2024-03-24T10:25:30.130 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-24T10:25:30.130 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_pdonnell@teuthology --name pdonnell-2024-03-24_04:56:01-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi --description fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} --archive /home/teuthworker/archive/pdonnell-2024-03-24_04:56:01-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7619070 --verbose -- /home/teuthworker/archive/pdonnell-2024-03-24_04:56:01-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7619070/orig.config.yaml 2024-03-24T10:25:30.179 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-24T10:25:30.272 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-03-24_04:56:01-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7619070 branch: wip-batrick-testing-20240323.003144-squid description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: pdonnell@redhat.com first_in_suite: false job_id: '7619070' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/quincy ' name: pdonnell-2024-03-24_04:56:01-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-batrick-testing-20240323.003144-squid ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 21b3bd672163092b4e542f772797fcba02d3d4de ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 21b3bd672163092b4e542f772797fcba02d3d4de kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-batrick-testing-20240323.003144-squid sha1: d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b owner: scheduled_pdonnell@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 4284 sha1: 21b3bd672163092b4e542f772797fcba02d3d4de sleep_before_teardown: 0 subset: 405/512 suite: fs suite_branch: wip-batrick-testing-20240323.003144-squid suite_path: /home/teuthworker/src/github.com_batrick_ceph_d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b/qa suite_relpath: qa suite_repo: https://github.com/batrick/ceph suite_sha1: d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b targets: smithi094.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLUkEj7js5B2LOwuV3cIofIodSHX3qWulagmUJDM+flT/yFZrCi2nuYPov/3ek9RIJmcmdXob/juWqdK+ysRkPE= smithi096.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJWsIwubLvHhvCvUaQT2eDd4GX0NcfHStLR85hOuWiDl3z52omxHrLa5qsLj3fs88euB9Nv5EQ7jV88mScZ0laA= tasks: - install: branch: quincy exclude_packages: - ceph-volume - print: '**** done install task...' - cephadm: cephadm_branch: quincy cephadm_git_url: https://github.com/ceph/ceph conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.ceph.io/ceph-ci/ceph:quincy roleless: true - print: '**** done end installing quincy cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 1 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay true - cephadm.shell: host.a: - ceph fs set cephfs inline_data false - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - kclient: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: - /cephfs/home/pdonnell/src/github.com_batrick_ceph_wip-batrick-testing-20240323.003144-squid/qa/suites/fs/upgrade/mds_upgrade_sequence/tasks/3-upgrade-mgr-staggered.yaml meta: {} postmerge: [] variables: fail_fs: false teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-24_04:56:01 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs false || true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-03-24T10:25:30.273 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_batrick_ceph_d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b/qa; will attempt to use it 2024-03-24T10:25:30.273 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_batrick_ceph_d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b/qa/tasks 2024-03-24T10:25:30.273 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-24T10:25:30.275 INFO:teuthology.task.internal:Checking packages... 2024-03-24T10:25:30.296 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '21b3bd672163092b4e542f772797fcba02d3d4de' 2024-03-24T10:25:30.296 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-24T10:25:30.296 INFO:teuthology.packaging:ref: None 2024-03-24T10:25:30.297 INFO:teuthology.packaging:tag: None 2024-03-24T10:25:30.297 INFO:teuthology.packaging:branch: wip-batrick-testing-20240323.003144-squid 2024-03-24T10:25:30.297 INFO:teuthology.packaging:sha1: 21b3bd672163092b4e542f772797fcba02d3d4de 2024-03-24T10:25:30.297 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-batrick-testing-20240323.003144-squid 2024-03-24T10:25:30.509 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1855.g21b3bd67 2024-03-24T10:25:30.510 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-24T10:25:30.515 INFO:teuthology.task.internal:no buildpackages task found 2024-03-24T10:25:30.516 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-24T10:25:30.562 INFO:teuthology.task.internal:Saving configuration 2024-03-24T10:25:30.581 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-24T10:25:30.626 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-24T10:25:30.652 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi094.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-24_04:56:01-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7619070', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-24 10:19:26.312724', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLUkEj7js5B2LOwuV3cIofIodSHX3qWulagmUJDM+flT/yFZrCi2nuYPov/3ek9RIJmcmdXob/juWqdK+ysRkPE='} 2024-03-24T10:25:30.671 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi096.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-24_04:56:01-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7619070', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-24 10:19:26.314032', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJWsIwubLvHhvCvUaQT2eDd4GX0NcfHStLR85hOuWiDl3z52omxHrLa5qsLj3fs88euB9Nv5EQ7jV88mScZ0laA='} 2024-03-24T10:25:30.671 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-24T10:25:30.677 INFO:teuthology.task.internal:roles: ubuntu@smithi094.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-03-24T10:25:30.678 INFO:teuthology.task.internal:roles: ubuntu@smithi096.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-03-24T10:25:30.678 INFO:teuthology.run_tasks:Running task console_log... 2024-03-24T10:25:30.749 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f687f595160>, signals=[15]) 2024-03-24T10:25:30.749 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-24T10:25:30.756 INFO:teuthology.task.internal:Opening connections... 2024-03-24T10:25:30.756 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi094.front.sepia.ceph.com 2024-03-24T10:25:30.758 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-24T10:25:30.845 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi096.front.sepia.ceph.com 2024-03-24T10:25:30.845 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-24T10:25:30.930 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-24T10:25:30.937 DEBUG:teuthology.orchestra.run.smithi094:> uname -m 2024-03-24T10:25:30.954 INFO:teuthology.orchestra.run.smithi094.stdout:x86_64 2024-03-24T10:25:30.954 DEBUG:teuthology.orchestra.run.smithi094:> cat /etc/os-release 2024-03-24T10:25:31.011 INFO:teuthology.orchestra.run.smithi094.stdout:NAME="CentOS Stream" 2024-03-24T10:25:31.011 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION="9" 2024-03-24T10:25:31.011 INFO:teuthology.orchestra.run.smithi094.stdout:ID="centos" 2024-03-24T10:25:31.011 INFO:teuthology.orchestra.run.smithi094.stdout:ID_LIKE="rhel fedora" 2024-03-24T10:25:31.011 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION_ID="9" 2024-03-24T10:25:31.011 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM_ID="platform:el9" 2024-03-24T10:25:31.011 INFO:teuthology.orchestra.run.smithi094.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-24T10:25:31.011 INFO:teuthology.orchestra.run.smithi094.stdout:ANSI_COLOR="0;31" 2024-03-24T10:25:31.011 INFO:teuthology.orchestra.run.smithi094.stdout:LOGO="fedora-logo-icon" 2024-03-24T10:25:31.011 INFO:teuthology.orchestra.run.smithi094.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-24T10:25:31.011 INFO:teuthology.orchestra.run.smithi094.stdout:HOME_URL="https://centos.org/" 2024-03-24T10:25:31.012 INFO:teuthology.orchestra.run.smithi094.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-24T10:25:31.012 INFO:teuthology.orchestra.run.smithi094.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-24T10:25:31.012 INFO:teuthology.orchestra.run.smithi094.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-24T10:25:31.012 INFO:teuthology.lock.ops:Updating smithi094.front.sepia.ceph.com on lock server 2024-03-24T10:25:31.051 DEBUG:teuthology.orchestra.run.smithi096:> uname -m 2024-03-24T10:25:31.068 INFO:teuthology.orchestra.run.smithi096.stdout:x86_64 2024-03-24T10:25:31.068 DEBUG:teuthology.orchestra.run.smithi096:> cat /etc/os-release 2024-03-24T10:25:31.124 INFO:teuthology.orchestra.run.smithi096.stdout:NAME="CentOS Stream" 2024-03-24T10:25:31.124 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION="9" 2024-03-24T10:25:31.124 INFO:teuthology.orchestra.run.smithi096.stdout:ID="centos" 2024-03-24T10:25:31.124 INFO:teuthology.orchestra.run.smithi096.stdout:ID_LIKE="rhel fedora" 2024-03-24T10:25:31.124 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION_ID="9" 2024-03-24T10:25:31.125 INFO:teuthology.orchestra.run.smithi096.stdout:PLATFORM_ID="platform:el9" 2024-03-24T10:25:31.125 INFO:teuthology.orchestra.run.smithi096.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-24T10:25:31.125 INFO:teuthology.orchestra.run.smithi096.stdout:ANSI_COLOR="0;31" 2024-03-24T10:25:31.125 INFO:teuthology.orchestra.run.smithi096.stdout:LOGO="fedora-logo-icon" 2024-03-24T10:25:31.125 INFO:teuthology.orchestra.run.smithi096.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-24T10:25:31.125 INFO:teuthology.orchestra.run.smithi096.stdout:HOME_URL="https://centos.org/" 2024-03-24T10:25:31.125 INFO:teuthology.orchestra.run.smithi096.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-24T10:25:31.125 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-24T10:25:31.125 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-24T10:25:31.125 INFO:teuthology.lock.ops:Updating smithi096.front.sepia.ceph.com on lock server 2024-03-24T10:25:31.159 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-24T10:25:31.168 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-24T10:25:31.177 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-24T10:25:31.178 DEBUG:teuthology.orchestra.run.smithi094:> test '!' -e /home/ubuntu/cephtest 2024-03-24T10:25:31.180 DEBUG:teuthology.orchestra.run.smithi096:> test '!' -e /home/ubuntu/cephtest 2024-03-24T10:25:31.195 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-24T10:25:31.201 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-24T10:25:31.202 DEBUG:teuthology.orchestra.run.smithi094:> test -z $(ls -A /var/lib/ceph) 2024-03-24T10:25:31.235 DEBUG:teuthology.orchestra.run.smithi096:> test -z $(ls -A /var/lib/ceph) 2024-03-24T10:25:31.326 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-24T10:25:31.381 INFO:teuthology.run_tasks:Running task kernel... 2024-03-24T10:25:31.395 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-03-24T10:25:31.395 INFO:teuthology.task.kernel:config {'host.a': {'kdb': True, 'sha1': 'distro'}, 'host.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-03-24T10:25:31.395 DEBUG:teuthology.orchestra.run.smithi094:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-24T10:25:31.395 DEBUG:teuthology.orchestra.run.smithi096:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-24T10:25:31.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-24T10:25:31.411 DEBUG:teuthology.orchestra.run.smithi096:> uname -r 2024-03-24T10:25:31.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-24T10:25:31.412 DEBUG:teuthology.orchestra.run.smithi094:> uname -r 2024-03-24T10:25:31.467 INFO:teuthology.orchestra.run.smithi096.stdout:5.14.0-361.el9.x86_64 2024-03-24T10:25:31.467 INFO:teuthology.task.kernel:Running kernel on smithi096: 5.14.0-361.el9.x86_64 2024-03-24T10:25:31.467 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum install -y kernel 2024-03-24T10:25:31.469 INFO:teuthology.orchestra.run.smithi094.stdout:5.14.0-361.el9.x86_64 2024-03-24T10:25:31.469 INFO:teuthology.task.kernel:Running kernel on smithi094: 5.14.0-361.el9.x86_64 2024-03-24T10:25:31.469 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum install -y kernel 2024-03-24T10:25:33.594 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-03-24T10:25:33.646 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-03-24T10:25:35.203 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS-9 - BaseOS 4.1 MB/s | 2.0 MB 00:00 2024-03-24T10:25:35.263 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS-9 - BaseOS 3.9 MB/s | 2.0 MB 00:00 2024-03-24T10:25:35.943 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - BaseOS 21 MB/s | 8.0 MB 00:00 2024-03-24T10:25:37.229 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - BaseOS 5.0 MB/s | 8.0 MB 00:01 2024-03-24T10:25:37.711 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - AppStream 26 MB/s | 19 MB 00:00 2024-03-24T10:25:39.148 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-03-24T10:25:43.551 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - CRB 6.7 MB/s | 6.1 MB 00:00 2024-03-24T10:25:43.878 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - CRB 2.5 MB/s | 6.1 MB 00:02 2024-03-24T10:25:45.470 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 16 kB 00:00 2024-03-24T10:25:45.850 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - Extras packages 13 kB/s | 16 kB 00:01 2024-03-24T10:25:46.221 INFO:teuthology.orchestra.run.smithi096.stdout:Extra Packages for Enterprise Linux 32 MB/s | 21 MB 00:00 2024-03-24T10:25:47.368 INFO:teuthology.orchestra.run.smithi094.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-03-24T10:25:51.095 INFO:teuthology.orchestra.run.smithi096.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-03-24T10:25:52.360 INFO:teuthology.orchestra.run.smithi094.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-03-24T10:25:53.177 INFO:teuthology.orchestra.run.smithi096.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-24T10:25:53.177 INFO:teuthology.orchestra.run.smithi096.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-24T10:25:53.267 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-24T10:25:53.279 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-24T10:25:53.279 INFO:teuthology.orchestra.run.smithi096.stdout: Package Architecture Version Repository Size 2024-03-24T10:25:53.279 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-24T10:25:53.279 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-03-24T10:25:53.279 INFO:teuthology.orchestra.run.smithi096.stdout: kernel x86_64 5.14.0-430.el9 baseos 44 k 2024-03-24T10:25:53.279 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-03-24T10:25:53.279 INFO:teuthology.orchestra.run.smithi096.stdout: kernel-core x86_64 5.14.0-430.el9 baseos 15 M 2024-03-24T10:25:53.280 INFO:teuthology.orchestra.run.smithi096.stdout: kernel-modules x86_64 5.14.0-430.el9 baseos 33 M 2024-03-24T10:25:53.280 INFO:teuthology.orchestra.run.smithi096.stdout: kernel-modules-core x86_64 5.14.0-430.el9 baseos 27 M 2024-03-24T10:25:53.280 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:25:53.280 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-03-24T10:25:53.280 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-24T10:25:53.280 INFO:teuthology.orchestra.run.smithi096.stdout:Install 4 Packages 2024-03-24T10:25:53.280 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:25:53.280 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 76 M 2024-03-24T10:25:53.280 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 123 M 2024-03-24T10:25:53.280 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-03-24T10:25:53.538 INFO:teuthology.orchestra.run.smithi096.stdout:(1/4): kernel-5.14.0-430.el9.x86_64.rpm 379 kB/s | 44 kB 00:00 2024-03-24T10:25:54.492 INFO:teuthology.orchestra.run.smithi094.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-24T10:25:54.492 INFO:teuthology.orchestra.run.smithi094.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-24T10:25:54.589 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-03-24T10:25:54.600 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-03-24T10:25:54.600 INFO:teuthology.orchestra.run.smithi094.stdout: Package Architecture Version Repository Size 2024-03-24T10:25:54.600 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-03-24T10:25:54.600 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-03-24T10:25:54.600 INFO:teuthology.orchestra.run.smithi094.stdout: kernel x86_64 5.14.0-430.el9 baseos 44 k 2024-03-24T10:25:54.600 INFO:teuthology.orchestra.run.smithi094.stdout:Installing dependencies: 2024-03-24T10:25:54.601 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-core x86_64 5.14.0-430.el9 baseos 15 M 2024-03-24T10:25:54.601 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-modules x86_64 5.14.0-430.el9 baseos 33 M 2024-03-24T10:25:54.601 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-modules-core x86_64 5.14.0-430.el9 baseos 27 M 2024-03-24T10:25:54.601 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:25:54.601 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-03-24T10:25:54.601 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-03-24T10:25:54.601 INFO:teuthology.orchestra.run.smithi094.stdout:Install 4 Packages 2024-03-24T10:25:54.601 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:25:54.601 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 76 M 2024-03-24T10:25:54.601 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 123 M 2024-03-24T10:25:54.601 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-03-24T10:25:54.778 INFO:teuthology.orchestra.run.smithi094.stdout:(1/4): kernel-5.14.0-430.el9.x86_64.rpm 398 kB/s | 44 kB 00:00 2024-03-24T10:25:55.423 INFO:teuthology.orchestra.run.smithi094.stdout:(2/4): kernel-core-5.14.0-430.el9.x86_64.rpm 20 MB/s | 15 MB 00:00 2024-03-24T10:25:56.014 INFO:teuthology.orchestra.run.smithi094.stdout:(3/4): kernel-modules-5.14.0-430.el9.x86_64.rpm 25 MB/s | 33 MB 00:01 2024-03-24T10:25:56.673 INFO:teuthology.orchestra.run.smithi094.stdout:(4/4): kernel-modules-core-5.14.0-430.el9.x86_6 15 MB/s | 27 MB 00:01 2024-03-24T10:25:56.673 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-03-24T10:25:56.673 INFO:teuthology.orchestra.run.smithi094.stdout:Total 37 MB/s | 76 MB 00:02 2024-03-24T10:25:57.005 INFO:teuthology.orchestra.run.smithi096.stdout:(2/4): kernel-core-5.14.0-430.el9.x86_64.rpm 4.2 MB/s | 15 MB 00:03 2024-03-24T10:25:57.332 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-03-24T10:25:57.703 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-03-24T10:25:57.704 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-03-24T10:25:58.732 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-03-24T10:25:58.732 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-03-24T10:25:59.654 INFO:teuthology.orchestra.run.smithi096.stdout:(3/4): kernel-modules-core-5.14.0-430.el9.x86_6 4.5 MB/s | 27 MB 00:06 2024-03-24T10:26:00.929 INFO:teuthology.orchestra.run.smithi096.stdout:(4/4): kernel-modules-5.14.0-430.el9.x86_64.rpm 4.5 MB/s | 33 MB 00:07 2024-03-24T10:26:00.930 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-03-24T10:26:00.930 INFO:teuthology.orchestra.run.smithi096.stdout:Total 9.9 MB/s | 76 MB 00:07 2024-03-24T10:26:01.208 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-03-24T10:26:01.599 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-03-24T10:26:01.826 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : kernel-modules-core-5.14.0-430.el9.x86_64 1/4 2024-03-24T10:26:01.927 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-24T10:26:01.965 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-03-24T10:26:01.965 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-03-24T10:26:02.965 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-03-24T10:26:02.965 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-03-24T10:26:04.670 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-24T10:26:04.801 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-24T10:26:05.444 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-03-24T10:26:06.060 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : kernel-modules-core-5.14.0-430.el9.x86_64 1/4 2024-03-24T10:26:06.156 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-24T10:26:08.922 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-24T10:26:09.065 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-24T10:26:09.544 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-24T10:26:09.593 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : kernel-5.14.0-430.el9.x86_64 4/4 2024-03-24T10:26:13.815 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-24T10:26:13.848 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : kernel-5.14.0-430.el9.x86_64 4/4 2024-03-24T10:26:14.336 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-24T10:26:18.563 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-24T10:26:43.655 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 4/4 2024-03-24T10:26:43.655 INFO:teuthology.orchestra.run.smithi094.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-24T10:26:43.656 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:26:43.700 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 4/4 2024-03-24T10:26:44.119 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: kernel-5.14.0-430.el9.x86_64 4/4 2024-03-24T10:26:44.119 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : kernel-5.14.0-430.el9.x86_64 1/4 2024-03-24T10:26:44.120 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-24T10:26:44.120 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-24T10:26:44.552 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-24T10:26:44.552 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:26:44.552 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-03-24T10:26:44.552 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-5.14.0-430.el9.x86_64 2024-03-24T10:26:44.552 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-core-5.14.0-430.el9.x86_64 2024-03-24T10:26:44.552 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-modules-5.14.0-430.el9.x86_64 2024-03-24T10:26:44.552 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-modules-core-5.14.0-430.el9.x86_64 2024-03-24T10:26:44.552 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:26:44.552 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-03-24T10:26:44.884 DEBUG:teuthology.orchestra.run.smithi094:> rpm -q kernel | sort -rV | head -n 1 2024-03-24T10:26:45.024 INFO:teuthology.orchestra.run.smithi094.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-24T10:26:45.024 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-24T10:26:45.024 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-430.el9.x86_64 2024-03-24T10:26:45.025 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-24T10:26:45.025 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-24T10:26:45.025 INFO:teuthology.task.kernel:Installing distro kernel on host.a... 2024-03-24T10:26:45.025 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi094.front.sepia.ceph.com, path=None, version=distro) 2024-03-24T10:26:45.025 DEBUG:teuthology.orchestra.run.smithi094:> rpm -q kernel | sort -rV | head -n 1 2024-03-24T10:26:45.104 INFO:teuthology.orchestra.run.smithi094.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-24T10:26:45.105 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-24T10:26:45.105 DEBUG:teuthology.orchestra.run.smithi094:> sudo rpm -qi grub2-tools 2024-03-24T10:26:45.146 INFO:teuthology.orchestra.run.smithi094.stdout:Name : grub2-tools 2024-03-24T10:26:45.146 INFO:teuthology.orchestra.run.smithi094.stdout:Epoch : 1 2024-03-24T10:26:45.146 INFO:teuthology.orchestra.run.smithi094.stdout:Version : 2.06 2024-03-24T10:26:45.146 INFO:teuthology.orchestra.run.smithi094.stdout:Release : 68.el9 2024-03-24T10:26:45.147 INFO:teuthology.orchestra.run.smithi094.stdout:Architecture: x86_64 2024-03-24T10:26:45.147 INFO:teuthology.orchestra.run.smithi094.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-24T10:26:45.147 INFO:teuthology.orchestra.run.smithi094.stdout:Group : Unspecified 2024-03-24T10:26:45.147 INFO:teuthology.orchestra.run.smithi094.stdout:Size : 8264254 2024-03-24T10:26:45.147 INFO:teuthology.orchestra.run.smithi094.stdout:License : GPLv3+ 2024-03-24T10:26:45.147 INFO:teuthology.orchestra.run.smithi094.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-24T10:26:45.147 INFO:teuthology.orchestra.run.smithi094.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-24T10:26:45.147 INFO:teuthology.orchestra.run.smithi094.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-24T10:26:45.147 INFO:teuthology.orchestra.run.smithi094.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-24T10:26:45.147 INFO:teuthology.orchestra.run.smithi094.stdout:Packager : builder@centos.org 2024-03-24T10:26:45.147 INFO:teuthology.orchestra.run.smithi094.stdout:Vendor : CentOS 2024-03-24T10:26:45.147 INFO:teuthology.orchestra.run.smithi094.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-24T10:26:45.147 INFO:teuthology.orchestra.run.smithi094.stdout:Summary : Support tools for GRUB. 2024-03-24T10:26:45.147 INFO:teuthology.orchestra.run.smithi094.stdout:Description : 2024-03-24T10:26:45.147 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:26:45.148 INFO:teuthology.orchestra.run.smithi094.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-24T10:26:45.148 INFO:teuthology.orchestra.run.smithi094.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-24T10:26:45.148 INFO:teuthology.orchestra.run.smithi094.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-24T10:26:45.148 INFO:teuthology.orchestra.run.smithi094.stdout:hardware devices. 2024-03-24T10:26:45.148 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:26:45.148 INFO:teuthology.orchestra.run.smithi094.stdout:This subpackage provides tools for support of all platforms. 2024-03-24T10:26:45.149 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-24T10:26:45.149 INFO:teuthology.task.kernel:Updating grub on smithi094 to boot 5.14.0-430.el9.x86_64 2024-03-24T10:26:45.149 DEBUG:teuthology.orchestra.run.smithi094:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-24T10:26:45.805 INFO:teuthology.orchestra.run.smithi094.stderr:Generating grub configuration file ... 2024-03-24T10:26:47.191 INFO:teuthology.orchestra.run.smithi094.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-24T10:26:47.240 INFO:teuthology.orchestra.run.smithi094.stderr:done 2024-03-24T10:26:47.242 DEBUG:teuthology.orchestra.run.smithi094:> mktemp 2024-03-24T10:26:47.257 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/tmp.Wvx5Zg0kqT 2024-03-24T10:26:47.257 DEBUG:teuthology.orchestra.run.smithi094:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Wvx5Zg0kqT 2024-03-24T10:26:47.327 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 0666 /tmp/tmp.Wvx5Zg0kqT 2024-03-24T10:26:47.469 DEBUG:teuthology.orchestra.remote:smithi094:/tmp/tmp.Wvx5Zg0kqT is 6KB 2024-03-24T10:26:47.520 DEBUG:teuthology.orchestra.run.smithi094:> rm -fr /tmp/tmp.Wvx5Zg0kqT 2024-03-24T10:26:47.534 DEBUG:teuthology.orchestra.run.smithi094:> sudo /bin/ls /boot/loader/entries || true 2024-03-24T10:26:47.602 INFO:teuthology.orchestra.run.smithi094.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-24T10:26:47.602 INFO:teuthology.orchestra.run.smithi094.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-24T10:26:47.602 INFO:teuthology.orchestra.run.smithi094.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-24T10:26:47.602 INFO:teuthology.orchestra.run.smithi094.stdout:15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64.conf 2024-03-24T10:26:47.605 DEBUG:teuthology.orchestra.run.smithi094:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64 2024-03-24T10:26:47.766 DEBUG:teuthology.orchestra.run.smithi094:> sudo shutdown -r now 2024-03-24T10:26:48.084 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 4/4 2024-03-24T10:26:48.084 INFO:teuthology.orchestra.run.smithi096.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-24T10:26:48.084 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:26:48.129 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 4/4 2024-03-24T10:26:48.521 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: kernel-5.14.0-430.el9.x86_64 4/4 2024-03-24T10:26:48.521 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : kernel-5.14.0-430.el9.x86_64 1/4 2024-03-24T10:26:48.522 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-24T10:26:48.522 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-24T10:26:48.880 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-24T10:26:48.880 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:26:48.880 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-03-24T10:26:48.880 INFO:teuthology.orchestra.run.smithi096.stdout: kernel-5.14.0-430.el9.x86_64 2024-03-24T10:26:48.881 INFO:teuthology.orchestra.run.smithi096.stdout: kernel-core-5.14.0-430.el9.x86_64 2024-03-24T10:26:48.881 INFO:teuthology.orchestra.run.smithi096.stdout: kernel-modules-5.14.0-430.el9.x86_64 2024-03-24T10:26:48.881 INFO:teuthology.orchestra.run.smithi096.stdout: kernel-modules-core-5.14.0-430.el9.x86_64 2024-03-24T10:26:48.881 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:26:48.881 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-24T10:26:49.210 DEBUG:teuthology.orchestra.run.smithi096:> rpm -q kernel | sort -rV | head -n 1 2024-03-24T10:26:49.330 INFO:teuthology.orchestra.run.smithi096.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-24T10:26:49.330 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-24T10:26:49.330 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-430.el9.x86_64 2024-03-24T10:26:49.331 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-24T10:26:49.331 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-24T10:26:49.331 INFO:teuthology.task.kernel:Installing distro kernel on host.b... 2024-03-24T10:26:49.331 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi096.front.sepia.ceph.com, path=None, version=distro) 2024-03-24T10:26:49.331 DEBUG:teuthology.orchestra.run.smithi096:> rpm -q kernel | sort -rV | head -n 1 2024-03-24T10:26:49.410 INFO:teuthology.orchestra.run.smithi096.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-24T10:26:49.411 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-24T10:26:49.411 DEBUG:teuthology.orchestra.run.smithi096:> sudo rpm -qi grub2-tools 2024-03-24T10:26:49.445 INFO:teuthology.orchestra.run.smithi096.stdout:Name : grub2-tools 2024-03-24T10:26:49.445 INFO:teuthology.orchestra.run.smithi096.stdout:Epoch : 1 2024-03-24T10:26:49.445 INFO:teuthology.orchestra.run.smithi096.stdout:Version : 2.06 2024-03-24T10:26:49.445 INFO:teuthology.orchestra.run.smithi096.stdout:Release : 68.el9 2024-03-24T10:26:49.445 INFO:teuthology.orchestra.run.smithi096.stdout:Architecture: x86_64 2024-03-24T10:26:49.445 INFO:teuthology.orchestra.run.smithi096.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-24T10:26:49.445 INFO:teuthology.orchestra.run.smithi096.stdout:Group : Unspecified 2024-03-24T10:26:49.446 INFO:teuthology.orchestra.run.smithi096.stdout:Size : 8264254 2024-03-24T10:26:49.446 INFO:teuthology.orchestra.run.smithi096.stdout:License : GPLv3+ 2024-03-24T10:26:49.446 INFO:teuthology.orchestra.run.smithi096.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-24T10:26:49.446 INFO:teuthology.orchestra.run.smithi096.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-24T10:26:49.446 INFO:teuthology.orchestra.run.smithi096.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-24T10:26:49.446 INFO:teuthology.orchestra.run.smithi096.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-24T10:26:49.446 INFO:teuthology.orchestra.run.smithi096.stdout:Packager : builder@centos.org 2024-03-24T10:26:49.446 INFO:teuthology.orchestra.run.smithi096.stdout:Vendor : CentOS 2024-03-24T10:26:49.446 INFO:teuthology.orchestra.run.smithi096.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-24T10:26:49.446 INFO:teuthology.orchestra.run.smithi096.stdout:Summary : Support tools for GRUB. 2024-03-24T10:26:49.446 INFO:teuthology.orchestra.run.smithi096.stdout:Description : 2024-03-24T10:26:49.446 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:26:49.446 INFO:teuthology.orchestra.run.smithi096.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-24T10:26:49.446 INFO:teuthology.orchestra.run.smithi096.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-24T10:26:49.447 INFO:teuthology.orchestra.run.smithi096.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-24T10:26:49.447 INFO:teuthology.orchestra.run.smithi096.stdout:hardware devices. 2024-03-24T10:26:49.447 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:26:49.447 INFO:teuthology.orchestra.run.smithi096.stdout:This subpackage provides tools for support of all platforms. 2024-03-24T10:26:49.448 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-24T10:26:49.448 INFO:teuthology.task.kernel:Updating grub on smithi096 to boot 5.14.0-430.el9.x86_64 2024-03-24T10:26:49.448 DEBUG:teuthology.orchestra.run.smithi096:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-24T10:26:50.125 INFO:teuthology.orchestra.run.smithi096.stderr:Generating grub configuration file ... 2024-03-24T10:26:51.515 INFO:teuthology.orchestra.run.smithi096.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-24T10:26:51.566 INFO:teuthology.orchestra.run.smithi096.stderr:done 2024-03-24T10:26:51.568 DEBUG:teuthology.orchestra.run.smithi096:> mktemp 2024-03-24T10:26:51.587 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/tmp.DeEncTP8OJ 2024-03-24T10:26:51.587 DEBUG:teuthology.orchestra.run.smithi096:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.DeEncTP8OJ 2024-03-24T10:26:51.655 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 0666 /tmp/tmp.DeEncTP8OJ 2024-03-24T10:26:51.795 DEBUG:teuthology.orchestra.remote:smithi096:/tmp/tmp.DeEncTP8OJ is 6KB 2024-03-24T10:26:51.844 DEBUG:teuthology.orchestra.run.smithi096:> rm -fr /tmp/tmp.DeEncTP8OJ 2024-03-24T10:26:51.858 DEBUG:teuthology.orchestra.run.smithi096:> sudo /bin/ls /boot/loader/entries || true 2024-03-24T10:26:51.928 INFO:teuthology.orchestra.run.smithi096.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-24T10:26:51.928 INFO:teuthology.orchestra.run.smithi096.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-24T10:26:51.928 INFO:teuthology.orchestra.run.smithi096.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-24T10:26:51.928 INFO:teuthology.orchestra.run.smithi096.stdout:15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64.conf 2024-03-24T10:26:51.929 DEBUG:teuthology.orchestra.run.smithi096:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64 2024-03-24T10:26:52.084 DEBUG:teuthology.orchestra.run.smithi096:> sudo shutdown -r now 2024-03-24T10:27:17.794 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-03-24T10:27:17.795 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-03-24T10:27:17.795 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-24T10:27:22.091 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-03-24T10:27:22.092 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-03-24T10:27:22.092 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-24T10:28:17.850 DEBUG:teuthology.orchestra.remote:timed out 2024-03-24T10:28:22.098 DEBUG:teuthology.orchestra.remote:timed out 2024-03-24T10:28:26.854 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-03-24T10:28:26.855 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-24T10:28:28.389 DEBUG:teuthology.orchestra.run.smithi094:> true 2024-03-24T10:28:28.882 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-03-24T10:28:28.882 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "5.14.0-430.el9.x86_64"... 2024-03-24T10:28:28.882 DEBUG:teuthology.orchestra.run.smithi094:> uname -r 2024-03-24T10:28:28.896 INFO:teuthology.orchestra.run.smithi094.stdout:5.14.0-430.el9.x86_64 2024-03-24T10:28:28.896 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-430.el9.x86_64 vs 5.14.0-430.el9.x86_64 2024-03-24T10:28:28.897 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-24T10:28:28.897 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-24T10:28:29.897 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-03-24T10:28:29.897 DEBUG:teuthology.orchestra.run.smithi094:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-24T10:28:30.003 INFO:teuthology.orchestra.run.smithi094.stdout:ttyS1 2024-03-24T10:28:30.037 DEBUG:teuthology.parallel:result is None 2024-03-24T10:28:31.099 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-03-24T10:28:31.100 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-24T10:28:31.656 DEBUG:teuthology.orchestra.run.smithi096:> true 2024-03-24T10:28:32.128 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-03-24T10:28:32.128 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "5.14.0-430.el9.x86_64"... 2024-03-24T10:28:32.128 DEBUG:teuthology.orchestra.run.smithi096:> uname -r 2024-03-24T10:28:32.143 INFO:teuthology.orchestra.run.smithi096.stdout:5.14.0-430.el9.x86_64 2024-03-24T10:28:32.143 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-430.el9.x86_64 vs 5.14.0-430.el9.x86_64 2024-03-24T10:28:32.143 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-24T10:28:32.143 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-24T10:28:33.144 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-03-24T10:28:33.144 DEBUG:teuthology.orchestra.run.smithi096:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-24T10:28:33.229 INFO:teuthology.orchestra.run.smithi096.stdout:ttyS1 2024-03-24T10:28:33.276 DEBUG:teuthology.parallel:result is None 2024-03-24T10:28:33.276 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-24T10:28:33.284 INFO:teuthology.task.internal:Creating test directory... 2024-03-24T10:28:33.284 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-24T10:28:33.287 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-24T10:28:33.302 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-24T10:28:33.308 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-24T10:28:33.315 INFO:teuthology.task.internal:Creating archive directory... 2024-03-24T10:28:33.315 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-24T10:28:33.342 DEBUG:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-24T10:28:33.372 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-24T10:28:33.379 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-24T10:28:33.379 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-24T10:28:33.402 DEBUG:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-24T10:28:33.437 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-24T10:28:33.445 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-24T10:28:33.449 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-24T10:28:33.458 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-24T10:28:33.459 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-24T10:28:33.466 INFO:teuthology.task.internal:Configuring sudo... 2024-03-24T10:28:33.466 DEBUG:teuthology.orchestra.run.smithi094:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-24T10:28:33.490 DEBUG:teuthology.orchestra.run.smithi096:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-24T10:28:33.527 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-24T10:28:33.535 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-24T10:28:33.535 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-24T10:28:33.555 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-24T10:28:33.581 DEBUG:teuthology.orchestra.run.smithi094:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-24T10:28:33.647 DEBUG:teuthology.orchestra.run.smithi094:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-24T10:28:33.693 DEBUG:teuthology.orchestra.run.smithi094:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-24T10:28:33.772 DEBUG:teuthology.orchestra.run.smithi094:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-24T10:28:33.809 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-03-24T10:28:33.809 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-24T10:28:33.874 DEBUG:teuthology.orchestra.run.smithi096:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-24T10:28:33.918 DEBUG:teuthology.orchestra.run.smithi096:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-24T10:28:33.964 DEBUG:teuthology.orchestra.run.smithi096:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-24T10:28:34.047 DEBUG:teuthology.orchestra.run.smithi096:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-24T10:28:34.090 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-24T10:28:34.090 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-24T10:28:34.158 DEBUG:teuthology.orchestra.run.smithi094:> sudo service rsyslog restart 2024-03-24T10:28:34.161 DEBUG:teuthology.orchestra.run.smithi096:> sudo service rsyslog restart 2024-03-24T10:28:34.202 INFO:teuthology.orchestra.run.smithi094.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-24T10:28:34.247 INFO:teuthology.orchestra.run.smithi096.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-24T10:28:34.630 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-24T10:28:34.638 INFO:teuthology.task.internal:Starting timer... 2024-03-24T10:28:34.639 INFO:teuthology.run_tasks:Running task pcp... 2024-03-24T10:28:34.649 INFO:teuthology.run_tasks:Running task selinux... 2024-03-24T10:28:34.658 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-03-24T10:28:34.659 DEBUG:teuthology.orchestra.run.smithi094:> sudo service auditd rotate 2024-03-24T10:28:34.719 INFO:teuthology.orchestra.run.smithi094.stdout:Rotating logs: 2024-03-24T10:28:34.721 DEBUG:teuthology.orchestra.run.smithi096:> sudo service auditd rotate 2024-03-24T10:28:34.784 INFO:teuthology.orchestra.run.smithi096.stdout:Rotating logs: 2024-03-24T10:28:34.786 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-24T10:28:34.786 DEBUG:teuthology.orchestra.run.smithi094:> /usr/sbin/getenforce 2024-03-24T10:28:34.839 INFO:teuthology.orchestra.run.smithi094.stdout:Permissive 2024-03-24T10:28:34.839 DEBUG:teuthology.orchestra.run.smithi096:> /usr/sbin/getenforce 2024-03-24T10:28:34.864 INFO:teuthology.orchestra.run.smithi096.stdout:Permissive 2024-03-24T10:28:34.864 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi094.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi096.front.sepia.ceph.com': 'permissive'} 2024-03-24T10:28:34.864 DEBUG:teuthology.orchestra.run.smithi094:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-24T10:28:34.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-24T10:28:34.893 DEBUG:teuthology.orchestra.run.smithi096:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-24T10:28:34.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-24T10:28:34.933 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-24T10:28:34.933 DEBUG:teuthology.orchestra.run.smithi094:> sudo /usr/sbin/setenforce permissive 2024-03-24T10:28:35.006 DEBUG:teuthology.orchestra.run.smithi096:> sudo /usr/sbin/setenforce permissive 2024-03-24T10:28:35.041 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-24T10:28:35.053 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-24T10:28:35.068 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-24T10:28:35.069 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi094.front.sepia.ceph.com,smithi096.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-24T10:33:11.496 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi094.front.sepia.ceph.com'), Remote(name='ubuntu@smithi096.front.sepia.ceph.com')] 2024-03-24T10:33:11.497 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-03-24T10:33:11.498 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-24T10:33:11.616 DEBUG:teuthology.orchestra.run.smithi094:> true 2024-03-24T10:33:11.701 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-03-24T10:33:11.701 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-03-24T10:33:11.702 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-24T10:33:11.824 DEBUG:teuthology.orchestra.run.smithi096:> true 2024-03-24T10:33:11.907 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-03-24T10:33:11.907 INFO:teuthology.run_tasks:Running task clock... 2024-03-24T10:33:11.922 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-24T10:33:11.923 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-24T10:33:11.923 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-24T10:33:11.926 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-24T10:33:11.927 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-24T10:33:11.956 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-24T10:33:11.973 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-24T10:33:11.982 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-24T10:33:11.999 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-24T10:33:12.002 INFO:teuthology.orchestra.run.smithi094.stderr:sudo: ntpd: command not found 2024-03-24T10:33:12.017 INFO:teuthology.orchestra.run.smithi094.stdout:506 Cannot talk to daemon 2024-03-24T10:33:12.028 INFO:teuthology.orchestra.run.smithi096.stderr:sudo: ntpd: command not found 2024-03-24T10:33:12.033 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-24T10:33:12.042 INFO:teuthology.orchestra.run.smithi096.stdout:506 Cannot talk to daemon 2024-03-24T10:33:12.049 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-24T10:33:12.059 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-24T10:33:12.075 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-24T10:33:12.099 INFO:teuthology.orchestra.run.smithi094.stderr:bash: line 1: ntpq: command not found 2024-03-24T10:33:12.102 INFO:teuthology.orchestra.run.smithi094.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-24T10:33:12.102 INFO:teuthology.orchestra.run.smithi094.stdout:=============================================================================== 2024-03-24T10:33:12.102 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T10:33:12.102 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T10:33:12.102 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T10:33:12.103 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T10:33:12.127 INFO:teuthology.orchestra.run.smithi096.stderr:bash: line 1: ntpq: command not found 2024-03-24T10:33:12.130 INFO:teuthology.orchestra.run.smithi096.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-24T10:33:12.130 INFO:teuthology.orchestra.run.smithi096.stdout:=============================================================================== 2024-03-24T10:33:12.130 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T10:33:12.130 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T10:33:12.131 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T10:33:12.131 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T10:33:12.132 INFO:teuthology.run_tasks:Running task install... 2024-03-24T10:33:12.140 DEBUG:teuthology.task.install:project ceph 2024-03-24T10:33:12.140 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '21b3bd672163092b4e542f772797fcba02d3d4de'}} 2024-03-24T10:33:12.140 DEBUG:teuthology.task.install:config {'branch': 'quincy', 'exclude_packages': ['ceph-volume'], 'flavor': 'default', 'sha1': '21b3bd672163092b4e542f772797fcba02d3d4de'} 2024-03-24T10:33:12.140 INFO:teuthology.task.install:Using flavor: default 2024-03-24T10:33:12.148 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-24T10:33:12.148 INFO:teuthology.task.install:extra packages: [] 2024-03-24T10:33:12.148 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '21b3bd672163092b4e542f772797fcba02d3d4de', 'tag': None, 'wait_for_package': False} 2024-03-24T10:33:12.149 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-24T10:33:12.149 INFO:teuthology.packaging:ref: None 2024-03-24T10:33:12.149 INFO:teuthology.packaging:tag: None 2024-03-24T10:33:12.149 INFO:teuthology.packaging:branch: quincy 2024-03-24T10:33:12.149 INFO:teuthology.packaging:sha1: 21b3bd672163092b4e542f772797fcba02d3d4de 2024-03-24T10:33:12.149 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-03-24T10:33:12.151 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '21b3bd672163092b4e542f772797fcba02d3d4de', 'tag': None, 'wait_for_package': False} 2024-03-24T10:33:12.151 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-24T10:33:12.151 INFO:teuthology.packaging:ref: None 2024-03-24T10:33:12.151 INFO:teuthology.packaging:tag: None 2024-03-24T10:33:12.151 INFO:teuthology.packaging:branch: quincy 2024-03-24T10:33:12.151 INFO:teuthology.packaging:sha1: 21b3bd672163092b4e542f772797fcba02d3d4de 2024-03-24T10:33:12.151 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-03-24T10:33:12.470 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/quincy/9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45/centos/9/flavors/default/ 2024-03-24T10:33:12.470 INFO:teuthology.task.install.rpm:Package version is 17.2.7-726.g9c4e0c34 2024-03-24T10:33:12.497 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/quincy/9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45/centos/9/flavors/default/ 2024-03-24T10:33:12.498 INFO:teuthology.task.install.rpm:Package version is 17.2.7-726.g9c4e0c34 2024-03-24T10:33:12.651 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/quincy/9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-24T10:33:12.652 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-24T10:33:12.652 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-24T10:33:12.661 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/quincy/9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-24T10:33:12.661 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-03-24T10:33:12.662 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-24T10:33:12.684 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-03-24T10:33:12.684 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-24T10:33:12.684 INFO:teuthology.packaging:ref: None 2024-03-24T10:33:12.684 INFO:teuthology.packaging:tag: None 2024-03-24T10:33:12.684 INFO:teuthology.packaging:branch: quincy 2024-03-24T10:33:12.684 INFO:teuthology.packaging:sha1: 21b3bd672163092b4e542f772797fcba02d3d4de 2024-03-24T10:33:12.685 DEBUG:teuthology.orchestra.run.smithi096:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-24T10:33:12.691 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-03-24T10:33:12.692 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-24T10:33:12.692 INFO:teuthology.packaging:ref: None 2024-03-24T10:33:12.692 INFO:teuthology.packaging:tag: None 2024-03-24T10:33:12.692 INFO:teuthology.packaging:branch: quincy 2024-03-24T10:33:12.692 INFO:teuthology.packaging:sha1: 21b3bd672163092b4e542f772797fcba02d3d4de 2024-03-24T10:33:12.692 DEBUG:teuthology.orchestra.run.smithi094:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-24T10:33:12.753 DEBUG:teuthology.orchestra.run.smithi096:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-24T10:33:12.760 DEBUG:teuthology.orchestra.run.smithi094:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-24T10:33:12.834 DEBUG:teuthology.orchestra.run.smithi096:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-24T10:33:12.839 DEBUG:teuthology.orchestra.run.smithi094:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-24T10:33:12.900 INFO:teuthology.orchestra.run.smithi096.stdout:check_obsoletes = 1 2024-03-24T10:33:12.902 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum clean all 2024-03-24T10:33:12.906 INFO:teuthology.orchestra.run.smithi094.stdout:check_obsoletes = 1 2024-03-24T10:33:12.907 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum clean all 2024-03-24T10:33:13.138 INFO:teuthology.orchestra.run.smithi096.stdout:74 files removed 2024-03-24T10:33:13.153 INFO:teuthology.orchestra.run.smithi094.stdout:74 files removed 2024-03-24T10:33:13.165 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-radosgw 2024-03-24T10:33:13.179 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install ceph-radosgw 2024-03-24T10:33:13.870 INFO:teuthology.orchestra.run.smithi096.stdout:ceph packages for x86_64 191 kB/s | 77 kB 00:00 2024-03-24T10:33:13.932 INFO:teuthology.orchestra.run.smithi094.stdout:ceph packages for x86_64 170 kB/s | 77 kB 00:00 2024-03-24T10:33:14.266 INFO:teuthology.orchestra.run.smithi094.stdout:ceph noarch packages 32 kB/s | 9.2 kB 00:00 2024-03-24T10:33:14.296 INFO:teuthology.orchestra.run.smithi096.stdout:ceph noarch packages 24 kB/s | 9.2 kB 00:00 2024-03-24T10:33:14.584 INFO:teuthology.orchestra.run.smithi094.stdout:ceph source packages 6.4 kB/s | 1.8 kB 00:00 2024-03-24T10:33:14.597 INFO:teuthology.orchestra.run.smithi096.stdout:ceph source packages 6.8 kB/s | 1.8 kB 00:00 2024-03-24T10:33:15.010 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS-9 - AppStream 17 MB/s | 6.7 MB 00:00 2024-03-24T10:33:15.440 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS-9 - AppStream 8.3 MB/s | 6.7 MB 00:00 2024-03-24T10:33:16.486 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS-9 - BaseOS 6.1 MB/s | 2.0 MB 00:00 2024-03-24T10:33:16.840 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS-9 - BaseOS 7.2 MB/s | 2.0 MB 00:00 2024-03-24T10:33:17.892 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-03-24T10:33:18.801 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - BaseOS 4.0 MB/s | 8.0 MB 00:01 2024-03-24T10:33:20.199 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - AppStream 15 MB/s | 19 MB 00:01 2024-03-24T10:33:22.327 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - AppStream 7.6 MB/s | 19 MB 00:02 2024-03-24T10:33:24.377 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - CRB 8.4 MB/s | 6.1 MB 00:00 2024-03-24T10:33:26.167 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 16 kB 00:00 2024-03-24T10:33:27.843 INFO:teuthology.orchestra.run.smithi096.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-03-24T10:33:28.970 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - CRB 1.9 MB/s | 6.1 MB 00:03 2024-03-24T10:33:30.395 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - Extras packages 63 kB/s | 16 kB 00:00 2024-03-24T10:33:32.697 INFO:teuthology.orchestra.run.smithi094.stdout:Extra Packages for Enterprise Linux 9.5 MB/s | 21 MB 00:02 2024-03-24T10:33:32.809 INFO:teuthology.orchestra.run.smithi096.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-03-24T10:33:35.455 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-24T10:33:35.468 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-24T10:33:35.468 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-03-24T10:33:35.468 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-24T10:33:35.468 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-03-24T10:33:35.468 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-radosgw x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 14 M 2024-03-24T10:33:35.468 INFO:teuthology.orchestra.run.smithi096.stdout:Upgrading: 2024-03-24T10:33:35.468 INFO:teuthology.orchestra.run.smithi096.stdout: librados2 x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 3.1 M 2024-03-24T10:33:35.468 INFO:teuthology.orchestra.run.smithi096.stdout: librbd1 x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 2.9 M 2024-03-24T10:33:35.468 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-03-24T10:33:35.468 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-base x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 5.0 M 2024-03-24T10:33:35.469 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-common x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 23 M 2024-03-24T10:33:35.469 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-selinux x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 25 k 2024-03-24T10:33:35.469 INFO:teuthology.orchestra.run.smithi096.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-24T10:33:35.469 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs2 x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 636 k 2024-03-24T10:33:35.469 INFO:teuthology.orchestra.run.smithi096.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-24T10:33:35.469 INFO:teuthology.orchestra.run.smithi096.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-24T10:33:35.469 INFO:teuthology.orchestra.run.smithi096.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-24T10:33:35.469 INFO:teuthology.orchestra.run.smithi096.stdout: libradosstriper1 x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 457 k 2024-03-24T10:33:35.469 INFO:teuthology.orchestra.run.smithi096.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-24T10:33:35.469 INFO:teuthology.orchestra.run.smithi096.stdout: librgw2 x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 4.9 M 2024-03-24T10:33:35.469 INFO:teuthology.orchestra.run.smithi096.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-24T10:33:35.469 INFO:teuthology.orchestra.run.smithi096.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-24T10:33:35.469 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-argparse x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 45 k 2024-03-24T10:33:35.469 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-common x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 117 k 2024-03-24T10:33:35.469 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cephfs x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 156 k 2024-03-24T10:33:35.470 INFO:teuthology.orchestra.run.smithi096.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-24T10:33:35.470 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rados x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 320 k 2024-03-24T10:33:35.470 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rbd x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 293 k 2024-03-24T10:33:35.470 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rgw x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 99 k 2024-03-24T10:33:35.470 INFO:teuthology.orchestra.run.smithi096.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-24T10:33:35.470 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:33:35.470 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-03-24T10:33:35.470 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-24T10:33:35.470 INFO:teuthology.orchestra.run.smithi096.stdout:Install 22 Packages 2024-03-24T10:33:35.470 INFO:teuthology.orchestra.run.smithi096.stdout:Upgrade 2 Packages 2024-03-24T10:33:35.470 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:33:35.471 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 58 M 2024-03-24T10:33:35.471 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-03-24T10:33:36.452 INFO:teuthology.orchestra.run.smithi096.stdout:(1/24): ceph-base-17.2.7-726.g9c4e0c34.el9.x86_ 6.5 MB/s | 5.0 MB 00:00 2024-03-24T10:33:36.527 INFO:teuthology.orchestra.run.smithi096.stdout:(2/24): ceph-selinux-17.2.7-726.g9c4e0c34.el9.x 336 kB/s | 25 kB 00:00 2024-03-24T10:33:36.635 INFO:teuthology.orchestra.run.smithi096.stdout:(3/24): libcephfs2-17.2.7-726.g9c4e0c34.el9.x86 5.7 MB/s | 636 kB 00:00 2024-03-24T10:33:36.736 INFO:teuthology.orchestra.run.smithi096.stdout:(4/24): libradosstriper1-17.2.7-726.g9c4e0c34.e 4.5 MB/s | 457 kB 00:00 2024-03-24T10:33:37.278 INFO:teuthology.orchestra.run.smithi096.stdout:(5/24): librgw2-17.2.7-726.g9c4e0c34.el9.x86_64 9.1 MB/s | 4.9 MB 00:00 2024-03-24T10:33:37.336 INFO:teuthology.orchestra.run.smithi096.stdout:(6/24): python3-ceph-argparse-17.2.7-726.g9c4e0 777 kB/s | 45 kB 00:00 2024-03-24T10:33:37.428 INFO:teuthology.orchestra.run.smithi096.stdout:(7/24): python3-ceph-common-17.2.7-726.g9c4e0c3 1.3 MB/s | 117 kB 00:00 2024-03-24T10:33:37.528 INFO:teuthology.orchestra.run.smithi096.stdout:(8/24): python3-cephfs-17.2.7-726.g9c4e0c34.el9 1.5 MB/s | 156 kB 00:00 2024-03-24T10:33:37.572 INFO:teuthology.orchestra.run.smithi094.stdout:lab-extras 40 kB/s | 1.7 kB 00:00 2024-03-24T10:33:37.620 INFO:teuthology.orchestra.run.smithi096.stdout:(9/24): python3-rados-17.2.7-726.g9c4e0c34.el9. 3.4 MB/s | 320 kB 00:00 2024-03-24T10:33:37.762 INFO:teuthology.orchestra.run.smithi096.stdout:(10/24): ceph-radosgw-17.2.7-726.g9c4e0c34.el9. 6.7 MB/s | 14 MB 00:02 2024-03-24T10:33:37.787 INFO:teuthology.orchestra.run.smithi096.stdout:(11/24): python3-rbd-17.2.7-726.g9c4e0c34.el9.x 1.7 MB/s | 293 kB 00:00 2024-03-24T10:33:37.863 INFO:teuthology.orchestra.run.smithi096.stdout:(12/24): python3-rgw-17.2.7-726.g9c4e0c34.el9.x 991 kB/s | 99 kB 00:00 2024-03-24T10:33:38.080 INFO:teuthology.orchestra.run.smithi096.stdout:(13/24): ceph-common-17.2.7-726.g9c4e0c34.el9.x 9.4 MB/s | 23 MB 00:02 2024-03-24T10:33:38.105 INFO:teuthology.orchestra.run.smithi096.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 183 kB/s | 44 kB 00:00 2024-03-24T10:33:38.163 INFO:teuthology.orchestra.run.smithi096.stdout:(15/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 423 kB/s | 159 kB 00:00 2024-03-24T10:33:38.265 INFO:teuthology.orchestra.run.smithi096.stdout:(16/24): librdkafka-1.6.1-102.el9.x86_64.rpm 3.5 MB/s | 661 kB 00:00 2024-03-24T10:33:38.297 INFO:teuthology.orchestra.run.smithi096.stdout:(17/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.4 MB/s | 282 kB 00:00 2024-03-24T10:33:38.322 INFO:teuthology.orchestra.run.smithi096.stdout:(18/24): python3-prettytable-0.7.2-27.el9.noarc 259 kB/s | 41 kB 00:00 2024-03-24T10:33:38.639 INFO:teuthology.orchestra.run.smithi096.stdout:(19/24): liboath-2.6.7-2.el9.x86_64.rpm 142 kB/s | 49 kB 00:00 2024-03-24T10:33:38.665 INFO:teuthology.orchestra.run.smithi096.stdout:(20/24): libunwind-1.6.2-1.el9.x86_64.rpm 197 kB/s | 67 kB 00:00 2024-03-24T10:33:38.774 INFO:teuthology.orchestra.run.smithi096.stdout:(21/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 607 kB/s | 309 kB 00:00 2024-03-24T10:33:38.957 INFO:teuthology.orchestra.run.smithi096.stdout:(22/24): librados2-17.2.7-726.g9c4e0c34.el9.x86 11 MB/s | 3.1 MB 00:00 2024-03-24T10:33:39.090 INFO:teuthology.orchestra.run.smithi096.stdout:(23/24): librbd1-17.2.7-726.g9c4e0c34.el9.x86_6 9.3 MB/s | 2.9 MB 00:00 2024-03-24T10:33:39.415 INFO:teuthology.orchestra.run.smithi096.stdout:(24/24): thrift-0.15.0-2.el9.x86_64.rpm 2.0 MB/s | 1.6 MB 00:00 2024-03-24T10:33:39.419 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-03-24T10:33:39.420 INFO:teuthology.orchestra.run.smithi096.stdout:Total 15 MB/s | 58 MB 00:03 2024-03-24T10:33:39.420 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-03-24T10:33:39.455 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-03-24T10:33:39.455 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-03-24T10:33:39.913 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-03-24T10:33:39.913 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-03-24T10:33:40.023 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-03-24T10:33:40.035 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-03-24T10:33:40.036 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repository Size 2024-03-24T10:33:40.036 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-03-24T10:33:40.036 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-03-24T10:33:40.036 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-radosgw x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 14 M 2024-03-24T10:33:40.036 INFO:teuthology.orchestra.run.smithi094.stdout:Upgrading: 2024-03-24T10:33:40.036 INFO:teuthology.orchestra.run.smithi094.stdout: librados2 x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 3.1 M 2024-03-24T10:33:40.036 INFO:teuthology.orchestra.run.smithi094.stdout: librbd1 x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 2.9 M 2024-03-24T10:33:40.036 INFO:teuthology.orchestra.run.smithi094.stdout:Installing dependencies: 2024-03-24T10:33:40.036 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-base x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 5.0 M 2024-03-24T10:33:40.036 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-common x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 23 M 2024-03-24T10:33:40.036 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-selinux x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 25 k 2024-03-24T10:33:40.036 INFO:teuthology.orchestra.run.smithi094.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-24T10:33:40.036 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs2 x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 636 k 2024-03-24T10:33:40.037 INFO:teuthology.orchestra.run.smithi094.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-24T10:33:40.037 INFO:teuthology.orchestra.run.smithi094.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-24T10:33:40.037 INFO:teuthology.orchestra.run.smithi094.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-24T10:33:40.037 INFO:teuthology.orchestra.run.smithi094.stdout: libradosstriper1 x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 457 k 2024-03-24T10:33:40.037 INFO:teuthology.orchestra.run.smithi094.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-24T10:33:40.037 INFO:teuthology.orchestra.run.smithi094.stdout: librgw2 x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 4.9 M 2024-03-24T10:33:40.037 INFO:teuthology.orchestra.run.smithi094.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-24T10:33:40.037 INFO:teuthology.orchestra.run.smithi094.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-24T10:33:40.037 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-argparse x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 45 k 2024-03-24T10:33:40.037 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-common x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 117 k 2024-03-24T10:33:40.037 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cephfs x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 156 k 2024-03-24T10:33:40.037 INFO:teuthology.orchestra.run.smithi094.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-24T10:33:40.037 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rados x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 320 k 2024-03-24T10:33:40.037 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rbd x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 293 k 2024-03-24T10:33:40.037 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rgw x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 99 k 2024-03-24T10:33:40.038 INFO:teuthology.orchestra.run.smithi094.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-24T10:33:40.038 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:33:40.038 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-03-24T10:33:40.038 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-03-24T10:33:40.038 INFO:teuthology.orchestra.run.smithi094.stdout:Install 22 Packages 2024-03-24T10:33:40.038 INFO:teuthology.orchestra.run.smithi094.stdout:Upgrade 2 Packages 2024-03-24T10:33:40.038 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:33:40.039 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 58 M 2024-03-24T10:33:40.039 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-03-24T10:33:40.973 INFO:teuthology.orchestra.run.smithi094.stdout:(1/24): ceph-base-17.2.7-726.g9c4e0c34.el9.x86_ 6.4 MB/s | 5.0 MB 00:00 2024-03-24T10:33:41.056 INFO:teuthology.orchestra.run.smithi094.stdout:(2/24): ceph-selinux-17.2.7-726.g9c4e0c34.el9.x 302 kB/s | 25 kB 00:00 2024-03-24T10:33:41.157 INFO:teuthology.orchestra.run.smithi094.stdout:(3/24): libcephfs2-17.2.7-726.g9c4e0c34.el9.x86 6.2 MB/s | 636 kB 00:00 2024-03-24T10:33:41.200 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-03-24T10:33:41.235 INFO:teuthology.orchestra.run.smithi094.stdout:(4/24): libradosstriper1-17.2.7-726.g9c4e0c34.e 6.0 MB/s | 457 kB 00:00 2024-03-24T10:33:41.519 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-03-24T10:33:41.585 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : librados2-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/26 2024-03-24T10:33:41.658 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librados2-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/26 2024-03-24T10:33:41.691 INFO:teuthology.orchestra.run.smithi094.stdout:(5/24): librgw2-17.2.7-726.g9c4e0c34.el9.x86_64 11 MB/s | 4.9 MB 00:00 2024-03-24T10:33:41.747 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rados-2:17.2.7-726.g9c4e0c34.el9.x86_64 3/26 2024-03-24T10:33:41.758 INFO:teuthology.orchestra.run.smithi094.stdout:(6/24): python3-ceph-argparse-17.2.7-726.g9c4e0 680 kB/s | 45 kB 00:00 2024-03-24T10:33:41.777 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libcephfs2-2:17.2.7-726.g9c4e0c34.el9.x86_64 4/26 2024-03-24T10:33:41.833 INFO:teuthology.orchestra.run.smithi094.stdout:(7/24): python3-ceph-common-17.2.7-726.g9c4e0c3 1.5 MB/s | 117 kB 00:00 2024-03-24T10:33:41.861 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libcephfs2-2:17.2.7-726.g9c4e0c34.el9.x86_64 4/26 2024-03-24T10:33:41.916 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-03-24T10:33:41.934 INFO:teuthology.orchestra.run.smithi094.stdout:(8/24): python3-cephfs-17.2.7-726.g9c4e0c34.el9 1.5 MB/s | 156 kB 00:00 2024-03-24T10:33:41.957 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-03-24T10:33:42.033 INFO:teuthology.orchestra.run.smithi094.stdout:(9/24): python3-rados-17.2.7-726.g9c4e0c34.el9. 3.1 MB/s | 320 kB 00:00 2024-03-24T10:33:42.142 INFO:teuthology.orchestra.run.smithi094.stdout:(10/24): python3-rbd-17.2.7-726.g9c4e0c34.el9.x 2.6 MB/s | 293 kB 00:00 2024-03-24T10:33:42.225 INFO:teuthology.orchestra.run.smithi094.stdout:(11/24): python3-rgw-17.2.7-726.g9c4e0c34.el9.x 1.2 MB/s | 99 kB 00:00 2024-03-24T10:33:42.334 INFO:teuthology.orchestra.run.smithi094.stdout:(12/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 159 kB 00:00 2024-03-24T10:33:42.368 INFO:teuthology.orchestra.run.smithi094.stdout:(13/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-03-24T10:33:42.394 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-03-24T10:33:42.430 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librgw2-2:17.2.7-726.g9c4e0c34.el9.x86_64 8/26 2024-03-24T10:33:42.434 INFO:teuthology.orchestra.run.smithi094.stdout:(14/24): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 661 kB 00:00 2024-03-24T10:33:42.493 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librgw2-2:17.2.7-726.g9c4e0c34.el9.x86_64 8/26 2024-03-24T10:33:42.521 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-03-24T10:33:42.565 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-ceph-argparse-2:17.2.7-726.g9c4e0c34.el9.x 10/26 2024-03-24T10:33:42.577 INFO:teuthology.orchestra.run.smithi094.stdout:(15/24): ceph-radosgw-17.2.7-726.g9c4e0c34.el9. 5.9 MB/s | 14 MB 00:02 2024-03-24T10:33:42.601 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cephfs-2:17.2.7-726.g9c4e0c34.el9.x86_64 11/26 2024-03-24T10:33:42.601 INFO:teuthology.orchestra.run.smithi094.stdout:(16/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.7 MB/s | 282 kB 00:00 2024-03-24T10:33:42.627 INFO:teuthology.orchestra.run.smithi094.stdout:(17/24): python3-prettytable-0.7.2-27.el9.noarc 834 kB/s | 41 kB 00:00 2024-03-24T10:33:42.664 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 12/26 2024-03-24T10:33:42.691 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libradosstriper1-2:17.2.7-726.g9c4e0c34.el9.x86_64 13/26 2024-03-24T10:33:42.742 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libradosstriper1-2:17.2.7-726.g9c4e0c34.el9.x86_64 13/26 2024-03-24T10:33:42.780 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-03-24T10:33:42.816 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-03-24T10:33:42.848 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-03-24T10:33:42.927 INFO:teuthology.orchestra.run.smithi094.stdout:(18/24): ceph-common-17.2.7-726.g9c4e0c34.el9.x 8.2 MB/s | 23 MB 00:02 2024-03-24T10:33:43.144 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-03-24T10:33:43.170 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : librbd1-2:17.2.7-726.g9c4e0c34.el9.x86_64 18/26 2024-03-24T10:33:43.260 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librbd1-2:17.2.7-726.g9c4e0c34.el9.x86_64 18/26 2024-03-24T10:33:43.333 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 19/26 2024-03-24T10:33:43.364 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86 20/26 2024-03-24T10:33:43.427 INFO:teuthology.orchestra.run.smithi094.stdout:(19/24): liboath-2.6.7-2.el9.x86_64.rpm 61 kB/s | 49 kB 00:00 2024-03-24T10:33:43.511 INFO:teuthology.orchestra.run.smithi094.stdout:(20/24): libunwind-1.6.2-1.el9.x86_64.rpm 126 kB/s | 67 kB 00:00 2024-03-24T10:33:43.619 INFO:teuthology.orchestra.run.smithi094.stdout:(21/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 303 kB/s | 309 kB 00:01 2024-03-24T10:33:43.886 INFO:teuthology.orchestra.run.smithi094.stdout:(22/24): librados2-17.2.7-726.g9c4e0c34.el9.x86 8.2 MB/s | 3.1 MB 00:00 2024-03-24T10:33:43.937 INFO:teuthology.orchestra.run.smithi094.stdout:(23/24): librbd1-17.2.7-726.g9c4e0c34.el9.x86_6 9.2 MB/s | 2.9 MB 00:00 2024-03-24T10:33:45.454 INFO:teuthology.orchestra.run.smithi094.stdout:(24/24): thrift-0.15.0-2.el9.x86_64.rpm 799 kB/s | 1.6 MB 00:02 2024-03-24T10:33:45.460 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-03-24T10:33:45.460 INFO:teuthology.orchestra.run.smithi094.stdout:Total 11 MB/s | 58 MB 00:05 2024-03-24T10:33:45.461 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-03-24T10:33:45.494 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-03-24T10:33:45.494 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-03-24T10:33:45.726 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86_64 21/26 2024-03-24T10:33:45.769 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86_64 21/26 2024-03-24T10:33:45.945 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-03-24T10:33:45.946 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-03-24T10:33:46.315 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86_64 21/26 2024-03-24T10:33:46.347 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-base-2:17.2.7-726.g9c4e0c34.el9.x86_64 22/26 2024-03-24T10:33:46.420 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-base-2:17.2.7-726.g9c4e0c34.el9.x86_64 22/26 2024-03-24T10:33:46.420 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-24T10:33:46.420 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-24T10:33:46.420 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:33:46.457 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-selinux-2:17.2.7-726.g9c4e0c34.el9.x86_64 23/26 2024-03-24T10:33:47.177 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-03-24T10:33:47.526 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-03-24T10:33:47.572 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : librados2-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/26 2024-03-24T10:33:47.655 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librados2-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/26 2024-03-24T10:33:47.726 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rados-2:17.2.7-726.g9c4e0c34.el9.x86_64 3/26 2024-03-24T10:33:47.757 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libcephfs2-2:17.2.7-726.g9c4e0c34.el9.x86_64 4/26 2024-03-24T10:33:47.844 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: libcephfs2-2:17.2.7-726.g9c4e0c34.el9.x86_64 4/26 2024-03-24T10:33:47.902 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-03-24T10:33:47.937 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-03-24T10:33:48.362 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-03-24T10:33:48.394 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librgw2-2:17.2.7-726.g9c4e0c34.el9.x86_64 8/26 2024-03-24T10:33:48.456 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librgw2-2:17.2.7-726.g9c4e0c34.el9.x86_64 8/26 2024-03-24T10:33:48.484 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-03-24T10:33:48.528 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-ceph-argparse-2:17.2.7-726.g9c4e0c34.el9.x 10/26 2024-03-24T10:33:48.564 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cephfs-2:17.2.7-726.g9c4e0c34.el9.x86_64 11/26 2024-03-24T10:33:48.626 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 12/26 2024-03-24T10:33:48.654 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libradosstriper1-2:17.2.7-726.g9c4e0c34.el9.x86_64 13/26 2024-03-24T10:33:48.720 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: libradosstriper1-2:17.2.7-726.g9c4e0c34.el9.x86_64 13/26 2024-03-24T10:33:48.759 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-03-24T10:33:48.796 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-03-24T10:33:48.828 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-03-24T10:33:49.126 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-03-24T10:33:49.174 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : librbd1-2:17.2.7-726.g9c4e0c34.el9.x86_64 18/26 2024-03-24T10:33:49.256 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librbd1-2:17.2.7-726.g9c4e0c34.el9.x86_64 18/26 2024-03-24T10:33:49.330 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 19/26 2024-03-24T10:33:49.360 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86 20/26 2024-03-24T10:33:51.767 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86_64 21/26 2024-03-24T10:33:51.807 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86_64 21/26 2024-03-24T10:33:52.339 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86_64 21/26 2024-03-24T10:33:52.376 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-base-2:17.2.7-726.g9c4e0c34.el9.x86_64 22/26 2024-03-24T10:33:52.464 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-base-2:17.2.7-726.g9c4e0c34.el9.x86_64 22/26 2024-03-24T10:33:52.464 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-24T10:33:52.464 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-24T10:33:52.464 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:33:52.503 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-selinux-2:17.2.7-726.g9c4e0c34.el9.x86_64 23/26 2024-03-24T10:33:56.027 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-selinux-2:17.2.7-726.g9c4e0c34.el9.x86_64 23/26 2024-03-24T10:33:56.027 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /sys 2024-03-24T10:33:56.027 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /proc 2024-03-24T10:33:56.027 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /mnt 2024-03-24T10:33:56.027 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /var/tmp 2024-03-24T10:33:56.027 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /home 2024-03-24T10:33:56.027 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /root 2024-03-24T10:33:56.027 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /tmp 2024-03-24T10:33:56.027 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:33:57.313 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-radosgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 24/26 2024-03-24T10:33:57.386 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 24/26 2024-03-24T10:33:57.386 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T10:33:57.387 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-24T10:33:57.387 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-24T10:33:57.387 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-24T10:33:57.387 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:33:57.388 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-03-24T10:33:57.470 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-03-24T10:33:57.470 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-03-24T10:33:58.645 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-03-24T10:33:58.645 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-base-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/26 2024-03-24T10:33:58.645 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/26 2024-03-24T10:33:58.645 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-radosgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 3/26 2024-03-24T10:33:58.645 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-selinux-2:17.2.7-726.g9c4e0c34.el9.x86_64 4/26 2024-03-24T10:33:58.646 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libcephfs2-2:17.2.7-726.g9c4e0c34.el9.x86_64 5/26 2024-03-24T10:33:58.646 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libradosstriper1-2:17.2.7-726.g9c4e0c34.el9.x86_64 6/26 2024-03-24T10:33:58.646 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librgw2-2:17.2.7-726.g9c4e0c34.el9.x86_64 7/26 2024-03-24T10:33:58.646 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-ceph-argparse-2:17.2.7-726.g9c4e0c34.el9.x 8/26 2024-03-24T10:33:58.646 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86 9/26 2024-03-24T10:33:58.646 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cephfs-2:17.2.7-726.g9c4e0c34.el9.x86_64 10/26 2024-03-24T10:33:58.646 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rados-2:17.2.7-726.g9c4e0c34.el9.x86_64 11/26 2024-03-24T10:33:58.646 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 12/26 2024-03-24T10:33:58.646 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 13/26 2024-03-24T10:33:58.646 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-03-24T10:33:58.646 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-03-24T10:33:58.646 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-03-24T10:33:58.647 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-03-24T10:33:58.647 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-03-24T10:33:58.647 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-03-24T10:33:58.647 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-03-24T10:33:58.647 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-03-24T10:33:58.647 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-03-24T10:33:58.647 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librados2-2:17.2.7-726.g9c4e0c34.el9.x86_64 23/26 2024-03-24T10:33:58.647 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-03-24T10:33:58.647 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librbd1-2:17.2.7-726.g9c4e0c34.el9.x86_64 25/26 2024-03-24T10:33:59.526 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-03-24T10:33:59.526 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:33:59.526 INFO:teuthology.orchestra.run.smithi096.stdout:Upgraded: 2024-03-24T10:33:59.526 INFO:teuthology.orchestra.run.smithi096.stdout: librados2-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:33:59.526 INFO:teuthology.orchestra.run.smithi096.stdout: librbd1-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:33:59.526 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-03-24T10:33:59.526 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-base-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:33:59.526 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:33:59.526 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-radosgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:33:59.526 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-selinux-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:33:59.526 INFO:teuthology.orchestra.run.smithi096.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-24T10:33:59.527 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs2-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:33:59.527 INFO:teuthology.orchestra.run.smithi096.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-24T10:33:59.527 INFO:teuthology.orchestra.run.smithi096.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-24T10:33:59.527 INFO:teuthology.orchestra.run.smithi096.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-24T10:33:59.527 INFO:teuthology.orchestra.run.smithi096.stdout: libradosstriper1-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:33:59.527 INFO:teuthology.orchestra.run.smithi096.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-24T10:33:59.527 INFO:teuthology.orchestra.run.smithi096.stdout: librgw2-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:33:59.527 INFO:teuthology.orchestra.run.smithi096.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-24T10:33:59.527 INFO:teuthology.orchestra.run.smithi096.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-24T10:33:59.527 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-argparse-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:33:59.527 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:33:59.527 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cephfs-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:33:59.527 INFO:teuthology.orchestra.run.smithi096.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-24T10:33:59.527 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rados-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:33:59.528 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:33:59.528 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:33:59.528 INFO:teuthology.orchestra.run.smithi096.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-24T10:33:59.528 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:33:59.528 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-24T10:33:59.883 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-test 2024-03-24T10:34:00.435 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:00:28 ago on Sun 24 Mar 2024 10:33:32 AM UTC. 2024-03-24T10:34:00.590 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-24T10:34:00.590 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-24T10:34:00.590 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-03-24T10:34:00.590 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-24T10:34:00.590 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-03-24T10:34:00.591 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-test x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 48 M 2024-03-24T10:34:00.591 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-03-24T10:34:00.591 INFO:teuthology.orchestra.run.smithi096.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-24T10:34:00.591 INFO:teuthology.orchestra.run.smithi096.stdout: libcephsqlite x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 163 k 2024-03-24T10:34:00.591 INFO:teuthology.orchestra.run.smithi096.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-24T10:34:00.591 INFO:teuthology.orchestra.run.smithi096.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-24T10:34:00.591 INFO:teuthology.orchestra.run.smithi096.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-24T10:34:00.591 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:34:00.591 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-03-24T10:34:00.591 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-24T10:34:00.591 INFO:teuthology.orchestra.run.smithi096.stdout:Install 6 Packages 2024-03-24T10:34:00.591 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:34:00.592 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 49 M 2024-03-24T10:34:00.592 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 196 M 2024-03-24T10:34:00.592 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-03-24T10:34:00.832 INFO:teuthology.orchestra.run.smithi096.stdout:(1/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.1 MB/s | 217 kB 00:00 2024-03-24T10:34:00.915 INFO:teuthology.orchestra.run.smithi096.stdout:(2/6): socat-1.7.4.1-5.el9.x86_64.rpm 3.5 MB/s | 300 kB 00:00 2024-03-24T10:34:00.952 INFO:teuthology.orchestra.run.smithi096.stdout:(3/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.7 MB/s | 63 kB 00:00 2024-03-24T10:34:01.068 INFO:teuthology.orchestra.run.smithi096.stdout:(4/6): libcephsqlite-17.2.7-726.g9c4e0c34.el9.x 375 kB/s | 163 kB 00:00 2024-03-24T10:34:01.111 INFO:teuthology.orchestra.run.smithi096.stdout:(5/6): jq-1.6-16.el9.x86_64.rpm 1.2 MB/s | 187 kB 00:00 2024-03-24T10:34:02.129 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-selinux-2:17.2.7-726.g9c4e0c34.el9.x86_64 23/26 2024-03-24T10:34:02.129 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /sys 2024-03-24T10:34:02.129 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /proc 2024-03-24T10:34:02.129 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /mnt 2024-03-24T10:34:02.129 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /var/tmp 2024-03-24T10:34:02.129 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /home 2024-03-24T10:34:02.129 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /root 2024-03-24T10:34:02.129 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /tmp 2024-03-24T10:34:02.129 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:34:03.443 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-radosgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 24/26 2024-03-24T10:34:03.521 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 24/26 2024-03-24T10:34:03.521 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T10:34:03.521 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-24T10:34:03.522 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-24T10:34:03.522 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-24T10:34:03.522 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:34:03.523 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-03-24T10:34:03.613 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-03-24T10:34:03.613 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-03-24T10:34:04.027 INFO:teuthology.orchestra.run.smithi096.stdout:(6/6): ceph-test-17.2.7-726.g9c4e0c34.el9.x86_6 14 MB/s | 48 MB 00:03 2024-03-24T10:34:04.030 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-03-24T10:34:04.030 INFO:teuthology.orchestra.run.smithi096.stdout:Total 14 MB/s | 49 MB 00:03 2024-03-24T10:34:04.045 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-03-24T10:34:04.066 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-03-24T10:34:04.066 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-03-24T10:34:04.445 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-03-24T10:34:04.445 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-03-24T10:34:04.766 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-03-24T10:34:04.767 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-base-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/26 2024-03-24T10:34:04.767 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/26 2024-03-24T10:34:04.767 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-radosgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 3/26 2024-03-24T10:34:04.767 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-selinux-2:17.2.7-726.g9c4e0c34.el9.x86_64 4/26 2024-03-24T10:34:04.767 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libcephfs2-2:17.2.7-726.g9c4e0c34.el9.x86_64 5/26 2024-03-24T10:34:04.767 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libradosstriper1-2:17.2.7-726.g9c4e0c34.el9.x86_64 6/26 2024-03-24T10:34:04.767 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librgw2-2:17.2.7-726.g9c4e0c34.el9.x86_64 7/26 2024-03-24T10:34:04.767 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-ceph-argparse-2:17.2.7-726.g9c4e0c34.el9.x 8/26 2024-03-24T10:34:04.767 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86 9/26 2024-03-24T10:34:04.767 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cephfs-2:17.2.7-726.g9c4e0c34.el9.x86_64 10/26 2024-03-24T10:34:04.767 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rados-2:17.2.7-726.g9c4e0c34.el9.x86_64 11/26 2024-03-24T10:34:04.767 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 12/26 2024-03-24T10:34:04.767 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 13/26 2024-03-24T10:34:04.767 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-03-24T10:34:04.767 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-03-24T10:34:04.768 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-03-24T10:34:04.768 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-03-24T10:34:04.768 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-03-24T10:34:04.768 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-03-24T10:34:04.768 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-03-24T10:34:04.768 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-03-24T10:34:04.768 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-03-24T10:34:04.768 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librados2-2:17.2.7-726.g9c4e0c34.el9.x86_64 23/26 2024-03-24T10:34:04.768 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-03-24T10:34:04.768 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librbd1-2:17.2.7-726.g9c4e0c34.el9.x86_64 25/26 2024-03-24T10:34:05.010 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-03-24T10:34:05.085 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2024-03-24T10:34:05.119 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2024-03-24T10:34:05.151 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-24T10:34:05.212 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : jq-1.6-16.el9.x86_64 4/6 2024-03-24T10:34:05.238 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libcephsqlite-2:17.2.7-726.g9c4e0c34.el9.x86_64 5/6 2024-03-24T10:34:05.667 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-03-24T10:34:05.667 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:34:05.668 INFO:teuthology.orchestra.run.smithi094.stdout:Upgraded: 2024-03-24T10:34:05.668 INFO:teuthology.orchestra.run.smithi094.stdout: librados2-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:34:05.668 INFO:teuthology.orchestra.run.smithi094.stdout: librbd1-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:34:05.668 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-03-24T10:34:05.668 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-base-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:34:05.668 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:34:05.668 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-radosgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:34:05.668 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-selinux-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:34:05.668 INFO:teuthology.orchestra.run.smithi094.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-24T10:34:05.668 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs2-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:34:05.668 INFO:teuthology.orchestra.run.smithi094.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-24T10:34:05.668 INFO:teuthology.orchestra.run.smithi094.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-24T10:34:05.668 INFO:teuthology.orchestra.run.smithi094.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-24T10:34:05.668 INFO:teuthology.orchestra.run.smithi094.stdout: libradosstriper1-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:34:05.668 INFO:teuthology.orchestra.run.smithi094.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-24T10:34:05.669 INFO:teuthology.orchestra.run.smithi094.stdout: librgw2-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:34:05.669 INFO:teuthology.orchestra.run.smithi094.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-24T10:34:05.669 INFO:teuthology.orchestra.run.smithi094.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-24T10:34:05.669 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-argparse-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:34:05.669 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:34:05.669 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cephfs-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:34:05.669 INFO:teuthology.orchestra.run.smithi094.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-24T10:34:05.669 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rados-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:34:05.669 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:34:05.669 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:34:05.669 INFO:teuthology.orchestra.run.smithi094.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-24T10:34:05.669 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:34:05.669 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-03-24T10:34:06.039 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install ceph-test 2024-03-24T10:34:06.599 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:00:29 ago on Sun 24 Mar 2024 10:33:37 AM UTC. 2024-03-24T10:34:06.756 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-03-24T10:34:06.756 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-03-24T10:34:06.757 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repository Size 2024-03-24T10:34:06.757 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-03-24T10:34:06.757 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-03-24T10:34:06.757 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-test x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 48 M 2024-03-24T10:34:06.757 INFO:teuthology.orchestra.run.smithi094.stdout:Installing dependencies: 2024-03-24T10:34:06.757 INFO:teuthology.orchestra.run.smithi094.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-24T10:34:06.757 INFO:teuthology.orchestra.run.smithi094.stdout: libcephsqlite x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 163 k 2024-03-24T10:34:06.758 INFO:teuthology.orchestra.run.smithi094.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-24T10:34:06.758 INFO:teuthology.orchestra.run.smithi094.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-24T10:34:06.758 INFO:teuthology.orchestra.run.smithi094.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-24T10:34:06.758 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:34:06.758 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-03-24T10:34:06.758 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-03-24T10:34:06.759 INFO:teuthology.orchestra.run.smithi094.stdout:Install 6 Packages 2024-03-24T10:34:06.759 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:34:06.760 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 49 M 2024-03-24T10:34:06.760 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 196 M 2024-03-24T10:34:06.760 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-03-24T10:34:07.179 INFO:teuthology.orchestra.run.smithi094.stdout:(1/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.4 MB/s | 217 kB 00:00 2024-03-24T10:34:07.230 INFO:teuthology.orchestra.run.smithi094.stdout:(2/6): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 300 kB 00:00 2024-03-24T10:34:07.255 INFO:teuthology.orchestra.run.smithi094.stdout:(3/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 2.5 MB/s | 63 kB 00:00 2024-03-24T10:34:07.280 INFO:teuthology.orchestra.run.smithi094.stdout:(4/6): libcephsqlite-17.2.7-726.g9c4e0c34.el9.x 640 kB/s | 163 kB 00:00 2024-03-24T10:34:07.447 INFO:teuthology.orchestra.run.smithi094.stdout:(5/6): jq-1.6-16.el9.x86_64.rpm 975 kB/s | 187 kB 00:00 2024-03-24T10:34:09.589 INFO:teuthology.orchestra.run.smithi094.stdout:(6/6): ceph-test-17.2.7-726.g9c4e0c34.el9.x86_6 19 MB/s | 48 MB 00:02 2024-03-24T10:34:09.592 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-03-24T10:34:09.592 INFO:teuthology.orchestra.run.smithi094.stdout:Total 17 MB/s | 49 MB 00:02 2024-03-24T10:34:09.607 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-03-24T10:34:09.627 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-03-24T10:34:09.627 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-03-24T10:34:09.669 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libcephsqlite-2:17.2.7-726.g9c4e0c34.el9.x86_64 5/6 2024-03-24T10:34:09.754 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-test-2:17.2.7-726.g9c4e0c34.el9.x86_64 6/6 2024-03-24T10:34:10.009 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-03-24T10:34:10.009 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-03-24T10:34:10.592 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-03-24T10:34:10.656 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2024-03-24T10:34:10.698 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2024-03-24T10:34:10.730 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-24T10:34:10.772 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : jq-1.6-16.el9.x86_64 4/6 2024-03-24T10:34:10.800 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libcephsqlite-2:17.2.7-726.g9c4e0c34.el9.x86_64 5/6 2024-03-24T10:34:12.001 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-test-2:17.2.7-726.g9c4e0c34.el9.x86_64 6/6 2024-03-24T10:34:12.001 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-test-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/6 2024-03-24T10:34:12.002 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libcephsqlite-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/6 2024-03-24T10:34:12.002 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-24T10:34:12.002 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 4/6 2024-03-24T10:34:12.002 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/6 2024-03-24T10:34:12.374 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : jq-1.6-16.el9.x86_64 6/6 2024-03-24T10:34:12.374 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:34:12.374 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-03-24T10:34:12.375 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-test-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:34:12.375 INFO:teuthology.orchestra.run.smithi096.stdout: jq-1.6-16.el9.x86_64 2024-03-24T10:34:12.375 INFO:teuthology.orchestra.run.smithi096.stdout: libcephsqlite-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:34:12.375 INFO:teuthology.orchestra.run.smithi096.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-24T10:34:12.375 INFO:teuthology.orchestra.run.smithi096.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-24T10:34:12.375 INFO:teuthology.orchestra.run.smithi096.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-24T10:34:12.375 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:34:12.375 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-24T10:34:12.686 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph 2024-03-24T10:34:13.254 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:00:41 ago on Sun 24 Mar 2024 10:33:32 AM UTC. 2024-03-24T10:34:13.423 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-24T10:34:13.426 INFO:teuthology.orchestra.run.smithi096.stdout:======================================================================================== 2024-03-24T10:34:13.426 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-03-24T10:34:13.426 INFO:teuthology.orchestra.run.smithi096.stdout:======================================================================================== 2024-03-24T10:34:13.426 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-03-24T10:34:13.426 INFO:teuthology.orchestra.run.smithi096.stdout: ceph x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 6.5 k 2024-03-24T10:34:13.426 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-03-24T10:34:13.426 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mds x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 2.1 M 2024-03-24T10:34:13.427 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 1.4 M 2024-03-24T10:34:13.427 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-modules-core noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 240 k 2024-03-24T10:34:13.427 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mon x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 3.7 M 2024-03-24T10:34:13.427 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-osd x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 15 M 2024-03-24T10:34:13.427 INFO:teuthology.orchestra.run.smithi096.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-24T10:34:13.427 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-24T10:34:13.427 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-24T10:34:13.427 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-24T10:34:13.427 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-24T10:34:13.427 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-24T10:34:13.427 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-24T10:34:13.427 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-24T10:34:13.427 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-24T10:34:13.427 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-24T10:34:13.427 INFO:teuthology.orchestra.run.smithi096.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-24T10:34:13.428 INFO:teuthology.orchestra.run.smithi096.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-24T10:34:13.428 INFO:teuthology.orchestra.run.smithi096.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-24T10:34:13.428 INFO:teuthology.orchestra.run.smithi096.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-24T10:34:13.428 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-24T10:34:13.428 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-24T10:34:13.428 INFO:teuthology.orchestra.run.smithi096.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-24T10:34:13.428 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-24T10:34:13.428 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-24T10:34:13.428 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-24T10:34:13.428 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-24T10:34:13.428 INFO:teuthology.orchestra.run.smithi096.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-24T10:34:13.428 INFO:teuthology.orchestra.run.smithi096.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-24T10:34:13.428 INFO:teuthology.orchestra.run.smithi096.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-24T10:34:13.428 INFO:teuthology.orchestra.run.smithi096.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-24T10:34:13.429 INFO:teuthology.orchestra.run.smithi096.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-24T10:34:13.429 INFO:teuthology.orchestra.run.smithi096.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-24T10:34:13.429 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:34:13.429 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-03-24T10:34:13.429 INFO:teuthology.orchestra.run.smithi096.stdout:======================================================================================== 2024-03-24T10:34:13.429 INFO:teuthology.orchestra.run.smithi096.stdout:Install 33 Packages 2024-03-24T10:34:13.429 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:34:13.430 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 27 M 2024-03-24T10:34:13.430 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 100 M 2024-03-24T10:34:13.430 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-03-24T10:34:14.038 INFO:teuthology.orchestra.run.smithi096.stdout:(1/33): ceph-17.2.7-726.g9c4e0c34.el9.x86_64.rp 33 kB/s | 6.5 kB 00:00 2024-03-24T10:34:14.163 INFO:teuthology.orchestra.run.smithi096.stdout:(2/33): ceph-mgr-17.2.7-726.g9c4e0c34.el9.x86_6 4.3 MB/s | 1.4 MB 00:00 2024-03-24T10:34:14.563 INFO:teuthology.orchestra.run.smithi096.stdout:(3/33): ceph-mds-17.2.7-726.g9c4e0c34.el9.x86_6 2.9 MB/s | 2.1 MB 00:00 2024-03-24T10:34:14.655 INFO:teuthology.orchestra.run.smithi096.stdout:(4/33): ceph-mgr-modules-core-17.2.7-726.g9c4e0 2.6 MB/s | 240 kB 00:00 2024-03-24T10:34:14.764 INFO:teuthology.orchestra.run.smithi096.stdout:(5/33): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 144 kB 00:00 2024-03-24T10:34:14.798 INFO:teuthology.orchestra.run.smithi096.stdout:(6/33): python3-markupsafe-1.1.1-12.el9.x86_64. 945 kB/s | 32 kB 00:00 2024-03-24T10:34:14.965 INFO:teuthology.orchestra.run.smithi096.stdout:(7/33): ceph-osd-17.2.7-726.g9c4e0c34.el9.x86_6 19 MB/s | 15 MB 00:00 2024-03-24T10:34:14.990 INFO:teuthology.orchestra.run.smithi096.stdout:(8/33): python3-pytz-2021.1-5.el9.noarch.rpm 246 kB/s | 47 kB 00:00 2024-03-24T10:34:15.015 INFO:teuthology.orchestra.run.smithi096.stdout:(9/33): python3-toml-0.10.2-6.el9.noarch.rpm 774 kB/s | 39 kB 00:00 2024-03-24T10:34:15.040 INFO:teuthology.orchestra.run.smithi096.stdout:(10/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-24T10:34:15.090 INFO:teuthology.orchestra.run.smithi096.stdout:(11/33): python3-cryptography-36.0.1-4.el9.x86_ 15 MB/s | 1.1 MB 00:00 2024-03-24T10:34:15.116 INFO:teuthology.orchestra.run.smithi096.stdout:(12/33): python3-ply-3.11-14.el9.noarch.rpm 1.3 MB/s | 103 kB 00:00 2024-03-24T10:34:15.141 INFO:teuthology.orchestra.run.smithi096.stdout:(13/33): python3-pycparser-2.20-6.el9.noarch.rp 2.4 MB/s | 124 kB 00:00 2024-03-24T10:34:15.166 INFO:teuthology.orchestra.run.smithi096.stdout:(14/33): python3-urllib3-1.26.5-5.el9.noarch.rp 3.6 MB/s | 187 kB 00:00 2024-03-24T10:34:15.236 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: libcephsqlite-2:17.2.7-726.g9c4e0c34.el9.x86_64 5/6 2024-03-24T10:34:15.267 INFO:teuthology.orchestra.run.smithi096.stdout:(15/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 435 kB/s | 43 kB 00:00 2024-03-24T10:34:15.308 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-test-2:17.2.7-726.g9c4e0c34.el9.x86_64 6/6 2024-03-24T10:34:15.342 INFO:teuthology.orchestra.run.smithi096.stdout:(16/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-03-24T10:34:15.426 INFO:teuthology.orchestra.run.smithi096.stdout:(17/33): ceph-mon-17.2.7-726.g9c4e0c34.el9.x86_ 2.7 MB/s | 3.7 MB 00:01 2024-03-24T10:34:15.451 INFO:teuthology.orchestra.run.smithi096.stdout:(18/33): python3-cherrypy-18.6.1-2.el9.noarch.r 3.2 MB/s | 358 kB 00:00 2024-03-24T10:34:15.493 INFO:teuthology.orchestra.run.smithi096.stdout:(19/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 160 kB/s | 11 kB 00:00 2024-03-24T10:34:15.518 INFO:teuthology.orchestra.run.smithi096.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 263 kB/s | 18 kB 00:00 2024-03-24T10:34:15.544 INFO:teuthology.orchestra.run.smithi096.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 457 kB/s | 23 kB 00:00 2024-03-24T10:34:15.569 INFO:teuthology.orchestra.run.smithi096.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 386 kB/s | 19 kB 00:00 2024-03-24T10:34:15.594 INFO:teuthology.orchestra.run.smithi096.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 391 kB/s | 20 kB 00:00 2024-03-24T10:34:15.620 INFO:teuthology.orchestra.run.smithi096.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 921 kB/s | 46 kB 00:00 2024-03-24T10:34:15.645 INFO:teuthology.orchestra.run.smithi096.stdout:(25/33): python3-requests-2.25.1-8.el9.noarch.r 248 kB/s | 125 kB 00:00 2024-03-24T10:34:15.670 INFO:teuthology.orchestra.run.smithi096.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-03-24T10:34:15.696 INFO:teuthology.orchestra.run.smithi096.stdout:(27/33): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-03-24T10:34:15.721 INFO:teuthology.orchestra.run.smithi096.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.8 MB/s | 90 kB 00:00 2024-03-24T10:34:15.746 INFO:teuthology.orchestra.run.smithi096.stdout:(29/33): python3-portend-3.1.0-2.el9.noarch.rpm 163 kB/s | 16 kB 00:00 2024-03-24T10:34:15.771 INFO:teuthology.orchestra.run.smithi096.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 474 kB/s | 36 kB 00:00 2024-03-24T10:34:15.797 INFO:teuthology.orchestra.run.smithi096.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-03-24T10:34:15.822 INFO:teuthology.orchestra.run.smithi096.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 397 kB/s | 20 kB 00:00 2024-03-24T10:34:15.897 INFO:teuthology.orchestra.run.smithi096.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 2.8 MB/s | 427 kB 00:00 2024-03-24T10:34:15.901 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-03-24T10:34:15.901 INFO:teuthology.orchestra.run.smithi096.stdout:Total 11 MB/s | 27 MB 00:02 2024-03-24T10:34:15.915 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-03-24T10:34:15.953 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-03-24T10:34:15.953 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-03-24T10:34:16.298 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-03-24T10:34:16.298 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-03-24T10:34:17.538 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-03-24T10:34:17.580 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-test-2:17.2.7-726.g9c4e0c34.el9.x86_64 6/6 2024-03-24T10:34:17.581 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-test-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/6 2024-03-24T10:34:17.581 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libcephsqlite-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/6 2024-03-24T10:34:17.581 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-24T10:34:17.581 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 4/6 2024-03-24T10:34:17.581 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/6 2024-03-24T10:34:17.600 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-03-24T10:34:17.632 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-03-24T10:34:17.673 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-03-24T10:34:17.764 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-03-24T10:34:17.852 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-03-24T10:34:17.892 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-03-24T10:34:17.921 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : jq-1.6-16.el9.x86_64 6/6 2024-03-24T10:34:17.921 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:34:17.921 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-03-24T10:34:17.921 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-test-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:34:17.921 INFO:teuthology.orchestra.run.smithi094.stdout: jq-1.6-16.el9.x86_64 2024-03-24T10:34:17.921 INFO:teuthology.orchestra.run.smithi094.stdout: libcephsqlite-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:34:17.921 INFO:teuthology.orchestra.run.smithi094.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-24T10:34:17.921 INFO:teuthology.orchestra.run.smithi094.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-24T10:34:17.921 INFO:teuthology.orchestra.run.smithi094.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-24T10:34:17.921 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:34:17.921 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-03-24T10:34:17.964 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-03-24T10:34:18.018 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 8/33 2024-03-24T10:34:18.073 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 9/33 2024-03-24T10:34:18.123 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-ply-3.11-14.el9.noarch 10/33 2024-03-24T10:34:18.185 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 11/33 2024-03-24T10:34:18.234 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install ceph 2024-03-24T10:34:18.371 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 12/33 2024-03-24T10:34:18.420 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 13/33 2024-03-24T10:34:18.458 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 14/33 2024-03-24T10:34:18.501 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 15/33 2024-03-24T10:34:18.531 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 16/33 2024-03-24T10:34:18.607 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 17/33 2024-03-24T10:34:18.642 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-24T10:34:18.675 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 19/33 2024-03-24T10:34:18.716 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 20/33 2024-03-24T10:34:18.751 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/33 2024-03-24T10:34:18.785 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 22/33 2024-03-24T10:34:18.794 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:00:41 ago on Sun 24 Mar 2024 10:33:37 AM UTC. 2024-03-24T10:34:18.874 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 23/33 2024-03-24T10:34:18.916 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 24/33 2024-03-24T10:34:18.945 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-03-24T10:34:18.947 INFO:teuthology.orchestra.run.smithi094.stdout:======================================================================================== 2024-03-24T10:34:18.947 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repository Size 2024-03-24T10:34:18.947 INFO:teuthology.orchestra.run.smithi094.stdout:======================================================================================== 2024-03-24T10:34:18.947 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-03-24T10:34:18.947 INFO:teuthology.orchestra.run.smithi094.stdout: ceph x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 6.5 k 2024-03-24T10:34:18.947 INFO:teuthology.orchestra.run.smithi094.stdout:Installing dependencies: 2024-03-24T10:34:18.948 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mds x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 2.1 M 2024-03-24T10:34:18.948 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 1.4 M 2024-03-24T10:34:18.948 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-modules-core noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 240 k 2024-03-24T10:34:18.948 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mon x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 3.7 M 2024-03-24T10:34:18.948 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-osd x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 15 M 2024-03-24T10:34:18.948 INFO:teuthology.orchestra.run.smithi094.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-24T10:34:18.948 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-24T10:34:18.948 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-24T10:34:18.948 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-24T10:34:18.948 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-24T10:34:18.948 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-24T10:34:18.948 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-24T10:34:18.948 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-24T10:34:18.949 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-24T10:34:18.949 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-24T10:34:18.949 INFO:teuthology.orchestra.run.smithi094.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-24T10:34:18.949 INFO:teuthology.orchestra.run.smithi094.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-24T10:34:18.949 INFO:teuthology.orchestra.run.smithi094.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-24T10:34:18.949 INFO:teuthology.orchestra.run.smithi094.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-24T10:34:18.949 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-24T10:34:18.949 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-24T10:34:18.949 INFO:teuthology.orchestra.run.smithi094.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-24T10:34:18.949 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-24T10:34:18.949 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-24T10:34:18.949 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-24T10:34:18.949 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-24T10:34:18.949 INFO:teuthology.orchestra.run.smithi094.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-24T10:34:18.949 INFO:teuthology.orchestra.run.smithi094.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-24T10:34:18.950 INFO:teuthology.orchestra.run.smithi094.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-24T10:34:18.950 INFO:teuthology.orchestra.run.smithi094.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-24T10:34:18.950 INFO:teuthology.orchestra.run.smithi094.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-24T10:34:18.950 INFO:teuthology.orchestra.run.smithi094.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-24T10:34:18.950 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:34:18.950 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-03-24T10:34:18.950 INFO:teuthology.orchestra.run.smithi094.stdout:======================================================================================== 2024-03-24T10:34:18.950 INFO:teuthology.orchestra.run.smithi094.stdout:Install 33 Packages 2024-03-24T10:34:18.950 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:34:18.951 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 27 M 2024-03-24T10:34:18.951 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 100 M 2024-03-24T10:34:18.951 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-03-24T10:34:18.989 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/33 2024-03-24T10:34:19.126 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/33 2024-03-24T10:34:19.250 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-24T10:34:19.376 INFO:teuthology.orchestra.run.smithi094.stdout:(1/33): ceph-17.2.7-726.g9c4e0c34.el9.x86_64.rp 32 kB/s | 6.5 kB 00:00 2024-03-24T10:34:19.406 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-726.g9c4e0c34.el9.n 28/33 2024-03-24T10:34:19.430 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-2:17.2.7-726.g9c4e0c34.el9.x86_64 29/33 2024-03-24T10:34:19.455 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-2:17.2.7-726.g9c4e0c34.el9.x86_64 29/33 2024-03-24T10:34:19.456 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T10:34:19.456 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-24T10:34:19.456 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-24T10:34:19.456 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-24T10:34:19.456 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:34:19.684 INFO:teuthology.orchestra.run.smithi094.stdout:(2/33): ceph-mds-17.2.7-726.g9c4e0c34.el9.x86_6 4.1 MB/s | 2.1 MB 00:00 2024-03-24T10:34:19.752 INFO:teuthology.orchestra.run.smithi094.stdout:(3/33): ceph-mgr-17.2.7-726.g9c4e0c34.el9.x86_6 2.4 MB/s | 1.4 MB 00:00 2024-03-24T10:34:19.860 INFO:teuthology.orchestra.run.smithi094.stdout:(4/33): ceph-mgr-modules-core-17.2.7-726.g9c4e0 2.2 MB/s | 240 kB 00:00 2024-03-24T10:34:19.977 INFO:teuthology.orchestra.run.smithi094.stdout:(5/33): python3-mako-1.1.4-6.el9.noarch.rpm 1.2 MB/s | 144 kB 00:00 2024-03-24T10:34:20.002 INFO:teuthology.orchestra.run.smithi094.stdout:(6/33): python3-markupsafe-1.1.1-12.el9.x86_64. 1.2 MB/s | 32 kB 00:00 2024-03-24T10:34:20.028 INFO:teuthology.orchestra.run.smithi094.stdout:(7/33): python3-pytz-2021.1-5.el9.noarch.rpm 1.9 MB/s | 47 kB 00:00 2024-03-24T10:34:20.053 INFO:teuthology.orchestra.run.smithi094.stdout:(8/33): python3-toml-0.10.2-6.el9.noarch.rpm 1.5 MB/s | 39 kB 00:00 2024-03-24T10:34:20.095 INFO:teuthology.orchestra.run.smithi094.stdout:(9/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.6 MB/s | 241 kB 00:00 2024-03-24T10:34:20.170 INFO:teuthology.orchestra.run.smithi094.stdout:(10/33): python3-cryptography-36.0.1-4.el9.x86_ 15 MB/s | 1.1 MB 00:00 2024-03-24T10:34:20.204 INFO:teuthology.orchestra.run.smithi094.stdout:(11/33): python3-ply-3.11-14.el9.noarch.rpm 3.0 MB/s | 103 kB 00:00 2024-03-24T10:34:20.238 INFO:teuthology.orchestra.run.smithi094.stdout:(12/33): python3-pycparser-2.20-6.el9.noarch.rp 3.6 MB/s | 124 kB 00:00 2024-03-24T10:34:20.280 INFO:teuthology.orchestra.run.smithi094.stdout:(13/33): python3-urllib3-1.26.5-5.el9.noarch.rp 4.4 MB/s | 187 kB 00:00 2024-03-24T10:34:20.471 INFO:teuthology.orchestra.run.smithi094.stdout:(14/33): ceph-osd-17.2.7-726.g9c4e0c34.el9.x86_ 19 MB/s | 15 MB 00:00 2024-03-24T10:34:20.530 INFO:teuthology.orchestra.run.smithi094.stdout:(15/33): python3-requests-2.25.1-8.el9.noarch.r 499 kB/s | 125 kB 00:00 2024-03-24T10:34:20.639 INFO:teuthology.orchestra.run.smithi094.stdout:(16/33): ceph-mon-17.2.7-726.g9c4e0c34.el9.x86_ 2.9 MB/s | 3.7 MB 00:01 2024-03-24T10:34:20.797 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-osd-2:17.2.7-726.g9c4e0c34.el9.x86_64 30/33 2024-03-24T10:34:20.805 INFO:teuthology.orchestra.run.smithi094.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 130 kB/s | 43 kB 00:00 2024-03-24T10:34:20.823 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-osd-2:17.2.7-726.g9c4e0c34.el9.x86_64 30/33 2024-03-24T10:34:20.824 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T10:34:20.824 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-24T10:34:20.824 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-24T10:34:20.824 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-24T10:34:20.824 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:34:20.989 INFO:teuthology.orchestra.run.smithi094.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 58 kB/s | 11 kB 00:00 2024-03-24T10:34:21.065 INFO:teuthology.orchestra.run.smithi094.stdout:(19/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 321 kB/s | 172 kB 00:00 2024-03-24T10:34:21.173 INFO:teuthology.orchestra.run.smithi094.stdout:(20/33): python3-cherrypy-18.6.1-2.el9.noarch.r 671 kB/s | 358 kB 00:00 2024-03-24T10:34:21.198 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mon-2:17.2.7-726.g9c4e0c34.el9.x86_64 31/33 2024-03-24T10:34:21.221 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mon-2:17.2.7-726.g9c4e0c34.el9.x86_64 31/33 2024-03-24T10:34:21.221 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T10:34:21.221 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-24T10:34:21.222 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-24T10:34:21.222 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-24T10:34:21.222 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:34:21.248 INFO:teuthology.orchestra.run.smithi094.stdout:(21/33): python3-jaraco-classes-3.2.1-5.el9.noa 68 kB/s | 18 kB 00:00 2024-03-24T10:34:21.324 INFO:teuthology.orchestra.run.smithi094.stdout:(22/33): python3-jaraco-collections-3.0.0-8.el9 89 kB/s | 23 kB 00:00 2024-03-24T10:34:21.424 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mds-2:17.2.7-726.g9c4e0c34.el9.x86_64 32/33 2024-03-24T10:34:21.441 INFO:teuthology.orchestra.run.smithi094.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 73 kB/s | 19 kB 00:00 2024-03-24T10:34:21.446 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mds-2:17.2.7-726.g9c4e0c34.el9.x86_64 32/33 2024-03-24T10:34:21.447 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T10:34:21.447 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-24T10:34:21.447 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-24T10:34:21.447 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-24T10:34:21.447 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:34:21.512 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-2:17.2.7-726.g9c4e0c34.el9.x86_64 33/33 2024-03-24T10:34:21.516 INFO:teuthology.orchestra.run.smithi094.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 74 kB/s | 20 kB 00:00 2024-03-24T10:34:21.666 INFO:teuthology.orchestra.run.smithi094.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 135 kB/s | 46 kB 00:00 2024-03-24T10:34:21.784 INFO:teuthology.orchestra.run.smithi094.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 230 kB/s | 79 kB 00:00 2024-03-24T10:34:21.934 INFO:teuthology.orchestra.run.smithi094.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 61 kB/s | 16 kB 00:00 2024-03-24T10:34:22.042 INFO:teuthology.orchestra.run.smithi094.stdout:(28/33): python3-pecan-1.4.2-3.el9.noarch.rpm 517 kB/s | 272 kB 00:00 2024-03-24T10:34:22.218 INFO:teuthology.orchestra.run.smithi094.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 207 kB/s | 90 kB 00:00 2024-03-24T10:34:22.243 INFO:teuthology.orchestra.run.smithi094.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 116 kB/s | 36 kB 00:00 2024-03-24T10:34:22.518 INFO:teuthology.orchestra.run.smithi094.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 73 kB/s | 20 kB 00:00 2024-03-24T10:34:22.543 INFO:teuthology.orchestra.run.smithi094.stdout:(32/33): python3-webob-1.8.7-6.el9.noarch.rpm 459 kB/s | 230 kB 00:00 2024-03-24T10:34:22.802 INFO:teuthology.orchestra.run.smithi094.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 732 kB/s | 427 kB 00:00 2024-03-24T10:34:22.806 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-03-24T10:34:22.806 INFO:teuthology.orchestra.run.smithi094.stdout:Total 6.9 MB/s | 27 MB 00:03 2024-03-24T10:34:22.820 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-03-24T10:34:22.859 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-03-24T10:34:22.859 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-03-24T10:34:23.197 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-03-24T10:34:23.197 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-03-24T10:34:23.253 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-2:17.2.7-726.g9c4e0c34.el9.x86_64 33/33 2024-03-24T10:34:23.253 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/33 2024-03-24T10:34:23.253 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mds-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/33 2024-03-24T10:34:23.253 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-2:17.2.7-726.g9c4e0c34.el9.x86_64 3/33 2024-03-24T10:34:23.253 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mon-2:17.2.7-726.g9c4e0c34.el9.x86_64 4/33 2024-03-24T10:34:23.253 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-osd-2:17.2.7-726.g9c4e0c34.el9.x86_64 5/33 2024-03-24T10:34:23.254 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-726.g9c4e0c34.el9.n 6/33 2024-03-24T10:34:23.254 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/33 2024-03-24T10:34:23.254 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/33 2024-03-24T10:34:23.254 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 9/33 2024-03-24T10:34:23.254 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 10/33 2024-03-24T10:34:23.254 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/33 2024-03-24T10:34:23.254 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 12/33 2024-03-24T10:34:23.254 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/33 2024-03-24T10:34:23.254 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/33 2024-03-24T10:34:23.254 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 15/33 2024-03-24T10:34:23.254 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 16/33 2024-03-24T10:34:23.254 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-03-24T10:34:23.254 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-24T10:34:23.254 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-03-24T10:34:23.254 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-03-24T10:34:23.255 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-03-24T10:34:23.255 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-03-24T10:34:23.255 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-03-24T10:34:23.255 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-03-24T10:34:23.255 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-03-24T10:34:23.256 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-03-24T10:34:23.256 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-24T10:34:23.256 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-03-24T10:34:23.256 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-03-24T10:34:23.256 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-03-24T10:34:23.256 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-03-24T10:34:23.256 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-03-24T10:34:24.015 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-03-24T10:34:24.015 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:34:24.015 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-03-24T10:34:24.015 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:34:24.015 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mds-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:34:24.015 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:34:24.015 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-modules-core-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-24T10:34:24.015 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mon-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:34:24.016 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-osd-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:34:24.016 INFO:teuthology.orchestra.run.smithi096.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-24T10:34:24.016 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-24T10:34:24.016 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-24T10:34:24.016 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-24T10:34:24.016 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-24T10:34:24.016 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-24T10:34:24.016 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-24T10:34:24.016 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-24T10:34:24.016 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-24T10:34:24.016 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-24T10:34:24.016 INFO:teuthology.orchestra.run.smithi096.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-24T10:34:24.016 INFO:teuthology.orchestra.run.smithi096.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-24T10:34:24.016 INFO:teuthology.orchestra.run.smithi096.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-24T10:34:24.017 INFO:teuthology.orchestra.run.smithi096.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-24T10:34:24.017 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-24T10:34:24.017 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ply-3.11-14.el9.noarch 2024-03-24T10:34:24.017 INFO:teuthology.orchestra.run.smithi096.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-24T10:34:24.017 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-24T10:34:24.017 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-24T10:34:24.017 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-24T10:34:24.017 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-24T10:34:24.017 INFO:teuthology.orchestra.run.smithi096.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-24T10:34:24.017 INFO:teuthology.orchestra.run.smithi096.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-24T10:34:24.017 INFO:teuthology.orchestra.run.smithi096.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-24T10:34:24.017 INFO:teuthology.orchestra.run.smithi096.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-24T10:34:24.017 INFO:teuthology.orchestra.run.smithi096.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-24T10:34:24.017 INFO:teuthology.orchestra.run.smithi096.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-24T10:34:24.018 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:34:24.018 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-24T10:34:24.395 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-base 2024-03-24T10:34:24.435 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-03-24T10:34:24.496 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-03-24T10:34:24.537 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-03-24T10:34:24.570 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-03-24T10:34:24.660 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-03-24T10:34:24.732 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-03-24T10:34:24.772 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-03-24T10:34:24.845 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-03-24T10:34:24.899 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 8/33 2024-03-24T10:34:24.937 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 9/33 2024-03-24T10:34:24.982 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:00:52 ago on Sun 24 Mar 2024 10:33:32 AM UTC. 2024-03-24T10:34:24.988 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-ply-3.11-14.el9.noarch 10/33 2024-03-24T10:34:25.048 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 11/33 2024-03-24T10:34:25.076 INFO:teuthology.orchestra.run.smithi096.stdout:Package ceph-base-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-24T10:34:25.127 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-24T10:34:25.128 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2024-03-24T10:34:25.128 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-24T10:34:25.196 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install cephadm 2024-03-24T10:34:25.240 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 12/33 2024-03-24T10:34:25.299 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 13/33 2024-03-24T10:34:25.331 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 14/33 2024-03-24T10:34:25.365 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 15/33 2024-03-24T10:34:25.395 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 16/33 2024-03-24T10:34:25.471 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 17/33 2024-03-24T10:34:25.505 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-24T10:34:25.538 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 19/33 2024-03-24T10:34:25.579 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 20/33 2024-03-24T10:34:25.614 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/33 2024-03-24T10:34:25.649 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 22/33 2024-03-24T10:34:25.737 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 23/33 2024-03-24T10:34:25.754 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:00:53 ago on Sun 24 Mar 2024 10:33:32 AM UTC. 2024-03-24T10:34:25.780 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 24/33 2024-03-24T10:34:25.851 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/33 2024-03-24T10:34:25.901 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-24T10:34:25.902 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-24T10:34:25.902 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-03-24T10:34:25.902 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-24T10:34:25.903 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-03-24T10:34:25.903 INFO:teuthology.orchestra.run.smithi096.stdout: cephadm noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 82 k 2024-03-24T10:34:25.903 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:34:25.903 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-03-24T10:34:25.903 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-24T10:34:25.903 INFO:teuthology.orchestra.run.smithi096.stdout:Install 1 Package 2024-03-24T10:34:25.903 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:34:25.903 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 82 k 2024-03-24T10:34:25.903 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 372 k 2024-03-24T10:34:25.903 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-03-24T10:34:25.997 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/33 2024-03-24T10:34:26.122 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-24T10:34:26.223 INFO:teuthology.orchestra.run.smithi096.stdout:cephadm-17.2.7-726.g9c4e0c34.el9.noarch.rpm 257 kB/s | 82 kB 00:00 2024-03-24T10:34:26.224 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-03-24T10:34:26.224 INFO:teuthology.orchestra.run.smithi096.stdout:Total 255 kB/s | 82 kB 00:00 2024-03-24T10:34:26.224 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-03-24T10:34:26.229 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-03-24T10:34:26.229 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-03-24T10:34:26.232 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-03-24T10:34:26.232 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-03-24T10:34:26.275 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-726.g9c4e0c34.el9.n 28/33 2024-03-24T10:34:26.302 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-2:17.2.7-726.g9c4e0c34.el9.x86_64 29/33 2024-03-24T10:34:26.308 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-03-24T10:34:26.328 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-2:17.2.7-726.g9c4e0c34.el9.x86_64 29/33 2024-03-24T10:34:26.328 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T10:34:26.328 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-24T10:34:26.328 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-24T10:34:26.328 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-24T10:34:26.328 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:34:26.712 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 1/1 2024-03-24T10:34:26.803 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 1/1 2024-03-24T10:34:27.178 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 1/1 2024-03-24T10:34:27.445 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 1/1 2024-03-24T10:34:27.445 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:34:27.445 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-03-24T10:34:27.445 INFO:teuthology.orchestra.run.smithi096.stdout: cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-24T10:34:27.445 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:34:27.445 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-24T10:34:27.663 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-immutable-object-cache 2024-03-24T10:34:27.703 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-osd-2:17.2.7-726.g9c4e0c34.el9.x86_64 30/33 2024-03-24T10:34:27.731 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-osd-2:17.2.7-726.g9c4e0c34.el9.x86_64 30/33 2024-03-24T10:34:27.731 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T10:34:27.731 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-24T10:34:27.731 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-24T10:34:27.731 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-24T10:34:27.731 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:34:28.104 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mon-2:17.2.7-726.g9c4e0c34.el9.x86_64 31/33 2024-03-24T10:34:28.127 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mon-2:17.2.7-726.g9c4e0c34.el9.x86_64 31/33 2024-03-24T10:34:28.128 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T10:34:28.128 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-24T10:34:28.128 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-24T10:34:28.128 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-24T10:34:28.128 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:34:28.235 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:00:56 ago on Sun 24 Mar 2024 10:33:32 AM UTC. 2024-03-24T10:34:28.330 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mds-2:17.2.7-726.g9c4e0c34.el9.x86_64 32/33 2024-03-24T10:34:28.353 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mds-2:17.2.7-726.g9c4e0c34.el9.x86_64 32/33 2024-03-24T10:34:28.353 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T10:34:28.353 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-24T10:34:28.353 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-24T10:34:28.353 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-24T10:34:28.353 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:34:28.389 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-24T10:34:28.390 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-24T10:34:28.390 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repo Size 2024-03-24T10:34:28.390 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-24T10:34:28.390 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-03-24T10:34:28.390 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 137 k 2024-03-24T10:34:28.390 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:34:28.391 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-03-24T10:34:28.391 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-24T10:34:28.391 INFO:teuthology.orchestra.run.smithi096.stdout:Install 1 Package 2024-03-24T10:34:28.391 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:34:28.391 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 137 k 2024-03-24T10:34:28.391 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 379 k 2024-03-24T10:34:28.391 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-03-24T10:34:28.419 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-2:17.2.7-726.g9c4e0c34.el9.x86_64 33/33 2024-03-24T10:34:28.592 INFO:teuthology.orchestra.run.smithi096.stdout:ceph-immutable-object-cache-17.2.7-726.g9c4e0c3 683 kB/s | 137 kB 00:00 2024-03-24T10:34:28.593 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-03-24T10:34:28.593 INFO:teuthology.orchestra.run.smithi096.stdout:Total 677 kB/s | 137 kB 00:00 2024-03-24T10:34:28.593 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-03-24T10:34:28.601 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-03-24T10:34:28.601 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-03-24T10:34:28.653 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-03-24T10:34:28.653 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-03-24T10:34:28.789 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-03-24T10:34:28.852 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-726.g9c4e0c34.e 1/1 2024-03-24T10:34:28.877 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-726.g9c4e0c34.e 1/1 2024-03-24T10:34:28.877 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T10:34:28.877 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-24T10:34:28.877 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:34:29.803 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-726.g9c4e0c34.e 1/1 2024-03-24T10:34:29.803 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:34:29.803 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-03-24T10:34:29.803 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-immutable-object-cache-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:34:29.803 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:34:29.804 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-24T10:34:30.046 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-mgr 2024-03-24T10:34:30.085 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-2:17.2.7-726.g9c4e0c34.el9.x86_64 33/33 2024-03-24T10:34:30.085 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/33 2024-03-24T10:34:30.085 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mds-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/33 2024-03-24T10:34:30.085 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-2:17.2.7-726.g9c4e0c34.el9.x86_64 3/33 2024-03-24T10:34:30.085 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mon-2:17.2.7-726.g9c4e0c34.el9.x86_64 4/33 2024-03-24T10:34:30.085 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-osd-2:17.2.7-726.g9c4e0c34.el9.x86_64 5/33 2024-03-24T10:34:30.085 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-726.g9c4e0c34.el9.n 6/33 2024-03-24T10:34:30.085 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/33 2024-03-24T10:34:30.085 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/33 2024-03-24T10:34:30.085 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 9/33 2024-03-24T10:34:30.085 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 10/33 2024-03-24T10:34:30.086 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/33 2024-03-24T10:34:30.086 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 12/33 2024-03-24T10:34:30.086 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/33 2024-03-24T10:34:30.086 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/33 2024-03-24T10:34:30.086 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 15/33 2024-03-24T10:34:30.086 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 16/33 2024-03-24T10:34:30.086 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-03-24T10:34:30.086 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-24T10:34:30.086 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-03-24T10:34:30.086 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-03-24T10:34:30.086 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-03-24T10:34:30.086 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-03-24T10:34:30.086 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-03-24T10:34:30.086 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-03-24T10:34:30.087 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-03-24T10:34:30.087 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-03-24T10:34:30.087 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-24T10:34:30.087 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-03-24T10:34:30.087 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-03-24T10:34:30.087 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-03-24T10:34:30.087 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-03-24T10:34:30.087 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-03-24T10:34:30.612 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:00:58 ago on Sun 24 Mar 2024 10:33:32 AM UTC. 2024-03-24T10:34:30.710 INFO:teuthology.orchestra.run.smithi096.stdout:Package ceph-mgr-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-24T10:34:30.759 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-24T10:34:30.760 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2024-03-24T10:34:30.760 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-24T10:34:30.829 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-mgr-dashboard 2024-03-24T10:34:30.966 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-03-24T10:34:30.966 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:34:30.966 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-03-24T10:34:30.966 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:34:30.966 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mds-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:34:30.967 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:34:30.967 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-modules-core-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-24T10:34:30.967 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mon-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:34:30.967 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-osd-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:34:30.967 INFO:teuthology.orchestra.run.smithi094.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-24T10:34:30.967 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-24T10:34:30.967 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-24T10:34:30.967 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-24T10:34:30.967 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-24T10:34:30.967 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-24T10:34:30.967 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-24T10:34:30.967 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-24T10:34:30.967 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-24T10:34:30.967 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-24T10:34:30.967 INFO:teuthology.orchestra.run.smithi094.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-24T10:34:30.968 INFO:teuthology.orchestra.run.smithi094.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-24T10:34:30.968 INFO:teuthology.orchestra.run.smithi094.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-24T10:34:30.968 INFO:teuthology.orchestra.run.smithi094.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-24T10:34:30.968 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-24T10:34:30.968 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ply-3.11-14.el9.noarch 2024-03-24T10:34:30.968 INFO:teuthology.orchestra.run.smithi094.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-24T10:34:30.968 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-24T10:34:30.968 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-24T10:34:30.968 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-24T10:34:30.968 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-24T10:34:30.968 INFO:teuthology.orchestra.run.smithi094.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-24T10:34:30.968 INFO:teuthology.orchestra.run.smithi094.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-24T10:34:30.968 INFO:teuthology.orchestra.run.smithi094.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-24T10:34:30.968 INFO:teuthology.orchestra.run.smithi094.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-24T10:34:30.968 INFO:teuthology.orchestra.run.smithi094.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-24T10:34:30.969 INFO:teuthology.orchestra.run.smithi094.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-24T10:34:30.969 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:34:30.969 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-03-24T10:34:31.287 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install ceph-base 2024-03-24T10:34:31.380 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:00:59 ago on Sun 24 Mar 2024 10:33:32 AM UTC. 2024-03-24T10:34:31.530 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-24T10:34:31.535 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-24T10:34:31.535 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-03-24T10:34:31.535 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-24T10:34:31.535 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-03-24T10:34:31.535 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-dashboard noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 1.7 M 2024-03-24T10:34:31.535 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-03-24T10:34:31.535 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-grafana-dashboards noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 23 k 2024-03-24T10:34:31.535 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-prometheus-alerts noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 15 k 2024-03-24T10:34:31.535 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-03-24T10:34:31.536 INFO:teuthology.orchestra.run.smithi096.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-24T10:34:31.536 INFO:teuthology.orchestra.run.smithi096.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-24T10:34:31.536 INFO:teuthology.orchestra.run.smithi096.stdout:Installing weak dependencies: 2024-03-24T10:34:31.536 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-03-24T10:34:31.536 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:34:31.536 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-03-24T10:34:31.536 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-24T10:34:31.536 INFO:teuthology.orchestra.run.smithi096.stdout:Install 7 Packages 2024-03-24T10:34:31.536 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:34:31.536 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 2.0 M 2024-03-24T10:34:31.536 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 8.1 M 2024-03-24T10:34:31.536 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-03-24T10:34:31.849 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:00:54 ago on Sun 24 Mar 2024 10:33:37 AM UTC. 2024-03-24T10:34:31.944 INFO:teuthology.orchestra.run.smithi094.stdout:Package ceph-base-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-24T10:34:31.996 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-03-24T10:34:31.996 INFO:teuthology.orchestra.run.smithi094.stdout:Nothing to do. 2024-03-24T10:34:31.997 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-03-24T10:34:32.066 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install cephadm 2024-03-24T10:34:32.080 INFO:teuthology.orchestra.run.smithi096.stdout:(1/7): ceph-prometheus-alerts-17.2.7-726.g9c4e0 74 kB/s | 15 kB 00:00 2024-03-24T10:34:32.189 INFO:teuthology.orchestra.run.smithi096.stdout:(2/7): ceph-grafana-dashboards-17.2.7-726.g9c4e 76 kB/s | 23 kB 00:00 2024-03-24T10:34:32.256 INFO:teuthology.orchestra.run.smithi096.stdout:(3/7): ceph-mgr-dashboard-17.2.7-726.g9c4e0c34. 4.5 MB/s | 1.7 MB 00:00 2024-03-24T10:34:32.523 INFO:teuthology.orchestra.run.smithi096.stdout:(4/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 20 kB/s | 9.0 kB 00:00 2024-03-24T10:34:32.625 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:00:55 ago on Sun 24 Mar 2024 10:33:37 AM UTC. 2024-03-24T10:34:32.727 INFO:teuthology.orchestra.run.smithi096.stdout:(5/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 66 kB/s | 31 kB 00:00 2024-03-24T10:34:32.766 INFO:teuthology.orchestra.run.smithi096.stdout:(6/7): python3-jwt-2.4.0-1.el9.noarch.rpm 74 kB/s | 41 kB 00:00 2024-03-24T10:34:32.772 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-03-24T10:34:32.773 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-03-24T10:34:32.773 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repository Size 2024-03-24T10:34:32.773 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-03-24T10:34:32.773 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-03-24T10:34:32.773 INFO:teuthology.orchestra.run.smithi094.stdout: cephadm noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 82 k 2024-03-24T10:34:32.773 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:34:32.773 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-03-24T10:34:32.773 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-03-24T10:34:32.773 INFO:teuthology.orchestra.run.smithi094.stdout:Install 1 Package 2024-03-24T10:34:32.773 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:34:32.774 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 82 k 2024-03-24T10:34:32.774 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 372 k 2024-03-24T10:34:32.774 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-03-24T10:34:33.024 INFO:teuthology.orchestra.run.smithi096.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 377 kB/s | 188 kB 00:00 2024-03-24T10:34:33.028 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-03-24T10:34:33.028 INFO:teuthology.orchestra.run.smithi096.stdout:Total 1.3 MB/s | 2.0 MB 00:01 2024-03-24T10:34:33.028 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-03-24T10:34:33.036 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-03-24T10:34:33.036 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-03-24T10:34:33.102 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-03-24T10:34:33.103 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-03-24T10:34:33.106 INFO:teuthology.orchestra.run.smithi094.stdout:cephadm-17.2.7-726.g9c4e0c34.el9.noarch.rpm 247 kB/s | 82 kB 00:00 2024-03-24T10:34:33.107 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-03-24T10:34:33.107 INFO:teuthology.orchestra.run.smithi094.stdout:Total 246 kB/s | 82 kB 00:00 2024-03-24T10:34:33.107 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-03-24T10:34:33.112 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-03-24T10:34:33.112 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-03-24T10:34:33.114 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-03-24T10:34:33.114 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-03-24T10:34:33.192 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-03-24T10:34:33.401 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-03-24T10:34:33.458 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-03-24T10:34:33.492 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-03-24T10:34:33.539 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-03-24T10:34:33.571 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-03-24T10:34:33.599 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-726.g9c4e0c34.el9.no 5/7 2024-03-24T10:34:33.654 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 1/1 2024-03-24T10:34:33.736 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 1/1 2024-03-24T10:34:33.822 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-726.g9c4e0c34.el9.n 6/7 2024-03-24T10:34:33.853 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-726.g9c4e0c34.el9.noarch 7/7 2024-03-24T10:34:34.119 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 1/1 2024-03-24T10:34:34.406 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-726.g9c4e0c34.el9.noarch 7/7 2024-03-24T10:34:34.407 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-726.g9c4e0c34.el9.n 1/7 2024-03-24T10:34:34.407 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-726.g9c4e0c34.el9.noarch 2/7 2024-03-24T10:34:34.407 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-726.g9c4e0c34.el9.no 3/7 2024-03-24T10:34:34.407 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-03-24T10:34:34.407 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-03-24T10:34:34.407 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-03-24T10:34:34.411 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 1/1 2024-03-24T10:34:34.411 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:34:34.411 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-03-24T10:34:34.411 INFO:teuthology.orchestra.run.smithi094.stdout: cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-24T10:34:34.411 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:34:34.411 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-03-24T10:34:34.617 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install ceph-immutable-object-cache 2024-03-24T10:34:34.702 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-03-24T10:34:34.703 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:34:34.703 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-03-24T10:34:34.703 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-grafana-dashboards-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-24T10:34:34.703 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-dashboard-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-24T10:34:34.703 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-prometheus-alerts-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-24T10:34:34.703 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-03-24T10:34:34.703 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-03-24T10:34:34.703 INFO:teuthology.orchestra.run.smithi096.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-24T10:34:34.703 INFO:teuthology.orchestra.run.smithi096.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-24T10:34:34.703 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:34:34.703 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-24T10:34:34.935 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-mgr-diskprediction-local 2024-03-24T10:34:35.193 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:00:58 ago on Sun 24 Mar 2024 10:33:37 AM UTC. 2024-03-24T10:34:35.342 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-03-24T10:34:35.342 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-03-24T10:34:35.342 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repo Size 2024-03-24T10:34:35.342 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-03-24T10:34:35.343 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-03-24T10:34:35.343 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 137 k 2024-03-24T10:34:35.343 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:34:35.343 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-03-24T10:34:35.343 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-03-24T10:34:35.343 INFO:teuthology.orchestra.run.smithi094.stdout:Install 1 Package 2024-03-24T10:34:35.343 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:34:35.343 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 137 k 2024-03-24T10:34:35.343 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 379 k 2024-03-24T10:34:35.343 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-03-24T10:34:35.486 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:01:03 ago on Sun 24 Mar 2024 10:33:32 AM UTC. 2024-03-24T10:34:35.567 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-immutable-object-cache-17.2.7-726.g9c4e0c3 612 kB/s | 137 kB 00:00 2024-03-24T10:34:35.568 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-03-24T10:34:35.568 INFO:teuthology.orchestra.run.smithi094.stdout:Total 609 kB/s | 137 kB 00:00 2024-03-24T10:34:35.568 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-03-24T10:34:35.576 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-03-24T10:34:35.576 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-03-24T10:34:35.628 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-03-24T10:34:35.628 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-03-24T10:34:35.635 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-24T10:34:35.636 INFO:teuthology.orchestra.run.smithi096.stdout:=========================================================================================== 2024-03-24T10:34:35.636 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-03-24T10:34:35.636 INFO:teuthology.orchestra.run.smithi096.stdout:=========================================================================================== 2024-03-24T10:34:35.636 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-03-24T10:34:35.636 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 7.4 M 2024-03-24T10:34:35.636 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-03-24T10:34:35.636 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-24T10:34:35.636 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-24T10:34:35.636 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-24T10:34:35.637 INFO:teuthology.orchestra.run.smithi096.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-24T10:34:35.637 INFO:teuthology.orchestra.run.smithi096.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-24T10:34:35.637 INFO:teuthology.orchestra.run.smithi096.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-24T10:34:35.637 INFO:teuthology.orchestra.run.smithi096.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-24T10:34:35.637 INFO:teuthology.orchestra.run.smithi096.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-24T10:34:35.637 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-24T10:34:35.637 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-24T10:34:35.637 INFO:teuthology.orchestra.run.smithi096.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-24T10:34:35.637 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:34:35.637 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-03-24T10:34:35.637 INFO:teuthology.orchestra.run.smithi096.stdout:=========================================================================================== 2024-03-24T10:34:35.637 INFO:teuthology.orchestra.run.smithi096.stdout:Install 12 Packages 2024-03-24T10:34:35.637 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:34:35.638 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 37 M 2024-03-24T10:34:35.638 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 209 M 2024-03-24T10:34:35.638 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-03-24T10:34:35.766 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-03-24T10:34:35.827 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-726.g9c4e0c34.e 1/1 2024-03-24T10:34:35.851 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-726.g9c4e0c34.e 1/1 2024-03-24T10:34:35.852 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T10:34:35.852 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-24T10:34:35.852 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:34:35.873 INFO:teuthology.orchestra.run.smithi096.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 240 kB/s | 32 kB 00:00 2024-03-24T10:34:35.906 INFO:teuthology.orchestra.run.smithi096.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 536 kB/s | 17 kB 00:00 2024-03-24T10:34:35.965 INFO:teuthology.orchestra.run.smithi096.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 591 kB/s | 34 kB 00:00 2024-03-24T10:34:36.057 INFO:teuthology.orchestra.run.smithi096.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 9.5 MB/s | 3.0 MB 00:00 2024-03-24T10:34:36.382 INFO:teuthology.orchestra.run.smithi096.stdout:(5/12): ceph-mgr-diskprediction-local-17.2.7-72 12 MB/s | 7.4 MB 00:00 2024-03-24T10:34:36.449 INFO:teuthology.orchestra.run.smithi096.stdout:(6/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 9.8 MB/s | 4.7 MB 00:00 2024-03-24T10:34:36.516 INFO:teuthology.orchestra.run.smithi096.stdout:(7/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 10 MB/s | 4.8 MB 00:00 2024-03-24T10:34:36.542 INFO:teuthology.orchestra.run.smithi096.stdout:(8/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 1.3 MB/s | 217 kB 00:00 2024-03-24T10:34:36.841 INFO:teuthology.orchestra.run.smithi096.stdout:(9/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 40 MB/s | 16 MB 00:00 2024-03-24T10:34:36.867 INFO:teuthology.orchestra.run.smithi096.stdout:(10/12): libquadmath-11.4.1-3.el9.x86_64.rpm 605 kB/s | 195 kB 00:00 2024-03-24T10:34:36.903 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-726.g9c4e0c34.e 1/1 2024-03-24T10:34:36.903 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:34:36.903 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-03-24T10:34:36.903 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-immutable-object-cache-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:34:36.903 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:34:36.903 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-03-24T10:34:36.958 INFO:teuthology.orchestra.run.smithi096.stdout:(11/12): libgfortran-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 803 kB 00:00 2024-03-24T10:34:37.067 INFO:teuthology.orchestra.run.smithi096.stdout:(12/12): python3-devel-3.9.17-2.el9.x86_64.rpm 1.1 MB/s | 245 kB 00:00 2024-03-24T10:34:37.069 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-03-24T10:34:37.069 INFO:teuthology.orchestra.run.smithi096.stdout:Total 26 MB/s | 37 MB 00:01 2024-03-24T10:34:37.110 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-03-24T10:34:37.130 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install ceph-mgr 2024-03-24T10:34:37.135 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-03-24T10:34:37.135 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-03-24T10:34:37.558 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-03-24T10:34:37.559 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-03-24T10:34:37.710 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:00 ago on Sun 24 Mar 2024 10:33:37 AM UTC. 2024-03-24T10:34:37.807 INFO:teuthology.orchestra.run.smithi094.stdout:Package ceph-mgr-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-24T10:34:37.858 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-03-24T10:34:37.859 INFO:teuthology.orchestra.run.smithi094.stdout:Nothing to do. 2024-03-24T10:34:37.859 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-03-24T10:34:37.925 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install ceph-mgr-dashboard 2024-03-24T10:34:38.258 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-03-24T10:34:38.334 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-03-24T10:34:38.372 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-03-24T10:34:38.471 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-03-24T10:34:38.479 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:01 ago on Sun 24 Mar 2024 10:33:37 AM UTC. 2024-03-24T10:34:38.517 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-03-24T10:34:38.628 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-03-24T10:34:38.630 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-03-24T10:34:38.630 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repository Size 2024-03-24T10:34:38.630 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-03-24T10:34:38.630 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-03-24T10:34:38.630 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-dashboard noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 1.7 M 2024-03-24T10:34:38.630 INFO:teuthology.orchestra.run.smithi094.stdout:Installing dependencies: 2024-03-24T10:34:38.631 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-grafana-dashboards noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 23 k 2024-03-24T10:34:38.631 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-prometheus-alerts noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 15 k 2024-03-24T10:34:38.631 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-03-24T10:34:38.631 INFO:teuthology.orchestra.run.smithi094.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-24T10:34:38.631 INFO:teuthology.orchestra.run.smithi094.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-24T10:34:38.631 INFO:teuthology.orchestra.run.smithi094.stdout:Installing weak dependencies: 2024-03-24T10:34:38.631 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-03-24T10:34:38.631 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:34:38.631 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-03-24T10:34:38.631 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-03-24T10:34:38.631 INFO:teuthology.orchestra.run.smithi094.stdout:Install 7 Packages 2024-03-24T10:34:38.631 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:34:38.631 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 2.0 M 2024-03-24T10:34:38.631 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 8.1 M 2024-03-24T10:34:38.631 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-03-24T10:34:38.693 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-24T10:34:38.724 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-24T10:34:38.801 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-03-24T10:34:39.014 INFO:teuthology.orchestra.run.smithi094.stdout:(1/7): ceph-grafana-dashboards-17.2.7-726.g9c4e 121 kB/s | 23 kB 00:00 2024-03-24T10:34:39.039 INFO:teuthology.orchestra.run.smithi094.stdout:(2/7): ceph-prometheus-alerts-17.2.7-726.g9c4e0 67 kB/s | 15 kB 00:00 2024-03-24T10:34:39.128 INFO:teuthology.orchestra.run.smithi094.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 83 kB/s | 9.0 kB 00:00 2024-03-24T10:34:39.183 INFO:teuthology.orchestra.run.smithi094.stdout:(4/7): python3-jwt-2.4.0-1.el9.noarch.rpm 288 kB/s | 41 kB 00:00 2024-03-24T10:34:39.207 INFO:teuthology.orchestra.run.smithi094.stdout:(5/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 367 kB/s | 31 kB 00:00 2024-03-24T10:34:39.299 INFO:teuthology.orchestra.run.smithi094.stdout:(6/7): ceph-mgr-dashboard-17.2.7-726.g9c4e0c34. 3.5 MB/s | 1.7 MB 00:00 2024-03-24T10:34:39.332 INFO:teuthology.orchestra.run.smithi094.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 1.2 MB/s | 188 kB 00:00 2024-03-24T10:34:39.334 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-03-24T10:34:39.334 INFO:teuthology.orchestra.run.smithi094.stdout:Total 2.8 MB/s | 2.0 MB 00:00 2024-03-24T10:34:39.334 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-03-24T10:34:39.342 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-03-24T10:34:39.342 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-03-24T10:34:39.401 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-03-24T10:34:39.402 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-03-24T10:34:39.500 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-03-24T10:34:39.606 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-03-24T10:34:39.699 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-03-24T10:34:39.792 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-03-24T10:34:39.826 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-03-24T10:34:39.872 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-03-24T10:34:39.905 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-03-24T10:34:39.933 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-726.g9c4e0c34.el9.no 5/7 2024-03-24T10:34:40.166 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-726.g9c4e0c34.el9.n 6/7 2024-03-24T10:34:40.212 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-726.g9c4e0c34.el9.noarch 7/7 2024-03-24T10:34:40.742 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-726.g9c4e0c34.el9.noarch 7/7 2024-03-24T10:34:40.742 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-726.g9c4e0c34.el9.n 1/7 2024-03-24T10:34:40.742 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-726.g9c4e0c34.el9.noarch 2/7 2024-03-24T10:34:40.742 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-726.g9c4e0c34.el9.no 3/7 2024-03-24T10:34:40.742 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-03-24T10:34:40.742 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-03-24T10:34:40.743 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-03-24T10:34:40.915 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-03-24T10:34:41.050 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-03-24T10:34:41.051 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:34:41.051 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-03-24T10:34:41.051 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-grafana-dashboards-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-24T10:34:41.051 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-dashboard-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-24T10:34:41.051 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-prometheus-alerts-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-24T10:34:41.051 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-03-24T10:34:41.051 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-03-24T10:34:41.051 INFO:teuthology.orchestra.run.smithi094.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-24T10:34:41.051 INFO:teuthology.orchestra.run.smithi094.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-24T10:34:41.051 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:34:41.051 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-03-24T10:34:41.272 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install ceph-mgr-diskprediction-local 2024-03-24T10:34:41.826 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:04 ago on Sun 24 Mar 2024 10:33:37 AM UTC. 2024-03-24T10:34:41.957 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-03-24T10:34:41.975 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-03-24T10:34:41.977 INFO:teuthology.orchestra.run.smithi094.stdout:=========================================================================================== 2024-03-24T10:34:41.977 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repository Size 2024-03-24T10:34:41.977 INFO:teuthology.orchestra.run.smithi094.stdout:=========================================================================================== 2024-03-24T10:34:41.977 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-03-24T10:34:41.977 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 7.4 M 2024-03-24T10:34:41.977 INFO:teuthology.orchestra.run.smithi094.stdout:Installing dependencies: 2024-03-24T10:34:41.978 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-24T10:34:41.978 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-24T10:34:41.978 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-24T10:34:41.978 INFO:teuthology.orchestra.run.smithi094.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-24T10:34:41.978 INFO:teuthology.orchestra.run.smithi094.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-24T10:34:41.978 INFO:teuthology.orchestra.run.smithi094.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-24T10:34:41.978 INFO:teuthology.orchestra.run.smithi094.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-24T10:34:41.978 INFO:teuthology.orchestra.run.smithi094.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-24T10:34:41.978 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-24T10:34:41.978 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-24T10:34:41.978 INFO:teuthology.orchestra.run.smithi094.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-24T10:34:41.978 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:34:41.979 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-03-24T10:34:41.979 INFO:teuthology.orchestra.run.smithi094.stdout:=========================================================================================== 2024-03-24T10:34:41.979 INFO:teuthology.orchestra.run.smithi094.stdout:Install 12 Packages 2024-03-24T10:34:41.979 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:34:41.979 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 37 M 2024-03-24T10:34:41.979 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 209 M 2024-03-24T10:34:41.979 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-03-24T10:34:41.991 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-726.g9c4e0c 12/12 2024-03-24T10:34:42.249 INFO:teuthology.orchestra.run.smithi094.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 213 kB/s | 32 kB 00:00 2024-03-24T10:34:42.290 INFO:teuthology.orchestra.run.smithi094.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 423 kB/s | 17 kB 00:00 2024-03-24T10:34:42.324 INFO:teuthology.orchestra.run.smithi094.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 1.0 MB/s | 34 kB 00:00 2024-03-24T10:34:42.482 INFO:teuthology.orchestra.run.smithi094.stdout:(4/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 30 MB/s | 4.7 MB 00:00 2024-03-24T10:34:42.550 INFO:teuthology.orchestra.run.smithi094.stdout:(5/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 6.6 MB/s | 3.0 MB 00:00 2024-03-24T10:34:42.625 INFO:teuthology.orchestra.run.smithi094.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 34 MB/s | 4.8 MB 00:00 2024-03-24T10:34:42.650 INFO:teuthology.orchestra.run.smithi094.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.2 MB/s | 217 kB 00:00 2024-03-24T10:34:43.000 INFO:teuthology.orchestra.run.smithi094.stdout:(8/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 42 MB/s | 16 MB 00:00 2024-03-24T10:34:43.142 INFO:teuthology.orchestra.run.smithi094.stdout:(9/12): libquadmath-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 195 kB 00:00 2024-03-24T10:34:43.226 INFO:teuthology.orchestra.run.smithi094.stdout:(10/12): ceph-mgr-diskprediction-local-17.2.7-7 6.6 MB/s | 7.4 MB 00:01 2024-03-24T10:34:43.284 INFO:teuthology.orchestra.run.smithi094.stdout:(11/12): libgfortran-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 803 kB 00:00 2024-03-24T10:34:43.543 INFO:teuthology.orchestra.run.smithi094.stdout:(12/12): python3-devel-3.9.17-2.el9.x86_64.rpm 613 kB/s | 245 kB 00:00 2024-03-24T10:34:43.545 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-03-24T10:34:43.546 INFO:teuthology.orchestra.run.smithi094.stdout:Total 24 MB/s | 37 MB 00:01 2024-03-24T10:34:43.586 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-03-24T10:34:43.609 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-03-24T10:34:43.609 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-03-24T10:34:44.049 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-03-24T10:34:44.049 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-03-24T10:34:44.573 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-726.g9c4e0c 12/12 2024-03-24T10:34:44.574 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-726.g9c4e0c 1/12 2024-03-24T10:34:44.574 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-03-24T10:34:44.574 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-03-24T10:34:44.574 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-03-24T10:34:44.574 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-24T10:34:44.574 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-24T10:34:44.574 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-03-24T10:34:44.574 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-03-24T10:34:44.574 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-03-24T10:34:44.574 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-03-24T10:34:44.574 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-03-24T10:34:44.745 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-03-24T10:34:44.822 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-03-24T10:34:44.857 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-03-24T10:34:44.932 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-03-24T10:34:44.932 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:34:44.932 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-03-24T10:34:44.932 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-diskprediction-local-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-24T10:34:44.932 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-24T10:34:44.932 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-24T10:34:44.932 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-24T10:34:44.932 INFO:teuthology.orchestra.run.smithi096.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-24T10:34:44.933 INFO:teuthology.orchestra.run.smithi096.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-24T10:34:44.933 INFO:teuthology.orchestra.run.smithi096.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-24T10:34:44.933 INFO:teuthology.orchestra.run.smithi096.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-24T10:34:44.933 INFO:teuthology.orchestra.run.smithi096.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-24T10:34:44.933 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-24T10:34:44.933 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-24T10:34:44.933 INFO:teuthology.orchestra.run.smithi096.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-24T10:34:44.933 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:34:44.933 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-24T10:34:44.949 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-03-24T10:34:44.984 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-03-24T10:34:45.152 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-24T10:34:45.183 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-24T10:34:45.259 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-03-24T10:34:45.297 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-mgr-rook 2024-03-24T10:34:45.884 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:01:13 ago on Sun 24 Mar 2024 10:33:32 AM UTC. 2024-03-24T10:34:45.911 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-03-24T10:34:46.037 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-03-24T10:34:46.045 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-24T10:34:46.046 INFO:teuthology.orchestra.run.smithi096.stdout:======================================================================================= 2024-03-24T10:34:46.046 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-03-24T10:34:46.046 INFO:teuthology.orchestra.run.smithi096.stdout:======================================================================================= 2024-03-24T10:34:46.046 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-03-24T10:34:46.047 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-rook noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 47 k 2024-03-24T10:34:46.047 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-03-24T10:34:46.047 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-24T10:34:46.047 INFO:teuthology.orchestra.run.smithi096.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-24T10:34:46.047 INFO:teuthology.orchestra.run.smithi096.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-24T10:34:46.047 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-24T10:34:46.047 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-24T10:34:46.047 INFO:teuthology.orchestra.run.smithi096.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-24T10:34:46.047 INFO:teuthology.orchestra.run.smithi096.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-24T10:34:46.047 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-24T10:34:46.047 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-24T10:34:46.047 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-24T10:34:46.047 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-24T10:34:46.047 INFO:teuthology.orchestra.run.smithi096.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-24T10:34:46.048 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:34:46.048 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-03-24T10:34:46.048 INFO:teuthology.orchestra.run.smithi096.stdout:======================================================================================= 2024-03-24T10:34:46.048 INFO:teuthology.orchestra.run.smithi096.stdout:Install 13 Packages 2024-03-24T10:34:46.048 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:34:46.048 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 2.0 M 2024-03-24T10:34:46.048 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 26 M 2024-03-24T10:34:46.048 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-03-24T10:34:46.359 INFO:teuthology.orchestra.run.smithi096.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 195 kB/s | 24 kB 00:00 2024-03-24T10:34:46.384 INFO:teuthology.orchestra.run.smithi096.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 120 kB/s | 18 kB 00:00 2024-03-24T10:34:46.443 INFO:teuthology.orchestra.run.smithi096.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.1 MB/s | 174 kB 00:00 2024-03-24T10:34:46.468 INFO:teuthology.orchestra.run.smithi096.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 132 kB 00:00 2024-03-24T10:34:46.502 INFO:teuthology.orchestra.run.smithi096.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 3.5 MB/s | 211 kB 00:00 2024-03-24T10:34:46.527 INFO:teuthology.orchestra.run.smithi096.stdout:(6/13): ceph-mgr-rook-17.2.7-726.g9c4e0c34.el9. 162 kB/s | 47 kB 00:00 2024-03-24T10:34:46.552 INFO:teuthology.orchestra.run.smithi096.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 512 kB/s | 43 kB 00:00 2024-03-24T10:34:46.719 INFO:teuthology.orchestra.run.smithi096.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 148 kB/s | 32 kB 00:00 2024-03-24T10:34:46.745 INFO:teuthology.orchestra.run.smithi096.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 78 kB/s | 15 kB 00:00 2024-03-24T10:34:46.837 INFO:teuthology.orchestra.run.smithi096.stdout:(10/13): python3-google-auth-2.28.2-1.el9.noarc 714 kB/s | 203 kB 00:00 2024-03-24T10:34:46.862 INFO:teuthology.orchestra.run.smithi096.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 505 kB/s | 59 kB 00:00 2024-03-24T10:34:46.895 INFO:teuthology.orchestra.run.smithi096.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.5 MB/s | 90 kB 00:00 2024-03-24T10:34:46.930 INFO:teuthology.orchestra.run.smithi096.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 4.9 MB/s | 1.0 MB 00:00 2024-03-24T10:34:46.933 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-03-24T10:34:46.933 INFO:teuthology.orchestra.run.smithi096.stdout:Total 2.3 MB/s | 2.0 MB 00:00 2024-03-24T10:34:46.933 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-03-24T10:34:46.946 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-03-24T10:34:46.946 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-03-24T10:34:47.033 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-03-24T10:34:47.034 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-03-24T10:34:47.284 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-03-24T10:34:47.556 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-03-24T10:34:47.718 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-03-24T10:34:47.788 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-03-24T10:34:47.844 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-03-24T10:34:47.881 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-03-24T10:34:47.922 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-03-24T10:34:48.016 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-03-24T10:34:48.133 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-03-24T10:34:48.190 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-03-24T10:34:48.308 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-03-24T10:34:48.334 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-726.g9c4e0c 12/12 2024-03-24T10:34:48.910 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-03-24T10:34:48.954 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-03-24T10:34:48.988 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-03-24T10:34:49.034 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-03-24T10:34:49.058 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-rook-2:17.2.7-726.g9c4e0c34.el9.noarch 13/13 2024-03-24T10:34:49.797 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-726.g9c4e0c34.el9.noarch 13/13 2024-03-24T10:34:49.797 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-rook-2:17.2.7-726.g9c4e0c34.el9.noarch 1/13 2024-03-24T10:34:49.797 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-03-24T10:34:49.797 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-03-24T10:34:49.797 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-03-24T10:34:49.798 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-03-24T10:34:49.798 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-03-24T10:34:49.798 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-03-24T10:34:49.798 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-03-24T10:34:49.798 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-03-24T10:34:49.798 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-03-24T10:34:49.798 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-03-24T10:34:49.798 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-03-24T10:34:50.149 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-03-24T10:34:50.149 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:34:50.149 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-03-24T10:34:50.149 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-rook-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-24T10:34:50.149 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-24T10:34:50.149 INFO:teuthology.orchestra.run.smithi096.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-24T10:34:50.149 INFO:teuthology.orchestra.run.smithi096.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-24T10:34:50.149 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-24T10:34:50.149 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-24T10:34:50.150 INFO:teuthology.orchestra.run.smithi096.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-24T10:34:50.150 INFO:teuthology.orchestra.run.smithi096.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-24T10:34:50.150 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-24T10:34:50.150 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-24T10:34:50.150 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-24T10:34:50.150 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-24T10:34:50.150 INFO:teuthology.orchestra.run.smithi096.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-24T10:34:50.150 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:34:50.150 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-24T10:34:50.388 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-mgr-cephadm 2024-03-24T10:34:50.944 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-726.g9c4e0c 12/12 2024-03-24T10:34:50.944 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-726.g9c4e0c 1/12 2024-03-24T10:34:50.944 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-03-24T10:34:50.944 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-03-24T10:34:50.944 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-03-24T10:34:50.944 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-24T10:34:50.944 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-24T10:34:50.944 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-03-24T10:34:50.944 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-03-24T10:34:50.944 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-03-24T10:34:50.944 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-03-24T10:34:50.944 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-03-24T10:34:50.946 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:01:18 ago on Sun 24 Mar 2024 10:33:32 AM UTC. 2024-03-24T10:34:51.094 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-24T10:34:51.095 INFO:teuthology.orchestra.run.smithi096.stdout:======================================================================================= 2024-03-24T10:34:51.095 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-03-24T10:34:51.095 INFO:teuthology.orchestra.run.smithi096.stdout:======================================================================================= 2024-03-24T10:34:51.095 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-03-24T10:34:51.095 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-cephadm noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 123 k 2024-03-24T10:34:51.096 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-03-24T10:34:51.096 INFO:teuthology.orchestra.run.smithi096.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-24T10:34:51.096 INFO:teuthology.orchestra.run.smithi096.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-24T10:34:51.096 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-24T10:34:51.096 INFO:teuthology.orchestra.run.smithi096.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-24T10:34:51.096 INFO:teuthology.orchestra.run.smithi096.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-24T10:34:51.096 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:34:51.096 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-03-24T10:34:51.096 INFO:teuthology.orchestra.run.smithi096.stdout:======================================================================================= 2024-03-24T10:34:51.096 INFO:teuthology.orchestra.run.smithi096.stdout:Install 6 Packages 2024-03-24T10:34:51.096 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:34:51.097 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 6.8 M 2024-03-24T10:34:51.097 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 33 M 2024-03-24T10:34:51.097 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-03-24T10:34:51.335 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-03-24T10:34:51.335 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:34:51.335 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-03-24T10:34:51.335 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-diskprediction-local-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-24T10:34:51.336 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-24T10:34:51.336 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-24T10:34:51.336 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-24T10:34:51.336 INFO:teuthology.orchestra.run.smithi094.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-24T10:34:51.336 INFO:teuthology.orchestra.run.smithi094.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-24T10:34:51.336 INFO:teuthology.orchestra.run.smithi094.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-24T10:34:51.336 INFO:teuthology.orchestra.run.smithi094.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-24T10:34:51.336 INFO:teuthology.orchestra.run.smithi094.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-24T10:34:51.336 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-24T10:34:51.336 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-24T10:34:51.336 INFO:teuthology.orchestra.run.smithi094.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-24T10:34:51.336 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:34:51.336 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-03-24T10:34:51.423 INFO:teuthology.orchestra.run.smithi096.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 2.2 MB/s | 248 kB 00:00 2024-03-24T10:34:51.532 INFO:teuthology.orchestra.run.smithi096.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 5.0 MB/s | 548 kB 00:00 2024-03-24T10:34:51.607 INFO:teuthology.orchestra.run.smithi096.stdout:(3/6): python3-babel-2.9.1-2.el9.noarch.rpm 19 MB/s | 5.8 MB 00:00 2024-03-24T10:34:51.624 INFO:teuthology.orchestra.run.smithi096.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 631 kB/s | 58 kB 00:00 2024-03-24T10:34:51.674 INFO:teuthology.orchestra.run.smithi096.stdout:(5/6): ceph-mgr-cephadm-17.2.7-726.g9c4e0c34.el 339 kB/s | 123 kB 00:00 2024-03-24T10:34:51.691 INFO:teuthology.orchestra.run.smithi096.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 610 kB/s | 51 kB 00:00 2024-03-24T10:34:51.692 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-03-24T10:34:51.693 INFO:teuthology.orchestra.run.smithi096.stdout:Total 11 MB/s | 6.8 MB 00:00 2024-03-24T10:34:51.702 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install ceph-mgr-rook 2024-03-24T10:34:51.705 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-03-24T10:34:51.715 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-03-24T10:34:51.716 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-03-24T10:34:51.804 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-03-24T10:34:51.805 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-03-24T10:34:52.093 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-03-24T10:34:52.272 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-03-24T10:34:52.275 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:15 ago on Sun 24 Mar 2024 10:33:37 AM UTC. 2024-03-24T10:34:52.317 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-03-24T10:34:52.425 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-03-24T10:34:52.426 INFO:teuthology.orchestra.run.smithi094.stdout:======================================================================================= 2024-03-24T10:34:52.427 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repository Size 2024-03-24T10:34:52.427 INFO:teuthology.orchestra.run.smithi094.stdout:======================================================================================= 2024-03-24T10:34:52.427 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-03-24T10:34:52.427 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-rook noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 47 k 2024-03-24T10:34:52.427 INFO:teuthology.orchestra.run.smithi094.stdout:Installing dependencies: 2024-03-24T10:34:52.427 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-24T10:34:52.427 INFO:teuthology.orchestra.run.smithi094.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-24T10:34:52.427 INFO:teuthology.orchestra.run.smithi094.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-24T10:34:52.427 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-24T10:34:52.427 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-24T10:34:52.427 INFO:teuthology.orchestra.run.smithi094.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-24T10:34:52.427 INFO:teuthology.orchestra.run.smithi094.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-24T10:34:52.428 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-24T10:34:52.428 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-24T10:34:52.428 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-24T10:34:52.428 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-24T10:34:52.428 INFO:teuthology.orchestra.run.smithi094.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-24T10:34:52.428 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:34:52.428 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-03-24T10:34:52.428 INFO:teuthology.orchestra.run.smithi094.stdout:======================================================================================= 2024-03-24T10:34:52.428 INFO:teuthology.orchestra.run.smithi094.stdout:Install 13 Packages 2024-03-24T10:34:52.428 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:34:52.429 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 2.0 M 2024-03-24T10:34:52.429 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 26 M 2024-03-24T10:34:52.429 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-03-24T10:34:52.594 INFO:teuthology.orchestra.run.smithi094.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 199 kB/s | 24 kB 00:00 2024-03-24T10:34:52.611 INFO:teuthology.orchestra.run.smithi094.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 130 kB/s | 18 kB 00:00 2024-03-24T10:34:52.663 INFO:teuthology.orchestra.run.smithi094.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.6 MB/s | 174 kB 00:00 2024-03-24T10:34:52.678 INFO:teuthology.orchestra.run.smithi094.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 132 kB 00:00 2024-03-24T10:34:52.704 INFO:teuthology.orchestra.run.smithi094.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 4.9 MB/s | 211 kB 00:00 2024-03-24T10:34:52.729 INFO:teuthology.orchestra.run.smithi094.stdout:(6/13): ceph-mgr-rook-17.2.7-726.g9c4e0c34.el9. 184 kB/s | 47 kB 00:00 2024-03-24T10:34:52.754 INFO:teuthology.orchestra.run.smithi094.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 569 kB/s | 43 kB 00:00 2024-03-24T10:34:52.833 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-03-24T10:34:52.863 INFO:teuthology.orchestra.run.smithi094.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 138 kB/s | 15 kB 00:00 2024-03-24T10:34:52.888 INFO:teuthology.orchestra.run.smithi094.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 175 kB/s | 32 kB 00:00 2024-03-24T10:34:52.912 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-03-24T10:34:52.975 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-03-24T10:34:52.988 INFO:teuthology.orchestra.run.smithi094.stdout:(10/13): python3-google-auth-2.28.2-1.el9.noarc 867 kB/s | 203 kB 00:00 2024-03-24T10:34:53.005 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 6/6 2024-03-24T10:34:53.013 INFO:teuthology.orchestra.run.smithi094.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 472 kB/s | 59 kB 00:00 2024-03-24T10:34:53.072 INFO:teuthology.orchestra.run.smithi094.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.0 MB/s | 90 kB 00:00 2024-03-24T10:34:53.131 INFO:teuthology.orchestra.run.smithi094.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 3.8 MB/s | 1.0 MB 00:00 2024-03-24T10:34:53.132 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-03-24T10:34:53.132 INFO:teuthology.orchestra.run.smithi094.stdout:Total 2.9 MB/s | 2.0 MB 00:00 2024-03-24T10:34:53.133 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-03-24T10:34:53.146 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-03-24T10:34:53.146 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-03-24T10:34:53.235 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-03-24T10:34:53.235 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-03-24T10:34:53.745 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-03-24T10:34:53.772 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 6/6 2024-03-24T10:34:53.772 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 1/6 2024-03-24T10:34:53.772 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-03-24T10:34:53.772 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-03-24T10:34:53.772 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-03-24T10:34:53.772 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-03-24T10:34:53.919 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-03-24T10:34:53.974 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-03-24T10:34:54.030 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-03-24T10:34:54.066 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-03-24T10:34:54.073 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-03-24T10:34:54.073 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:34:54.073 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-03-24T10:34:54.074 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-24T10:34:54.074 INFO:teuthology.orchestra.run.smithi096.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-24T10:34:54.074 INFO:teuthology.orchestra.run.smithi096.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-24T10:34:54.074 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-24T10:34:54.074 INFO:teuthology.orchestra.run.smithi096.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-24T10:34:54.074 INFO:teuthology.orchestra.run.smithi096.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-24T10:34:54.074 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:34:54.074 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-24T10:34:54.108 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-03-24T10:34:54.203 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-03-24T10:34:54.327 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-03-24T10:34:54.332 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-fuse 2024-03-24T10:34:54.376 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-03-24T10:34:54.896 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:01:22 ago on Sun 24 Mar 2024 10:33:32 AM UTC. 2024-03-24T10:34:55.045 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-24T10:34:55.046 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-24T10:34:55.046 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-03-24T10:34:55.046 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-24T10:34:55.046 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-03-24T10:34:55.046 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-fuse x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 812 k 2024-03-24T10:34:55.046 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-03-24T10:34:55.046 INFO:teuthology.orchestra.run.smithi096.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-24T10:34:55.046 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:34:55.046 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-03-24T10:34:55.047 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-24T10:34:55.047 INFO:teuthology.orchestra.run.smithi096.stdout:Install 2 Packages 2024-03-24T10:34:55.047 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:34:55.047 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 891 k 2024-03-24T10:34:55.047 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 2.6 M 2024-03-24T10:34:55.047 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-03-24T10:34:55.096 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-03-24T10:34:55.148 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-03-24T10:34:55.176 INFO:teuthology.orchestra.run.smithi096.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 610 kB/s | 79 kB 00:00 2024-03-24T10:34:55.183 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-03-24T10:34:55.232 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-03-24T10:34:55.252 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-rook-2:17.2.7-726.g9c4e0c34.el9.noarch 13/13 2024-03-24T10:34:55.368 INFO:teuthology.orchestra.run.smithi096.stdout:(2/2): ceph-fuse-17.2.7-726.g9c4e0c34.el9.x86_6 2.5 MB/s | 812 kB 00:00 2024-03-24T10:34:55.369 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-03-24T10:34:55.370 INFO:teuthology.orchestra.run.smithi096.stdout:Total 2.7 MB/s | 891 kB 00:00 2024-03-24T10:34:55.370 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-03-24T10:34:55.390 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-03-24T10:34:55.390 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-03-24T10:34:55.464 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-03-24T10:34:55.464 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-03-24T10:34:55.639 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-03-24T10:34:55.788 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-03-24T10:34:55.850 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/2 2024-03-24T10:34:56.042 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-726.g9c4e0c34.el9.noarch 13/13 2024-03-24T10:34:56.043 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-rook-2:17.2.7-726.g9c4e0c34.el9.noarch 1/13 2024-03-24T10:34:56.043 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-03-24T10:34:56.043 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-03-24T10:34:56.043 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-03-24T10:34:56.043 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-03-24T10:34:56.043 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-03-24T10:34:56.043 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-03-24T10:34:56.043 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-03-24T10:34:56.043 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-03-24T10:34:56.043 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-03-24T10:34:56.043 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-03-24T10:34:56.043 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-03-24T10:34:56.410 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-03-24T10:34:56.411 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:34:56.411 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-03-24T10:34:56.411 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-rook-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-24T10:34:56.411 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-24T10:34:56.411 INFO:teuthology.orchestra.run.smithi094.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-24T10:34:56.411 INFO:teuthology.orchestra.run.smithi094.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-24T10:34:56.411 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-24T10:34:56.411 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-24T10:34:56.411 INFO:teuthology.orchestra.run.smithi094.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-24T10:34:56.411 INFO:teuthology.orchestra.run.smithi094.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-24T10:34:56.411 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-24T10:34:56.411 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-24T10:34:56.412 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-24T10:34:56.412 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-24T10:34:56.412 INFO:teuthology.orchestra.run.smithi094.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-24T10:34:56.412 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:34:56.412 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-03-24T10:34:56.560 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/2 2024-03-24T10:34:56.560 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/2 2024-03-24T10:34:56.659 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install ceph-mgr-cephadm 2024-03-24T10:34:56.864 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-03-24T10:34:56.864 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:34:56.864 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-03-24T10:34:56.864 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-03-24T10:34:56.864 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:34:56.865 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-24T10:34:57.141 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install librados-devel 2024-03-24T10:34:57.236 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:20 ago on Sun 24 Mar 2024 10:33:37 AM UTC. 2024-03-24T10:34:57.394 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-03-24T10:34:57.394 INFO:teuthology.orchestra.run.smithi094.stdout:======================================================================================= 2024-03-24T10:34:57.395 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repository Size 2024-03-24T10:34:57.395 INFO:teuthology.orchestra.run.smithi094.stdout:======================================================================================= 2024-03-24T10:34:57.395 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-03-24T10:34:57.395 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-cephadm noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 123 k 2024-03-24T10:34:57.395 INFO:teuthology.orchestra.run.smithi094.stdout:Installing dependencies: 2024-03-24T10:34:57.395 INFO:teuthology.orchestra.run.smithi094.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-24T10:34:57.395 INFO:teuthology.orchestra.run.smithi094.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-24T10:34:57.395 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-24T10:34:57.395 INFO:teuthology.orchestra.run.smithi094.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-24T10:34:57.395 INFO:teuthology.orchestra.run.smithi094.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-24T10:34:57.395 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:34:57.395 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-03-24T10:34:57.395 INFO:teuthology.orchestra.run.smithi094.stdout:======================================================================================= 2024-03-24T10:34:57.395 INFO:teuthology.orchestra.run.smithi094.stdout:Install 6 Packages 2024-03-24T10:34:57.396 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:34:57.396 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 6.8 M 2024-03-24T10:34:57.396 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 33 M 2024-03-24T10:34:57.396 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-03-24T10:34:57.709 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:01:25 ago on Sun 24 Mar 2024 10:33:32 AM UTC. 2024-03-24T10:34:57.860 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-24T10:34:57.861 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-24T10:34:57.861 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repo Size 2024-03-24T10:34:57.861 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-24T10:34:57.861 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-03-24T10:34:57.861 INFO:teuthology.orchestra.run.smithi096.stdout: librados-devel x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 126 k 2024-03-24T10:34:57.861 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:34:57.861 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-03-24T10:34:57.861 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-24T10:34:57.861 INFO:teuthology.orchestra.run.smithi096.stdout:Install 1 Package 2024-03-24T10:34:57.861 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:34:57.861 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 126 k 2024-03-24T10:34:57.861 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 448 k 2024-03-24T10:34:57.862 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-03-24T10:34:57.886 INFO:teuthology.orchestra.run.smithi094.stdout:(1/6): python3-babel-2.9.1-2.el9.noarch.rpm 22 MB/s | 5.8 MB 00:00 2024-03-24T10:34:57.912 INFO:teuthology.orchestra.run.smithi094.stdout:(2/6): ceph-mgr-cephadm-17.2.7-726.g9c4e0c34.el 432 kB/s | 123 kB 00:00 2024-03-24T10:34:57.970 INFO:teuthology.orchestra.run.smithi094.stdout:(3/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 721 kB/s | 248 kB 00:00 2024-03-24T10:34:58.004 INFO:teuthology.orchestra.run.smithi094.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 627 kB/s | 58 kB 00:00 2024-03-24T10:34:58.045 INFO:teuthology.orchestra.run.smithi094.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 3.4 MB/s | 548 kB 00:00 2024-03-24T10:34:58.067 INFO:teuthology.orchestra.run.smithi096.stdout:librados-devel-17.2.7-726.g9c4e0c34.el9.x86_64. 617 kB/s | 126 kB 00:00 2024-03-24T10:34:58.067 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-03-24T10:34:58.067 INFO:teuthology.orchestra.run.smithi096.stdout:Total 612 kB/s | 126 kB 00:00 2024-03-24T10:34:58.067 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-03-24T10:34:58.070 INFO:teuthology.orchestra.run.smithi094.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 508 kB/s | 51 kB 00:00 2024-03-24T10:34:58.073 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-03-24T10:34:58.074 INFO:teuthology.orchestra.run.smithi094.stdout:Total 10 MB/s | 6.8 MB 00:00 2024-03-24T10:34:58.074 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-03-24T10:34:58.074 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-03-24T10:34:58.086 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-03-24T10:34:58.097 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-03-24T10:34:58.098 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-03-24T10:34:58.128 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-03-24T10:34:58.129 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-03-24T10:34:58.188 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-03-24T10:34:58.189 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-03-24T10:34:58.264 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-03-24T10:34:58.364 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librados-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-24T10:34:58.518 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-03-24T10:34:58.693 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-03-24T10:34:58.747 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-03-24T10:34:58.767 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librados-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-24T10:34:59.069 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librados-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-24T10:34:59.069 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:34:59.069 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-03-24T10:34:59.069 INFO:teuthology.orchestra.run.smithi096.stdout: librados-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:34:59.069 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:34:59.069 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-24T10:34:59.264 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-03-24T10:34:59.335 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install libcephfs2 2024-03-24T10:34:59.350 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-03-24T10:34:59.406 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-03-24T10:34:59.435 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 6/6 2024-03-24T10:34:59.910 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:01:27 ago on Sun 24 Mar 2024 10:33:32 AM UTC. 2024-03-24T10:35:00.009 INFO:teuthology.orchestra.run.smithi096.stdout:Package libcephfs2-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-24T10:35:00.061 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-24T10:35:00.062 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2024-03-24T10:35:00.062 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-24T10:35:00.134 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install libcephfs-devel 2024-03-24T10:35:00.263 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 6/6 2024-03-24T10:35:00.264 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 1/6 2024-03-24T10:35:00.264 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-03-24T10:35:00.264 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-03-24T10:35:00.264 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-03-24T10:35:00.264 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-03-24T10:35:00.567 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-03-24T10:35:00.567 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:35:00.567 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-03-24T10:35:00.567 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-24T10:35:00.567 INFO:teuthology.orchestra.run.smithi094.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-24T10:35:00.567 INFO:teuthology.orchestra.run.smithi094.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-24T10:35:00.567 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-24T10:35:00.568 INFO:teuthology.orchestra.run.smithi094.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-24T10:35:00.568 INFO:teuthology.orchestra.run.smithi094.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-24T10:35:00.568 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:35:00.568 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-03-24T10:35:00.692 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:01:28 ago on Sun 24 Mar 2024 10:33:32 AM UTC. 2024-03-24T10:35:00.830 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install ceph-fuse 2024-03-24T10:35:00.840 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-24T10:35:00.841 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-24T10:35:00.841 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repo Size 2024-03-24T10:35:00.841 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-24T10:35:00.841 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-03-24T10:35:00.841 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs-devel x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 25 k 2024-03-24T10:35:00.841 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:35:00.841 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-03-24T10:35:00.841 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-24T10:35:00.841 INFO:teuthology.orchestra.run.smithi096.stdout:Install 1 Package 2024-03-24T10:35:00.842 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:35:00.842 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 25 k 2024-03-24T10:35:00.842 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 110 k 2024-03-24T10:35:00.842 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-03-24T10:35:01.087 INFO:teuthology.orchestra.run.smithi096.stdout:libcephfs-devel-17.2.7-726.g9c4e0c34.el9.x86_64 102 kB/s | 25 kB 00:00 2024-03-24T10:35:01.088 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-03-24T10:35:01.088 INFO:teuthology.orchestra.run.smithi096.stdout:Total 101 kB/s | 25 kB 00:00 2024-03-24T10:35:01.088 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-03-24T10:35:01.091 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-03-24T10:35:01.092 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-03-24T10:35:01.093 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-03-24T10:35:01.093 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-03-24T10:35:01.176 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-03-24T10:35:01.277 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libcephfs-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-24T10:35:01.394 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:24 ago on Sun 24 Mar 2024 10:33:37 AM UTC. 2024-03-24T10:35:01.544 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-03-24T10:35:01.545 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-03-24T10:35:01.545 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repository Size 2024-03-24T10:35:01.545 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-03-24T10:35:01.545 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-03-24T10:35:01.545 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-fuse x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 812 k 2024-03-24T10:35:01.545 INFO:teuthology.orchestra.run.smithi094.stdout:Installing dependencies: 2024-03-24T10:35:01.545 INFO:teuthology.orchestra.run.smithi094.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-24T10:35:01.545 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:35:01.545 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-03-24T10:35:01.545 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-03-24T10:35:01.546 INFO:teuthology.orchestra.run.smithi094.stdout:Install 2 Packages 2024-03-24T10:35:01.546 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:35:01.546 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 891 k 2024-03-24T10:35:01.546 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 2.6 M 2024-03-24T10:35:01.546 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-03-24T10:35:01.636 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-24T10:35:01.682 INFO:teuthology.orchestra.run.smithi094.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 579 kB/s | 79 kB 00:00 2024-03-24T10:35:01.887 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libcephfs-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-24T10:35:01.887 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:35:01.887 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-03-24T10:35:01.887 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:35:01.887 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:35:01.887 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-24T10:35:02.021 INFO:teuthology.orchestra.run.smithi094.stdout:(2/2): ceph-fuse-17.2.7-726.g9c4e0c34.el9.x86_6 1.7 MB/s | 812 kB 00:00 2024-03-24T10:35:02.021 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-03-24T10:35:02.021 INFO:teuthology.orchestra.run.smithi094.stdout:Total 1.8 MB/s | 891 kB 00:00 2024-03-24T10:35:02.021 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-03-24T10:35:02.037 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-03-24T10:35:02.037 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-03-24T10:35:02.059 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install librados2 2024-03-24T10:35:02.110 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-03-24T10:35:02.110 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-03-24T10:35:02.289 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-03-24T10:35:02.419 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-03-24T10:35:02.482 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/2 2024-03-24T10:35:02.614 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:01:30 ago on Sun 24 Mar 2024 10:33:32 AM UTC. 2024-03-24T10:35:02.710 INFO:teuthology.orchestra.run.smithi096.stdout:Package librados2-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-24T10:35:02.762 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-24T10:35:02.763 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2024-03-24T10:35:02.763 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-24T10:35:02.833 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install librbd1 2024-03-24T10:35:03.196 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/2 2024-03-24T10:35:03.196 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/2 2024-03-24T10:35:03.389 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:01:31 ago on Sun 24 Mar 2024 10:33:32 AM UTC. 2024-03-24T10:35:03.486 INFO:teuthology.orchestra.run.smithi096.stdout:Package librbd1-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-24T10:35:03.538 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-24T10:35:03.538 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2024-03-24T10:35:03.539 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-24T10:35:03.539 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-03-24T10:35:03.539 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:35:03.540 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-03-24T10:35:03.540 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-03-24T10:35:03.540 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:35:03.540 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-03-24T10:35:03.602 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install python3-rados 2024-03-24T10:35:03.795 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install librados-devel 2024-03-24T10:35:04.157 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:01:32 ago on Sun 24 Mar 2024 10:33:32 AM UTC. 2024-03-24T10:35:04.253 INFO:teuthology.orchestra.run.smithi096.stdout:Package python3-rados-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-24T10:35:04.305 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-24T10:35:04.306 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2024-03-24T10:35:04.306 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-24T10:35:04.369 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install python3-rgw 2024-03-24T10:35:04.373 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:27 ago on Sun 24 Mar 2024 10:33:37 AM UTC. 2024-03-24T10:35:04.525 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-03-24T10:35:04.525 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-03-24T10:35:04.525 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repo Size 2024-03-24T10:35:04.525 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-03-24T10:35:04.525 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-03-24T10:35:04.525 INFO:teuthology.orchestra.run.smithi094.stdout: librados-devel x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 126 k 2024-03-24T10:35:04.525 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:35:04.525 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-03-24T10:35:04.525 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-03-24T10:35:04.525 INFO:teuthology.orchestra.run.smithi094.stdout:Install 1 Package 2024-03-24T10:35:04.525 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:35:04.525 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 126 k 2024-03-24T10:35:04.526 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 448 k 2024-03-24T10:35:04.526 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-03-24T10:35:04.837 INFO:teuthology.orchestra.run.smithi094.stdout:librados-devel-17.2.7-726.g9c4e0c34.el9.x86_64. 405 kB/s | 126 kB 00:00 2024-03-24T10:35:04.838 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-03-24T10:35:04.839 INFO:teuthology.orchestra.run.smithi094.stdout:Total 403 kB/s | 126 kB 00:00 2024-03-24T10:35:04.839 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-03-24T10:35:04.844 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-03-24T10:35:04.845 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-03-24T10:35:04.899 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-03-24T10:35:04.899 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-03-24T10:35:04.919 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:01:32 ago on Sun 24 Mar 2024 10:33:32 AM UTC. 2024-03-24T10:35:05.015 INFO:teuthology.orchestra.run.smithi096.stdout:Package python3-rgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-24T10:35:05.045 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-03-24T10:35:05.067 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-24T10:35:05.069 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2024-03-24T10:35:05.069 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-24T10:35:05.137 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install python3-cephfs 2024-03-24T10:35:05.144 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librados-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-24T10:35:05.551 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librados-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-24T10:35:05.699 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:01:33 ago on Sun 24 Mar 2024 10:33:32 AM UTC. 2024-03-24T10:35:05.797 INFO:teuthology.orchestra.run.smithi096.stdout:Package python3-cephfs-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-24T10:35:05.840 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librados-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-24T10:35:05.840 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:35:05.840 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-03-24T10:35:05.840 INFO:teuthology.orchestra.run.smithi094.stdout: librados-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:35:05.840 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:35:05.840 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-03-24T10:35:05.848 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-24T10:35:05.849 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2024-03-24T10:35:05.849 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-24T10:35:05.924 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install python3-rbd 2024-03-24T10:35:06.098 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install libcephfs2 2024-03-24T10:35:06.480 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:01:34 ago on Sun 24 Mar 2024 10:33:32 AM UTC. 2024-03-24T10:35:06.578 INFO:teuthology.orchestra.run.smithi096.stdout:Package python3-rbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-24T10:35:06.631 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-24T10:35:06.632 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2024-03-24T10:35:06.632 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-24T10:35:06.664 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:29 ago on Sun 24 Mar 2024 10:33:37 AM UTC. 2024-03-24T10:35:06.704 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install rbd-fuse 2024-03-24T10:35:06.763 INFO:teuthology.orchestra.run.smithi094.stdout:Package libcephfs2-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-24T10:35:06.818 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-03-24T10:35:06.821 INFO:teuthology.orchestra.run.smithi094.stdout:Nothing to do. 2024-03-24T10:35:06.821 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-03-24T10:35:06.876 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install libcephfs-devel 2024-03-24T10:35:07.264 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:01:35 ago on Sun 24 Mar 2024 10:33:32 AM UTC. 2024-03-24T10:35:07.412 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-24T10:35:07.413 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-24T10:35:07.413 INFO:teuthology.orchestra.run.smithi096.stdout: Package Architecture Version Repository Size 2024-03-24T10:35:07.413 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-24T10:35:07.414 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-03-24T10:35:07.414 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-fuse x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 82 k 2024-03-24T10:35:07.414 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:35:07.414 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-03-24T10:35:07.414 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-24T10:35:07.414 INFO:teuthology.orchestra.run.smithi096.stdout:Install 1 Package 2024-03-24T10:35:07.414 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:35:07.414 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 82 k 2024-03-24T10:35:07.414 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 222 k 2024-03-24T10:35:07.414 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-03-24T10:35:07.435 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:30 ago on Sun 24 Mar 2024 10:33:37 AM UTC. 2024-03-24T10:35:07.586 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-03-24T10:35:07.587 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-03-24T10:35:07.587 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repo Size 2024-03-24T10:35:07.587 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-03-24T10:35:07.587 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-03-24T10:35:07.587 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs-devel x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 25 k 2024-03-24T10:35:07.587 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:35:07.587 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-03-24T10:35:07.587 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-03-24T10:35:07.587 INFO:teuthology.orchestra.run.smithi094.stdout:Install 1 Package 2024-03-24T10:35:07.587 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:35:07.587 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 25 k 2024-03-24T10:35:07.588 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 110 k 2024-03-24T10:35:07.588 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-03-24T10:35:07.693 INFO:teuthology.orchestra.run.smithi096.stdout:rbd-fuse-17.2.7-726.g9c4e0c34.el9.x86_64.rpm 296 kB/s | 82 kB 00:00 2024-03-24T10:35:07.693 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-03-24T10:35:07.693 INFO:teuthology.orchestra.run.smithi096.stdout:Total 294 kB/s | 82 kB 00:00 2024-03-24T10:35:07.693 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-03-24T10:35:07.698 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-03-24T10:35:07.698 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-03-24T10:35:07.752 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-03-24T10:35:07.752 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-03-24T10:35:07.842 INFO:teuthology.orchestra.run.smithi094.stdout:libcephfs-devel-17.2.7-726.g9c4e0c34.el9.x86_64 98 kB/s | 25 kB 00:00 2024-03-24T10:35:07.842 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-03-24T10:35:07.843 INFO:teuthology.orchestra.run.smithi094.stdout:Total 97 kB/s | 25 kB 00:00 2024-03-24T10:35:07.843 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-03-24T10:35:07.846 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-03-24T10:35:07.847 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-03-24T10:35:07.848 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-03-24T10:35:07.848 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-03-24T10:35:07.891 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-03-24T10:35:07.932 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-03-24T10:35:08.006 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : rbd-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-24T10:35:08.032 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libcephfs-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-24T10:35:08.372 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: rbd-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-24T10:35:08.393 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-24T10:35:08.644 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : rbd-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-24T10:35:08.644 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:35:08.644 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-03-24T10:35:08.644 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:35:08.644 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:35:08.644 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-24T10:35:08.683 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libcephfs-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-24T10:35:08.684 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:35:08.684 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-03-24T10:35:08.684 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:35:08.684 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:35:08.684 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-03-24T10:35:08.848 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install librados2 2024-03-24T10:35:08.880 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install rbd-mirror 2024-03-24T10:35:09.402 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:32 ago on Sun 24 Mar 2024 10:33:37 AM UTC. 2024-03-24T10:35:09.441 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:01:37 ago on Sun 24 Mar 2024 10:33:32 AM UTC. 2024-03-24T10:35:09.499 INFO:teuthology.orchestra.run.smithi094.stdout:Package librados2-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-24T10:35:09.551 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-03-24T10:35:09.552 INFO:teuthology.orchestra.run.smithi094.stdout:Nothing to do. 2024-03-24T10:35:09.552 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-03-24T10:35:09.594 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-24T10:35:09.595 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-24T10:35:09.595 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repo Size 2024-03-24T10:35:09.595 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-24T10:35:09.595 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-03-24T10:35:09.595 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-mirror x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 2.9 M 2024-03-24T10:35:09.596 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:35:09.596 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-03-24T10:35:09.596 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-24T10:35:09.596 INFO:teuthology.orchestra.run.smithi096.stdout:Install 1 Package 2024-03-24T10:35:09.596 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:35:09.596 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 2.9 M 2024-03-24T10:35:09.596 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 12 M 2024-03-24T10:35:09.596 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-03-24T10:35:09.608 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install librbd1 2024-03-24T10:35:10.016 INFO:teuthology.orchestra.run.smithi096.stdout:rbd-mirror-17.2.7-726.g9c4e0c34.el9.x86_64.rpm 7.1 MB/s | 2.9 MB 00:00 2024-03-24T10:35:10.016 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-03-24T10:35:10.016 INFO:teuthology.orchestra.run.smithi096.stdout:Total 7.1 MB/s | 2.9 MB 00:00 2024-03-24T10:35:10.016 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-03-24T10:35:10.022 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-03-24T10:35:10.022 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-03-24T10:35:10.093 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-03-24T10:35:10.093 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-03-24T10:35:10.163 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:33 ago on Sun 24 Mar 2024 10:33:37 AM UTC. 2024-03-24T10:35:10.262 INFO:teuthology.orchestra.run.smithi094.stdout:Package librbd1-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-24T10:35:10.315 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-03-24T10:35:10.316 INFO:teuthology.orchestra.run.smithi094.stdout:Nothing to do. 2024-03-24T10:35:10.316 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-03-24T10:35:10.368 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install python3-rados 2024-03-24T10:35:10.501 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-03-24T10:35:10.572 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : rbd-mirror-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-24T10:35:10.594 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: rbd-mirror-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-24T10:35:10.594 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T10:35:10.594 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-24T10:35:10.594 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-24T10:35:10.594 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-24T10:35:10.594 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:35:10.932 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:33 ago on Sun 24 Mar 2024 10:33:37 AM UTC. 2024-03-24T10:35:11.029 INFO:teuthology.orchestra.run.smithi094.stdout:Package python3-rados-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-24T10:35:11.081 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-03-24T10:35:11.082 INFO:teuthology.orchestra.run.smithi094.stdout:Nothing to do. 2024-03-24T10:35:11.082 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-03-24T10:35:11.137 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install python3-rgw 2024-03-24T10:35:11.687 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : rbd-mirror-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-24T10:35:11.687 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:35:11.687 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-03-24T10:35:11.688 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-mirror-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:35:11.688 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:35:11.688 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-24T10:35:11.705 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:34 ago on Sun 24 Mar 2024 10:33:37 AM UTC. 2024-03-24T10:35:11.802 INFO:teuthology.orchestra.run.smithi094.stdout:Package python3-rgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-24T10:35:11.854 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-03-24T10:35:11.854 INFO:teuthology.orchestra.run.smithi094.stdout:Nothing to do. 2024-03-24T10:35:11.855 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-03-24T10:35:11.915 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install rbd-nbd 2024-03-24T10:35:11.918 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install python3-cephfs 2024-03-24T10:35:12.481 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:35 ago on Sun 24 Mar 2024 10:33:37 AM UTC. 2024-03-24T10:35:12.487 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:01:40 ago on Sun 24 Mar 2024 10:33:32 AM UTC. 2024-03-24T10:35:12.580 INFO:teuthology.orchestra.run.smithi094.stdout:Package python3-cephfs-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-24T10:35:12.632 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-03-24T10:35:12.633 INFO:teuthology.orchestra.run.smithi094.stdout:Nothing to do. 2024-03-24T10:35:12.633 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-03-24T10:35:12.639 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-24T10:35:12.639 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-24T10:35:12.639 INFO:teuthology.orchestra.run.smithi096.stdout: Package Architecture Version Repository Size 2024-03-24T10:35:12.639 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-24T10:35:12.639 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-03-24T10:35:12.639 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-nbd x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 165 k 2024-03-24T10:35:12.639 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:35:12.639 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-03-24T10:35:12.640 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-24T10:35:12.640 INFO:teuthology.orchestra.run.smithi096.stdout:Install 1 Package 2024-03-24T10:35:12.640 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:35:12.640 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 165 k 2024-03-24T10:35:12.640 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 477 k 2024-03-24T10:35:12.640 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-03-24T10:35:12.687 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install python3-rbd 2024-03-24T10:35:13.049 INFO:teuthology.orchestra.run.smithi096.stdout:rbd-nbd-17.2.7-726.g9c4e0c34.el9.x86_64.rpm 404 kB/s | 165 kB 00:00 2024-03-24T10:35:13.049 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-03-24T10:35:13.050 INFO:teuthology.orchestra.run.smithi096.stdout:Total 403 kB/s | 165 kB 00:00 2024-03-24T10:35:13.050 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-03-24T10:35:13.057 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-03-24T10:35:13.057 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-03-24T10:35:13.111 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-03-24T10:35:13.111 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-03-24T10:35:13.248 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:36 ago on Sun 24 Mar 2024 10:33:37 AM UTC. 2024-03-24T10:35:13.259 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-03-24T10:35:13.344 INFO:teuthology.orchestra.run.smithi094.stdout:Package python3-rbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-24T10:35:13.373 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : rbd-nbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-24T10:35:13.397 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-03-24T10:35:13.398 INFO:teuthology.orchestra.run.smithi094.stdout:Nothing to do. 2024-03-24T10:35:13.398 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-03-24T10:35:13.460 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install rbd-fuse 2024-03-24T10:35:13.813 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: rbd-nbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-24T10:35:14.018 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:37 ago on Sun 24 Mar 2024 10:33:37 AM UTC. 2024-03-24T10:35:14.070 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : rbd-nbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-24T10:35:14.070 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:35:14.070 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-03-24T10:35:14.070 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-nbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:35:14.070 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:35:14.070 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-24T10:35:14.167 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-03-24T10:35:14.167 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-03-24T10:35:14.167 INFO:teuthology.orchestra.run.smithi094.stdout: Package Architecture Version Repository Size 2024-03-24T10:35:14.168 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-03-24T10:35:14.168 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-03-24T10:35:14.168 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-fuse x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 82 k 2024-03-24T10:35:14.168 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:35:14.168 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-03-24T10:35:14.168 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-03-24T10:35:14.168 INFO:teuthology.orchestra.run.smithi094.stdout:Install 1 Package 2024-03-24T10:35:14.168 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:35:14.168 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 82 k 2024-03-24T10:35:14.168 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 222 k 2024-03-24T10:35:14.168 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-03-24T10:35:14.280 DEBUG:teuthology.parallel:result is None 2024-03-24T10:35:14.504 INFO:teuthology.orchestra.run.smithi094.stdout:rbd-fuse-17.2.7-726.g9c4e0c34.el9.x86_64.rpm 243 kB/s | 82 kB 00:00 2024-03-24T10:35:14.504 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-03-24T10:35:14.505 INFO:teuthology.orchestra.run.smithi094.stdout:Total 242 kB/s | 82 kB 00:00 2024-03-24T10:35:14.505 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-03-24T10:35:14.512 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-03-24T10:35:14.512 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-03-24T10:35:14.566 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-03-24T10:35:14.566 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-03-24T10:35:14.705 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-03-24T10:35:14.818 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : rbd-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-24T10:35:15.181 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: rbd-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-24T10:35:15.471 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : rbd-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-24T10:35:15.471 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:35:15.471 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-03-24T10:35:15.471 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:35:15.471 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:35:15.471 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-03-24T10:35:15.688 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install rbd-mirror 2024-03-24T10:35:16.253 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:39 ago on Sun 24 Mar 2024 10:33:37 AM UTC. 2024-03-24T10:35:16.409 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-03-24T10:35:16.409 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-03-24T10:35:16.410 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repo Size 2024-03-24T10:35:16.410 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-03-24T10:35:16.410 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-03-24T10:35:16.410 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-mirror x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 2.9 M 2024-03-24T10:35:16.410 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:35:16.410 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-03-24T10:35:16.410 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-03-24T10:35:16.410 INFO:teuthology.orchestra.run.smithi094.stdout:Install 1 Package 2024-03-24T10:35:16.410 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:35:16.410 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 2.9 M 2024-03-24T10:35:16.410 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 12 M 2024-03-24T10:35:16.410 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-03-24T10:35:17.558 INFO:teuthology.orchestra.run.smithi094.stdout:rbd-mirror-17.2.7-726.g9c4e0c34.el9.x86_64.rpm 2.6 MB/s | 2.9 MB 00:01 2024-03-24T10:35:17.559 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-03-24T10:35:17.559 INFO:teuthology.orchestra.run.smithi094.stdout:Total 2.6 MB/s | 2.9 MB 00:01 2024-03-24T10:35:17.559 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-03-24T10:35:17.576 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-03-24T10:35:17.576 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-03-24T10:35:17.641 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-03-24T10:35:17.642 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-03-24T10:35:18.047 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-03-24T10:35:18.117 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : rbd-mirror-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-24T10:35:18.141 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: rbd-mirror-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-24T10:35:18.141 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T10:35:18.141 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-24T10:35:18.141 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-24T10:35:18.142 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-24T10:35:18.142 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:35:19.278 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : rbd-mirror-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-24T10:35:19.278 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:35:19.278 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-03-24T10:35:19.278 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-mirror-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:35:19.278 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:35:19.278 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-03-24T10:35:19.516 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install rbd-nbd 2024-03-24T10:35:20.082 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:43 ago on Sun 24 Mar 2024 10:33:37 AM UTC. 2024-03-24T10:35:20.233 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-03-24T10:35:20.234 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-03-24T10:35:20.234 INFO:teuthology.orchestra.run.smithi094.stdout: Package Architecture Version Repository Size 2024-03-24T10:35:20.234 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-03-24T10:35:20.234 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-03-24T10:35:20.234 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-nbd x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 165 k 2024-03-24T10:35:20.234 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:35:20.234 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-03-24T10:35:20.234 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-03-24T10:35:20.234 INFO:teuthology.orchestra.run.smithi094.stdout:Install 1 Package 2024-03-24T10:35:20.234 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:35:20.234 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 165 k 2024-03-24T10:35:20.235 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 477 k 2024-03-24T10:35:20.235 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-03-24T10:35:20.579 INFO:teuthology.orchestra.run.smithi094.stdout:rbd-nbd-17.2.7-726.g9c4e0c34.el9.x86_64.rpm 479 kB/s | 165 kB 00:00 2024-03-24T10:35:20.580 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-03-24T10:35:20.580 INFO:teuthology.orchestra.run.smithi094.stdout:Total 477 kB/s | 165 kB 00:00 2024-03-24T10:35:20.580 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-03-24T10:35:20.588 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-03-24T10:35:20.588 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-03-24T10:35:20.642 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-03-24T10:35:20.643 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-03-24T10:35:20.788 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-03-24T10:35:20.876 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : rbd-nbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-24T10:35:21.298 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: rbd-nbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-24T10:35:21.596 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : rbd-nbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-24T10:35:21.596 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:35:21.596 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-03-24T10:35:21.596 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-nbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-24T10:35:21.596 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:35:21.596 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-03-24T10:35:21.833 DEBUG:teuthology.parallel:result is None 2024-03-24T10:35:21.833 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-24T10:35:21.833 INFO:teuthology.packaging:ref: None 2024-03-24T10:35:21.833 INFO:teuthology.packaging:tag: None 2024-03-24T10:35:21.833 INFO:teuthology.packaging:branch: quincy 2024-03-24T10:35:21.833 INFO:teuthology.packaging:sha1: 21b3bd672163092b4e542f772797fcba02d3d4de 2024-03-24T10:35:21.834 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-03-24T10:35:22.080 DEBUG:teuthology.orchestra.run.smithi094:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-24T10:35:22.102 INFO:teuthology.orchestra.run.smithi094.stdout:17.2.7-726.g9c4e0c34.el9 2024-03-24T10:35:22.103 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-726.g9c4e0c34.el9 2024-03-24T10:35:22.103 INFO:teuthology.task.install:The correct ceph version 17.2.7-726.g9c4e0c34 is installed. 2024-03-24T10:35:22.105 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-24T10:35:22.105 INFO:teuthology.packaging:ref: None 2024-03-24T10:35:22.105 INFO:teuthology.packaging:tag: None 2024-03-24T10:35:22.105 INFO:teuthology.packaging:branch: quincy 2024-03-24T10:35:22.105 INFO:teuthology.packaging:sha1: 21b3bd672163092b4e542f772797fcba02d3d4de 2024-03-24T10:35:22.105 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-03-24T10:35:22.461 DEBUG:teuthology.orchestra.run.smithi096:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-24T10:35:22.487 INFO:teuthology.orchestra.run.smithi096.stdout:17.2.7-726.g9c4e0c34.el9 2024-03-24T10:35:22.487 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-726.g9c4e0c34.el9 2024-03-24T10:35:22.487 INFO:teuthology.task.install:The correct ceph version 17.2.7-726.g9c4e0c34 is installed. 2024-03-24T10:35:22.489 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-24T10:35:22.489 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-03-24T10:35:22.489 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-24T10:35:22.522 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-24T10:35:22.522 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-24T10:35:22.560 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-24T10:35:22.561 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-03-24T10:35:22.561 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/daemon-helper 2024-03-24T10:35:22.592 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-24T10:35:22.662 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-24T10:35:22.662 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/daemon-helper 2024-03-24T10:35:22.692 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-24T10:35:22.767 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-24T10:35:22.767 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-03-24T10:35:22.767 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-24T10:35:22.797 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-24T10:35:22.867 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-24T10:35:22.867 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-24T10:35:22.900 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-24T10:35:22.971 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-24T10:35:22.971 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-03-24T10:35:22.972 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/stdin-killer 2024-03-24T10:35:23.000 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-24T10:35:23.069 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-24T10:35:23.069 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/stdin-killer 2024-03-24T10:35:23.098 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-24T10:35:23.171 INFO:teuthology.run_tasks:Running task print... 2024-03-24T10:35:23.181 INFO:teuthology.task.print:**** done install task... 2024-03-24T10:35:23.181 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-24T10:35:23.360 INFO:tasks.cephadm:Config: {'cephadm_branch': 'quincy', 'cephadm_git_url': 'https://github.com/ceph/ceph', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}}, 'image': 'quay.ceph.io/ceph-ci/ceph:quincy', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '21b3bd672163092b4e542f772797fcba02d3d4de'} 2024-03-24T10:35:23.361 INFO:tasks.cephadm:Cluster image is quay.ceph.io/ceph-ci/ceph:quincy 2024-03-24T10:35:23.361 INFO:tasks.cephadm:Cluster fsid is 386110d4-e9ca-11ee-95cd-87774f69a715 2024-03-24T10:35:23.361 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-24T10:35:23.361 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-03-24T10:35:23.362 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi094': '172.21.15.94', 'mon.smithi096': '172.21.15.96'} 2024-03-24T10:35:23.362 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-24T10:35:23.362 DEBUG:teuthology.orchestra.run.smithi094:> sudo hostname $(hostname -s) 2024-03-24T10:35:23.391 DEBUG:teuthology.orchestra.run.smithi096:> sudo hostname $(hostname -s) 2024-03-24T10:35:23.420 INFO:tasks.cephadm:Downloading cephadm (repo https://github.com/ceph/ceph ref quincy)... 2024-03-24T10:35:23.421 DEBUG:teuthology.orchestra.run.smithi094:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-24T10:35:23.671 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r--. 1 ubuntu ubuntu 375153 Mar 24 10:35 /home/ubuntu/cephtest/cephadm 2024-03-24T10:35:23.672 DEBUG:teuthology.orchestra.run.smithi096:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-24T10:35:23.859 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-r--r--. 1 ubuntu ubuntu 375153 Mar 24 10:35 /home/ubuntu/cephtest/cephadm 2024-03-24T10:35:23.859 DEBUG:teuthology.orchestra.run.smithi094:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-24T10:35:23.876 DEBUG:teuthology.orchestra.run.smithi096:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-24T10:35:23.913 INFO:tasks.cephadm:Pulling image quay.ceph.io/ceph-ci/ceph:quincy on all hosts... 2024-03-24T10:35:23.913 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-03-24T10:35:23.919 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-03-24T10:35:24.765 INFO:teuthology.orchestra.run.smithi094.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-03-24T10:35:24.784 INFO:teuthology.orchestra.run.smithi096.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-03-24T10:35:51.032 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-03-24T10:35:51.033 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_version": "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)", 2024-03-24T10:35:51.033 INFO:teuthology.orchestra.run.smithi094.stdout: "image_id": "837124da804ffab28571a3e286aff1c534a701f228c0d3d7a309fcb128aca1de", 2024-03-24T10:35:51.033 INFO:teuthology.orchestra.run.smithi094.stdout: "repo_digests": [ 2024-03-24T10:35:51.033 INFO:teuthology.orchestra.run.smithi094.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232" 2024-03-24T10:35:51.033 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-03-24T10:35:51.033 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-03-24T10:36:00.746 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-24T10:36:00.746 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_version": "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)", 2024-03-24T10:36:00.746 INFO:teuthology.orchestra.run.smithi096.stdout: "image_id": "837124da804ffab28571a3e286aff1c534a701f228c0d3d7a309fcb128aca1de", 2024-03-24T10:36:00.746 INFO:teuthology.orchestra.run.smithi096.stdout: "repo_digests": [ 2024-03-24T10:36:00.746 INFO:teuthology.orchestra.run.smithi096.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232" 2024-03-24T10:36:00.746 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-03-24T10:36:00.746 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-24T10:36:00.765 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /etc/ceph 2024-03-24T10:36:00.800 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /etc/ceph 2024-03-24T10:36:00.830 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 777 /etc/ceph 2024-03-24T10:36:00.870 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 777 /etc/ceph 2024-03-24T10:36:00.897 INFO:tasks.cephadm:Writing seed config... 2024-03-24T10:36:00.898 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-03-24T10:36:00.898 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-03-24T10:36:00.898 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-03-24T10:36:00.898 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-03-24T10:36:00.898 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-03-24T10:36:00.899 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-24T10:36:00.899 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-24T10:36:00.899 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-03-24T10:36:00.899 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-03-24T10:36:00.899 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-24T10:36:00.899 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-24T10:36:00.899 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-03-24T10:36:00.899 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-24T10:36:00.899 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-24T10:36:00.899 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-24T10:36:00.899 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-24T10:36:00.899 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-24T10:36:00.899 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-24T10:36:00.899 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-24T10:36:00.899 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-24T10:36:00.900 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-24T10:36:00.900 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-24T10:36:00.900 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-24T10:36:00.900 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-03-24T10:36:00.900 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-24T10:36:00.900 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-24T10:36:00.900 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-24T10:36:00.900 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-24T10:36:00.900 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-24T10:36:00.900 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-24T10:36:00.900 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-24T10:36:00.900 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-24T10:36:00.900 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-24T10:36:00.900 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-24T10:36:00.900 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-03-24T10:36:00.901 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-24T10:36:00.901 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-24T10:36:00.901 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-24T10:36:00.901 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-24T10:36:00.901 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-24T10:36:00.901 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-03-24T10:36:00.901 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-24T10:36:00.902 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-03-24T10:36:00.902 DEBUG:teuthology.orchestra.run.smithi094:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-24T10:36:00.927 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 386110d4-e9ca-11ee-95cd-87774f69a715 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-24T10:36:00.927 DEBUG:teuthology.orchestra.run.smithi094:mon.smithi094> sudo journalctl -f -n 0 -u ceph-386110d4-e9ca-11ee-95cd-87774f69a715@mon.smithi094.service 2024-03-24T10:36:00.969 INFO:tasks.cephadm:Bootstrapping... 2024-03-24T10:36:00.969 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy -v bootstrap --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.94 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-24T10:36:01.176 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-03-24T10:36:01.176 INFO:teuthology.orchestra.run.smithi094.stdout:cephadm ['--image', 'quay.ceph.io/ceph-ci/ceph:quincy', '-v', 'bootstrap', '--fsid', '386110d4-e9ca-11ee-95cd-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.94', '--skip-admin-label'] 2024-03-24T10:36:01.201 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-24T10:36:01.201 INFO:teuthology.orchestra.run.smithi094.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-24T10:36:01.202 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying podman|docker is present... 2024-03-24T10:36:01.225 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-24T10:36:01.226 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying lvm2 is present... 2024-03-24T10:36:01.226 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying time synchronization is in place... 2024-03-24T10:36:01.233 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-24T10:36:01.233 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-24T10:36:01.239 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-24T10:36:01.239 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout inactive 2024-03-24T10:36:01.246 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout enabled 2024-03-24T10:36:01.252 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout active 2024-03-24T10:36:01.252 INFO:teuthology.orchestra.run.smithi094.stdout:Unit chronyd.service is enabled and running 2024-03-24T10:36:01.252 INFO:teuthology.orchestra.run.smithi094.stdout:Repeating the final host check... 2024-03-24T10:36:01.276 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-24T10:36:01.276 INFO:teuthology.orchestra.run.smithi094.stdout:podman (/bin/podman) version 4.9.3 is present 2024-03-24T10:36:01.276 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl is present 2024-03-24T10:36:01.276 INFO:teuthology.orchestra.run.smithi094.stdout:lvcreate is present 2024-03-24T10:36:01.282 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-24T10:36:01.283 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-24T10:36:01.288 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-24T10:36:01.288 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout inactive 2024-03-24T10:36:01.295 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout enabled 2024-03-24T10:36:01.301 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout active 2024-03-24T10:36:01.302 INFO:teuthology.orchestra.run.smithi094.stdout:Unit chronyd.service is enabled and running 2024-03-24T10:36:01.302 INFO:teuthology.orchestra.run.smithi094.stdout:Host looks OK 2024-03-24T10:36:01.302 INFO:teuthology.orchestra.run.smithi094.stdout:Cluster fsid: 386110d4-e9ca-11ee-95cd-87774f69a715 2024-03-24T10:36:01.302 INFO:teuthology.orchestra.run.smithi094.stdout:Acquiring lock 140601279489120 on /run/cephadm/386110d4-e9ca-11ee-95cd-87774f69a715.lock 2024-03-24T10:36:01.302 INFO:teuthology.orchestra.run.smithi094.stdout:Lock 140601279489120 acquired on /run/cephadm/386110d4-e9ca-11ee-95cd-87774f69a715.lock 2024-03-24T10:36:01.302 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying IP 172.21.15.94 port 3300 ... 2024-03-24T10:36:01.302 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying IP 172.21.15.94 port 6789 ... 2024-03-24T10:36:01.303 INFO:teuthology.orchestra.run.smithi094.stdout:Base mon IP(s) is [172.21.15.94:3300, 172.21.15.94:6789], mon addrv is [v2:172.21.15.94:3300,v1:172.21.15.94:6789] 2024-03-24T10:36:01.306 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.94 metric 100 2024-03-24T10:36:01.306 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.94 metric 100 2024-03-24T10:36:01.308 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-24T10:36:01.309 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-03-24T10:36:01.309 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1781sec hoplimit 64 pref medium 2024-03-24T10:36:01.311 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-24T10:36:01.311 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-24T10:36:01.311 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-24T10:36:01.311 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-03-24T10:36:01.311 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d763/64 scope link 2024-03-24T10:36:01.312 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-24T10:36:01.312 INFO:teuthology.orchestra.run.smithi094.stdout:Mon IP `172.21.15.94` is in CIDR network `172.21.0.0/20` 2024-03-24T10:36:01.312 INFO:teuthology.orchestra.run.smithi094.stdout:Mon IP `172.21.15.94` is in CIDR network `172.21.0.0/20` 2024-03-24T10:36:01.312 INFO:teuthology.orchestra.run.smithi094.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-24T10:36:01.313 INFO:teuthology.orchestra.run.smithi094.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-24T10:36:01.313 INFO:teuthology.orchestra.run.smithi094.stdout:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-03-24T10:36:01.789 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stdout 837124da804ffab28571a3e286aff1c534a701f228c0d3d7a309fcb128aca1de 2024-03-24T10:36:01.789 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stderr Trying to pull quay.ceph.io/ceph-ci/ceph:quincy... 2024-03-24T10:36:01.789 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stderr Getting image source signatures 2024-03-24T10:36:01.790 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stderr Copying blob sha256:601ec453fe24458667695dda21742a4f3ea719c31ad8aa8ec48bf585cc42a7d7 2024-03-24T10:36:01.790 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stderr Copying blob sha256:9516588f908cdb14b5e4062a6063db2d9291b355c3f32e2a99864608a4a7b649 2024-03-24T10:36:01.790 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stderr Copying config sha256:837124da804ffab28571a3e286aff1c534a701f228c0d3d7a309fcb128aca1de 2024-03-24T10:36:01.790 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-24T10:36:02.720 INFO:teuthology.orchestra.run.smithi094.stdout:ceph: stdout ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable) 2024-03-24T10:36:02.720 INFO:teuthology.orchestra.run.smithi094.stdout:Ceph version: ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable) 2024-03-24T10:36:02.720 INFO:teuthology.orchestra.run.smithi094.stdout:Extracting ceph user uid/gid from container image... 2024-03-24T10:36:03.711 INFO:teuthology.orchestra.run.smithi094.stdout:stat: stdout 167 167 2024-03-24T10:36:03.711 INFO:teuthology.orchestra.run.smithi094.stdout:Creating initial keys... 2024-03-24T10:36:04.508 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph-authtool: stdout AQATAgBm0H24OhAA8HEqzm9qfJWe2j9GuZatPw== 2024-03-24T10:36:05.364 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph-authtool: stdout AQAUAgBm9smOMBAAWpb3SndLZUV/SWSJCI/Exw== 2024-03-24T10:36:06.344 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph-authtool: stdout AQAVAgBmrPrxKBAAu0Gb0bFCJB/1bstfa3LTAQ== 2024-03-24T10:36:06.345 INFO:teuthology.orchestra.run.smithi094.stdout:Creating initial monmap... 2024-03-24T10:36:07.267 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-24T10:36:07.267 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2024-03-24T10:36:07.268 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 386110d4-e9ca-11ee-95cd-87774f69a715 2024-03-24T10:36:07.268 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-24T10:36:07.268 INFO:teuthology.orchestra.run.smithi094.stdout:monmaptool for smithi094 [v2:172.21.15.94:3300,v1:172.21.15.94:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-24T10:36:07.268 INFO:teuthology.orchestra.run.smithi094.stdout:setting min_mon_release = octopus 2024-03-24T10:36:07.268 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/monmaptool: set fsid to 386110d4-e9ca-11ee-95cd-87774f69a715 2024-03-24T10:36:07.268 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-24T10:36:07.268 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:36:07.268 INFO:teuthology.orchestra.run.smithi094.stdout:Creating mon... 2024-03-24T10:36:08.198 INFO:teuthology.orchestra.run.smithi094.stdout:create mon.smithi094 on 2024-03-24T10:36:08.408 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-24T10:36:08.553 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-24T10:36:08.701 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-386110d4-e9ca-11ee-95cd-87774f69a715.target → /etc/systemd/system/ceph-386110d4-e9ca-11ee-95cd-87774f69a715.target. 2024-03-24T10:36:08.702 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-386110d4-e9ca-11ee-95cd-87774f69a715.target → /etc/systemd/system/ceph-386110d4-e9ca-11ee-95cd-87774f69a715.target. 2024-03-24T10:36:08.869 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-386110d4-e9ca-11ee-95cd-87774f69a715@mon.smithi094 2024-03-24T10:36:08.869 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Failed to reset failed state of unit ceph-386110d4-e9ca-11ee-95cd-87774f69a715@mon.smithi094.service: Unit ceph-386110d4-e9ca-11ee-95cd-87774f69a715@mon.smithi094.service not loaded. 2024-03-24T10:36:09.037 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-386110d4-e9ca-11ee-95cd-87774f69a715.target.wants/ceph-386110d4-e9ca-11ee-95cd-87774f69a715@mon.smithi094.service → /etc/systemd/system/ceph-386110d4-e9ca-11ee-95cd-87774f69a715@.service. 2024-03-24T10:36:09.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:09 smithi094 podman[29055]: 2024-03-24 10:36:09.179761119 +0000 UTC m=+0.017815323 image pull 837124da804ffab28571a3e286aff1c534a701f228c0d3d7a309fcb128aca1de quay.ceph.io/ceph-ci/ceph:quincy 2024-03-24T10:36:10.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:10 smithi094 podman[29055]: 2024-03-24 10:36:10.118000458 +0000 UTC m=+0.956054706 container create 2f276eaee9d0c7d106f9fc6d2460304d296d27af493b4a935146244faec37400 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-mon-smithi094, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, RELEASE=quincy-9c4e0c3, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-24T10:36:12.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:11 smithi094 podman[29055]: 2024-03-24 10:36:11.955368804 +0000 UTC m=+2.793423023 container init 2f276eaee9d0c7d106f9fc6d2460304d296d27af493b4a935146244faec37400 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-mon-smithi094, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, ceph=True, RELEASE=quincy-9c4e0c3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T10:36:12.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:11 smithi094 podman[29055]: 2024-03-24 10:36:11.959226481 +0000 UTC m=+2.797280685 container start 2f276eaee9d0c7d106f9fc6d2460304d296d27af493b4a935146244faec37400 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-mon-smithi094, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=quincy-9c4e0c3, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319) 2024-03-24T10:36:12.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:12 smithi094 ceph-mon[29074]: mkfs 386110d4-e9ca-11ee-95cd-87774f69a715 2024-03-24T10:36:12.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:12 smithi094 ceph-mon[29074]: mon.smithi094 is new leader, mons smithi094 in quorum (ranks 0) 2024-03-24T10:36:12.949 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-24T10:36:12.949 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout disabled 2024-03-24T10:36:12.957 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-24T10:36:12.957 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout inactive 2024-03-24T10:36:12.957 INFO:teuthology.orchestra.run.smithi094.stdout:firewalld.service is not enabled 2024-03-24T10:36:12.957 INFO:teuthology.orchestra.run.smithi094.stdout:Not possible to enable service . firewalld.service is not available 2024-03-24T10:36:12.958 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for mon to start... 2024-03-24T10:36:12.958 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for mon... 2024-03-24T10:36:13.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:12 smithi094 bash[29055]: 2f276eaee9d0c7d106f9fc6d2460304d296d27af493b4a935146244faec37400 2024-03-24T10:36:13.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:12 smithi094 systemd[1]: Started Ceph mon.smithi094 for 386110d4-e9ca-11ee-95cd-87774f69a715. 2024-03-24T10:36:13.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:13 smithi094 ceph-mon[29074]: mon.smithi094 is new leader, mons smithi094 in quorum (ranks 0) 2024-03-24T10:36:13.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:13 smithi094 ceph-mon[29074]: monmap e1: 1 mons at {smithi094=[v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-24T10:36:13.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:13 smithi094 ceph-mon[29074]: fsmap 2024-03-24T10:36:13.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:13 smithi094 ceph-mon[29074]: osdmap e1: 0 total, 0 up, 0 in 2024-03-24T10:36:13.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:13 smithi094 ceph-mon[29074]: mgrmap e1: no daemons active 2024-03-24T10:36:16.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:16 smithi094 ceph-mon[29074]: from='client.? 172.21.15.94:0/530168215' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-24T10:36:22.686 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout cluster: 2024-03-24T10:36:22.686 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout id: 386110d4-e9ca-11ee-95cd-87774f69a715 2024-03-24T10:36:22.686 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-24T10:36:22.686 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-03-24T10:36:22.686 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout services: 2024-03-24T10:36:22.687 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi094 (age 4s) 2024-03-24T10:36:22.687 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-24T10:36:22.687 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-24T10:36:22.687 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-03-24T10:36:22.687 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout data: 2024-03-24T10:36:22.687 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-24T10:36:22.687 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-24T10:36:22.687 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-24T10:36:22.687 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout pgs: 2024-03-24T10:36:22.687 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-03-24T10:36:22.687 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:16.079+0000 7f8c53e22700 1 Processor -- start 2024-03-24T10:36:22.687 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:16.080+0000 7f8c53e22700 1 -- start start 2024-03-24T10:36:22.687 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:16.080+0000 7f8c53e22700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c4c076b40 0x7f8c4c105560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:36:22.687 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:16.080+0000 7f8c53e22700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4c076f50 con 0x7f8c4c076b40 2024-03-24T10:36:22.687 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:16.080+0000 7f8c51bbe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c4c076b40 0x7f8c4c105560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:36:22.688 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:16.080+0000 7f8c51bbe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c4c076b40 0x7f8c4c105560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43752/0 (socket says 172.21.15.94:43752) 2024-03-24T10:36:22.688 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:16.080+0000 7f8c51bbe700 1 -- 172.21.15.94:0/1547426665 learned_addr learned my addr 172.21.15.94:0/1547426665 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:36:22.688 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:16.081+0000 7f8c51bbe700 1 -- 172.21.15.94:0/1547426665 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c4c105aa0 con 0x7f8c4c076b40 2024-03-24T10:36:22.688 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:16.081+0000 7f8c51bbe700 1 --2- 172.21.15.94:0/1547426665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c4c076b40 0x7f8c4c105560 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8c44009e50 tx=0x7f8c44009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d71bd9c0ee7e21ed server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:36:22.688 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:16.081+0000 7f8c50bbc700 1 -- 172.21.15.94:0/1547426665 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8c440092e0 con 0x7f8c4c076b40 2024-03-24T10:36:22.688 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:16.081+0000 7f8c50bbc700 1 -- 172.21.15.94:0/1547426665 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c4400ba20 con 0x7f8c4c076b40 2024-03-24T10:36:22.688 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:16.081+0000 7f8c50bbc700 1 -- 172.21.15.94:0/1547426665 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8c4400bd30 con 0x7f8c4c076b40 2024-03-24T10:36:22.689 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:16.081+0000 7f8c53e22700 1 -- 172.21.15.94:0/1547426665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c4c076b40 msgr2=0x7f8c4c105560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:36:22.689 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:16.081+0000 7f8c53e22700 1 --2- 172.21.15.94:0/1547426665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c4c076b40 0x7f8c4c105560 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8c44009e50 tx=0x7f8c44009840 comp rx=0 tx=0).stop 2024-03-24T10:36:22.689 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:16.081+0000 7f8c53e22700 1 -- 172.21.15.94:0/1547426665 shutdown_connections 2024-03-24T10:36:22.689 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:16.081+0000 7f8c53e22700 1 --2- 172.21.15.94:0/1547426665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c4c076b40 0x7f8c4c105560 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:22.689 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:16.081+0000 7f8c53e22700 1 -- 172.21.15.94:0/1547426665 >> 172.21.15.94:0/1547426665 conn(0x7f8c4c0fcf90 msgr2=0x7f8c4c0ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:36:22.689 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:16.082+0000 7f8c53e22700 1 -- 172.21.15.94:0/1547426665 shutdown_connections 2024-03-24T10:36:22.689 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:16.082+0000 7f8c53e22700 1 -- 172.21.15.94:0/1547426665 wait complete. 2024-03-24T10:36:22.689 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:16.082+0000 7f8c53e22700 1 Processor -- start 2024-03-24T10:36:22.689 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:16.082+0000 7f8c53e22700 1 -- start start 2024-03-24T10:36:22.689 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:16.082+0000 7f8c53e22700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c4c076b40 0x7f8c4c102eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:36:22.689 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:16.082+0000 7f8c53e22700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4c1033f0 con 0x7f8c4c076b40 2024-03-24T10:36:22.689 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:16.082+0000 7f8c51bbe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c4c076b40 0x7f8c4c102eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:36:22.689 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:16.082+0000 7f8c51bbe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c4c076b40 0x7f8c4c102eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43756/0 (socket says 172.21.15.94:43756) 2024-03-24T10:36:22.689 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:16.082+0000 7f8c51bbe700 1 -- 172.21.15.94:0/530168215 learned_addr learned my addr 172.21.15.94:0/530168215 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:36:22.689 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:16.083+0000 7f8c51bbe700 1 -- 172.21.15.94:0/530168215 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c44009ae0 con 0x7f8c4c076b40 2024-03-24T10:36:22.690 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:16.083+0000 7f8c51bbe700 1 --2- 172.21.15.94:0/530168215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c4c076b40 0x7f8c4c102eb0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8c44003000 tx=0x7f8c44017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:36:22.690 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:16.083+0000 7f8c4a7fc700 1 -- 172.21.15.94:0/530168215 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8c44012b90 con 0x7f8c4c076b40 2024-03-24T10:36:22.690 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:16.083+0000 7f8c4a7fc700 1 -- 172.21.15.94:0/530168215 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c44012d10 con 0x7f8c4c076b40 2024-03-24T10:36:22.690 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:16.083+0000 7f8c53e22700 1 -- 172.21.15.94:0/530168215 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c4c1014c0 con 0x7f8c4c076b40 2024-03-24T10:36:22.690 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:16.083+0000 7f8c4a7fc700 1 -- 172.21.15.94:0/530168215 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8c4401e400 con 0x7f8c4c076b40 2024-03-24T10:36:22.690 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:16.083+0000 7f8c53e22700 1 -- 172.21.15.94:0/530168215 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c4c1019b0 con 0x7f8c4c076b40 2024-03-24T10:36:22.690 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:16.083+0000 7f8c4a7fc700 1 -- 172.21.15.94:0/530168215 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f8c44019030 con 0x7f8c4c076b40 2024-03-24T10:36:22.690 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:16.083+0000 7f8c4a7fc700 1 -- 172.21.15.94:0/530168215 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f8c4401ed00 con 0x7f8c4c076b40 2024-03-24T10:36:22.690 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:16.084+0000 7f8c53e22700 1 -- 172.21.15.94:0/530168215 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c4c04bed0 con 0x7f8c4c076b40 2024-03-24T10:36:22.690 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:16.086+0000 7f8c4a7fc700 1 -- 172.21.15.94:0/530168215 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f8c44017070 con 0x7f8c4c076b40 2024-03-24T10:36:22.690 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:16.157+0000 7f8c53e22700 1 -- 172.21.15.94:0/530168215 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f8c4c105e10 con 0x7f8c4c076b40 2024-03-24T10:36:22.690 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:16.157+0000 7f8c4a7fc700 1 -- 172.21.15.94:0/530168215 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+318 (secure 0 0 0) 0x7f8c4401e680 con 0x7f8c4c076b40 2024-03-24T10:36:22.690 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:16.158+0000 7f8c53e22700 1 -- 172.21.15.94:0/530168215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c4c076b40 msgr2=0x7f8c4c102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:36:22.691 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:16.158+0000 7f8c53e22700 1 --2- 172.21.15.94:0/530168215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c4c076b40 0x7f8c4c102eb0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8c44003000 tx=0x7f8c44017000 comp rx=0 tx=0).stop 2024-03-24T10:36:22.691 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:16.158+0000 7f8c53e22700 1 -- 172.21.15.94:0/530168215 shutdown_connections 2024-03-24T10:36:22.691 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:16.158+0000 7f8c53e22700 1 --2- 172.21.15.94:0/530168215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c4c076b40 0x7f8c4c102eb0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:22.691 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:16.158+0000 7f8c53e22700 1 -- 172.21.15.94:0/530168215 >> 172.21.15.94:0/530168215 conn(0x7f8c4c0fcf90 msgr2=0x7f8c4c0fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:36:22.691 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:16.158+0000 7f8c53e22700 1 -- 172.21.15.94:0/530168215 shutdown_connections 2024-03-24T10:36:22.691 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:16.158+0000 7f8c53e22700 1 -- 172.21.15.94:0/530168215 wait complete. 2024-03-24T10:36:22.691 INFO:teuthology.orchestra.run.smithi094.stdout:mon is available 2024-03-24T10:36:22.691 INFO:teuthology.orchestra.run.smithi094.stdout:Assimilating anything we can from ceph.conf... 2024-03-24T10:36:23.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:23 smithi094 ceph-mon[29074]: from='client.? 172.21.15.94:0/2654679239' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-24T10:36:23.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:23 smithi094 ceph-mon[29074]: from='client.? 172.21.15.94:0/2654679239' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-24T10:36:23.859 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-03-24T10:36:23.859 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout [global] 2024-03-24T10:36:23.859 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout fsid = 386110d4-e9ca-11ee-95cd-87774f69a715 2024-03-24T10:36:23.859 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.94:3300,v1:172.21.15.94:6789] 2024-03-24T10:36:23.859 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-24T10:36:23.859 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-24T10:36:23.859 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-24T10:36:23.860 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-24T10:36:23.860 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-03-24T10:36:23.860 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-24T10:36:23.860 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-24T10:36:23.860 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-03-24T10:36:23.860 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout [osd] 2024-03-24T10:36:23.860 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-24T10:36:23.860 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-24T10:36:23.860 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-24T10:36:23.860 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-24T10:36:23.860 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-24T10:36:23.860 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:23.201+0000 7f79584a0700 1 Processor -- start 2024-03-24T10:36:23.860 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:23.201+0000 7f79584a0700 1 -- start start 2024-03-24T10:36:23.860 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:23.201+0000 7f79584a0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7950078c90 0x7f7950077290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:36:23.860 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:23.201+0000 7f79584a0700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79500790a0 con 0x7f7950078c90 2024-03-24T10:36:23.861 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:23.201+0000 7f795623c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7950078c90 0x7f7950077290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:36:23.861 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:23.201+0000 7f795623c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7950078c90 0x7f7950077290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38212/0 (socket says 172.21.15.94:38212) 2024-03-24T10:36:23.861 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:23.201+0000 7f795623c700 1 -- 172.21.15.94:0/2294005117 learned_addr learned my addr 172.21.15.94:0/2294005117 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:36:23.861 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:23.201+0000 7f795623c700 1 -- 172.21.15.94:0/2294005117 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79500777d0 con 0x7f7950078c90 2024-03-24T10:36:23.861 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:23.202+0000 7f795623c700 1 --2- 172.21.15.94:0/2294005117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7950078c90 0x7f7950077290 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f794c0098e0 tx=0x7f794c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af1af06966d23bcd server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:36:23.861 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:23.202+0000 7f795523a700 1 -- 172.21.15.94:0/2294005117 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f794c00b860 con 0x7f7950078c90 2024-03-24T10:36:23.861 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:23.202+0000 7f795523a700 1 -- 172.21.15.94:0/2294005117 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f794c00b9c0 con 0x7f7950078c90 2024-03-24T10:36:23.861 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:23.202+0000 7f795523a700 1 -- 172.21.15.94:0/2294005117 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f794c00bc90 con 0x7f7950078c90 2024-03-24T10:36:23.861 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:23.202+0000 7f79584a0700 1 -- 172.21.15.94:0/2294005117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7950078c90 msgr2=0x7f7950077290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:36:23.861 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:23.202+0000 7f79584a0700 1 --2- 172.21.15.94:0/2294005117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7950078c90 0x7f7950077290 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f794c0098e0 tx=0x7f794c009250 comp rx=0 tx=0).stop 2024-03-24T10:36:23.861 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:23.202+0000 7f79584a0700 1 -- 172.21.15.94:0/2294005117 shutdown_connections 2024-03-24T10:36:23.861 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:23.202+0000 7f79584a0700 1 --2- 172.21.15.94:0/2294005117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7950078c90 0x7f7950077290 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:23.861 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:23.202+0000 7f79584a0700 1 -- 172.21.15.94:0/2294005117 >> 172.21.15.94:0/2294005117 conn(0x7f79500fe070 msgr2=0x7f79501004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:36:23.861 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:23.202+0000 7f79584a0700 1 -- 172.21.15.94:0/2294005117 shutdown_connections 2024-03-24T10:36:23.861 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:23.203+0000 7f79584a0700 1 -- 172.21.15.94:0/2294005117 wait complete. 2024-03-24T10:36:23.862 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:23.203+0000 7f79584a0700 1 Processor -- start 2024-03-24T10:36:23.862 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:23.203+0000 7f79584a0700 1 -- start start 2024-03-24T10:36:23.862 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:23.203+0000 7f79584a0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7950078c90 0x7f79501130f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:36:23.862 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:23.203+0000 7f79584a0700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7950113630 con 0x7f7950078c90 2024-03-24T10:36:23.862 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:23.203+0000 7f795623c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7950078c90 0x7f79501130f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:36:23.862 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:23.203+0000 7f795623c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7950078c90 0x7f79501130f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38222/0 (socket says 172.21.15.94:38222) 2024-03-24T10:36:23.862 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:23.203+0000 7f795623c700 1 -- 172.21.15.94:0/2654679239 learned_addr learned my addr 172.21.15.94:0/2654679239 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:36:23.862 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:23.203+0000 7f795623c700 1 -- 172.21.15.94:0/2654679239 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f794c0094f0 con 0x7f7950078c90 2024-03-24T10:36:23.862 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:23.204+0000 7f795623c700 1 --2- 172.21.15.94:0/2654679239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7950078c90 0x7f79501130f0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f794c009290 tx=0x7f794c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:36:23.862 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:23.204+0000 7f7946ffd700 1 -- 172.21.15.94:0/2654679239 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f794c01c5c0 con 0x7f7950078c90 2024-03-24T10:36:23.862 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:23.204+0000 7f7946ffd700 1 -- 172.21.15.94:0/2654679239 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f794c01c740 con 0x7f7950078c90 2024-03-24T10:36:23.862 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:23.204+0000 7f7946ffd700 1 -- 172.21.15.94:0/2654679239 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f794c01ca50 con 0x7f7950078c90 2024-03-24T10:36:23.862 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:23.204+0000 7f79584a0700 1 -- 172.21.15.94:0/2654679239 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7950113830 con 0x7f7950078c90 2024-03-24T10:36:23.862 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:23.204+0000 7f79584a0700 1 -- 172.21.15.94:0/2654679239 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79501143a0 con 0x7f7950078c90 2024-03-24T10:36:23.863 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:23.204+0000 7f7946ffd700 1 -- 172.21.15.94:0/2654679239 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f794c01a450 con 0x7f7950078c90 2024-03-24T10:36:23.863 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:23.204+0000 7f7946ffd700 1 -- 172.21.15.94:0/2654679239 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f794c02e790 con 0x7f7950078c90 2024-03-24T10:36:23.863 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:23.205+0000 7f79584a0700 1 -- 172.21.15.94:0/2654679239 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f795005f5f0 con 0x7f7950078c90 2024-03-24T10:36:23.863 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:23.207+0000 7f7946ffd700 1 -- 172.21.15.94:0/2654679239 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f794c01a890 con 0x7f7950078c90 2024-03-24T10:36:23.863 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:23.275+0000 7f79584a0700 1 -- 172.21.15.94:0/2654679239 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f795010cd80 con 0x7f7950078c90 2024-03-24T10:36:23.863 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:23.279+0000 7f7946ffd700 1 -- 172.21.15.94:0/2654679239 <== mon.0 v2:172.21.15.94:3300/0 7 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f794c01aa30 con 0x7f7950078c90 2024-03-24T10:36:23.863 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:23.279+0000 7f7946ffd700 1 -- 172.21.15.94:0/2654679239 <== mon.0 v2:172.21.15.94:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f794c02ea70 con 0x7f7950078c90 2024-03-24T10:36:23.863 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:23.280+0000 7f79584a0700 1 -- 172.21.15.94:0/2654679239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7950078c90 msgr2=0x7f79501130f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:36:23.863 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:23.280+0000 7f79584a0700 1 --2- 172.21.15.94:0/2654679239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7950078c90 0x7f79501130f0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f794c009290 tx=0x7f794c016000 comp rx=0 tx=0).stop 2024-03-24T10:36:23.863 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:23.280+0000 7f79584a0700 1 -- 172.21.15.94:0/2654679239 shutdown_connections 2024-03-24T10:36:23.863 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:23.280+0000 7f79584a0700 1 --2- 172.21.15.94:0/2654679239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7950078c90 0x7f79501130f0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:23.863 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:23.280+0000 7f79584a0700 1 -- 172.21.15.94:0/2654679239 >> 172.21.15.94:0/2654679239 conn(0x7f79500fe070 msgr2=0x7f7950109f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:36:23.863 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:23.280+0000 7f79584a0700 1 -- 172.21.15.94:0/2654679239 shutdown_connections 2024-03-24T10:36:23.863 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:23.280+0000 7f79584a0700 1 -- 172.21.15.94:0/2654679239 wait complete. 2024-03-24T10:36:23.863 INFO:teuthology.orchestra.run.smithi094.stdout:Generating new minimal ceph.conf... 2024-03-24T10:36:24.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:24 smithi094 ceph-mon[29074]: from='client.? 172.21.15.94:0/2345033814' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:36:24.862 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:24.237+0000 7f694d6e3700 1 Processor -- start 2024-03-24T10:36:24.862 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:24.237+0000 7f694d6e3700 1 -- start start 2024-03-24T10:36:24.862 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:24.237+0000 7f694d6e3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6948102910 0x7f6948102d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:36:24.862 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:24.237+0000 7f694d6e3700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6948103260 con 0x7f6948102910 2024-03-24T10:36:24.862 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:24.237+0000 7f6946ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6948102910 0x7f6948102d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:36:24.862 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:24.237+0000 7f6946ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6948102910 0x7f6948102d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38234/0 (socket says 172.21.15.94:38234) 2024-03-24T10:36:24.862 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:24.237+0000 7f6946ffd700 1 -- 172.21.15.94:0/3979959381 learned_addr learned my addr 172.21.15.94:0/3979959381 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:36:24.862 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:24.238+0000 7f6946ffd700 1 -- 172.21.15.94:0/3979959381 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69481033a0 con 0x7f6948102910 2024-03-24T10:36:24.862 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:24.238+0000 7f6946ffd700 1 --2- 172.21.15.94:0/3979959381 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6948102910 0x7f6948102d20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f69380098e0 tx=0x7f6938009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12e7d1fa1775f54f server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:36:24.862 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:24.238+0000 7f6945ffb700 1 -- 172.21.15.94:0/3979959381 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f693800b860 con 0x7f6948102910 2024-03-24T10:36:24.862 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:24.238+0000 7f6945ffb700 1 -- 172.21.15.94:0/3979959381 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f693800be50 con 0x7f6948102910 2024-03-24T10:36:24.863 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:24.238+0000 7f6945ffb700 1 -- 172.21.15.94:0/3979959381 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6938010380 con 0x7f6948102910 2024-03-24T10:36:24.863 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:24.238+0000 7f694d6e3700 1 -- 172.21.15.94:0/3979959381 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6948102910 msgr2=0x7f6948102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:36:24.863 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:24.238+0000 7f694d6e3700 1 --2- 172.21.15.94:0/3979959381 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6948102910 0x7f6948102d20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f69380098e0 tx=0x7f6938009250 comp rx=0 tx=0).stop 2024-03-24T10:36:24.863 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:24.238+0000 7f694d6e3700 1 -- 172.21.15.94:0/3979959381 shutdown_connections 2024-03-24T10:36:24.863 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:24.238+0000 7f694d6e3700 1 --2- 172.21.15.94:0/3979959381 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6948102910 0x7f6948102d20 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:24.863 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:24.238+0000 7f694d6e3700 1 -- 172.21.15.94:0/3979959381 >> 172.21.15.94:0/3979959381 conn(0x7f69480fe070 msgr2=0x7f69481004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:36:24.863 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:24.239+0000 7f694d6e3700 1 -- 172.21.15.94:0/3979959381 shutdown_connections 2024-03-24T10:36:24.863 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:24.239+0000 7f694d6e3700 1 -- 172.21.15.94:0/3979959381 wait complete. 2024-03-24T10:36:24.863 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:24.239+0000 7f694d6e3700 1 Processor -- start 2024-03-24T10:36:24.863 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:24.239+0000 7f694d6e3700 1 -- start start 2024-03-24T10:36:24.863 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:24.239+0000 7f694d6e3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6948102910 0x7f6948111700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:36:24.863 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:24.239+0000 7f694d6e3700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6948111c40 con 0x7f6948102910 2024-03-24T10:36:24.863 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:24.240+0000 7f6946ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6948102910 0x7f6948111700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:36:24.863 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:24.240+0000 7f6946ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6948102910 0x7f6948111700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38242/0 (socket says 172.21.15.94:38242) 2024-03-24T10:36:24.863 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:24.240+0000 7f6946ffd700 1 -- 172.21.15.94:0/2345033814 learned_addr learned my addr 172.21.15.94:0/2345033814 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:36:24.864 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:24.240+0000 7f6946ffd700 1 -- 172.21.15.94:0/2345033814 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69380094f0 con 0x7f6948102910 2024-03-24T10:36:24.864 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:24.240+0000 7f6946ffd700 1 --2- 172.21.15.94:0/2345033814 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6948102910 0x7f6948111700 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6938000c00 tx=0x7f6938009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:36:24.864 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:24.240+0000 7f692f7fe700 1 -- 172.21.15.94:0/2345033814 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f69380107c0 con 0x7f6948102910 2024-03-24T10:36:24.864 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:24.240+0000 7f692f7fe700 1 -- 172.21.15.94:0/2345033814 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f6938010db0 con 0x7f6948102910 2024-03-24T10:36:24.864 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:24.240+0000 7f694d6e3700 1 -- 172.21.15.94:0/2345033814 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6948111e40 con 0x7f6948102910 2024-03-24T10:36:24.864 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:24.240+0000 7f692f7fe700 1 -- 172.21.15.94:0/2345033814 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f693801ad00 con 0x7f6948102910 2024-03-24T10:36:24.864 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:24.240+0000 7f694d6e3700 1 -- 172.21.15.94:0/2345033814 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69481129b0 con 0x7f6948102910 2024-03-24T10:36:24.864 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:24.240+0000 7f692f7fe700 1 -- 172.21.15.94:0/2345033814 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f6938024440 con 0x7f6948102910 2024-03-24T10:36:24.865 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:24.240+0000 7f692f7fe700 1 -- 172.21.15.94:0/2345033814 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f6938019710 con 0x7f6948102910 2024-03-24T10:36:24.865 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:24.241+0000 7f694d6e3700 1 -- 172.21.15.94:0/2345033814 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f694805f5f0 con 0x7f6948102910 2024-03-24T10:36:24.865 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:24.243+0000 7f692f7fe700 1 -- 172.21.15.94:0/2345033814 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f69380426d0 con 0x7f6948102910 2024-03-24T10:36:24.865 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:24.308+0000 7f694d6e3700 1 -- 172.21.15.94:0/2345033814 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f6948104690 con 0x7f6948102910 2024-03-24T10:36:24.865 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:24.309+0000 7f692f7fe700 1 -- 172.21.15.94:0/2345033814 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f6938031020 con 0x7f6948102910 2024-03-24T10:36:24.865 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:24.309+0000 7f694d6e3700 1 -- 172.21.15.94:0/2345033814 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6948102910 msgr2=0x7f6948111700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:36:24.865 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:24.309+0000 7f694d6e3700 1 --2- 172.21.15.94:0/2345033814 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6948102910 0x7f6948111700 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6938000c00 tx=0x7f6938009250 comp rx=0 tx=0).stop 2024-03-24T10:36:24.865 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:24.309+0000 7f694d6e3700 1 -- 172.21.15.94:0/2345033814 shutdown_connections 2024-03-24T10:36:24.865 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:24.309+0000 7f694d6e3700 1 --2- 172.21.15.94:0/2345033814 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6948102910 0x7f6948111700 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:24.865 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:24.309+0000 7f694d6e3700 1 -- 172.21.15.94:0/2345033814 >> 172.21.15.94:0/2345033814 conn(0x7f69480fe070 msgr2=0x7f69480fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:36:24.866 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:24.310+0000 7f694d6e3700 1 -- 172.21.15.94:0/2345033814 shutdown_connections 2024-03-24T10:36:24.866 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:24.310+0000 7f694d6e3700 1 -- 172.21.15.94:0/2345033814 wait complete. 2024-03-24T10:36:24.866 INFO:teuthology.orchestra.run.smithi094.stdout:Restarting the monitor... 2024-03-24T10:36:25.127 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:24 smithi094 systemd[1]: Stopping Ceph mon.smithi094 for 386110d4-e9ca-11ee-95cd-87774f69a715... 2024-03-24T10:36:25.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:25 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-mon-smithi094[29070]: 2024-03-24T10:36:25.124+0000 7f209ca1a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi094 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T10:36:25.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:25 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-mon-smithi094[29070]: 2024-03-24T10:36:25.124+0000 7f209ca1a700 -1 mon.smithi094@0(leader) e1 *** Got Signal Terminated *** 2024-03-24T10:36:25.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:25 smithi094 podman[29318]: 2024-03-24 10:36:25.340034273 +0000 UTC m=+0.322764634 container died 2f276eaee9d0c7d106f9fc6d2460304d296d27af493b4a935146244faec37400 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-mon-smithi094, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, RELEASE=quincy-9c4e0c3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T10:36:25.934 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:25 smithi094 podman[29318]: 2024-03-24 10:36:25.716909086 +0000 UTC m=+0.699639461 container cleanup 2f276eaee9d0c7d106f9fc6d2460304d296d27af493b4a935146244faec37400 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-mon-smithi094, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=quincy-9c4e0c3, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-24T10:36:25.935 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:25 smithi094 bash[29318]: ceph-386110d4-e9ca-11ee-95cd-87774f69a715-mon-smithi094 2024-03-24T10:36:26.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:25 smithi094 podman[29332]: 2024-03-24 10:36:25.934557122 +0000 UTC m=+0.591899628 container remove 2f276eaee9d0c7d106f9fc6d2460304d296d27af493b4a935146244faec37400 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-mon-smithi094, RELEASE=quincy-9c4e0c3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T10:36:26.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 systemd[1]: ceph-386110d4-e9ca-11ee-95cd-87774f69a715@mon.smithi094.service: Deactivated successfully. 2024-03-24T10:36:26.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 systemd[1]: Stopped Ceph mon.smithi094 for 386110d4-e9ca-11ee-95cd-87774f69a715. 2024-03-24T10:36:26.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 systemd[1]: Starting Ceph mon.smithi094 for 386110d4-e9ca-11ee-95cd-87774f69a715... 2024-03-24T10:36:26.579 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 podman[29428]: 2024-03-24 10:36:26.187552968 +0000 UTC m=+0.018040384 image pull 837124da804ffab28571a3e286aff1c534a701f228c0d3d7a309fcb128aca1de quay.ceph.io/ceph-ci/ceph:quincy 2024-03-24T10:36:26.579 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 podman[29428]: 2024-03-24 10:36:26.28526588 +0000 UTC m=+0.115753289 container create 76584fceb32b32ae4332ea8019fd710a55ac7733c1b6c174d655481b8e7115ae (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-mon-smithi094, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-9c4e0c3, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True) 2024-03-24T10:36:26.579 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 podman[29428]: 2024-03-24 10:36:26.49596946 +0000 UTC m=+0.326456869 container init 76584fceb32b32ae4332ea8019fd710a55ac7733c1b6c174d655481b8e7115ae (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-mon-smithi094, org.label-schema.vendor=CentOS, RELEASE=quincy-9c4e0c3, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=) 2024-03-24T10:36:26.579 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 podman[29428]: 2024-03-24 10:36:26.498943052 +0000 UTC m=+0.329430462 container start 76584fceb32b32ae4332ea8019fd710a55ac7733c1b6c174d655481b8e7115ae (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-mon-smithi094, org.label-schema.build-date=20240319, GIT_CLEAN=True, RELEASE=quincy-9c4e0c3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-24T10:36:26.579 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: set uid:gid to 167:167 (ceph:ceph) 2024-03-24T10:36:26.579 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable), process ceph-mon, pid 2 2024-03-24T10:36:26.580 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: pidfile_write: ignore empty --pid-file 2024-03-24T10:36:26.580 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: load: jerasure load: lrc 2024-03-24T10:36:26.580 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: RocksDB version: 6.15.5 2024-03-24T10:36:26.580 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-03-24T10:36:26.580 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Compile date Mar 21 2024 2024-03-24T10:36:26.580 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: DB SUMMARY 2024-03-24T10:36:26.580 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: DB Session ID: 4JK57Z0E2FLZMEOZ4R41 2024-03-24T10:36:26.580 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: CURRENT file: CURRENT 2024-03-24T10:36:26.580 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: IDENTITY file: IDENTITY 2024-03-24T10:36:26.580 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-03-24T10:36:26.580 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi094/store.db dir, Total Num: 1, files: 000008.sst 2024-03-24T10:36:26.580 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi094/store.db: 000010.log size: 98448 ; 2024-03-24T10:36:26.580 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.error_if_exists: 0 2024-03-24T10:36:26.580 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.create_if_missing: 0 2024-03-24T10:36:26.581 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.paranoid_checks: 1 2024-03-24T10:36:26.582 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-24T10:36:26.582 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.env: 0x557fc7389a80 2024-03-24T10:36:26.582 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.fs: Posix File System 2024-03-24T10:36:26.582 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.info_log: 0x557fc8465c00 2024-03-24T10:36:26.582 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.max_file_opening_threads: 16 2024-03-24T10:36:26.582 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.statistics: (nil) 2024-03-24T10:36:26.582 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.use_fsync: 0 2024-03-24T10:36:26.582 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.max_log_file_size: 0 2024-03-24T10:36:26.582 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-24T10:36:26.582 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-24T10:36:26.582 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.keep_log_file_num: 1000 2024-03-24T10:36:26.582 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.recycle_log_file_num: 0 2024-03-24T10:36:26.582 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.allow_fallocate: 1 2024-03-24T10:36:26.583 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.allow_mmap_reads: 0 2024-03-24T10:36:26.583 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.allow_mmap_writes: 0 2024-03-24T10:36:26.583 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.use_direct_reads: 0 2024-03-24T10:36:26.583 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-24T10:36:26.583 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.create_missing_column_families: 0 2024-03-24T10:36:26.583 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.db_log_dir: 2024-03-24T10:36:26.583 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi094/store.db 2024-03-24T10:36:26.583 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-24T10:36:26.583 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-24T10:36:26.583 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-24T10:36:26.583 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-24T10:36:26.583 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-24T10:36:26.583 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-24T10:36:26.584 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.advise_random_on_open: 1 2024-03-24T10:36:26.584 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.db_write_buffer_size: 0 2024-03-24T10:36:26.584 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.write_buffer_manager: 0x557fc91daab0 2024-03-24T10:36:26.584 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-24T10:36:26.584 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-03-24T10:36:26.584 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-24T10:36:26.584 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-24T10:36:26.584 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.rate_limiter: (nil) 2024-03-24T10:36:26.584 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-24T10:36:26.584 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.wal_recovery_mode: 2 2024-03-24T10:36:26.584 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.enable_thread_tracking: 0 2024-03-24T10:36:26.584 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.enable_pipelined_write: 0 2024-03-24T10:36:26.584 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.unordered_write: 0 2024-03-24T10:36:26.585 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-24T10:36:26.585 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-24T10:36:26.585 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-24T10:36:26.585 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-24T10:36:26.585 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.row_cache: None 2024-03-24T10:36:26.585 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.wal_filter: None 2024-03-24T10:36:26.585 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-24T10:36:26.585 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.allow_ingest_behind: 0 2024-03-24T10:36:26.585 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.preserve_deletes: 0 2024-03-24T10:36:26.585 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.two_write_queues: 0 2024-03-24T10:36:26.585 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.manual_wal_flush: 0 2024-03-24T10:36:26.585 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.atomic_flush: 0 2024-03-24T10:36:26.585 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-24T10:36:26.585 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-24T10:36:26.585 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-24T10:36:26.586 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.log_readahead_size: 0 2024-03-24T10:36:26.586 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-24T10:36:26.586 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.best_efforts_recovery: 0 2024-03-24T10:36:26.586 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-24T10:36:26.586 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-24T10:36:26.586 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.allow_data_in_errors: 0 2024-03-24T10:36:26.586 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.db_host_id: __hostname__ 2024-03-24T10:36:26.586 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.max_background_jobs: 2 2024-03-24T10:36:26.586 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.max_background_compactions: -1 2024-03-24T10:36:26.586 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.max_subcompactions: 1 2024-03-24T10:36:26.586 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-24T10:36:26.586 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-24T10:36:26.586 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-24T10:36:26.586 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.max_total_wal_size: 0 2024-03-24T10:36:26.586 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-24T10:36:26.587 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-24T10:36:26.587 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-24T10:36:26.587 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-24T10:36:26.587 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.max_open_files: -1 2024-03-24T10:36:26.587 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.bytes_per_sync: 0 2024-03-24T10:36:26.587 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-24T10:36:26.587 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-24T10:36:26.587 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.compaction_readahead_size: 0 2024-03-24T10:36:26.587 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.max_background_flushes: -1 2024-03-24T10:36:26.587 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Compression algorithms supported: 2024-03-24T10:36:26.588 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-24T10:36:26.588 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: kZSTD supported: 0 2024-03-24T10:36:26.588 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: kXpressCompression supported: 0 2024-03-24T10:36:26.588 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: kLZ4HCCompression supported: 1 2024-03-24T10:36:26.588 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: kZlibCompression supported: 1 2024-03-24T10:36:26.588 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: kSnappyCompression supported: 1 2024-03-24T10:36:26.588 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: kLZ4Compression supported: 1 2024-03-24T10:36:26.588 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: kBZip2Compression supported: 0 2024-03-24T10:36:26.588 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-24T10:36:26.588 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi094/store.db/MANIFEST-000009 2024-03-24T10:36:26.588 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-03-24T10:36:26.588 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-24T10:36:26.588 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.merge_operator: 2024-03-24T10:36:26.589 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.compaction_filter: None 2024-03-24T10:36:26.589 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.compaction_filter_factory: None 2024-03-24T10:36:26.589 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.sst_partitioner_factory: None 2024-03-24T10:36:26.589 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-24T10:36:26.589 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-24T10:36:26.589 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x557fc84baf58) 2024-03-24T10:36:26.589 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: cache_index_and_filter_blocks: 1 2024-03-24T10:36:26.589 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-24T10:36:26.589 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-24T10:36:26.589 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: pin_top_level_index_and_filter: 1 2024-03-24T10:36:26.589 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: index_type: 0 2024-03-24T10:36:26.589 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: data_block_index_type: 0 2024-03-24T10:36:26.589 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: index_shortening: 1 2024-03-24T10:36:26.590 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-24T10:36:26.590 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: hash_index_allow_collision: 1 2024-03-24T10:36:26.590 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: checksum: 1 2024-03-24T10:36:26.590 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: no_block_cache: 0 2024-03-24T10:36:26.590 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: block_cache: 0x557fc84ed090 2024-03-24T10:36:26.590 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: block_cache_name: BinnedLRUCache 2024-03-24T10:36:26.590 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: block_cache_options: 2024-03-24T10:36:26.590 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: capacity : 536870912 2024-03-24T10:36:26.590 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: num_shard_bits : 4 2024-03-24T10:36:26.590 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: strict_capacity_limit : 0 2024-03-24T10:36:26.590 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: high_pri_pool_ratio: 0.000 2024-03-24T10:36:26.590 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: block_cache_compressed: (nil) 2024-03-24T10:36:26.591 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: persistent_cache: (nil) 2024-03-24T10:36:26.591 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: block_size: 4096 2024-03-24T10:36:26.591 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: block_size_deviation: 10 2024-03-24T10:36:26.591 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: block_restart_interval: 16 2024-03-24T10:36:26.591 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: index_block_restart_interval: 1 2024-03-24T10:36:26.591 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: metadata_block_size: 4096 2024-03-24T10:36:26.591 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: partition_filters: 0 2024-03-24T10:36:26.591 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: use_delta_encoding: 1 2024-03-24T10:36:26.591 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-03-24T10:36:26.591 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: whole_key_filtering: 1 2024-03-24T10:36:26.591 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: verify_compression: 0 2024-03-24T10:36:26.591 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: read_amp_bytes_per_bit: 0 2024-03-24T10:36:26.591 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: format_version: 4 2024-03-24T10:36:26.591 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: enable_index_compression: 1 2024-03-24T10:36:26.591 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: block_align: 0 2024-03-24T10:36:26.592 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.write_buffer_size: 33554432 2024-03-24T10:36:26.592 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.max_write_buffer_number: 2 2024-03-24T10:36:26.592 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.compression: NoCompression 2024-03-24T10:36:26.592 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.bottommost_compression: Disabled 2024-03-24T10:36:26.592 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.prefix_extractor: nullptr 2024-03-24T10:36:26.592 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-24T10:36:26.592 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.num_levels: 7 2024-03-24T10:36:26.592 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-24T10:36:26.592 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-24T10:36:26.592 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-24T10:36:26.592 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-24T10:36:26.592 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-24T10:36:26.592 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-24T10:36:26.592 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-24T10:36:26.592 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-24T10:36:26.593 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-24T10:36:26.593 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-24T10:36:26.593 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-24T10:36:26.593 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.compression_opts.level: 32767 2024-03-24T10:36:26.593 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.compression_opts.strategy: 0 2024-03-24T10:36:26.593 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-24T10:36:26.593 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-24T10:36:26.593 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-24T10:36:26.593 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.compression_opts.enabled: false 2024-03-24T10:36:26.593 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-24T10:36:26.593 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-24T10:36:26.593 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-24T10:36:26.593 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.target_file_size_base: 67108864 2024-03-24T10:36:26.593 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-24T10:36:26.593 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-24T10:36:26.594 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-24T10:36:26.594 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-24T10:36:26.594 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-24T10:36:26.594 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-24T10:36:26.594 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-24T10:36:26.594 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-24T10:36:26.594 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-24T10:36:26.594 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-24T10:36:26.594 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-24T10:36:26.594 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-24T10:36:26.594 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-24T10:36:26.594 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.arena_block_size: 4194304 2024-03-24T10:36:26.594 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-24T10:36:26.594 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-24T10:36:26.594 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-03-24T10:36:26.595 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.disable_auto_compactions: 0 2024-03-24T10:36:26.595 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-24T10:36:26.595 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-24T10:36:26.595 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-24T10:36:26.595 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-24T10:36:26.595 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-24T10:36:26.595 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-24T10:36:26.595 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-24T10:36:26.595 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-24T10:36:26.595 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-24T10:36:26.595 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-24T10:36:26.595 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.table_properties_collectors: 2024-03-24T10:36:26.595 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.inplace_update_support: 0 2024-03-24T10:36:26.595 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-24T10:36:26.595 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-24T10:36:26.596 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-24T10:36:26.596 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-24T10:36:26.596 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.bloom_locality: 0 2024-03-24T10:36:26.596 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.max_successive_merges: 0 2024-03-24T10:36:26.596 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-24T10:36:26.596 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.paranoid_file_checks: 0 2024-03-24T10:36:26.596 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.force_consistency_checks: 1 2024-03-24T10:36:26.596 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.report_bg_io_stats: 0 2024-03-24T10:36:26.596 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.ttl: 2592000 2024-03-24T10:36:26.596 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-24T10:36:26.596 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.enable_blob_files: false 2024-03-24T10:36:26.596 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.min_blob_size: 0 2024-03-24T10:36:26.596 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.blob_file_size: 268435456 2024-03-24T10:36:26.597 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-24T10:36:26.597 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-24T10:36:26.597 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-24T10:36:26.597 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi094/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-24T10:36:26.597 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2024-03-24T10:36:26.597 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2024-03-24T10:36:26.598 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711276586524325, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-03-24T10:36:26.598 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2024-03-24T10:36:26.598 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-03-24T10:36:26.598 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711276586525190, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 93499, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 91611, "index_size": 248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13606, "raw_average_key_size": 49, "raw_value_size": 84149, "raw_average_value_size": 305, "num_data_blocks": 11, "num_entries": 275, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1711276586, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "1ae0d5c3-c4e6-450d-81ad-f4149dcd597c", "db_session_id": "4JK57Z0E2FLZMEOZ4R41"}} 2024-03-24T10:36:26.598 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2024-03-24T10:36:26.598 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711276586525510, "job": 1, "event": "recovery_finished"} 2024-03-24T10:36:26.598 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-smithi094/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-24T10:36:26.598 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x557fc8481180 2024-03-24T10:36:26.598 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: DB pointer 0x557fc8556000 2024-03-24T10:36:26.598 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-03-24T10:36:26.598 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-03-24T10:36:26.598 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: ** DB Stats ** 2024-03-24T10:36:26.598 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-24T10:36:26.598 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-24T10:36:26.599 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-24T10:36:26.599 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-24T10:36:26.599 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-24T10:36:26.599 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-03-24T10:36:26.599 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-24T10:36:26.599 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: 2024-03-24T10:36:26.599 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: ** Compaction Stats [default] ** 2024-03-24T10:36:26.599 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-24T10:36:26.599 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-24T10:36:26.599 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: L0 2/0 93.11 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 150.1 0.00 0.00 1 0.001 0 0 2024-03-24T10:36:26.599 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Sum 2/0 93.11 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 150.1 0.00 0.00 1 0.001 0 0 2024-03-24T10:36:26.599 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 150.1 0.00 0.00 1 0.001 0 0 2024-03-24T10:36:26.599 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: 2024-03-24T10:36:26.599 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: ** Compaction Stats [default] ** 2024-03-24T10:36:26.599 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-24T10:36:26.599 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-24T10:36:26.600 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 150.1 0.00 0.00 1 0.001 0 0 2024-03-24T10:36:26.600 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-24T10:36:26.600 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-24T10:36:26.600 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-24T10:36:26.600 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-24T10:36:26.600 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-24T10:36:26.600 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-24T10:36:26.600 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Cumulative compaction: 0.00 GB write, 29.85 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-24T10:36:26.600 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Interval compaction: 0.00 GB write, 29.85 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-24T10:36:26.600 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-24T10:36:26.600 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: 2024-03-24T10:36:26.600 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-24T10:36:26.600 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: 2024-03-24T10:36:26.600 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: ** Compaction Stats [default] ** 2024-03-24T10:36:26.600 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-24T10:36:26.600 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-24T10:36:26.601 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: L0 2/0 93.11 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 150.1 0.00 0.00 1 0.001 0 0 2024-03-24T10:36:26.601 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Sum 2/0 93.11 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 150.1 0.00 0.00 1 0.001 0 0 2024-03-24T10:36:26.601 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-03-24T10:36:26.601 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: 2024-03-24T10:36:26.601 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: ** Compaction Stats [default] ** 2024-03-24T10:36:26.601 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-24T10:36:26.601 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-24T10:36:26.601 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 150.1 0.00 0.00 1 0.001 0 0 2024-03-24T10:36:26.601 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-24T10:36:26.601 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-24T10:36:26.601 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-24T10:36:26.601 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-24T10:36:26.601 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-24T10:36:26.601 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-24T10:36:26.602 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Cumulative compaction: 0.00 GB write, 29.52 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-24T10:36:26.602 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-24T10:36:26.602 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-24T10:36:26.602 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: 2024-03-24T10:36:26.602 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-24T10:36:26.602 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: starting mon.smithi094 rank 0 at public addrs [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] at bind addrs [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi094 fsid 386110d4-e9ca-11ee-95cd-87774f69a715 2024-03-24T10:36:26.602 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: mon.smithi094@-1(???) e1 preinit fsid 386110d4-e9ca-11ee-95cd-87774f69a715 2024-03-24T10:36:26.602 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: mon.smithi094@-1(???).mds e1 new map 2024-03-24T10:36:26.602 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: mon.smithi094@-1(???).mds e1 print_map 2024-03-24T10:36:26.602 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: e1 2024-03-24T10:36:26.602 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T10:36:26.602 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:36:26.602 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: legacy client fscid: -1 2024-03-24T10:36:26.602 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: 2024-03-24T10:36:26.603 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: No filesystems configured 2024-03-24T10:36:26.603 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: mon.smithi094@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-24T10:36:26.603 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: mon.smithi094@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-24T10:36:26.603 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: mon.smithi094@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-24T10:36:26.603 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: mon.smithi094@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-24T10:36:26.603 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: mon.smithi094@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-24T10:36:26.603 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 bash[29428]: 76584fceb32b32ae4332ea8019fd710a55ac7733c1b6c174d655481b8e7115ae 2024-03-24T10:36:26.603 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 systemd[1]: Started Ceph mon.smithi094 for 386110d4-e9ca-11ee-95cd-87774f69a715. 2024-03-24T10:36:26.603 INFO:teuthology.orchestra.run.smithi094.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-24T10:36:26.848 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: mon.smithi094 is new leader, mons smithi094 in quorum (ranks 0) 2024-03-24T10:36:26.848 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: monmap e1: 1 mons at {smithi094=[v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-24T10:36:26.848 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: fsmap 2024-03-24T10:36:26.848 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: osdmap e1: 0 total, 0 up, 0 in 2024-03-24T10:36:26.848 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:26 smithi094 ceph-mon[29446]: mgrmap e1: no daemons active 2024-03-24T10:36:27.626 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:26.946+0000 7fc40f7f5700 1 Processor -- start 2024-03-24T10:36:27.626 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:26.946+0000 7fc40f7f5700 1 -- start start 2024-03-24T10:36:27.626 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:26.947+0000 7fc40f7f5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc408101e10 0x7fc408102220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:36:27.626 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:26.947+0000 7fc40f7f5700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc408102760 con 0x7fc408101e10 2024-03-24T10:36:27.626 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:26.947+0000 7fc40d591700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc408101e10 0x7fc408102220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:36:27.626 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:26.947+0000 7fc40d591700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc408101e10 0x7fc408102220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38252/0 (socket says 172.21.15.94:38252) 2024-03-24T10:36:27.626 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:26.947+0000 7fc40d591700 1 -- 172.21.15.94:0/1011984747 learned_addr learned my addr 172.21.15.94:0/1011984747 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:36:27.626 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:26.947+0000 7fc40d591700 1 -- 172.21.15.94:0/1011984747 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4081028a0 con 0x7fc408101e10 2024-03-24T10:36:27.626 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:26.947+0000 7fc40d591700 1 --2- 172.21.15.94:0/1011984747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc408101e10 0x7fc408102220 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8009e50 tx=0x7fc3f8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e56d317331e6c11 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:36:27.627 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:26.947+0000 7fc407fff700 1 -- 172.21.15.94:0/1011984747 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc3f80092e0 con 0x7fc408101e10 2024-03-24T10:36:27.627 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:26.947+0000 7fc407fff700 1 -- 172.21.15.94:0/1011984747 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fc3f8012430 con 0x7fc408101e10 2024-03-24T10:36:27.627 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:26.947+0000 7fc407fff700 1 -- 172.21.15.94:0/1011984747 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc3f80075c0 con 0x7fc408101e10 2024-03-24T10:36:27.627 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:26.948+0000 7fc40f7f5700 1 -- 172.21.15.94:0/1011984747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc408101e10 msgr2=0x7fc408102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:36:27.627 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:26.948+0000 7fc40f7f5700 1 --2- 172.21.15.94:0/1011984747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc408101e10 0x7fc408102220 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8009e50 tx=0x7fc3f8009840 comp rx=0 tx=0).stop 2024-03-24T10:36:27.627 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:26.948+0000 7fc40f7f5700 1 -- 172.21.15.94:0/1011984747 shutdown_connections 2024-03-24T10:36:27.627 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:26.948+0000 7fc40f7f5700 1 --2- 172.21.15.94:0/1011984747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc408101e10 0x7fc408102220 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:27.627 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:26.948+0000 7fc40f7f5700 1 -- 172.21.15.94:0/1011984747 >> 172.21.15.94:0/1011984747 conn(0x7fc4080fc700 msgr2=0x7fc4080feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:36:27.627 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:26.948+0000 7fc40f7f5700 1 -- 172.21.15.94:0/1011984747 shutdown_connections 2024-03-24T10:36:27.627 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:26.948+0000 7fc40f7f5700 1 -- 172.21.15.94:0/1011984747 wait complete. 2024-03-24T10:36:27.627 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:26.949+0000 7fc40f7f5700 1 Processor -- start 2024-03-24T10:36:27.627 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:26.949+0000 7fc40f7f5700 1 -- start start 2024-03-24T10:36:27.627 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:26.949+0000 7fc40f7f5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc408101e10 0x7fc408101630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:36:27.627 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:26.949+0000 7fc40f7f5700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc408114410 con 0x7fc408101e10 2024-03-24T10:36:27.627 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:26.949+0000 7fc40d591700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc408101e10 0x7fc408101630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:36:27.627 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:26.949+0000 7fc40d591700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc408101e10 0x7fc408101630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38262/0 (socket says 172.21.15.94:38262) 2024-03-24T10:36:27.628 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:26.949+0000 7fc40d591700 1 -- 172.21.15.94:0/4274619891 learned_addr learned my addr 172.21.15.94:0/4274619891 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:36:27.628 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:26.949+0000 7fc40d591700 1 -- 172.21.15.94:0/4274619891 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f8009ae0 con 0x7fc408101e10 2024-03-24T10:36:27.628 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:26.949+0000 7fc40d591700 1 --2- 172.21.15.94:0/4274619891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc408101e10 0x7fc408101630 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8009fd0 tx=0x7fc3f8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:36:27.628 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:26.950+0000 7fc405ffb700 1 -- 172.21.15.94:0/4274619891 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc3f80092e0 con 0x7fc408101e10 2024-03-24T10:36:27.628 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:26.950+0000 7fc405ffb700 1 -- 172.21.15.94:0/4274619891 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fc3f8007d10 con 0x7fc408101e10 2024-03-24T10:36:27.628 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:26.950+0000 7fc40f7f5700 1 -- 172.21.15.94:0/4274619891 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc408114550 con 0x7fc408101e10 2024-03-24T10:36:27.628 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:26.950+0000 7fc405ffb700 1 -- 172.21.15.94:0/4274619891 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc3f801c880 con 0x7fc408101e10 2024-03-24T10:36:27.628 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:26.950+0000 7fc40f7f5700 1 -- 172.21.15.94:0/4274619891 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc408114a40 con 0x7fc408101e10 2024-03-24T10:36:27.628 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:26.950+0000 7fc405ffb700 1 -- 172.21.15.94:0/4274619891 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fc3f8019070 con 0x7fc408101e10 2024-03-24T10:36:27.628 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:26.950+0000 7fc405ffb700 1 -- 172.21.15.94:0/4274619891 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fc3f8025b00 con 0x7fc408101e10 2024-03-24T10:36:27.628 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:26.951+0000 7fc40f7f5700 1 -- 172.21.15.94:0/4274619891 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc40804bed0 con 0x7fc408101e10 2024-03-24T10:36:27.628 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:26.953+0000 7fc405ffb700 1 -- 172.21.15.94:0/4274619891 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fc3f802a060 con 0x7fc408101e10 2024-03-24T10:36:27.628 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:27.019+0000 7fc40f7f5700 1 -- 172.21.15.94:0/4274619891 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7fc4081057a0 con 0x7fc408101e10 2024-03-24T10:36:27.628 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:27.021+0000 7fc405ffb700 1 -- 172.21.15.94:0/4274619891 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7fc4081057a0 con 0x7fc408101e10 2024-03-24T10:36:27.628 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:27.022+0000 7fc40f7f5700 1 -- 172.21.15.94:0/4274619891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc408101e10 msgr2=0x7fc408101630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:36:27.629 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:27.022+0000 7fc40f7f5700 1 --2- 172.21.15.94:0/4274619891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc408101e10 0x7fc408101630 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8009fd0 tx=0x7fc3f8009840 comp rx=0 tx=0).stop 2024-03-24T10:36:27.629 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:27.022+0000 7fc40f7f5700 1 -- 172.21.15.94:0/4274619891 shutdown_connections 2024-03-24T10:36:27.629 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:27.022+0000 7fc40f7f5700 1 --2- 172.21.15.94:0/4274619891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc408101e10 0x7fc408101630 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:27.629 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:27.022+0000 7fc40f7f5700 1 -- 172.21.15.94:0/4274619891 >> 172.21.15.94:0/4274619891 conn(0x7fc4080fc700 msgr2=0x7fc4080fd320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:36:27.629 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:27.022+0000 7fc40f7f5700 1 -- 172.21.15.94:0/4274619891 shutdown_connections 2024-03-24T10:36:27.629 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:27.022+0000 7fc40f7f5700 1 -- 172.21.15.94:0/4274619891 wait complete. 2024-03-24T10:36:27.629 INFO:teuthology.orchestra.run.smithi094.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-24T10:36:27.629 INFO:teuthology.orchestra.run.smithi094.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-24T10:36:27.629 INFO:teuthology.orchestra.run.smithi094.stdout:Creating mgr... 2024-03-24T10:36:27.629 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying port 9283 ... 2024-03-24T10:36:27.804 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-386110d4-e9ca-11ee-95cd-87774f69a715@mgr.smithi094.yaethu 2024-03-24T10:36:27.804 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Failed to reset failed state of unit ceph-386110d4-e9ca-11ee-95cd-87774f69a715@mgr.smithi094.yaethu.service: Unit ceph-386110d4-e9ca-11ee-95cd-87774f69a715@mgr.smithi094.yaethu.service not loaded. 2024-03-24T10:36:27.945 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-386110d4-e9ca-11ee-95cd-87774f69a715.target.wants/ceph-386110d4-e9ca-11ee-95cd-87774f69a715@mgr.smithi094.yaethu.service → /etc/systemd/system/ceph-386110d4-e9ca-11ee-95cd-87774f69a715@.service. 2024-03-24T10:36:28.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:28 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/4274619891' entity='client.admin' 2024-03-24T10:36:29.617 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-24T10:36:29.618 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout disabled 2024-03-24T10:36:29.625 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-24T10:36:29.625 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout inactive 2024-03-24T10:36:29.625 INFO:teuthology.orchestra.run.smithi094.stdout:firewalld.service is not enabled 2024-03-24T10:36:29.625 INFO:teuthology.orchestra.run.smithi094.stdout:Not possible to enable service . firewalld.service is not available 2024-03-24T10:36:29.633 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-24T10:36:29.633 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout disabled 2024-03-24T10:36:29.641 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-24T10:36:29.641 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout inactive 2024-03-24T10:36:29.641 INFO:teuthology.orchestra.run.smithi094.stdout:firewalld.service is not enabled 2024-03-24T10:36:29.641 INFO:teuthology.orchestra.run.smithi094.stdout:Not possible to open ports <[9283]>. firewalld.service is not available 2024-03-24T10:36:29.641 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for mgr to start... 2024-03-24T10:36:29.641 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for mgr... 2024-03-24T10:36:30.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:30 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/1672009455' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-24T10:36:30.791 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-03-24T10:36:30.791 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-03-24T10:36:30.791 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsid": "386110d4-e9ca-11ee-95cd-87774f69a715", 2024-03-24T10:36:30.791 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "health": { 2024-03-24T10:36:30.791 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-24T10:36:30.791 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-24T10:36:30.792 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-24T10:36:30.792 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-03-24T10:36:30.792 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-24T10:36:30.792 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-24T10:36:30.792 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 0 2024-03-24T10:36:30.792 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-03-24T10:36:30.792 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-24T10:36:30.792 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "smithi094" 2024-03-24T10:36:30.792 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-03-24T10:36:30.792 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-03-24T10:36:30.792 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-24T10:36:30.792 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T10:36:30.792 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-03-24T10:36:30.792 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-24T10:36:30.792 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-03-24T10:36:30.793 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-24T10:36:30.793 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T10:36:30.793 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-24T10:36:30.793 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-24T10:36:30.793 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-24T10:36:30.801 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-24T10:36:30.801 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-24T10:36:30.801 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-24T10:36:30.801 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-03-24T10:36:30.801 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-24T10:36:30.802 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-24T10:36:30.802 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-24T10:36:30.802 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-24T10:36:30.802 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-24T10:36:30.802 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-24T10:36:30.802 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-24T10:36:30.802 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-24T10:36:30.802 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-24T10:36:30.802 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-03-24T10:36:30.802 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-24T10:36:30.802 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T10:36:30.802 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-24T10:36:30.802 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-24T10:36:30.802 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-03-24T10:36:30.802 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-24T10:36:30.803 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-24T10:36:30.803 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-24T10:36:30.803 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-24T10:36:30.803 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-24T10:36:30.803 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-24T10:36:30.803 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "restful" 2024-03-24T10:36:30.803 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-03-24T10:36:30.803 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-24T10:36:30.803 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-03-24T10:36:30.803 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-24T10:36:30.803 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T10:36:30.803 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modified": "2024-03-24T10:36:12.012396+0000", 2024-03-24T10:36:30.803 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-24T10:36:30.803 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-03-24T10:36:30.804 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-24T10:36:30.804 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-03-24T10:36:30.804 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:30.095+0000 7f9835f23700 1 Processor -- start 2024-03-24T10:36:30.804 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:30.096+0000 7f9835f23700 1 -- start start 2024-03-24T10:36:30.804 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:30.096+0000 7f9835f23700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9830071250 0x7f9830105540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:36:30.804 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:30.096+0000 7f9835f23700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9830071660 con 0x7f9830071250 2024-03-24T10:36:30.804 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:30.096+0000 7f982f7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9830071250 0x7f9830105540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:36:30.804 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:30.096+0000 7f982f7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9830071250 0x7f9830105540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56686/0 (socket says 172.21.15.94:56686) 2024-03-24T10:36:30.804 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:30.096+0000 7f982f7fe700 1 -- 172.21.15.94:0/3859537304 learned_addr learned my addr 172.21.15.94:0/3859537304 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:36:30.804 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:30.096+0000 7f982f7fe700 1 -- 172.21.15.94:0/3859537304 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9830105a80 con 0x7f9830071250 2024-03-24T10:36:30.804 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:30.096+0000 7f982f7fe700 1 --2- 172.21.15.94:0/3859537304 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9830071250 0x7f9830105540 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f98200098e0 tx=0x7f9820009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f57cafeac413e8e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:36:30.804 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:30.097+0000 7f982e7fc700 1 -- 172.21.15.94:0/3859537304 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f982000b860 con 0x7f9830071250 2024-03-24T10:36:30.805 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:30.097+0000 7f982e7fc700 1 -- 172.21.15.94:0/3859537304 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f982000be50 con 0x7f9830071250 2024-03-24T10:36:30.805 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:30.097+0000 7f982e7fc700 1 -- 172.21.15.94:0/3859537304 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9820010380 con 0x7f9830071250 2024-03-24T10:36:30.805 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:30.097+0000 7f9835f23700 1 -- 172.21.15.94:0/3859537304 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9830071250 msgr2=0x7f9830105540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:36:30.805 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:30.097+0000 7f9835f23700 1 --2- 172.21.15.94:0/3859537304 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9830071250 0x7f9830105540 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f98200098e0 tx=0x7f9820009250 comp rx=0 tx=0).stop 2024-03-24T10:36:30.805 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:30.097+0000 7f9835f23700 1 -- 172.21.15.94:0/3859537304 shutdown_connections 2024-03-24T10:36:30.805 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:30.097+0000 7f9835f23700 1 --2- 172.21.15.94:0/3859537304 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9830071250 0x7f9830105540 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:30.805 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:30.097+0000 7f9835f23700 1 -- 172.21.15.94:0/3859537304 >> 172.21.15.94:0/3859537304 conn(0x7f98300fcfb0 msgr2=0x7f98300ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:36:30.805 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:30.097+0000 7f9835f23700 1 -- 172.21.15.94:0/3859537304 shutdown_connections 2024-03-24T10:36:30.805 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:30.097+0000 7f9835f23700 1 -- 172.21.15.94:0/3859537304 wait complete. 2024-03-24T10:36:30.805 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:30.098+0000 7f9835f23700 1 Processor -- start 2024-03-24T10:36:30.805 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:30.098+0000 7f9835f23700 1 -- start start 2024-03-24T10:36:30.805 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:30.098+0000 7f9835f23700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9830071250 0x7f9830111760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:36:30.805 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:30.098+0000 7f9835f23700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9830111ca0 con 0x7f9830071250 2024-03-24T10:36:30.805 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:30.098+0000 7f982f7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9830071250 0x7f9830111760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:36:30.805 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:30.098+0000 7f982f7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9830071250 0x7f9830111760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56690/0 (socket says 172.21.15.94:56690) 2024-03-24T10:36:30.805 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:30.098+0000 7f982f7fe700 1 -- 172.21.15.94:0/1672009455 learned_addr learned my addr 172.21.15.94:0/1672009455 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:36:30.806 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:30.099+0000 7f982f7fe700 1 -- 172.21.15.94:0/1672009455 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98200094f0 con 0x7f9830071250 2024-03-24T10:36:30.806 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:30.099+0000 7f982f7fe700 1 --2- 172.21.15.94:0/1672009455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9830071250 0x7f9830111760 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9820011bd0 tx=0x7f9820009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:36:30.806 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:30.099+0000 7f9817fff700 1 -- 172.21.15.94:0/1672009455 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f98200107c0 con 0x7f9830071250 2024-03-24T10:36:30.806 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:30.099+0000 7f9817fff700 1 -- 172.21.15.94:0/1672009455 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f9820010db0 con 0x7f9830071250 2024-03-24T10:36:30.806 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:30.099+0000 7f9817fff700 1 -- 172.21.15.94:0/1672009455 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f982001ad00 con 0x7f9830071250 2024-03-24T10:36:30.806 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:30.099+0000 7f9835f23700 1 -- 172.21.15.94:0/1672009455 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9830111ea0 con 0x7f9830071250 2024-03-24T10:36:30.806 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:30.099+0000 7f9835f23700 1 -- 172.21.15.94:0/1672009455 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9830112a10 con 0x7f9830071250 2024-03-24T10:36:30.806 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:30.099+0000 7f9817fff700 1 -- 172.21.15.94:0/1672009455 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f98200256d0 con 0x7f9830071250 2024-03-24T10:36:30.806 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:30.099+0000 7f9817fff700 1 -- 172.21.15.94:0/1672009455 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f98200198e0 con 0x7f9830071250 2024-03-24T10:36:30.806 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:30.100+0000 7f9835f23700 1 -- 172.21.15.94:0/1672009455 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f983005f5f0 con 0x7f9830071250 2024-03-24T10:36:30.806 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:30.104+0000 7f9817fff700 1 -- 172.21.15.94:0/1672009455 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f982002a060 con 0x7f9830071250 2024-03-24T10:36:30.806 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:30.174+0000 7f9835f23700 1 -- 172.21.15.94:0/1672009455 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f98301068c0 con 0x7f9830071250 2024-03-24T10:36:30.806 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:30.174+0000 7f9817fff700 1 -- 172.21.15.94:0/1672009455 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f9820020070 con 0x7f9830071250 2024-03-24T10:36:30.807 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:30.175+0000 7f9835f23700 1 -- 172.21.15.94:0/1672009455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9830071250 msgr2=0x7f9830111760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:36:30.807 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:30.175+0000 7f9835f23700 1 --2- 172.21.15.94:0/1672009455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9830071250 0x7f9830111760 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9820011bd0 tx=0x7f9820009250 comp rx=0 tx=0).stop 2024-03-24T10:36:30.807 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:30.175+0000 7f9835f23700 1 -- 172.21.15.94:0/1672009455 shutdown_connections 2024-03-24T10:36:30.807 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:30.175+0000 7f9835f23700 1 --2- 172.21.15.94:0/1672009455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9830071250 0x7f9830111760 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:30.807 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:30.175+0000 7f9835f23700 1 -- 172.21.15.94:0/1672009455 >> 172.21.15.94:0/1672009455 conn(0x7f98300fcfb0 msgr2=0x7f98300fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:36:30.807 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:30.175+0000 7f9835f23700 1 -- 172.21.15.94:0/1672009455 shutdown_connections 2024-03-24T10:36:30.807 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:30.175+0000 7f9835f23700 1 -- 172.21.15.94:0/1672009455 wait complete. 2024-03-24T10:36:30.807 INFO:teuthology.orchestra.run.smithi094.stdout:mgr not available, waiting (1/15)... 2024-03-24T10:36:33.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:33 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/3380853142' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-24T10:36:33.788 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-03-24T10:36:33.788 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-03-24T10:36:33.789 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsid": "386110d4-e9ca-11ee-95cd-87774f69a715", 2024-03-24T10:36:33.789 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "health": { 2024-03-24T10:36:33.789 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-24T10:36:33.789 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-24T10:36:33.789 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-24T10:36:33.789 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-03-24T10:36:33.789 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-24T10:36:33.789 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-24T10:36:33.789 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 0 2024-03-24T10:36:33.789 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-03-24T10:36:33.789 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-24T10:36:33.789 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "smithi094" 2024-03-24T10:36:33.789 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-03-24T10:36:33.790 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-03-24T10:36:33.790 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-24T10:36:33.790 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T10:36:33.790 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-03-24T10:36:33.790 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-24T10:36:33.790 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-03-24T10:36:33.790 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-24T10:36:33.790 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T10:36:33.790 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-24T10:36:33.790 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-24T10:36:33.790 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-24T10:36:33.799 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-24T10:36:33.799 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-24T10:36:33.799 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-24T10:36:33.799 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-03-24T10:36:33.799 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-24T10:36:33.799 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-24T10:36:33.799 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-24T10:36:33.800 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-24T10:36:33.800 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-24T10:36:33.800 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-24T10:36:33.800 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-24T10:36:33.800 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-24T10:36:33.800 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-24T10:36:33.800 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-03-24T10:36:33.800 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-24T10:36:33.800 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T10:36:33.800 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-24T10:36:33.800 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-24T10:36:33.800 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-03-24T10:36:33.800 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-24T10:36:33.800 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-24T10:36:33.800 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-24T10:36:33.801 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-24T10:36:33.801 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-24T10:36:33.801 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-24T10:36:33.801 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "restful" 2024-03-24T10:36:33.801 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-03-24T10:36:33.801 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-24T10:36:33.801 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-03-24T10:36:33.801 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-24T10:36:33.801 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T10:36:33.801 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modified": "2024-03-24T10:36:12.012396+0000", 2024-03-24T10:36:33.801 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-24T10:36:33.801 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-03-24T10:36:33.801 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-24T10:36:33.801 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-03-24T10:36:33.801 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:33.201+0000 7faacfe6e700 1 Processor -- start 2024-03-24T10:36:33.802 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:33.202+0000 7faacfe6e700 1 -- start start 2024-03-24T10:36:33.802 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:33.202+0000 7faacfe6e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faac81028f0 0x7faac8102d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:36:33.802 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:33.202+0000 7faacfe6e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac8103240 con 0x7faac81028f0 2024-03-24T10:36:33.802 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:33.202+0000 7faacdc0a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faac81028f0 0x7faac8102d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:36:33.802 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:33.202+0000 7faacdc0a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faac81028f0 0x7faac8102d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56692/0 (socket says 172.21.15.94:56692) 2024-03-24T10:36:33.802 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:33.202+0000 7faacdc0a700 1 -- 172.21.15.94:0/1695100303 learned_addr learned my addr 172.21.15.94:0/1695100303 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:36:33.802 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:33.202+0000 7faacdc0a700 1 -- 172.21.15.94:0/1695100303 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac8103380 con 0x7faac81028f0 2024-03-24T10:36:33.802 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:33.203+0000 7faacdc0a700 1 --2- 172.21.15.94:0/1695100303 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faac81028f0 0x7faac8102d00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7faab80098e0 tx=0x7faab8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a19e880b5773a726 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:36:33.802 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:33.203+0000 7faaccc08700 1 -- 172.21.15.94:0/1695100303 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faab800b860 con 0x7faac81028f0 2024-03-24T10:36:33.802 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:33.203+0000 7faaccc08700 1 -- 172.21.15.94:0/1695100303 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7faab800be50 con 0x7faac81028f0 2024-03-24T10:36:33.802 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:33.203+0000 7faaccc08700 1 -- 172.21.15.94:0/1695100303 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faab8010380 con 0x7faac81028f0 2024-03-24T10:36:33.803 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:33.204+0000 7faacfe6e700 1 -- 172.21.15.94:0/1695100303 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faac81028f0 msgr2=0x7faac8102d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:36:33.803 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:33.204+0000 7faacfe6e700 1 --2- 172.21.15.94:0/1695100303 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faac81028f0 0x7faac8102d00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7faab80098e0 tx=0x7faab8009250 comp rx=0 tx=0).stop 2024-03-24T10:36:33.803 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:33.204+0000 7faacfe6e700 1 -- 172.21.15.94:0/1695100303 shutdown_connections 2024-03-24T10:36:33.803 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:33.204+0000 7faacfe6e700 1 --2- 172.21.15.94:0/1695100303 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faac81028f0 0x7faac8102d00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:33.803 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:33.204+0000 7faacfe6e700 1 -- 172.21.15.94:0/1695100303 >> 172.21.15.94:0/1695100303 conn(0x7faac80fe070 msgr2=0x7faac8100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:36:33.803 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:33.204+0000 7faacfe6e700 1 -- 172.21.15.94:0/1695100303 shutdown_connections 2024-03-24T10:36:33.803 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:33.204+0000 7faacfe6e700 1 -- 172.21.15.94:0/1695100303 wait complete. 2024-03-24T10:36:33.803 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:33.204+0000 7faacfe6e700 1 Processor -- start 2024-03-24T10:36:33.803 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:33.204+0000 7faacfe6e700 1 -- start start 2024-03-24T10:36:33.803 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:33.204+0000 7faacfe6e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faac81028f0 0x7faac8078c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:36:33.804 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:33.204+0000 7faacfe6e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac8079180 con 0x7faac81028f0 2024-03-24T10:36:33.804 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:33.205+0000 7faacdc0a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faac81028f0 0x7faac8078c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:36:33.804 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:33.205+0000 7faacdc0a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faac81028f0 0x7faac8078c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56700/0 (socket says 172.21.15.94:56700) 2024-03-24T10:36:33.804 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:33.205+0000 7faacdc0a700 1 -- 172.21.15.94:0/3380853142 learned_addr learned my addr 172.21.15.94:0/3380853142 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:36:33.804 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:33.205+0000 7faacdc0a700 1 -- 172.21.15.94:0/3380853142 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab80094f0 con 0x7faac81028f0 2024-03-24T10:36:33.804 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:33.205+0000 7faacdc0a700 1 --2- 172.21.15.94:0/3380853142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faac81028f0 0x7faac8078c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7faab800b440 tx=0x7faab8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:36:33.804 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:33.205+0000 7faabe7fc700 1 -- 172.21.15.94:0/3380853142 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faab8010760 con 0x7faac81028f0 2024-03-24T10:36:33.804 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:33.205+0000 7faabe7fc700 1 -- 172.21.15.94:0/3380853142 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7faab8010d50 con 0x7faac81028f0 2024-03-24T10:36:33.804 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:33.205+0000 7faacfe6e700 1 -- 172.21.15.94:0/3380853142 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faac80771f0 con 0x7faac81028f0 2024-03-24T10:36:33.804 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:33.205+0000 7faabe7fc700 1 -- 172.21.15.94:0/3380853142 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faab801ae20 con 0x7faac81028f0 2024-03-24T10:36:33.804 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:33.205+0000 7faacfe6e700 1 -- 172.21.15.94:0/3380853142 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faac80776e0 con 0x7faac81028f0 2024-03-24T10:36:33.804 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:33.205+0000 7faabe7fc700 1 -- 172.21.15.94:0/3380853142 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7faab802e6c0 con 0x7faac81028f0 2024-03-24T10:36:33.805 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:33.205+0000 7faabe7fc700 1 -- 172.21.15.94:0/3380853142 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7faab8024910 con 0x7faac81028f0 2024-03-24T10:36:33.805 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:33.206+0000 7faacfe6e700 1 -- 172.21.15.94:0/3380853142 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faac804bed0 con 0x7faac81028f0 2024-03-24T10:36:33.805 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:33.208+0000 7faabe7fc700 1 -- 172.21.15.94:0/3380853142 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7faab8029070 con 0x7faac81028f0 2024-03-24T10:36:33.805 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:33.280+0000 7faacfe6e700 1 -- 172.21.15.94:0/3380853142 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7faac8104670 con 0x7faac81028f0 2024-03-24T10:36:33.805 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:33.280+0000 7faabe7fc700 1 -- 172.21.15.94:0/3380853142 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7faab801a6a0 con 0x7faac81028f0 2024-03-24T10:36:33.805 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:33.281+0000 7faacfe6e700 1 -- 172.21.15.94:0/3380853142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faac81028f0 msgr2=0x7faac8078c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:36:33.805 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:33.281+0000 7faacfe6e700 1 --2- 172.21.15.94:0/3380853142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faac81028f0 0x7faac8078c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7faab800b440 tx=0x7faab8009250 comp rx=0 tx=0).stop 2024-03-24T10:36:33.805 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:33.281+0000 7faacfe6e700 1 -- 172.21.15.94:0/3380853142 shutdown_connections 2024-03-24T10:36:33.806 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:33.281+0000 7faacfe6e700 1 --2- 172.21.15.94:0/3380853142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faac81028f0 0x7faac8078c40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:33.806 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:33.281+0000 7faacfe6e700 1 -- 172.21.15.94:0/3380853142 >> 172.21.15.94:0/3380853142 conn(0x7faac80fe070 msgr2=0x7faac80feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:36:33.806 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:33.281+0000 7faacfe6e700 1 -- 172.21.15.94:0/3380853142 shutdown_connections 2024-03-24T10:36:33.806 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:33.281+0000 7faacfe6e700 1 -- 172.21.15.94:0/3380853142 wait complete. 2024-03-24T10:36:33.806 INFO:teuthology.orchestra.run.smithi094.stdout:mgr not available, waiting (2/15)... 2024-03-24T10:36:36.190 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:36 smithi094 ceph-mon[29446]: Activating manager daemon smithi094.yaethu 2024-03-24T10:36:36.190 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:36 smithi094 ceph-mon[29446]: mgrmap e2: smithi094.yaethu(active, starting, since 0.0028306s) 2024-03-24T10:36:36.190 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:36 smithi094 ceph-mon[29446]: from='mgr.14100 172.21.15.94:0/3741353702' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-24T10:36:36.190 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:36 smithi094 ceph-mon[29446]: from='mgr.14100 172.21.15.94:0/3741353702' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-24T10:36:36.190 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:36 smithi094 ceph-mon[29446]: from='mgr.14100 172.21.15.94:0/3741353702' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-24T10:36:36.190 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:36 smithi094 ceph-mon[29446]: from='mgr.14100 172.21.15.94:0/3741353702' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata", "id": "smithi094"}]: dispatch 2024-03-24T10:36:36.190 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:36 smithi094 ceph-mon[29446]: from='mgr.14100 172.21.15.94:0/3741353702' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mgr metadata", "who": "smithi094.yaethu", "id": "smithi094.yaethu"}]: dispatch 2024-03-24T10:36:36.190 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:36 smithi094 ceph-mon[29446]: Manager daemon smithi094.yaethu is now available 2024-03-24T10:36:36.191 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:36 smithi094 ceph-mon[29446]: from='mgr.14100 172.21.15.94:0/3741353702' entity='mgr.smithi094.yaethu' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi094.yaethu/mirror_snapshot_schedule"}]: dispatch 2024-03-24T10:36:36.191 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:36 smithi094 ceph-mon[29446]: from='mgr.14100 172.21.15.94:0/3741353702' entity='mgr.smithi094.yaethu' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi094.yaethu/trash_purge_schedule"}]: dispatch 2024-03-24T10:36:36.191 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:36 smithi094 ceph-mon[29446]: from='mgr.14100 172.21.15.94:0/3741353702' entity='mgr.smithi094.yaethu' 2024-03-24T10:36:36.191 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:36 smithi094 ceph-mon[29446]: from='mgr.14100 172.21.15.94:0/3741353702' entity='mgr.smithi094.yaethu' 2024-03-24T10:36:36.191 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:36 smithi094 ceph-mon[29446]: from='mgr.14100 172.21.15.94:0/3741353702' entity='mgr.smithi094.yaethu' 2024-03-24T10:36:36.971 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-03-24T10:36:36.971 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-03-24T10:36:36.971 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsid": "386110d4-e9ca-11ee-95cd-87774f69a715", 2024-03-24T10:36:36.971 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "health": { 2024-03-24T10:36:36.971 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-24T10:36:36.971 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-24T10:36:36.971 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-24T10:36:36.971 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-03-24T10:36:36.971 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-24T10:36:36.971 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-24T10:36:36.972 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 0 2024-03-24T10:36:36.972 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-03-24T10:36:36.972 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-24T10:36:36.972 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "smithi094" 2024-03-24T10:36:36.972 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-03-24T10:36:36.972 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-03-24T10:36:36.972 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-24T10:36:36.972 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T10:36:36.972 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-03-24T10:36:36.972 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-24T10:36:36.972 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-03-24T10:36:36.972 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-24T10:36:36.972 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T10:36:36.972 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-24T10:36:36.972 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-24T10:36:36.973 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-24T10:36:36.973 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-24T10:36:36.973 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-24T10:36:36.973 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-24T10:36:36.973 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-03-24T10:36:36.973 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-24T10:36:36.973 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-24T10:36:36.973 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-24T10:36:36.973 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-24T10:36:36.973 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-24T10:36:36.973 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-24T10:36:36.973 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-24T10:36:36.973 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-24T10:36:36.973 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-24T10:36:36.973 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-03-24T10:36:36.973 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-24T10:36:36.974 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T10:36:36.974 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-24T10:36:36.974 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-24T10:36:36.974 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-03-24T10:36:36.974 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-24T10:36:36.974 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-24T10:36:36.974 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-24T10:36:36.974 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-24T10:36:36.974 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-24T10:36:36.974 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-24T10:36:36.974 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "restful" 2024-03-24T10:36:36.974 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-03-24T10:36:36.974 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-24T10:36:36.974 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-03-24T10:36:36.974 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-24T10:36:36.975 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T10:36:36.975 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modified": "2024-03-24T10:36:12.012396+0000", 2024-03-24T10:36:36.975 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-24T10:36:36.975 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-03-24T10:36:36.975 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-24T10:36:36.975 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-03-24T10:36:36.975 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:36.280+0000 7eff0176b700 1 Processor -- start 2024-03-24T10:36:36.975 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:36.280+0000 7eff0176b700 1 -- start start 2024-03-24T10:36:36.975 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:36.280+0000 7eff0176b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efefc102920 0x7efefc102d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:36:36.975 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:36.280+0000 7eff0176b700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efefc103270 con 0x7efefc102920 2024-03-24T10:36:36.975 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:36.281+0000 7efefaffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efefc102920 0x7efefc102d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:36:36.975 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:36.281+0000 7efefaffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efefc102920 0x7efefc102d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56780/0 (socket says 172.21.15.94:56780) 2024-03-24T10:36:36.975 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:36.281+0000 7efefaffd700 1 -- 172.21.15.94:0/1280983456 learned_addr learned my addr 172.21.15.94:0/1280983456 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:36:36.975 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:36.281+0000 7efefaffd700 1 -- 172.21.15.94:0/1280983456 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efefc1033b0 con 0x7efefc102920 2024-03-24T10:36:36.975 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:36.281+0000 7efefaffd700 1 --2- 172.21.15.94:0/1280983456 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efefc102920 0x7efefc102d30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7efeec0098e0 tx=0x7efeec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c501cc305e04f071 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:36:36.976 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:36.281+0000 7efef9ffb700 1 -- 172.21.15.94:0/1280983456 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efeec00b860 con 0x7efefc102920 2024-03-24T10:36:36.976 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:36.281+0000 7efef9ffb700 1 -- 172.21.15.94:0/1280983456 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7efeec00be50 con 0x7efefc102920 2024-03-24T10:36:36.976 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:36.281+0000 7efef9ffb700 1 -- 172.21.15.94:0/1280983456 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efeec010380 con 0x7efefc102920 2024-03-24T10:36:36.976 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:36.282+0000 7eff0176b700 1 -- 172.21.15.94:0/1280983456 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efefc102920 msgr2=0x7efefc102d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:36:36.976 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:36.282+0000 7eff0176b700 1 --2- 172.21.15.94:0/1280983456 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efefc102920 0x7efefc102d30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7efeec0098e0 tx=0x7efeec009250 comp rx=0 tx=0).stop 2024-03-24T10:36:36.976 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:36.282+0000 7eff0176b700 1 -- 172.21.15.94:0/1280983456 shutdown_connections 2024-03-24T10:36:36.976 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:36.282+0000 7eff0176b700 1 --2- 172.21.15.94:0/1280983456 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efefc102920 0x7efefc102d30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:36.976 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:36.282+0000 7eff0176b700 1 -- 172.21.15.94:0/1280983456 >> 172.21.15.94:0/1280983456 conn(0x7efefc0fe0a0 msgr2=0x7efefc1004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:36:36.976 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:36.282+0000 7eff0176b700 1 -- 172.21.15.94:0/1280983456 shutdown_connections 2024-03-24T10:36:36.976 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:36.282+0000 7eff0176b700 1 -- 172.21.15.94:0/1280983456 wait complete. 2024-03-24T10:36:36.976 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:36.283+0000 7eff0176b700 1 Processor -- start 2024-03-24T10:36:36.976 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:36.283+0000 7eff0176b700 1 -- start start 2024-03-24T10:36:36.976 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:36.283+0000 7eff0176b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efefc102920 0x7efefc1116a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:36:36.976 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:36.283+0000 7eff0176b700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efefc111be0 con 0x7efefc102920 2024-03-24T10:36:36.976 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:36.283+0000 7efefaffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efefc102920 0x7efefc1116a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:36:36.976 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:36.283+0000 7efefaffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efefc102920 0x7efefc1116a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56794/0 (socket says 172.21.15.94:56794) 2024-03-24T10:36:36.977 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:36.283+0000 7efefaffd700 1 -- 172.21.15.94:0/2920528033 learned_addr learned my addr 172.21.15.94:0/2920528033 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:36:36.977 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:36.283+0000 7efefaffd700 1 -- 172.21.15.94:0/2920528033 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeec0094f0 con 0x7efefc102920 2024-03-24T10:36:36.977 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:36.283+0000 7efefaffd700 1 --2- 172.21.15.94:0/2920528033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efefc102920 0x7efefc1116a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7efeec00b440 tx=0x7efeec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:36:36.977 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:36.283+0000 7efee37fe700 1 -- 172.21.15.94:0/2920528033 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efeec010760 con 0x7efefc102920 2024-03-24T10:36:36.977 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:36.284+0000 7eff0176b700 1 -- 172.21.15.94:0/2920528033 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efefc111de0 con 0x7efefc102920 2024-03-24T10:36:36.977 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:36.284+0000 7efee37fe700 1 -- 172.21.15.94:0/2920528033 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7efeec010d50 con 0x7efefc102920 2024-03-24T10:36:36.977 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:36.284+0000 7efee37fe700 1 -- 172.21.15.94:0/2920528033 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efeec01ae20 con 0x7efefc102920 2024-03-24T10:36:36.977 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:36.284+0000 7eff0176b700 1 -- 172.21.15.94:0/2920528033 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efefc112950 con 0x7efefc102920 2024-03-24T10:36:36.977 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:36.284+0000 7efee37fe700 1 -- 172.21.15.94:0/2920528033 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 2) v1 ==== 43124+0+0 (secure 0 0 0) 0x7efeec02e440 con 0x7efefc102920 2024-03-24T10:36:36.977 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:36.284+0000 7eff0176b700 1 -- 172.21.15.94:0/2920528033 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efefc05f5f0 con 0x7efefc102920 2024-03-24T10:36:36.977 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:36.285+0000 7efee37fe700 1 -- 172.21.15.94:0/2920528033 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7efeec04a0f0 con 0x7efefc102920 2024-03-24T10:36:36.977 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:36.288+0000 7efee37fe700 1 -- 172.21.15.94:0/2920528033 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7efeec018940 con 0x7efefc102920 2024-03-24T10:36:36.977 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:36.360+0000 7eff0176b700 1 -- 172.21.15.94:0/2920528033 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7efefc04bed0 con 0x7efefc102920 2024-03-24T10:36:36.977 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:36.360+0000 7efee37fe700 1 -- 172.21.15.94:0/2920528033 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7efeec018350 con 0x7efefc102920 2024-03-24T10:36:36.977 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:36.361+0000 7eff0176b700 1 -- 172.21.15.94:0/2920528033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efefc102920 msgr2=0x7efefc1116a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:36:36.978 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:36.361+0000 7eff0176b700 1 --2- 172.21.15.94:0/2920528033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efefc102920 0x7efefc1116a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7efeec00b440 tx=0x7efeec009250 comp rx=0 tx=0).stop 2024-03-24T10:36:36.978 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:36.361+0000 7eff0176b700 1 -- 172.21.15.94:0/2920528033 shutdown_connections 2024-03-24T10:36:36.978 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:36.361+0000 7eff0176b700 1 --2- 172.21.15.94:0/2920528033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efefc102920 0x7efefc1116a0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:36.978 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:36.361+0000 7eff0176b700 1 -- 172.21.15.94:0/2920528033 >> 172.21.15.94:0/2920528033 conn(0x7efefc0fe0a0 msgr2=0x7efefc0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:36:36.978 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:36.361+0000 7eff0176b700 1 -- 172.21.15.94:0/2920528033 shutdown_connections 2024-03-24T10:36:36.978 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:36.361+0000 7eff0176b700 1 -- 172.21.15.94:0/2920528033 wait complete. 2024-03-24T10:36:36.978 INFO:teuthology.orchestra.run.smithi094.stdout:mgr not available, waiting (3/15)... 2024-03-24T10:36:37.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:37 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/2920528033' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-24T10:36:37.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:37 smithi094 ceph-mon[29446]: mgrmap e3: smithi094.yaethu(active, since 1.00629s) 2024-03-24T10:36:39.327 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:39 smithi094 ceph-mon[29446]: mgrmap e4: smithi094.yaethu(active, since 2s) 2024-03-24T10:36:40.163 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:40 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/2531350584' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-24T10:36:40.168 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-03-24T10:36:40.168 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-03-24T10:36:40.168 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsid": "386110d4-e9ca-11ee-95cd-87774f69a715", 2024-03-24T10:36:40.168 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "health": { 2024-03-24T10:36:40.168 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-24T10:36:40.168 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-24T10:36:40.168 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-24T10:36:40.168 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-03-24T10:36:40.168 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-24T10:36:40.168 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-24T10:36:40.168 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 0 2024-03-24T10:36:40.168 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-03-24T10:36:40.168 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-24T10:36:40.169 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "smithi094" 2024-03-24T10:36:40.169 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-03-24T10:36:40.169 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_age": 13, 2024-03-24T10:36:40.169 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-24T10:36:40.169 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T10:36:40.169 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-03-24T10:36:40.169 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-24T10:36:40.169 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-03-24T10:36:40.169 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-24T10:36:40.169 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T10:36:40.169 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-24T10:36:40.169 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-24T10:36:40.179 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-24T10:36:40.179 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-24T10:36:40.180 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-24T10:36:40.180 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-24T10:36:40.180 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-03-24T10:36:40.180 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-24T10:36:40.180 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-24T10:36:40.180 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-24T10:36:40.180 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-24T10:36:40.180 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-24T10:36:40.180 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-24T10:36:40.181 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-24T10:36:40.181 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-24T10:36:40.181 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-24T10:36:40.181 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-03-24T10:36:40.181 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-24T10:36:40.181 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T10:36:40.181 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-24T10:36:40.181 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-24T10:36:40.181 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-03-24T10:36:40.182 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-24T10:36:40.182 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-24T10:36:40.182 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-24T10:36:40.182 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-24T10:36:40.182 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-24T10:36:40.182 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-24T10:36:40.182 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "restful" 2024-03-24T10:36:40.182 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-03-24T10:36:40.183 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-24T10:36:40.183 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-03-24T10:36:40.183 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-24T10:36:40.183 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T10:36:40.183 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modified": "2024-03-24T10:36:12.012396+0000", 2024-03-24T10:36:40.184 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-24T10:36:40.184 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-03-24T10:36:40.184 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-24T10:36:40.184 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-03-24T10:36:40.184 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.353+0000 7faee124f700 1 Processor -- start 2024-03-24T10:36:40.184 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.353+0000 7faee124f700 1 -- start start 2024-03-24T10:36:40.184 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.353+0000 7faee124f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faedc078c30 0x7faedc077230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:36:40.184 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.353+0000 7faee124f700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faedc079040 con 0x7faedc078c30 2024-03-24T10:36:40.184 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.353+0000 7faedad9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faedc078c30 0x7faedc077230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:36:40.184 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.353+0000 7faedad9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faedc078c30 0x7faedc077230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56806/0 (socket says 172.21.15.94:56806) 2024-03-24T10:36:40.184 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.353+0000 7faedad9d700 1 -- 172.21.15.94:0/2557419640 learned_addr learned my addr 172.21.15.94:0/2557419640 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:36:40.185 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.353+0000 7faedad9d700 1 -- 172.21.15.94:0/2557419640 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faedc077770 con 0x7faedc078c30 2024-03-24T10:36:40.185 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.353+0000 7faedad9d700 1 --2- 172.21.15.94:0/2557419640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faedc078c30 0x7faedc077230 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7faed00098e0 tx=0x7faed0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=592bc5492c9d05d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:36:40.185 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.354+0000 7faed9d9b700 1 -- 172.21.15.94:0/2557419640 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faed000b860 con 0x7faedc078c30 2024-03-24T10:36:40.185 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.354+0000 7faed9d9b700 1 -- 172.21.15.94:0/2557419640 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7faed000be50 con 0x7faedc078c30 2024-03-24T10:36:40.185 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.354+0000 7faed9d9b700 1 -- 172.21.15.94:0/2557419640 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faed0010380 con 0x7faedc078c30 2024-03-24T10:36:40.185 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.354+0000 7faee124f700 1 -- 172.21.15.94:0/2557419640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faedc078c30 msgr2=0x7faedc077230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:36:40.185 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.354+0000 7faee124f700 1 --2- 172.21.15.94:0/2557419640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faedc078c30 0x7faedc077230 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7faed00098e0 tx=0x7faed0009250 comp rx=0 tx=0).stop 2024-03-24T10:36:40.185 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.354+0000 7faee124f700 1 -- 172.21.15.94:0/2557419640 shutdown_connections 2024-03-24T10:36:40.185 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.354+0000 7faee124f700 1 --2- 172.21.15.94:0/2557419640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faedc078c30 0x7faedc077230 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:40.185 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.354+0000 7faee124f700 1 -- 172.21.15.94:0/2557419640 >> 172.21.15.94:0/2557419640 conn(0x7faedc0fe070 msgr2=0x7faedc100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:36:40.185 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.354+0000 7faee124f700 1 -- 172.21.15.94:0/2557419640 shutdown_connections 2024-03-24T10:36:40.185 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.354+0000 7faee124f700 1 -- 172.21.15.94:0/2557419640 wait complete. 2024-03-24T10:36:40.185 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.355+0000 7faee124f700 1 Processor -- start 2024-03-24T10:36:40.185 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.355+0000 7faee124f700 1 -- start start 2024-03-24T10:36:40.186 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.355+0000 7faee124f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faedc078c30 0x7faedc1139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:36:40.186 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.355+0000 7faee124f700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faedc113ee0 con 0x7faedc078c30 2024-03-24T10:36:40.186 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.355+0000 7faedad9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faedc078c30 0x7faedc1139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:36:40.186 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.355+0000 7faedad9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faedc078c30 0x7faedc1139a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56816/0 (socket says 172.21.15.94:56816) 2024-03-24T10:36:40.186 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.355+0000 7faedad9d700 1 -- 172.21.15.94:0/2531350584 learned_addr learned my addr 172.21.15.94:0/2531350584 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:36:40.186 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.355+0000 7faedad9d700 1 -- 172.21.15.94:0/2531350584 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed00094f0 con 0x7faedc078c30 2024-03-24T10:36:40.186 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.356+0000 7faedad9d700 1 --2- 172.21.15.94:0/2531350584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faedc078c30 0x7faedc1139a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7faed0006fd0 tx=0x7faed0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:36:40.187 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.356+0000 7faec37fe700 1 -- 172.21.15.94:0/2531350584 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faed00107c0 con 0x7faedc078c30 2024-03-24T10:36:40.187 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.356+0000 7faec37fe700 1 -- 172.21.15.94:0/2531350584 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7faed0010db0 con 0x7faedc078c30 2024-03-24T10:36:40.187 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.356+0000 7faee124f700 1 -- 172.21.15.94:0/2531350584 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faedc1140e0 con 0x7faedc078c30 2024-03-24T10:36:40.187 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.356+0000 7faec37fe700 1 -- 172.21.15.94:0/2531350584 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faed001ad40 con 0x7faedc078c30 2024-03-24T10:36:40.187 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.356+0000 7faee124f700 1 -- 172.21.15.94:0/2531350584 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faedc114c50 con 0x7faedc078c30 2024-03-24T10:36:40.187 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.357+0000 7faec37fe700 1 -- 172.21.15.94:0/2531350584 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 4) v1 ==== 43357+0+0 (secure 0 0 0) 0x7faed00193c0 con 0x7faedc078c30 2024-03-24T10:36:40.187 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.357+0000 7faec37fe700 1 --2- 172.21.15.94:0/2531350584 >> [v2:172.21.15.94:6800/1013449112,v1:172.21.15.94:6801/1013449112] conn(0x7faec4036740 0x7faec4038bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:36:40.187 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.357+0000 7faec37fe700 1 -- 172.21.15.94:0/2531350584 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7faed004a250 con 0x7faedc078c30 2024-03-24T10:36:40.187 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.357+0000 7faee124f700 1 -- 172.21.15.94:0/2531350584 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faedc05f5f0 con 0x7faedc078c30 2024-03-24T10:36:40.187 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.357+0000 7faeda59c700 1 --2- 172.21.15.94:0/2531350584 >> [v2:172.21.15.94:6800/1013449112,v1:172.21.15.94:6801/1013449112] conn(0x7faec4036740 0x7faec4038bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:36:40.187 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.357+0000 7faeda59c700 1 --2- 172.21.15.94:0/2531350584 >> [v2:172.21.15.94:6800/1013449112,v1:172.21.15.94:6801/1013449112] conn(0x7faec4036740 0x7faec4038bf0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7faecc009990 tx=0x7faecc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:36:40.187 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.362+0000 7faec37fe700 1 -- 172.21.15.94:0/2531350584 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7faed002c300 con 0x7faedc078c30 2024-03-24T10:36:40.187 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.618+0000 7faee124f700 1 -- 172.21.15.94:0/2531350584 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7faedc114f30 con 0x7faedc078c30 2024-03-24T10:36:40.187 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.619+0000 7faec37fe700 1 -- 172.21.15.94:0/2531350584 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7faed0024b60 con 0x7faedc078c30 2024-03-24T10:36:40.188 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.622+0000 7faee124f700 1 -- 172.21.15.94:0/2531350584 >> [v2:172.21.15.94:6800/1013449112,v1:172.21.15.94:6801/1013449112] conn(0x7faec4036740 msgr2=0x7faec4038bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:36:40.188 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.622+0000 7faee124f700 1 --2- 172.21.15.94:0/2531350584 >> [v2:172.21.15.94:6800/1013449112,v1:172.21.15.94:6801/1013449112] conn(0x7faec4036740 0x7faec4038bf0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7faecc009990 tx=0x7faecc006e40 comp rx=0 tx=0).stop 2024-03-24T10:36:40.188 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.622+0000 7faee124f700 1 -- 172.21.15.94:0/2531350584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faedc078c30 msgr2=0x7faedc1139a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:36:40.188 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.622+0000 7faee124f700 1 --2- 172.21.15.94:0/2531350584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faedc078c30 0x7faedc1139a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7faed0006fd0 tx=0x7faed0009250 comp rx=0 tx=0).stop 2024-03-24T10:36:40.188 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.622+0000 7faee124f700 1 -- 172.21.15.94:0/2531350584 shutdown_connections 2024-03-24T10:36:40.188 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.622+0000 7faee124f700 1 --2- 172.21.15.94:0/2531350584 >> [v2:172.21.15.94:6800/1013449112,v1:172.21.15.94:6801/1013449112] conn(0x7faec4036740 0x7faec4038bf0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:40.188 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.622+0000 7faee124f700 1 --2- 172.21.15.94:0/2531350584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faedc078c30 0x7faedc1139a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:40.188 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.622+0000 7faee124f700 1 -- 172.21.15.94:0/2531350584 >> 172.21.15.94:0/2531350584 conn(0x7faedc0fe070 msgr2=0x7faedc0fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:36:40.188 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.622+0000 7faee124f700 1 -- 172.21.15.94:0/2531350584 shutdown_connections 2024-03-24T10:36:40.188 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:39.622+0000 7faee124f700 1 -- 172.21.15.94:0/2531350584 wait complete. 2024-03-24T10:36:40.188 INFO:teuthology.orchestra.run.smithi094.stdout:mgr is available 2024-03-24T10:36:41.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:41 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/1349379696' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-24T10:36:41.351 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-03-24T10:36:41.351 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout [global] 2024-03-24T10:36:41.351 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout fsid = 386110d4-e9ca-11ee-95cd-87774f69a715 2024-03-24T10:36:41.351 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-24T10:36:41.352 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-24T10:36:41.352 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-24T10:36:41.352 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-24T10:36:41.352 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-03-24T10:36:41.352 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-24T10:36:41.352 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-24T10:36:41.352 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-03-24T10:36:41.352 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout [osd] 2024-03-24T10:36:41.352 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-24T10:36:41.352 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-24T10:36:41.352 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-24T10:36:41.353 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-24T10:36:41.353 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-24T10:36:41.353 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.593+0000 7f51a3d7a700 1 Processor -- start 2024-03-24T10:36:41.353 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.593+0000 7f51a3d7a700 1 -- start start 2024-03-24T10:36:41.353 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.593+0000 7f51a3d7a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f519c102920 0x7f519c102d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:36:41.353 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.593+0000 7f51a3d7a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519c103270 con 0x7f519c102920 2024-03-24T10:36:41.353 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.593+0000 7f51a1b16700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f519c102920 0x7f519c102d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:36:41.353 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.593+0000 7f51a1b16700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f519c102920 0x7f519c102d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56824/0 (socket says 172.21.15.94:56824) 2024-03-24T10:36:41.353 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.593+0000 7f51a1b16700 1 -- 172.21.15.94:0/3197243680 learned_addr learned my addr 172.21.15.94:0/3197243680 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:36:41.353 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.593+0000 7f51a1b16700 1 -- 172.21.15.94:0/3197243680 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f519c1033b0 con 0x7f519c102920 2024-03-24T10:36:41.353 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.594+0000 7f51a1b16700 1 --2- 172.21.15.94:0/3197243680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f519c102920 0x7f519c102d30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f51980098e0 tx=0x7f5198009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e81b231c7ad37bf5 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:36:41.353 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.594+0000 7f51a0b14700 1 -- 172.21.15.94:0/3197243680 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f519800b860 con 0x7f519c102920 2024-03-24T10:36:41.353 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.594+0000 7f51a0b14700 1 -- 172.21.15.94:0/3197243680 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f519800be50 con 0x7f519c102920 2024-03-24T10:36:41.353 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.594+0000 7f51a0b14700 1 -- 172.21.15.94:0/3197243680 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5198010380 con 0x7f519c102920 2024-03-24T10:36:41.354 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.594+0000 7f51a3d7a700 1 -- 172.21.15.94:0/3197243680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f519c102920 msgr2=0x7f519c102d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:36:41.354 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.594+0000 7f51a3d7a700 1 --2- 172.21.15.94:0/3197243680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f519c102920 0x7f519c102d30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f51980098e0 tx=0x7f5198009250 comp rx=0 tx=0).stop 2024-03-24T10:36:41.354 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.594+0000 7f51a3d7a700 1 -- 172.21.15.94:0/3197243680 shutdown_connections 2024-03-24T10:36:41.354 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.594+0000 7f51a3d7a700 1 --2- 172.21.15.94:0/3197243680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f519c102920 0x7f519c102d30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:41.354 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.594+0000 7f51a3d7a700 1 -- 172.21.15.94:0/3197243680 >> 172.21.15.94:0/3197243680 conn(0x7f519c0fe0a0 msgr2=0x7f519c1004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:36:41.354 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.594+0000 7f51a3d7a700 1 -- 172.21.15.94:0/3197243680 shutdown_connections 2024-03-24T10:36:41.354 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.594+0000 7f51a3d7a700 1 -- 172.21.15.94:0/3197243680 wait complete. 2024-03-24T10:36:41.354 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.595+0000 7f51a3d7a700 1 Processor -- start 2024-03-24T10:36:41.354 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.595+0000 7f51a3d7a700 1 -- start start 2024-03-24T10:36:41.354 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.595+0000 7f51a3d7a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f519c102920 0x7f519c1116a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:36:41.355 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.595+0000 7f51a3d7a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519c111be0 con 0x7f519c102920 2024-03-24T10:36:41.355 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.595+0000 7f51a1b16700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f519c102920 0x7f519c1116a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:36:41.355 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.595+0000 7f51a1b16700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f519c102920 0x7f519c1116a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56828/0 (socket says 172.21.15.94:56828) 2024-03-24T10:36:41.355 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.595+0000 7f51a1b16700 1 -- 172.21.15.94:0/1349379696 learned_addr learned my addr 172.21.15.94:0/1349379696 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:36:41.355 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.595+0000 7f51a1b16700 1 -- 172.21.15.94:0/1349379696 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51980094f0 con 0x7f519c102920 2024-03-24T10:36:41.355 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.596+0000 7f51a1b16700 1 --2- 172.21.15.94:0/1349379696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f519c102920 0x7f519c1116a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5198000c00 tx=0x7f5198009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:36:41.356 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.596+0000 7f51927fc700 1 -- 172.21.15.94:0/1349379696 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f51980107c0 con 0x7f519c102920 2024-03-24T10:36:41.356 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.596+0000 7f51927fc700 1 -- 172.21.15.94:0/1349379696 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f5198010db0 con 0x7f519c102920 2024-03-24T10:36:41.356 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.596+0000 7f51a3d7a700 1 -- 172.21.15.94:0/1349379696 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f519c111de0 con 0x7f519c102920 2024-03-24T10:36:41.356 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.596+0000 7f51927fc700 1 -- 172.21.15.94:0/1349379696 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f519801ad00 con 0x7f519c102920 2024-03-24T10:36:41.356 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.596+0000 7f51a3d7a700 1 -- 172.21.15.94:0/1349379696 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f519c112950 con 0x7f519c102920 2024-03-24T10:36:41.356 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.597+0000 7f51927fc700 1 -- 172.21.15.94:0/1349379696 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 4) v1 ==== 43357+0+0 (secure 0 0 0) 0x7f51980193c0 con 0x7f519c102920 2024-03-24T10:36:41.356 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.597+0000 7f51927fc700 1 --2- 172.21.15.94:0/1349379696 >> [v2:172.21.15.94:6800/1013449112,v1:172.21.15.94:6801/1013449112] conn(0x7f5188036790 0x7f5188038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:36:41.356 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.597+0000 7f51927fc700 1 -- 172.21.15.94:0/1349379696 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f519804a5b0 con 0x7f519c102920 2024-03-24T10:36:41.356 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.597+0000 7f51a3d7a700 1 -- 172.21.15.94:0/1349379696 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f519c05f5f0 con 0x7f519c102920 2024-03-24T10:36:41.357 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.597+0000 7f51a1315700 1 --2- 172.21.15.94:0/1349379696 >> [v2:172.21.15.94:6800/1013449112,v1:172.21.15.94:6801/1013449112] conn(0x7f5188036790 0x7f5188038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:36:41.357 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.597+0000 7f51a1315700 1 --2- 172.21.15.94:0/1349379696 >> [v2:172.21.15.94:6800/1013449112,v1:172.21.15.94:6801/1013449112] conn(0x7f5188036790 0x7f5188038c40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f518c009990 tx=0x7f518c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:36:41.357 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.601+0000 7f51927fc700 1 -- 172.21.15.94:0/1349379696 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f5198019670 con 0x7f519c102920 2024-03-24T10:36:41.357 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.784+0000 7f51a3d7a700 1 -- 172.21.15.94:0/1349379696 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f519c04bed0 con 0x7f519c102920 2024-03-24T10:36:41.357 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.785+0000 7f51927fc700 1 -- 172.21.15.94:0/1349379696 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f5198022340 con 0x7f519c102920 2024-03-24T10:36:41.357 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.787+0000 7f51a3d7a700 1 -- 172.21.15.94:0/1349379696 >> [v2:172.21.15.94:6800/1013449112,v1:172.21.15.94:6801/1013449112] conn(0x7f5188036790 msgr2=0x7f5188038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:36:41.357 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.787+0000 7f51a3d7a700 1 --2- 172.21.15.94:0/1349379696 >> [v2:172.21.15.94:6800/1013449112,v1:172.21.15.94:6801/1013449112] conn(0x7f5188036790 0x7f5188038c40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f518c009990 tx=0x7f518c006e40 comp rx=0 tx=0).stop 2024-03-24T10:36:41.357 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.787+0000 7f51a3d7a700 1 -- 172.21.15.94:0/1349379696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f519c102920 msgr2=0x7f519c1116a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:36:41.358 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.787+0000 7f51a3d7a700 1 --2- 172.21.15.94:0/1349379696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f519c102920 0x7f519c1116a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5198000c00 tx=0x7f5198009250 comp rx=0 tx=0).stop 2024-03-24T10:36:41.358 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.788+0000 7f51a3d7a700 1 -- 172.21.15.94:0/1349379696 shutdown_connections 2024-03-24T10:36:41.358 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.788+0000 7f51a3d7a700 1 --2- 172.21.15.94:0/1349379696 >> [v2:172.21.15.94:6800/1013449112,v1:172.21.15.94:6801/1013449112] conn(0x7f5188036790 0x7f5188038c40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:41.358 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.788+0000 7f51a3d7a700 1 --2- 172.21.15.94:0/1349379696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f519c102920 0x7f519c1116a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:41.358 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.788+0000 7f51a3d7a700 1 -- 172.21.15.94:0/1349379696 >> 172.21.15.94:0/1349379696 conn(0x7f519c0fe0a0 msgr2=0x7f519c0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:36:41.358 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.788+0000 7f51a3d7a700 1 -- 172.21.15.94:0/1349379696 shutdown_connections 2024-03-24T10:36:41.358 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:40.788+0000 7f51a3d7a700 1 -- 172.21.15.94:0/1349379696 wait complete. 2024-03-24T10:36:41.358 INFO:teuthology.orchestra.run.smithi094.stdout:Enabling cephadm module... 2024-03-24T10:36:42.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:42 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/3662547557' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-24T10:36:42.575 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:41.719+0000 7f8afb7b3700 1 Processor -- start 2024-03-24T10:36:42.576 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:41.719+0000 7f8afb7b3700 1 -- start start 2024-03-24T10:36:42.576 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:41.719+0000 7f8afb7b3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8af4102910 0x7f8af4102d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:36:42.576 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:41.719+0000 7f8afb7b3700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af4103260 con 0x7f8af4102910 2024-03-24T10:36:42.576 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:41.719+0000 7f8af954f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8af4102910 0x7f8af4102d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:36:42.576 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:41.719+0000 7f8af954f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8af4102910 0x7f8af4102d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56830/0 (socket says 172.21.15.94:56830) 2024-03-24T10:36:42.576 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:41.719+0000 7f8af954f700 1 -- 172.21.15.94:0/3798991920 learned_addr learned my addr 172.21.15.94:0/3798991920 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:36:42.576 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:41.719+0000 7f8af954f700 1 -- 172.21.15.94:0/3798991920 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af41033a0 con 0x7f8af4102910 2024-03-24T10:36:42.576 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:41.720+0000 7f8af954f700 1 --2- 172.21.15.94:0/3798991920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8af4102910 0x7f8af4102d20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8af00098e0 tx=0x7f8af0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40b4899c969c25b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:36:42.576 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:41.720+0000 7f8aebfff700 1 -- 172.21.15.94:0/3798991920 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8af000b860 con 0x7f8af4102910 2024-03-24T10:36:42.576 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:41.720+0000 7f8aebfff700 1 -- 172.21.15.94:0/3798991920 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f8af000be50 con 0x7f8af4102910 2024-03-24T10:36:42.576 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:41.720+0000 7f8aebfff700 1 -- 172.21.15.94:0/3798991920 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8af0010380 con 0x7f8af4102910 2024-03-24T10:36:42.576 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:41.720+0000 7f8afb7b3700 1 -- 172.21.15.94:0/3798991920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8af4102910 msgr2=0x7f8af4102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:36:42.576 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:41.720+0000 7f8afb7b3700 1 --2- 172.21.15.94:0/3798991920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8af4102910 0x7f8af4102d20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8af00098e0 tx=0x7f8af0009250 comp rx=0 tx=0).stop 2024-03-24T10:36:42.577 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:41.720+0000 7f8afb7b3700 1 -- 172.21.15.94:0/3798991920 shutdown_connections 2024-03-24T10:36:42.577 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:41.720+0000 7f8afb7b3700 1 --2- 172.21.15.94:0/3798991920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8af4102910 0x7f8af4102d20 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:42.577 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:41.720+0000 7f8afb7b3700 1 -- 172.21.15.94:0/3798991920 >> 172.21.15.94:0/3798991920 conn(0x7f8af40fe070 msgr2=0x7f8af41004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:36:42.577 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:41.720+0000 7f8afb7b3700 1 -- 172.21.15.94:0/3798991920 shutdown_connections 2024-03-24T10:36:42.577 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:41.720+0000 7f8afb7b3700 1 -- 172.21.15.94:0/3798991920 wait complete. 2024-03-24T10:36:42.577 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:41.721+0000 7f8afb7b3700 1 Processor -- start 2024-03-24T10:36:42.577 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:41.721+0000 7f8afb7b3700 1 -- start start 2024-03-24T10:36:42.577 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:41.721+0000 7f8afb7b3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8af4102910 0x7f8af41116d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:36:42.577 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:41.721+0000 7f8afb7b3700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af4111c10 con 0x7f8af4102910 2024-03-24T10:36:42.577 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:41.721+0000 7f8af954f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8af4102910 0x7f8af41116d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:36:42.577 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:41.721+0000 7f8af954f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8af4102910 0x7f8af41116d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56842/0 (socket says 172.21.15.94:56842) 2024-03-24T10:36:42.577 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:41.721+0000 7f8af954f700 1 -- 172.21.15.94:0/3662547557 learned_addr learned my addr 172.21.15.94:0/3662547557 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:36:42.577 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:41.721+0000 7f8af954f700 1 -- 172.21.15.94:0/3662547557 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af00094f0 con 0x7f8af4102910 2024-03-24T10:36:42.577 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:41.722+0000 7f8af954f700 1 --2- 172.21.15.94:0/3662547557 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8af4102910 0x7f8af41116d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8af000b440 tx=0x7f8af0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:36:42.577 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:41.722+0000 7f8ae9ffb700 1 -- 172.21.15.94:0/3662547557 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8af0010760 con 0x7f8af4102910 2024-03-24T10:36:42.577 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:41.722+0000 7f8ae9ffb700 1 -- 172.21.15.94:0/3662547557 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f8af0010d50 con 0x7f8af4102910 2024-03-24T10:36:42.578 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:41.722+0000 7f8afb7b3700 1 -- 172.21.15.94:0/3662547557 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8af4111e10 con 0x7f8af4102910 2024-03-24T10:36:42.578 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:41.722+0000 7f8ae9ffb700 1 -- 172.21.15.94:0/3662547557 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8af001ae20 con 0x7f8af4102910 2024-03-24T10:36:42.578 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:41.722+0000 7f8afb7b3700 1 -- 172.21.15.94:0/3662547557 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8af4112980 con 0x7f8af4102910 2024-03-24T10:36:42.578 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:41.723+0000 7f8ae9ffb700 1 -- 172.21.15.94:0/3662547557 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 4) v1 ==== 43357+0+0 (secure 0 0 0) 0x7f8af002e440 con 0x7f8af4102910 2024-03-24T10:36:42.578 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:41.723+0000 7f8ae9ffb700 1 --2- 172.21.15.94:0/3662547557 >> [v2:172.21.15.94:6800/1013449112,v1:172.21.15.94:6801/1013449112] conn(0x7f8ae0036790 0x7f8ae0038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:36:42.578 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:41.723+0000 7f8ae9ffb700 1 -- 172.21.15.94:0/3662547557 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f8af004a690 con 0x7f8af4102910 2024-03-24T10:36:42.578 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:41.723+0000 7f8afb7b3700 1 -- 172.21.15.94:0/3662547557 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8af405f5f0 con 0x7f8af4102910 2024-03-24T10:36:42.578 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:41.723+0000 7f8af8d4e700 1 --2- 172.21.15.94:0/3662547557 >> [v2:172.21.15.94:6800/1013449112,v1:172.21.15.94:6801/1013449112] conn(0x7f8ae0036790 0x7f8ae0038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:36:42.578 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:41.723+0000 7f8af8d4e700 1 --2- 172.21.15.94:0/3662547557 >> [v2:172.21.15.94:6800/1013449112,v1:172.21.15.94:6801/1013449112] conn(0x7f8ae0036790 0x7f8ae0038c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4009990 tx=0x7f8ae4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:36:42.578 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:41.728+0000 7f8ae9ffb700 1 -- 172.21.15.94:0/3662547557 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f8af0018940 con 0x7f8af4102910 2024-03-24T10:36:42.578 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:41.960+0000 7f8afb7b3700 1 -- 172.21.15.94:0/3662547557 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f8af4112bc0 con 0x7f8af4102910 2024-03-24T10:36:42.578 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.010+0000 7f8ae9ffb700 1 -- 172.21.15.94:0/3662547557 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 5) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f8af002e740 con 0x7f8af4102910 2024-03-24T10:36:42.578 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.010+0000 7f8ae9ffb700 1 -- 172.21.15.94:0/3662547557 <== mon.0 v2:172.21.15.94:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f8af0049880 con 0x7f8af4102910 2024-03-24T10:36:42.578 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.013+0000 7f8afb7b3700 1 -- 172.21.15.94:0/3662547557 >> [v2:172.21.15.94:6800/1013449112,v1:172.21.15.94:6801/1013449112] conn(0x7f8ae0036790 msgr2=0x7f8ae0038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:36:42.579 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.013+0000 7f8afb7b3700 1 --2- 172.21.15.94:0/3662547557 >> [v2:172.21.15.94:6800/1013449112,v1:172.21.15.94:6801/1013449112] conn(0x7f8ae0036790 0x7f8ae0038c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4009990 tx=0x7f8ae4006e40 comp rx=0 tx=0).stop 2024-03-24T10:36:42.579 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.013+0000 7f8afb7b3700 1 -- 172.21.15.94:0/3662547557 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8af4102910 msgr2=0x7f8af41116d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:36:42.579 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.013+0000 7f8afb7b3700 1 --2- 172.21.15.94:0/3662547557 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8af4102910 0x7f8af41116d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8af000b440 tx=0x7f8af0009250 comp rx=0 tx=0).stop 2024-03-24T10:36:42.579 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.013+0000 7f8afb7b3700 1 -- 172.21.15.94:0/3662547557 shutdown_connections 2024-03-24T10:36:42.579 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.013+0000 7f8afb7b3700 1 --2- 172.21.15.94:0/3662547557 >> [v2:172.21.15.94:6800/1013449112,v1:172.21.15.94:6801/1013449112] conn(0x7f8ae0036790 0x7f8ae0038c40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:42.579 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.013+0000 7f8afb7b3700 1 --2- 172.21.15.94:0/3662547557 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8af4102910 0x7f8af41116d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:42.579 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.013+0000 7f8afb7b3700 1 -- 172.21.15.94:0/3662547557 >> 172.21.15.94:0/3662547557 conn(0x7f8af40fe070 msgr2=0x7f8af40feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:36:42.579 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.013+0000 7f8afb7b3700 1 -- 172.21.15.94:0/3662547557 shutdown_connections 2024-03-24T10:36:42.579 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.013+0000 7f8afb7b3700 1 -- 172.21.15.94:0/3662547557 wait complete. 2024-03-24T10:36:43.220 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:43 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/3662547557' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-24T10:36:43.220 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:43 smithi094 ceph-mon[29446]: mgrmap e5: smithi094.yaethu(active, since 6s) 2024-03-24T10:36:43.732 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-03-24T10:36:43.732 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-24T10:36:43.732 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-24T10:36:43.732 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "active_name": "smithi094.yaethu", 2024-03-24T10:36:43.732 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-24T10:36:43.732 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-03-24T10:36:43.732 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.965+0000 7f08afaff700 1 Processor -- start 2024-03-24T10:36:43.732 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.965+0000 7f08afaff700 1 -- start start 2024-03-24T10:36:43.732 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.965+0000 7f08afaff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08a8100c80 0x7f08a8101090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:36:43.733 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.965+0000 7f08afaff700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a81015d0 con 0x7f08a8100c80 2024-03-24T10:36:43.733 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.965+0000 7f08ad89b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08a8100c80 0x7f08a8101090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:36:43.733 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.966+0000 7f08ad89b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08a8100c80 0x7f08a8101090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56862/0 (socket says 172.21.15.94:56862) 2024-03-24T10:36:43.733 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.966+0000 7f08ad89b700 1 -- 172.21.15.94:0/3497812583 learned_addr learned my addr 172.21.15.94:0/3497812583 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:36:43.733 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.966+0000 7f08ad89b700 1 -- 172.21.15.94:0/3497812583 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a8101710 con 0x7f08a8100c80 2024-03-24T10:36:43.733 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.966+0000 7f08ad89b700 1 --2- 172.21.15.94:0/3497812583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08a8100c80 0x7f08a8101090 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f08a40098e0 tx=0x7f08a4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5d4f2e307dc2cea server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:36:43.733 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.966+0000 7f08ac899700 1 -- 172.21.15.94:0/3497812583 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f08a400b860 con 0x7f08a8100c80 2024-03-24T10:36:43.733 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.966+0000 7f08ac899700 1 -- 172.21.15.94:0/3497812583 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f08a400be50 con 0x7f08a8100c80 2024-03-24T10:36:43.733 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.966+0000 7f08ac899700 1 -- 172.21.15.94:0/3497812583 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f08a4010380 con 0x7f08a8100c80 2024-03-24T10:36:43.733 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.967+0000 7f08afaff700 1 -- 172.21.15.94:0/3497812583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08a8100c80 msgr2=0x7f08a8101090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:36:43.733 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.967+0000 7f08afaff700 1 --2- 172.21.15.94:0/3497812583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08a8100c80 0x7f08a8101090 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f08a40098e0 tx=0x7f08a4009250 comp rx=0 tx=0).stop 2024-03-24T10:36:43.733 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.967+0000 7f08afaff700 1 -- 172.21.15.94:0/3497812583 shutdown_connections 2024-03-24T10:36:43.734 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.967+0000 7f08afaff700 1 --2- 172.21.15.94:0/3497812583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08a8100c80 0x7f08a8101090 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:43.734 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.967+0000 7f08afaff700 1 -- 172.21.15.94:0/3497812583 >> 172.21.15.94:0/3497812583 conn(0x7f08a80fc700 msgr2=0x7f08a80feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:36:43.734 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.967+0000 7f08afaff700 1 -- 172.21.15.94:0/3497812583 shutdown_connections 2024-03-24T10:36:43.734 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.967+0000 7f08afaff700 1 -- 172.21.15.94:0/3497812583 wait complete. 2024-03-24T10:36:43.734 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.967+0000 7f08afaff700 1 Processor -- start 2024-03-24T10:36:43.734 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.968+0000 7f08afaff700 1 -- start start 2024-03-24T10:36:43.734 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.968+0000 7f08afaff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08a8100c80 0x7f08a8111680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:36:43.734 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.968+0000 7f08afaff700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a8111bc0 con 0x7f08a8100c80 2024-03-24T10:36:43.734 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.968+0000 7f08ad89b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08a8100c80 0x7f08a8111680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:36:43.734 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.968+0000 7f08ad89b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08a8100c80 0x7f08a8111680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56864/0 (socket says 172.21.15.94:56864) 2024-03-24T10:36:43.734 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.968+0000 7f08ad89b700 1 -- 172.21.15.94:0/905114045 learned_addr learned my addr 172.21.15.94:0/905114045 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:36:43.734 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.968+0000 7f08ad89b700 1 -- 172.21.15.94:0/905114045 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a40094f0 con 0x7f08a8100c80 2024-03-24T10:36:43.735 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.968+0000 7f08ad89b700 1 --2- 172.21.15.94:0/905114045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08a8100c80 0x7f08a8111680 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f08a4000c00 tx=0x7f08a4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:36:43.735 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.968+0000 7f089e7fc700 1 -- 172.21.15.94:0/905114045 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f08a40107c0 con 0x7f08a8100c80 2024-03-24T10:36:43.735 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.968+0000 7f089e7fc700 1 -- 172.21.15.94:0/905114045 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f08a4010db0 con 0x7f08a8100c80 2024-03-24T10:36:43.735 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.968+0000 7f089e7fc700 1 -- 172.21.15.94:0/905114045 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f08a401ad00 con 0x7f08a8100c80 2024-03-24T10:36:43.735 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.969+0000 7f08afaff700 1 -- 172.21.15.94:0/905114045 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08a8111dc0 con 0x7f08a8100c80 2024-03-24T10:36:43.735 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.969+0000 7f08afaff700 1 -- 172.21.15.94:0/905114045 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08a8112930 con 0x7f08a8100c80 2024-03-24T10:36:43.735 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.969+0000 7f089e7fc700 1 -- 172.21.15.94:0/905114045 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f08a40193c0 con 0x7f08a8100c80 2024-03-24T10:36:43.735 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.969+0000 7f08afaff700 1 -- 172.21.15.94:0/905114045 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08a805f5f0 con 0x7f08a8100c80 2024-03-24T10:36:43.735 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.970+0000 7f089e7fc700 1 --2- 172.21.15.94:0/905114045 >> [v2:172.21.15.94:6800/1013449112,v1:172.21.15.94:6801/1013449112] conn(0x7f08940367e0 0x7f0894038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:36:43.735 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.970+0000 7f089e7fc700 1 -- 172.21.15.94:0/905114045 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f08a404a650 con 0x7f08a8100c80 2024-03-24T10:36:43.735 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.970+0000 7f08ad09a700 1 -- 172.21.15.94:0/905114045 >> [v2:172.21.15.94:6800/1013449112,v1:172.21.15.94:6801/1013449112] conn(0x7f08940367e0 msgr2=0x7f0894038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/1013449112 2024-03-24T10:36:43.735 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.970+0000 7f08ad09a700 1 --2- 172.21.15.94:0/905114045 >> [v2:172.21.15.94:6800/1013449112,v1:172.21.15.94:6801/1013449112] conn(0x7f08940367e0 0x7f0894038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-24T10:36:43.735 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:42.974+0000 7f089e7fc700 1 -- 172.21.15.94:0/905114045 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f08a4022930 con 0x7f08a8100c80 2024-03-24T10:36:43.735 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:43.171+0000 7f08ad09a700 1 -- 172.21.15.94:0/905114045 >> [v2:172.21.15.94:6800/1013449112,v1:172.21.15.94:6801/1013449112] conn(0x7f08940367e0 msgr2=0x7f0894038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/1013449112 2024-03-24T10:36:43.735 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:43.171+0000 7f08ad09a700 1 --2- 172.21.15.94:0/905114045 >> [v2:172.21.15.94:6800/1013449112,v1:172.21.15.94:6801/1013449112] conn(0x7f08940367e0 0x7f0894038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-24T10:36:43.736 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:43.220+0000 7f08afaff700 1 -- 172.21.15.94:0/905114045 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f08a8112c80 con 0x7f08a8100c80 2024-03-24T10:36:43.736 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:43.220+0000 7f089e7fc700 1 -- 172.21.15.94:0/905114045 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7f08a4022340 con 0x7f08a8100c80 2024-03-24T10:36:43.736 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:43.222+0000 7f08afaff700 1 -- 172.21.15.94:0/905114045 >> [v2:172.21.15.94:6800/1013449112,v1:172.21.15.94:6801/1013449112] conn(0x7f08940367e0 msgr2=0x7f0894038c90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-24T10:36:43.736 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:43.222+0000 7f08afaff700 1 --2- 172.21.15.94:0/905114045 >> [v2:172.21.15.94:6800/1013449112,v1:172.21.15.94:6801/1013449112] conn(0x7f08940367e0 0x7f0894038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:43.736 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:43.222+0000 7f08afaff700 1 -- 172.21.15.94:0/905114045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08a8100c80 msgr2=0x7f08a8111680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:36:43.736 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:43.222+0000 7f08afaff700 1 --2- 172.21.15.94:0/905114045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08a8100c80 0x7f08a8111680 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f08a4000c00 tx=0x7f08a4009250 comp rx=0 tx=0).stop 2024-03-24T10:36:43.736 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:43.222+0000 7f08afaff700 1 -- 172.21.15.94:0/905114045 shutdown_connections 2024-03-24T10:36:43.736 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:43.222+0000 7f08afaff700 1 --2- 172.21.15.94:0/905114045 >> [v2:172.21.15.94:6800/1013449112,v1:172.21.15.94:6801/1013449112] conn(0x7f08940367e0 0x7f0894038c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:43.736 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:43.222+0000 7f08afaff700 1 --2- 172.21.15.94:0/905114045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08a8100c80 0x7f08a8111680 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:43.736 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:43.222+0000 7f08afaff700 1 -- 172.21.15.94:0/905114045 >> 172.21.15.94:0/905114045 conn(0x7f08a80fc700 msgr2=0x7f08a80fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:36:43.736 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:43.223+0000 7f08afaff700 1 -- 172.21.15.94:0/905114045 shutdown_connections 2024-03-24T10:36:43.736 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:43.223+0000 7f08afaff700 1 -- 172.21.15.94:0/905114045 wait complete. 2024-03-24T10:36:43.736 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for the mgr to restart... 2024-03-24T10:36:43.736 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for mgr epoch 5... 2024-03-24T10:36:44.291 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:44 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/905114045' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-24T10:36:48.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:48 smithi094 ceph-mon[29446]: Active manager daemon smithi094.yaethu restarted 2024-03-24T10:36:48.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:48 smithi094 ceph-mon[29446]: Activating manager daemon smithi094.yaethu 2024-03-24T10:36:48.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:48 smithi094 ceph-mon[29446]: osdmap e2: 0 total, 0 up, 0 in 2024-03-24T10:36:48.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:48 smithi094 ceph-mon[29446]: mgrmap e6: smithi094.yaethu(active, starting, since 0.00329339s) 2024-03-24T10:36:48.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:48 smithi094 ceph-mon[29446]: from='mgr.14120 172.21.15.94:0/773982920' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata", "id": "smithi094"}]: dispatch 2024-03-24T10:36:48.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:48 smithi094 ceph-mon[29446]: from='mgr.14120 172.21.15.94:0/773982920' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mgr metadata", "who": "smithi094.yaethu", "id": "smithi094.yaethu"}]: dispatch 2024-03-24T10:36:48.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:48 smithi094 ceph-mon[29446]: from='mgr.14120 172.21.15.94:0/773982920' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-24T10:36:48.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:48 smithi094 ceph-mon[29446]: from='mgr.14120 172.21.15.94:0/773982920' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-24T10:36:48.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:48 smithi094 ceph-mon[29446]: from='mgr.14120 172.21.15.94:0/773982920' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-24T10:36:48.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:48 smithi094 ceph-mon[29446]: Manager daemon smithi094.yaethu is now available 2024-03-24T10:36:48.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:48 smithi094 ceph-mon[29446]: from='mgr.14120 172.21.15.94:0/773982920' entity='mgr.smithi094.yaethu' 2024-03-24T10:36:48.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:48 smithi094 ceph-mon[29446]: from='mgr.14120 172.21.15.94:0/773982920' entity='mgr.smithi094.yaethu' 2024-03-24T10:36:48.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:48 smithi094 ceph-mon[29446]: from='mgr.14120 172.21.15.94:0/773982920' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:36:48.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:48 smithi094 ceph-mon[29446]: from='mgr.14120 172.21.15.94:0/773982920' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:36:48.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:48 smithi094 ceph-mon[29446]: from='mgr.14120 172.21.15.94:0/773982920' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:36:49.746 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:49 smithi094 ceph-mon[29446]: Found migration_current of "None". Setting to last migration. 2024-03-24T10:36:49.746 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:49 smithi094 ceph-mon[29446]: from='mgr.14120 172.21.15.94:0/773982920' entity='mgr.smithi094.yaethu' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi094.yaethu/mirror_snapshot_schedule"}]: dispatch 2024-03-24T10:36:49.746 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:49 smithi094 ceph-mon[29446]: from='mgr.14120 172.21.15.94:0/773982920' entity='mgr.smithi094.yaethu' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi094.yaethu/trash_purge_schedule"}]: dispatch 2024-03-24T10:36:49.746 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:49 smithi094 ceph-mon[29446]: mgrmap e7: smithi094.yaethu(active, since 1.00533s) 2024-03-24T10:36:50.015 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-03-24T10:36:50.015 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-24T10:36:50.015 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-24T10:36:50.015 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-03-24T10:36:50.015 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:44.148+0000 7f5244525700 1 Processor -- start 2024-03-24T10:36:50.015 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:44.148+0000 7f5244525700 1 -- start start 2024-03-24T10:36:50.015 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:44.148+0000 7f5244525700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f523c071230 0x7f523c105560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:36:50.015 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:44.148+0000 7f5244525700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523c071640 con 0x7f523c071230 2024-03-24T10:36:50.015 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:44.148+0000 7f52422c1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f523c071230 0x7f523c105560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:36:50.016 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:44.149+0000 7f52422c1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f523c071230 0x7f523c105560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56870/0 (socket says 172.21.15.94:56870) 2024-03-24T10:36:50.016 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:44.149+0000 7f52422c1700 1 -- 172.21.15.94:0/4247865454 learned_addr learned my addr 172.21.15.94:0/4247865454 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:36:50.016 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:44.149+0000 7f52422c1700 1 -- 172.21.15.94:0/4247865454 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f523c105aa0 con 0x7f523c071230 2024-03-24T10:36:50.016 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:44.149+0000 7f52422c1700 1 --2- 172.21.15.94:0/4247865454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f523c071230 0x7f523c105560 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f522c0098e0 tx=0x7f522c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c647ac21fdd6280 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:36:50.016 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:44.149+0000 7f52412bf700 1 -- 172.21.15.94:0/4247865454 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f522c00b860 con 0x7f523c071230 2024-03-24T10:36:50.016 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:44.149+0000 7f52412bf700 1 -- 172.21.15.94:0/4247865454 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f522c00be50 con 0x7f523c071230 2024-03-24T10:36:50.016 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:44.149+0000 7f52412bf700 1 -- 172.21.15.94:0/4247865454 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f522c010380 con 0x7f523c071230 2024-03-24T10:36:50.016 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:44.150+0000 7f5244525700 1 -- 172.21.15.94:0/4247865454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f523c071230 msgr2=0x7f523c105560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:36:50.016 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:44.150+0000 7f5244525700 1 --2- 172.21.15.94:0/4247865454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f523c071230 0x7f523c105560 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f522c0098e0 tx=0x7f522c009250 comp rx=0 tx=0).stop 2024-03-24T10:36:50.016 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:44.150+0000 7f5244525700 1 -- 172.21.15.94:0/4247865454 shutdown_connections 2024-03-24T10:36:50.016 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:44.150+0000 7f5244525700 1 --2- 172.21.15.94:0/4247865454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f523c071230 0x7f523c105560 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:50.016 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:44.150+0000 7f5244525700 1 -- 172.21.15.94:0/4247865454 >> 172.21.15.94:0/4247865454 conn(0x7f523c0fcfb0 msgr2=0x7f523c0ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:36:50.016 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:44.150+0000 7f5244525700 1 -- 172.21.15.94:0/4247865454 shutdown_connections 2024-03-24T10:36:50.016 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:44.150+0000 7f5244525700 1 -- 172.21.15.94:0/4247865454 wait complete. 2024-03-24T10:36:50.016 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:44.150+0000 7f5244525700 1 Processor -- start 2024-03-24T10:36:50.017 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:44.150+0000 7f5244525700 1 -- start start 2024-03-24T10:36:50.017 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:44.151+0000 7f5244525700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f523c071230 0x7f523c102eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:36:50.017 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:44.151+0000 7f5244525700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523c1033f0 con 0x7f523c071230 2024-03-24T10:36:50.017 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:44.151+0000 7f52422c1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f523c071230 0x7f523c102eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:36:50.017 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:44.151+0000 7f52422c1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f523c071230 0x7f523c102eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56874/0 (socket says 172.21.15.94:56874) 2024-03-24T10:36:50.017 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:44.151+0000 7f52422c1700 1 -- 172.21.15.94:0/3648923806 learned_addr learned my addr 172.21.15.94:0/3648923806 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:36:50.017 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:44.151+0000 7f52422c1700 1 -- 172.21.15.94:0/3648923806 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f522c0094f0 con 0x7f523c071230 2024-03-24T10:36:50.017 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:44.151+0000 7f52422c1700 1 --2- 172.21.15.94:0/3648923806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f523c071230 0x7f523c102eb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f522c0090b0 tx=0x7f522c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:36:50.017 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:44.151+0000 7f5232ffd700 1 -- 172.21.15.94:0/3648923806 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f522c010760 con 0x7f523c071230 2024-03-24T10:36:50.017 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:44.151+0000 7f5244525700 1 -- 172.21.15.94:0/3648923806 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f523c101460 con 0x7f523c071230 2024-03-24T10:36:50.017 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:44.151+0000 7f5232ffd700 1 -- 172.21.15.94:0/3648923806 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f522c010d50 con 0x7f523c071230 2024-03-24T10:36:50.017 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:44.151+0000 7f5232ffd700 1 -- 172.21.15.94:0/3648923806 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f522c01ace0 con 0x7f523c071230 2024-03-24T10:36:50.017 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:44.151+0000 7f5244525700 1 -- 172.21.15.94:0/3648923806 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f523c101950 con 0x7f523c071230 2024-03-24T10:36:50.017 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:44.152+0000 7f5232ffd700 1 -- 172.21.15.94:0/3648923806 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f522c02d3f0 con 0x7f523c071230 2024-03-24T10:36:50.018 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:44.152+0000 7f5232ffd700 1 --2- 172.21.15.94:0/3648923806 >> [v2:172.21.15.94:6800/1013449112,v1:172.21.15.94:6801/1013449112] conn(0x7f52280367e0 0x7f5228038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:36:50.018 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:44.152+0000 7f5232ffd700 1 -- 172.21.15.94:0/3648923806 --> [v2:172.21.15.94:6800/1013449112,v1:172.21.15.94:6801/1013449112] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f52280393a0 con 0x7f52280367e0 2024-03-24T10:36:50.018 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:44.153+0000 7f5232ffd700 1 -- 172.21.15.94:0/3648923806 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f522c04a260 con 0x7f523c071230 2024-03-24T10:36:50.018 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:44.153+0000 7f5241ac0700 1 -- 172.21.15.94:0/3648923806 >> [v2:172.21.15.94:6800/1013449112,v1:172.21.15.94:6801/1013449112] conn(0x7f52280367e0 msgr2=0x7f5228038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/1013449112 2024-03-24T10:36:50.018 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:44.153+0000 7f5241ac0700 1 --2- 172.21.15.94:0/3648923806 >> [v2:172.21.15.94:6800/1013449112,v1:172.21.15.94:6801/1013449112] conn(0x7f52280367e0 0x7f5228038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-24T10:36:50.018 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:44.354+0000 7f5241ac0700 1 -- 172.21.15.94:0/3648923806 >> [v2:172.21.15.94:6800/1013449112,v1:172.21.15.94:6801/1013449112] conn(0x7f52280367e0 msgr2=0x7f5228038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/1013449112 2024-03-24T10:36:50.018 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:44.354+0000 7f5241ac0700 1 --2- 172.21.15.94:0/3648923806 >> [v2:172.21.15.94:6800/1013449112,v1:172.21.15.94:6801/1013449112] conn(0x7f52280367e0 0x7f5228038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-24T10:36:50.018 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:44.755+0000 7f5241ac0700 1 -- 172.21.15.94:0/3648923806 >> [v2:172.21.15.94:6800/1013449112,v1:172.21.15.94:6801/1013449112] conn(0x7f52280367e0 msgr2=0x7f5228038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/1013449112 2024-03-24T10:36:50.018 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:44.755+0000 7f5241ac0700 1 --2- 172.21.15.94:0/3648923806 >> [v2:172.21.15.94:6800/1013449112,v1:172.21.15.94:6801/1013449112] conn(0x7f52280367e0 0x7f5228038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-24T10:36:50.018 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:45.556+0000 7f5241ac0700 1 -- 172.21.15.94:0/3648923806 >> [v2:172.21.15.94:6800/1013449112,v1:172.21.15.94:6801/1013449112] conn(0x7f52280367e0 msgr2=0x7f5228038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/1013449112 2024-03-24T10:36:50.019 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:45.556+0000 7f5241ac0700 1 --2- 172.21.15.94:0/3648923806 >> [v2:172.21.15.94:6800/1013449112,v1:172.21.15.94:6801/1013449112] conn(0x7f52280367e0 0x7f5228038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-24T10:36:50.019 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:47.157+0000 7f5241ac0700 1 -- 172.21.15.94:0/3648923806 >> [v2:172.21.15.94:6800/1013449112,v1:172.21.15.94:6801/1013449112] conn(0x7f52280367e0 msgr2=0x7f5228038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/1013449112 2024-03-24T10:36:50.019 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:47.157+0000 7f5241ac0700 1 --2- 172.21.15.94:0/3648923806 >> [v2:172.21.15.94:6800/1013449112,v1:172.21.15.94:6801/1013449112] conn(0x7f52280367e0 0x7f5228038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-24T10:36:50.019 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:48.434+0000 7f5232ffd700 1 -- 172.21.15.94:0/3648923806 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mgrmap(e 6) v1 ==== 43135+0+0 (secure 0 0 0) 0x7f522c02d6f0 con 0x7f523c071230 2024-03-24T10:36:50.019 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:48.434+0000 7f5232ffd700 1 -- 172.21.15.94:0/3648923806 >> [v2:172.21.15.94:6800/1013449112,v1:172.21.15.94:6801/1013449112] conn(0x7f52280367e0 msgr2=0x7f5228038c90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-24T10:36:50.019 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:48.434+0000 7f5232ffd700 1 --2- 172.21.15.94:0/3648923806 >> [v2:172.21.15.94:6800/1013449112,v1:172.21.15.94:6801/1013449112] conn(0x7f52280367e0 0x7f5228038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:50.019 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:49.435+0000 7f5232ffd700 1 -- 172.21.15.94:0/3648923806 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 7) v1 ==== 43262+0+0 (secure 0 0 0) 0x7f522c02da70 con 0x7f523c071230 2024-03-24T10:36:50.019 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:49.435+0000 7f5232ffd700 1 --2- 172.21.15.94:0/3648923806 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f5228039fc0 0x7f522803c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:36:50.019 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:49.435+0000 7f5232ffd700 1 -- 172.21.15.94:0/3648923806 --> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f52280393a0 con 0x7f5228039fc0 2024-03-24T10:36:50.019 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:49.435+0000 7f5241ac0700 1 --2- 172.21.15.94:0/3648923806 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f5228039fc0 0x7f522803c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:36:50.019 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:49.436+0000 7f5241ac0700 1 --2- 172.21.15.94:0/3648923806 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f5228039fc0 0x7f522803c3a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f52380039d0 tx=0x7f5238009350 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:36:50.019 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:49.437+0000 7f5232ffd700 1 -- 172.21.15.94:0/3648923806 <== mgr.14120 v2:172.21.15.94:6800/1996608546 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7f52280393a0 con 0x7f5228039fc0 2024-03-24T10:36:50.019 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:49.443+0000 7f5244525700 1 -- 172.21.15.94:0/3648923806 --> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f523c04bed0 con 0x7f5228039fc0 2024-03-24T10:36:50.019 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:49.443+0000 7f5232ffd700 1 -- 172.21.15.94:0/3648923806 <== mgr.14120 v2:172.21.15.94:6800/1996608546 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f523c04bed0 con 0x7f5228039fc0 2024-03-24T10:36:50.019 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:49.443+0000 7f5244525700 1 -- 172.21.15.94:0/3648923806 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f5228039fc0 msgr2=0x7f522803c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:36:50.020 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:49.443+0000 7f5244525700 1 --2- 172.21.15.94:0/3648923806 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f5228039fc0 0x7f522803c3a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f52380039d0 tx=0x7f5238009350 comp rx=0 tx=0).stop 2024-03-24T10:36:50.020 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:49.443+0000 7f5244525700 1 -- 172.21.15.94:0/3648923806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f523c071230 msgr2=0x7f523c102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:36:50.020 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:49.443+0000 7f5244525700 1 --2- 172.21.15.94:0/3648923806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f523c071230 0x7f523c102eb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f522c0090b0 tx=0x7f522c009250 comp rx=0 tx=0).stop 2024-03-24T10:36:50.020 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:49.443+0000 7f5244525700 1 -- 172.21.15.94:0/3648923806 shutdown_connections 2024-03-24T10:36:50.020 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:49.443+0000 7f5244525700 1 --2- 172.21.15.94:0/3648923806 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f5228039fc0 0x7f522803c3a0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:50.020 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:49.443+0000 7f5244525700 1 --2- 172.21.15.94:0/3648923806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f523c071230 0x7f523c102eb0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:50.020 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:49.443+0000 7f5244525700 1 --2- 172.21.15.94:0/3648923806 >> [v2:172.21.15.94:6800/1013449112,v1:172.21.15.94:6801/1013449112] conn(0x7f52280367e0 0x7f5228038c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:50.020 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:49.443+0000 7f5244525700 1 -- 172.21.15.94:0/3648923806 >> 172.21.15.94:0/3648923806 conn(0x7f523c0fcfb0 msgr2=0x7f523c0fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:36:50.020 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:49.443+0000 7f5244525700 1 -- 172.21.15.94:0/3648923806 shutdown_connections 2024-03-24T10:36:50.020 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:49.443+0000 7f5244525700 1 -- 172.21.15.94:0/3648923806 wait complete. 2024-03-24T10:36:50.020 INFO:teuthology.orchestra.run.smithi094.stdout:mgr epoch 5 is available 2024-03-24T10:36:50.020 INFO:teuthology.orchestra.run.smithi094.stdout:Setting orchestrator backend to cephadm... 2024-03-24T10:36:50.672 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:50 smithi094 ceph-mon[29446]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-24T10:36:50.672 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:50 smithi094 ceph-mon[29446]: [24/Mar/2024:10:36:49] ENGINE Bus STARTING 2024-03-24T10:36:50.672 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:50 smithi094 ceph-mon[29446]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-24T10:36:50.673 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:50 smithi094 ceph-mon[29446]: [24/Mar/2024:10:36:49] ENGINE Serving on https://172.21.15.94:7150 2024-03-24T10:36:50.673 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:50 smithi094 ceph-mon[29446]: [24/Mar/2024:10:36:49] ENGINE Bus STARTED 2024-03-24T10:36:50.673 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:50 smithi094 ceph-mon[29446]: from='mgr.14120 172.21.15.94:0/773982920' entity='mgr.smithi094.yaethu' 2024-03-24T10:36:50.673 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:50 smithi094 ceph-mon[29446]: from='mgr.14120 172.21.15.94:0/773982920' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:36:51.210 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.463+0000 7fb5d2b26700 1 Processor -- start 2024-03-24T10:36:51.210 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.463+0000 7fb5d2b26700 1 -- start start 2024-03-24T10:36:51.210 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.464+0000 7fb5d2b26700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5cc101750 0x7fb5cc101b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:36:51.210 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.464+0000 7fb5d2b26700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5cc1020a0 con 0x7fb5cc101750 2024-03-24T10:36:51.210 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.464+0000 7fb5d08c2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5cc101750 0x7fb5cc101b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:36:51.210 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.464+0000 7fb5d08c2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5cc101750 0x7fb5cc101b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34480/0 (socket says 172.21.15.94:34480) 2024-03-24T10:36:51.211 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.464+0000 7fb5d08c2700 1 -- 172.21.15.94:0/1201101407 learned_addr learned my addr 172.21.15.94:0/1201101407 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:36:51.211 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.464+0000 7fb5d08c2700 1 -- 172.21.15.94:0/1201101407 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5cc1021e0 con 0x7fb5cc101750 2024-03-24T10:36:51.211 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.464+0000 7fb5d08c2700 1 --2- 172.21.15.94:0/1201101407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5cc101750 0x7fb5cc101b60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb5c000bb80 tx=0x7fb5c000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b319477bb4e5e2bf server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:36:51.211 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.464+0000 7fb5cb7fe700 1 -- 172.21.15.94:0/1201101407 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb5c000dd00 con 0x7fb5cc101750 2024-03-24T10:36:51.211 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.464+0000 7fb5cb7fe700 1 -- 172.21.15.94:0/1201101407 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fb5c000de60 con 0x7fb5cc101750 2024-03-24T10:36:51.211 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.464+0000 7fb5cb7fe700 1 -- 172.21.15.94:0/1201101407 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb5c00188a0 con 0x7fb5cc101750 2024-03-24T10:36:51.211 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.465+0000 7fb5d2b26700 1 -- 172.21.15.94:0/1201101407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5cc101750 msgr2=0x7fb5cc101b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:36:51.211 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.465+0000 7fb5d2b26700 1 --2- 172.21.15.94:0/1201101407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5cc101750 0x7fb5cc101b60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb5c000bb80 tx=0x7fb5c000b4f0 comp rx=0 tx=0).stop 2024-03-24T10:36:51.211 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.465+0000 7fb5d2b26700 1 -- 172.21.15.94:0/1201101407 shutdown_connections 2024-03-24T10:36:51.211 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.465+0000 7fb5d2b26700 1 --2- 172.21.15.94:0/1201101407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5cc101750 0x7fb5cc101b60 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:51.211 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.465+0000 7fb5d2b26700 1 -- 172.21.15.94:0/1201101407 >> 172.21.15.94:0/1201101407 conn(0x7fb5cc0fcf30 msgr2=0x7fb5cc0ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:36:51.211 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.465+0000 7fb5d2b26700 1 -- 172.21.15.94:0/1201101407 shutdown_connections 2024-03-24T10:36:51.211 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.465+0000 7fb5d2b26700 1 -- 172.21.15.94:0/1201101407 wait complete. 2024-03-24T10:36:51.211 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.465+0000 7fb5d2b26700 1 Processor -- start 2024-03-24T10:36:51.212 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.466+0000 7fb5d2b26700 1 -- start start 2024-03-24T10:36:51.212 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.466+0000 7fb5d2b26700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5cc101750 0x7fb5cc10f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:36:51.212 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.466+0000 7fb5d2b26700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5cc10fa10 con 0x7fb5cc101750 2024-03-24T10:36:51.212 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.466+0000 7fb5d08c2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5cc101750 0x7fb5cc10f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:36:51.212 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.466+0000 7fb5d08c2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5cc101750 0x7fb5cc10f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34482/0 (socket says 172.21.15.94:34482) 2024-03-24T10:36:51.212 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.466+0000 7fb5d08c2700 1 -- 172.21.15.94:0/3918502328 learned_addr learned my addr 172.21.15.94:0/3918502328 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:36:51.212 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.466+0000 7fb5d08c2700 1 -- 172.21.15.94:0/3918502328 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c000b790 con 0x7fb5cc101750 2024-03-24T10:36:51.212 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.466+0000 7fb5d08c2700 1 --2- 172.21.15.94:0/3918502328 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5cc101750 0x7fb5cc10f4d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0018600 tx=0x7fb5c000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:36:51.212 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.466+0000 7fb5c97fa700 1 -- 172.21.15.94:0/3918502328 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb5c0010b70 con 0x7fb5cc101750 2024-03-24T10:36:51.212 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.467+0000 7fb5c97fa700 1 -- 172.21.15.94:0/3918502328 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fb5c001d050 con 0x7fb5cc101750 2024-03-24T10:36:51.212 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.467+0000 7fb5d2b26700 1 -- 172.21.15.94:0/3918502328 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5cc10fc10 con 0x7fb5cc101750 2024-03-24T10:36:51.212 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.467+0000 7fb5c97fa700 1 -- 172.21.15.94:0/3918502328 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb5c00225b0 con 0x7fb5cc101750 2024-03-24T10:36:51.212 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.467+0000 7fb5d2b26700 1 -- 172.21.15.94:0/3918502328 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5cc110730 con 0x7fb5cc101750 2024-03-24T10:36:51.212 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.467+0000 7fb5c97fa700 1 -- 172.21.15.94:0/3918502328 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) v1 ==== 43262+0+0 (secure 0 0 0) 0x7fb5c0022760 con 0x7fb5cc101750 2024-03-24T10:36:51.212 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.468+0000 7fb5c97fa700 1 --2- 172.21.15.94:0/3918502328 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7fb5bc0366b0 0x7fb5bc038b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:36:51.212 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.468+0000 7fb5d2b26700 1 -- 172.21.15.94:0/3918502328 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5cc05f5f0 con 0x7fb5cc101750 2024-03-24T10:36:51.213 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.468+0000 7fb5c97fa700 1 -- 172.21.15.94:0/3918502328 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fb5c0014070 con 0x7fb5cc101750 2024-03-24T10:36:51.213 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.468+0000 7fb5cbfff700 1 --2- 172.21.15.94:0/3918502328 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7fb5bc0366b0 0x7fb5bc038b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:36:51.213 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.468+0000 7fb5cbfff700 1 --2- 172.21.15.94:0/3918502328 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7fb5bc0366b0 0x7fb5bc038b60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8009990 tx=0x7fb5b8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:36:51.213 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.473+0000 7fb5c97fa700 1 -- 172.21.15.94:0/3918502328 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fb5c00218d0 con 0x7fb5cc101750 2024-03-24T10:36:51.213 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.558+0000 7fb5c97fa700 1 -- 172.21.15.94:0/3918502328 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fb5c002d980 con 0x7fb5cc101750 2024-03-24T10:36:51.213 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.672+0000 7fb5d2b26700 1 -- 172.21.15.94:0/3918502328 --> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fb5cc110ae0 con 0x7fb5bc0366b0 2024-03-24T10:36:51.213 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.679+0000 7fb5c97fa700 1 -- 172.21.15.94:0/3918502328 <== mgr.14120 v2:172.21.15.94:6800/1996608546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fb5cc110ae0 con 0x7fb5bc0366b0 2024-03-24T10:36:51.213 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.681+0000 7fb5d2b26700 1 -- 172.21.15.94:0/3918502328 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7fb5bc0366b0 msgr2=0x7fb5bc038b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:36:51.213 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.681+0000 7fb5d2b26700 1 --2- 172.21.15.94:0/3918502328 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7fb5bc0366b0 0x7fb5bc038b60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8009990 tx=0x7fb5b8006e40 comp rx=0 tx=0).stop 2024-03-24T10:36:51.213 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.681+0000 7fb5d2b26700 1 -- 172.21.15.94:0/3918502328 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5cc101750 msgr2=0x7fb5cc10f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:36:51.213 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.681+0000 7fb5d2b26700 1 --2- 172.21.15.94:0/3918502328 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5cc101750 0x7fb5cc10f4d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0018600 tx=0x7fb5c000b4f0 comp rx=0 tx=0).stop 2024-03-24T10:36:51.213 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.681+0000 7fb5d2b26700 1 -- 172.21.15.94:0/3918502328 shutdown_connections 2024-03-24T10:36:51.213 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.681+0000 7fb5d2b26700 1 --2- 172.21.15.94:0/3918502328 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7fb5bc0366b0 0x7fb5bc038b60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:51.213 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.681+0000 7fb5d2b26700 1 --2- 172.21.15.94:0/3918502328 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5cc101750 0x7fb5cc10f4d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:51.213 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.681+0000 7fb5d2b26700 1 -- 172.21.15.94:0/3918502328 >> 172.21.15.94:0/3918502328 conn(0x7fb5cc0fcf30 msgr2=0x7fb5cc105ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:36:51.214 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.682+0000 7fb5d2b26700 1 -- 172.21.15.94:0/3918502328 shutdown_connections 2024-03-24T10:36:51.214 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:50.682+0000 7fb5d2b26700 1 -- 172.21.15.94:0/3918502328 wait complete. 2024-03-24T10:36:51.605 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:51 smithi094 ceph-mon[29446]: mgrmap e8: smithi094.yaethu(active, since 2s) 2024-03-24T10:36:51.605 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:51 smithi094 ceph-mon[29446]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:36:51.605 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:51 smithi094 ceph-mon[29446]: from='mgr.14120 172.21.15.94:0/773982920' entity='mgr.smithi094.yaethu' 2024-03-24T10:36:51.606 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:51 smithi094 ceph-mon[29446]: from='mgr.14120 172.21.15.94:0/773982920' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:36:52.359 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-24T10:36:52.359 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.610+0000 7f8f9dee2700 1 Processor -- start 2024-03-24T10:36:52.359 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.610+0000 7f8f9dee2700 1 -- start start 2024-03-24T10:36:52.359 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.610+0000 7f8f9dee2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f98102920 0x7f8f98102d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:36:52.359 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.610+0000 7f8f9dee2700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f98103270 con 0x7f8f98102920 2024-03-24T10:36:52.360 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.610+0000 7f8f977fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f98102920 0x7f8f98102d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:36:52.360 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.610+0000 7f8f977fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f98102920 0x7f8f98102d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34484/0 (socket says 172.21.15.94:34484) 2024-03-24T10:36:52.360 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.610+0000 7f8f977fe700 1 -- 172.21.15.94:0/1750717209 learned_addr learned my addr 172.21.15.94:0/1750717209 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:36:52.360 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.610+0000 7f8f977fe700 1 -- 172.21.15.94:0/1750717209 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f981033b0 con 0x7f8f98102920 2024-03-24T10:36:52.360 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.611+0000 7f8f977fe700 1 --2- 172.21.15.94:0/1750717209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f98102920 0x7f8f98102d30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8f880098e0 tx=0x7f8f88009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a54c5f7adcf1ffa7 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:36:52.360 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.611+0000 7f8f967fc700 1 -- 172.21.15.94:0/1750717209 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8f8800b860 con 0x7f8f98102920 2024-03-24T10:36:52.360 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.611+0000 7f8f967fc700 1 -- 172.21.15.94:0/1750717209 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f8f8800be50 con 0x7f8f98102920 2024-03-24T10:36:52.360 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.611+0000 7f8f967fc700 1 -- 172.21.15.94:0/1750717209 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8f88010380 con 0x7f8f98102920 2024-03-24T10:36:52.360 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.611+0000 7f8f9dee2700 1 -- 172.21.15.94:0/1750717209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f98102920 msgr2=0x7f8f98102d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:36:52.360 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.611+0000 7f8f9dee2700 1 --2- 172.21.15.94:0/1750717209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f98102920 0x7f8f98102d30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8f880098e0 tx=0x7f8f88009250 comp rx=0 tx=0).stop 2024-03-24T10:36:52.360 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.611+0000 7f8f9dee2700 1 -- 172.21.15.94:0/1750717209 shutdown_connections 2024-03-24T10:36:52.360 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.611+0000 7f8f9dee2700 1 --2- 172.21.15.94:0/1750717209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f98102920 0x7f8f98102d30 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:52.360 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.611+0000 7f8f9dee2700 1 -- 172.21.15.94:0/1750717209 >> 172.21.15.94:0/1750717209 conn(0x7f8f980fe0a0 msgr2=0x7f8f981004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:36:52.361 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.611+0000 7f8f9dee2700 1 -- 172.21.15.94:0/1750717209 shutdown_connections 2024-03-24T10:36:52.361 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.611+0000 7f8f9dee2700 1 -- 172.21.15.94:0/1750717209 wait complete. 2024-03-24T10:36:52.361 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.612+0000 7f8f9dee2700 1 Processor -- start 2024-03-24T10:36:52.361 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.612+0000 7f8f9dee2700 1 -- start start 2024-03-24T10:36:52.361 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.612+0000 7f8f9dee2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f98102920 0x7f8f981116a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:36:52.361 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.612+0000 7f8f9dee2700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f98111be0 con 0x7f8f98102920 2024-03-24T10:36:52.361 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.612+0000 7f8f977fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f98102920 0x7f8f981116a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:36:52.361 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.612+0000 7f8f977fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f98102920 0x7f8f981116a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34496/0 (socket says 172.21.15.94:34496) 2024-03-24T10:36:52.361 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.612+0000 7f8f977fe700 1 -- 172.21.15.94:0/430269430 learned_addr learned my addr 172.21.15.94:0/430269430 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:36:52.361 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.613+0000 7f8f977fe700 1 -- 172.21.15.94:0/430269430 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f880094f0 con 0x7f8f98102920 2024-03-24T10:36:52.361 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.613+0000 7f8f977fe700 1 --2- 172.21.15.94:0/430269430 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f98102920 0x7f8f981116a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8f88000c00 tx=0x7f8f88009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:36:52.361 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.613+0000 7f8f7ffff700 1 -- 172.21.15.94:0/430269430 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8f880107c0 con 0x7f8f98102920 2024-03-24T10:36:52.361 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.613+0000 7f8f7ffff700 1 -- 172.21.15.94:0/430269430 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f8f88010db0 con 0x7f8f98102920 2024-03-24T10:36:52.361 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.613+0000 7f8f9dee2700 1 -- 172.21.15.94:0/430269430 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f98111de0 con 0x7f8f98102920 2024-03-24T10:36:52.361 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.613+0000 7f8f7ffff700 1 -- 172.21.15.94:0/430269430 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8f8801ad00 con 0x7f8f98102920 2024-03-24T10:36:52.362 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.613+0000 7f8f9dee2700 1 -- 172.21.15.94:0/430269430 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f98112950 con 0x7f8f98102920 2024-03-24T10:36:52.362 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.614+0000 7f8f7ffff700 1 -- 172.21.15.94:0/430269430 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f8f880193c0 con 0x7f8f98102920 2024-03-24T10:36:52.362 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.614+0000 7f8f7ffff700 1 --2- 172.21.15.94:0/430269430 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f8f800367e0 0x7f8f80038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:36:52.362 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.614+0000 7f8f9dee2700 1 -- 172.21.15.94:0/430269430 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f9805f5f0 con 0x7f8f98102920 2024-03-24T10:36:52.362 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.614+0000 7f8f7ffff700 1 -- 172.21.15.94:0/430269430 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f8f8804a620 con 0x7f8f98102920 2024-03-24T10:36:52.362 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.614+0000 7f8f96ffd700 1 --2- 172.21.15.94:0/430269430 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f8f800367e0 0x7f8f80038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:36:52.362 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.614+0000 7f8f96ffd700 1 --2- 172.21.15.94:0/430269430 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f8f800367e0 0x7f8f80038c90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c009990 tx=0x7f8f8c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:36:52.362 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.619+0000 7f8f7ffff700 1 -- 172.21.15.94:0/430269430 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f8f880196c0 con 0x7f8f98102920 2024-03-24T10:36:52.362 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.804+0000 7f8f9dee2700 1 -- 172.21.15.94:0/430269430 --> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f980ff310 con 0x7f8f800367e0 2024-03-24T10:36:52.362 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.804+0000 7f8f7ffff700 1 -- 172.21.15.94:0/430269430 <== mgr.14120 v2:172.21.15.94:6800/1996608546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f8f980ff310 con 0x7f8f800367e0 2024-03-24T10:36:52.362 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.806+0000 7f8f9dee2700 1 -- 172.21.15.94:0/430269430 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f8f800367e0 msgr2=0x7f8f80038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:36:52.362 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.806+0000 7f8f9dee2700 1 --2- 172.21.15.94:0/430269430 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f8f800367e0 0x7f8f80038c90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c009990 tx=0x7f8f8c006e40 comp rx=0 tx=0).stop 2024-03-24T10:36:52.362 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.806+0000 7f8f9dee2700 1 -- 172.21.15.94:0/430269430 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f98102920 msgr2=0x7f8f981116a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:36:52.362 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.806+0000 7f8f9dee2700 1 --2- 172.21.15.94:0/430269430 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f98102920 0x7f8f981116a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8f88000c00 tx=0x7f8f88009250 comp rx=0 tx=0).stop 2024-03-24T10:36:52.363 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.806+0000 7f8f9dee2700 1 -- 172.21.15.94:0/430269430 shutdown_connections 2024-03-24T10:36:52.363 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.806+0000 7f8f9dee2700 1 --2- 172.21.15.94:0/430269430 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f8f800367e0 0x7f8f80038c90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:52.363 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.806+0000 7f8f9dee2700 1 --2- 172.21.15.94:0/430269430 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f98102920 0x7f8f981116a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:52.363 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.806+0000 7f8f9dee2700 1 -- 172.21.15.94:0/430269430 >> 172.21.15.94:0/430269430 conn(0x7f8f980fe0a0 msgr2=0x7f8f980fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:36:52.363 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.807+0000 7f8f9dee2700 1 -- 172.21.15.94:0/430269430 shutdown_connections 2024-03-24T10:36:52.363 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:51.807+0000 7f8f9dee2700 1 -- 172.21.15.94:0/430269430 wait complete. 2024-03-24T10:36:52.363 INFO:teuthology.orchestra.run.smithi094.stdout:Generating ssh key... 2024-03-24T10:36:52.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:52 smithi094 ceph-mon[29446]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:36:53.598 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:52.773+0000 7ff8eef05700 1 Processor -- start 2024-03-24T10:36:53.601 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:52.773+0000 7ff8eef05700 1 -- start start 2024-03-24T10:36:53.601 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:52.774+0000 7ff8eef05700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8e8101dc0 0x7ff8e81021d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:36:53.601 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:52.774+0000 7ff8eef05700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e8102710 con 0x7ff8e8101dc0 2024-03-24T10:36:53.601 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:52.774+0000 7ff8ecca1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8e8101dc0 0x7ff8e81021d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:36:53.601 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:52.774+0000 7ff8ecca1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8e8101dc0 0x7ff8e81021d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34508/0 (socket says 172.21.15.94:34508) 2024-03-24T10:36:53.601 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:52.774+0000 7ff8ecca1700 1 -- 172.21.15.94:0/3166718155 learned_addr learned my addr 172.21.15.94:0/3166718155 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:36:53.601 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:52.774+0000 7ff8ecca1700 1 -- 172.21.15.94:0/3166718155 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e8102850 con 0x7ff8e8101dc0 2024-03-24T10:36:53.602 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:52.774+0000 7ff8ecca1700 1 --2- 172.21.15.94:0/3166718155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8e8101dc0 0x7ff8e81021d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc0098e0 tx=0x7ff8dc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd723f35b962bdb8 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:36:53.602 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:52.774+0000 7ff8e77fe700 1 -- 172.21.15.94:0/3166718155 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff8dc00b860 con 0x7ff8e8101dc0 2024-03-24T10:36:53.602 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:52.774+0000 7ff8e77fe700 1 -- 172.21.15.94:0/3166718155 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7ff8dc00be50 con 0x7ff8e8101dc0 2024-03-24T10:36:53.602 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:52.774+0000 7ff8e77fe700 1 -- 172.21.15.94:0/3166718155 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff8dc010380 con 0x7ff8e8101dc0 2024-03-24T10:36:53.602 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:52.775+0000 7ff8eef05700 1 -- 172.21.15.94:0/3166718155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8e8101dc0 msgr2=0x7ff8e81021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:36:53.602 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:52.775+0000 7ff8eef05700 1 --2- 172.21.15.94:0/3166718155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8e8101dc0 0x7ff8e81021d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc0098e0 tx=0x7ff8dc009250 comp rx=0 tx=0).stop 2024-03-24T10:36:53.602 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:52.775+0000 7ff8eef05700 1 -- 172.21.15.94:0/3166718155 shutdown_connections 2024-03-24T10:36:53.602 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:52.775+0000 7ff8eef05700 1 --2- 172.21.15.94:0/3166718155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8e8101dc0 0x7ff8e81021d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:53.602 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:52.775+0000 7ff8eef05700 1 -- 172.21.15.94:0/3166718155 >> 172.21.15.94:0/3166718155 conn(0x7ff8e80fc750 msgr2=0x7ff8e80feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:36:53.602 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:52.775+0000 7ff8eef05700 1 -- 172.21.15.94:0/3166718155 shutdown_connections 2024-03-24T10:36:53.602 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:52.775+0000 7ff8eef05700 1 -- 172.21.15.94:0/3166718155 wait complete. 2024-03-24T10:36:53.602 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:52.775+0000 7ff8eef05700 1 Processor -- start 2024-03-24T10:36:53.602 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:52.776+0000 7ff8eef05700 1 -- start start 2024-03-24T10:36:53.603 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:52.776+0000 7ff8eef05700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8e8101dc0 0x7ff8e81015c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:36:53.603 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:52.776+0000 7ff8eef05700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e81143c0 con 0x7ff8e8101dc0 2024-03-24T10:36:53.603 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:52.776+0000 7ff8ecca1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8e8101dc0 0x7ff8e81015c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:36:53.603 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:52.776+0000 7ff8ecca1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8e8101dc0 0x7ff8e81015c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34524/0 (socket says 172.21.15.94:34524) 2024-03-24T10:36:53.603 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:52.776+0000 7ff8ecca1700 1 -- 172.21.15.94:0/640863954 learned_addr learned my addr 172.21.15.94:0/640863954 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:36:53.603 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:52.776+0000 7ff8ecca1700 1 -- 172.21.15.94:0/640863954 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8dc0094f0 con 0x7ff8e8101dc0 2024-03-24T10:36:53.603 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:52.776+0000 7ff8ecca1700 1 --2- 172.21.15.94:0/640863954 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8e8101dc0 0x7ff8e81015c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc00b440 tx=0x7ff8dc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:36:53.603 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:52.776+0000 7ff8e57fa700 1 -- 172.21.15.94:0/640863954 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff8dc010760 con 0x7ff8e8101dc0 2024-03-24T10:36:53.604 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:52.776+0000 7ff8e57fa700 1 -- 172.21.15.94:0/640863954 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7ff8dc010d50 con 0x7ff8e8101dc0 2024-03-24T10:36:53.604 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:52.777+0000 7ff8eef05700 1 -- 172.21.15.94:0/640863954 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8e8114500 con 0x7ff8e8101dc0 2024-03-24T10:36:53.604 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:52.777+0000 7ff8e57fa700 1 -- 172.21.15.94:0/640863954 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff8dc01ae20 con 0x7ff8e8101dc0 2024-03-24T10:36:53.604 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:52.777+0000 7ff8eef05700 1 -- 172.21.15.94:0/640863954 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8e8114990 con 0x7ff8e8101dc0 2024-03-24T10:36:53.604 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:52.777+0000 7ff8e57fa700 1 -- 172.21.15.94:0/640863954 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7ff8dc02e440 con 0x7ff8e8101dc0 2024-03-24T10:36:53.604 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:52.777+0000 7ff8eef05700 1 -- 172.21.15.94:0/640863954 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8e804bed0 con 0x7ff8e8101dc0 2024-03-24T10:36:53.604 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:52.777+0000 7ff8e57fa700 1 --2- 172.21.15.94:0/640863954 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7ff8d80367e0 0x7ff8d8038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:36:53.604 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:52.777+0000 7ff8e57fa700 1 -- 172.21.15.94:0/640863954 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7ff8dc04a5b0 con 0x7ff8e8101dc0 2024-03-24T10:36:53.604 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:52.778+0000 7ff8e7fff700 1 --2- 172.21.15.94:0/640863954 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7ff8d80367e0 0x7ff8d8038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:36:53.604 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:52.778+0000 7ff8e7fff700 1 --2- 172.21.15.94:0/640863954 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7ff8d80367e0 0x7ff8d8038c90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff8d4009990 tx=0x7ff8d4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:36:53.604 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:52.782+0000 7ff8e57fa700 1 -- 172.21.15.94:0/640863954 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7ff8dc02eb10 con 0x7ff8e8101dc0 2024-03-24T10:36:53.604 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:52.971+0000 7ff8eef05700 1 -- 172.21.15.94:0/640863954 --> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7ff8e80fd9b0 con 0x7ff8d80367e0 2024-03-24T10:36:53.604 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.038+0000 7ff8e57fa700 1 -- 172.21.15.94:0/640863954 <== mgr.14120 v2:172.21.15.94:6800/1996608546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff8e80fd9b0 con 0x7ff8d80367e0 2024-03-24T10:36:53.605 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.040+0000 7ff8eef05700 1 -- 172.21.15.94:0/640863954 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7ff8d80367e0 msgr2=0x7ff8d8038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:36:53.605 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.040+0000 7ff8eef05700 1 --2- 172.21.15.94:0/640863954 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7ff8d80367e0 0x7ff8d8038c90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff8d4009990 tx=0x7ff8d4006e40 comp rx=0 tx=0).stop 2024-03-24T10:36:53.605 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.040+0000 7ff8eef05700 1 -- 172.21.15.94:0/640863954 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8e8101dc0 msgr2=0x7ff8e81015c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:36:53.605 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.040+0000 7ff8eef05700 1 --2- 172.21.15.94:0/640863954 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8e8101dc0 0x7ff8e81015c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc00b440 tx=0x7ff8dc009250 comp rx=0 tx=0).stop 2024-03-24T10:36:53.605 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.040+0000 7ff8eef05700 1 -- 172.21.15.94:0/640863954 shutdown_connections 2024-03-24T10:36:53.605 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.040+0000 7ff8eef05700 1 --2- 172.21.15.94:0/640863954 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7ff8d80367e0 0x7ff8d8038c90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:53.605 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.040+0000 7ff8eef05700 1 --2- 172.21.15.94:0/640863954 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8e8101dc0 0x7ff8e81015c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:53.605 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.040+0000 7ff8eef05700 1 -- 172.21.15.94:0/640863954 >> 172.21.15.94:0/640863954 conn(0x7ff8e80fc750 msgr2=0x7ff8e80fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:36:53.605 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.040+0000 7ff8eef05700 1 -- 172.21.15.94:0/640863954 shutdown_connections 2024-03-24T10:36:53.605 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.040+0000 7ff8eef05700 1 -- 172.21.15.94:0/640863954 wait complete. 2024-03-24T10:36:54.165 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:54 smithi094 ceph-mon[29446]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:36:54.165 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:54 smithi094 ceph-mon[29446]: Generating ssh key... 2024-03-24T10:36:54.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:54 smithi094 ceph-mon[29446]: from='mgr.14120 172.21.15.94:0/773982920' entity='mgr.smithi094.yaethu' 2024-03-24T10:36:54.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:54 smithi094 ceph-mon[29446]: from='mgr.14120 172.21.15.94:0/773982920' entity='mgr.smithi094.yaethu' 2024-03-24T10:36:54.721 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9yYa357KmpLvSmOZJWVx3l32qF6ao/UG82f7zdsM18tnK0HTqk0IuTpSGrwIiMvUp/aDYgTwETqH/aC99p7eMH+QqXrVAIXWIJVh6fzKizbSCBDoU+owWoNp8CraV0JO5ms0ggk/jO4zufDNiRhAaeSS/DsLs6zPKKt10UB7GEG3Bh2VxHnQajVU2HtCEVeFWlHQmPmjsS7e9ZusMbmISbgW11+b8RFxix51/GdazcO8umJACOBhG2eHi0rIR/Ois13tNDDVS7sS8pd+Y0MLYUiXqaE0ImvqZ8W7Cwzw+B2VJwbJmyZ33uzSOeSG/IStzlQ3vAwtpEir7zeHTUoubPT3+ss9O0eDVm5Q6asfjia3jI0F44aJaesiPXxOx1jYXTYv6CSg5NDd5jMEbQnQ5MzIqwt20P646HKfeyU0N7/gM5qL1MxqQe5GTkJrge98LJYkl92aOWS850zebAe/m0iwih4uHcbC4OGSi64Wq4HWlROyRCh4qRSKNKeq4ohE= ceph-386110d4-e9ca-11ee-95cd-87774f69a715 2024-03-24T10:36:54.743 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.971+0000 7f7d18048700 1 Processor -- start 2024-03-24T10:36:54.743 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.971+0000 7f7d18048700 1 -- start start 2024-03-24T10:36:54.743 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.972+0000 7f7d18048700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d10102920 0x7f7d10102d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:36:54.743 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.972+0000 7f7d18048700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d10103270 con 0x7f7d10102920 2024-03-24T10:36:54.743 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.972+0000 7f7d15de4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d10102920 0x7f7d10102d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:36:54.743 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.972+0000 7f7d15de4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d10102920 0x7f7d10102d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34536/0 (socket says 172.21.15.94:34536) 2024-03-24T10:36:54.743 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.972+0000 7f7d15de4700 1 -- 172.21.15.94:0/3007063920 learned_addr learned my addr 172.21.15.94:0/3007063920 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:36:54.744 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.972+0000 7f7d15de4700 1 -- 172.21.15.94:0/3007063920 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d101033b0 con 0x7f7d10102920 2024-03-24T10:36:54.744 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.972+0000 7f7d15de4700 1 --2- 172.21.15.94:0/3007063920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d10102920 0x7f7d10102d30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7d000098e0 tx=0x7f7d00009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4caadf7519becc0f server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:36:54.744 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.972+0000 7f7d14de2700 1 -- 172.21.15.94:0/3007063920 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7d0000b860 con 0x7f7d10102920 2024-03-24T10:36:54.744 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.972+0000 7f7d14de2700 1 -- 172.21.15.94:0/3007063920 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f7d0000be50 con 0x7f7d10102920 2024-03-24T10:36:54.744 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.972+0000 7f7d14de2700 1 -- 172.21.15.94:0/3007063920 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7d00010380 con 0x7f7d10102920 2024-03-24T10:36:54.744 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.973+0000 7f7d18048700 1 -- 172.21.15.94:0/3007063920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d10102920 msgr2=0x7f7d10102d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:36:54.744 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.973+0000 7f7d18048700 1 --2- 172.21.15.94:0/3007063920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d10102920 0x7f7d10102d30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7d000098e0 tx=0x7f7d00009250 comp rx=0 tx=0).stop 2024-03-24T10:36:54.744 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.973+0000 7f7d18048700 1 -- 172.21.15.94:0/3007063920 shutdown_connections 2024-03-24T10:36:54.744 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.973+0000 7f7d18048700 1 --2- 172.21.15.94:0/3007063920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d10102920 0x7f7d10102d30 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:54.744 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.973+0000 7f7d18048700 1 -- 172.21.15.94:0/3007063920 >> 172.21.15.94:0/3007063920 conn(0x7f7d100fe0a0 msgr2=0x7f7d101004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:36:54.744 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.973+0000 7f7d18048700 1 -- 172.21.15.94:0/3007063920 shutdown_connections 2024-03-24T10:36:54.744 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.973+0000 7f7d18048700 1 -- 172.21.15.94:0/3007063920 wait complete. 2024-03-24T10:36:54.744 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.974+0000 7f7d18048700 1 Processor -- start 2024-03-24T10:36:54.744 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.974+0000 7f7d18048700 1 -- start start 2024-03-24T10:36:54.744 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.974+0000 7f7d18048700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d10102920 0x7f7d10078c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:36:54.745 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.974+0000 7f7d18048700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d10079180 con 0x7f7d10102920 2024-03-24T10:36:54.745 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.974+0000 7f7d15de4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d10102920 0x7f7d10078c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:36:54.745 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.974+0000 7f7d15de4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d10102920 0x7f7d10078c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34548/0 (socket says 172.21.15.94:34548) 2024-03-24T10:36:54.745 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.974+0000 7f7d15de4700 1 -- 172.21.15.94:0/1334551529 learned_addr learned my addr 172.21.15.94:0/1334551529 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:36:54.745 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.974+0000 7f7d15de4700 1 -- 172.21.15.94:0/1334551529 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d000094f0 con 0x7f7d10102920 2024-03-24T10:36:54.745 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.974+0000 7f7d15de4700 1 --2- 172.21.15.94:0/1334551529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d10102920 0x7f7d10078c40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7d0000b440 tx=0x7f7d00009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:36:54.745 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.974+0000 7f7d067fc700 1 -- 172.21.15.94:0/1334551529 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7d00010760 con 0x7f7d10102920 2024-03-24T10:36:54.745 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.975+0000 7f7d067fc700 1 -- 172.21.15.94:0/1334551529 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f7d00010d50 con 0x7f7d10102920 2024-03-24T10:36:54.745 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.975+0000 7f7d18048700 1 -- 172.21.15.94:0/1334551529 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d100771f0 con 0x7f7d10102920 2024-03-24T10:36:54.745 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.975+0000 7f7d067fc700 1 -- 172.21.15.94:0/1334551529 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7d0001ae20 con 0x7f7d10102920 2024-03-24T10:36:54.745 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.975+0000 7f7d18048700 1 -- 172.21.15.94:0/1334551529 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d100776e0 con 0x7f7d10102920 2024-03-24T10:36:54.745 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.975+0000 7f7d067fc700 1 -- 172.21.15.94:0/1334551529 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f7d0002e440 con 0x7f7d10102920 2024-03-24T10:36:54.745 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.975+0000 7f7d18048700 1 -- 172.21.15.94:0/1334551529 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d1004bed0 con 0x7f7d10102920 2024-03-24T10:36:54.745 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.976+0000 7f7d067fc700 1 --2- 172.21.15.94:0/1334551529 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f7cfc036770 0x7f7cfc038c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:36:54.745 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.976+0000 7f7d067fc700 1 -- 172.21.15.94:0/1334551529 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f7d0004a840 con 0x7f7d10102920 2024-03-24T10:36:54.746 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.976+0000 7f7d155e3700 1 --2- 172.21.15.94:0/1334551529 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f7cfc036770 0x7f7cfc038c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:36:54.746 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.976+0000 7f7d155e3700 1 --2- 172.21.15.94:0/1334551529 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f7cfc036770 0x7f7cfc038c20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c009990 tx=0x7f7d0c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:36:54.746 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:53.980+0000 7f7d067fc700 1 -- 172.21.15.94:0/1334551529 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f7d00018940 con 0x7f7d10102920 2024-03-24T10:36:54.746 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:54.165+0000 7f7d18048700 1 -- 172.21.15.94:0/1334551529 --> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d10077df0 con 0x7f7cfc036770 2024-03-24T10:36:54.746 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:54.166+0000 7f7d067fc700 1 -- 172.21.15.94:0/1334551529 <== mgr.14120 v2:172.21.15.94:6800/1996608546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f7d10077df0 con 0x7f7cfc036770 2024-03-24T10:36:54.746 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:54.168+0000 7f7d18048700 1 -- 172.21.15.94:0/1334551529 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f7cfc036770 msgr2=0x7f7cfc038c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:36:54.746 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:54.168+0000 7f7d18048700 1 --2- 172.21.15.94:0/1334551529 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f7cfc036770 0x7f7cfc038c20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c009990 tx=0x7f7d0c006e40 comp rx=0 tx=0).stop 2024-03-24T10:36:54.746 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:54.168+0000 7f7d18048700 1 -- 172.21.15.94:0/1334551529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d10102920 msgr2=0x7f7d10078c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:36:54.746 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:54.168+0000 7f7d18048700 1 --2- 172.21.15.94:0/1334551529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d10102920 0x7f7d10078c40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7d0000b440 tx=0x7f7d00009250 comp rx=0 tx=0).stop 2024-03-24T10:36:54.746 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:54.168+0000 7f7d18048700 1 -- 172.21.15.94:0/1334551529 shutdown_connections 2024-03-24T10:36:54.746 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:54.168+0000 7f7d18048700 1 --2- 172.21.15.94:0/1334551529 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f7cfc036770 0x7f7cfc038c20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:54.746 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:54.168+0000 7f7d18048700 1 --2- 172.21.15.94:0/1334551529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d10102920 0x7f7d10078c40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:54.746 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:54.168+0000 7f7d18048700 1 -- 172.21.15.94:0/1334551529 >> 172.21.15.94:0/1334551529 conn(0x7f7d100fe0a0 msgr2=0x7f7d100fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:36:54.746 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:54.168+0000 7f7d18048700 1 -- 172.21.15.94:0/1334551529 shutdown_connections 2024-03-24T10:36:54.747 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:54.168+0000 7f7d18048700 1 -- 172.21.15.94:0/1334551529 wait complete. 2024-03-24T10:36:54.747 INFO:teuthology.orchestra.run.smithi094.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-24T10:36:54.747 INFO:teuthology.orchestra.run.smithi094.stdout:Adding key to root@localhost authorized_keys... 2024-03-24T10:36:54.747 INFO:teuthology.orchestra.run.smithi094.stdout:Adding host smithi094... 2024-03-24T10:36:55.316 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:55 smithi094 ceph-mon[29446]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:36:56.250 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:56 smithi094 ceph-mon[29446]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi094", "addr": "172.21.15.94", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:36:56.251 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:56 smithi094 ceph-mon[29446]: Deploying cephadm binary to smithi094 2024-03-24T10:36:57.921 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout Added host 'smithi094' with addr '172.21.15.94' 2024-03-24T10:36:57.921 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:55.116+0000 7f534c6d6700 1 Processor -- start 2024-03-24T10:36:57.922 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:55.117+0000 7f534c6d6700 1 -- start start 2024-03-24T10:36:57.922 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:55.117+0000 7f534c6d6700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5344100c60 0x7f5344101070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:36:57.922 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:55.117+0000 7f534c6d6700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53441015b0 con 0x7f5344100c60 2024-03-24T10:36:57.922 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:55.117+0000 7f534a472700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5344100c60 0x7f5344101070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:36:57.922 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:55.117+0000 7f534a472700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5344100c60 0x7f5344101070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34556/0 (socket says 172.21.15.94:34556) 2024-03-24T10:36:57.922 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:55.117+0000 7f534a472700 1 -- 172.21.15.94:0/4192330208 learned_addr learned my addr 172.21.15.94:0/4192330208 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:36:57.922 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:55.117+0000 7f534a472700 1 -- 172.21.15.94:0/4192330208 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53441016f0 con 0x7f5344100c60 2024-03-24T10:36:57.922 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:55.117+0000 7f534a472700 1 --2- 172.21.15.94:0/4192330208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5344100c60 0x7f5344101070 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f53340098e0 tx=0x7f5334009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc05efc7a36e19fe server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:36:57.922 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:55.118+0000 7f5349470700 1 -- 172.21.15.94:0/4192330208 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f533400b860 con 0x7f5344100c60 2024-03-24T10:36:57.922 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:55.118+0000 7f5349470700 1 -- 172.21.15.94:0/4192330208 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f533400be50 con 0x7f5344100c60 2024-03-24T10:36:57.922 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:55.118+0000 7f5349470700 1 -- 172.21.15.94:0/4192330208 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5334010380 con 0x7f5344100c60 2024-03-24T10:36:57.922 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:55.118+0000 7f534c6d6700 1 -- 172.21.15.94:0/4192330208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5344100c60 msgr2=0x7f5344101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:36:57.922 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:55.118+0000 7f534c6d6700 1 --2- 172.21.15.94:0/4192330208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5344100c60 0x7f5344101070 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f53340098e0 tx=0x7f5334009250 comp rx=0 tx=0).stop 2024-03-24T10:36:57.923 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:55.118+0000 7f534c6d6700 1 -- 172.21.15.94:0/4192330208 shutdown_connections 2024-03-24T10:36:57.923 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:55.118+0000 7f534c6d6700 1 --2- 172.21.15.94:0/4192330208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5344100c60 0x7f5344101070 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:57.923 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:55.118+0000 7f534c6d6700 1 -- 172.21.15.94:0/4192330208 >> 172.21.15.94:0/4192330208 conn(0x7f53440fc6c0 msgr2=0x7f53440fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:36:57.923 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:55.118+0000 7f534c6d6700 1 -- 172.21.15.94:0/4192330208 shutdown_connections 2024-03-24T10:36:57.923 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:55.118+0000 7f534c6d6700 1 -- 172.21.15.94:0/4192330208 wait complete. 2024-03-24T10:36:57.923 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:55.119+0000 7f534c6d6700 1 Processor -- start 2024-03-24T10:36:57.923 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:55.119+0000 7f534c6d6700 1 -- start start 2024-03-24T10:36:57.923 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:55.119+0000 7f534c6d6700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5344100c60 0x7f5344111710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:36:57.923 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:55.119+0000 7f534c6d6700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5344111c50 con 0x7f5344100c60 2024-03-24T10:36:57.923 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:55.119+0000 7f534a472700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5344100c60 0x7f5344111710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:36:57.923 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:55.119+0000 7f534a472700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5344100c60 0x7f5344111710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34570/0 (socket says 172.21.15.94:34570) 2024-03-24T10:36:57.923 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:55.119+0000 7f534a472700 1 -- 172.21.15.94:0/3675351645 learned_addr learned my addr 172.21.15.94:0/3675351645 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:36:57.924 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:55.119+0000 7f534a472700 1 -- 172.21.15.94:0/3675351645 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53340094f0 con 0x7f5344100c60 2024-03-24T10:36:57.924 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:55.120+0000 7f534a472700 1 --2- 172.21.15.94:0/3675351645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5344100c60 0x7f5344111710 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5334006b00 tx=0x7f5334009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:36:57.924 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:55.120+0000 7f533affd700 1 -- 172.21.15.94:0/3675351645 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5334010760 con 0x7f5344100c60 2024-03-24T10:36:57.924 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:55.120+0000 7f534c6d6700 1 -- 172.21.15.94:0/3675351645 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5344111e50 con 0x7f5344100c60 2024-03-24T10:36:57.924 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:55.120+0000 7f533affd700 1 -- 172.21.15.94:0/3675351645 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f5334010d50 con 0x7f5344100c60 2024-03-24T10:36:57.924 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:55.120+0000 7f533affd700 1 -- 172.21.15.94:0/3675351645 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f533401ad40 con 0x7f5344100c60 2024-03-24T10:36:57.924 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:55.120+0000 7f534c6d6700 1 -- 172.21.15.94:0/3675351645 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53441129c0 con 0x7f5344100c60 2024-03-24T10:36:57.924 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:55.121+0000 7f533affd700 1 -- 172.21.15.94:0/3675351645 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f53340108c0 con 0x7f5344100c60 2024-03-24T10:36:57.924 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:55.121+0000 7f533affd700 1 --2- 172.21.15.94:0/3675351645 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f5330036790 0x7f5330038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:36:57.924 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:55.121+0000 7f533affd700 1 -- 172.21.15.94:0/3675351645 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f533404a5c0 con 0x7f5344100c60 2024-03-24T10:36:57.924 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:55.121+0000 7f534c6d6700 1 -- 172.21.15.94:0/3675351645 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f534405f5f0 con 0x7f5344100c60 2024-03-24T10:36:57.925 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:55.121+0000 7f5349c71700 1 --2- 172.21.15.94:0/3675351645 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f5330036790 0x7f5330038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:36:57.925 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:55.121+0000 7f5349c71700 1 --2- 172.21.15.94:0/3675351645 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f5330036790 0x7f5330038c40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5340009990 tx=0x7f5340006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:36:57.925 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:55.126+0000 7f533affd700 1 -- 172.21.15.94:0/3675351645 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f5334018b40 con 0x7f5344100c60 2024-03-24T10:36:57.925 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:55.315+0000 7f534c6d6700 1 -- 172.21.15.94:0/3675351645 --> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi094", "addr": "172.21.15.94", "target": ["mon-mgr", ""]}) v1 -- 0x7f53440fda10 con 0x7f5330036790 2024-03-24T10:36:57.925 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:57.341+0000 7f533affd700 1 -- 172.21.15.94:0/3675351645 <== mgr.14120 v2:172.21.15.94:6800/1996608546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f53440fda10 con 0x7f5330036790 2024-03-24T10:36:57.925 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:57.344+0000 7f534c6d6700 1 -- 172.21.15.94:0/3675351645 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f5330036790 msgr2=0x7f5330038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:36:57.925 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:57.344+0000 7f534c6d6700 1 --2- 172.21.15.94:0/3675351645 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f5330036790 0x7f5330038c40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5340009990 tx=0x7f5340006e40 comp rx=0 tx=0).stop 2024-03-24T10:36:57.925 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:57.344+0000 7f534c6d6700 1 -- 172.21.15.94:0/3675351645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5344100c60 msgr2=0x7f5344111710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:36:57.925 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:57.344+0000 7f534c6d6700 1 --2- 172.21.15.94:0/3675351645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5344100c60 0x7f5344111710 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5334006b00 tx=0x7f5334009250 comp rx=0 tx=0).stop 2024-03-24T10:36:57.925 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:57.344+0000 7f534c6d6700 1 -- 172.21.15.94:0/3675351645 shutdown_connections 2024-03-24T10:36:57.925 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:57.344+0000 7f534c6d6700 1 --2- 172.21.15.94:0/3675351645 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f5330036790 0x7f5330038c40 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:57.925 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:57.344+0000 7f534c6d6700 1 --2- 172.21.15.94:0/3675351645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5344100c60 0x7f5344111710 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:57.925 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:57.344+0000 7f534c6d6700 1 -- 172.21.15.94:0/3675351645 >> 172.21.15.94:0/3675351645 conn(0x7f53440fc6c0 msgr2=0x7f53440fd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:36:57.926 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:57.344+0000 7f534c6d6700 1 -- 172.21.15.94:0/3675351645 shutdown_connections 2024-03-24T10:36:57.926 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:57.344+0000 7f534c6d6700 1 -- 172.21.15.94:0/3675351645 wait complete. 2024-03-24T10:36:57.926 INFO:teuthology.orchestra.run.smithi094.stdout:Deploying mon service with default placement... 2024-03-24T10:36:58.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:58 smithi094 ceph-mon[29446]: from='mgr.14120 172.21.15.94:0/773982920' entity='mgr.smithi094.yaethu' 2024-03-24T10:36:58.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:58 smithi094 ceph-mon[29446]: Added host smithi094 2024-03-24T10:36:58.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:58 smithi094 ceph-mon[29446]: from='mgr.14120 172.21.15.94:0/773982920' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:36:59.221 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-24T10:36:59.222 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.323+0000 7fbb929d4700 1 Processor -- start 2024-03-24T10:36:59.222 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.324+0000 7fbb929d4700 1 -- start start 2024-03-24T10:36:59.222 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.324+0000 7fbb929d4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb8c1028f0 0x7fbb8c102d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:36:59.222 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.324+0000 7fbb929d4700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8c103240 con 0x7fbb8c1028f0 2024-03-24T10:36:59.222 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.324+0000 7fbb8bfff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb8c1028f0 0x7fbb8c102d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:36:59.222 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.324+0000 7fbb8bfff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb8c1028f0 0x7fbb8c102d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34580/0 (socket says 172.21.15.94:34580) 2024-03-24T10:36:59.222 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.324+0000 7fbb8bfff700 1 -- 172.21.15.94:0/2241575671 learned_addr learned my addr 172.21.15.94:0/2241575671 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:36:59.222 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.325+0000 7fbb8bfff700 1 -- 172.21.15.94:0/2241575671 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb8c103380 con 0x7fbb8c1028f0 2024-03-24T10:36:59.222 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.325+0000 7fbb8bfff700 1 --2- 172.21.15.94:0/2241575671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb8c1028f0 0x7fbb8c102d00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c0098e0 tx=0x7fbb7c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fce3addd1bcc7011 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:36:59.222 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.325+0000 7fbb8affd700 1 -- 172.21.15.94:0/2241575671 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbb7c00b860 con 0x7fbb8c1028f0 2024-03-24T10:36:59.222 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.325+0000 7fbb8affd700 1 -- 172.21.15.94:0/2241575671 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fbb7c00be50 con 0x7fbb8c1028f0 2024-03-24T10:36:59.222 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.325+0000 7fbb8affd700 1 -- 172.21.15.94:0/2241575671 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbb7c010380 con 0x7fbb8c1028f0 2024-03-24T10:36:59.222 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.326+0000 7fbb929d4700 1 -- 172.21.15.94:0/2241575671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb8c1028f0 msgr2=0x7fbb8c102d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:36:59.223 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.326+0000 7fbb929d4700 1 --2- 172.21.15.94:0/2241575671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb8c1028f0 0x7fbb8c102d00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c0098e0 tx=0x7fbb7c009250 comp rx=0 tx=0).stop 2024-03-24T10:36:59.223 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.326+0000 7fbb929d4700 1 -- 172.21.15.94:0/2241575671 shutdown_connections 2024-03-24T10:36:59.223 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.326+0000 7fbb929d4700 1 --2- 172.21.15.94:0/2241575671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb8c1028f0 0x7fbb8c102d00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:59.223 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.326+0000 7fbb929d4700 1 -- 172.21.15.94:0/2241575671 >> 172.21.15.94:0/2241575671 conn(0x7fbb8c0fe070 msgr2=0x7fbb8c100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:36:59.223 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.326+0000 7fbb929d4700 1 -- 172.21.15.94:0/2241575671 shutdown_connections 2024-03-24T10:36:59.223 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.326+0000 7fbb929d4700 1 -- 172.21.15.94:0/2241575671 wait complete. 2024-03-24T10:36:59.223 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.326+0000 7fbb929d4700 1 Processor -- start 2024-03-24T10:36:59.223 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.327+0000 7fbb929d4700 1 -- start start 2024-03-24T10:36:59.223 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.327+0000 7fbb929d4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb8c1028f0 0x7fbb8c111650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:36:59.223 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.327+0000 7fbb929d4700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8c111b90 con 0x7fbb8c1028f0 2024-03-24T10:36:59.223 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.327+0000 7fbb8bfff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb8c1028f0 0x7fbb8c111650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:36:59.223 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.327+0000 7fbb8bfff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb8c1028f0 0x7fbb8c111650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34590/0 (socket says 172.21.15.94:34590) 2024-03-24T10:36:59.223 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.327+0000 7fbb8bfff700 1 -- 172.21.15.94:0/1190671407 learned_addr learned my addr 172.21.15.94:0/1190671407 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:36:59.224 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.327+0000 7fbb8bfff700 1 -- 172.21.15.94:0/1190671407 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb7c0094f0 con 0x7fbb8c1028f0 2024-03-24T10:36:59.224 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.327+0000 7fbb8bfff700 1 --2- 172.21.15.94:0/1190671407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb8c1028f0 0x7fbb8c111650 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c000c00 tx=0x7fbb7c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:36:59.224 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.327+0000 7fbb88ff9700 1 -- 172.21.15.94:0/1190671407 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbb7c0107c0 con 0x7fbb8c1028f0 2024-03-24T10:36:59.224 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.328+0000 7fbb88ff9700 1 -- 172.21.15.94:0/1190671407 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fbb7c010db0 con 0x7fbb8c1028f0 2024-03-24T10:36:59.224 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.328+0000 7fbb929d4700 1 -- 172.21.15.94:0/1190671407 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb8c111d90 con 0x7fbb8c1028f0 2024-03-24T10:36:59.224 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.328+0000 7fbb88ff9700 1 -- 172.21.15.94:0/1190671407 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbb7c01ad00 con 0x7fbb8c1028f0 2024-03-24T10:36:59.224 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.328+0000 7fbb929d4700 1 -- 172.21.15.94:0/1190671407 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb8c112900 con 0x7fbb8c1028f0 2024-03-24T10:36:59.224 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.328+0000 7fbb88ff9700 1 -- 172.21.15.94:0/1190671407 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fbb7c0193c0 con 0x7fbb8c1028f0 2024-03-24T10:36:59.224 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.329+0000 7fbb88ff9700 1 --2- 172.21.15.94:0/1190671407 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7fbb740367e0 0x7fbb74038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:36:59.224 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.329+0000 7fbb88ff9700 1 -- 172.21.15.94:0/1190671407 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fbb7c04b5b0 con 0x7fbb8c1028f0 2024-03-24T10:36:59.224 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.329+0000 7fbb929d4700 1 -- 172.21.15.94:0/1190671407 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb8c05f5f0 con 0x7fbb8c1028f0 2024-03-24T10:36:59.224 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.329+0000 7fbb8b7fe700 1 --2- 172.21.15.94:0/1190671407 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7fbb740367e0 0x7fbb74038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:36:59.224 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.329+0000 7fbb8b7fe700 1 --2- 172.21.15.94:0/1190671407 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7fbb740367e0 0x7fbb74038c90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbb80009990 tx=0x7fbb80006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:36:59.224 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.334+0000 7fbb88ff9700 1 -- 172.21.15.94:0/1190671407 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fbb7c0196c0 con 0x7fbb8c1028f0 2024-03-24T10:36:59.225 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.541+0000 7fbb929d4700 1 -- 172.21.15.94:0/1190671407 --> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb8c0ff240 con 0x7fbb740367e0 2024-03-24T10:36:59.225 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.545+0000 7fbb88ff9700 1 -- 172.21.15.94:0/1190671407 <== mgr.14120 v2:172.21.15.94:6800/1996608546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fbb8c0ff240 con 0x7fbb740367e0 2024-03-24T10:36:59.225 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.547+0000 7fbb929d4700 1 -- 172.21.15.94:0/1190671407 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7fbb740367e0 msgr2=0x7fbb74038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:36:59.225 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.547+0000 7fbb929d4700 1 --2- 172.21.15.94:0/1190671407 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7fbb740367e0 0x7fbb74038c90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbb80009990 tx=0x7fbb80006e40 comp rx=0 tx=0).stop 2024-03-24T10:36:59.225 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.547+0000 7fbb929d4700 1 -- 172.21.15.94:0/1190671407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb8c1028f0 msgr2=0x7fbb8c111650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:36:59.225 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.547+0000 7fbb929d4700 1 --2- 172.21.15.94:0/1190671407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb8c1028f0 0x7fbb8c111650 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c000c00 tx=0x7fbb7c009250 comp rx=0 tx=0).stop 2024-03-24T10:36:59.225 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.547+0000 7fbb929d4700 1 -- 172.21.15.94:0/1190671407 shutdown_connections 2024-03-24T10:36:59.225 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.547+0000 7fbb929d4700 1 --2- 172.21.15.94:0/1190671407 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7fbb740367e0 0x7fbb74038c90 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:59.225 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.547+0000 7fbb929d4700 1 --2- 172.21.15.94:0/1190671407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb8c1028f0 0x7fbb8c111650 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:36:59.225 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.547+0000 7fbb929d4700 1 -- 172.21.15.94:0/1190671407 >> 172.21.15.94:0/1190671407 conn(0x7fbb8c0fe070 msgr2=0x7fbb8c0feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:36:59.225 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.547+0000 7fbb929d4700 1 -- 172.21.15.94:0/1190671407 shutdown_connections 2024-03-24T10:36:59.225 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:58.547+0000 7fbb929d4700 1 -- 172.21.15.94:0/1190671407 wait complete. 2024-03-24T10:36:59.225 INFO:teuthology.orchestra.run.smithi094.stdout:Deploying mgr service with default placement... 2024-03-24T10:36:59.798 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:59 smithi094 ceph-mon[29446]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:36:59.798 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:59 smithi094 ceph-mon[29446]: Saving service mon spec with placement count:5 2024-03-24T10:36:59.798 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:36:59 smithi094 ceph-mon[29446]: from='mgr.14120 172.21.15.94:0/773982920' entity='mgr.smithi094.yaethu' 2024-03-24T10:37:00.757 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-24T10:37:00.757 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.628+0000 7f8ea9e21700 1 Processor -- start 2024-03-24T10:37:00.757 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.628+0000 7f8ea9e21700 1 -- start start 2024-03-24T10:37:00.757 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.628+0000 7f8ea9e21700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ea4071230 0x7f8ea4107960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:00.757 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.628+0000 7f8ea9e21700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea4071640 con 0x7f8ea4071230 2024-03-24T10:37:00.757 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.629+0000 7f8ea37fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ea4071230 0x7f8ea4107960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:00.757 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.629+0000 7f8ea37fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ea4071230 0x7f8ea4107960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56580/0 (socket says 172.21.15.94:56580) 2024-03-24T10:37:00.757 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.629+0000 7f8ea37fe700 1 -- 172.21.15.94:0/2548499017 learned_addr learned my addr 172.21.15.94:0/2548499017 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:37:00.757 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.629+0000 7f8ea37fe700 1 -- 172.21.15.94:0/2548499017 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea4103350 con 0x7f8ea4071230 2024-03-24T10:37:00.757 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.629+0000 7f8ea37fe700 1 --2- 172.21.15.94:0/2548499017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ea4071230 0x7f8ea4107960 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8e94009e50 tx=0x7f8e94009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=662621870a2f770c server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:00.757 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.629+0000 7f8ea27fc700 1 -- 172.21.15.94:0/2548499017 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8e940092e0 con 0x7f8ea4071230 2024-03-24T10:37:00.757 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.629+0000 7f8ea27fc700 1 -- 172.21.15.94:0/2548499017 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f8e94012430 con 0x7f8ea4071230 2024-03-24T10:37:00.758 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.629+0000 7f8ea27fc700 1 -- 172.21.15.94:0/2548499017 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8e940075c0 con 0x7f8ea4071230 2024-03-24T10:37:00.758 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.630+0000 7f8ea9e21700 1 -- 172.21.15.94:0/2548499017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ea4071230 msgr2=0x7f8ea4107960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:00.758 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.630+0000 7f8ea9e21700 1 --2- 172.21.15.94:0/2548499017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ea4071230 0x7f8ea4107960 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8e94009e50 tx=0x7f8e94009840 comp rx=0 tx=0).stop 2024-03-24T10:37:00.758 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.630+0000 7f8ea9e21700 1 -- 172.21.15.94:0/2548499017 shutdown_connections 2024-03-24T10:37:00.758 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.630+0000 7f8ea9e21700 1 --2- 172.21.15.94:0/2548499017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ea4071230 0x7f8ea4107960 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:00.758 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.630+0000 7f8ea9e21700 1 -- 172.21.15.94:0/2548499017 >> 172.21.15.94:0/2548499017 conn(0x7f8ea40fcfb0 msgr2=0x7f8ea40ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:37:00.758 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.630+0000 7f8ea9e21700 1 -- 172.21.15.94:0/2548499017 shutdown_connections 2024-03-24T10:37:00.758 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.630+0000 7f8ea9e21700 1 -- 172.21.15.94:0/2548499017 wait complete. 2024-03-24T10:37:00.758 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.630+0000 7f8ea9e21700 1 Processor -- start 2024-03-24T10:37:00.758 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.631+0000 7f8ea9e21700 1 -- start start 2024-03-24T10:37:00.758 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.631+0000 7f8ea9e21700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ea4071230 0x7f8ea4113900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:00.758 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.631+0000 7f8ea9e21700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea4113e40 con 0x7f8ea4071230 2024-03-24T10:37:00.758 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.631+0000 7f8ea37fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ea4071230 0x7f8ea4113900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:00.758 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.631+0000 7f8ea37fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ea4071230 0x7f8ea4113900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56586/0 (socket says 172.21.15.94:56586) 2024-03-24T10:37:00.758 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.631+0000 7f8ea37fe700 1 -- 172.21.15.94:0/3360324879 learned_addr learned my addr 172.21.15.94:0/3360324879 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:37:00.759 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.631+0000 7f8ea37fe700 1 -- 172.21.15.94:0/3360324879 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e94009ae0 con 0x7f8ea4071230 2024-03-24T10:37:00.759 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.631+0000 7f8ea37fe700 1 --2- 172.21.15.94:0/3360324879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ea4071230 0x7f8ea4113900 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8e94012da0 tx=0x7f8e94009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:00.759 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.631+0000 7f8e8bfff700 1 -- 172.21.15.94:0/3360324879 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8e940092e0 con 0x7f8ea4071230 2024-03-24T10:37:00.759 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.631+0000 7f8ea9e21700 1 -- 172.21.15.94:0/3360324879 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ea4114040 con 0x7f8ea4071230 2024-03-24T10:37:00.759 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.631+0000 7f8e8bfff700 1 -- 172.21.15.94:0/3360324879 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f8e94007d10 con 0x7f8ea4071230 2024-03-24T10:37:00.759 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.631+0000 7f8e8bfff700 1 -- 172.21.15.94:0/3360324879 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8e9401c910 con 0x7f8ea4071230 2024-03-24T10:37:00.759 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.631+0000 7f8ea9e21700 1 -- 172.21.15.94:0/3360324879 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ea4114bb0 con 0x7f8ea4071230 2024-03-24T10:37:00.759 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.632+0000 7f8e8bfff700 1 -- 172.21.15.94:0/3360324879 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f8e94019070 con 0x7f8ea4071230 2024-03-24T10:37:00.759 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.632+0000 7f8ea9e21700 1 -- 172.21.15.94:0/3360324879 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ea405f5f0 con 0x7f8ea4071230 2024-03-24T10:37:00.759 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.633+0000 7f8e8bfff700 1 --2- 172.21.15.94:0/3360324879 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f8e8c036790 0x7f8e8c038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:00.759 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.633+0000 7f8e8bfff700 1 -- 172.21.15.94:0/3360324879 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f8e94049ef0 con 0x7f8ea4071230 2024-03-24T10:37:00.759 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.633+0000 7f8ea2ffd700 1 --2- 172.21.15.94:0/3360324879 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f8e8c036790 0x7f8e8c038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:00.759 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.633+0000 7f8ea2ffd700 1 --2- 172.21.15.94:0/3360324879 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f8e8c036790 0x7f8e8c038c40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8e98009990 tx=0x7f8e98006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:00.760 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.637+0000 7f8e8bfff700 1 -- 172.21.15.94:0/3360324879 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f8e94019910 con 0x7f8ea4071230 2024-03-24T10:37:00.760 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.837+0000 7f8ea9e21700 1 -- 172.21.15.94:0/3360324879 --> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ea40fe420 con 0x7f8e8c036790 2024-03-24T10:37:00.760 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.842+0000 7f8e8bfff700 1 -- 172.21.15.94:0/3360324879 <== mgr.14120 v2:172.21.15.94:6800/1996608546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f8ea40fe420 con 0x7f8e8c036790 2024-03-24T10:37:00.761 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.844+0000 7f8ea9e21700 1 -- 172.21.15.94:0/3360324879 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f8e8c036790 msgr2=0x7f8e8c038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:00.761 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.844+0000 7f8ea9e21700 1 --2- 172.21.15.94:0/3360324879 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f8e8c036790 0x7f8e8c038c40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8e98009990 tx=0x7f8e98006e40 comp rx=0 tx=0).stop 2024-03-24T10:37:00.761 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.844+0000 7f8ea9e21700 1 -- 172.21.15.94:0/3360324879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ea4071230 msgr2=0x7f8ea4113900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:00.761 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.844+0000 7f8ea9e21700 1 --2- 172.21.15.94:0/3360324879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ea4071230 0x7f8ea4113900 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8e94012da0 tx=0x7f8e94009840 comp rx=0 tx=0).stop 2024-03-24T10:37:00.761 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.844+0000 7f8ea9e21700 1 -- 172.21.15.94:0/3360324879 shutdown_connections 2024-03-24T10:37:00.762 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.844+0000 7f8ea9e21700 1 --2- 172.21.15.94:0/3360324879 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f8e8c036790 0x7f8e8c038c40 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:00.762 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.844+0000 7f8ea9e21700 1 --2- 172.21.15.94:0/3360324879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ea4071230 0x7f8ea4113900 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:00.762 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.844+0000 7f8ea9e21700 1 -- 172.21.15.94:0/3360324879 >> 172.21.15.94:0/3360324879 conn(0x7f8ea40fcfb0 msgr2=0x7f8ea40fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:37:00.762 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.844+0000 7f8ea9e21700 1 -- 172.21.15.94:0/3360324879 shutdown_connections 2024-03-24T10:37:00.762 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:36:59.845+0000 7f8ea9e21700 1 -- 172.21.15.94:0/3360324879 wait complete. 2024-03-24T10:37:00.762 INFO:teuthology.orchestra.run.smithi094.stdout:Deploying crash service with default placement... 2024-03-24T10:37:01.122 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:00 smithi094 ceph-mon[29446]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:37:01.122 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:00 smithi094 ceph-mon[29446]: Saving service mgr spec with placement count:2 2024-03-24T10:37:01.122 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:00 smithi094 ceph-mon[29446]: from='mgr.14120 172.21.15.94:0/773982920' entity='mgr.smithi094.yaethu' 2024-03-24T10:37:02.074 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-03-24T10:37:02.074 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.122+0000 7fb301df7700 1 Processor -- start 2024-03-24T10:37:02.074 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.122+0000 7fb301df7700 1 -- start start 2024-03-24T10:37:02.074 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.122+0000 7fb301df7700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2fc1028b0 0x7fb2fc102cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:02.074 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.122+0000 7fb301df7700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2fc103290 con 0x7fb2fc1028b0 2024-03-24T10:37:02.074 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.122+0000 7fb2fb7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2fc1028b0 0x7fb2fc102cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:02.074 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.122+0000 7fb2fb7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2fc1028b0 0x7fb2fc102cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56602/0 (socket says 172.21.15.94:56602) 2024-03-24T10:37:02.074 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.122+0000 7fb2fb7fe700 1 -- 172.21.15.94:0/538977622 learned_addr learned my addr 172.21.15.94:0/538977622 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:37:02.074 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.122+0000 7fb2fb7fe700 1 -- 172.21.15.94:0/538977622 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2fc1033d0 con 0x7fb2fc1028b0 2024-03-24T10:37:02.075 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.123+0000 7fb2fb7fe700 1 --2- 172.21.15.94:0/538977622 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2fc1028b0 0x7fb2fc102cc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec0098e0 tx=0x7fb2ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9f93e63acfbf08 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:02.075 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.123+0000 7fb2fa7fc700 1 -- 172.21.15.94:0/538977622 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb2ec00b860 con 0x7fb2fc1028b0 2024-03-24T10:37:02.075 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.123+0000 7fb2fa7fc700 1 -- 172.21.15.94:0/538977622 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fb2ec00be50 con 0x7fb2fc1028b0 2024-03-24T10:37:02.075 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.123+0000 7fb2fa7fc700 1 -- 172.21.15.94:0/538977622 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb2ec010380 con 0x7fb2fc1028b0 2024-03-24T10:37:02.075 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.123+0000 7fb301df7700 1 -- 172.21.15.94:0/538977622 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2fc1028b0 msgr2=0x7fb2fc102cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:02.075 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.123+0000 7fb301df7700 1 --2- 172.21.15.94:0/538977622 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2fc1028b0 0x7fb2fc102cc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec0098e0 tx=0x7fb2ec009250 comp rx=0 tx=0).stop 2024-03-24T10:37:02.075 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.124+0000 7fb301df7700 1 -- 172.21.15.94:0/538977622 shutdown_connections 2024-03-24T10:37:02.075 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.124+0000 7fb301df7700 1 --2- 172.21.15.94:0/538977622 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2fc1028b0 0x7fb2fc102cc0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:02.075 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.124+0000 7fb301df7700 1 -- 172.21.15.94:0/538977622 >> 172.21.15.94:0/538977622 conn(0x7fb2fc0fdf90 msgr2=0x7fb2fc100400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:37:02.075 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.124+0000 7fb301df7700 1 -- 172.21.15.94:0/538977622 shutdown_connections 2024-03-24T10:37:02.075 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.124+0000 7fb301df7700 1 -- 172.21.15.94:0/538977622 wait complete. 2024-03-24T10:37:02.075 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.124+0000 7fb301df7700 1 Processor -- start 2024-03-24T10:37:02.075 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.124+0000 7fb301df7700 1 -- start start 2024-03-24T10:37:02.075 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.124+0000 7fb301df7700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2fc1028b0 0x7fb2fc078c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:02.075 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.124+0000 7fb301df7700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2fc079180 con 0x7fb2fc1028b0 2024-03-24T10:37:02.076 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.124+0000 7fb2fb7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2fc1028b0 0x7fb2fc078c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:02.076 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.125+0000 7fb2fb7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2fc1028b0 0x7fb2fc078c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56612/0 (socket says 172.21.15.94:56612) 2024-03-24T10:37:02.076 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.125+0000 7fb2fb7fe700 1 -- 172.21.15.94:0/300568885 learned_addr learned my addr 172.21.15.94:0/300568885 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:37:02.076 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.125+0000 7fb2fb7fe700 1 -- 172.21.15.94:0/300568885 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2ec0094f0 con 0x7fb2fc1028b0 2024-03-24T10:37:02.076 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.125+0000 7fb2fb7fe700 1 --2- 172.21.15.94:0/300568885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2fc1028b0 0x7fb2fc078c40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec00b420 tx=0x7fb2ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:02.076 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.125+0000 7fb2e3fff700 1 -- 172.21.15.94:0/300568885 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb2ec010600 con 0x7fb2fc1028b0 2024-03-24T10:37:02.076 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.125+0000 7fb301df7700 1 -- 172.21.15.94:0/300568885 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2fc0771f0 con 0x7fb2fc1028b0 2024-03-24T10:37:02.076 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.125+0000 7fb2e3fff700 1 -- 172.21.15.94:0/300568885 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fb2ec010bf0 con 0x7fb2fc1028b0 2024-03-24T10:37:02.076 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.125+0000 7fb2e3fff700 1 -- 172.21.15.94:0/300568885 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb2ec01acb0 con 0x7fb2fc1028b0 2024-03-24T10:37:02.076 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.125+0000 7fb301df7700 1 -- 172.21.15.94:0/300568885 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2fc0776e0 con 0x7fb2fc1028b0 2024-03-24T10:37:02.076 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.126+0000 7fb2e3fff700 1 -- 172.21.15.94:0/300568885 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fb2ec02d3f0 con 0x7fb2fc1028b0 2024-03-24T10:37:02.076 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.126+0000 7fb2e3fff700 1 --2- 172.21.15.94:0/300568885 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7fb2e4036790 0x7fb2e4038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:02.076 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.126+0000 7fb2e3fff700 1 -- 172.21.15.94:0/300568885 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fb2ec04b0a0 con 0x7fb2fc1028b0 2024-03-24T10:37:02.076 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.126+0000 7fb2faffd700 1 --2- 172.21.15.94:0/300568885 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7fb2e4036790 0x7fb2e4038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:02.076 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.126+0000 7fb301df7700 1 -- 172.21.15.94:0/300568885 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2fc04bed0 con 0x7fb2fc1028b0 2024-03-24T10:37:02.077 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.126+0000 7fb2faffd700 1 --2- 172.21.15.94:0/300568885 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7fb2e4036790 0x7fb2e4038c40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb2f0009990 tx=0x7fb2f0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:02.077 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.131+0000 7fb2e3fff700 1 -- 172.21.15.94:0/300568885 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fb2ec047080 con 0x7fb2fc1028b0 2024-03-24T10:37:02.077 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.329+0000 7fb301df7700 1 -- 172.21.15.94:0/300568885 --> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7fb2fc077d70 con 0x7fb2e4036790 2024-03-24T10:37:02.077 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.333+0000 7fb2e3fff700 1 -- 172.21.15.94:0/300568885 <== mgr.14120 v2:172.21.15.94:6800/1996608546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7fb2fc077d70 con 0x7fb2e4036790 2024-03-24T10:37:02.077 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.335+0000 7fb301df7700 1 -- 172.21.15.94:0/300568885 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7fb2e4036790 msgr2=0x7fb2e4038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:02.077 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.335+0000 7fb301df7700 1 --2- 172.21.15.94:0/300568885 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7fb2e4036790 0x7fb2e4038c40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb2f0009990 tx=0x7fb2f0006e40 comp rx=0 tx=0).stop 2024-03-24T10:37:02.077 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.335+0000 7fb301df7700 1 -- 172.21.15.94:0/300568885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2fc1028b0 msgr2=0x7fb2fc078c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:02.077 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.335+0000 7fb301df7700 1 --2- 172.21.15.94:0/300568885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2fc1028b0 0x7fb2fc078c40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec00b420 tx=0x7fb2ec009250 comp rx=0 tx=0).stop 2024-03-24T10:37:02.077 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.335+0000 7fb301df7700 1 -- 172.21.15.94:0/300568885 shutdown_connections 2024-03-24T10:37:02.077 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.335+0000 7fb301df7700 1 --2- 172.21.15.94:0/300568885 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7fb2e4036790 0x7fb2e4038c40 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:02.077 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.335+0000 7fb301df7700 1 --2- 172.21.15.94:0/300568885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2fc1028b0 0x7fb2fc078c40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:02.077 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.335+0000 7fb301df7700 1 -- 172.21.15.94:0/300568885 >> 172.21.15.94:0/300568885 conn(0x7fb2fc0fdf90 msgr2=0x7fb2fc0feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:37:02.077 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.335+0000 7fb301df7700 1 -- 172.21.15.94:0/300568885 shutdown_connections 2024-03-24T10:37:02.077 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:01.335+0000 7fb301df7700 1 -- 172.21.15.94:0/300568885 wait complete. 2024-03-24T10:37:02.077 INFO:teuthology.orchestra.run.smithi094.stdout:Deploying prometheus service with default placement... 2024-03-24T10:37:02.648 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:02 smithi094 ceph-mon[29446]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:37:02.648 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:02 smithi094 ceph-mon[29446]: Saving service crash spec with placement * 2024-03-24T10:37:02.649 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:02 smithi094 ceph-mon[29446]: from='mgr.14120 172.21.15.94:0/773982920' entity='mgr.smithi094.yaethu' 2024-03-24T10:37:02.649 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:02 smithi094 ceph-mon[29446]: from='mgr.14120 172.21.15.94:0/773982920' entity='mgr.smithi094.yaethu' 2024-03-24T10:37:02.649 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:02 smithi094 ceph-mon[29446]: from='mgr.14120 172.21.15.94:0/773982920' entity='mgr.smithi094.yaethu' 2024-03-24T10:37:03.656 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-03-24T10:37:03.656 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.420+0000 7fd4f21b2700 1 Processor -- start 2024-03-24T10:37:03.656 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.420+0000 7fd4f21b2700 1 -- start start 2024-03-24T10:37:03.656 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.421+0000 7fd4f21b2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4ec101750 0x7fd4ec101b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:03.656 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.421+0000 7fd4f21b2700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ec1020a0 con 0x7fd4ec101750 2024-03-24T10:37:03.656 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.421+0000 7fd4eb7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4ec101750 0x7fd4ec101b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:03.656 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.421+0000 7fd4eb7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4ec101750 0x7fd4ec101b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56626/0 (socket says 172.21.15.94:56626) 2024-03-24T10:37:03.657 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.421+0000 7fd4eb7fe700 1 -- 172.21.15.94:0/254865949 learned_addr learned my addr 172.21.15.94:0/254865949 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:37:03.657 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.421+0000 7fd4eb7fe700 1 -- 172.21.15.94:0/254865949 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ec1021e0 con 0x7fd4ec101750 2024-03-24T10:37:03.657 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.421+0000 7fd4eb7fe700 1 --2- 172.21.15.94:0/254865949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4ec101750 0x7fd4ec101b60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc0098e0 tx=0x7fd4dc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=254f33e3d07313d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:03.657 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.421+0000 7fd4eaffd700 1 -- 172.21.15.94:0/254865949 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd4dc00b860 con 0x7fd4ec101750 2024-03-24T10:37:03.657 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.421+0000 7fd4eaffd700 1 -- 172.21.15.94:0/254865949 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd4dc00be90 con 0x7fd4ec101750 2024-03-24T10:37:03.657 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.421+0000 7fd4eaffd700 1 -- 172.21.15.94:0/254865949 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd4dc0103c0 con 0x7fd4ec101750 2024-03-24T10:37:03.657 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.422+0000 7fd4f21b2700 1 -- 172.21.15.94:0/254865949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4ec101750 msgr2=0x7fd4ec101b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:03.657 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.422+0000 7fd4f21b2700 1 --2- 172.21.15.94:0/254865949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4ec101750 0x7fd4ec101b60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc0098e0 tx=0x7fd4dc009250 comp rx=0 tx=0).stop 2024-03-24T10:37:03.657 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.422+0000 7fd4f21b2700 1 -- 172.21.15.94:0/254865949 shutdown_connections 2024-03-24T10:37:03.657 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.422+0000 7fd4f21b2700 1 --2- 172.21.15.94:0/254865949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4ec101750 0x7fd4ec101b60 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:03.657 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.422+0000 7fd4f21b2700 1 -- 172.21.15.94:0/254865949 >> 172.21.15.94:0/254865949 conn(0x7fd4ec0fcf30 msgr2=0x7fd4ec0ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:37:03.657 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.422+0000 7fd4f21b2700 1 -- 172.21.15.94:0/254865949 shutdown_connections 2024-03-24T10:37:03.657 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.422+0000 7fd4f21b2700 1 -- 172.21.15.94:0/254865949 wait complete. 2024-03-24T10:37:03.657 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.422+0000 7fd4f21b2700 1 Processor -- start 2024-03-24T10:37:03.658 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.423+0000 7fd4f21b2700 1 -- start start 2024-03-24T10:37:03.658 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.423+0000 7fd4f21b2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4ec101750 0x7fd4ec113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:03.658 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.423+0000 7fd4f21b2700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ec113e70 con 0x7fd4ec101750 2024-03-24T10:37:03.658 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.423+0000 7fd4eb7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4ec101750 0x7fd4ec113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:03.658 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.423+0000 7fd4eb7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4ec101750 0x7fd4ec113930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56638/0 (socket says 172.21.15.94:56638) 2024-03-24T10:37:03.658 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.423+0000 7fd4eb7fe700 1 -- 172.21.15.94:0/2098211728 learned_addr learned my addr 172.21.15.94:0/2098211728 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:37:03.658 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.423+0000 7fd4eb7fe700 1 -- 172.21.15.94:0/2098211728 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4dc0094f0 con 0x7fd4ec101750 2024-03-24T10:37:03.658 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.423+0000 7fd4eb7fe700 1 --2- 172.21.15.94:0/2098211728 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4ec101750 0x7fd4ec113930 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc011c10 tx=0x7fd4dc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:03.658 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.423+0000 7fd4e8ff9700 1 -- 172.21.15.94:0/2098211728 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd4dc010890 con 0x7fd4ec101750 2024-03-24T10:37:03.658 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.423+0000 7fd4e8ff9700 1 -- 172.21.15.94:0/2098211728 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd4dc010ec0 con 0x7fd4ec101750 2024-03-24T10:37:03.658 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.423+0000 7fd4f21b2700 1 -- 172.21.15.94:0/2098211728 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4ec114070 con 0x7fd4ec101750 2024-03-24T10:37:03.658 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.423+0000 7fd4e8ff9700 1 -- 172.21.15.94:0/2098211728 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd4dc0243f0 con 0x7fd4ec101750 2024-03-24T10:37:03.658 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.423+0000 7fd4f21b2700 1 -- 172.21.15.94:0/2098211728 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4ec114be0 con 0x7fd4ec101750 2024-03-24T10:37:03.658 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.424+0000 7fd4e8ff9700 1 -- 172.21.15.94:0/2098211728 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fd4dc0109f0 con 0x7fd4ec101750 2024-03-24T10:37:03.658 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.424+0000 7fd4f21b2700 1 -- 172.21.15.94:0/2098211728 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4ec05f5f0 con 0x7fd4ec101750 2024-03-24T10:37:03.659 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.424+0000 7fd4e8ff9700 1 --2- 172.21.15.94:0/2098211728 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7fd4d80367e0 0x7fd4d8038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:03.659 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.424+0000 7fd4e8ff9700 1 -- 172.21.15.94:0/2098211728 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fd4dc04a190 con 0x7fd4ec101750 2024-03-24T10:37:03.659 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.425+0000 7fd4e2ffd700 1 --2- 172.21.15.94:0/2098211728 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7fd4d80367e0 0x7fd4d8038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:03.659 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.425+0000 7fd4e2ffd700 1 --2- 172.21.15.94:0/2098211728 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7fd4d80367e0 0x7fd4d8038c90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4009990 tx=0x7fd4d4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:03.659 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.429+0000 7fd4e8ff9700 1 -- 172.21.15.94:0/2098211728 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fd4dc02b4d0 con 0x7fd4ec101750 2024-03-24T10:37:03.659 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.648+0000 7fd4f21b2700 1 -- 172.21.15.94:0/2098211728 --> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7fd4ec115760 con 0x7fd4d80367e0 2024-03-24T10:37:03.659 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.653+0000 7fd4e8ff9700 1 -- 172.21.15.94:0/2098211728 <== mgr.14120 v2:172.21.15.94:6800/1996608546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7fd4ec115760 con 0x7fd4d80367e0 2024-03-24T10:37:03.659 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.654+0000 7fd4f21b2700 1 -- 172.21.15.94:0/2098211728 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7fd4d80367e0 msgr2=0x7fd4d8038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:03.659 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.654+0000 7fd4f21b2700 1 --2- 172.21.15.94:0/2098211728 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7fd4d80367e0 0x7fd4d8038c90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4009990 tx=0x7fd4d4006e40 comp rx=0 tx=0).stop 2024-03-24T10:37:03.659 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.655+0000 7fd4f21b2700 1 -- 172.21.15.94:0/2098211728 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4ec101750 msgr2=0x7fd4ec113930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:03.659 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.655+0000 7fd4f21b2700 1 --2- 172.21.15.94:0/2098211728 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4ec101750 0x7fd4ec113930 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc011c10 tx=0x7fd4dc009250 comp rx=0 tx=0).stop 2024-03-24T10:37:03.659 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.655+0000 7fd4f21b2700 1 -- 172.21.15.94:0/2098211728 shutdown_connections 2024-03-24T10:37:03.659 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.655+0000 7fd4f21b2700 1 --2- 172.21.15.94:0/2098211728 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7fd4d80367e0 0x7fd4d8038c90 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:03.659 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.655+0000 7fd4f21b2700 1 --2- 172.21.15.94:0/2098211728 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4ec101750 0x7fd4ec113930 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:03.660 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.655+0000 7fd4f21b2700 1 -- 172.21.15.94:0/2098211728 >> 172.21.15.94:0/2098211728 conn(0x7fd4ec0fcf30 msgr2=0x7fd4ec0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:37:03.660 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.655+0000 7fd4f21b2700 1 -- 172.21.15.94:0/2098211728 shutdown_connections 2024-03-24T10:37:03.660 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:02.655+0000 7fd4f21b2700 1 -- 172.21.15.94:0/2098211728 wait complete. 2024-03-24T10:37:03.660 INFO:teuthology.orchestra.run.smithi094.stdout:Deploying grafana service with default placement... 2024-03-24T10:37:03.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:03 smithi094 ceph-mon[29446]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:37:03.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:03 smithi094 ceph-mon[29446]: Saving service prometheus spec with placement count:1 2024-03-24T10:37:03.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:03 smithi094 ceph-mon[29446]: from='mgr.14120 172.21.15.94:0/773982920' entity='mgr.smithi094.yaethu' 2024-03-24T10:37:04.839 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-03-24T10:37:04.839 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.072+0000 7f2c4c6db700 1 Processor -- start 2024-03-24T10:37:04.839 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.072+0000 7f2c4c6db700 1 -- start start 2024-03-24T10:37:04.839 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.072+0000 7f2c4c6db700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c44101dc0 0x7f2c441021d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:04.839 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.072+0000 7f2c4c6db700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c44102710 con 0x7f2c44101dc0 2024-03-24T10:37:04.839 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.073+0000 7f2c4a477700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c44101dc0 0x7f2c441021d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:04.840 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.073+0000 7f2c4a477700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c44101dc0 0x7f2c441021d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56650/0 (socket says 172.21.15.94:56650) 2024-03-24T10:37:04.840 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.073+0000 7f2c4a477700 1 -- 172.21.15.94:0/305564254 learned_addr learned my addr 172.21.15.94:0/305564254 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:37:04.840 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.073+0000 7f2c4a477700 1 -- 172.21.15.94:0/305564254 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c44102850 con 0x7f2c44101dc0 2024-03-24T10:37:04.840 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.073+0000 7f2c4a477700 1 --2- 172.21.15.94:0/305564254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c44101dc0 0x7f2c441021d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c00bb80 tx=0x7f2c3c00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c6bbf1959f0af45 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:04.840 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.073+0000 7f2c49475700 1 -- 172.21.15.94:0/305564254 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2c3c00dd00 con 0x7f2c44101dc0 2024-03-24T10:37:04.840 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.073+0000 7f2c49475700 1 -- 172.21.15.94:0/305564254 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2c3c00de60 con 0x7f2c44101dc0 2024-03-24T10:37:04.840 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.073+0000 7f2c49475700 1 -- 172.21.15.94:0/305564254 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2c3c018920 con 0x7f2c44101dc0 2024-03-24T10:37:04.840 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.074+0000 7f2c4c6db700 1 -- 172.21.15.94:0/305564254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c44101dc0 msgr2=0x7f2c441021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:04.840 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.074+0000 7f2c4c6db700 1 --2- 172.21.15.94:0/305564254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c44101dc0 0x7f2c441021d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c00bb80 tx=0x7f2c3c00b4f0 comp rx=0 tx=0).stop 2024-03-24T10:37:04.840 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.074+0000 7f2c4c6db700 1 -- 172.21.15.94:0/305564254 shutdown_connections 2024-03-24T10:37:04.840 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.074+0000 7f2c4c6db700 1 --2- 172.21.15.94:0/305564254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c44101dc0 0x7f2c441021d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:04.840 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.074+0000 7f2c4c6db700 1 -- 172.21.15.94:0/305564254 >> 172.21.15.94:0/305564254 conn(0x7f2c440fc750 msgr2=0x7f2c440feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:37:04.840 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.074+0000 7f2c4c6db700 1 -- 172.21.15.94:0/305564254 shutdown_connections 2024-03-24T10:37:04.840 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.074+0000 7f2c4c6db700 1 -- 172.21.15.94:0/305564254 wait complete. 2024-03-24T10:37:04.841 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.074+0000 7f2c4c6db700 1 Processor -- start 2024-03-24T10:37:04.841 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.074+0000 7f2c4c6db700 1 -- start start 2024-03-24T10:37:04.841 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.075+0000 7f2c4c6db700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c44101dc0 0x7f2c4410f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:04.841 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.075+0000 7f2c4c6db700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4410f990 con 0x7f2c44101dc0 2024-03-24T10:37:04.841 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.075+0000 7f2c4a477700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c44101dc0 0x7f2c4410f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:04.841 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.075+0000 7f2c4a477700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c44101dc0 0x7f2c4410f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56662/0 (socket says 172.21.15.94:56662) 2024-03-24T10:37:04.841 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.075+0000 7f2c4a477700 1 -- 172.21.15.94:0/1802489658 learned_addr learned my addr 172.21.15.94:0/1802489658 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:37:04.841 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.075+0000 7f2c4a477700 1 -- 172.21.15.94:0/1802489658 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c3c00b790 con 0x7f2c44101dc0 2024-03-24T10:37:04.841 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.075+0000 7f2c4a477700 1 --2- 172.21.15.94:0/1802489658 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c44101dc0 0x7f2c4410f450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c006b00 tx=0x7f2c3c00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:04.841 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.075+0000 7f2c3affd700 1 -- 172.21.15.94:0/1802489658 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2c3c010bd0 con 0x7f2c44101dc0 2024-03-24T10:37:04.841 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.075+0000 7f2c3affd700 1 -- 172.21.15.94:0/1802489658 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2c3c01d050 con 0x7f2c44101dc0 2024-03-24T10:37:04.841 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.075+0000 7f2c4c6db700 1 -- 172.21.15.94:0/1802489658 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c4410fb90 con 0x7f2c44101dc0 2024-03-24T10:37:04.841 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.075+0000 7f2c3affd700 1 -- 172.21.15.94:0/1802489658 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2c3c022590 con 0x7f2c44101dc0 2024-03-24T10:37:04.841 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.075+0000 7f2c4c6db700 1 -- 172.21.15.94:0/1802489658 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c44110700 con 0x7f2c44101dc0 2024-03-24T10:37:04.841 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.076+0000 7f2c3affd700 1 -- 172.21.15.94:0/1802489658 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f2c3c022810 con 0x7f2c44101dc0 2024-03-24T10:37:04.842 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.076+0000 7f2c4c6db700 1 -- 172.21.15.94:0/1802489658 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c4405f5f0 con 0x7f2c44101dc0 2024-03-24T10:37:04.842 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.076+0000 7f2c3affd700 1 --2- 172.21.15.94:0/1802489658 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f2c30036790 0x7f2c30038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:04.842 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.076+0000 7f2c3affd700 1 -- 172.21.15.94:0/1802489658 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f2c3c014070 con 0x7f2c44101dc0 2024-03-24T10:37:04.842 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.076+0000 7f2c49c76700 1 --2- 172.21.15.94:0/1802489658 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f2c30036790 0x7f2c30038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:04.842 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.077+0000 7f2c49c76700 1 --2- 172.21.15.94:0/1802489658 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f2c30036790 0x7f2c30038c40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2c34009990 tx=0x7f2c34006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:04.842 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.081+0000 7f2c3affd700 1 -- 172.21.15.94:0/1802489658 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f2c3c021a70 con 0x7f2c44101dc0 2024-03-24T10:37:04.842 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.283+0000 7f2c4c6db700 1 -- 172.21.15.94:0/1802489658 --> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7f2c441112e0 con 0x7f2c30036790 2024-03-24T10:37:04.842 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.290+0000 7f2c3affd700 1 -- 172.21.15.94:0/1802489658 <== mgr.14120 v2:172.21.15.94:6800/1996608546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f2c441112e0 con 0x7f2c30036790 2024-03-24T10:37:04.842 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.292+0000 7f2c4c6db700 1 -- 172.21.15.94:0/1802489658 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f2c30036790 msgr2=0x7f2c30038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:04.842 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.292+0000 7f2c4c6db700 1 --2- 172.21.15.94:0/1802489658 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f2c30036790 0x7f2c30038c40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2c34009990 tx=0x7f2c34006e40 comp rx=0 tx=0).stop 2024-03-24T10:37:04.842 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.292+0000 7f2c4c6db700 1 -- 172.21.15.94:0/1802489658 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c44101dc0 msgr2=0x7f2c4410f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:04.842 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.292+0000 7f2c4c6db700 1 --2- 172.21.15.94:0/1802489658 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c44101dc0 0x7f2c4410f450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c006b00 tx=0x7f2c3c00b4f0 comp rx=0 tx=0).stop 2024-03-24T10:37:04.842 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.292+0000 7f2c4c6db700 1 -- 172.21.15.94:0/1802489658 shutdown_connections 2024-03-24T10:37:04.842 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.292+0000 7f2c4c6db700 1 --2- 172.21.15.94:0/1802489658 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f2c30036790 0x7f2c30038c40 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:04.843 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.292+0000 7f2c4c6db700 1 --2- 172.21.15.94:0/1802489658 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c44101dc0 0x7f2c4410f450 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:04.843 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.292+0000 7f2c4c6db700 1 -- 172.21.15.94:0/1802489658 >> 172.21.15.94:0/1802489658 conn(0x7f2c440fc750 msgr2=0x7f2c44106230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:37:04.843 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.292+0000 7f2c4c6db700 1 -- 172.21.15.94:0/1802489658 shutdown_connections 2024-03-24T10:37:04.843 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:04.292+0000 7f2c4c6db700 1 -- 172.21.15.94:0/1802489658 wait complete. 2024-03-24T10:37:04.843 INFO:teuthology.orchestra.run.smithi094.stdout:Deploying node-exporter service with default placement... 2024-03-24T10:37:05.000 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:04 smithi094 ceph-mon[29446]: from='mgr.14120 172.21.15.94:0/773982920' entity='mgr.smithi094.yaethu' 2024-03-24T10:37:05.000 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:04 smithi094 ceph-mon[29446]: from='mgr.14120 172.21.15.94:0/773982920' entity='mgr.smithi094.yaethu' 2024-03-24T10:37:05.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:05 smithi094 ceph-mon[29446]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:37:05.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:05 smithi094 ceph-mon[29446]: Saving service grafana spec with placement count:1 2024-03-24T10:37:05.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:05 smithi094 ceph-mon[29446]: from='mgr.14120 172.21.15.94:0/773982920' entity='mgr.smithi094.yaethu' 2024-03-24T10:37:05.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:05 smithi094 ceph-mon[29446]: from='mgr.14120 172.21.15.94:0/773982920' entity='mgr.smithi094.yaethu' 2024-03-24T10:37:06.079 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-03-24T10:37:06.079 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.239+0000 7f1cbd67c700 1 Processor -- start 2024-03-24T10:37:06.080 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.239+0000 7f1cbd67c700 1 -- start start 2024-03-24T10:37:06.080 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.239+0000 7f1cbd67c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cb8102cc0 0x7f1cb81030d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:06.080 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.239+0000 7f1cbd67c700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb81036a0 con 0x7f1cb8102cc0 2024-03-24T10:37:06.080 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.239+0000 7f1cb6ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cb8102cc0 0x7f1cb81030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:06.080 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.239+0000 7f1cb6ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cb8102cc0 0x7f1cb81030d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56666/0 (socket says 172.21.15.94:56666) 2024-03-24T10:37:06.080 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.239+0000 7f1cb6ffd700 1 -- 172.21.15.94:0/3278469797 learned_addr learned my addr 172.21.15.94:0/3278469797 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:37:06.080 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.239+0000 7f1cb6ffd700 1 -- 172.21.15.94:0/3278469797 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb81037e0 con 0x7f1cb8102cc0 2024-03-24T10:37:06.080 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.239+0000 7f1cb6ffd700 1 --2- 172.21.15.94:0/3278469797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cb8102cc0 0x7f1cb81030d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1ca80098e0 tx=0x7f1ca8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e74fd9ec399c819f server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:06.080 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.240+0000 7f1cb5ffb700 1 -- 172.21.15.94:0/3278469797 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1ca800b860 con 0x7f1cb8102cc0 2024-03-24T10:37:06.080 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.240+0000 7f1cb5ffb700 1 -- 172.21.15.94:0/3278469797 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1ca800be90 con 0x7f1cb8102cc0 2024-03-24T10:37:06.080 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.240+0000 7f1cb5ffb700 1 -- 172.21.15.94:0/3278469797 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1ca80103c0 con 0x7f1cb8102cc0 2024-03-24T10:37:06.080 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.240+0000 7f1cbd67c700 1 -- 172.21.15.94:0/3278469797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cb8102cc0 msgr2=0x7f1cb81030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:06.081 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.240+0000 7f1cbd67c700 1 --2- 172.21.15.94:0/3278469797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cb8102cc0 0x7f1cb81030d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1ca80098e0 tx=0x7f1ca8009250 comp rx=0 tx=0).stop 2024-03-24T10:37:06.081 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.240+0000 7f1cbd67c700 1 -- 172.21.15.94:0/3278469797 shutdown_connections 2024-03-24T10:37:06.081 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.240+0000 7f1cbd67c700 1 --2- 172.21.15.94:0/3278469797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cb8102cc0 0x7f1cb81030d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:06.081 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.240+0000 7f1cbd67c700 1 -- 172.21.15.94:0/3278469797 >> 172.21.15.94:0/3278469797 conn(0x7f1cb80fc700 msgr2=0x7f1cb80feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:37:06.081 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.240+0000 7f1cbd67c700 1 -- 172.21.15.94:0/3278469797 shutdown_connections 2024-03-24T10:37:06.081 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.240+0000 7f1cbd67c700 1 -- 172.21.15.94:0/3278469797 wait complete. 2024-03-24T10:37:06.081 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.241+0000 7f1cbd67c700 1 Processor -- start 2024-03-24T10:37:06.081 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.241+0000 7f1cbd67c700 1 -- start start 2024-03-24T10:37:06.081 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.241+0000 7f1cbd67c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cb8102cc0 0x7f1cb81138f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:06.081 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.241+0000 7f1cbd67c700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb8113e30 con 0x7f1cb8102cc0 2024-03-24T10:37:06.081 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.241+0000 7f1cb6ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cb8102cc0 0x7f1cb81138f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:06.081 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.241+0000 7f1cb6ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cb8102cc0 0x7f1cb81138f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56670/0 (socket says 172.21.15.94:56670) 2024-03-24T10:37:06.081 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.241+0000 7f1cb6ffd700 1 -- 172.21.15.94:0/3774545115 learned_addr learned my addr 172.21.15.94:0/3774545115 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:37:06.081 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.242+0000 7f1cb6ffd700 1 -- 172.21.15.94:0/3774545115 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca80094f0 con 0x7f1cb8102cc0 2024-03-24T10:37:06.081 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.242+0000 7f1cb6ffd700 1 --2- 172.21.15.94:0/3774545115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cb8102cc0 0x7f1cb81138f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8006d40 tx=0x7f1ca8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:06.082 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.242+0000 7f1c9f7fe700 1 -- 172.21.15.94:0/3774545115 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1ca8010840 con 0x7f1cb8102cc0 2024-03-24T10:37:06.082 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.242+0000 7f1cbd67c700 1 -- 172.21.15.94:0/3774545115 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cb8114030 con 0x7f1cb8102cc0 2024-03-24T10:37:06.082 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.242+0000 7f1c9f7fe700 1 -- 172.21.15.94:0/3774545115 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1ca8010e70 con 0x7f1cb8102cc0 2024-03-24T10:37:06.082 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.242+0000 7f1c9f7fe700 1 -- 172.21.15.94:0/3774545115 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1ca8024ea0 con 0x7f1cb8102cc0 2024-03-24T10:37:06.082 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.242+0000 7f1cbd67c700 1 -- 172.21.15.94:0/3774545115 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cb8114ba0 con 0x7f1cb8102cc0 2024-03-24T10:37:06.082 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.243+0000 7f1c9f7fe700 1 -- 172.21.15.94:0/3774545115 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f1ca801a510 con 0x7f1cb8102cc0 2024-03-24T10:37:06.082 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.243+0000 7f1c9f7fe700 1 --2- 172.21.15.94:0/3774545115 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f1ca0036740 0x7f1ca0038bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:06.082 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.243+0000 7f1cbd67c700 1 -- 172.21.15.94:0/3774545115 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cb805f5f0 con 0x7f1cb8102cc0 2024-03-24T10:37:06.082 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.243+0000 7f1c9f7fe700 1 -- 172.21.15.94:0/3774545115 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f1ca804a870 con 0x7f1cb8102cc0 2024-03-24T10:37:06.082 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.243+0000 7f1cb67fc700 1 --2- 172.21.15.94:0/3774545115 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f1ca0036740 0x7f1ca0038bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:06.082 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.243+0000 7f1cb67fc700 1 --2- 172.21.15.94:0/3774545115 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f1ca0036740 0x7f1ca0038bf0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1cac009990 tx=0x7f1cac006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:06.082 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.248+0000 7f1c9f7fe700 1 -- 172.21.15.94:0/3774545115 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f1ca8018530 con 0x7f1cb8102cc0 2024-03-24T10:37:06.082 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.449+0000 7f1cbd67c700 1 -- 172.21.15.94:0/3774545115 --> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f1cb80fd9d0 con 0x7f1ca0036740 2024-03-24T10:37:06.082 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.455+0000 7f1c9f7fe700 1 -- 172.21.15.94:0/3774545115 <== mgr.14120 v2:172.21.15.94:6800/1996608546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f1cb80fd9d0 con 0x7f1ca0036740 2024-03-24T10:37:06.082 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.457+0000 7f1cbd67c700 1 -- 172.21.15.94:0/3774545115 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f1ca0036740 msgr2=0x7f1ca0038bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:06.083 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.457+0000 7f1cbd67c700 1 --2- 172.21.15.94:0/3774545115 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f1ca0036740 0x7f1ca0038bf0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1cac009990 tx=0x7f1cac006e40 comp rx=0 tx=0).stop 2024-03-24T10:37:06.083 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.457+0000 7f1cbd67c700 1 -- 172.21.15.94:0/3774545115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cb8102cc0 msgr2=0x7f1cb81138f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:06.083 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.457+0000 7f1cbd67c700 1 --2- 172.21.15.94:0/3774545115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cb8102cc0 0x7f1cb81138f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8006d40 tx=0x7f1ca8009250 comp rx=0 tx=0).stop 2024-03-24T10:37:06.083 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.457+0000 7f1cbd67c700 1 -- 172.21.15.94:0/3774545115 shutdown_connections 2024-03-24T10:37:06.083 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.457+0000 7f1cbd67c700 1 --2- 172.21.15.94:0/3774545115 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f1ca0036740 0x7f1ca0038bf0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:06.083 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.457+0000 7f1cbd67c700 1 --2- 172.21.15.94:0/3774545115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cb8102cc0 0x7f1cb81138f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:06.083 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.457+0000 7f1cbd67c700 1 -- 172.21.15.94:0/3774545115 >> 172.21.15.94:0/3774545115 conn(0x7f1cb80fc700 msgr2=0x7f1cb80fd280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:37:06.083 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.458+0000 7f1cbd67c700 1 -- 172.21.15.94:0/3774545115 shutdown_connections 2024-03-24T10:37:06.083 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:05.458+0000 7f1cbd67c700 1 -- 172.21.15.94:0/3774545115 wait complete. 2024-03-24T10:37:06.083 INFO:teuthology.orchestra.run.smithi094.stdout:Deploying alertmanager service with default placement... 2024-03-24T10:37:06.981 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:06 smithi094 ceph-mon[29446]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:37:06.981 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:06 smithi094 ceph-mon[29446]: Saving service node-exporter spec with placement * 2024-03-24T10:37:06.982 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:06 smithi094 ceph-mon[29446]: from='mgr.14120 172.21.15.94:0/773982920' entity='mgr.smithi094.yaethu' 2024-03-24T10:37:07.880 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:07 smithi094 ceph-mon[29446]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:37:07.880 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:07 smithi094 ceph-mon[29446]: Saving service alertmanager spec with placement count:1 2024-03-24T10:37:07.887 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-03-24T10:37:07.888 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.483+0000 7fb598dcf700 1 Processor -- start 2024-03-24T10:37:07.888 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.483+0000 7fb598dcf700 1 -- start start 2024-03-24T10:37:07.888 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.483+0000 7fb598dcf700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb594102860 0x7fb594102c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:07.888 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.483+0000 7fb598dcf700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5941031b0 con 0x7fb594102860 2024-03-24T10:37:07.888 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.484+0000 7fb59259c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb594102860 0x7fb594102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:07.888 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.484+0000 7fb59259c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb594102860 0x7fb594102c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56682/0 (socket says 172.21.15.94:56682) 2024-03-24T10:37:07.888 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.484+0000 7fb59259c700 1 -- 172.21.15.94:0/2847172469 learned_addr learned my addr 172.21.15.94:0/2847172469 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:37:07.888 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.484+0000 7fb59259c700 1 -- 172.21.15.94:0/2847172469 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5941032f0 con 0x7fb594102860 2024-03-24T10:37:07.888 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.484+0000 7fb59259c700 1 --2- 172.21.15.94:0/2847172469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb594102860 0x7fb594102c70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb5840098e0 tx=0x7fb584009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2721d90621d7792c server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:07.888 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.484+0000 7fb59159a700 1 -- 172.21.15.94:0/2847172469 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb58400b860 con 0x7fb594102860 2024-03-24T10:37:07.888 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.484+0000 7fb59159a700 1 -- 172.21.15.94:0/2847172469 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb58400be90 con 0x7fb594102860 2024-03-24T10:37:07.888 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.484+0000 7fb59159a700 1 -- 172.21.15.94:0/2847172469 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb5840103c0 con 0x7fb594102860 2024-03-24T10:37:07.888 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.485+0000 7fb598dcf700 1 -- 172.21.15.94:0/2847172469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb594102860 msgr2=0x7fb594102c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:07.888 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.485+0000 7fb598dcf700 1 --2- 172.21.15.94:0/2847172469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb594102860 0x7fb594102c70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb5840098e0 tx=0x7fb584009250 comp rx=0 tx=0).stop 2024-03-24T10:37:07.888 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.485+0000 7fb598dcf700 1 -- 172.21.15.94:0/2847172469 shutdown_connections 2024-03-24T10:37:07.889 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.485+0000 7fb598dcf700 1 --2- 172.21.15.94:0/2847172469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb594102860 0x7fb594102c70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:07.889 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.485+0000 7fb598dcf700 1 -- 172.21.15.94:0/2847172469 >> 172.21.15.94:0/2847172469 conn(0x7fb5940fe000 msgr2=0x7fb594100410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:37:07.889 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.485+0000 7fb598dcf700 1 -- 172.21.15.94:0/2847172469 shutdown_connections 2024-03-24T10:37:07.889 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.485+0000 7fb598dcf700 1 -- 172.21.15.94:0/2847172469 wait complete. 2024-03-24T10:37:07.889 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.486+0000 7fb598dcf700 1 Processor -- start 2024-03-24T10:37:07.889 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.486+0000 7fb598dcf700 1 -- start start 2024-03-24T10:37:07.889 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.486+0000 7fb598dcf700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb594102860 0x7fb594078c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:07.889 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.486+0000 7fb598dcf700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb594079180 con 0x7fb594102860 2024-03-24T10:37:07.889 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.486+0000 7fb59259c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb594102860 0x7fb594078c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:07.889 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.486+0000 7fb59259c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb594102860 0x7fb594078c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56696/0 (socket says 172.21.15.94:56696) 2024-03-24T10:37:07.889 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.486+0000 7fb59259c700 1 -- 172.21.15.94:0/3209062120 learned_addr learned my addr 172.21.15.94:0/3209062120 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:37:07.889 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.487+0000 7fb59259c700 1 -- 172.21.15.94:0/3209062120 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5840094f0 con 0x7fb594102860 2024-03-24T10:37:07.889 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.487+0000 7fb59259c700 1 --2- 172.21.15.94:0/3209062120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb594102860 0x7fb594078c40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb584011680 tx=0x7fb584009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:07.889 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.487+0000 7fb582ffd700 1 -- 172.21.15.94:0/3209062120 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb584010890 con 0x7fb594102860 2024-03-24T10:37:07.889 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.487+0000 7fb598dcf700 1 -- 172.21.15.94:0/3209062120 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5940771f0 con 0x7fb594102860 2024-03-24T10:37:07.889 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.487+0000 7fb582ffd700 1 -- 172.21.15.94:0/3209062120 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb584010ec0 con 0x7fb594102860 2024-03-24T10:37:07.890 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.487+0000 7fb598dcf700 1 -- 172.21.15.94:0/3209062120 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb594077620 con 0x7fb594102860 2024-03-24T10:37:07.890 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.487+0000 7fb582ffd700 1 -- 172.21.15.94:0/3209062120 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb5840243f0 con 0x7fb594102860 2024-03-24T10:37:07.890 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.488+0000 7fb582ffd700 1 -- 172.21.15.94:0/3209062120 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fb58402d3f0 con 0x7fb594102860 2024-03-24T10:37:07.890 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.488+0000 7fb582ffd700 1 --2- 172.21.15.94:0/3209062120 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7fb57c0367e0 0x7fb57c038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:07.890 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.488+0000 7fb582ffd700 1 -- 172.21.15.94:0/3209062120 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fb58404b2d0 con 0x7fb594102860 2024-03-24T10:37:07.890 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.488+0000 7fb598dcf700 1 -- 172.21.15.94:0/3209062120 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5740052e0 con 0x7fb594102860 2024-03-24T10:37:07.890 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.488+0000 7fb591d9b700 1 --2- 172.21.15.94:0/3209062120 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7fb57c0367e0 0x7fb57c038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:07.890 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.489+0000 7fb591d9b700 1 --2- 172.21.15.94:0/3209062120 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7fb57c0367e0 0x7fb57c038c90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb588009990 tx=0x7fb588006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:07.890 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.493+0000 7fb582ffd700 1 -- 172.21.15.94:0/3209062120 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fb5840193c0 con 0x7fb594102860 2024-03-24T10:37:07.890 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.700+0000 7fb598dcf700 1 -- 172.21.15.94:0/3209062120 --> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7fb574000bf0 con 0x7fb57c0367e0 2024-03-24T10:37:07.890 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.705+0000 7fb582ffd700 1 -- 172.21.15.94:0/3209062120 <== mgr.14120 v2:172.21.15.94:6800/1996608546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7fb574000bf0 con 0x7fb57c0367e0 2024-03-24T10:37:07.890 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.707+0000 7fb598dcf700 1 -- 172.21.15.94:0/3209062120 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7fb57c0367e0 msgr2=0x7fb57c038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:07.890 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.707+0000 7fb598dcf700 1 --2- 172.21.15.94:0/3209062120 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7fb57c0367e0 0x7fb57c038c90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb588009990 tx=0x7fb588006e40 comp rx=0 tx=0).stop 2024-03-24T10:37:07.890 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.707+0000 7fb598dcf700 1 -- 172.21.15.94:0/3209062120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb594102860 msgr2=0x7fb594078c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:07.891 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.707+0000 7fb598dcf700 1 --2- 172.21.15.94:0/3209062120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb594102860 0x7fb594078c40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb584011680 tx=0x7fb584009250 comp rx=0 tx=0).stop 2024-03-24T10:37:07.891 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.707+0000 7fb598dcf700 1 -- 172.21.15.94:0/3209062120 shutdown_connections 2024-03-24T10:37:07.891 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.707+0000 7fb598dcf700 1 --2- 172.21.15.94:0/3209062120 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7fb57c0367e0 0x7fb57c038c90 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:07.891 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.707+0000 7fb598dcf700 1 --2- 172.21.15.94:0/3209062120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb594102860 0x7fb594078c40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:07.891 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.707+0000 7fb598dcf700 1 -- 172.21.15.94:0/3209062120 >> 172.21.15.94:0/3209062120 conn(0x7fb5940fe000 msgr2=0x7fb5940feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:37:07.891 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.707+0000 7fb598dcf700 1 -- 172.21.15.94:0/3209062120 shutdown_connections 2024-03-24T10:37:07.891 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:06.707+0000 7fb598dcf700 1 -- 172.21.15.94:0/3209062120 wait complete. 2024-03-24T10:37:09.022 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.264+0000 7f168e6cf700 1 Processor -- start 2024-03-24T10:37:09.022 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.264+0000 7f168e6cf700 1 -- start start 2024-03-24T10:37:09.022 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.264+0000 7f168e6cf700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1688101d70 0x7f1688102180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:09.022 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.264+0000 7f168e6cf700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16881026c0 con 0x7f1688101d70 2024-03-24T10:37:09.022 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.264+0000 7f1687fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1688101d70 0x7f1688102180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:09.023 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.264+0000 7f1687fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1688101d70 0x7f1688102180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56708/0 (socket says 172.21.15.94:56708) 2024-03-24T10:37:09.023 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.264+0000 7f1687fff700 1 -- 172.21.15.94:0/2778706286 learned_addr learned my addr 172.21.15.94:0/2778706286 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:37:09.023 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.264+0000 7f1687fff700 1 -- 172.21.15.94:0/2778706286 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1688102800 con 0x7f1688101d70 2024-03-24T10:37:09.023 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.265+0000 7f1687fff700 1 --2- 172.21.15.94:0/2778706286 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1688101d70 0x7f1688102180 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f16780098e0 tx=0x7f1678009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e12a4421b74a50 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:09.023 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.265+0000 7f1686ffd700 1 -- 172.21.15.94:0/2778706286 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f167800b860 con 0x7f1688101d70 2024-03-24T10:37:09.023 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.265+0000 7f1686ffd700 1 -- 172.21.15.94:0/2778706286 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f167800be90 con 0x7f1688101d70 2024-03-24T10:37:09.023 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.265+0000 7f1686ffd700 1 -- 172.21.15.94:0/2778706286 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f16780103c0 con 0x7f1688101d70 2024-03-24T10:37:09.023 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.265+0000 7f168e6cf700 1 -- 172.21.15.94:0/2778706286 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1688101d70 msgr2=0x7f1688102180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:09.023 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.265+0000 7f168e6cf700 1 --2- 172.21.15.94:0/2778706286 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1688101d70 0x7f1688102180 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f16780098e0 tx=0x7f1678009250 comp rx=0 tx=0).stop 2024-03-24T10:37:09.023 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.265+0000 7f168e6cf700 1 -- 172.21.15.94:0/2778706286 shutdown_connections 2024-03-24T10:37:09.023 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.265+0000 7f168e6cf700 1 --2- 172.21.15.94:0/2778706286 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1688101d70 0x7f1688102180 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:09.023 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.265+0000 7f168e6cf700 1 -- 172.21.15.94:0/2778706286 >> 172.21.15.94:0/2778706286 conn(0x7f16880fc700 msgr2=0x7f16880feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:37:09.023 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.265+0000 7f168e6cf700 1 -- 172.21.15.94:0/2778706286 shutdown_connections 2024-03-24T10:37:09.023 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.265+0000 7f168e6cf700 1 -- 172.21.15.94:0/2778706286 wait complete. 2024-03-24T10:37:09.023 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.266+0000 7f168e6cf700 1 Processor -- start 2024-03-24T10:37:09.024 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.266+0000 7f168e6cf700 1 -- start start 2024-03-24T10:37:09.024 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.266+0000 7f168e6cf700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1688101d70 0x7f1688107a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:09.024 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.266+0000 7f168e6cf700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1688107f70 con 0x7f1688101d70 2024-03-24T10:37:09.024 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.266+0000 7f1687fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1688101d70 0x7f1688107a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:09.024 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.266+0000 7f1687fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1688101d70 0x7f1688107a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56716/0 (socket says 172.21.15.94:56716) 2024-03-24T10:37:09.024 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.266+0000 7f1687fff700 1 -- 172.21.15.94:0/2066611401 learned_addr learned my addr 172.21.15.94:0/2066611401 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:37:09.024 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.266+0000 7f1687fff700 1 -- 172.21.15.94:0/2066611401 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16780094f0 con 0x7f1688101d70 2024-03-24T10:37:09.024 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.267+0000 7f1687fff700 1 --2- 172.21.15.94:0/2066611401 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1688101d70 0x7f1688107a30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1678011680 tx=0x7f1678009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:09.024 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.267+0000 7f1684ff9700 1 -- 172.21.15.94:0/2066611401 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1678010890 con 0x7f1688101d70 2024-03-24T10:37:09.024 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.267+0000 7f168e6cf700 1 -- 172.21.15.94:0/2066611401 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1688105fe0 con 0x7f1688101d70 2024-03-24T10:37:09.024 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.267+0000 7f1684ff9700 1 -- 172.21.15.94:0/2066611401 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1678010ec0 con 0x7f1688101d70 2024-03-24T10:37:09.024 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.267+0000 7f1684ff9700 1 -- 172.21.15.94:0/2066611401 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f16780243f0 con 0x7f1688101d70 2024-03-24T10:37:09.024 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.267+0000 7f168e6cf700 1 -- 172.21.15.94:0/2066611401 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16881064d0 con 0x7f1688101d70 2024-03-24T10:37:09.024 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.268+0000 7f1684ff9700 1 -- 172.21.15.94:0/2066611401 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f16780109f0 con 0x7f1688101d70 2024-03-24T10:37:09.024 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.268+0000 7f1684ff9700 1 --2- 172.21.15.94:0/2066611401 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f1670036790 0x7f1670038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:09.025 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.268+0000 7f168e6cf700 1 -- 172.21.15.94:0/2066611401 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f168804bed0 con 0x7f1688101d70 2024-03-24T10:37:09.025 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.268+0000 7f1684ff9700 1 -- 172.21.15.94:0/2066611401 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f167804a2f0 con 0x7f1688101d70 2024-03-24T10:37:09.025 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.268+0000 7f16877fe700 1 --2- 172.21.15.94:0/2066611401 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f1670036790 0x7f1670038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:09.025 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.268+0000 7f16877fe700 1 --2- 172.21.15.94:0/2066611401 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f1670036790 0x7f1670038c40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f167c009990 tx=0x7f167c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:09.025 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.273+0000 7f1684ff9700 1 -- 172.21.15.94:0/2066611401 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f167802d3f0 con 0x7f1688101d70 2024-03-24T10:37:09.025 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.457+0000 7f168e6cf700 1 -- 172.21.15.94:0/2066611401 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f1688103640 con 0x7f1688101d70 2024-03-24T10:37:09.025 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.460+0000 7f1684ff9700 1 -- 172.21.15.94:0/2066611401 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f16780166e0 con 0x7f1688101d70 2024-03-24T10:37:09.025 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.462+0000 7f168e6cf700 1 -- 172.21.15.94:0/2066611401 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f1670036790 msgr2=0x7f1670038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:09.025 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.462+0000 7f168e6cf700 1 --2- 172.21.15.94:0/2066611401 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f1670036790 0x7f1670038c40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f167c009990 tx=0x7f167c006e40 comp rx=0 tx=0).stop 2024-03-24T10:37:09.025 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.462+0000 7f168e6cf700 1 -- 172.21.15.94:0/2066611401 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1688101d70 msgr2=0x7f1688107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:09.025 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.462+0000 7f168e6cf700 1 --2- 172.21.15.94:0/2066611401 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1688101d70 0x7f1688107a30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1678011680 tx=0x7f1678009250 comp rx=0 tx=0).stop 2024-03-24T10:37:09.025 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.462+0000 7f168e6cf700 1 -- 172.21.15.94:0/2066611401 shutdown_connections 2024-03-24T10:37:09.025 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.462+0000 7f168e6cf700 1 --2- 172.21.15.94:0/2066611401 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f1670036790 0x7f1670038c40 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:09.025 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.462+0000 7f168e6cf700 1 --2- 172.21.15.94:0/2066611401 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1688101d70 0x7f1688107a30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:09.025 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.462+0000 7f168e6cf700 1 -- 172.21.15.94:0/2066611401 >> 172.21.15.94:0/2066611401 conn(0x7f16880fc700 msgr2=0x7f16880fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:37:09.026 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.462+0000 7f168e6cf700 1 -- 172.21.15.94:0/2066611401 shutdown_connections 2024-03-24T10:37:09.026 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:08.462+0000 7f168e6cf700 1 -- 172.21.15.94:0/2066611401 wait complete. 2024-03-24T10:37:09.651 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:09 smithi094 ceph-mon[29446]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:37:09.651 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:09 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/2066611401' entity='client.admin' 2024-03-24T10:37:10.170 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.440+0000 7f4cfa6b6700 1 Processor -- start 2024-03-24T10:37:10.171 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.440+0000 7f4cfa6b6700 1 -- start start 2024-03-24T10:37:10.171 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.440+0000 7f4cfa6b6700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cf4102910 0x7f4cf4102d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:10.171 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.440+0000 7f4cfa6b6700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf4103260 con 0x7f4cf4102910 2024-03-24T10:37:10.171 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.440+0000 7f4cf3fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cf4102910 0x7f4cf4102d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:10.171 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.440+0000 7f4cf3fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cf4102910 0x7f4cf4102d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56730/0 (socket says 172.21.15.94:56730) 2024-03-24T10:37:10.171 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.440+0000 7f4cf3fff700 1 -- 172.21.15.94:0/4020524205 learned_addr learned my addr 172.21.15.94:0/4020524205 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:37:10.171 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.440+0000 7f4cf3fff700 1 -- 172.21.15.94:0/4020524205 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf41033a0 con 0x7f4cf4102910 2024-03-24T10:37:10.171 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.440+0000 7f4cf3fff700 1 --2- 172.21.15.94:0/4020524205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cf4102910 0x7f4cf4102d20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4ce40098e0 tx=0x7f4ce4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcda39584270b295 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:10.171 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.441+0000 7f4cf2ffd700 1 -- 172.21.15.94:0/4020524205 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4ce400b860 con 0x7f4cf4102910 2024-03-24T10:37:10.171 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.441+0000 7f4cf2ffd700 1 -- 172.21.15.94:0/4020524205 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ce400be90 con 0x7f4cf4102910 2024-03-24T10:37:10.171 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.441+0000 7f4cf2ffd700 1 -- 172.21.15.94:0/4020524205 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4ce40103c0 con 0x7f4cf4102910 2024-03-24T10:37:10.171 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.441+0000 7f4cfa6b6700 1 -- 172.21.15.94:0/4020524205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cf4102910 msgr2=0x7f4cf4102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:10.171 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.441+0000 7f4cfa6b6700 1 --2- 172.21.15.94:0/4020524205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cf4102910 0x7f4cf4102d20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4ce40098e0 tx=0x7f4ce4009250 comp rx=0 tx=0).stop 2024-03-24T10:37:10.171 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.441+0000 7f4cfa6b6700 1 -- 172.21.15.94:0/4020524205 shutdown_connections 2024-03-24T10:37:10.172 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.441+0000 7f4cfa6b6700 1 --2- 172.21.15.94:0/4020524205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cf4102910 0x7f4cf4102d20 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:10.172 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.441+0000 7f4cfa6b6700 1 -- 172.21.15.94:0/4020524205 >> 172.21.15.94:0/4020524205 conn(0x7f4cf40fe070 msgr2=0x7f4cf41004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:37:10.172 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.441+0000 7f4cfa6b6700 1 -- 172.21.15.94:0/4020524205 shutdown_connections 2024-03-24T10:37:10.172 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.441+0000 7f4cfa6b6700 1 -- 172.21.15.94:0/4020524205 wait complete. 2024-03-24T10:37:10.172 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.442+0000 7f4cfa6b6700 1 Processor -- start 2024-03-24T10:37:10.172 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.442+0000 7f4cfa6b6700 1 -- start start 2024-03-24T10:37:10.172 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.442+0000 7f4cfa6b6700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cf4102910 0x7f4cf4111710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:10.172 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.442+0000 7f4cfa6b6700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf4111c50 con 0x7f4cf4102910 2024-03-24T10:37:10.172 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.442+0000 7f4cf3fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cf4102910 0x7f4cf4111710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:10.172 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.442+0000 7f4cf3fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cf4102910 0x7f4cf4111710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56738/0 (socket says 172.21.15.94:56738) 2024-03-24T10:37:10.172 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.442+0000 7f4cf3fff700 1 -- 172.21.15.94:0/525469828 learned_addr learned my addr 172.21.15.94:0/525469828 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:37:10.172 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.442+0000 7f4cf3fff700 1 -- 172.21.15.94:0/525469828 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce40094f0 con 0x7f4cf4102910 2024-03-24T10:37:10.172 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.442+0000 7f4cf3fff700 1 --2- 172.21.15.94:0/525469828 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cf4102910 0x7f4cf4111710 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4ce40090b0 tx=0x7f4ce4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:10.172 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.443+0000 7f4cf0ff9700 1 -- 172.21.15.94:0/525469828 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4ce4010840 con 0x7f4cf4102910 2024-03-24T10:37:10.172 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.443+0000 7f4cf0ff9700 1 -- 172.21.15.94:0/525469828 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ce4010e70 con 0x7f4cf4102910 2024-03-24T10:37:10.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.443+0000 7f4cfa6b6700 1 -- 172.21.15.94:0/525469828 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cf4111e50 con 0x7f4cf4102910 2024-03-24T10:37:10.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.443+0000 7f4cf0ff9700 1 -- 172.21.15.94:0/525469828 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4ce4024e40 con 0x7f4cf4102910 2024-03-24T10:37:10.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.443+0000 7f4cfa6b6700 1 -- 172.21.15.94:0/525469828 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cf41129c0 con 0x7f4cf4102910 2024-03-24T10:37:10.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.443+0000 7f4cf0ff9700 1 -- 172.21.15.94:0/525469828 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f4ce401a4e0 con 0x7f4cf4102910 2024-03-24T10:37:10.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.444+0000 7f4cf0ff9700 1 --2- 172.21.15.94:0/525469828 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f4cdc036790 0x7f4cdc038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:10.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.444+0000 7f4cf0ff9700 1 -- 172.21.15.94:0/525469828 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f4ce404a9d0 con 0x7f4cf4102910 2024-03-24T10:37:10.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.444+0000 7f4cfa6b6700 1 -- 172.21.15.94:0/525469828 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cf405f5f0 con 0x7f4cf4102910 2024-03-24T10:37:10.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.444+0000 7f4cf37fe700 1 --2- 172.21.15.94:0/525469828 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f4cdc036790 0x7f4cdc038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:10.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.444+0000 7f4cf37fe700 1 --2- 172.21.15.94:0/525469828 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f4cdc036790 0x7f4cdc038c40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8009990 tx=0x7f4ce8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:10.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.448+0000 7f4cf0ff9700 1 -- 172.21.15.94:0/525469828 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f4ce40188f0 con 0x7f4cf4102910 2024-03-24T10:37:10.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.634+0000 7f4cfa6b6700 1 -- 172.21.15.94:0/525469828 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f4cf404bed0 con 0x7f4cf4102910 2024-03-24T10:37:10.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.637+0000 7f4cf0ff9700 1 -- 172.21.15.94:0/525469828 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f4ce402d3a0 con 0x7f4cf4102910 2024-03-24T10:37:10.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.639+0000 7f4cfa6b6700 1 -- 172.21.15.94:0/525469828 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f4cdc036790 msgr2=0x7f4cdc038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:10.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.639+0000 7f4cfa6b6700 1 --2- 172.21.15.94:0/525469828 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f4cdc036790 0x7f4cdc038c40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8009990 tx=0x7f4ce8006e40 comp rx=0 tx=0).stop 2024-03-24T10:37:10.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.639+0000 7f4cfa6b6700 1 -- 172.21.15.94:0/525469828 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cf4102910 msgr2=0x7f4cf4111710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:10.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.639+0000 7f4cfa6b6700 1 --2- 172.21.15.94:0/525469828 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cf4102910 0x7f4cf4111710 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4ce40090b0 tx=0x7f4ce4009250 comp rx=0 tx=0).stop 2024-03-24T10:37:10.174 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.639+0000 7f4cfa6b6700 1 -- 172.21.15.94:0/525469828 shutdown_connections 2024-03-24T10:37:10.174 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.639+0000 7f4cfa6b6700 1 --2- 172.21.15.94:0/525469828 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f4cdc036790 0x7f4cdc038c40 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:10.174 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.639+0000 7f4cfa6b6700 1 --2- 172.21.15.94:0/525469828 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cf4102910 0x7f4cf4111710 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:10.174 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.639+0000 7f4cfa6b6700 1 -- 172.21.15.94:0/525469828 >> 172.21.15.94:0/525469828 conn(0x7f4cf40fe070 msgr2=0x7f4cf40feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:37:10.174 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.640+0000 7f4cfa6b6700 1 -- 172.21.15.94:0/525469828 shutdown_connections 2024-03-24T10:37:10.174 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:09.640+0000 7f4cfa6b6700 1 -- 172.21.15.94:0/525469828 wait complete. 2024-03-24T10:37:10.174 INFO:teuthology.orchestra.run.smithi094.stdout:Enabling the dashboard module... 2024-03-24T10:37:10.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:10 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/525469828' entity='client.admin' 2024-03-24T10:37:11.953 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:11 smithi094 ceph-mon[29446]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:37:11.953 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:11 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/3214251049' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-24T10:37:12.303 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:10.532+0000 7fd10bdc2700 1 Processor -- start 2024-03-24T10:37:12.304 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:10.532+0000 7fd10bdc2700 1 -- start start 2024-03-24T10:37:12.304 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:10.532+0000 7fd10bdc2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd104076c20 0x7fd104105470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:12.305 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:10.532+0000 7fd10bdc2700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1041059b0 con 0x7fd104076c20 2024-03-24T10:37:12.305 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:10.532+0000 7fd109b5e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd104076c20 0x7fd104105470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:12.305 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:10.532+0000 7fd109b5e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd104076c20 0x7fd104105470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40014/0 (socket says 172.21.15.94:40014) 2024-03-24T10:37:12.305 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:10.532+0000 7fd109b5e700 1 -- 172.21.15.94:0/3210805673 learned_addr learned my addr 172.21.15.94:0/3210805673 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:37:12.305 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:10.532+0000 7fd109b5e700 1 -- 172.21.15.94:0/3210805673 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd104105af0 con 0x7fd104076c20 2024-03-24T10:37:12.305 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:10.532+0000 7fd109b5e700 1 --2- 172.21.15.94:0/3210805673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd104076c20 0x7fd104105470 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd1000098e0 tx=0x7fd100009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4efe3dd0ef21290a server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:12.305 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:10.533+0000 7fd108b5c700 1 -- 172.21.15.94:0/3210805673 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd10000b860 con 0x7fd104076c20 2024-03-24T10:37:12.305 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:10.533+0000 7fd108b5c700 1 -- 172.21.15.94:0/3210805673 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd10000be90 con 0x7fd104076c20 2024-03-24T10:37:12.305 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:10.533+0000 7fd108b5c700 1 -- 172.21.15.94:0/3210805673 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd1000103c0 con 0x7fd104076c20 2024-03-24T10:37:12.305 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:10.533+0000 7fd10bdc2700 1 -- 172.21.15.94:0/3210805673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd104076c20 msgr2=0x7fd104105470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:12.306 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:10.533+0000 7fd10bdc2700 1 --2- 172.21.15.94:0/3210805673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd104076c20 0x7fd104105470 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd1000098e0 tx=0x7fd100009250 comp rx=0 tx=0).stop 2024-03-24T10:37:12.306 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:10.533+0000 7fd10bdc2700 1 -- 172.21.15.94:0/3210805673 shutdown_connections 2024-03-24T10:37:12.306 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:10.533+0000 7fd10bdc2700 1 --2- 172.21.15.94:0/3210805673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd104076c20 0x7fd104105470 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:12.306 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:10.533+0000 7fd10bdc2700 1 -- 172.21.15.94:0/3210805673 >> 172.21.15.94:0/3210805673 conn(0x7fd1040fcf40 msgr2=0x7fd1040ff350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:37:12.306 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:10.533+0000 7fd10bdc2700 1 -- 172.21.15.94:0/3210805673 shutdown_connections 2024-03-24T10:37:12.306 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:10.533+0000 7fd10bdc2700 1 -- 172.21.15.94:0/3210805673 wait complete. 2024-03-24T10:37:12.306 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:10.534+0000 7fd10bdc2700 1 Processor -- start 2024-03-24T10:37:12.306 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:10.534+0000 7fd10bdc2700 1 -- start start 2024-03-24T10:37:12.306 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:10.534+0000 7fd10bdc2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd104076c20 0x7fd104111710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:12.306 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:10.534+0000 7fd10bdc2700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd104111c50 con 0x7fd104076c20 2024-03-24T10:37:12.306 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:10.534+0000 7fd109b5e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd104076c20 0x7fd104111710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:12.306 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:10.534+0000 7fd109b5e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd104076c20 0x7fd104111710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40024/0 (socket says 172.21.15.94:40024) 2024-03-24T10:37:12.306 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:10.534+0000 7fd109b5e700 1 -- 172.21.15.94:0/3214251049 learned_addr learned my addr 172.21.15.94:0/3214251049 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:37:12.306 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:10.534+0000 7fd109b5e700 1 -- 172.21.15.94:0/3214251049 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1000094f0 con 0x7fd104076c20 2024-03-24T10:37:12.306 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:10.534+0000 7fd109b5e700 1 --2- 172.21.15.94:0/3214251049 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd104076c20 0x7fd104111710 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd100006d40 tx=0x7fd100009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:12.307 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:10.535+0000 7fd0fa7fc700 1 -- 172.21.15.94:0/3214251049 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd100010840 con 0x7fd104076c20 2024-03-24T10:37:12.307 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:10.535+0000 7fd0fa7fc700 1 -- 172.21.15.94:0/3214251049 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd100010e70 con 0x7fd104076c20 2024-03-24T10:37:12.307 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:10.535+0000 7fd0fa7fc700 1 -- 172.21.15.94:0/3214251049 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd100024ea0 con 0x7fd104076c20 2024-03-24T10:37:12.307 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:10.535+0000 7fd10bdc2700 1 -- 172.21.15.94:0/3214251049 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd104111e50 con 0x7fd104076c20 2024-03-24T10:37:12.307 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:10.535+0000 7fd10bdc2700 1 -- 172.21.15.94:0/3214251049 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1041129c0 con 0x7fd104076c20 2024-03-24T10:37:12.307 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:10.535+0000 7fd0fa7fc700 1 -- 172.21.15.94:0/3214251049 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fd100024780 con 0x7fd104076c20 2024-03-24T10:37:12.307 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:10.535+0000 7fd10bdc2700 1 -- 172.21.15.94:0/3214251049 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd10405f5f0 con 0x7fd104076c20 2024-03-24T10:37:12.307 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:10.536+0000 7fd0fa7fc700 1 --2- 172.21.15.94:0/3214251049 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7fd0f0036790 0x7fd0f0038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:12.307 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:10.536+0000 7fd0fa7fc700 1 -- 172.21.15.94:0/3214251049 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fd10004a680 con 0x7fd104076c20 2024-03-24T10:37:12.307 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:10.536+0000 7fd10935d700 1 --2- 172.21.15.94:0/3214251049 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7fd0f0036790 0x7fd0f0038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:12.308 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:10.536+0000 7fd10935d700 1 --2- 172.21.15.94:0/3214251049 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7fd0f0036790 0x7fd0f0038c40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4009990 tx=0x7fd0f4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:12.308 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:10.540+0000 7fd0fa7fc700 1 -- 172.21.15.94:0/3214251049 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fd10002c930 con 0x7fd104076c20 2024-03-24T10:37:12.308 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:10.780+0000 7fd10bdc2700 1 -- 172.21.15.94:0/3214251049 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fd10404bed0 con 0x7fd104076c20 2024-03-24T10:37:12.308 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:11.642+0000 7fd0fa7fc700 1 -- 172.21.15.94:0/3214251049 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 9) v1 ==== 43381+0+0 (secure 0 0 0) 0x7fd10002e440 con 0x7fd104076c20 2024-03-24T10:37:12.308 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:11.644+0000 7fd0fa7fc700 1 -- 172.21.15.94:0/3214251049 <== mon.0 v2:172.21.15.94:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7fd10000f4c0 con 0x7fd104076c20 2024-03-24T10:37:12.308 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:11.647+0000 7fd10bdc2700 1 -- 172.21.15.94:0/3214251049 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7fd0f0036790 msgr2=0x7fd0f0038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:12.308 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:11.647+0000 7fd10bdc2700 1 --2- 172.21.15.94:0/3214251049 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7fd0f0036790 0x7fd0f0038c40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4009990 tx=0x7fd0f4006e40 comp rx=0 tx=0).stop 2024-03-24T10:37:12.308 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:11.647+0000 7fd10bdc2700 1 -- 172.21.15.94:0/3214251049 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd104076c20 msgr2=0x7fd104111710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:12.308 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:11.647+0000 7fd10bdc2700 1 --2- 172.21.15.94:0/3214251049 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd104076c20 0x7fd104111710 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd100006d40 tx=0x7fd100009250 comp rx=0 tx=0).stop 2024-03-24T10:37:12.308 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:11.647+0000 7fd10bdc2700 1 -- 172.21.15.94:0/3214251049 shutdown_connections 2024-03-24T10:37:12.308 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:11.647+0000 7fd10bdc2700 1 --2- 172.21.15.94:0/3214251049 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7fd0f0036790 0x7fd0f0038c40 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:12.308 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:11.647+0000 7fd10bdc2700 1 --2- 172.21.15.94:0/3214251049 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd104076c20 0x7fd104111710 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:12.308 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:11.647+0000 7fd10bdc2700 1 -- 172.21.15.94:0/3214251049 >> 172.21.15.94:0/3214251049 conn(0x7fd1040fcf40 msgr2=0x7fd1040fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:37:12.308 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:11.648+0000 7fd10bdc2700 1 -- 172.21.15.94:0/3214251049 shutdown_connections 2024-03-24T10:37:12.308 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:11.648+0000 7fd10bdc2700 1 -- 172.21.15.94:0/3214251049 wait complete. 2024-03-24T10:37:12.913 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:12 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/3214251049' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-24T10:37:12.913 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:12 smithi094 ceph-mon[29446]: mgrmap e9: smithi094.yaethu(active, since 23s) 2024-03-24T10:37:13.530 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-03-24T10:37:13.530 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-24T10:37:13.530 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-24T10:37:13.530 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "active_name": "smithi094.yaethu", 2024-03-24T10:37:13.530 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-24T10:37:13.530 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-03-24T10:37:13.530 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.719+0000 7f733435d700 1 Processor -- start 2024-03-24T10:37:13.530 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.719+0000 7f733435d700 1 -- start start 2024-03-24T10:37:13.530 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.720+0000 7f733435d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f732c1028c0 0x7f732c102cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:13.531 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.720+0000 7f733435d700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f732c103210 con 0x7f732c1028c0 2024-03-24T10:37:13.531 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.720+0000 7f73320f9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f732c1028c0 0x7f732c102cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:13.531 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.720+0000 7f73320f9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f732c1028c0 0x7f732c102cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40048/0 (socket says 172.21.15.94:40048) 2024-03-24T10:37:13.531 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.720+0000 7f73320f9700 1 -- 172.21.15.94:0/2868978161 learned_addr learned my addr 172.21.15.94:0/2868978161 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:37:13.531 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.720+0000 7f73320f9700 1 -- 172.21.15.94:0/2868978161 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f732c103350 con 0x7f732c1028c0 2024-03-24T10:37:13.531 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.720+0000 7f73320f9700 1 --2- 172.21.15.94:0/2868978161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f732c1028c0 0x7f732c102cd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f731c0098e0 tx=0x7f731c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d00e9295995087e server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:13.531 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.720+0000 7f73310f7700 1 -- 172.21.15.94:0/2868978161 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f731c00b860 con 0x7f732c1028c0 2024-03-24T10:37:13.531 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.721+0000 7f73310f7700 1 -- 172.21.15.94:0/2868978161 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f731c00be90 con 0x7f732c1028c0 2024-03-24T10:37:13.531 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.721+0000 7f73310f7700 1 -- 172.21.15.94:0/2868978161 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f731c0103c0 con 0x7f732c1028c0 2024-03-24T10:37:13.531 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.721+0000 7f733435d700 1 -- 172.21.15.94:0/2868978161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f732c1028c0 msgr2=0x7f732c102cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:13.531 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.721+0000 7f733435d700 1 --2- 172.21.15.94:0/2868978161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f732c1028c0 0x7f732c102cd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f731c0098e0 tx=0x7f731c009250 comp rx=0 tx=0).stop 2024-03-24T10:37:13.531 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.721+0000 7f733435d700 1 -- 172.21.15.94:0/2868978161 shutdown_connections 2024-03-24T10:37:13.531 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.721+0000 7f733435d700 1 --2- 172.21.15.94:0/2868978161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f732c1028c0 0x7f732c102cd0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:13.531 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.721+0000 7f733435d700 1 -- 172.21.15.94:0/2868978161 >> 172.21.15.94:0/2868978161 conn(0x7f732c0fe080 msgr2=0x7f732c100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:37:13.532 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.721+0000 7f733435d700 1 -- 172.21.15.94:0/2868978161 shutdown_connections 2024-03-24T10:37:13.532 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.721+0000 7f733435d700 1 -- 172.21.15.94:0/2868978161 wait complete. 2024-03-24T10:37:13.532 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.722+0000 7f733435d700 1 Processor -- start 2024-03-24T10:37:13.532 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.722+0000 7f733435d700 1 -- start start 2024-03-24T10:37:13.532 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.722+0000 7f733435d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f732c1028c0 0x7f732c1116c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:13.533 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.722+0000 7f733435d700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f732c111c00 con 0x7f732c1028c0 2024-03-24T10:37:13.533 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.722+0000 7f73320f9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f732c1028c0 0x7f732c1116c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:13.533 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.722+0000 7f73320f9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f732c1028c0 0x7f732c1116c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40050/0 (socket says 172.21.15.94:40050) 2024-03-24T10:37:13.533 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.722+0000 7f73320f9700 1 -- 172.21.15.94:0/385015867 learned_addr learned my addr 172.21.15.94:0/385015867 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:37:13.533 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.722+0000 7f73320f9700 1 -- 172.21.15.94:0/385015867 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f731c0094f0 con 0x7f732c1028c0 2024-03-24T10:37:13.533 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.723+0000 7f73320f9700 1 --2- 172.21.15.94:0/385015867 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f732c1028c0 0x7f732c1116c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f731c011680 tx=0x7f731c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:13.533 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.723+0000 7f7322ffd700 1 -- 172.21.15.94:0/385015867 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f731c010890 con 0x7f732c1028c0 2024-03-24T10:37:13.533 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.723+0000 7f7322ffd700 1 -- 172.21.15.94:0/385015867 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f731c010ec0 con 0x7f732c1028c0 2024-03-24T10:37:13.533 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.723+0000 7f7322ffd700 1 -- 172.21.15.94:0/385015867 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f731c0243f0 con 0x7f732c1028c0 2024-03-24T10:37:13.533 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.723+0000 7f733435d700 1 -- 172.21.15.94:0/385015867 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f732c111e00 con 0x7f732c1028c0 2024-03-24T10:37:13.533 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.723+0000 7f733435d700 1 -- 172.21.15.94:0/385015867 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f732c112970 con 0x7f732c1028c0 2024-03-24T10:37:13.533 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.724+0000 7f7322ffd700 1 -- 172.21.15.94:0/385015867 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f731c02d3f0 con 0x7f732c1028c0 2024-03-24T10:37:13.533 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.724+0000 7f7322ffd700 1 --2- 172.21.15.94:0/385015867 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f7318036830 0x7f7318038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:13.533 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.724+0000 7f7322ffd700 1 -- 172.21.15.94:0/385015867 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f731c04a2a0 con 0x7f732c1028c0 2024-03-24T10:37:13.534 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.724+0000 7f73318f8700 1 -- 172.21.15.94:0/385015867 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f7318036830 msgr2=0x7f7318038ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/1996608546 2024-03-24T10:37:13.534 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.724+0000 7f73318f8700 1 --2- 172.21.15.94:0/385015867 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f7318036830 0x7f7318038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-24T10:37:13.534 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.724+0000 7f733435d700 1 -- 172.21.15.94:0/385015867 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f732c05f5f0 con 0x7f732c1028c0 2024-03-24T10:37:13.534 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.729+0000 7f7322ffd700 1 -- 172.21.15.94:0/385015867 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f731c02b8e0 con 0x7f732c1028c0 2024-03-24T10:37:13.534 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.925+0000 7f73318f8700 1 -- 172.21.15.94:0/385015867 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f7318036830 msgr2=0x7f7318038ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/1996608546 2024-03-24T10:37:13.534 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.925+0000 7f73318f8700 1 --2- 172.21.15.94:0/385015867 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f7318036830 0x7f7318038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-24T10:37:13.534 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.975+0000 7f733435d700 1 -- 172.21.15.94:0/385015867 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f732c112bf0 con 0x7f732c1028c0 2024-03-24T10:37:13.534 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.975+0000 7f7322ffd700 1 -- 172.21.15.94:0/385015867 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7f731c02c350 con 0x7f732c1028c0 2024-03-24T10:37:13.534 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.977+0000 7f733435d700 1 -- 172.21.15.94:0/385015867 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f7318036830 msgr2=0x7f7318038ce0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-24T10:37:13.534 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.977+0000 7f733435d700 1 --2- 172.21.15.94:0/385015867 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f7318036830 0x7f7318038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:13.534 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.977+0000 7f733435d700 1 -- 172.21.15.94:0/385015867 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f732c1028c0 msgr2=0x7f732c1116c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:13.534 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.977+0000 7f733435d700 1 --2- 172.21.15.94:0/385015867 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f732c1028c0 0x7f732c1116c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f731c011680 tx=0x7f731c009250 comp rx=0 tx=0).stop 2024-03-24T10:37:13.534 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.977+0000 7f733435d700 1 -- 172.21.15.94:0/385015867 shutdown_connections 2024-03-24T10:37:13.534 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.977+0000 7f733435d700 1 --2- 172.21.15.94:0/385015867 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f7318036830 0x7f7318038ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:13.534 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.977+0000 7f733435d700 1 --2- 172.21.15.94:0/385015867 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f732c1028c0 0x7f732c1116c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:13.535 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.977+0000 7f733435d700 1 -- 172.21.15.94:0/385015867 >> 172.21.15.94:0/385015867 conn(0x7f732c0fe080 msgr2=0x7f732c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:37:13.535 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.977+0000 7f733435d700 1 -- 172.21.15.94:0/385015867 shutdown_connections 2024-03-24T10:37:13.535 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:12.977+0000 7f733435d700 1 -- 172.21.15.94:0/385015867 wait complete. 2024-03-24T10:37:13.535 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for the mgr to restart... 2024-03-24T10:37:13.535 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for mgr epoch 9... 2024-03-24T10:37:13.744 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:13 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/385015867' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-24T10:37:18.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:18 smithi094 ceph-mon[29446]: Active manager daemon smithi094.yaethu restarted 2024-03-24T10:37:18.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:18 smithi094 ceph-mon[29446]: Activating manager daemon smithi094.yaethu 2024-03-24T10:37:18.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:18 smithi094 ceph-mon[29446]: osdmap e3: 0 total, 0 up, 0 in 2024-03-24T10:37:18.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:18 smithi094 ceph-mon[29446]: mgrmap e10: smithi094.yaethu(active, starting, since 0.00328796s) 2024-03-24T10:37:18.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:18 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata", "id": "smithi094"}]: dispatch 2024-03-24T10:37:18.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:18 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mgr metadata", "who": "smithi094.yaethu", "id": "smithi094.yaethu"}]: dispatch 2024-03-24T10:37:18.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:18 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-24T10:37:18.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:18 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-24T10:37:18.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:18 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-24T10:37:18.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:18 smithi094 ceph-mon[29446]: Manager daemon smithi094.yaethu is now available 2024-03-24T10:37:18.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:18 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:37:18.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:18 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:37:19.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:19 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi094.yaethu/mirror_snapshot_schedule"}]: dispatch 2024-03-24T10:37:19.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:19 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi094.yaethu/trash_purge_schedule"}]: dispatch 2024-03-24T10:37:19.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:19 smithi094 ceph-mon[29446]: [24/Mar/2024:10:37:18] ENGINE Bus STARTING 2024-03-24T10:37:19.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:19 smithi094 ceph-mon[29446]: [24/Mar/2024:10:37:18] ENGINE Serving on https://172.21.15.94:7150 2024-03-24T10:37:19.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:19 smithi094 ceph-mon[29446]: [24/Mar/2024:10:37:18] ENGINE Bus STARTED 2024-03-24T10:37:19.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:19 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' 2024-03-24T10:37:19.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:19 smithi094 ceph-mon[29446]: mgrmap e11: smithi094.yaethu(active, since 1.00569s) 2024-03-24T10:37:19.952 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-03-24T10:37:19.952 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-24T10:37:19.952 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-24T10:37:19.952 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-03-24T10:37:19.952 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:13.954+0000 7f8cd5a67700 1 Processor -- start 2024-03-24T10:37:19.952 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:13.954+0000 7f8cd5a67700 1 -- start start 2024-03-24T10:37:19.952 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:13.954+0000 7f8cd5a67700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cd01028c0 0x7f8cd0102cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:19.953 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:13.954+0000 7f8cd5a67700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd0103210 con 0x7f8cd01028c0 2024-03-24T10:37:19.953 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:13.954+0000 7f8ccf7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cd01028c0 0x7f8cd0102cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:19.953 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:13.954+0000 7f8ccf7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cd01028c0 0x7f8cd0102cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40052/0 (socket says 172.21.15.94:40052) 2024-03-24T10:37:19.953 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:13.954+0000 7f8ccf7fe700 1 -- 172.21.15.94:0/2348848946 learned_addr learned my addr 172.21.15.94:0/2348848946 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:37:19.953 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:13.955+0000 7f8ccf7fe700 1 -- 172.21.15.94:0/2348848946 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd0103350 con 0x7f8cd01028c0 2024-03-24T10:37:19.953 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:13.955+0000 7f8ccf7fe700 1 --2- 172.21.15.94:0/2348848946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cd01028c0 0x7f8cd0102cd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8cc40098e0 tx=0x7f8cc4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e71bc000814980d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:19.954 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:13.955+0000 7f8cce7fc700 1 -- 172.21.15.94:0/2348848946 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8cc400b860 con 0x7f8cd01028c0 2024-03-24T10:37:19.954 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:13.955+0000 7f8cce7fc700 1 -- 172.21.15.94:0/2348848946 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8cc400be90 con 0x7f8cd01028c0 2024-03-24T10:37:19.954 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:13.955+0000 7f8cce7fc700 1 -- 172.21.15.94:0/2348848946 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8cc40103c0 con 0x7f8cd01028c0 2024-03-24T10:37:19.954 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:13.956+0000 7f8cd5a67700 1 -- 172.21.15.94:0/2348848946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cd01028c0 msgr2=0x7f8cd0102cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:19.954 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:13.956+0000 7f8cd5a67700 1 --2- 172.21.15.94:0/2348848946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cd01028c0 0x7f8cd0102cd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8cc40098e0 tx=0x7f8cc4009250 comp rx=0 tx=0).stop 2024-03-24T10:37:19.954 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:13.956+0000 7f8cd5a67700 1 -- 172.21.15.94:0/2348848946 shutdown_connections 2024-03-24T10:37:19.954 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:13.956+0000 7f8cd5a67700 1 --2- 172.21.15.94:0/2348848946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cd01028c0 0x7f8cd0102cd0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:19.954 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:13.956+0000 7f8cd5a67700 1 -- 172.21.15.94:0/2348848946 >> 172.21.15.94:0/2348848946 conn(0x7f8cd00fe080 msgr2=0x7f8cd0100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:37:19.954 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:13.956+0000 7f8cd5a67700 1 -- 172.21.15.94:0/2348848946 shutdown_connections 2024-03-24T10:37:19.954 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:13.956+0000 7f8cd5a67700 1 -- 172.21.15.94:0/2348848946 wait complete. 2024-03-24T10:37:19.954 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:13.956+0000 7f8cd5a67700 1 Processor -- start 2024-03-24T10:37:19.954 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:13.956+0000 7f8cd5a67700 1 -- start start 2024-03-24T10:37:19.954 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:13.957+0000 7f8cd5a67700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cd01028c0 0x7f8cd0078c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:19.954 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:13.957+0000 7f8cd5a67700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd0079180 con 0x7f8cd01028c0 2024-03-24T10:37:19.954 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:13.957+0000 7f8ccf7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cd01028c0 0x7f8cd0078c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:19.955 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:13.957+0000 7f8ccf7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cd01028c0 0x7f8cd0078c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40060/0 (socket says 172.21.15.94:40060) 2024-03-24T10:37:19.955 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:13.957+0000 7f8ccf7fe700 1 -- 172.21.15.94:0/2296167683 learned_addr learned my addr 172.21.15.94:0/2296167683 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:37:19.955 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:13.957+0000 7f8ccf7fe700 1 -- 172.21.15.94:0/2296167683 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc40094f0 con 0x7f8cd01028c0 2024-03-24T10:37:19.955 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:13.957+0000 7f8ccf7fe700 1 --2- 172.21.15.94:0/2296167683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cd01028c0 0x7f8cd0078c40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8cc40090b0 tx=0x7f8cc4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:19.955 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:13.957+0000 7f8cb7fff700 1 -- 172.21.15.94:0/2296167683 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8cc4010840 con 0x7f8cd01028c0 2024-03-24T10:37:19.955 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:13.957+0000 7f8cb7fff700 1 -- 172.21.15.94:0/2296167683 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8cc4010e70 con 0x7f8cd01028c0 2024-03-24T10:37:19.955 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:13.957+0000 7f8cd5a67700 1 -- 172.21.15.94:0/2296167683 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cd00771f0 con 0x7f8cd01028c0 2024-03-24T10:37:19.955 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:13.957+0000 7f8cb7fff700 1 -- 172.21.15.94:0/2296167683 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8cc4024e40 con 0x7f8cd01028c0 2024-03-24T10:37:19.955 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:13.957+0000 7f8cd5a67700 1 -- 172.21.15.94:0/2296167683 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cd00776e0 con 0x7f8cd01028c0 2024-03-24T10:37:19.955 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:13.958+0000 7f8cb7fff700 1 -- 172.21.15.94:0/2296167683 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f8cc401a510 con 0x7f8cd01028c0 2024-03-24T10:37:19.955 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:13.958+0000 7f8cb7fff700 1 --2- 172.21.15.94:0/2296167683 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f8cb8036790 0x7f8cb8038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:19.955 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:13.958+0000 7f8cb7fff700 1 -- 172.21.15.94:0/2296167683 --> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8cb8039350 con 0x7f8cb8036790 2024-03-24T10:37:19.955 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:13.958+0000 7f8cceffd700 1 -- 172.21.15.94:0/2296167683 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f8cb8036790 msgr2=0x7f8cb8038c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/1996608546 2024-03-24T10:37:19.955 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:13.958+0000 7f8cb7fff700 1 -- 172.21.15.94:0/2296167683 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f8cc404b860 con 0x7f8cd01028c0 2024-03-24T10:37:19.956 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:13.958+0000 7f8cceffd700 1 --2- 172.21.15.94:0/2296167683 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f8cb8036790 0x7f8cb8038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-24T10:37:19.956 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:14.159+0000 7f8cceffd700 1 -- 172.21.15.94:0/2296167683 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f8cb8036790 msgr2=0x7f8cb8038c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/1996608546 2024-03-24T10:37:19.956 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:14.159+0000 7f8cceffd700 1 --2- 172.21.15.94:0/2296167683 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f8cb8036790 0x7f8cb8038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-24T10:37:19.956 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:14.560+0000 7f8cceffd700 1 -- 172.21.15.94:0/2296167683 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f8cb8036790 msgr2=0x7f8cb8038c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/1996608546 2024-03-24T10:37:19.956 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:14.560+0000 7f8cceffd700 1 --2- 172.21.15.94:0/2296167683 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f8cb8036790 0x7f8cb8038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-24T10:37:19.956 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:15.361+0000 7f8cceffd700 1 -- 172.21.15.94:0/2296167683 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f8cb8036790 msgr2=0x7f8cb8038c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/1996608546 2024-03-24T10:37:19.956 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:15.361+0000 7f8cceffd700 1 --2- 172.21.15.94:0/2296167683 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f8cb8036790 0x7f8cb8038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-24T10:37:19.956 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:16.962+0000 7f8cceffd700 1 -- 172.21.15.94:0/2296167683 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f8cb8036790 msgr2=0x7f8cb8038c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/1996608546 2024-03-24T10:37:19.956 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:16.962+0000 7f8cceffd700 1 --2- 172.21.15.94:0/2296167683 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f8cb8036790 0x7f8cb8038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-24T10:37:19.956 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:18.183+0000 7f8cb7fff700 1 -- 172.21.15.94:0/2296167683 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mgrmap(e 10) v1 ==== 43148+0+0 (secure 0 0 0) 0x7f8cc4024430 con 0x7f8cd01028c0 2024-03-24T10:37:19.956 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:18.184+0000 7f8cb7fff700 1 -- 172.21.15.94:0/2296167683 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f8cb8036790 msgr2=0x7f8cb8038c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-24T10:37:19.956 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:18.184+0000 7f8cb7fff700 1 --2- 172.21.15.94:0/2296167683 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f8cb8036790 0x7f8cb8038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:19.956 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:19.185+0000 7f8cb7fff700 1 -- 172.21.15.94:0/2296167683 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 11) v1 ==== 43275+0+0 (secure 0 0 0) 0x7f8cc401abe0 con 0x7f8cd01028c0 2024-03-24T10:37:19.956 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:19.185+0000 7f8cb7fff700 1 --2- 172.21.15.94:0/2296167683 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f8cb803a070 0x7f8cb803c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:19.956 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:19.185+0000 7f8cb7fff700 1 -- 172.21.15.94:0/2296167683 --> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8cb8039350 con 0x7f8cb803a070 2024-03-24T10:37:19.956 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:19.186+0000 7f8cceffd700 1 --2- 172.21.15.94:0/2296167683 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f8cb803a070 0x7f8cb803c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:19.957 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:19.186+0000 7f8cceffd700 1 --2- 172.21.15.94:0/2296167683 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f8cb803a070 0x7f8cb803c450 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00039d0 tx=0x7f8cc00092f0 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:19.957 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:19.187+0000 7f8cb7fff700 1 -- 172.21.15.94:0/2296167683 <== mgr.14162 v2:172.21.15.94:6800/2406899470 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7f8cb8039350 con 0x7f8cb803a070 2024-03-24T10:37:19.957 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:19.193+0000 7f8cd5a67700 1 -- 172.21.15.94:0/2296167683 --> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f8cd004bed0 con 0x7f8cb803a070 2024-03-24T10:37:19.957 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:19.193+0000 7f8cb7fff700 1 -- 172.21.15.94:0/2296167683 <== mgr.14162 v2:172.21.15.94:6800/2406899470 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f8cd004bed0 con 0x7f8cb803a070 2024-03-24T10:37:19.957 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:19.193+0000 7f8cd5a67700 1 -- 172.21.15.94:0/2296167683 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f8cb803a070 msgr2=0x7f8cb803c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:19.957 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:19.193+0000 7f8cd5a67700 1 --2- 172.21.15.94:0/2296167683 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f8cb803a070 0x7f8cb803c450 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00039d0 tx=0x7f8cc00092f0 comp rx=0 tx=0).stop 2024-03-24T10:37:19.957 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:19.193+0000 7f8cd5a67700 1 -- 172.21.15.94:0/2296167683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cd01028c0 msgr2=0x7f8cd0078c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:19.957 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:19.193+0000 7f8cd5a67700 1 --2- 172.21.15.94:0/2296167683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cd01028c0 0x7f8cd0078c40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8cc40090b0 tx=0x7f8cc4009250 comp rx=0 tx=0).stop 2024-03-24T10:37:19.957 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:19.193+0000 7f8cd5a67700 1 -- 172.21.15.94:0/2296167683 shutdown_connections 2024-03-24T10:37:19.957 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:19.193+0000 7f8cd5a67700 1 --2- 172.21.15.94:0/2296167683 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f8cb803a070 0x7f8cb803c450 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:19.957 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:19.193+0000 7f8cd5a67700 1 --2- 172.21.15.94:0/2296167683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cd01028c0 0x7f8cd0078c40 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:19.957 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:19.193+0000 7f8cd5a67700 1 --2- 172.21.15.94:0/2296167683 >> [v2:172.21.15.94:6800/1996608546,v1:172.21.15.94:6801/1996608546] conn(0x7f8cb8036790 0x7f8cb8038c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:19.957 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:19.194+0000 7f8cd5a67700 1 -- 172.21.15.94:0/2296167683 >> 172.21.15.94:0/2296167683 conn(0x7f8cd00fe080 msgr2=0x7f8cd00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:37:19.957 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:19.194+0000 7f8cd5a67700 1 -- 172.21.15.94:0/2296167683 shutdown_connections 2024-03-24T10:37:19.957 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:19.194+0000 7f8cd5a67700 1 -- 172.21.15.94:0/2296167683 wait complete. 2024-03-24T10:37:19.958 INFO:teuthology.orchestra.run.smithi094.stdout:mgr epoch 9 is available 2024-03-24T10:37:19.958 INFO:teuthology.orchestra.run.smithi094.stdout:Generating a dashboard self-signed certificate... 2024-03-24T10:37:20.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:20 smithi094 ceph-mon[29446]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-24T10:37:20.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:20 smithi094 ceph-mon[29446]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-24T10:37:20.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:20 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' 2024-03-24T10:37:21.514 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-24T10:37:21.514 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.365+0000 7fb041459700 1 Processor -- start 2024-03-24T10:37:21.514 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.365+0000 7fb041459700 1 -- start start 2024-03-24T10:37:21.514 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.365+0000 7fb041459700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb03c101890 0x7fb03c101ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:21.515 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.365+0000 7fb041459700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c1021e0 con 0x7fb03c101890 2024-03-24T10:37:21.515 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.365+0000 7fb03affd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb03c101890 0x7fb03c101ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:21.515 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.365+0000 7fb03affd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb03c101890 0x7fb03c101ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49694/0 (socket says 172.21.15.94:49694) 2024-03-24T10:37:21.515 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.365+0000 7fb03affd700 1 -- 172.21.15.94:0/1928884870 learned_addr learned my addr 172.21.15.94:0/1928884870 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:37:21.515 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.365+0000 7fb03affd700 1 -- 172.21.15.94:0/1928884870 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb03c102320 con 0x7fb03c101890 2024-03-24T10:37:21.516 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.365+0000 7fb03affd700 1 --2- 172.21.15.94:0/1928884870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb03c101890 0x7fb03c101ca0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb02c0098e0 tx=0x7fb02c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1681df4d76cf8da4 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:21.516 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.366+0000 7fb039ffb700 1 -- 172.21.15.94:0/1928884870 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb02c00b8f0 con 0x7fb03c101890 2024-03-24T10:37:21.516 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.366+0000 7fb039ffb700 1 -- 172.21.15.94:0/1928884870 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb02c011420 con 0x7fb03c101890 2024-03-24T10:37:21.516 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.366+0000 7fb039ffb700 1 -- 172.21.15.94:0/1928884870 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb02c010540 con 0x7fb03c101890 2024-03-24T10:37:21.516 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.366+0000 7fb041459700 1 -- 172.21.15.94:0/1928884870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb03c101890 msgr2=0x7fb03c101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:21.516 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.366+0000 7fb041459700 1 --2- 172.21.15.94:0/1928884870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb03c101890 0x7fb03c101ca0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb02c0098e0 tx=0x7fb02c009250 comp rx=0 tx=0).stop 2024-03-24T10:37:21.516 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.366+0000 7fb041459700 1 -- 172.21.15.94:0/1928884870 shutdown_connections 2024-03-24T10:37:21.517 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.366+0000 7fb041459700 1 --2- 172.21.15.94:0/1928884870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb03c101890 0x7fb03c101ca0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:21.517 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.366+0000 7fb041459700 1 -- 172.21.15.94:0/1928884870 >> 172.21.15.94:0/1928884870 conn(0x7fb03c0fd010 msgr2=0x7fb03c0ff420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:37:21.517 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.366+0000 7fb041459700 1 -- 172.21.15.94:0/1928884870 shutdown_connections 2024-03-24T10:37:21.517 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.366+0000 7fb041459700 1 -- 172.21.15.94:0/1928884870 wait complete. 2024-03-24T10:37:21.517 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.367+0000 7fb041459700 1 Processor -- start 2024-03-24T10:37:21.517 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.367+0000 7fb041459700 1 -- start start 2024-03-24T10:37:21.517 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.367+0000 7fb041459700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb03c101890 0x7fb03c076b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:21.517 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.367+0000 7fb041459700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c114480 con 0x7fb03c101890 2024-03-24T10:37:21.518 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.367+0000 7fb03affd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb03c101890 0x7fb03c076b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:21.518 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.367+0000 7fb03affd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb03c101890 0x7fb03c076b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49696/0 (socket says 172.21.15.94:49696) 2024-03-24T10:37:21.518 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.367+0000 7fb03affd700 1 -- 172.21.15.94:0/3750924868 learned_addr learned my addr 172.21.15.94:0/3750924868 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:37:21.518 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.368+0000 7fb03affd700 1 -- 172.21.15.94:0/3750924868 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb02c0094f0 con 0x7fb03c101890 2024-03-24T10:37:21.518 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.368+0000 7fb03affd700 1 --2- 172.21.15.94:0/3750924868 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb03c101890 0x7fb03c076b20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb02c006d40 tx=0x7fb02c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:21.518 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.368+0000 7fb0237fe700 1 -- 172.21.15.94:0/3750924868 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb02c00beb0 con 0x7fb03c101890 2024-03-24T10:37:21.519 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.368+0000 7fb0237fe700 1 -- 172.21.15.94:0/3750924868 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb02c010c80 con 0x7fb03c101890 2024-03-24T10:37:21.519 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.368+0000 7fb041459700 1 -- 172.21.15.94:0/3750924868 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb03c1145c0 con 0x7fb03c101890 2024-03-24T10:37:21.519 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.368+0000 7fb0237fe700 1 -- 172.21.15.94:0/3750924868 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb02c024d40 con 0x7fb03c101890 2024-03-24T10:37:21.519 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.368+0000 7fb041459700 1 -- 172.21.15.94:0/3750924868 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb03c114a50 con 0x7fb03c101890 2024-03-24T10:37:21.519 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.369+0000 7fb0237fe700 1 -- 172.21.15.94:0/3750924868 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 11) v1 ==== 43275+0+0 (secure 0 0 0) 0x7fb02c01a430 con 0x7fb03c101890 2024-03-24T10:37:21.519 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.369+0000 7fb0237fe700 1 --2- 172.21.15.94:0/3750924868 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fb024036740 0x7fb024038bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:21.519 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.369+0000 7fb0237fe700 1 -- 172.21.15.94:0/3750924868 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fb02c04a150 con 0x7fb03c101890 2024-03-24T10:37:21.519 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.369+0000 7fb03a7fc700 1 --2- 172.21.15.94:0/3750924868 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fb024036740 0x7fb024038bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:21.520 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.369+0000 7fb041459700 1 -- 172.21.15.94:0/3750924868 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb03c04bed0 con 0x7fb03c101890 2024-03-24T10:37:21.520 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.369+0000 7fb03a7fc700 1 --2- 172.21.15.94:0/3750924868 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fb024036740 0x7fb024038bf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb030009990 tx=0x7fb030006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:21.520 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.374+0000 7fb0237fe700 1 -- 172.21.15.94:0/3750924868 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fb02c018940 con 0x7fb03c101890 2024-03-24T10:37:21.520 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.495+0000 7fb0237fe700 1 -- 172.21.15.94:0/3750924868 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 12) v1 ==== 43381+0+0 (secure 0 0 0) 0x7fb02c00fda0 con 0x7fb03c101890 2024-03-24T10:37:21.520 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.590+0000 7fb041459700 1 -- 172.21.15.94:0/3750924868 --> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fb03c0fe3d0 con 0x7fb024036740 2024-03-24T10:37:21.520 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.836+0000 7fb0237fe700 1 -- 172.21.15.94:0/3750924868 <== mgr.14162 v2:172.21.15.94:6800/2406899470 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fb03c0fe3d0 con 0x7fb024036740 2024-03-24T10:37:21.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.839+0000 7fb041459700 1 -- 172.21.15.94:0/3750924868 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fb024036740 msgr2=0x7fb024038bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:21.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.839+0000 7fb041459700 1 --2- 172.21.15.94:0/3750924868 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fb024036740 0x7fb024038bf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb030009990 tx=0x7fb030006e40 comp rx=0 tx=0).stop 2024-03-24T10:37:21.522 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.839+0000 7fb041459700 1 -- 172.21.15.94:0/3750924868 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb03c101890 msgr2=0x7fb03c076b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:21.522 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.839+0000 7fb041459700 1 --2- 172.21.15.94:0/3750924868 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb03c101890 0x7fb03c076b20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb02c006d40 tx=0x7fb02c009250 comp rx=0 tx=0).stop 2024-03-24T10:37:21.522 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.840+0000 7fb041459700 1 -- 172.21.15.94:0/3750924868 shutdown_connections 2024-03-24T10:37:21.522 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.840+0000 7fb041459700 1 --2- 172.21.15.94:0/3750924868 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fb024036740 0x7fb024038bf0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:21.522 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.840+0000 7fb041459700 1 --2- 172.21.15.94:0/3750924868 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb03c101890 0x7fb03c076b20 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:21.522 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.840+0000 7fb041459700 1 -- 172.21.15.94:0/3750924868 >> 172.21.15.94:0/3750924868 conn(0x7fb03c0fd010 msgr2=0x7fb03c0fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:37:21.522 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.840+0000 7fb041459700 1 -- 172.21.15.94:0/3750924868 shutdown_connections 2024-03-24T10:37:21.523 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:20.840+0000 7fb041459700 1 -- 172.21.15.94:0/3750924868 wait complete. 2024-03-24T10:37:21.523 INFO:teuthology.orchestra.run.smithi094.stdout:Creating initial admin user... 2024-03-24T10:37:21.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:21 smithi094 ceph-mon[29446]: mgrmap e12: smithi094.yaethu(active, since 2s) 2024-03-24T10:37:21.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:21 smithi094 ceph-mon[29446]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:37:21.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:21 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' 2024-03-24T10:37:21.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:21 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' 2024-03-24T10:37:21.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:21 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' 2024-03-24T10:37:23.365 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$tZTD/CaFKOeXMmm5GfiHV.2TCr0ICGxRDtSgbXh5zSNuFliknrIeu", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1711276642, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-24T10:37:23.365 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:21.935+0000 7f589c135700 1 Processor -- start 2024-03-24T10:37:23.365 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:21.935+0000 7f589c135700 1 -- start start 2024-03-24T10:37:23.366 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:21.935+0000 7f589c135700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5894076af0 0x7f58941055a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:23.366 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:21.935+0000 7f589c135700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5894076f00 con 0x7f5894076af0 2024-03-24T10:37:23.366 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:21.935+0000 7f5899ed1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5894076af0 0x7f58941055a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:23.366 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:21.935+0000 7f5899ed1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5894076af0 0x7f58941055a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49698/0 (socket says 172.21.15.94:49698) 2024-03-24T10:37:23.366 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:21.935+0000 7f5899ed1700 1 -- 172.21.15.94:0/1226564666 learned_addr learned my addr 172.21.15.94:0/1226564666 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:37:23.367 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:21.936+0000 7f5899ed1700 1 -- 172.21.15.94:0/1226564666 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5894105ae0 con 0x7f5894076af0 2024-03-24T10:37:23.367 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:21.936+0000 7f5899ed1700 1 --2- 172.21.15.94:0/1226564666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5894076af0 0x7f58941055a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f58900098e0 tx=0x7f5890009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68dbec3559731a1f server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:23.367 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:21.936+0000 7f5898ecf700 1 -- 172.21.15.94:0/1226564666 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f589000b8f0 con 0x7f5894076af0 2024-03-24T10:37:23.367 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:21.936+0000 7f5898ecf700 1 -- 172.21.15.94:0/1226564666 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5890011420 con 0x7f5894076af0 2024-03-24T10:37:23.367 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:21.936+0000 7f5898ecf700 1 -- 172.21.15.94:0/1226564666 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5890010540 con 0x7f5894076af0 2024-03-24T10:37:23.367 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:21.936+0000 7f589c135700 1 -- 172.21.15.94:0/1226564666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5894076af0 msgr2=0x7f58941055a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:23.367 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:21.936+0000 7f589c135700 1 --2- 172.21.15.94:0/1226564666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5894076af0 0x7f58941055a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f58900098e0 tx=0x7f5890009250 comp rx=0 tx=0).stop 2024-03-24T10:37:23.367 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:21.937+0000 7f589c135700 1 -- 172.21.15.94:0/1226564666 shutdown_connections 2024-03-24T10:37:23.367 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:21.937+0000 7f589c135700 1 --2- 172.21.15.94:0/1226564666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5894076af0 0x7f58941055a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:23.368 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:21.937+0000 7f589c135700 1 -- 172.21.15.94:0/1226564666 >> 172.21.15.94:0/1226564666 conn(0x7f58940fd010 msgr2=0x7f58940ff420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:37:23.368 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:21.937+0000 7f589c135700 1 -- 172.21.15.94:0/1226564666 shutdown_connections 2024-03-24T10:37:23.368 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:21.937+0000 7f589c135700 1 -- 172.21.15.94:0/1226564666 wait complete. 2024-03-24T10:37:23.368 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:21.937+0000 7f589c135700 1 Processor -- start 2024-03-24T10:37:23.368 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:21.937+0000 7f589c135700 1 -- start start 2024-03-24T10:37:23.368 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:21.937+0000 7f589c135700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5894076af0 0x7f58941117a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:23.368 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:21.937+0000 7f589c135700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5894111ce0 con 0x7f5894076af0 2024-03-24T10:37:23.368 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:21.937+0000 7f5899ed1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5894076af0 0x7f58941117a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:23.368 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:21.937+0000 7f5899ed1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5894076af0 0x7f58941117a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49702/0 (socket says 172.21.15.94:49702) 2024-03-24T10:37:23.368 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:21.937+0000 7f5899ed1700 1 -- 172.21.15.94:0/2036595882 learned_addr learned my addr 172.21.15.94:0/2036595882 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:37:23.368 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:21.938+0000 7f5899ed1700 1 -- 172.21.15.94:0/2036595882 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58900094f0 con 0x7f5894076af0 2024-03-24T10:37:23.368 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:21.938+0000 7f5899ed1700 1 --2- 172.21.15.94:0/2036595882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5894076af0 0x7f58941117a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5890011e90 tx=0x7f5890009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:23.368 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:21.938+0000 7f588a7fc700 1 -- 172.21.15.94:0/2036595882 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f589000bd60 con 0x7f5894076af0 2024-03-24T10:37:23.368 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:21.938+0000 7f588a7fc700 1 -- 172.21.15.94:0/2036595882 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f589000bec0 con 0x7f5894076af0 2024-03-24T10:37:23.369 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:21.938+0000 7f588a7fc700 1 -- 172.21.15.94:0/2036595882 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f589001abe0 con 0x7f5894076af0 2024-03-24T10:37:23.369 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:21.938+0000 7f589c135700 1 -- 172.21.15.94:0/2036595882 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5894111ee0 con 0x7f5894076af0 2024-03-24T10:37:23.369 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:21.938+0000 7f589c135700 1 -- 172.21.15.94:0/2036595882 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5894112a50 con 0x7f5894076af0 2024-03-24T10:37:23.369 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:21.939+0000 7f588a7fc700 1 -- 172.21.15.94:0/2036595882 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 12) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f58900193c0 con 0x7f5894076af0 2024-03-24T10:37:23.369 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:21.939+0000 7f589c135700 1 -- 172.21.15.94:0/2036595882 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f589405f5f0 con 0x7f5894076af0 2024-03-24T10:37:23.369 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:21.939+0000 7f588a7fc700 1 --2- 172.21.15.94:0/2036595882 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f58800368c0 0x7f5880038d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:23.369 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:21.939+0000 7f588a7fc700 1 -- 172.21.15.94:0/2036595882 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f589004a680 con 0x7f5894076af0 2024-03-24T10:37:23.369 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:21.939+0000 7f58996d0700 1 --2- 172.21.15.94:0/2036595882 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f58800368c0 0x7f5880038d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:23.369 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:21.939+0000 7f58996d0700 1 --2- 172.21.15.94:0/2036595882 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f58800368c0 0x7f5880038d70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5884009990 tx=0x7f5884006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:23.370 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:21.944+0000 7f588a7fc700 1 -- 172.21.15.94:0/2036595882 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f5890018ce0 con 0x7f5894076af0 2024-03-24T10:37:23.370 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:22.152+0000 7f589c135700 1 -- 172.21.15.94:0/2036595882 --> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f589405f410 con 0x7f58800368c0 2024-03-24T10:37:23.370 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:22.376+0000 7f588a7fc700 1 -- 172.21.15.94:0/2036595882 <== mgr.14162 v2:172.21.15.94:6800/2406899470 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f589405f410 con 0x7f58800368c0 2024-03-24T10:37:23.370 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:22.379+0000 7f589c135700 1 -- 172.21.15.94:0/2036595882 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f58800368c0 msgr2=0x7f5880038d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:23.370 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:22.379+0000 7f589c135700 1 --2- 172.21.15.94:0/2036595882 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f58800368c0 0x7f5880038d70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5884009990 tx=0x7f5884006e40 comp rx=0 tx=0).stop 2024-03-24T10:37:23.370 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:22.379+0000 7f589c135700 1 -- 172.21.15.94:0/2036595882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5894076af0 msgr2=0x7f58941117a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:23.370 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:22.379+0000 7f589c135700 1 --2- 172.21.15.94:0/2036595882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5894076af0 0x7f58941117a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5890011e90 tx=0x7f5890009250 comp rx=0 tx=0).stop 2024-03-24T10:37:23.370 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:22.379+0000 7f589c135700 1 -- 172.21.15.94:0/2036595882 shutdown_connections 2024-03-24T10:37:23.371 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:22.379+0000 7f589c135700 1 --2- 172.21.15.94:0/2036595882 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f58800368c0 0x7f5880038d70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:23.371 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:22.379+0000 7f589c135700 1 --2- 172.21.15.94:0/2036595882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5894076af0 0x7f58941117a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:23.371 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:22.379+0000 7f589c135700 1 -- 172.21.15.94:0/2036595882 >> 172.21.15.94:0/2036595882 conn(0x7f58940fd010 msgr2=0x7f58940fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:37:23.371 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:22.379+0000 7f589c135700 1 -- 172.21.15.94:0/2036595882 shutdown_connections 2024-03-24T10:37:23.372 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:22.379+0000 7f589c135700 1 -- 172.21.15.94:0/2036595882 wait complete. 2024-03-24T10:37:23.372 INFO:teuthology.orchestra.run.smithi094.stdout:Fetching dashboard port number... 2024-03-24T10:37:23.615 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:23 smithi094 ceph-mon[29446]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:37:23.615 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:23 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' 2024-03-24T10:37:24.723 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:24 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/1017770123' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-24T10:37:24.731 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 8443 2024-03-24T10:37:24.731 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:23.814+0000 7f45c3ac1700 1 Processor -- start 2024-03-24T10:37:24.731 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:23.815+0000 7f45c3ac1700 1 -- start start 2024-03-24T10:37:24.731 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:23.815+0000 7f45c3ac1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45bc0712c0 0x7f45bc1054e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:24.731 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:23.815+0000 7f45c3ac1700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45bc105a20 con 0x7f45bc0712c0 2024-03-24T10:37:24.731 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:23.815+0000 7f45c185d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45bc0712c0 0x7f45bc1054e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:24.731 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:23.815+0000 7f45c185d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45bc0712c0 0x7f45bc1054e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49712/0 (socket says 172.21.15.94:49712) 2024-03-24T10:37:24.731 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:23.815+0000 7f45c185d700 1 -- 172.21.15.94:0/1937705614 learned_addr learned my addr 172.21.15.94:0/1937705614 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:37:24.731 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:23.815+0000 7f45c185d700 1 -- 172.21.15.94:0/1937705614 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45bc105b60 con 0x7f45bc0712c0 2024-03-24T10:37:24.731 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:23.815+0000 7f45c185d700 1 --2- 172.21.15.94:0/1937705614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45bc0712c0 0x7f45bc1054e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f45b80098e0 tx=0x7f45b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71fccb65cf29baa7 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:24.732 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:23.815+0000 7f45c085b700 1 -- 172.21.15.94:0/1937705614 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f45b800b860 con 0x7f45bc0712c0 2024-03-24T10:37:24.732 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:23.816+0000 7f45c085b700 1 -- 172.21.15.94:0/1937705614 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f45b800be90 con 0x7f45bc0712c0 2024-03-24T10:37:24.732 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:23.816+0000 7f45c085b700 1 -- 172.21.15.94:0/1937705614 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f45b80103c0 con 0x7f45bc0712c0 2024-03-24T10:37:24.732 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:23.816+0000 7f45c3ac1700 1 -- 172.21.15.94:0/1937705614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45bc0712c0 msgr2=0x7f45bc1054e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:24.732 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:23.816+0000 7f45c3ac1700 1 --2- 172.21.15.94:0/1937705614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45bc0712c0 0x7f45bc1054e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f45b80098e0 tx=0x7f45b8009250 comp rx=0 tx=0).stop 2024-03-24T10:37:24.732 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:23.816+0000 7f45c3ac1700 1 -- 172.21.15.94:0/1937705614 shutdown_connections 2024-03-24T10:37:24.732 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:23.816+0000 7f45c3ac1700 1 --2- 172.21.15.94:0/1937705614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45bc0712c0 0x7f45bc1054e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:24.732 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:23.816+0000 7f45c3ac1700 1 -- 172.21.15.94:0/1937705614 >> 172.21.15.94:0/1937705614 conn(0x7f45bc0fcfb0 msgr2=0x7f45bc0ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:37:24.732 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:23.816+0000 7f45c3ac1700 1 -- 172.21.15.94:0/1937705614 shutdown_connections 2024-03-24T10:37:24.732 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:23.816+0000 7f45c3ac1700 1 -- 172.21.15.94:0/1937705614 wait complete. 2024-03-24T10:37:24.732 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:23.817+0000 7f45c3ac1700 1 Processor -- start 2024-03-24T10:37:24.732 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:23.817+0000 7f45c3ac1700 1 -- start start 2024-03-24T10:37:24.732 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:23.817+0000 7f45c3ac1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45bc0712c0 0x7f45bc102e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:24.732 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:23.817+0000 7f45c3ac1700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45bc103370 con 0x7f45bc0712c0 2024-03-24T10:37:24.732 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:23.817+0000 7f45c185d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45bc0712c0 0x7f45bc102e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:24.733 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:23.817+0000 7f45c185d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45bc0712c0 0x7f45bc102e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49724/0 (socket says 172.21.15.94:49724) 2024-03-24T10:37:24.733 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:23.817+0000 7f45c185d700 1 -- 172.21.15.94:0/1017770123 learned_addr learned my addr 172.21.15.94:0/1017770123 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:37:24.733 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:23.817+0000 7f45c185d700 1 -- 172.21.15.94:0/1017770123 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b80094f0 con 0x7f45bc0712c0 2024-03-24T10:37:24.733 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:23.817+0000 7f45c185d700 1 --2- 172.21.15.94:0/1017770123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45bc0712c0 0x7f45bc102e30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f45b80094c0 tx=0x7f45b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:24.733 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:23.817+0000 7f45b27fc700 1 -- 172.21.15.94:0/1017770123 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f45b80118f0 con 0x7f45bc0712c0 2024-03-24T10:37:24.733 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:23.817+0000 7f45b27fc700 1 -- 172.21.15.94:0/1017770123 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f45b8010d60 con 0x7f45bc0712c0 2024-03-24T10:37:24.733 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:23.817+0000 7f45c3ac1700 1 -- 172.21.15.94:0/1017770123 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45bc1013e0 con 0x7f45bc0712c0 2024-03-24T10:37:24.733 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:23.817+0000 7f45b27fc700 1 -- 172.21.15.94:0/1017770123 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f45b801a430 con 0x7f45bc0712c0 2024-03-24T10:37:24.733 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:23.818+0000 7f45c3ac1700 1 -- 172.21.15.94:0/1017770123 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45bc1018d0 con 0x7f45bc0712c0 2024-03-24T10:37:24.733 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:23.818+0000 7f45b27fc700 1 -- 172.21.15.94:0/1017770123 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 12) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f45b8010890 con 0x7f45bc0712c0 2024-03-24T10:37:24.733 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:23.818+0000 7f45c3ac1700 1 -- 172.21.15.94:0/1017770123 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45bc04bed0 con 0x7f45bc0712c0 2024-03-24T10:37:24.733 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:23.818+0000 7f45b27fc700 1 --2- 172.21.15.94:0/1017770123 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f45a80367e0 0x7f45a8038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:24.733 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:23.818+0000 7f45b27fc700 1 -- 172.21.15.94:0/1017770123 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f45b804a620 con 0x7f45bc0712c0 2024-03-24T10:37:24.733 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:23.819+0000 7f45c105c700 1 --2- 172.21.15.94:0/1017770123 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f45a80367e0 0x7f45a8038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:24.733 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:23.819+0000 7f45c105c700 1 --2- 172.21.15.94:0/1017770123 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f45a80367e0 0x7f45a8038c90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f45ac009990 tx=0x7f45ac006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:24.734 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:23.823+0000 7f45b27fc700 1 -- 172.21.15.94:0/1017770123 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f45b80227b0 con 0x7f45bc0712c0 2024-03-24T10:37:24.734 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:24.008+0000 7f45c3ac1700 1 -- 172.21.15.94:0/1017770123 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f45bc101d50 con 0x7f45bc0712c0 2024-03-24T10:37:24.734 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:24.009+0000 7f45b27fc700 1 -- 172.21.15.94:0/1017770123 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f45b8010ed0 con 0x7f45bc0712c0 2024-03-24T10:37:24.734 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:24.010+0000 7f45c3ac1700 1 -- 172.21.15.94:0/1017770123 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f45a80367e0 msgr2=0x7f45a8038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:24.734 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:24.010+0000 7f45c3ac1700 1 --2- 172.21.15.94:0/1017770123 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f45a80367e0 0x7f45a8038c90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f45ac009990 tx=0x7f45ac006e40 comp rx=0 tx=0).stop 2024-03-24T10:37:24.734 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:24.010+0000 7f45c3ac1700 1 -- 172.21.15.94:0/1017770123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45bc0712c0 msgr2=0x7f45bc102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:24.734 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:24.010+0000 7f45c3ac1700 1 --2- 172.21.15.94:0/1017770123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45bc0712c0 0x7f45bc102e30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f45b80094c0 tx=0x7f45b8009250 comp rx=0 tx=0).stop 2024-03-24T10:37:24.734 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:24.011+0000 7f45c3ac1700 1 -- 172.21.15.94:0/1017770123 shutdown_connections 2024-03-24T10:37:24.734 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:24.011+0000 7f45c3ac1700 1 --2- 172.21.15.94:0/1017770123 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f45a80367e0 0x7f45a8038c90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:24.734 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:24.011+0000 7f45c3ac1700 1 --2- 172.21.15.94:0/1017770123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45bc0712c0 0x7f45bc102e30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:24.734 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:24.011+0000 7f45c3ac1700 1 -- 172.21.15.94:0/1017770123 >> 172.21.15.94:0/1017770123 conn(0x7f45bc0fcfb0 msgr2=0x7f45bc0fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:37:24.734 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:24.011+0000 7f45c3ac1700 1 -- 172.21.15.94:0/1017770123 shutdown_connections 2024-03-24T10:37:24.734 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:24.011+0000 7f45c3ac1700 1 -- 172.21.15.94:0/1017770123 wait complete. 2024-03-24T10:37:24.738 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-24T10:37:24.738 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout disabled 2024-03-24T10:37:24.745 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-24T10:37:24.745 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout inactive 2024-03-24T10:37:24.745 INFO:teuthology.orchestra.run.smithi094.stdout:firewalld.service is not enabled 2024-03-24T10:37:24.745 INFO:teuthology.orchestra.run.smithi094.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-24T10:37:24.747 INFO:teuthology.orchestra.run.smithi094.stdout:Ceph Dashboard is now available at: 2024-03-24T10:37:24.747 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:37:24.747 INFO:teuthology.orchestra.run.smithi094.stdout: URL: https://smithi094.front.sepia.ceph.com:8443/ 2024-03-24T10:37:24.747 INFO:teuthology.orchestra.run.smithi094.stdout: User: admin 2024-03-24T10:37:24.747 INFO:teuthology.orchestra.run.smithi094.stdout: Password: 2ncglrxb5t 2024-03-24T10:37:24.747 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:37:24.748 INFO:teuthology.orchestra.run.smithi094.stdout:Saving cluster configuration to /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/config directory 2024-03-24T10:37:24.748 INFO:teuthology.orchestra.run.smithi094.stdout:Enabling autotune for osd_memory_target 2024-03-24T10:37:25.655 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:25 smithi094 ceph-mon[29446]: mgrmap e13: smithi094.yaethu(active, since 6s) 2024-03-24T10:37:25.931 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.160+0000 7f350d014700 1 Processor -- start 2024-03-24T10:37:25.932 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.160+0000 7f350d014700 1 -- start start 2024-03-24T10:37:25.932 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.160+0000 7f350d014700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3508071280 0x7f3508105520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:25.932 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.160+0000 7f350d014700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3508071690 con 0x7f3508071280 2024-03-24T10:37:25.932 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.160+0000 7f3506d9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3508071280 0x7f3508105520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:25.932 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.161+0000 7f3506d9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3508071280 0x7f3508105520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49738/0 (socket says 172.21.15.94:49738) 2024-03-24T10:37:25.932 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.161+0000 7f3506d9d700 1 -- 172.21.15.94:0/4073484028 learned_addr learned my addr 172.21.15.94:0/4073484028 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:37:25.932 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.161+0000 7f3506d9d700 1 -- 172.21.15.94:0/4073484028 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3508105a60 con 0x7f3508071280 2024-03-24T10:37:25.932 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.161+0000 7f3506d9d700 1 --2- 172.21.15.94:0/4073484028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3508071280 0x7f3508105520 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f34f80098e0 tx=0x7f34f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9805015781249145 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:25.932 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.161+0000 7f3505d9b700 1 -- 172.21.15.94:0/4073484028 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f34f800b860 con 0x7f3508071280 2024-03-24T10:37:25.932 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.161+0000 7f3505d9b700 1 -- 172.21.15.94:0/4073484028 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f34f800be90 con 0x7f3508071280 2024-03-24T10:37:25.932 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.161+0000 7f3505d9b700 1 -- 172.21.15.94:0/4073484028 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f34f80103c0 con 0x7f3508071280 2024-03-24T10:37:25.932 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.162+0000 7f350d014700 1 -- 172.21.15.94:0/4073484028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3508071280 msgr2=0x7f3508105520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:25.932 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.162+0000 7f350d014700 1 --2- 172.21.15.94:0/4073484028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3508071280 0x7f3508105520 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f34f80098e0 tx=0x7f34f8009250 comp rx=0 tx=0).stop 2024-03-24T10:37:25.933 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.162+0000 7f350d014700 1 -- 172.21.15.94:0/4073484028 shutdown_connections 2024-03-24T10:37:25.933 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.162+0000 7f350d014700 1 --2- 172.21.15.94:0/4073484028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3508071280 0x7f3508105520 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:25.933 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.162+0000 7f350d014700 1 -- 172.21.15.94:0/4073484028 >> 172.21.15.94:0/4073484028 conn(0x7f35080fcfb0 msgr2=0x7f35080ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:37:25.933 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.162+0000 7f350d014700 1 -- 172.21.15.94:0/4073484028 shutdown_connections 2024-03-24T10:37:25.933 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.162+0000 7f350d014700 1 -- 172.21.15.94:0/4073484028 wait complete. 2024-03-24T10:37:25.933 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.162+0000 7f350d014700 1 Processor -- start 2024-03-24T10:37:25.933 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.162+0000 7f350d014700 1 -- start start 2024-03-24T10:37:25.933 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.162+0000 7f350d014700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3508071280 0x7f3508111770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:25.933 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.162+0000 7f350d014700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3508111cb0 con 0x7f3508071280 2024-03-24T10:37:25.933 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.163+0000 7f3506d9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3508071280 0x7f3508111770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:25.933 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.163+0000 7f3506d9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3508071280 0x7f3508111770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49754/0 (socket says 172.21.15.94:49754) 2024-03-24T10:37:25.933 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.163+0000 7f3506d9d700 1 -- 172.21.15.94:0/2408624558 learned_addr learned my addr 172.21.15.94:0/2408624558 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:37:25.933 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.163+0000 7f3506d9d700 1 -- 172.21.15.94:0/2408624558 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f80094f0 con 0x7f3508071280 2024-03-24T10:37:25.933 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.163+0000 7f3506d9d700 1 --2- 172.21.15.94:0/2408624558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3508071280 0x7f3508111770 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f34f80094c0 tx=0x7f34f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:25.934 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.163+0000 7f34ef7fe700 1 -- 172.21.15.94:0/2408624558 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f34f80118f0 con 0x7f3508071280 2024-03-24T10:37:25.934 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.163+0000 7f350d014700 1 -- 172.21.15.94:0/2408624558 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3508111eb0 con 0x7f3508071280 2024-03-24T10:37:25.934 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.163+0000 7f34ef7fe700 1 -- 172.21.15.94:0/2408624558 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f34f8010d60 con 0x7f3508071280 2024-03-24T10:37:25.934 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.163+0000 7f34ef7fe700 1 -- 172.21.15.94:0/2408624558 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f34f801a430 con 0x7f3508071280 2024-03-24T10:37:25.934 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.163+0000 7f350d014700 1 -- 172.21.15.94:0/2408624558 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3508112a20 con 0x7f3508071280 2024-03-24T10:37:25.934 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.164+0000 7f34ef7fe700 1 -- 172.21.15.94:0/2408624558 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f34f801a590 con 0x7f3508071280 2024-03-24T10:37:25.934 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.164+0000 7f34ef7fe700 1 --2- 172.21.15.94:0/2408624558 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f34f0036880 0x7f34f0038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:25.934 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.164+0000 7f34ef7fe700 1 -- 172.21.15.94:0/2408624558 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f34f804a7e0 con 0x7f3508071280 2024-03-24T10:37:25.934 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.164+0000 7f350d014700 1 -- 172.21.15.94:0/2408624558 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f350805f5f0 con 0x7f3508071280 2024-03-24T10:37:25.934 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.164+0000 7f350659c700 1 --2- 172.21.15.94:0/2408624558 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f34f0036880 0x7f34f0038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:25.934 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.165+0000 7f350659c700 1 --2- 172.21.15.94:0/2408624558 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f34f0036880 0x7f34f0038d30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f34fc009990 tx=0x7f34fc006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:25.934 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.169+0000 7f34ef7fe700 1 -- 172.21.15.94:0/2408624558 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f34f802d3f0 con 0x7f3508071280 2024-03-24T10:37:25.934 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.354+0000 7f350d014700 1 -- 172.21.15.94:0/2408624558 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f350804bed0 con 0x7f3508071280 2024-03-24T10:37:25.934 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.354+0000 7f34ef7fe700 1 -- 172.21.15.94:0/2408624558 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f34f8010ed0 con 0x7f3508071280 2024-03-24T10:37:25.935 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.356+0000 7f350d014700 1 -- 172.21.15.94:0/2408624558 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f34f0036880 msgr2=0x7f34f0038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:25.935 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.356+0000 7f350d014700 1 --2- 172.21.15.94:0/2408624558 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f34f0036880 0x7f34f0038d30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f34fc009990 tx=0x7f34fc006e40 comp rx=0 tx=0).stop 2024-03-24T10:37:25.935 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.356+0000 7f350d014700 1 -- 172.21.15.94:0/2408624558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3508071280 msgr2=0x7f3508111770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:25.935 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.356+0000 7f350d014700 1 --2- 172.21.15.94:0/2408624558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3508071280 0x7f3508111770 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f34f80094c0 tx=0x7f34f8009250 comp rx=0 tx=0).stop 2024-03-24T10:37:25.935 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.356+0000 7f350d014700 1 -- 172.21.15.94:0/2408624558 shutdown_connections 2024-03-24T10:37:25.935 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.356+0000 7f350d014700 1 --2- 172.21.15.94:0/2408624558 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f34f0036880 0x7f34f0038d30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:25.935 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.356+0000 7f350d014700 1 --2- 172.21.15.94:0/2408624558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3508071280 0x7f3508111770 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:25.935 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.356+0000 7f350d014700 1 -- 172.21.15.94:0/2408624558 >> 172.21.15.94:0/2408624558 conn(0x7f35080fcfb0 msgr2=0x7f35080fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:37:25.935 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.357+0000 7f350d014700 1 -- 172.21.15.94:0/2408624558 shutdown_connections 2024-03-24T10:37:25.935 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:25.357+0000 7f350d014700 1 -- 172.21.15.94:0/2408624558 wait complete. 2024-03-24T10:37:27.189 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.322+0000 7fa27b905700 1 Processor -- start 2024-03-24T10:37:27.190 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.322+0000 7fa27b905700 1 -- start start 2024-03-24T10:37:27.190 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.322+0000 7fa27b905700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa274076b20 0x7fa274107980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:27.190 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.322+0000 7fa27b905700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa274076f30 con 0x7fa274076b20 2024-03-24T10:37:27.190 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.322+0000 7fa2796a1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa274076b20 0x7fa274107980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:27.190 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.322+0000 7fa2796a1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa274076b20 0x7fa274107980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49768/0 (socket says 172.21.15.94:49768) 2024-03-24T10:37:27.190 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.322+0000 7fa2796a1700 1 -- 172.21.15.94:0/3683973862 learned_addr learned my addr 172.21.15.94:0/3683973862 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:37:27.190 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.322+0000 7fa2796a1700 1 -- 172.21.15.94:0/3683973862 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa274103370 con 0x7fa274076b20 2024-03-24T10:37:27.190 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.323+0000 7fa2796a1700 1 --2- 172.21.15.94:0/3683973862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa274076b20 0x7fa274107980 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa2700098e0 tx=0x7fa270009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9005311a49e0ac1 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:27.190 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.323+0000 7fa26bfff700 1 -- 172.21.15.94:0/3683973862 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa27000b860 con 0x7fa274076b20 2024-03-24T10:37:27.190 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.323+0000 7fa26bfff700 1 -- 172.21.15.94:0/3683973862 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa27000be90 con 0x7fa274076b20 2024-03-24T10:37:27.190 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.323+0000 7fa26bfff700 1 -- 172.21.15.94:0/3683973862 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa2700103c0 con 0x7fa274076b20 2024-03-24T10:37:27.190 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.323+0000 7fa27b905700 1 -- 172.21.15.94:0/3683973862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa274076b20 msgr2=0x7fa274107980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:27.191 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.323+0000 7fa27b905700 1 --2- 172.21.15.94:0/3683973862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa274076b20 0x7fa274107980 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa2700098e0 tx=0x7fa270009250 comp rx=0 tx=0).stop 2024-03-24T10:37:27.191 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.323+0000 7fa27b905700 1 -- 172.21.15.94:0/3683973862 shutdown_connections 2024-03-24T10:37:27.191 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.323+0000 7fa27b905700 1 --2- 172.21.15.94:0/3683973862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa274076b20 0x7fa274107980 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:27.191 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.323+0000 7fa27b905700 1 -- 172.21.15.94:0/3683973862 >> 172.21.15.94:0/3683973862 conn(0x7fa2740fcf90 msgr2=0x7fa2740ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:37:27.191 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.323+0000 7fa27b905700 1 -- 172.21.15.94:0/3683973862 shutdown_connections 2024-03-24T10:37:27.191 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.323+0000 7fa27b905700 1 -- 172.21.15.94:0/3683973862 wait complete. 2024-03-24T10:37:27.191 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.324+0000 7fa27b905700 1 Processor -- start 2024-03-24T10:37:27.191 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.324+0000 7fa27b905700 1 -- start start 2024-03-24T10:37:27.191 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.324+0000 7fa27b905700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa274076b20 0x7fa274113900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:27.191 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.324+0000 7fa27b905700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa274113e40 con 0x7fa274076b20 2024-03-24T10:37:27.191 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.324+0000 7fa2796a1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa274076b20 0x7fa274113900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:27.191 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.324+0000 7fa2796a1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa274076b20 0x7fa274113900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49784/0 (socket says 172.21.15.94:49784) 2024-03-24T10:37:27.191 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.324+0000 7fa2796a1700 1 -- 172.21.15.94:0/4149302593 learned_addr learned my addr 172.21.15.94:0/4149302593 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:37:27.192 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.324+0000 7fa2796a1700 1 -- 172.21.15.94:0/4149302593 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2700094f0 con 0x7fa274076b20 2024-03-24T10:37:27.192 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.325+0000 7fa2796a1700 1 --2- 172.21.15.94:0/4149302593 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa274076b20 0x7fa274113900 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa2700094c0 tx=0x7fa270009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:27.192 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.325+0000 7fa269ffb700 1 -- 172.21.15.94:0/4149302593 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa270010890 con 0x7fa274076b20 2024-03-24T10:37:27.192 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.325+0000 7fa27b905700 1 -- 172.21.15.94:0/4149302593 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2741140a0 con 0x7fa274076b20 2024-03-24T10:37:27.192 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.325+0000 7fa269ffb700 1 -- 172.21.15.94:0/4149302593 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa270010ec0 con 0x7fa274076b20 2024-03-24T10:37:27.192 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.325+0000 7fa269ffb700 1 -- 172.21.15.94:0/4149302593 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa27001aea0 con 0x7fa274076b20 2024-03-24T10:37:27.192 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.325+0000 7fa27b905700 1 -- 172.21.15.94:0/4149302593 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa274114bb0 con 0x7fa274076b20 2024-03-24T10:37:27.192 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.326+0000 7fa269ffb700 1 -- 172.21.15.94:0/4149302593 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fa27002d3f0 con 0x7fa274076b20 2024-03-24T10:37:27.192 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.326+0000 7fa269ffb700 1 --2- 172.21.15.94:0/4149302593 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fa260036850 0x7fa260038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:27.193 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.326+0000 7fa269ffb700 1 -- 172.21.15.94:0/4149302593 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fa27004a3d0 con 0x7fa274076b20 2024-03-24T10:37:27.193 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.326+0000 7fa27b905700 1 -- 172.21.15.94:0/4149302593 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa27405f5f0 con 0x7fa274076b20 2024-03-24T10:37:27.193 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.326+0000 7fa278ea0700 1 --2- 172.21.15.94:0/4149302593 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fa260036850 0x7fa260038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:27.193 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.326+0000 7fa278ea0700 1 --2- 172.21.15.94:0/4149302593 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fa260036850 0x7fa260038d00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa264009990 tx=0x7fa264006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:27.193 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.331+0000 7fa269ffb700 1 -- 172.21.15.94:0/4149302593 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fa27002b8e0 con 0x7fa274076b20 2024-03-24T10:37:27.194 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.622+0000 7fa27b905700 1 -- 172.21.15.94:0/4149302593 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7fa27404bed0 con 0x7fa274076b20 2024-03-24T10:37:27.194 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.624+0000 7fa269ffb700 1 -- 172.21.15.94:0/4149302593 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v33)=0 set mgr/dashboard/cluster/status v33) v1 ==== 153+0+0 (secure 0 0 0) 0x7fa27002b2f0 con 0x7fa274076b20 2024-03-24T10:37:27.194 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.626+0000 7fa27b905700 1 -- 172.21.15.94:0/4149302593 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fa260036850 msgr2=0x7fa260038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:27.194 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.626+0000 7fa27b905700 1 --2- 172.21.15.94:0/4149302593 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fa260036850 0x7fa260038d00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa264009990 tx=0x7fa264006e40 comp rx=0 tx=0).stop 2024-03-24T10:37:27.194 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.626+0000 7fa27b905700 1 -- 172.21.15.94:0/4149302593 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa274076b20 msgr2=0x7fa274113900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:27.194 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.626+0000 7fa27b905700 1 --2- 172.21.15.94:0/4149302593 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa274076b20 0x7fa274113900 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa2700094c0 tx=0x7fa270009250 comp rx=0 tx=0).stop 2024-03-24T10:37:27.194 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.626+0000 7fa27b905700 1 -- 172.21.15.94:0/4149302593 shutdown_connections 2024-03-24T10:37:27.194 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.626+0000 7fa27b905700 1 --2- 172.21.15.94:0/4149302593 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fa260036850 0x7fa260038d00 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:27.194 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.626+0000 7fa27b905700 1 --2- 172.21.15.94:0/4149302593 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa274076b20 0x7fa274113900 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:27.194 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.626+0000 7fa27b905700 1 -- 172.21.15.94:0/4149302593 >> 172.21.15.94:0/4149302593 conn(0x7fa2740fcf90 msgr2=0x7fa2740fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:37:27.194 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.627+0000 7fa27b905700 1 -- 172.21.15.94:0/4149302593 shutdown_connections 2024-03-24T10:37:27.194 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-03-24T10:37:26.627+0000 7fa27b905700 1 -- 172.21.15.94:0/4149302593 wait complete. 2024-03-24T10:37:27.194 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-24T10:37:27.194 INFO:teuthology.orchestra.run.smithi094.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-24T10:37:27.195 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:37:27.195 INFO:teuthology.orchestra.run.smithi094.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-24T10:37:27.195 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:37:27.195 INFO:teuthology.orchestra.run.smithi094.stdout:Or, if you are only running a single cluster on this host: 2024-03-24T10:37:27.195 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:37:27.195 INFO:teuthology.orchestra.run.smithi094.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-24T10:37:27.195 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:37:27.195 INFO:teuthology.orchestra.run.smithi094.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-24T10:37:27.195 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:37:27.195 INFO:teuthology.orchestra.run.smithi094.stdout: ceph telemetry on 2024-03-24T10:37:27.195 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:37:27.195 INFO:teuthology.orchestra.run.smithi094.stdout:For more information see: 2024-03-24T10:37:27.195 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:37:27.196 INFO:teuthology.orchestra.run.smithi094.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-03-24T10:37:27.196 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:37:27.196 INFO:teuthology.orchestra.run.smithi094.stdout:Bootstrap complete. 2024-03-24T10:37:27.223 INFO:tasks.cephadm:Fetching config... 2024-03-24T10:37:27.224 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-03-24T10:37:27.224 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-24T10:37:27.279 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-24T10:37:27.279 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-03-24T10:37:27.279 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-24T10:37:27.335 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-24T10:37:27.335 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-03-24T10:37:27.335 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/keyring of=/dev/stdout 2024-03-24T10:37:27.402 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-24T10:37:27.402 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-03-24T10:37:27.402 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-24T10:37:27.458 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-24T10:37:27.459 DEBUG:teuthology.orchestra.run.smithi094:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9yYa357KmpLvSmOZJWVx3l32qF6ao/UG82f7zdsM18tnK0HTqk0IuTpSGrwIiMvUp/aDYgTwETqH/aC99p7eMH+QqXrVAIXWIJVh6fzKizbSCBDoU+owWoNp8CraV0JO5ms0ggk/jO4zufDNiRhAaeSS/DsLs6zPKKt10UB7GEG3Bh2VxHnQajVU2HtCEVeFWlHQmPmjsS7e9ZusMbmISbgW11+b8RFxix51/GdazcO8umJACOBhG2eHi0rIR/Ois13tNDDVS7sS8pd+Y0MLYUiXqaE0ImvqZ8W7Cwzw+B2VJwbJmyZ33uzSOeSG/IStzlQ3vAwtpEir7zeHTUoubPT3+ss9O0eDVm5Q6asfjia3jI0F44aJaesiPXxOx1jYXTYv6CSg5NDd5jMEbQnQ5MzIqwt20P646HKfeyU0N7/gM5qL1MxqQe5GTkJrge98LJYkl92aOWS850zebAe/m0iwih4uHcbC4OGSi64Wq4HWlROyRCh4qRSKNKeq4ohE= ceph-386110d4-e9ca-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-24T10:37:27.538 INFO:teuthology.orchestra.run.smithi094.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9yYa357KmpLvSmOZJWVx3l32qF6ao/UG82f7zdsM18tnK0HTqk0IuTpSGrwIiMvUp/aDYgTwETqH/aC99p7eMH+QqXrVAIXWIJVh6fzKizbSCBDoU+owWoNp8CraV0JO5ms0ggk/jO4zufDNiRhAaeSS/DsLs6zPKKt10UB7GEG3Bh2VxHnQajVU2HtCEVeFWlHQmPmjsS7e9ZusMbmISbgW11+b8RFxix51/GdazcO8umJACOBhG2eHi0rIR/Ois13tNDDVS7sS8pd+Y0MLYUiXqaE0ImvqZ8W7Cwzw+B2VJwbJmyZ33uzSOeSG/IStzlQ3vAwtpEir7zeHTUoubPT3+ss9O0eDVm5Q6asfjia3jI0F44aJaesiPXxOx1jYXTYv6CSg5NDd5jMEbQnQ5MzIqwt20P646HKfeyU0N7/gM5qL1MxqQe5GTkJrge98LJYkl92aOWS850zebAe/m0iwih4uHcbC4OGSi64Wq4HWlROyRCh4qRSKNKeq4ohE= ceph-386110d4-e9ca-11ee-95cd-87774f69a715 2024-03-24T10:37:27.552 DEBUG:teuthology.orchestra.run.smithi096:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9yYa357KmpLvSmOZJWVx3l32qF6ao/UG82f7zdsM18tnK0HTqk0IuTpSGrwIiMvUp/aDYgTwETqH/aC99p7eMH+QqXrVAIXWIJVh6fzKizbSCBDoU+owWoNp8CraV0JO5ms0ggk/jO4zufDNiRhAaeSS/DsLs6zPKKt10UB7GEG3Bh2VxHnQajVU2HtCEVeFWlHQmPmjsS7e9ZusMbmISbgW11+b8RFxix51/GdazcO8umJACOBhG2eHi0rIR/Ois13tNDDVS7sS8pd+Y0MLYUiXqaE0ImvqZ8W7Cwzw+B2VJwbJmyZ33uzSOeSG/IStzlQ3vAwtpEir7zeHTUoubPT3+ss9O0eDVm5Q6asfjia3jI0F44aJaesiPXxOx1jYXTYv6CSg5NDd5jMEbQnQ5MzIqwt20P646HKfeyU0N7/gM5qL1MxqQe5GTkJrge98LJYkl92aOWS850zebAe/m0iwih4uHcbC4OGSi64Wq4HWlROyRCh4qRSKNKeq4ohE= ceph-386110d4-e9ca-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-24T10:37:27.598 INFO:teuthology.orchestra.run.smithi096.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9yYa357KmpLvSmOZJWVx3l32qF6ao/UG82f7zdsM18tnK0HTqk0IuTpSGrwIiMvUp/aDYgTwETqH/aC99p7eMH+QqXrVAIXWIJVh6fzKizbSCBDoU+owWoNp8CraV0JO5ms0ggk/jO4zufDNiRhAaeSS/DsLs6zPKKt10UB7GEG3Bh2VxHnQajVU2HtCEVeFWlHQmPmjsS7e9ZusMbmISbgW11+b8RFxix51/GdazcO8umJACOBhG2eHi0rIR/Ois13tNDDVS7sS8pd+Y0MLYUiXqaE0ImvqZ8W7Cwzw+B2VJwbJmyZ33uzSOeSG/IStzlQ3vAwtpEir7zeHTUoubPT3+ss9O0eDVm5Q6asfjia3jI0F44aJaesiPXxOx1jYXTYv6CSg5NDd5jMEbQnQ5MzIqwt20P646HKfeyU0N7/gM5qL1MxqQe5GTkJrge98LJYkl92aOWS850zebAe/m0iwih4uHcbC4OGSi64Wq4HWlROyRCh4qRSKNKeq4ohE= ceph-386110d4-e9ca-11ee-95cd-87774f69a715 2024-03-24T10:37:27.616 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-24T10:37:27.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:27 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/4149302593' entity='client.admin' 2024-03-24T10:37:27.869 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:37:30.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:30.154+0000 7f9d3e2fa700 1 -- 172.21.15.94:0/1720051885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d380fd050 msgr2=0x7f9d380fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:30.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:30.154+0000 7f9d3e2fa700 1 --2- 172.21.15.94:0/1720051885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d380fd050 0x7f9d380fd460 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9d280048c0 tx=0x7f9d28009e30 comp rx=0 tx=0).stop 2024-03-24T10:37:30.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:30.155+0000 7f9d3e2fa700 1 -- 172.21.15.94:0/1720051885 shutdown_connections 2024-03-24T10:37:30.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:30.155+0000 7f9d3e2fa700 1 --2- 172.21.15.94:0/1720051885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d380fd050 0x7f9d380fd460 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:30.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:30.155+0000 7f9d3e2fa700 1 -- 172.21.15.94:0/1720051885 >> 172.21.15.94:0/1720051885 conn(0x7f9d380f70d0 msgr2=0x7f9d380f9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:37:30.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:30.155+0000 7f9d3e2fa700 1 -- 172.21.15.94:0/1720051885 shutdown_connections 2024-03-24T10:37:30.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:30.155+0000 7f9d3e2fa700 1 -- 172.21.15.94:0/1720051885 wait complete. 2024-03-24T10:37:30.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:30.155+0000 7f9d3e2fa700 1 Processor -- start 2024-03-24T10:37:30.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:30.155+0000 7f9d3e2fa700 1 -- start start 2024-03-24T10:37:30.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:30.155+0000 7f9d3e2fa700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d380fd050 0x7f9d3810d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:30.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:30.155+0000 7f9d3e2fa700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3810d620 con 0x7f9d380fd050 2024-03-24T10:37:30.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:30.156+0000 7f9d37fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d380fd050 0x7f9d3810d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:30.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:30.156+0000 7f9d37fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d380fd050 0x7f9d3810d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35848/0 (socket says 172.21.15.94:35848) 2024-03-24T10:37:30.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:30.156+0000 7f9d37fff700 1 -- 172.21.15.94:0/813343282 learned_addr learned my addr 172.21.15.94:0/813343282 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:37:30.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:30.156+0000 7f9d37fff700 1 -- 172.21.15.94:0/813343282 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d28005040 con 0x7f9d380fd050 2024-03-24T10:37:30.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:30.156+0000 7f9d37fff700 1 --2- 172.21.15.94:0/813343282 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d380fd050 0x7f9d3810d0e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9d280096b0 tx=0x7f9d28009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:30.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:30.156+0000 7f9d34ff9700 1 -- 172.21.15.94:0/813343282 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9d2800bed0 con 0x7f9d380fd050 2024-03-24T10:37:30.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:30.156+0000 7f9d34ff9700 1 -- 172.21.15.94:0/813343282 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9d2800b5c0 con 0x7f9d380fd050 2024-03-24T10:37:30.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:30.156+0000 7f9d3e2fa700 1 -- 172.21.15.94:0/813343282 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d3810d820 con 0x7f9d380fd050 2024-03-24T10:37:30.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:30.156+0000 7f9d34ff9700 1 -- 172.21.15.94:0/813343282 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9d2801c5b0 con 0x7f9d380fd050 2024-03-24T10:37:30.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:30.156+0000 7f9d3e2fa700 1 -- 172.21.15.94:0/813343282 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d3810e390 con 0x7f9d380fd050 2024-03-24T10:37:30.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:30.157+0000 7f9d34ff9700 1 -- 172.21.15.94:0/813343282 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f9d28013420 con 0x7f9d380fd050 2024-03-24T10:37:30.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:30.157+0000 7f9d3e2fa700 1 -- 172.21.15.94:0/813343282 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d38101e70 con 0x7f9d380fd050 2024-03-24T10:37:30.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:30.157+0000 7f9d34ff9700 1 --2- 172.21.15.94:0/813343282 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f9d20036880 0x7f9d20038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:30.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:30.157+0000 7f9d34ff9700 1 -- 172.21.15.94:0/813343282 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f9d28049a50 con 0x7f9d380fd050 2024-03-24T10:37:30.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:30.157+0000 7f9d377fe700 1 --2- 172.21.15.94:0/813343282 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f9d20036880 0x7f9d20038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:30.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:30.158+0000 7f9d377fe700 1 --2- 172.21.15.94:0/813343282 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f9d20036880 0x7f9d20038d30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c009990 tx=0x7f9d2c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:30.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:30.162+0000 7f9d34ff9700 1 -- 172.21.15.94:0/813343282 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f9d28013a80 con 0x7f9d380fd050 2024-03-24T10:37:30.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:30.352+0000 7f9d3e2fa700 1 -- 172.21.15.94:0/813343282 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f9d381017e0 con 0x7f9d380fd050 2024-03-24T10:37:30.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:30.355+0000 7f9d34ff9700 1 -- 172.21.15.94:0/813343282 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f9d28011370 con 0x7f9d380fd050 2024-03-24T10:37:30.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:30.357+0000 7f9d3e2fa700 1 -- 172.21.15.94:0/813343282 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f9d20036880 msgr2=0x7f9d20038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:30.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:30.357+0000 7f9d3e2fa700 1 --2- 172.21.15.94:0/813343282 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f9d20036880 0x7f9d20038d30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c009990 tx=0x7f9d2c006e40 comp rx=0 tx=0).stop 2024-03-24T10:37:30.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:30.357+0000 7f9d3e2fa700 1 -- 172.21.15.94:0/813343282 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d380fd050 msgr2=0x7f9d3810d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:30.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:30.357+0000 7f9d3e2fa700 1 --2- 172.21.15.94:0/813343282 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d380fd050 0x7f9d3810d0e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9d280096b0 tx=0x7f9d28009e30 comp rx=0 tx=0).stop 2024-03-24T10:37:30.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:30.357+0000 7f9d3e2fa700 1 -- 172.21.15.94:0/813343282 shutdown_connections 2024-03-24T10:37:30.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:30.357+0000 7f9d3e2fa700 1 --2- 172.21.15.94:0/813343282 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f9d20036880 0x7f9d20038d30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:30.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:30.357+0000 7f9d3e2fa700 1 --2- 172.21.15.94:0/813343282 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d380fd050 0x7f9d3810d0e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:30.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:30.357+0000 7f9d3e2fa700 1 -- 172.21.15.94:0/813343282 >> 172.21.15.94:0/813343282 conn(0x7f9d380f70d0 msgr2=0x7f9d38065070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:37:30.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:30.357+0000 7f9d3e2fa700 1 -- 172.21.15.94:0/813343282 shutdown_connections 2024-03-24T10:37:30.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:30.357+0000 7f9d3e2fa700 1 -- 172.21.15.94:0/813343282 wait complete. 2024-03-24T10:37:31.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:31 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/813343282' entity='client.admin' 2024-03-24T10:37:31.649 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-24T10:37:31.649 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-24T10:37:32.429 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:37:32.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:32 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' 2024-03-24T10:37:32.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:32 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' 2024-03-24T10:37:32.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:32 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "who": "osd/host:smithi094", "name": "osd_memory_target"}]: dispatch 2024-03-24T10:37:32.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:32 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' 2024-03-24T10:37:32.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:32 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi094", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-24T10:37:32.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:32 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi094", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-03-24T10:37:32.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:32 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:37:32.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:32 smithi094 ceph-mon[29446]: Deploying daemon crash.smithi094 on smithi094 2024-03-24T10:37:33.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:33.699+0000 7fca2c896700 1 -- 172.21.15.94:0/2292715965 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca280fb960 msgr2=0x7fca280fbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:33.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:33.699+0000 7fca2c896700 1 --2- 172.21.15.94:0/2292715965 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca280fb960 0x7fca280fbd70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fca180048c0 tx=0x7fca18009e30 comp rx=0 tx=0).stop 2024-03-24T10:37:33.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:33.699+0000 7fca2c896700 1 -- 172.21.15.94:0/2292715965 shutdown_connections 2024-03-24T10:37:33.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:33.699+0000 7fca2c896700 1 --2- 172.21.15.94:0/2292715965 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca280fb960 0x7fca280fbd70 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:33.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:33.699+0000 7fca2c896700 1 -- 172.21.15.94:0/2292715965 >> 172.21.15.94:0/2292715965 conn(0x7fca280f70c0 msgr2=0x7fca280f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:37:33.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:33.699+0000 7fca2c896700 1 -- 172.21.15.94:0/2292715965 shutdown_connections 2024-03-24T10:37:33.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:33.699+0000 7fca2c896700 1 -- 172.21.15.94:0/2292715965 wait complete. 2024-03-24T10:37:33.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:33.699+0000 7fca2c896700 1 Processor -- start 2024-03-24T10:37:33.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:33.699+0000 7fca2c896700 1 -- start start 2024-03-24T10:37:33.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:33.699+0000 7fca2c896700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca280fb960 0x7fca28066860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:33.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:33.699+0000 7fca2c896700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2810fde0 con 0x7fca280fb960 2024-03-24T10:37:33.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:33.699+0000 7fca2659c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca280fb960 0x7fca28066860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:33.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:33.699+0000 7fca2659c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca280fb960 0x7fca28066860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35866/0 (socket says 172.21.15.94:35866) 2024-03-24T10:37:33.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:33.699+0000 7fca2659c700 1 -- 172.21.15.94:0/2461626733 learned_addr learned my addr 172.21.15.94:0/2461626733 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:37:33.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:33.700+0000 7fca2659c700 1 -- 172.21.15.94:0/2461626733 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca18005040 con 0x7fca280fb960 2024-03-24T10:37:33.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:33.700+0000 7fca2659c700 1 --2- 172.21.15.94:0/2461626733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca280fb960 0x7fca28066860 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fca180096b0 tx=0x7fca18009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:33.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:33.700+0000 7fca16ffd700 1 -- 172.21.15.94:0/2461626733 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fca1800bed0 con 0x7fca280fb960 2024-03-24T10:37:33.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:33.700+0000 7fca16ffd700 1 -- 172.21.15.94:0/2461626733 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fca1800b5c0 con 0x7fca280fb960 2024-03-24T10:37:33.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:33.700+0000 7fca2c896700 1 -- 172.21.15.94:0/2461626733 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca2810ff20 con 0x7fca280fb960 2024-03-24T10:37:33.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:33.700+0000 7fca16ffd700 1 -- 172.21.15.94:0/2461626733 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fca1801c5b0 con 0x7fca280fb960 2024-03-24T10:37:33.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:33.700+0000 7fca2c896700 1 -- 172.21.15.94:0/2461626733 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca28110410 con 0x7fca280fb960 2024-03-24T10:37:33.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:33.701+0000 7fca16ffd700 1 -- 172.21.15.94:0/2461626733 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fca18013420 con 0x7fca280fb960 2024-03-24T10:37:33.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:33.701+0000 7fca16ffd700 1 --2- 172.21.15.94:0/2461626733 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fca100368d0 0x7fca10038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:33.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:33.701+0000 7fca2c896700 1 -- 172.21.15.94:0/2461626733 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca28105920 con 0x7fca280fb960 2024-03-24T10:37:33.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:33.701+0000 7fca16ffd700 1 -- 172.21.15.94:0/2461626733 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fca180499d0 con 0x7fca280fb960 2024-03-24T10:37:33.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:33.701+0000 7fca25d9b700 1 --2- 172.21.15.94:0/2461626733 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fca100368d0 0x7fca10038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:33.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:33.701+0000 7fca25d9b700 1 --2- 172.21.15.94:0/2461626733 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fca100368d0 0x7fca10038d80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fca1c009990 tx=0x7fca1c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:33.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:33.706+0000 7fca16ffd700 1 -- 172.21.15.94:0/2461626733 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fca18013a80 con 0x7fca280fb960 2024-03-24T10:37:33.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:33.898+0000 7fca2c896700 1 -- 172.21.15.94:0/2461626733 --> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fca28101850 con 0x7fca100368d0 2024-03-24T10:37:33.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:33.902+0000 7fca16ffd700 1 -- 172.21.15.94:0/2461626733 <== mgr.14162 v2:172.21.15.94:6800/2406899470 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fca1c009590 con 0x7fca100368d0 2024-03-24T10:37:33.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:33.904+0000 7fca2c896700 1 -- 172.21.15.94:0/2461626733 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fca100368d0 msgr2=0x7fca10038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:33.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:33.904+0000 7fca2c896700 1 --2- 172.21.15.94:0/2461626733 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fca100368d0 0x7fca10038d80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fca1c009990 tx=0x7fca1c006e40 comp rx=0 tx=0).stop 2024-03-24T10:37:33.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:33.904+0000 7fca2c896700 1 -- 172.21.15.94:0/2461626733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca280fb960 msgr2=0x7fca28066860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:33.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:33.904+0000 7fca2c896700 1 --2- 172.21.15.94:0/2461626733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca280fb960 0x7fca28066860 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fca180096b0 tx=0x7fca18009e30 comp rx=0 tx=0).stop 2024-03-24T10:37:33.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:33.904+0000 7fca2c896700 1 -- 172.21.15.94:0/2461626733 shutdown_connections 2024-03-24T10:37:33.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:33.904+0000 7fca2c896700 1 --2- 172.21.15.94:0/2461626733 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fca100368d0 0x7fca10038d80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:33.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:33.904+0000 7fca2c896700 1 --2- 172.21.15.94:0/2461626733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca280fb960 0x7fca28066860 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:33.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:33.904+0000 7fca2c896700 1 -- 172.21.15.94:0/2461626733 >> 172.21.15.94:0/2461626733 conn(0x7fca280f70c0 msgr2=0x7fca28102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:37:33.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:33.904+0000 7fca2c896700 1 -- 172.21.15.94:0/2461626733 shutdown_connections 2024-03-24T10:37:33.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:33.904+0000 7fca2c896700 1 -- 172.21.15.94:0/2461626733 wait complete. 2024-03-24T10:37:34.872 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi096 2024-03-24T10:37:34.873 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-24T10:37:34.873 DEBUG:teuthology.orchestra.run.smithi096:> dd of=/etc/ceph/ceph.conf 2024-03-24T10:37:34.891 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-24T10:37:34.891 DEBUG:teuthology.orchestra.run.smithi096:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-24T10:37:34.949 INFO:tasks.cephadm:Adding host smithi096 to orchestrator... 2024-03-24T10:37:34.949 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph orch host add smithi096 2024-03-24T10:37:35.099 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:34 smithi094 ceph-mon[29446]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:37:35.099 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:34 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' 2024-03-24T10:37:35.449 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:37:36.663 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:36 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' 2024-03-24T10:37:36.663 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:36 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' 2024-03-24T10:37:36.663 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:36 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' 2024-03-24T10:37:36.663 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:36 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' 2024-03-24T10:37:36.663 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:36 smithi094 ceph-mon[29446]: Deploying daemon node-exporter.smithi094 on smithi094 2024-03-24T10:37:36.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:36.897+0000 7f71709fe700 1 -- 172.21.15.94:0/2464353189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f716c0fb4a0 msgr2=0x7f716c0fb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:36.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:36.897+0000 7f71709fe700 1 --2- 172.21.15.94:0/2464353189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f716c0fb4a0 0x7f716c0fb8b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f715c0048c0 tx=0x7f715c009e30 comp rx=0 tx=0).stop 2024-03-24T10:37:36.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:36.897+0000 7f71709fe700 1 -- 172.21.15.94:0/2464353189 shutdown_connections 2024-03-24T10:37:36.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:36.897+0000 7f71709fe700 1 --2- 172.21.15.94:0/2464353189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f716c0fb4a0 0x7f716c0fb8b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:36.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:36.897+0000 7f71709fe700 1 -- 172.21.15.94:0/2464353189 >> 172.21.15.94:0/2464353189 conn(0x7f716c0f70d0 msgr2=0x7f716c0f9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:37:36.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:36.897+0000 7f71709fe700 1 -- 172.21.15.94:0/2464353189 shutdown_connections 2024-03-24T10:37:36.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:36.897+0000 7f71709fe700 1 -- 172.21.15.94:0/2464353189 wait complete. 2024-03-24T10:37:36.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:36.898+0000 7f71709fe700 1 Processor -- start 2024-03-24T10:37:36.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:36.898+0000 7f71709fe700 1 -- start start 2024-03-24T10:37:36.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:36.898+0000 7f71709fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f716c0fb4a0 0x7f716c10f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:36.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:36.898+0000 7f71709fe700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716c10f800 con 0x7f716c0fb4a0 2024-03-24T10:37:36.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:36.898+0000 7f716a59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f716c0fb4a0 0x7f716c10f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:36.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:36.898+0000 7f716a59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f716c0fb4a0 0x7f716c10f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35892/0 (socket says 172.21.15.94:35892) 2024-03-24T10:37:36.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:36.898+0000 7f716a59c700 1 -- 172.21.15.94:0/187830428 learned_addr learned my addr 172.21.15.94:0/187830428 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:37:36.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:36.899+0000 7f716a59c700 1 -- 172.21.15.94:0/187830428 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f715c005040 con 0x7f716c0fb4a0 2024-03-24T10:37:36.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:36.899+0000 7f716a59c700 1 --2- 172.21.15.94:0/187830428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f716c0fb4a0 0x7f716c10f2c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f715c009680 tx=0x7f715c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:36.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:36.899+0000 7f715affd700 1 -- 172.21.15.94:0/187830428 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f715c00bed0 con 0x7f716c0fb4a0 2024-03-24T10:37:36.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:36.899+0000 7f71709fe700 1 -- 172.21.15.94:0/187830428 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f716c10fa00 con 0x7f716c0fb4a0 2024-03-24T10:37:36.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:36.899+0000 7f715affd700 1 -- 172.21.15.94:0/187830428 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f715c00b5c0 con 0x7f716c0fb4a0 2024-03-24T10:37:36.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:36.899+0000 7f715affd700 1 -- 172.21.15.94:0/187830428 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f715c01c5b0 con 0x7f716c0fb4a0 2024-03-24T10:37:36.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:36.899+0000 7f71709fe700 1 -- 172.21.15.94:0/187830428 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f716c110570 con 0x7f716c0fb4a0 2024-03-24T10:37:36.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:36.900+0000 7f715affd700 1 -- 172.21.15.94:0/187830428 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f715c01c710 con 0x7f716c0fb4a0 2024-03-24T10:37:36.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:36.900+0000 7f715affd700 1 --2- 172.21.15.94:0/187830428 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f7154036880 0x7f7154038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:36.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:36.900+0000 7f715affd700 1 -- 172.21.15.94:0/187830428 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f715c049900 con 0x7f716c0fb4a0 2024-03-24T10:37:36.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:36.900+0000 7f7169d9b700 1 --2- 172.21.15.94:0/187830428 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f7154036880 0x7f7154038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:36.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:36.900+0000 7f71709fe700 1 -- 172.21.15.94:0/187830428 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f716c105170 con 0x7f716c0fb4a0 2024-03-24T10:37:36.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:36.901+0000 7f7169d9b700 1 --2- 172.21.15.94:0/187830428 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f7154036880 0x7f7154038d30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7160009990 tx=0x7f7160006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:36.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:36.905+0000 7f715affd700 1 -- 172.21.15.94:0/187830428 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f715c025400 con 0x7f716c0fb4a0 2024-03-24T10:37:37.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:37.111+0000 7f71709fe700 1 -- 172.21.15.94:0/187830428 --> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi096", "target": ["mon-mgr", ""]}) v1 -- 0x7f716c04bed0 con 0x7f7154036880 2024-03-24T10:37:38.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:37 smithi094 ceph-mon[29446]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi096", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:37:39.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:39.243+0000 7f715affd700 1 -- 172.21.15.94:0/187830428 <== mgr.14162 v2:172.21.15.94:6800/2406899470 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f716c04bed0 con 0x7f7154036880 2024-03-24T10:37:39.243 INFO:teuthology.orchestra.run.smithi094.stdout:Added host 'smithi096' with addr '172.21.15.96' 2024-03-24T10:37:39.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:39.245+0000 7f71709fe700 1 -- 172.21.15.94:0/187830428 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f7154036880 msgr2=0x7f7154038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:39.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:39.245+0000 7f71709fe700 1 --2- 172.21.15.94:0/187830428 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f7154036880 0x7f7154038d30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7160009990 tx=0x7f7160006e40 comp rx=0 tx=0).stop 2024-03-24T10:37:39.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:39.245+0000 7f71709fe700 1 -- 172.21.15.94:0/187830428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f716c0fb4a0 msgr2=0x7f716c10f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:39.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:39.245+0000 7f71709fe700 1 --2- 172.21.15.94:0/187830428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f716c0fb4a0 0x7f716c10f2c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f715c009680 tx=0x7f715c009e30 comp rx=0 tx=0).stop 2024-03-24T10:37:39.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:39.245+0000 7f71709fe700 1 -- 172.21.15.94:0/187830428 shutdown_connections 2024-03-24T10:37:39.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:39.245+0000 7f71709fe700 1 --2- 172.21.15.94:0/187830428 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f7154036880 0x7f7154038d30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:39.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:39.245+0000 7f71709fe700 1 --2- 172.21.15.94:0/187830428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f716c0fb4a0 0x7f716c10f2c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:39.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:39.245+0000 7f71709fe700 1 -- 172.21.15.94:0/187830428 >> 172.21.15.94:0/187830428 conn(0x7f716c0f70d0 msgr2=0x7f716c105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:37:39.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:39.245+0000 7f71709fe700 1 -- 172.21.15.94:0/187830428 shutdown_connections 2024-03-24T10:37:39.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:39.245+0000 7f71709fe700 1 -- 172.21.15.94:0/187830428 wait complete. 2024-03-24T10:37:39.491 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:39 smithi094 ceph-mon[29446]: Deploying cephadm binary to smithi096 2024-03-24T10:37:39.491 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:39 smithi094 ceph-mon[29446]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:37:39.491 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:39 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' 2024-03-24T10:37:40.323 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph orch host ls --format=json 2024-03-24T10:37:40.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:40 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' 2024-03-24T10:37:40.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:40 smithi094 ceph-mon[29446]: Added host smithi096 2024-03-24T10:37:40.766 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:37:41.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:41 smithi094 ceph-mon[29446]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:37:41.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:41 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' 2024-03-24T10:37:41.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:41 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' 2024-03-24T10:37:41.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:41 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' 2024-03-24T10:37:41.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:41 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' 2024-03-24T10:37:42.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:42 smithi094 ceph-mon[29446]: Deploying daemon alertmanager.smithi094 on smithi094 2024-03-24T10:37:42.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:42.380+0000 7fdd93666700 1 -- 172.21.15.94:0/1466019727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd8c0fd6c0 msgr2=0x7fdd8c0ffaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:42.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:42.380+0000 7fdd93666700 1 --2- 172.21.15.94:0/1466019727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd8c0fd6c0 0x7fdd8c0ffaa0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdd880048c0 tx=0x7fdd88009e30 comp rx=0 tx=0).stop 2024-03-24T10:37:42.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:42.380+0000 7fdd93666700 1 -- 172.21.15.94:0/1466019727 shutdown_connections 2024-03-24T10:37:42.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:42.380+0000 7fdd93666700 1 --2- 172.21.15.94:0/1466019727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd8c0fd6c0 0x7fdd8c0ffaa0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:42.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:42.380+0000 7fdd93666700 1 -- 172.21.15.94:0/1466019727 >> 172.21.15.94:0/1466019727 conn(0x7fdd8c0f70c0 msgr2=0x7fdd8c0f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:37:42.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:42.380+0000 7fdd93666700 1 -- 172.21.15.94:0/1466019727 shutdown_connections 2024-03-24T10:37:42.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:42.380+0000 7fdd93666700 1 -- 172.21.15.94:0/1466019727 wait complete. 2024-03-24T10:37:42.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:42.381+0000 7fdd93666700 1 Processor -- start 2024-03-24T10:37:42.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:42.381+0000 7fdd93666700 1 -- start start 2024-03-24T10:37:42.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:42.381+0000 7fdd93666700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd8c0fd6c0 0x7fdd8c10d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:42.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:42.381+0000 7fdd93666700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8c10d600 con 0x7fdd8c0fd6c0 2024-03-24T10:37:42.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:42.381+0000 7fdd91402700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd8c0fd6c0 0x7fdd8c10d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:42.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:42.381+0000 7fdd91402700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd8c0fd6c0 0x7fdd8c10d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39352/0 (socket says 172.21.15.94:39352) 2024-03-24T10:37:42.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:42.381+0000 7fdd91402700 1 -- 172.21.15.94:0/732808321 learned_addr learned my addr 172.21.15.94:0/732808321 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:37:42.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:42.381+0000 7fdd91402700 1 -- 172.21.15.94:0/732808321 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd88005040 con 0x7fdd8c0fd6c0 2024-03-24T10:37:42.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:42.381+0000 7fdd91402700 1 --2- 172.21.15.94:0/732808321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd8c0fd6c0 0x7fdd8c10d0c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdd8800b900 tx=0x7fdd88009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:42.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:42.381+0000 7fdd81ffb700 1 -- 172.21.15.94:0/732808321 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdd8800b5c0 con 0x7fdd8c0fd6c0 2024-03-24T10:37:42.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:42.382+0000 7fdd81ffb700 1 -- 172.21.15.94:0/732808321 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd8800be70 con 0x7fdd8c0fd6c0 2024-03-24T10:37:42.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:42.382+0000 7fdd93666700 1 -- 172.21.15.94:0/732808321 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd8c10d800 con 0x7fdd8c0fd6c0 2024-03-24T10:37:42.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:42.382+0000 7fdd81ffb700 1 -- 172.21.15.94:0/732808321 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdd8801c590 con 0x7fdd8c0fd6c0 2024-03-24T10:37:42.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:42.382+0000 7fdd93666700 1 -- 172.21.15.94:0/732808321 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd8c10e370 con 0x7fdd8c0fd6c0 2024-03-24T10:37:42.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:42.382+0000 7fdd81ffb700 1 -- 172.21.15.94:0/732808321 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fdd8801c850 con 0x7fdd8c0fd6c0 2024-03-24T10:37:42.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:42.383+0000 7fdd93666700 1 -- 172.21.15.94:0/732808321 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd8c102f50 con 0x7fdd8c0fd6c0 2024-03-24T10:37:42.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:42.383+0000 7fdd81ffb700 1 --2- 172.21.15.94:0/732808321 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fdd78036880 0x7fdd78038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:42.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:42.383+0000 7fdd81ffb700 1 -- 172.21.15.94:0/732808321 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fdd88049350 con 0x7fdd8c0fd6c0 2024-03-24T10:37:42.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:42.383+0000 7fdd90c01700 1 --2- 172.21.15.94:0/732808321 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fdd78036880 0x7fdd78038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:42.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:42.383+0000 7fdd90c01700 1 --2- 172.21.15.94:0/732808321 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fdd78036880 0x7fdd78038d30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c009990 tx=0x7fdd7c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:42.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:42.387+0000 7fdd81ffb700 1 -- 172.21.15.94:0/732808321 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fdd88011960 con 0x7fdd8c0fd6c0 2024-03-24T10:37:42.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:42.613+0000 7fdd93666700 1 -- 172.21.15.94:0/732808321 --> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdd8c10e670 con 0x7fdd78036880 2024-03-24T10:37:42.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:42.615+0000 7fdd81ffb700 1 -- 172.21.15.94:0/732808321 <== mgr.14162 v2:172.21.15.94:6800/2406899470 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7fdd8c10e670 con 0x7fdd78036880 2024-03-24T10:37:42.615 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:37:42.615 INFO:teuthology.orchestra.run.smithi094.stdout:[{"addr": "172.21.15.94", "hostname": "smithi094", "labels": [], "status": ""}, {"addr": "172.21.15.96", "hostname": "smithi096", "labels": [], "status": ""}] 2024-03-24T10:37:42.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:42.617+0000 7fdd93666700 1 -- 172.21.15.94:0/732808321 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fdd78036880 msgr2=0x7fdd78038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:42.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:42.617+0000 7fdd93666700 1 --2- 172.21.15.94:0/732808321 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fdd78036880 0x7fdd78038d30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c009990 tx=0x7fdd7c006e40 comp rx=0 tx=0).stop 2024-03-24T10:37:42.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:42.617+0000 7fdd93666700 1 -- 172.21.15.94:0/732808321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd8c0fd6c0 msgr2=0x7fdd8c10d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:42.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:42.617+0000 7fdd93666700 1 --2- 172.21.15.94:0/732808321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd8c0fd6c0 0x7fdd8c10d0c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdd8800b900 tx=0x7fdd88009e30 comp rx=0 tx=0).stop 2024-03-24T10:37:42.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:42.617+0000 7fdd93666700 1 -- 172.21.15.94:0/732808321 shutdown_connections 2024-03-24T10:37:42.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:42.617+0000 7fdd93666700 1 --2- 172.21.15.94:0/732808321 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fdd78036880 0x7fdd78038d30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:42.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:42.617+0000 7fdd93666700 1 --2- 172.21.15.94:0/732808321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd8c0fd6c0 0x7fdd8c10d0c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:42.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:42.617+0000 7fdd93666700 1 -- 172.21.15.94:0/732808321 >> 172.21.15.94:0/732808321 conn(0x7fdd8c0f70c0 msgr2=0x7fdd8c103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:37:42.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:42.617+0000 7fdd93666700 1 -- 172.21.15.94:0/732808321 shutdown_connections 2024-03-24T10:37:42.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:42.617+0000 7fdd93666700 1 -- 172.21.15.94:0/732808321 wait complete. 2024-03-24T10:37:43.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:43 smithi094 ceph-mon[29446]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:37:43.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:43 smithi094 ceph-mon[29446]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-24T10:37:43.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:43 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' 2024-03-24T10:37:43.598 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-24T10:37:43.599 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph osd crush tunables default 2024-03-24T10:37:43.848 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:37:45.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:45 smithi094 ceph-mon[29446]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:37:47.288 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:47 smithi094 ceph-mon[29446]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:37:47.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:47.928+0000 7f7e6c0fe700 1 -- 172.21.15.94:0/2948051000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e640fe0a0 msgr2=0x7f7e640fe4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:47.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:47.928+0000 7f7e6c0fe700 1 --2- 172.21.15.94:0/2948051000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e640fe0a0 0x7f7e640fe4b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7e540048c0 tx=0x7f7e54009e30 comp rx=0 tx=0).stop 2024-03-24T10:37:47.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:47.928+0000 7f7e6c0fe700 1 -- 172.21.15.94:0/2948051000 shutdown_connections 2024-03-24T10:37:47.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:47.928+0000 7f7e6c0fe700 1 --2- 172.21.15.94:0/2948051000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e640fe0a0 0x7f7e640fe4b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:47.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:47.928+0000 7f7e6c0fe700 1 -- 172.21.15.94:0/2948051000 >> 172.21.15.94:0/2948051000 conn(0x7f7e640f6840 msgr2=0x7f7e640f8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:37:47.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:47.928+0000 7f7e6c0fe700 1 -- 172.21.15.94:0/2948051000 shutdown_connections 2024-03-24T10:37:47.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:47.928+0000 7f7e6c0fe700 1 -- 172.21.15.94:0/2948051000 wait complete. 2024-03-24T10:37:47.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:47.929+0000 7f7e6c0fe700 1 Processor -- start 2024-03-24T10:37:47.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:47.929+0000 7f7e6c0fe700 1 -- start start 2024-03-24T10:37:47.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:47.929+0000 7f7e6c0fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e640fe0a0 0x7f7e6410d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:47.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:47.929+0000 7f7e6c0fe700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6410d600 con 0x7f7e640fe0a0 2024-03-24T10:37:47.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:47.929+0000 7f7e69e9a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e640fe0a0 0x7f7e6410d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:47.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:47.929+0000 7f7e69e9a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e640fe0a0 0x7f7e6410d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39372/0 (socket says 172.21.15.94:39372) 2024-03-24T10:37:47.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:47.929+0000 7f7e69e9a700 1 -- 172.21.15.94:0/3493904431 learned_addr learned my addr 172.21.15.94:0/3493904431 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:37:47.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:47.930+0000 7f7e69e9a700 1 -- 172.21.15.94:0/3493904431 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e54005040 con 0x7f7e640fe0a0 2024-03-24T10:37:47.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:47.930+0000 7f7e69e9a700 1 --2- 172.21.15.94:0/3493904431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e640fe0a0 0x7f7e6410d0c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7e54009680 tx=0x7f7e54009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:47.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:47.930+0000 7f7e5a7fc700 1 -- 172.21.15.94:0/3493904431 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7e5400b5c0 con 0x7f7e640fe0a0 2024-03-24T10:37:47.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:47.930+0000 7f7e5a7fc700 1 -- 172.21.15.94:0/3493904431 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7e5400be70 con 0x7f7e640fe0a0 2024-03-24T10:37:47.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:47.930+0000 7f7e6c0fe700 1 -- 172.21.15.94:0/3493904431 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e6410d800 con 0x7f7e640fe0a0 2024-03-24T10:37:47.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:47.930+0000 7f7e5a7fc700 1 -- 172.21.15.94:0/3493904431 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7e540135a0 con 0x7f7e640fe0a0 2024-03-24T10:37:47.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:47.930+0000 7f7e6c0fe700 1 -- 172.21.15.94:0/3493904431 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e6410e370 con 0x7f7e640fe0a0 2024-03-24T10:37:47.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:47.931+0000 7f7e5a7fc700 1 -- 172.21.15.94:0/3493904431 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f7e54013860 con 0x7f7e640fe0a0 2024-03-24T10:37:47.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:47.931+0000 7f7e5a7fc700 1 --2- 172.21.15.94:0/3493904431 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f7e50036880 0x7f7e50038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:47.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:47.931+0000 7f7e5a7fc700 1 -- 172.21.15.94:0/3493904431 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f7e540493a0 con 0x7f7e640fe0a0 2024-03-24T10:37:47.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:47.931+0000 7f7e69699700 1 --2- 172.21.15.94:0/3493904431 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f7e50036880 0x7f7e50038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:47.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:47.931+0000 7f7e6c0fe700 1 -- 172.21.15.94:0/3493904431 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e64102f70 con 0x7f7e640fe0a0 2024-03-24T10:37:47.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:47.931+0000 7f7e69699700 1 --2- 172.21.15.94:0/3493904431 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f7e50036880 0x7f7e50038d30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7e60009990 tx=0x7f7e60006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:47.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:47.936+0000 7f7e5a7fc700 1 -- 172.21.15.94:0/3493904431 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f7e54021020 con 0x7f7e640fe0a0 2024-03-24T10:37:48.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:48.131+0000 7f7e6c0fe700 1 -- 172.21.15.94:0/3493904431 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f7e6405f5f0 con 0x7f7e640fe0a0 2024-03-24T10:37:48.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:48.252+0000 7f7e5a7fc700 1 -- 172.21.15.94:0/3493904431 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f7e54011960 con 0x7f7e640fe0a0 2024-03-24T10:37:48.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:48.254+0000 7f7e6c0fe700 1 -- 172.21.15.94:0/3493904431 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f7e50036880 msgr2=0x7f7e50038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:48.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:48.254+0000 7f7e6c0fe700 1 --2- 172.21.15.94:0/3493904431 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f7e50036880 0x7f7e50038d30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7e60009990 tx=0x7f7e60006e40 comp rx=0 tx=0).stop 2024-03-24T10:37:48.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:48.254+0000 7f7e6c0fe700 1 -- 172.21.15.94:0/3493904431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e640fe0a0 msgr2=0x7f7e6410d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:48.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:48.254+0000 7f7e6c0fe700 1 --2- 172.21.15.94:0/3493904431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e640fe0a0 0x7f7e6410d0c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7e54009680 tx=0x7f7e54009e30 comp rx=0 tx=0).stop 2024-03-24T10:37:48.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:48.254+0000 7f7e6c0fe700 1 -- 172.21.15.94:0/3493904431 shutdown_connections 2024-03-24T10:37:48.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:48.254+0000 7f7e6c0fe700 1 --2- 172.21.15.94:0/3493904431 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f7e50036880 0x7f7e50038d30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:48.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:48.254+0000 7f7e6c0fe700 1 --2- 172.21.15.94:0/3493904431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e640fe0a0 0x7f7e6410d0c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:48.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:48.254+0000 7f7e6c0fe700 1 -- 172.21.15.94:0/3493904431 >> 172.21.15.94:0/3493904431 conn(0x7f7e640f6840 msgr2=0x7f7e64103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:37:48.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:48.254+0000 7f7e6c0fe700 1 -- 172.21.15.94:0/3493904431 shutdown_connections 2024-03-24T10:37:48.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:37:48.254+0000 7f7e6c0fe700 1 -- 172.21.15.94:0/3493904431 wait complete. 2024-03-24T10:37:48.255 INFO:teuthology.orchestra.run.smithi094.stderr:adjusted tunables profile to default 2024-03-24T10:37:48.544 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:48 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/3493904431' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-24T10:37:49.148 INFO:tasks.cephadm:Adding mon.smithi094 on smithi094 2024-03-24T10:37:49.149 INFO:tasks.cephadm:Adding mon.smithi096 on smithi096 2024-03-24T10:37:49.149 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph orch apply mon '2;smithi094:172.21.15.94=smithi094;smithi096:172.21.15.96=smithi096' 2024-03-24T10:37:49.399 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T10:37:49.445 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T10:37:49.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:49 smithi094 ceph-mon[29446]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:37:49.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:49 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/3493904431' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-24T10:37:49.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:49 smithi094 ceph-mon[29446]: osdmap e4: 0 total, 0 up, 0 in 2024-03-24T10:37:51.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:51.002+0000 7fdcf0c43700 1 -- 172.21.15.96:0/2563394694 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcec0fb960 msgr2=0x7fdcec0fbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:51.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:51.002+0000 7fdcf0c43700 1 --2- 172.21.15.96:0/2563394694 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcec0fb960 0x7fdcec0fbd70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc0048c0 tx=0x7fdcdc009e30 comp rx=0 tx=0).stop 2024-03-24T10:37:51.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:51.002+0000 7fdcf0c43700 1 -- 172.21.15.96:0/2563394694 shutdown_connections 2024-03-24T10:37:51.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:51.002+0000 7fdcf0c43700 1 --2- 172.21.15.96:0/2563394694 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcec0fb960 0x7fdcec0fbd70 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:51.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:51.002+0000 7fdcf0c43700 1 -- 172.21.15.96:0/2563394694 >> 172.21.15.96:0/2563394694 conn(0x7fdcec0f70c0 msgr2=0x7fdcec0f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:37:51.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:51.002+0000 7fdcf0c43700 1 -- 172.21.15.96:0/2563394694 shutdown_connections 2024-03-24T10:37:51.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:51.002+0000 7fdcf0c43700 1 -- 172.21.15.96:0/2563394694 wait complete. 2024-03-24T10:37:51.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:51.003+0000 7fdcf0c43700 1 Processor -- start 2024-03-24T10:37:51.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:51.003+0000 7fdcf0c43700 1 -- start start 2024-03-24T10:37:51.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:51.003+0000 7fdcf0c43700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcec0fb960 0x7fdcec10d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:51.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:51.003+0000 7fdcf0c43700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcec10d600 con 0x7fdcec0fb960 2024-03-24T10:37:51.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:51.003+0000 7fdcea59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcec0fb960 0x7fdcec10d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:51.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:51.003+0000 7fdcea59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcec0fb960 0x7fdcec10d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.96:38688/0 (socket says 172.21.15.96:38688) 2024-03-24T10:37:51.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:51.003+0000 7fdcea59c700 1 -- 172.21.15.96:0/202795684 learned_addr learned my addr 172.21.15.96:0/202795684 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-24T10:37:51.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:51.003+0000 7fdcea59c700 1 -- 172.21.15.96:0/202795684 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcdc005040 con 0x7fdcec0fb960 2024-03-24T10:37:51.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:51.003+0000 7fdcea59c700 1 --2- 172.21.15.96:0/202795684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcec0fb960 0x7fdcec10d0c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc0096b0 tx=0x7fdcdc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:51.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:51.004+0000 7fdce37fe700 1 -- 172.21.15.96:0/202795684 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdcdc00bed0 con 0x7fdcec0fb960 2024-03-24T10:37:51.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:51.004+0000 7fdcf0c43700 1 -- 172.21.15.96:0/202795684 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcec10d800 con 0x7fdcec0fb960 2024-03-24T10:37:51.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:51.004+0000 7fdce37fe700 1 -- 172.21.15.96:0/202795684 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdcdc00b5c0 con 0x7fdcec0fb960 2024-03-24T10:37:51.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:51.004+0000 7fdce37fe700 1 -- 172.21.15.96:0/202795684 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdcdc01c5b0 con 0x7fdcec0fb960 2024-03-24T10:37:51.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:51.004+0000 7fdcf0c43700 1 -- 172.21.15.96:0/202795684 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcec10e370 con 0x7fdcec0fb960 2024-03-24T10:37:51.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:51.005+0000 7fdce37fe700 1 -- 172.21.15.96:0/202795684 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fdcdc01c820 con 0x7fdcec0fb960 2024-03-24T10:37:51.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:51.005+0000 7fdcf0c43700 1 -- 172.21.15.96:0/202795684 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcec101e70 con 0x7fdcec0fb960 2024-03-24T10:37:51.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:51.005+0000 7fdce37fe700 1 --2- 172.21.15.96:0/202795684 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fdcd8036880 0x7fdcd8038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:51.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:51.005+0000 7fdce37fe700 1 -- 172.21.15.96:0/202795684 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fdcdc049c00 con 0x7fdcec0fb960 2024-03-24T10:37:51.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:51.005+0000 7fdce1d9b700 1 --2- 172.21.15.96:0/202795684 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fdcd8036880 0x7fdcd8038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:51.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:51.005+0000 7fdce1d9b700 1 --2- 172.21.15.96:0/202795684 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fdcd8036880 0x7fdcd8038d30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4009990 tx=0x7fdcd4006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:51.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:51.013+0000 7fdce37fe700 1 -- 172.21.15.96:0/202795684 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fdcdc013420 con 0x7fdcec0fb960 2024-03-24T10:37:51.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:51.208+0000 7fdcf0c43700 1 -- 172.21.15.96:0/202795684 --> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi094:172.21.15.94=smithi094;smithi096:172.21.15.96=smithi096", "target": ["mon-mgr", ""]}) v1 -- 0x7fdcec0664c0 con 0x7fdcd8036880 2024-03-24T10:37:51.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:51.214+0000 7fdce37fe700 1 -- 172.21.15.96:0/202795684 <== mgr.14162 v2:172.21.15.94:6800/2406899470 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fdcec0664c0 con 0x7fdcd8036880 2024-03-24T10:37:51.214 INFO:teuthology.orchestra.run.smithi096.stdout:Scheduled mon update... 2024-03-24T10:37:51.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:51.215+0000 7fdcf0c43700 1 -- 172.21.15.96:0/202795684 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fdcd8036880 msgr2=0x7fdcd8038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:51.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:51.215+0000 7fdcf0c43700 1 --2- 172.21.15.96:0/202795684 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fdcd8036880 0x7fdcd8038d30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4009990 tx=0x7fdcd4006e40 comp rx=0 tx=0).stop 2024-03-24T10:37:51.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:51.215+0000 7fdcf0c43700 1 -- 172.21.15.96:0/202795684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcec0fb960 msgr2=0x7fdcec10d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:51.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:51.215+0000 7fdcf0c43700 1 --2- 172.21.15.96:0/202795684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcec0fb960 0x7fdcec10d0c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc0096b0 tx=0x7fdcdc009e30 comp rx=0 tx=0).stop 2024-03-24T10:37:51.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:51.215+0000 7fdcf0c43700 1 -- 172.21.15.96:0/202795684 shutdown_connections 2024-03-24T10:37:51.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:51.215+0000 7fdcf0c43700 1 --2- 172.21.15.96:0/202795684 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fdcd8036880 0x7fdcd8038d30 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:51.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:51.215+0000 7fdcf0c43700 1 --2- 172.21.15.96:0/202795684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcec0fb960 0x7fdcec10d0c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:51.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:51.215+0000 7fdcf0c43700 1 -- 172.21.15.96:0/202795684 >> 172.21.15.96:0/202795684 conn(0x7fdcec0f70c0 msgr2=0x7fdcec065d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:37:51.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:51.216+0000 7fdcf0c43700 1 -- 172.21.15.96:0/202795684 shutdown_connections 2024-03-24T10:37:51.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:51.216+0000 7fdcf0c43700 1 -- 172.21.15.96:0/202795684 wait complete. 2024-03-24T10:37:51.774 DEBUG:teuthology.orchestra.run.smithi096:mon.smithi096> sudo journalctl -f -n 0 -u ceph-386110d4-e9ca-11ee-95cd-87774f69a715@mon.smithi096.service 2024-03-24T10:37:51.777 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-24T10:37:51.778 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T10:37:51.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:51 smithi094 ceph-mon[29446]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:37:51.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:51 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' 2024-03-24T10:37:51.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:51 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' 2024-03-24T10:37:51.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:51 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' 2024-03-24T10:37:51.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:51 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' 2024-03-24T10:37:51.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:51 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' 2024-03-24T10:37:51.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:51 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' 2024-03-24T10:37:51.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:51 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-24T10:37:51.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:51 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' 2024-03-24T10:37:51.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:51 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' 2024-03-24T10:37:52.054 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T10:37:52.100 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T10:37:52.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:52 smithi094 ceph-mon[29446]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-24T10:37:52.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:52 smithi094 ceph-mon[29446]: Deploying daemon grafana.smithi094 on smithi094 2024-03-24T10:37:52.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:52 smithi094 ceph-mon[29446]: from='client.14194 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi094:172.21.15.94=smithi094;smithi096:172.21.15.96=smithi096", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:37:52.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:52 smithi094 ceph-mon[29446]: Saving service mon spec with placement smithi094:172.21.15.94=smithi094;smithi096:172.21.15.96=smithi096;count:2 2024-03-24T10:37:53.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:53.578+0000 7f7504af1700 1 -- 172.21.15.96:0/3527386840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75000fb960 msgr2=0x7f75000fbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:53.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:53.578+0000 7f7504af1700 1 --2- 172.21.15.96:0/3527386840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75000fb960 0x7f75000fbd70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f74f00048c0 tx=0x7f74f0009e30 comp rx=0 tx=0).stop 2024-03-24T10:37:53.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:53.578+0000 7f7504af1700 1 -- 172.21.15.96:0/3527386840 shutdown_connections 2024-03-24T10:37:53.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:53.578+0000 7f7504af1700 1 --2- 172.21.15.96:0/3527386840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75000fb960 0x7f75000fbd70 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:53.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:53.578+0000 7f7504af1700 1 -- 172.21.15.96:0/3527386840 >> 172.21.15.96:0/3527386840 conn(0x7f75000f70c0 msgr2=0x7f75000f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:37:53.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:53.578+0000 7f7504af1700 1 -- 172.21.15.96:0/3527386840 shutdown_connections 2024-03-24T10:37:53.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:53.578+0000 7f7504af1700 1 -- 172.21.15.96:0/3527386840 wait complete. 2024-03-24T10:37:53.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:53.578+0000 7f7504af1700 1 Processor -- start 2024-03-24T10:37:53.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:53.578+0000 7f7504af1700 1 -- start start 2024-03-24T10:37:53.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:53.578+0000 7f7504af1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75000fb960 0x7f75000668c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:53.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:53.578+0000 7f7504af1700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f750010fd50 con 0x7f75000fb960 2024-03-24T10:37:53.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:53.579+0000 7f74fe59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75000fb960 0x7f75000668c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:53.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:53.579+0000 7f74fe59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75000fb960 0x7f75000668c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.96:38696/0 (socket says 172.21.15.96:38696) 2024-03-24T10:37:53.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:53.579+0000 7f74fe59c700 1 -- 172.21.15.96:0/978142015 learned_addr learned my addr 172.21.15.96:0/978142015 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-24T10:37:53.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:53.579+0000 7f74fe59c700 1 -- 172.21.15.96:0/978142015 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f0005040 con 0x7f75000fb960 2024-03-24T10:37:53.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:53.579+0000 7f74fe59c700 1 --2- 172.21.15.96:0/978142015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75000fb960 0x7f75000668c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f74f00096b0 tx=0x7f74f0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:53.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:53.579+0000 7f74eeffd700 1 -- 172.21.15.96:0/978142015 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f74f000bed0 con 0x7f75000fb960 2024-03-24T10:37:53.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:53.579+0000 7f74eeffd700 1 -- 172.21.15.96:0/978142015 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f74f000b5c0 con 0x7f75000fb960 2024-03-24T10:37:53.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:53.579+0000 7f74eeffd700 1 -- 172.21.15.96:0/978142015 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f74f001c5b0 con 0x7f75000fb960 2024-03-24T10:37:53.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:53.579+0000 7f7504af1700 1 -- 172.21.15.96:0/978142015 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f750010fef0 con 0x7f75000fb960 2024-03-24T10:37:53.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:53.579+0000 7f7504af1700 1 -- 172.21.15.96:0/978142015 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7500110380 con 0x7f75000fb960 2024-03-24T10:37:53.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:53.580+0000 7f74eeffd700 1 -- 172.21.15.96:0/978142015 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f74f001c820 con 0x7f75000fb960 2024-03-24T10:37:53.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:53.580+0000 7f7504af1700 1 -- 172.21.15.96:0/978142015 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7500105920 con 0x7f75000fb960 2024-03-24T10:37:53.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:53.580+0000 7f74eeffd700 1 --2- 172.21.15.96:0/978142015 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f74e8036880 0x7f74e8038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:53.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:53.580+0000 7f74eeffd700 1 -- 172.21.15.96:0/978142015 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f74f0049a80 con 0x7f75000fb960 2024-03-24T10:37:53.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:53.580+0000 7f74fdd9b700 1 --2- 172.21.15.96:0/978142015 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f74e8036880 0x7f74e8038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:53.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:53.581+0000 7f74fdd9b700 1 --2- 172.21.15.96:0/978142015 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f74e8036880 0x7f74e8038d30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f74f4009990 tx=0x7f74f4006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:53.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:53.585+0000 7f74eeffd700 1 -- 172.21.15.96:0/978142015 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f74f0023300 con 0x7f75000fb960 2024-03-24T10:37:53.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:53.845+0000 7f7504af1700 1 -- 172.21.15.96:0/978142015 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f750005f5f0 con 0x7f75000fb960 2024-03-24T10:37:53.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:53.846+0000 7f74eeffd700 1 -- 172.21.15.96:0/978142015 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f74f0011b60 con 0x7f75000fb960 2024-03-24T10:37:53.851 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:37:53.851 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":1,"fsid":"386110d4-e9ca-11ee-95cd-87774f69a715","modified":"2024-03-24T10:36:06.632144Z","created":"2024-03-24T10:36:06.632144Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi094","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-24T10:37:53.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:53.848+0000 7f7504af1700 1 -- 172.21.15.96:0/978142015 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f74e8036880 msgr2=0x7f74e8038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:53.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:53.848+0000 7f7504af1700 1 --2- 172.21.15.96:0/978142015 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f74e8036880 0x7f74e8038d30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f74f4009990 tx=0x7f74f4006e40 comp rx=0 tx=0).stop 2024-03-24T10:37:53.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:53.848+0000 7f7504af1700 1 -- 172.21.15.96:0/978142015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75000fb960 msgr2=0x7f75000668c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:53.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:53.848+0000 7f7504af1700 1 --2- 172.21.15.96:0/978142015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75000fb960 0x7f75000668c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f74f00096b0 tx=0x7f74f0009e30 comp rx=0 tx=0).stop 2024-03-24T10:37:53.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:53.848+0000 7f7504af1700 1 -- 172.21.15.96:0/978142015 shutdown_connections 2024-03-24T10:37:53.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:53.848+0000 7f7504af1700 1 --2- 172.21.15.96:0/978142015 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f74e8036880 0x7f74e8038d30 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:53.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:53.848+0000 7f7504af1700 1 --2- 172.21.15.96:0/978142015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75000fb960 0x7f75000668c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:53.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:53.848+0000 7f7504af1700 1 -- 172.21.15.96:0/978142015 >> 172.21.15.96:0/978142015 conn(0x7f75000f70c0 msgr2=0x7f7500102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:37:53.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:53.848+0000 7f7504af1700 1 -- 172.21.15.96:0/978142015 shutdown_connections 2024-03-24T10:37:53.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:53.848+0000 7f7504af1700 1 -- 172.21.15.96:0/978142015 wait complete. 2024-03-24T10:37:53.853 INFO:teuthology.orchestra.run.smithi096.stderr:dumped monmap epoch 1 2024-03-24T10:37:53.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:53 smithi094 ceph-mon[29446]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:37:53.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:53 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' 2024-03-24T10:37:54.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:54 smithi094 ceph-mon[29446]: from='client.? 172.21.15.96:0/978142015' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T10:37:55.434 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-24T10:37:55.434 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T10:37:55.685 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T10:37:55.731 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T10:37:55.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:55 smithi094 ceph-mon[29446]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:37:57.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:57.121+0000 7ffa24f60700 1 -- 172.21.15.96:0/503469418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa200fd050 msgr2=0x7ffa200fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:57.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:57.121+0000 7ffa24f60700 1 --2- 172.21.15.96:0/503469418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa200fd050 0x7ffa200fd460 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ffa180048c0 tx=0x7ffa18009e30 comp rx=0 tx=0).stop 2024-03-24T10:37:57.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:57.121+0000 7ffa24f60700 1 -- 172.21.15.96:0/503469418 shutdown_connections 2024-03-24T10:37:57.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:57.121+0000 7ffa24f60700 1 --2- 172.21.15.96:0/503469418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa200fd050 0x7ffa200fd460 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:57.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:57.121+0000 7ffa24f60700 1 -- 172.21.15.96:0/503469418 >> 172.21.15.96:0/503469418 conn(0x7ffa200f70d0 msgr2=0x7ffa200f9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:37:57.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:57.121+0000 7ffa24f60700 1 -- 172.21.15.96:0/503469418 shutdown_connections 2024-03-24T10:37:57.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:57.121+0000 7ffa24f60700 1 -- 172.21.15.96:0/503469418 wait complete. 2024-03-24T10:37:57.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:57.122+0000 7ffa24f60700 1 Processor -- start 2024-03-24T10:37:57.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:57.122+0000 7ffa24f60700 1 -- start start 2024-03-24T10:37:57.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:57.122+0000 7ffa24f60700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa200fd050 0x7ffa20065bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:57.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:57.122+0000 7ffa24f60700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2010fdc0 con 0x7ffa200fd050 2024-03-24T10:37:57.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:57.122+0000 7ffa1e59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa200fd050 0x7ffa20065bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:57.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:57.122+0000 7ffa1e59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa200fd050 0x7ffa20065bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.96:60820/0 (socket says 172.21.15.96:60820) 2024-03-24T10:37:57.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:57.122+0000 7ffa1e59c700 1 -- 172.21.15.96:0/54136462 learned_addr learned my addr 172.21.15.96:0/54136462 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-24T10:37:57.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:57.122+0000 7ffa1e59c700 1 -- 172.21.15.96:0/54136462 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa18005040 con 0x7ffa200fd050 2024-03-24T10:37:57.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:57.122+0000 7ffa1e59c700 1 --2- 172.21.15.96:0/54136462 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa200fd050 0x7ffa20065bf0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ffa180096b0 tx=0x7ffa18009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:57.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:57.123+0000 7ffa16ffd700 1 -- 172.21.15.96:0/54136462 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffa1800bed0 con 0x7ffa200fd050 2024-03-24T10:37:57.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:57.123+0000 7ffa16ffd700 1 -- 172.21.15.96:0/54136462 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffa1800b5c0 con 0x7ffa200fd050 2024-03-24T10:37:57.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:57.123+0000 7ffa24f60700 1 -- 172.21.15.96:0/54136462 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa2010ff00 con 0x7ffa200fd050 2024-03-24T10:37:57.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:57.123+0000 7ffa16ffd700 1 -- 172.21.15.96:0/54136462 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffa1801c5b0 con 0x7ffa200fd050 2024-03-24T10:37:57.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:57.123+0000 7ffa24f60700 1 -- 172.21.15.96:0/54136462 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa201103f0 con 0x7ffa200fd050 2024-03-24T10:37:57.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:57.123+0000 7ffa24f60700 1 -- 172.21.15.96:0/54136462 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa20105920 con 0x7ffa200fd050 2024-03-24T10:37:57.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:57.124+0000 7ffa16ffd700 1 -- 172.21.15.96:0/54136462 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7ffa18013420 con 0x7ffa200fd050 2024-03-24T10:37:57.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:57.124+0000 7ffa16ffd700 1 --2- 172.21.15.96:0/54136462 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7ffa0c0368d0 0x7ffa0c038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:37:57.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:57.124+0000 7ffa16ffd700 1 -- 172.21.15.96:0/54136462 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7ffa18049a40 con 0x7ffa200fd050 2024-03-24T10:37:57.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:57.124+0000 7ffa1dd9b700 1 --2- 172.21.15.96:0/54136462 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7ffa0c0368d0 0x7ffa0c038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:37:57.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:57.124+0000 7ffa1dd9b700 1 --2- 172.21.15.96:0/54136462 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7ffa0c0368d0 0x7ffa0c038d80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ffa08009990 tx=0x7ffa08006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:37:57.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:57.129+0000 7ffa16ffd700 1 -- 172.21.15.96:0/54136462 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7ffa18013a80 con 0x7ffa200fd050 2024-03-24T10:37:57.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:57.389+0000 7ffa24f60700 1 -- 172.21.15.96:0/54136462 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ffa20101e70 con 0x7ffa200fd050 2024-03-24T10:37:57.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:57.390+0000 7ffa16ffd700 1 -- 172.21.15.96:0/54136462 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7ffa18011370 con 0x7ffa200fd050 2024-03-24T10:37:57.391 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:37:57.391 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":1,"fsid":"386110d4-e9ca-11ee-95cd-87774f69a715","modified":"2024-03-24T10:36:06.632144Z","created":"2024-03-24T10:36:06.632144Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi094","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-24T10:37:57.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:57.392+0000 7ffa24f60700 1 -- 172.21.15.96:0/54136462 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7ffa0c0368d0 msgr2=0x7ffa0c038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:57.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:57.392+0000 7ffa24f60700 1 --2- 172.21.15.96:0/54136462 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7ffa0c0368d0 0x7ffa0c038d80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ffa08009990 tx=0x7ffa08006e40 comp rx=0 tx=0).stop 2024-03-24T10:37:57.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:57.392+0000 7ffa24f60700 1 -- 172.21.15.96:0/54136462 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa200fd050 msgr2=0x7ffa20065bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:37:57.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:57.392+0000 7ffa24f60700 1 --2- 172.21.15.96:0/54136462 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa200fd050 0x7ffa20065bf0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ffa180096b0 tx=0x7ffa18009e30 comp rx=0 tx=0).stop 2024-03-24T10:37:57.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:57.392+0000 7ffa24f60700 1 -- 172.21.15.96:0/54136462 shutdown_connections 2024-03-24T10:37:57.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:57.392+0000 7ffa24f60700 1 --2- 172.21.15.96:0/54136462 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7ffa0c0368d0 0x7ffa0c038d80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:57.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:57.392+0000 7ffa24f60700 1 --2- 172.21.15.96:0/54136462 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa200fd050 0x7ffa20065bf0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:37:57.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:57.392+0000 7ffa24f60700 1 -- 172.21.15.96:0/54136462 >> 172.21.15.96:0/54136462 conn(0x7ffa200f70d0 msgr2=0x7ffa20102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:37:57.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:57.392+0000 7ffa24f60700 1 -- 172.21.15.96:0/54136462 shutdown_connections 2024-03-24T10:37:57.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:37:57.392+0000 7ffa24f60700 1 -- 172.21.15.96:0/54136462 wait complete. 2024-03-24T10:37:57.393 INFO:teuthology.orchestra.run.smithi096.stderr:dumped monmap epoch 1 2024-03-24T10:37:57.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:57 smithi094 ceph-mon[29446]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:37:57.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:57 smithi094 ceph-mon[29446]: from='client.? 172.21.15.96:0/54136462' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T10:37:58.921 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-24T10:37:58.921 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T10:37:59.166 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T10:37:59.212 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T10:37:59.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:37:59 smithi094 ceph-mon[29446]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:38:00.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:00.586+0000 7fed68a1a700 1 -- 172.21.15.96:0/3789787705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed640fd040 msgr2=0x7fed640fd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:38:00.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:00.586+0000 7fed68a1a700 1 --2- 172.21.15.96:0/3789787705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed640fd040 0x7fed640fd450 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fed540048c0 tx=0x7fed54009e30 comp rx=0 tx=0).stop 2024-03-24T10:38:00.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:00.586+0000 7fed68a1a700 1 -- 172.21.15.96:0/3789787705 shutdown_connections 2024-03-24T10:38:00.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:00.586+0000 7fed68a1a700 1 --2- 172.21.15.96:0/3789787705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed640fd040 0x7fed640fd450 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:00.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:00.586+0000 7fed68a1a700 1 -- 172.21.15.96:0/3789787705 >> 172.21.15.96:0/3789787705 conn(0x7fed640f70c0 msgr2=0x7fed640f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:38:00.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:00.586+0000 7fed68a1a700 1 -- 172.21.15.96:0/3789787705 shutdown_connections 2024-03-24T10:38:00.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:00.586+0000 7fed68a1a700 1 -- 172.21.15.96:0/3789787705 wait complete. 2024-03-24T10:38:00.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:00.586+0000 7fed68a1a700 1 Processor -- start 2024-03-24T10:38:00.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:00.587+0000 7fed68a1a700 1 -- start start 2024-03-24T10:38:00.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:00.587+0000 7fed68a1a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed640fd040 0x7fed6410bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:38:00.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:00.587+0000 7fed68a1a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6410c4c0 con 0x7fed640fd040 2024-03-24T10:38:00.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:00.587+0000 7fed6259c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed640fd040 0x7fed6410bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:38:00.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:00.587+0000 7fed6259c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed640fd040 0x7fed6410bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.96:60838/0 (socket says 172.21.15.96:60838) 2024-03-24T10:38:00.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:00.587+0000 7fed6259c700 1 -- 172.21.15.96:0/1761015349 learned_addr learned my addr 172.21.15.96:0/1761015349 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-24T10:38:00.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:00.587+0000 7fed6259c700 1 -- 172.21.15.96:0/1761015349 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed54005040 con 0x7fed640fd040 2024-03-24T10:38:00.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:00.587+0000 7fed6259c700 1 --2- 172.21.15.96:0/1761015349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed640fd040 0x7fed6410bf80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fed540096b0 tx=0x7fed54009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:38:00.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:00.588+0000 7fed52ffd700 1 -- 172.21.15.96:0/1761015349 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fed5400bed0 con 0x7fed640fd040 2024-03-24T10:38:00.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:00.588+0000 7fed68a1a700 1 -- 172.21.15.96:0/1761015349 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed6410c6c0 con 0x7fed640fd040 2024-03-24T10:38:00.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:00.588+0000 7fed68a1a700 1 -- 172.21.15.96:0/1761015349 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed6410d230 con 0x7fed640fd040 2024-03-24T10:38:00.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:00.588+0000 7fed52ffd700 1 -- 172.21.15.96:0/1761015349 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fed5400b5c0 con 0x7fed640fd040 2024-03-24T10:38:00.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:00.588+0000 7fed52ffd700 1 -- 172.21.15.96:0/1761015349 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fed54025400 con 0x7fed640fd040 2024-03-24T10:38:00.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:00.589+0000 7fed68a1a700 1 -- 172.21.15.96:0/1761015349 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed440052e0 con 0x7fed640fd040 2024-03-24T10:38:00.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:00.589+0000 7fed52ffd700 1 -- 172.21.15.96:0/1761015349 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fed54013420 con 0x7fed640fd040 2024-03-24T10:38:00.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:00.589+0000 7fed52ffd700 1 --2- 172.21.15.96:0/1761015349 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fed4c036880 0x7fed4c038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:38:00.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:00.589+0000 7fed52ffd700 1 -- 172.21.15.96:0/1761015349 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fed54049990 con 0x7fed640fd040 2024-03-24T10:38:00.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:00.589+0000 7fed61d9b700 1 --2- 172.21.15.96:0/1761015349 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fed4c036880 0x7fed4c038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:38:00.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:00.590+0000 7fed61d9b700 1 --2- 172.21.15.96:0/1761015349 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fed4c036880 0x7fed4c038d30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fed58009990 tx=0x7fed58006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:38:00.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:00.594+0000 7fed52ffd700 1 -- 172.21.15.96:0/1761015349 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fed54011d40 con 0x7fed640fd040 2024-03-24T10:38:00.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:00.857+0000 7fed68a1a700 1 -- 172.21.15.96:0/1761015349 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fed44005590 con 0x7fed640fd040 2024-03-24T10:38:00.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:00.857+0000 7fed52ffd700 1 -- 172.21.15.96:0/1761015349 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fed54011b60 con 0x7fed640fd040 2024-03-24T10:38:00.858 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:38:00.858 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":1,"fsid":"386110d4-e9ca-11ee-95cd-87774f69a715","modified":"2024-03-24T10:36:06.632144Z","created":"2024-03-24T10:36:06.632144Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi094","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-24T10:38:00.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:00.859+0000 7fed68a1a700 1 -- 172.21.15.96:0/1761015349 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fed4c036880 msgr2=0x7fed4c038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:38:00.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:00.859+0000 7fed68a1a700 1 --2- 172.21.15.96:0/1761015349 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fed4c036880 0x7fed4c038d30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fed58009990 tx=0x7fed58006e40 comp rx=0 tx=0).stop 2024-03-24T10:38:00.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:00.859+0000 7fed68a1a700 1 -- 172.21.15.96:0/1761015349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed640fd040 msgr2=0x7fed6410bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:38:00.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:00.859+0000 7fed68a1a700 1 --2- 172.21.15.96:0/1761015349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed640fd040 0x7fed6410bf80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fed540096b0 tx=0x7fed54009e30 comp rx=0 tx=0).stop 2024-03-24T10:38:00.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:00.859+0000 7fed68a1a700 1 -- 172.21.15.96:0/1761015349 shutdown_connections 2024-03-24T10:38:00.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:00.859+0000 7fed68a1a700 1 --2- 172.21.15.96:0/1761015349 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fed4c036880 0x7fed4c038d30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:00.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:00.859+0000 7fed68a1a700 1 --2- 172.21.15.96:0/1761015349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed640fd040 0x7fed6410bf80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:00.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:00.859+0000 7fed68a1a700 1 -- 172.21.15.96:0/1761015349 >> 172.21.15.96:0/1761015349 conn(0x7fed640f70c0 msgr2=0x7fed64102a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:38:00.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:00.859+0000 7fed68a1a700 1 -- 172.21.15.96:0/1761015349 shutdown_connections 2024-03-24T10:38:00.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:00.859+0000 7fed68a1a700 1 -- 172.21.15.96:0/1761015349 wait complete. 2024-03-24T10:38:00.861 INFO:teuthology.orchestra.run.smithi096.stderr:dumped monmap epoch 1 2024-03-24T10:38:01.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:01 smithi094 ceph-mon[29446]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:38:01.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:01 smithi094 ceph-mon[29446]: from='client.? 172.21.15.96:0/1761015349' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T10:38:02.415 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-24T10:38:02.416 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T10:38:02.666 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T10:38:02.713 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T10:38:03.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:03 smithi094 ceph-mon[29446]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:38:04.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:04.134+0000 7f8cf19b0700 1 -- 172.21.15.96:0/1596672500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cec066560 msgr2=0x7f8cec0ff6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:38:04.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:04.134+0000 7f8cf19b0700 1 --2- 172.21.15.96:0/1596672500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cec066560 0x7f8cec0ff6d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc009c60 tx=0x7f8cdc004d10 comp rx=0 tx=0).stop 2024-03-24T10:38:04.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:04.134+0000 7f8cf19b0700 1 -- 172.21.15.96:0/1596672500 shutdown_connections 2024-03-24T10:38:04.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:04.134+0000 7f8cf19b0700 1 --2- 172.21.15.96:0/1596672500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cec066560 0x7f8cec0ff6d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:04.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:04.134+0000 7f8cf19b0700 1 -- 172.21.15.96:0/1596672500 >> 172.21.15.96:0/1596672500 conn(0x7f8cec0f70c0 msgr2=0x7f8cec0f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:38:04.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:04.134+0000 7f8cf19b0700 1 -- 172.21.15.96:0/1596672500 shutdown_connections 2024-03-24T10:38:04.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:04.134+0000 7f8cf19b0700 1 -- 172.21.15.96:0/1596672500 wait complete. 2024-03-24T10:38:04.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:04.135+0000 7f8cf19b0700 1 Processor -- start 2024-03-24T10:38:04.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:04.135+0000 7f8cf19b0700 1 -- start start 2024-03-24T10:38:04.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:04.135+0000 7f8cf19b0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cec066560 0x7f8cec10d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:38:04.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:04.135+0000 7f8cf19b0700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cec10d5f0 con 0x7f8cec066560 2024-03-24T10:38:04.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:04.135+0000 7f8ceaffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cec066560 0x7f8cec10d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:38:04.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:04.135+0000 7f8ceaffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cec066560 0x7f8cec10d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.96:60848/0 (socket says 172.21.15.96:60848) 2024-03-24T10:38:04.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:04.135+0000 7f8ceaffd700 1 -- 172.21.15.96:0/2103632037 learned_addr learned my addr 172.21.15.96:0/2103632037 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-24T10:38:04.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:04.135+0000 7f8ceaffd700 1 -- 172.21.15.96:0/2103632037 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cdc009940 con 0x7f8cec066560 2024-03-24T10:38:04.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:04.136+0000 7f8ceaffd700 1 --2- 172.21.15.96:0/2103632037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cec066560 0x7f8cec10d0b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc0097b0 tx=0x7f8cdc004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:38:04.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:04.136+0000 7f8ce3fff700 1 -- 172.21.15.96:0/2103632037 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8cdc012410 con 0x7f8cec066560 2024-03-24T10:38:04.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:04.136+0000 7f8cf19b0700 1 -- 172.21.15.96:0/2103632037 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cec10d7f0 con 0x7f8cec066560 2024-03-24T10:38:04.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:04.136+0000 7f8ce3fff700 1 -- 172.21.15.96:0/2103632037 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8cdc012a40 con 0x7f8cec066560 2024-03-24T10:38:04.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:04.136+0000 7f8ce3fff700 1 -- 172.21.15.96:0/2103632037 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8cdc01b980 con 0x7f8cec066560 2024-03-24T10:38:04.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:04.136+0000 7f8cf19b0700 1 -- 172.21.15.96:0/2103632037 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cec10e360 con 0x7f8cec066560 2024-03-24T10:38:04.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:04.137+0000 7f8cf19b0700 1 -- 172.21.15.96:0/2103632037 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cec102f60 con 0x7f8cec066560 2024-03-24T10:38:04.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:04.137+0000 7f8ce3fff700 1 -- 172.21.15.96:0/2103632037 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f8cdc019040 con 0x7f8cec066560 2024-03-24T10:38:04.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:04.137+0000 7f8ce3fff700 1 --2- 172.21.15.96:0/2103632037 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f8cd8036880 0x7f8cd8038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:38:04.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:04.137+0000 7f8ce3fff700 1 -- 172.21.15.96:0/2103632037 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f8cdc049cb0 con 0x7f8cec066560 2024-03-24T10:38:04.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:04.137+0000 7f8ce27fc700 1 --2- 172.21.15.96:0/2103632037 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f8cd8036880 0x7f8cd8038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:38:04.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:04.138+0000 7f8ce27fc700 1 --2- 172.21.15.96:0/2103632037 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f8cd8036880 0x7f8cd8038d30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4009990 tx=0x7f8cd4006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:38:04.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:04.143+0000 7f8ce3fff700 1 -- 172.21.15.96:0/2103632037 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f8cdc012bb0 con 0x7f8cec066560 2024-03-24T10:38:04.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:04.403+0000 7f8cf19b0700 1 -- 172.21.15.96:0/2103632037 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8cec10ed70 con 0x7f8cec066560 2024-03-24T10:38:04.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:04.404+0000 7f8ce3fff700 1 -- 172.21.15.96:0/2103632037 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f8cdc017090 con 0x7f8cec066560 2024-03-24T10:38:04.405 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:38:04.406 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":1,"fsid":"386110d4-e9ca-11ee-95cd-87774f69a715","modified":"2024-03-24T10:36:06.632144Z","created":"2024-03-24T10:36:06.632144Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi094","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-24T10:38:04.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:04.406+0000 7f8cf19b0700 1 -- 172.21.15.96:0/2103632037 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f8cd8036880 msgr2=0x7f8cd8038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:38:04.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:04.406+0000 7f8cf19b0700 1 --2- 172.21.15.96:0/2103632037 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f8cd8036880 0x7f8cd8038d30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4009990 tx=0x7f8cd4006e40 comp rx=0 tx=0).stop 2024-03-24T10:38:04.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:04.406+0000 7f8cf19b0700 1 -- 172.21.15.96:0/2103632037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cec066560 msgr2=0x7f8cec10d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:38:04.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:04.406+0000 7f8cf19b0700 1 --2- 172.21.15.96:0/2103632037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cec066560 0x7f8cec10d0b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc0097b0 tx=0x7f8cdc004d10 comp rx=0 tx=0).stop 2024-03-24T10:38:04.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:04.406+0000 7f8cf19b0700 1 -- 172.21.15.96:0/2103632037 shutdown_connections 2024-03-24T10:38:04.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:04.406+0000 7f8cf19b0700 1 --2- 172.21.15.96:0/2103632037 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f8cd8036880 0x7f8cd8038d30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:04.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:04.406+0000 7f8cf19b0700 1 --2- 172.21.15.96:0/2103632037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cec066560 0x7f8cec10d0b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:04.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:04.406+0000 7f8cf19b0700 1 -- 172.21.15.96:0/2103632037 >> 172.21.15.96:0/2103632037 conn(0x7f8cec0f70c0 msgr2=0x7f8cec103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:38:04.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:04.406+0000 7f8cf19b0700 1 -- 172.21.15.96:0/2103632037 shutdown_connections 2024-03-24T10:38:04.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:04.406+0000 7f8cf19b0700 1 -- 172.21.15.96:0/2103632037 wait complete. 2024-03-24T10:38:04.408 INFO:teuthology.orchestra.run.smithi096.stderr:dumped monmap epoch 1 2024-03-24T10:38:04.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:04 smithi094 ceph-mon[29446]: from='client.? 172.21.15.96:0/2103632037' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T10:38:05.946 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-24T10:38:05.947 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T10:38:05.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:05 smithi094 ceph-mon[29446]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:38:06.197 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T10:38:06.245 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T10:38:07.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:07.706+0000 7f7f4e920700 1 -- 172.21.15.96:0/3477289334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f480faea0 msgr2=0x7f7f480fb2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:38:07.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:07.706+0000 7f7f4e920700 1 --2- 172.21.15.96:0/3477289334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f480faea0 0x7f7f480fb2b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7f380048c0 tx=0x7f7f38009e30 comp rx=0 tx=0).stop 2024-03-24T10:38:07.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:07.706+0000 7f7f4e920700 1 -- 172.21.15.96:0/3477289334 shutdown_connections 2024-03-24T10:38:07.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:07.706+0000 7f7f4e920700 1 --2- 172.21.15.96:0/3477289334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f480faea0 0x7f7f480fb2b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:07.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:07.706+0000 7f7f4e920700 1 -- 172.21.15.96:0/3477289334 >> 172.21.15.96:0/3477289334 conn(0x7f7f480f6820 msgr2=0x7f7f480f8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:38:07.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:07.706+0000 7f7f4e920700 1 -- 172.21.15.96:0/3477289334 shutdown_connections 2024-03-24T10:38:07.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:07.706+0000 7f7f4e920700 1 -- 172.21.15.96:0/3477289334 wait complete. 2024-03-24T10:38:07.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:07.706+0000 7f7f4e920700 1 Processor -- start 2024-03-24T10:38:07.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:07.707+0000 7f7f4e920700 1 -- start start 2024-03-24T10:38:07.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:07.707+0000 7f7f4e920700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f480faea0 0x7f7f480fd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:38:07.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:07.707+0000 7f7f4e920700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f480fde10 con 0x7f7f480faea0 2024-03-24T10:38:07.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:07.707+0000 7f7f47fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f480faea0 0x7f7f480fd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:38:07.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:07.707+0000 7f7f47fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f480faea0 0x7f7f480fd8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.96:33512/0 (socket says 172.21.15.96:33512) 2024-03-24T10:38:07.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:07.707+0000 7f7f47fff700 1 -- 172.21.15.96:0/2680795493 learned_addr learned my addr 172.21.15.96:0/2680795493 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-24T10:38:07.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:07.707+0000 7f7f47fff700 1 -- 172.21.15.96:0/2680795493 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f38005040 con 0x7f7f480faea0 2024-03-24T10:38:07.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:07.707+0000 7f7f47fff700 1 --2- 172.21.15.96:0/2680795493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f480faea0 0x7f7f480fd8d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7f38009680 tx=0x7f7f38009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:38:07.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:07.707+0000 7f7f44ff9700 1 -- 172.21.15.96:0/2680795493 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7f3800b5c0 con 0x7f7f480faea0 2024-03-24T10:38:07.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:07.707+0000 7f7f44ff9700 1 -- 172.21.15.96:0/2680795493 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7f3800be70 con 0x7f7f480faea0 2024-03-24T10:38:07.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:07.707+0000 7f7f4e920700 1 -- 172.21.15.96:0/2680795493 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f480fbee0 con 0x7f7f480faea0 2024-03-24T10:38:07.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:07.707+0000 7f7f44ff9700 1 -- 172.21.15.96:0/2680795493 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7f380135a0 con 0x7f7f480faea0 2024-03-24T10:38:07.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:07.708+0000 7f7f4e920700 1 -- 172.21.15.96:0/2680795493 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f480fc3d0 con 0x7f7f480faea0 2024-03-24T10:38:07.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:07.708+0000 7f7f44ff9700 1 -- 172.21.15.96:0/2680795493 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f7f38013860 con 0x7f7f480faea0 2024-03-24T10:38:07.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:07.708+0000 7f7f4e920700 1 -- 172.21.15.96:0/2680795493 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f4805f5f0 con 0x7f7f480faea0 2024-03-24T10:38:07.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:07.708+0000 7f7f44ff9700 1 --2- 172.21.15.96:0/2680795493 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f7f30036880 0x7f7f30038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:38:07.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:07.708+0000 7f7f44ff9700 1 -- 172.21.15.96:0/2680795493 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f7f38049540 con 0x7f7f480faea0 2024-03-24T10:38:07.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:07.709+0000 7f7f477fe700 1 --2- 172.21.15.96:0/2680795493 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f7f30036880 0x7f7f30038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:38:07.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:07.709+0000 7f7f477fe700 1 --2- 172.21.15.96:0/2680795493 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f7f30036880 0x7f7f30038d30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c009990 tx=0x7f7f3c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:38:07.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:07.713+0000 7f7f44ff9700 1 -- 172.21.15.96:0/2680795493 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f7f380123c0 con 0x7f7f480faea0 2024-03-24T10:38:07.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:07.974+0000 7f7f4e920700 1 -- 172.21.15.96:0/2680795493 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f7f48106450 con 0x7f7f480faea0 2024-03-24T10:38:07.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:07.975+0000 7f7f44ff9700 1 -- 172.21.15.96:0/2680795493 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f7f38011370 con 0x7f7f480faea0 2024-03-24T10:38:07.976 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:38:07.976 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":1,"fsid":"386110d4-e9ca-11ee-95cd-87774f69a715","modified":"2024-03-24T10:36:06.632144Z","created":"2024-03-24T10:36:06.632144Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi094","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-24T10:38:07.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:07.977+0000 7f7f4e920700 1 -- 172.21.15.96:0/2680795493 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f7f30036880 msgr2=0x7f7f30038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:38:07.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:07.977+0000 7f7f4e920700 1 --2- 172.21.15.96:0/2680795493 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f7f30036880 0x7f7f30038d30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c009990 tx=0x7f7f3c006e40 comp rx=0 tx=0).stop 2024-03-24T10:38:07.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:07.977+0000 7f7f4e920700 1 -- 172.21.15.96:0/2680795493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f480faea0 msgr2=0x7f7f480fd8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:38:07.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:07.977+0000 7f7f4e920700 1 --2- 172.21.15.96:0/2680795493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f480faea0 0x7f7f480fd8d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7f38009680 tx=0x7f7f38009e30 comp rx=0 tx=0).stop 2024-03-24T10:38:07.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:07.977+0000 7f7f4e920700 1 -- 172.21.15.96:0/2680795493 shutdown_connections 2024-03-24T10:38:07.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:07.977+0000 7f7f4e920700 1 --2- 172.21.15.96:0/2680795493 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f7f30036880 0x7f7f30038d30 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:07.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:07.977+0000 7f7f4e920700 1 --2- 172.21.15.96:0/2680795493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f480faea0 0x7f7f480fd8d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:07.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:07.977+0000 7f7f4e920700 1 -- 172.21.15.96:0/2680795493 >> 172.21.15.96:0/2680795493 conn(0x7f7f480f6820 msgr2=0x7f7f48103bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:38:07.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:07.977+0000 7f7f4e920700 1 -- 172.21.15.96:0/2680795493 shutdown_connections 2024-03-24T10:38:07.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:07.977+0000 7f7f4e920700 1 -- 172.21.15.96:0/2680795493 wait complete. 2024-03-24T10:38:07.979 INFO:teuthology.orchestra.run.smithi096.stderr:dumped monmap epoch 1 2024-03-24T10:38:07.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:07 smithi094 ceph-mon[29446]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:38:08.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:08 smithi094 ceph-mon[29446]: from='client.? 172.21.15.96:0/2680795493' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T10:38:09.445 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-24T10:38:09.446 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T10:38:09.691 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T10:38:09.733 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T10:38:09.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:09 smithi094 ceph-mon[29446]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:38:11.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:11.384+0000 7fdf5565d700 1 -- 172.21.15.96:0/4252281177 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf500fca60 msgr2=0x7fdf500fce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:38:11.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:11.384+0000 7fdf5565d700 1 --2- 172.21.15.96:0/4252281177 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf500fca60 0x7fdf500fce70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdf400048c0 tx=0x7fdf40009e30 comp rx=0 tx=0).stop 2024-03-24T10:38:11.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:11.384+0000 7fdf5565d700 1 -- 172.21.15.96:0/4252281177 shutdown_connections 2024-03-24T10:38:11.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:11.384+0000 7fdf5565d700 1 --2- 172.21.15.96:0/4252281177 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf500fca60 0x7fdf500fce70 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:11.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:11.384+0000 7fdf5565d700 1 -- 172.21.15.96:0/4252281177 >> 172.21.15.96:0/4252281177 conn(0x7fdf500f81c0 msgr2=0x7fdf500fa5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:38:11.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:11.384+0000 7fdf5565d700 1 -- 172.21.15.96:0/4252281177 shutdown_connections 2024-03-24T10:38:11.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:11.384+0000 7fdf5565d700 1 -- 172.21.15.96:0/4252281177 wait complete. 2024-03-24T10:38:11.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:11.384+0000 7fdf5565d700 1 Processor -- start 2024-03-24T10:38:11.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:11.385+0000 7fdf5565d700 1 -- start start 2024-03-24T10:38:11.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:11.385+0000 7fdf5565d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf500fca60 0x7fdf5010d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:38:11.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:11.385+0000 7fdf5565d700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5010d5f0 con 0x7fdf500fca60 2024-03-24T10:38:11.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:11.385+0000 7fdf4effd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf500fca60 0x7fdf5010d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:38:11.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:11.385+0000 7fdf4effd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf500fca60 0x7fdf5010d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.96:33526/0 (socket says 172.21.15.96:33526) 2024-03-24T10:38:11.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:11.385+0000 7fdf4effd700 1 -- 172.21.15.96:0/3267874972 learned_addr learned my addr 172.21.15.96:0/3267874972 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-24T10:38:11.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:11.385+0000 7fdf4effd700 1 -- 172.21.15.96:0/3267874972 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf40005040 con 0x7fdf500fca60 2024-03-24T10:38:11.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:11.385+0000 7fdf4effd700 1 --2- 172.21.15.96:0/3267874972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf500fca60 0x7fdf5010d0b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdf400096b0 tx=0x7fdf40009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:38:11.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:11.386+0000 7fdf47fff700 1 -- 172.21.15.96:0/3267874972 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdf4000bed0 con 0x7fdf500fca60 2024-03-24T10:38:11.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:11.386+0000 7fdf47fff700 1 -- 172.21.15.96:0/3267874972 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdf4000b5c0 con 0x7fdf500fca60 2024-03-24T10:38:11.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:11.386+0000 7fdf5565d700 1 -- 172.21.15.96:0/3267874972 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf5010d7f0 con 0x7fdf500fca60 2024-03-24T10:38:11.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:11.386+0000 7fdf47fff700 1 -- 172.21.15.96:0/3267874972 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdf4001c5b0 con 0x7fdf500fca60 2024-03-24T10:38:11.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:11.386+0000 7fdf5565d700 1 -- 172.21.15.96:0/3267874972 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf5010e360 con 0x7fdf500fca60 2024-03-24T10:38:11.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:11.386+0000 7fdf47fff700 1 -- 172.21.15.96:0/3267874972 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fdf40013a80 con 0x7fdf500fca60 2024-03-24T10:38:11.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:11.386+0000 7fdf5565d700 1 -- 172.21.15.96:0/3267874972 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf50102f70 con 0x7fdf500fca60 2024-03-24T10:38:11.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:11.387+0000 7fdf47fff700 1 --2- 172.21.15.96:0/3267874972 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fdf3c036880 0x7fdf3c038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:38:11.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:11.387+0000 7fdf47fff700 1 -- 172.21.15.96:0/3267874972 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fdf400499d0 con 0x7fdf500fca60 2024-03-24T10:38:11.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:11.387+0000 7fdf467fc700 1 --2- 172.21.15.96:0/3267874972 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fdf3c036880 0x7fdf3c038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:38:11.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:11.387+0000 7fdf467fc700 1 --2- 172.21.15.96:0/3267874972 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fdf3c036880 0x7fdf3c038d30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fdf38009990 tx=0x7fdf38006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:38:11.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:11.392+0000 7fdf47fff700 1 -- 172.21.15.96:0/3267874972 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fdf40011550 con 0x7fdf500fca60 2024-03-24T10:38:11.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:11.651+0000 7fdf5565d700 1 -- 172.21.15.96:0/3267874972 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fdf5010ed70 con 0x7fdf500fca60 2024-03-24T10:38:11.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:11.651+0000 7fdf47fff700 1 -- 172.21.15.96:0/3267874972 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fdf40011370 con 0x7fdf500fca60 2024-03-24T10:38:11.652 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:38:11.652 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":1,"fsid":"386110d4-e9ca-11ee-95cd-87774f69a715","modified":"2024-03-24T10:36:06.632144Z","created":"2024-03-24T10:36:06.632144Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi094","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-24T10:38:11.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:11.653+0000 7fdf5565d700 1 -- 172.21.15.96:0/3267874972 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fdf3c036880 msgr2=0x7fdf3c038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:38:11.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:11.653+0000 7fdf5565d700 1 --2- 172.21.15.96:0/3267874972 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fdf3c036880 0x7fdf3c038d30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fdf38009990 tx=0x7fdf38006e40 comp rx=0 tx=0).stop 2024-03-24T10:38:11.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:11.653+0000 7fdf5565d700 1 -- 172.21.15.96:0/3267874972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf500fca60 msgr2=0x7fdf5010d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:38:11.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:11.653+0000 7fdf5565d700 1 --2- 172.21.15.96:0/3267874972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf500fca60 0x7fdf5010d0b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdf400096b0 tx=0x7fdf40009e30 comp rx=0 tx=0).stop 2024-03-24T10:38:11.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:11.653+0000 7fdf5565d700 1 -- 172.21.15.96:0/3267874972 shutdown_connections 2024-03-24T10:38:11.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:11.653+0000 7fdf5565d700 1 --2- 172.21.15.96:0/3267874972 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fdf3c036880 0x7fdf3c038d30 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:11.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:11.653+0000 7fdf5565d700 1 --2- 172.21.15.96:0/3267874972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf500fca60 0x7fdf5010d0b0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:11.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:11.653+0000 7fdf5565d700 1 -- 172.21.15.96:0/3267874972 >> 172.21.15.96:0/3267874972 conn(0x7fdf500f81c0 msgr2=0x7fdf50103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:38:11.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:11.654+0000 7fdf5565d700 1 -- 172.21.15.96:0/3267874972 shutdown_connections 2024-03-24T10:38:11.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:11.654+0000 7fdf5565d700 1 -- 172.21.15.96:0/3267874972 wait complete. 2024-03-24T10:38:11.655 INFO:teuthology.orchestra.run.smithi096.stderr:dumped monmap epoch 1 2024-03-24T10:38:11.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:11 smithi094 ceph-mon[29446]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:38:11.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:11 smithi094 ceph-mon[29446]: from='client.? 172.21.15.96:0/3267874972' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T10:38:13.208 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-24T10:38:13.208 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T10:38:13.456 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T10:38:13.502 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T10:38:13.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:13 smithi094 ceph-mon[29446]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:38:14.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:14.844+0000 7f40ce0f6700 1 -- 172.21.15.96:0/3880645175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40c80fcf30 msgr2=0x7f40c80fd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:38:14.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:14.844+0000 7f40ce0f6700 1 --2- 172.21.15.96:0/3880645175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40c80fcf30 0x7f40c80fd340 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f40b8009c60 tx=0x7f40b8004d10 comp rx=0 tx=0).stop 2024-03-24T10:38:14.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:14.844+0000 7f40ce0f6700 1 -- 172.21.15.96:0/3880645175 shutdown_connections 2024-03-24T10:38:14.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:14.844+0000 7f40ce0f6700 1 --2- 172.21.15.96:0/3880645175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40c80fcf30 0x7f40c80fd340 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:14.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:14.844+0000 7f40ce0f6700 1 -- 172.21.15.96:0/3880645175 >> 172.21.15.96:0/3880645175 conn(0x7f40c80f6820 msgr2=0x7f40c80f8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:38:14.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:14.844+0000 7f40ce0f6700 1 -- 172.21.15.96:0/3880645175 shutdown_connections 2024-03-24T10:38:14.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:14.844+0000 7f40ce0f6700 1 -- 172.21.15.96:0/3880645175 wait complete. 2024-03-24T10:38:14.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:14.845+0000 7f40ce0f6700 1 Processor -- start 2024-03-24T10:38:14.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:14.845+0000 7f40ce0f6700 1 -- start start 2024-03-24T10:38:14.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:14.845+0000 7f40ce0f6700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40c80fcf30 0x7f40c810d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:38:14.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:14.845+0000 7f40ce0f6700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c810d640 con 0x7f40c80fcf30 2024-03-24T10:38:14.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:14.845+0000 7f40c77fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40c80fcf30 0x7f40c810d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:38:14.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:14.845+0000 7f40c77fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40c80fcf30 0x7f40c810d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.96:47140/0 (socket says 172.21.15.96:47140) 2024-03-24T10:38:14.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:14.845+0000 7f40c77fe700 1 -- 172.21.15.96:0/1531611618 learned_addr learned my addr 172.21.15.96:0/1531611618 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-24T10:38:14.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:14.846+0000 7f40c77fe700 1 -- 172.21.15.96:0/1531611618 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b8009940 con 0x7f40c80fcf30 2024-03-24T10:38:14.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:14.846+0000 7f40c77fe700 1 --2- 172.21.15.96:0/1531611618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40c80fcf30 0x7f40c810d100 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f40b80097b0 tx=0x7f40b8004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:38:14.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:14.846+0000 7f40bffff700 1 -- 172.21.15.96:0/1531611618 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f40b8012410 con 0x7f40c80fcf30 2024-03-24T10:38:14.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:14.846+0000 7f40ce0f6700 1 -- 172.21.15.96:0/1531611618 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40c810d840 con 0x7f40c80fcf30 2024-03-24T10:38:14.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:14.846+0000 7f40bffff700 1 -- 172.21.15.96:0/1531611618 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f40b8012a40 con 0x7f40c80fcf30 2024-03-24T10:38:14.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:14.846+0000 7f40ce0f6700 1 -- 172.21.15.96:0/1531611618 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40c810e3b0 con 0x7f40c80fcf30 2024-03-24T10:38:14.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:14.846+0000 7f40bffff700 1 -- 172.21.15.96:0/1531611618 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f40b801b980 con 0x7f40c80fcf30 2024-03-24T10:38:14.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:14.847+0000 7f40ce0f6700 1 -- 172.21.15.96:0/1531611618 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40c8101e60 con 0x7f40c80fcf30 2024-03-24T10:38:14.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:14.847+0000 7f40bffff700 1 -- 172.21.15.96:0/1531611618 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f40b8019040 con 0x7f40c80fcf30 2024-03-24T10:38:14.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:14.847+0000 7f40bffff700 1 --2- 172.21.15.96:0/1531611618 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f40b40368d0 0x7f40b4038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:38:14.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:14.847+0000 7f40bffff700 1 -- 172.21.15.96:0/1531611618 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f40b8049c20 con 0x7f40c80fcf30 2024-03-24T10:38:14.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:14.847+0000 7f40c6ffd700 1 --2- 172.21.15.96:0/1531611618 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f40b40368d0 0x7f40b4038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:38:14.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:14.848+0000 7f40c6ffd700 1 --2- 172.21.15.96:0/1531611618 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f40b40368d0 0x7f40b4038d80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f40b0009990 tx=0x7f40b0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:38:14.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:14.852+0000 7f40bffff700 1 -- 172.21.15.96:0/1531611618 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f40b8012bb0 con 0x7f40c80fcf30 2024-03-24T10:38:15.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:15.112+0000 7f40ce0f6700 1 -- 172.21.15.96:0/1531611618 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f40c810e5f0 con 0x7f40c80fcf30 2024-03-24T10:38:15.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:15.113+0000 7f40bffff700 1 -- 172.21.15.96:0/1531611618 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f40b8017090 con 0x7f40c80fcf30 2024-03-24T10:38:15.114 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:38:15.114 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":1,"fsid":"386110d4-e9ca-11ee-95cd-87774f69a715","modified":"2024-03-24T10:36:06.632144Z","created":"2024-03-24T10:36:06.632144Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi094","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-24T10:38:15.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:15.116+0000 7f40ce0f6700 1 -- 172.21.15.96:0/1531611618 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f40b40368d0 msgr2=0x7f40b4038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:38:15.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:15.116+0000 7f40ce0f6700 1 --2- 172.21.15.96:0/1531611618 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f40b40368d0 0x7f40b4038d80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f40b0009990 tx=0x7f40b0006e40 comp rx=0 tx=0).stop 2024-03-24T10:38:15.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:15.116+0000 7f40ce0f6700 1 -- 172.21.15.96:0/1531611618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40c80fcf30 msgr2=0x7f40c810d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:38:15.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:15.116+0000 7f40ce0f6700 1 --2- 172.21.15.96:0/1531611618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40c80fcf30 0x7f40c810d100 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f40b80097b0 tx=0x7f40b8004d10 comp rx=0 tx=0).stop 2024-03-24T10:38:15.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:15.116+0000 7f40ce0f6700 1 -- 172.21.15.96:0/1531611618 shutdown_connections 2024-03-24T10:38:15.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:15.116+0000 7f40ce0f6700 1 --2- 172.21.15.96:0/1531611618 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f40b40368d0 0x7f40b4038d80 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:15.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:15.116+0000 7f40ce0f6700 1 --2- 172.21.15.96:0/1531611618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40c80fcf30 0x7f40c810d100 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:15.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:15.116+0000 7f40ce0f6700 1 -- 172.21.15.96:0/1531611618 >> 172.21.15.96:0/1531611618 conn(0x7f40c80f6820 msgr2=0x7f40c80fab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:38:15.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:15.116+0000 7f40ce0f6700 1 -- 172.21.15.96:0/1531611618 shutdown_connections 2024-03-24T10:38:15.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:15.116+0000 7f40ce0f6700 1 -- 172.21.15.96:0/1531611618 wait complete. 2024-03-24T10:38:15.117 INFO:teuthology.orchestra.run.smithi096.stderr:dumped monmap epoch 1 2024-03-24T10:38:15.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:15 smithi094 ceph-mon[29446]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:38:15.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:15 smithi094 ceph-mon[29446]: from='client.? 172.21.15.96:0/1531611618' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T10:38:16.652 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-24T10:38:16.652 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T10:38:16.896 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T10:38:16.942 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T10:38:17.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:17 smithi094 ceph-mon[29446]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:38:18.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:18.338+0000 7f14111c1700 1 -- 172.21.15.96:0/22911949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f140c0fb960 msgr2=0x7f140c0fbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:38:18.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:18.338+0000 7f14111c1700 1 --2- 172.21.15.96:0/22911949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f140c0fb960 0x7f140c0fbd70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f14000048c0 tx=0x7f1400009e30 comp rx=0 tx=0).stop 2024-03-24T10:38:18.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:18.338+0000 7f14111c1700 1 -- 172.21.15.96:0/22911949 shutdown_connections 2024-03-24T10:38:18.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:18.338+0000 7f14111c1700 1 --2- 172.21.15.96:0/22911949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f140c0fb960 0x7f140c0fbd70 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:18.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:18.338+0000 7f14111c1700 1 -- 172.21.15.96:0/22911949 >> 172.21.15.96:0/22911949 conn(0x7f140c0f70c0 msgr2=0x7f140c0f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:38:18.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:18.338+0000 7f14111c1700 1 -- 172.21.15.96:0/22911949 shutdown_connections 2024-03-24T10:38:18.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:18.338+0000 7f14111c1700 1 -- 172.21.15.96:0/22911949 wait complete. 2024-03-24T10:38:18.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:18.339+0000 7f14111c1700 1 Processor -- start 2024-03-24T10:38:18.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:18.339+0000 7f14111c1700 1 -- start start 2024-03-24T10:38:18.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:18.339+0000 7f14111c1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f140c0fb960 0x7f140c066860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:38:18.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:18.339+0000 7f14111c1700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140c10fdb0 con 0x7f140c0fb960 2024-03-24T10:38:18.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:18.339+0000 7f140ad9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f140c0fb960 0x7f140c066860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:38:18.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:18.339+0000 7f140ad9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f140c0fb960 0x7f140c066860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.96:47146/0 (socket says 172.21.15.96:47146) 2024-03-24T10:38:18.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:18.339+0000 7f140ad9d700 1 -- 172.21.15.96:0/1682321475 learned_addr learned my addr 172.21.15.96:0/1682321475 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-24T10:38:18.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:18.340+0000 7f140ad9d700 1 -- 172.21.15.96:0/1682321475 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1400005040 con 0x7f140c0fb960 2024-03-24T10:38:18.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:18.340+0000 7f140ad9d700 1 --2- 172.21.15.96:0/1682321475 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f140c0fb960 0x7f140c066860 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f14000096b0 tx=0x7f1400009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:38:18.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:18.340+0000 7f13f37fe700 1 -- 172.21.15.96:0/1682321475 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f140000bed0 con 0x7f140c0fb960 2024-03-24T10:38:18.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:18.340+0000 7f13f37fe700 1 -- 172.21.15.96:0/1682321475 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f140000b5c0 con 0x7f140c0fb960 2024-03-24T10:38:18.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:18.340+0000 7f14111c1700 1 -- 172.21.15.96:0/1682321475 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f140c10fef0 con 0x7f140c0fb960 2024-03-24T10:38:18.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:18.340+0000 7f13f37fe700 1 -- 172.21.15.96:0/1682321475 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f140001c5b0 con 0x7f140c0fb960 2024-03-24T10:38:18.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:18.340+0000 7f14111c1700 1 -- 172.21.15.96:0/1682321475 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f140c1103e0 con 0x7f140c0fb960 2024-03-24T10:38:18.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:18.341+0000 7f14111c1700 1 -- 172.21.15.96:0/1682321475 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f140c105920 con 0x7f140c0fb960 2024-03-24T10:38:18.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:18.341+0000 7f13f37fe700 1 -- 172.21.15.96:0/1682321475 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f1400013a80 con 0x7f140c0fb960 2024-03-24T10:38:18.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:18.341+0000 7f13f37fe700 1 --2- 172.21.15.96:0/1682321475 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f13f4036880 0x7f13f4038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:38:18.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:18.341+0000 7f13f37fe700 1 -- 172.21.15.96:0/1682321475 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f14000499d0 con 0x7f140c0fb960 2024-03-24T10:38:18.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:18.341+0000 7f140a59c700 1 --2- 172.21.15.96:0/1682321475 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f13f4036880 0x7f13f4038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:38:18.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:18.342+0000 7f140a59c700 1 --2- 172.21.15.96:0/1682321475 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f13f4036880 0x7f13f4038d30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f13fc009990 tx=0x7f13fc006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:38:18.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:18.346+0000 7f13f37fe700 1 -- 172.21.15.96:0/1682321475 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f1400011550 con 0x7f140c0fb960 2024-03-24T10:38:18.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:18.606+0000 7f14111c1700 1 -- 172.21.15.96:0/1682321475 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f140c05f5f0 con 0x7f140c0fb960 2024-03-24T10:38:18.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:18.607+0000 7f13f37fe700 1 -- 172.21.15.96:0/1682321475 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f1400011370 con 0x7f140c0fb960 2024-03-24T10:38:18.607 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:38:18.608 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":1,"fsid":"386110d4-e9ca-11ee-95cd-87774f69a715","modified":"2024-03-24T10:36:06.632144Z","created":"2024-03-24T10:36:06.632144Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi094","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-24T10:38:18.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:18.609+0000 7f14111c1700 1 -- 172.21.15.96:0/1682321475 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f13f4036880 msgr2=0x7f13f4038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:38:18.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:18.609+0000 7f14111c1700 1 --2- 172.21.15.96:0/1682321475 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f13f4036880 0x7f13f4038d30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f13fc009990 tx=0x7f13fc006e40 comp rx=0 tx=0).stop 2024-03-24T10:38:18.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:18.609+0000 7f14111c1700 1 -- 172.21.15.96:0/1682321475 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f140c0fb960 msgr2=0x7f140c066860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:38:18.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:18.609+0000 7f14111c1700 1 --2- 172.21.15.96:0/1682321475 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f140c0fb960 0x7f140c066860 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f14000096b0 tx=0x7f1400009e30 comp rx=0 tx=0).stop 2024-03-24T10:38:18.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:18.609+0000 7f14111c1700 1 -- 172.21.15.96:0/1682321475 shutdown_connections 2024-03-24T10:38:18.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:18.609+0000 7f14111c1700 1 --2- 172.21.15.96:0/1682321475 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f13f4036880 0x7f13f4038d30 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:18.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:18.609+0000 7f14111c1700 1 --2- 172.21.15.96:0/1682321475 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f140c0fb960 0x7f140c066860 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:18.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:18.609+0000 7f14111c1700 1 -- 172.21.15.96:0/1682321475 >> 172.21.15.96:0/1682321475 conn(0x7f140c0f70c0 msgr2=0x7f140c102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:38:18.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:18.609+0000 7f14111c1700 1 -- 172.21.15.96:0/1682321475 shutdown_connections 2024-03-24T10:38:18.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:18.609+0000 7f14111c1700 1 -- 172.21.15.96:0/1682321475 wait complete. 2024-03-24T10:38:18.610 INFO:teuthology.orchestra.run.smithi096.stderr:dumped monmap epoch 1 2024-03-24T10:38:18.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:18 smithi094 ceph-mon[29446]: from='client.? 172.21.15.96:0/1682321475' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T10:38:19.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:19 smithi094 ceph-mon[29446]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:38:20.172 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-24T10:38:20.172 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T10:38:20.418 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T10:38:20.465 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T10:38:21.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:21.850+0000 7f4b9c962700 1 -- 172.21.15.96:0/2215290361 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b980fca40 msgr2=0x7f4b980fce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:38:21.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:21.850+0000 7f4b9c962700 1 --2- 172.21.15.96:0/2215290361 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b980fca40 0x7f4b980fce50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c0048c0 tx=0x7f4b8c009e30 comp rx=0 tx=0).stop 2024-03-24T10:38:21.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:21.850+0000 7f4b9c962700 1 -- 172.21.15.96:0/2215290361 shutdown_connections 2024-03-24T10:38:21.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:21.850+0000 7f4b9c962700 1 --2- 172.21.15.96:0/2215290361 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b980fca40 0x7f4b980fce50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:21.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:21.850+0000 7f4b9c962700 1 -- 172.21.15.96:0/2215290361 >> 172.21.15.96:0/2215290361 conn(0x7f4b980f81c0 msgr2=0x7f4b980fa5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:38:21.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:21.851+0000 7f4b9c962700 1 -- 172.21.15.96:0/2215290361 shutdown_connections 2024-03-24T10:38:21.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:21.851+0000 7f4b9c962700 1 -- 172.21.15.96:0/2215290361 wait complete. 2024-03-24T10:38:21.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:21.851+0000 7f4b9c962700 1 Processor -- start 2024-03-24T10:38:21.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:21.851+0000 7f4b9c962700 1 -- start start 2024-03-24T10:38:21.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:21.851+0000 7f4b9c962700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b980fca40 0x7f4b9810d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:38:21.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:21.851+0000 7f4b9c962700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9810d5d0 con 0x7f4b980fca40 2024-03-24T10:38:21.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:21.851+0000 7f4b9659c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b980fca40 0x7f4b9810d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:38:21.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:21.851+0000 7f4b9659c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b980fca40 0x7f4b9810d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.96:47164/0 (socket says 172.21.15.96:47164) 2024-03-24T10:38:21.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:21.851+0000 7f4b9659c700 1 -- 172.21.15.96:0/3628644398 learned_addr learned my addr 172.21.15.96:0/3628644398 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-24T10:38:21.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:21.852+0000 7f4b9659c700 1 -- 172.21.15.96:0/3628644398 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b8c005040 con 0x7f4b980fca40 2024-03-24T10:38:21.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:21.852+0000 7f4b9659c700 1 --2- 172.21.15.96:0/3628644398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b980fca40 0x7f4b9810d090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c0096b0 tx=0x7f4b8c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:38:21.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:21.852+0000 7f4b86ffd700 1 -- 172.21.15.96:0/3628644398 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4b8c00bed0 con 0x7f4b980fca40 2024-03-24T10:38:21.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:21.852+0000 7f4b9c962700 1 -- 172.21.15.96:0/3628644398 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b9810d7d0 con 0x7f4b980fca40 2024-03-24T10:38:21.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:21.852+0000 7f4b86ffd700 1 -- 172.21.15.96:0/3628644398 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b8c00b5c0 con 0x7f4b980fca40 2024-03-24T10:38:21.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:21.852+0000 7f4b86ffd700 1 -- 172.21.15.96:0/3628644398 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4b8c01c5b0 con 0x7f4b980fca40 2024-03-24T10:38:21.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:21.852+0000 7f4b9c962700 1 -- 172.21.15.96:0/3628644398 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b9810e340 con 0x7f4b980fca40 2024-03-24T10:38:21.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:21.853+0000 7f4b86ffd700 1 -- 172.21.15.96:0/3628644398 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f4b8c013a80 con 0x7f4b980fca40 2024-03-24T10:38:21.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:21.853+0000 7f4b9c962700 1 -- 172.21.15.96:0/3628644398 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b98102f50 con 0x7f4b980fca40 2024-03-24T10:38:21.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:21.853+0000 7f4b86ffd700 1 --2- 172.21.15.96:0/3628644398 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f4b80036880 0x7f4b80038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:38:21.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:21.853+0000 7f4b86ffd700 1 -- 172.21.15.96:0/3628644398 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f4b8c0499d0 con 0x7f4b980fca40 2024-03-24T10:38:21.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:21.853+0000 7f4b95d9b700 1 --2- 172.21.15.96:0/3628644398 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f4b80036880 0x7f4b80038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:38:21.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:21.854+0000 7f4b95d9b700 1 --2- 172.21.15.96:0/3628644398 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f4b80036880 0x7f4b80038d30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4b88009990 tx=0x7f4b88006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:38:21.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:21.859+0000 7f4b86ffd700 1 -- 172.21.15.96:0/3628644398 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f4b8c011550 con 0x7f4b980fca40 2024-03-24T10:38:21.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:21 smithi094 ceph-mon[29446]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:38:22.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:22.117+0000 7f4b9c962700 1 -- 172.21.15.96:0/3628644398 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4b9810ed50 con 0x7f4b980fca40 2024-03-24T10:38:22.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:22.118+0000 7f4b86ffd700 1 -- 172.21.15.96:0/3628644398 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f4b8c011370 con 0x7f4b980fca40 2024-03-24T10:38:22.118 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:38:22.118 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":1,"fsid":"386110d4-e9ca-11ee-95cd-87774f69a715","modified":"2024-03-24T10:36:06.632144Z","created":"2024-03-24T10:36:06.632144Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi094","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-24T10:38:22.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:22.119+0000 7f4b9c962700 1 -- 172.21.15.96:0/3628644398 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f4b80036880 msgr2=0x7f4b80038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:38:22.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:22.119+0000 7f4b9c962700 1 --2- 172.21.15.96:0/3628644398 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f4b80036880 0x7f4b80038d30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4b88009990 tx=0x7f4b88006e40 comp rx=0 tx=0).stop 2024-03-24T10:38:22.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:22.119+0000 7f4b9c962700 1 -- 172.21.15.96:0/3628644398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b980fca40 msgr2=0x7f4b9810d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:38:22.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:22.119+0000 7f4b9c962700 1 --2- 172.21.15.96:0/3628644398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b980fca40 0x7f4b9810d090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c0096b0 tx=0x7f4b8c009e30 comp rx=0 tx=0).stop 2024-03-24T10:38:22.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:22.120+0000 7f4b9c962700 1 -- 172.21.15.96:0/3628644398 shutdown_connections 2024-03-24T10:38:22.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:22.120+0000 7f4b9c962700 1 --2- 172.21.15.96:0/3628644398 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f4b80036880 0x7f4b80038d30 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:22.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:22.120+0000 7f4b9c962700 1 --2- 172.21.15.96:0/3628644398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b980fca40 0x7f4b9810d090 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:22.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:22.120+0000 7f4b9c962700 1 -- 172.21.15.96:0/3628644398 >> 172.21.15.96:0/3628644398 conn(0x7f4b980f81c0 msgr2=0x7f4b98103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:38:22.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:22.120+0000 7f4b9c962700 1 -- 172.21.15.96:0/3628644398 shutdown_connections 2024-03-24T10:38:22.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:22.120+0000 7f4b9c962700 1 -- 172.21.15.96:0/3628644398 wait complete. 2024-03-24T10:38:22.121 INFO:teuthology.orchestra.run.smithi096.stderr:dumped monmap epoch 1 2024-03-24T10:38:22.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:22 smithi094 ceph-mon[29446]: from='client.? 172.21.15.96:0/3628644398' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T10:38:23.755 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-24T10:38:23.755 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T10:38:23.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:23 smithi094 ceph-mon[29446]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:38:24.004 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T10:38:24.051 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T10:38:24.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:24 smithi094 ceph-mon[29446]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:38:25.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:25.410+0000 7fcce9d17700 1 -- 172.21.15.96:0/3744333921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcce40fca60 msgr2=0x7fcce40fce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:38:25.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:25.410+0000 7fcce9d17700 1 --2- 172.21.15.96:0/3744333921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcce40fca60 0x7fcce40fce70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fccd40048c0 tx=0x7fccd4009e30 comp rx=0 tx=0).stop 2024-03-24T10:38:25.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:25.410+0000 7fcce9d17700 1 -- 172.21.15.96:0/3744333921 shutdown_connections 2024-03-24T10:38:25.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:25.410+0000 7fcce9d17700 1 --2- 172.21.15.96:0/3744333921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcce40fca60 0x7fcce40fce70 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:25.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:25.410+0000 7fcce9d17700 1 -- 172.21.15.96:0/3744333921 >> 172.21.15.96:0/3744333921 conn(0x7fcce40f81c0 msgr2=0x7fcce40fa5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:38:25.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:25.410+0000 7fcce9d17700 1 -- 172.21.15.96:0/3744333921 shutdown_connections 2024-03-24T10:38:25.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:25.410+0000 7fcce9d17700 1 -- 172.21.15.96:0/3744333921 wait complete. 2024-03-24T10:38:25.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:25.411+0000 7fcce9d17700 1 Processor -- start 2024-03-24T10:38:25.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:25.411+0000 7fcce9d17700 1 -- start start 2024-03-24T10:38:25.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:25.411+0000 7fcce9d17700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcce40fca60 0x7fcce410d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:38:25.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:25.411+0000 7fcce9d17700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce410d5d0 con 0x7fcce40fca60 2024-03-24T10:38:25.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:25.412+0000 7fcce37fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcce40fca60 0x7fcce410d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:38:25.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:25.412+0000 7fcce37fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcce40fca60 0x7fcce410d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.96:49936/0 (socket says 172.21.15.96:49936) 2024-03-24T10:38:25.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:25.412+0000 7fcce37fe700 1 -- 172.21.15.96:0/3207824821 learned_addr learned my addr 172.21.15.96:0/3207824821 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-24T10:38:25.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:25.412+0000 7fcce37fe700 1 -- 172.21.15.96:0/3207824821 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd4005040 con 0x7fcce40fca60 2024-03-24T10:38:25.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:25.412+0000 7fcce37fe700 1 --2- 172.21.15.96:0/3207824821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcce40fca60 0x7fcce410d090 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fccd40096b0 tx=0x7fccd4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:38:25.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:25.412+0000 7fcccbfff700 1 -- 172.21.15.96:0/3207824821 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fccd400bed0 con 0x7fcce40fca60 2024-03-24T10:38:25.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:25.412+0000 7fcce9d17700 1 -- 172.21.15.96:0/3207824821 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcce410d7d0 con 0x7fcce40fca60 2024-03-24T10:38:25.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:25.412+0000 7fcccbfff700 1 -- 172.21.15.96:0/3207824821 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fccd400b5c0 con 0x7fcce40fca60 2024-03-24T10:38:25.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:25.412+0000 7fcccbfff700 1 -- 172.21.15.96:0/3207824821 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fccd401c5b0 con 0x7fcce40fca60 2024-03-24T10:38:25.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:25.412+0000 7fcce9d17700 1 -- 172.21.15.96:0/3207824821 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcce410e340 con 0x7fcce40fca60 2024-03-24T10:38:25.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:25.413+0000 7fcccbfff700 1 -- 172.21.15.96:0/3207824821 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fccd401c820 con 0x7fcce40fca60 2024-03-24T10:38:25.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:25.413+0000 7fcccbfff700 1 --2- 172.21.15.96:0/3207824821 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fcccc036880 0x7fcccc038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:38:25.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:25.413+0000 7fcccbfff700 1 -- 172.21.15.96:0/3207824821 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fccd4049b00 con 0x7fcce40fca60 2024-03-24T10:38:25.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:25.413+0000 7fcce9d17700 1 -- 172.21.15.96:0/3207824821 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcce4102f70 con 0x7fcce40fca60 2024-03-24T10:38:25.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:25.413+0000 7fcce2ffd700 1 --2- 172.21.15.96:0/3207824821 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fcccc036880 0x7fcccc038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:38:25.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:25.414+0000 7fcce2ffd700 1 --2- 172.21.15.96:0/3207824821 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fcccc036880 0x7fcccc038d30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fccd8009990 tx=0x7fccd8006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:38:25.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:25.418+0000 7fcccbfff700 1 -- 172.21.15.96:0/3207824821 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fccd4023300 con 0x7fcce40fca60 2024-03-24T10:38:25.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:25.679+0000 7fcce9d17700 1 -- 172.21.15.96:0/3207824821 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fcce41028a0 con 0x7fcce40fca60 2024-03-24T10:38:25.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:25.680+0000 7fcccbfff700 1 -- 172.21.15.96:0/3207824821 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fccd4011b60 con 0x7fcce40fca60 2024-03-24T10:38:25.681 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:38:25.681 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":1,"fsid":"386110d4-e9ca-11ee-95cd-87774f69a715","modified":"2024-03-24T10:36:06.632144Z","created":"2024-03-24T10:36:06.632144Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi094","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-24T10:38:25.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:25.682+0000 7fcce9d17700 1 -- 172.21.15.96:0/3207824821 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fcccc036880 msgr2=0x7fcccc038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:38:25.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:25.682+0000 7fcce9d17700 1 --2- 172.21.15.96:0/3207824821 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fcccc036880 0x7fcccc038d30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fccd8009990 tx=0x7fccd8006e40 comp rx=0 tx=0).stop 2024-03-24T10:38:25.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:25.682+0000 7fcce9d17700 1 -- 172.21.15.96:0/3207824821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcce40fca60 msgr2=0x7fcce410d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:38:25.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:25.682+0000 7fcce9d17700 1 --2- 172.21.15.96:0/3207824821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcce40fca60 0x7fcce410d090 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fccd40096b0 tx=0x7fccd4009e30 comp rx=0 tx=0).stop 2024-03-24T10:38:25.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:25.682+0000 7fcce9d17700 1 -- 172.21.15.96:0/3207824821 shutdown_connections 2024-03-24T10:38:25.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:25.682+0000 7fcce9d17700 1 --2- 172.21.15.96:0/3207824821 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fcccc036880 0x7fcccc038d30 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:25.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:25.682+0000 7fcce9d17700 1 --2- 172.21.15.96:0/3207824821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcce40fca60 0x7fcce410d090 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:25.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:25.682+0000 7fcce9d17700 1 -- 172.21.15.96:0/3207824821 >> 172.21.15.96:0/3207824821 conn(0x7fcce40f81c0 msgr2=0x7fcce4103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:38:25.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:25.682+0000 7fcce9d17700 1 -- 172.21.15.96:0/3207824821 shutdown_connections 2024-03-24T10:38:25.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:25.682+0000 7fcce9d17700 1 -- 172.21.15.96:0/3207824821 wait complete. 2024-03-24T10:38:25.683 INFO:teuthology.orchestra.run.smithi096.stderr:dumped monmap epoch 1 2024-03-24T10:38:25.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:25 smithi094 ceph-mon[29446]: from='client.? 172.21.15.96:0/3207824821' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T10:38:26.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:26 smithi094 ceph-mon[29446]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:38:27.168 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-24T10:38:27.169 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T10:38:27.419 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T10:38:27.465 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T10:38:28.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:28.873+0000 7f8579631700 1 -- 172.21.15.96:0/1204333168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85740fca40 msgr2=0x7f85740fce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:38:28.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:28.873+0000 7f8579631700 1 --2- 172.21.15.96:0/1204333168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85740fca40 0x7f85740fce50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f856c009c60 tx=0x7f856c004d10 comp rx=0 tx=0).stop 2024-03-24T10:38:28.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:28.873+0000 7f8579631700 1 -- 172.21.15.96:0/1204333168 shutdown_connections 2024-03-24T10:38:28.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:28.873+0000 7f8579631700 1 --2- 172.21.15.96:0/1204333168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85740fca40 0x7f85740fce50 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:28.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:28.873+0000 7f8579631700 1 -- 172.21.15.96:0/1204333168 >> 172.21.15.96:0/1204333168 conn(0x7f85740f81c0 msgr2=0x7f85740fa5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:38:28.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:28.873+0000 7f8579631700 1 -- 172.21.15.96:0/1204333168 shutdown_connections 2024-03-24T10:38:28.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:28.873+0000 7f8579631700 1 -- 172.21.15.96:0/1204333168 wait complete. 2024-03-24T10:38:28.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:28.874+0000 7f8579631700 1 Processor -- start 2024-03-24T10:38:28.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:28.874+0000 7f8579631700 1 -- start start 2024-03-24T10:38:28.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:28.874+0000 7f8579631700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85740fca40 0x7f857410d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:38:28.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:28.874+0000 7f8579631700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f857410d5d0 con 0x7f85740fca40 2024-03-24T10:38:28.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:28.874+0000 7f8572ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85740fca40 0x7f857410d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:38:28.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:28.874+0000 7f8572ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85740fca40 0x7f857410d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.96:49948/0 (socket says 172.21.15.96:49948) 2024-03-24T10:38:28.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:28.874+0000 7f8572ffd700 1 -- 172.21.15.96:0/4087033222 learned_addr learned my addr 172.21.15.96:0/4087033222 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-24T10:38:28.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:28.874+0000 7f8572ffd700 1 -- 172.21.15.96:0/4087033222 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f856c009940 con 0x7f85740fca40 2024-03-24T10:38:28.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:28.875+0000 7f8572ffd700 1 --2- 172.21.15.96:0/4087033222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85740fca40 0x7f857410d090 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f856c009500 tx=0x7f856c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:38:28.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:28.875+0000 7f855b7fe700 1 -- 172.21.15.96:0/4087033222 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f856c007c80 con 0x7f85740fca40 2024-03-24T10:38:28.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:28.875+0000 7f855b7fe700 1 -- 172.21.15.96:0/4087033222 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f856c01d910 con 0x7f85740fca40 2024-03-24T10:38:28.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:28.875+0000 7f8579631700 1 -- 172.21.15.96:0/4087033222 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f857410d7d0 con 0x7f85740fca40 2024-03-24T10:38:28.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:28.875+0000 7f855b7fe700 1 -- 172.21.15.96:0/4087033222 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f856c0129d0 con 0x7f85740fca40 2024-03-24T10:38:28.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:28.875+0000 7f8579631700 1 -- 172.21.15.96:0/4087033222 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f857410e340 con 0x7f85740fca40 2024-03-24T10:38:28.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:28.876+0000 7f855b7fe700 1 -- 172.21.15.96:0/4087033222 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f856c019040 con 0x7f85740fca40 2024-03-24T10:38:28.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:28.876+0000 7f8579631700 1 -- 172.21.15.96:0/4087033222 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8574102f50 con 0x7f85740fca40 2024-03-24T10:38:28.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:28.876+0000 7f855b7fe700 1 --2- 172.21.15.96:0/4087033222 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f855c0368d0 0x7f855c038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:38:28.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:28.876+0000 7f855b7fe700 1 -- 172.21.15.96:0/4087033222 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f856c04a3b0 con 0x7f85740fca40 2024-03-24T10:38:28.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:28.876+0000 7f85727fc700 1 --2- 172.21.15.96:0/4087033222 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f855c0368d0 0x7f855c038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:38:28.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:28.876+0000 7f85727fc700 1 --2- 172.21.15.96:0/4087033222 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f855c0368d0 0x7f855c038d80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8568009990 tx=0x7f8568006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:38:28.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:28.881+0000 7f855b7fe700 1 -- 172.21.15.96:0/4087033222 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f856c024310 con 0x7f85740fca40 2024-03-24T10:38:29.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:29.140+0000 7f8579631700 1 -- 172.21.15.96:0/4087033222 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f857410eca0 con 0x7f85740fca40 2024-03-24T10:38:29.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:29.140+0000 7f855b7fe700 1 -- 172.21.15.96:0/4087033222 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f856c017020 con 0x7f85740fca40 2024-03-24T10:38:29.141 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:38:29.141 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":1,"fsid":"386110d4-e9ca-11ee-95cd-87774f69a715","modified":"2024-03-24T10:36:06.632144Z","created":"2024-03-24T10:36:06.632144Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi094","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-24T10:38:29.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:29.143+0000 7f8579631700 1 -- 172.21.15.96:0/4087033222 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f855c0368d0 msgr2=0x7f855c038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:38:29.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:29.143+0000 7f8579631700 1 --2- 172.21.15.96:0/4087033222 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f855c0368d0 0x7f855c038d80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8568009990 tx=0x7f8568006e40 comp rx=0 tx=0).stop 2024-03-24T10:38:29.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:29.143+0000 7f8579631700 1 -- 172.21.15.96:0/4087033222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85740fca40 msgr2=0x7f857410d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:38:29.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:29.143+0000 7f8579631700 1 --2- 172.21.15.96:0/4087033222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85740fca40 0x7f857410d090 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f856c009500 tx=0x7f856c004d10 comp rx=0 tx=0).stop 2024-03-24T10:38:29.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:29.143+0000 7f8579631700 1 -- 172.21.15.96:0/4087033222 shutdown_connections 2024-03-24T10:38:29.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:29.143+0000 7f8579631700 1 --2- 172.21.15.96:0/4087033222 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f855c0368d0 0x7f855c038d80 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:29.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:29.143+0000 7f8579631700 1 --2- 172.21.15.96:0/4087033222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85740fca40 0x7f857410d090 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:29.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:29.143+0000 7f8579631700 1 -- 172.21.15.96:0/4087033222 >> 172.21.15.96:0/4087033222 conn(0x7f85740f81c0 msgr2=0x7f8574103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:38:29.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:29.143+0000 7f8579631700 1 -- 172.21.15.96:0/4087033222 shutdown_connections 2024-03-24T10:38:29.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:29.143+0000 7f8579631700 1 -- 172.21.15.96:0/4087033222 wait complete. 2024-03-24T10:38:29.144 INFO:teuthology.orchestra.run.smithi096.stderr:dumped monmap epoch 1 2024-03-24T10:38:29.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:29 smithi094 ceph-mon[29446]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:38:29.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:29 smithi094 ceph-mon[29446]: from='client.? 172.21.15.96:0/4087033222' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T10:38:30.714 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-24T10:38:30.715 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T10:38:30.957 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T10:38:31.001 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T10:38:31.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:31 smithi094 ceph-mon[29446]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:38:32.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:32.451+0000 7fd97018a700 1 -- 172.21.15.96:0/4042858540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9680fca60 msgr2=0x7fd9680fce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:38:32.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:32.451+0000 7fd97018a700 1 --2- 172.21.15.96:0/4042858540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9680fca60 0x7fd9680fce70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd9580048c0 tx=0x7fd958009e30 comp rx=0 tx=0).stop 2024-03-24T10:38:32.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:32.451+0000 7fd97018a700 1 -- 172.21.15.96:0/4042858540 shutdown_connections 2024-03-24T10:38:32.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:32.451+0000 7fd97018a700 1 --2- 172.21.15.96:0/4042858540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9680fca60 0x7fd9680fce70 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:32.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:32.451+0000 7fd97018a700 1 -- 172.21.15.96:0/4042858540 >> 172.21.15.96:0/4042858540 conn(0x7fd9680f81c0 msgr2=0x7fd9680fa5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:38:32.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:32.451+0000 7fd97018a700 1 -- 172.21.15.96:0/4042858540 shutdown_connections 2024-03-24T10:38:32.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:32.451+0000 7fd97018a700 1 -- 172.21.15.96:0/4042858540 wait complete. 2024-03-24T10:38:32.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:32.451+0000 7fd97018a700 1 Processor -- start 2024-03-24T10:38:32.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:32.451+0000 7fd97018a700 1 -- start start 2024-03-24T10:38:32.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:32.451+0000 7fd97018a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9680fca60 0x7fd96810d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:38:32.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:32.451+0000 7fd97018a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96810d5d0 con 0x7fd9680fca60 2024-03-24T10:38:32.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:32.452+0000 7fd96df26700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9680fca60 0x7fd96810d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:38:32.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:32.452+0000 7fd96df26700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9680fca60 0x7fd96810d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.96:49962/0 (socket says 172.21.15.96:49962) 2024-03-24T10:38:32.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:32.452+0000 7fd96df26700 1 -- 172.21.15.96:0/1878243260 learned_addr learned my addr 172.21.15.96:0/1878243260 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-24T10:38:32.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:32.452+0000 7fd96df26700 1 -- 172.21.15.96:0/1878243260 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd958005040 con 0x7fd9680fca60 2024-03-24T10:38:32.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:32.452+0000 7fd96df26700 1 --2- 172.21.15.96:0/1878243260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9680fca60 0x7fd96810d090 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd9580096b0 tx=0x7fd958009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:38:32.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:32.452+0000 7fd95e7fc700 1 -- 172.21.15.96:0/1878243260 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd95800bed0 con 0x7fd9680fca60 2024-03-24T10:38:32.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:32.452+0000 7fd95e7fc700 1 -- 172.21.15.96:0/1878243260 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd95800b5c0 con 0x7fd9680fca60 2024-03-24T10:38:32.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:32.452+0000 7fd97018a700 1 -- 172.21.15.96:0/1878243260 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd96810d7d0 con 0x7fd9680fca60 2024-03-24T10:38:32.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:32.452+0000 7fd95e7fc700 1 -- 172.21.15.96:0/1878243260 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd95801c5b0 con 0x7fd9680fca60 2024-03-24T10:38:32.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:32.452+0000 7fd97018a700 1 -- 172.21.15.96:0/1878243260 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd96810e340 con 0x7fd9680fca60 2024-03-24T10:38:32.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:32.453+0000 7fd95e7fc700 1 -- 172.21.15.96:0/1878243260 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fd95801c820 con 0x7fd9680fca60 2024-03-24T10:38:32.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:32.453+0000 7fd97018a700 1 -- 172.21.15.96:0/1878243260 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd968102f70 con 0x7fd9680fca60 2024-03-24T10:38:32.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:32.453+0000 7fd95e7fc700 1 --2- 172.21.15.96:0/1878243260 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fd954036880 0x7fd954038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:38:32.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:32.453+0000 7fd95e7fc700 1 -- 172.21.15.96:0/1878243260 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fd958049d20 con 0x7fd9680fca60 2024-03-24T10:38:32.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:32.453+0000 7fd96d725700 1 --2- 172.21.15.96:0/1878243260 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fd954036880 0x7fd954038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:38:32.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:32.454+0000 7fd96d725700 1 --2- 172.21.15.96:0/1878243260 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fd954036880 0x7fd954038d30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd964009990 tx=0x7fd964006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:38:32.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:32.458+0000 7fd95e7fc700 1 -- 172.21.15.96:0/1878243260 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fd958023300 con 0x7fd9680fca60 2024-03-24T10:38:32.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:32.720+0000 7fd97018a700 1 -- 172.21.15.96:0/1878243260 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd96810ed50 con 0x7fd9680fca60 2024-03-24T10:38:32.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:32.720+0000 7fd95e7fc700 1 -- 172.21.15.96:0/1878243260 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fd958011b60 con 0x7fd9680fca60 2024-03-24T10:38:32.721 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:38:32.721 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":1,"fsid":"386110d4-e9ca-11ee-95cd-87774f69a715","modified":"2024-03-24T10:36:06.632144Z","created":"2024-03-24T10:36:06.632144Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi094","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-24T10:38:32.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:32.722+0000 7fd97018a700 1 -- 172.21.15.96:0/1878243260 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fd954036880 msgr2=0x7fd954038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:38:32.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:32.722+0000 7fd97018a700 1 --2- 172.21.15.96:0/1878243260 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fd954036880 0x7fd954038d30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd964009990 tx=0x7fd964006e40 comp rx=0 tx=0).stop 2024-03-24T10:38:32.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:32.722+0000 7fd97018a700 1 -- 172.21.15.96:0/1878243260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9680fca60 msgr2=0x7fd96810d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:38:32.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:32.722+0000 7fd97018a700 1 --2- 172.21.15.96:0/1878243260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9680fca60 0x7fd96810d090 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd9580096b0 tx=0x7fd958009e30 comp rx=0 tx=0).stop 2024-03-24T10:38:32.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:32.722+0000 7fd97018a700 1 -- 172.21.15.96:0/1878243260 shutdown_connections 2024-03-24T10:38:32.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:32.722+0000 7fd97018a700 1 --2- 172.21.15.96:0/1878243260 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fd954036880 0x7fd954038d30 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:32.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:32.722+0000 7fd97018a700 1 --2- 172.21.15.96:0/1878243260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9680fca60 0x7fd96810d090 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:32.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:32.722+0000 7fd97018a700 1 -- 172.21.15.96:0/1878243260 >> 172.21.15.96:0/1878243260 conn(0x7fd9680f81c0 msgr2=0x7fd968103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:38:32.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:32.722+0000 7fd97018a700 1 -- 172.21.15.96:0/1878243260 shutdown_connections 2024-03-24T10:38:32.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:32.722+0000 7fd97018a700 1 -- 172.21.15.96:0/1878243260 wait complete. 2024-03-24T10:38:32.724 INFO:teuthology.orchestra.run.smithi096.stderr:dumped monmap epoch 1 2024-03-24T10:38:33.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:33 smithi094 ceph-mon[29446]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:38:33.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:33 smithi094 ceph-mon[29446]: from='client.? 172.21.15.96:0/1878243260' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T10:38:34.308 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-24T10:38:34.308 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T10:38:34.560 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T10:38:34.607 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T10:38:35.549 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:35 smithi094 ceph-mon[29446]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:38:36.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:36.270+0000 7efc52284700 1 -- 172.21.15.96:0/3415082689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc4c0669e0 msgr2=0x7efc4c0ff650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:38:36.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:36.270+0000 7efc52284700 1 --2- 172.21.15.96:0/3415082689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc4c0669e0 0x7efc4c0ff650 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efc3c0048c0 tx=0x7efc3c009e30 comp rx=0 tx=0).stop 2024-03-24T10:38:36.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:36.271+0000 7efc52284700 1 -- 172.21.15.96:0/3415082689 shutdown_connections 2024-03-24T10:38:36.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:36.271+0000 7efc52284700 1 --2- 172.21.15.96:0/3415082689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc4c0669e0 0x7efc4c0ff650 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:36.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:36.271+0000 7efc52284700 1 -- 172.21.15.96:0/3415082689 >> 172.21.15.96:0/3415082689 conn(0x7efc4c0f70c0 msgr2=0x7efc4c0f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:38:36.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:36.271+0000 7efc52284700 1 -- 172.21.15.96:0/3415082689 shutdown_connections 2024-03-24T10:38:36.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:36.271+0000 7efc52284700 1 -- 172.21.15.96:0/3415082689 wait complete. 2024-03-24T10:38:36.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:36.271+0000 7efc52284700 1 Processor -- start 2024-03-24T10:38:36.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:36.271+0000 7efc52284700 1 -- start start 2024-03-24T10:38:36.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:36.271+0000 7efc52284700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc4c0669e0 0x7efc4c10d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:38:36.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:36.271+0000 7efc52284700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4c10d630 con 0x7efc4c0669e0 2024-03-24T10:38:36.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:36.272+0000 7efc4bfff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc4c0669e0 0x7efc4c10d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:38:36.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:36.272+0000 7efc4bfff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc4c0669e0 0x7efc4c10d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.96:54566/0 (socket says 172.21.15.96:54566) 2024-03-24T10:38:36.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:36.272+0000 7efc4bfff700 1 -- 172.21.15.96:0/548227238 learned_addr learned my addr 172.21.15.96:0/548227238 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-24T10:38:36.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:36.272+0000 7efc4bfff700 1 -- 172.21.15.96:0/548227238 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc3c005040 con 0x7efc4c0669e0 2024-03-24T10:38:36.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:36.272+0000 7efc4bfff700 1 --2- 172.21.15.96:0/548227238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc4c0669e0 0x7efc4c10d0f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7efc3c00b900 tx=0x7efc3c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:38:36.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:36.272+0000 7efc48ff9700 1 -- 172.21.15.96:0/548227238 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efc3c00b5c0 con 0x7efc4c0669e0 2024-03-24T10:38:36.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:36.272+0000 7efc52284700 1 -- 172.21.15.96:0/548227238 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc4c10d830 con 0x7efc4c0669e0 2024-03-24T10:38:36.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:36.272+0000 7efc48ff9700 1 -- 172.21.15.96:0/548227238 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efc3c00be70 con 0x7efc4c0669e0 2024-03-24T10:38:36.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:36.272+0000 7efc48ff9700 1 -- 172.21.15.96:0/548227238 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efc3c01c590 con 0x7efc4c0669e0 2024-03-24T10:38:36.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:36.272+0000 7efc52284700 1 -- 172.21.15.96:0/548227238 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc4c10e3a0 con 0x7efc4c0669e0 2024-03-24T10:38:36.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:36.273+0000 7efc48ff9700 1 -- 172.21.15.96:0/548227238 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7efc3c01c850 con 0x7efc4c0669e0 2024-03-24T10:38:36.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:36.273+0000 7efc48ff9700 1 --2- 172.21.15.96:0/548227238 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7efc34036880 0x7efc34038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:38:36.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:36.273+0000 7efc48ff9700 1 -- 172.21.15.96:0/548227238 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7efc3c049440 con 0x7efc4c0669e0 2024-03-24T10:38:36.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:36.273+0000 7efc52284700 1 -- 172.21.15.96:0/548227238 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc4c102f60 con 0x7efc4c0669e0 2024-03-24T10:38:36.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:36.274+0000 7efc4b7fe700 1 --2- 172.21.15.96:0/548227238 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7efc34036880 0x7efc34038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:38:36.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:36.274+0000 7efc4b7fe700 1 --2- 172.21.15.96:0/548227238 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7efc34036880 0x7efc34038d30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7efc40009990 tx=0x7efc40006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:38:36.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:36.279+0000 7efc48ff9700 1 -- 172.21.15.96:0/548227238 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7efc3c021070 con 0x7efc4c0669e0 2024-03-24T10:38:36.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:36.540+0000 7efc52284700 1 -- 172.21.15.96:0/548227238 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7efc4c05f5f0 con 0x7efc4c0669e0 2024-03-24T10:38:36.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:36.540+0000 7efc48ff9700 1 -- 172.21.15.96:0/548227238 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7efc4c05f5f0 con 0x7efc4c0669e0 2024-03-24T10:38:36.541 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:38:36.541 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":1,"fsid":"386110d4-e9ca-11ee-95cd-87774f69a715","modified":"2024-03-24T10:36:06.632144Z","created":"2024-03-24T10:36:06.632144Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi094","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-24T10:38:36.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:36.542+0000 7efc52284700 1 -- 172.21.15.96:0/548227238 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7efc34036880 msgr2=0x7efc34038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:38:36.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:36.542+0000 7efc52284700 1 --2- 172.21.15.96:0/548227238 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7efc34036880 0x7efc34038d30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7efc40009990 tx=0x7efc40006e40 comp rx=0 tx=0).stop 2024-03-24T10:38:36.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:36.542+0000 7efc52284700 1 -- 172.21.15.96:0/548227238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc4c0669e0 msgr2=0x7efc4c10d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:38:36.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:36.542+0000 7efc52284700 1 --2- 172.21.15.96:0/548227238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc4c0669e0 0x7efc4c10d0f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7efc3c00b900 tx=0x7efc3c009e30 comp rx=0 tx=0).stop 2024-03-24T10:38:36.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:36.542+0000 7efc52284700 1 -- 172.21.15.96:0/548227238 shutdown_connections 2024-03-24T10:38:36.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:36.542+0000 7efc52284700 1 --2- 172.21.15.96:0/548227238 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7efc34036880 0x7efc34038d30 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:36.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:36.542+0000 7efc52284700 1 --2- 172.21.15.96:0/548227238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc4c0669e0 0x7efc4c10d0f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:36.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:36.542+0000 7efc52284700 1 -- 172.21.15.96:0/548227238 >> 172.21.15.96:0/548227238 conn(0x7efc4c0f70c0 msgr2=0x7efc4c103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:38:36.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:36.542+0000 7efc52284700 1 -- 172.21.15.96:0/548227238 shutdown_connections 2024-03-24T10:38:36.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:36.542+0000 7efc52284700 1 -- 172.21.15.96:0/548227238 wait complete. 2024-03-24T10:38:36.544 INFO:teuthology.orchestra.run.smithi096.stderr:dumped monmap epoch 1 2024-03-24T10:38:37.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:37 smithi094 ceph-mon[29446]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:38:37.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:37 smithi094 ceph-mon[29446]: from='client.? 172.21.15.96:0/548227238' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T10:38:38.094 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-24T10:38:38.094 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T10:38:38.341 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T10:38:38.386 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T10:38:39.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:39 smithi094 ceph-mon[29446]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:38:39.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:39.839+0000 7fb80e136700 1 -- 172.21.15.96:0/1095072389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8080fca50 msgr2=0x7fb8080fce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:38:39.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:39.839+0000 7fb80e136700 1 --2- 172.21.15.96:0/1095072389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8080fca50 0x7fb8080fce60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb7f80048c0 tx=0x7fb7f8009e30 comp rx=0 tx=0).stop 2024-03-24T10:38:39.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:39.839+0000 7fb80e136700 1 -- 172.21.15.96:0/1095072389 shutdown_connections 2024-03-24T10:38:39.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:39.839+0000 7fb80e136700 1 --2- 172.21.15.96:0/1095072389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8080fca50 0x7fb8080fce60 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:39.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:39.839+0000 7fb80e136700 1 -- 172.21.15.96:0/1095072389 >> 172.21.15.96:0/1095072389 conn(0x7fb8080f8190 msgr2=0x7fb8080fa5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:38:39.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:39.839+0000 7fb80e136700 1 -- 172.21.15.96:0/1095072389 shutdown_connections 2024-03-24T10:38:39.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:39.839+0000 7fb80e136700 1 -- 172.21.15.96:0/1095072389 wait complete. 2024-03-24T10:38:39.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:39.839+0000 7fb80e136700 1 Processor -- start 2024-03-24T10:38:39.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:39.840+0000 7fb80e136700 1 -- start start 2024-03-24T10:38:39.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:39.840+0000 7fb80e136700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8080fca50 0x7fb80810d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:38:39.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:39.840+0000 7fb80e136700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80810d5f0 con 0x7fb8080fca50 2024-03-24T10:38:39.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:39.840+0000 7fb8077fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8080fca50 0x7fb80810d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:38:39.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:39.840+0000 7fb8077fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8080fca50 0x7fb80810d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.96:54582/0 (socket says 172.21.15.96:54582) 2024-03-24T10:38:39.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:39.840+0000 7fb8077fe700 1 -- 172.21.15.96:0/1586464824 learned_addr learned my addr 172.21.15.96:0/1586464824 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-24T10:38:39.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:39.840+0000 7fb8077fe700 1 -- 172.21.15.96:0/1586464824 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f8005040 con 0x7fb8080fca50 2024-03-24T10:38:39.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:39.840+0000 7fb8077fe700 1 --2- 172.21.15.96:0/1586464824 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8080fca50 0x7fb80810d0b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb7f80096b0 tx=0x7fb7f8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:38:39.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:39.841+0000 7fb7effff700 1 -- 172.21.15.96:0/1586464824 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb7f800bed0 con 0x7fb8080fca50 2024-03-24T10:38:39.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:39.841+0000 7fb80e136700 1 -- 172.21.15.96:0/1586464824 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb80810d7f0 con 0x7fb8080fca50 2024-03-24T10:38:39.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:39.841+0000 7fb7effff700 1 -- 172.21.15.96:0/1586464824 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb7f800b5c0 con 0x7fb8080fca50 2024-03-24T10:38:39.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:39.841+0000 7fb7effff700 1 -- 172.21.15.96:0/1586464824 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb7f801c5b0 con 0x7fb8080fca50 2024-03-24T10:38:39.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:39.841+0000 7fb80e136700 1 -- 172.21.15.96:0/1586464824 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb80810e360 con 0x7fb8080fca50 2024-03-24T10:38:39.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:39.842+0000 7fb7effff700 1 -- 172.21.15.96:0/1586464824 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fb7f801c710 con 0x7fb8080fca50 2024-03-24T10:38:39.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:39.842+0000 7fb7effff700 1 --2- 172.21.15.96:0/1586464824 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fb7f00368d0 0x7fb7f0038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:38:39.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:39.842+0000 7fb7effff700 1 -- 172.21.15.96:0/1586464824 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fb7f8049a10 con 0x7fb8080fca50 2024-03-24T10:38:39.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:39.842+0000 7fb80e136700 1 -- 172.21.15.96:0/1586464824 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb808102f60 con 0x7fb8080fca50 2024-03-24T10:38:39.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:39.842+0000 7fb806ffd700 1 --2- 172.21.15.96:0/1586464824 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fb7f00368d0 0x7fb7f0038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:38:39.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:39.842+0000 7fb806ffd700 1 --2- 172.21.15.96:0/1586464824 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fb7f00368d0 0x7fb7f0038d80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc009990 tx=0x7fb7fc006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:38:39.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:39.847+0000 7fb7effff700 1 -- 172.21.15.96:0/1586464824 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fb7f8025400 con 0x7fb8080fca50 2024-03-24T10:38:40.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:40.105+0000 7fb80e136700 1 -- 172.21.15.96:0/1586464824 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb808102890 con 0x7fb8080fca50 2024-03-24T10:38:40.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:40.106+0000 7fb7effff700 1 -- 172.21.15.96:0/1586464824 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fb7f8011370 con 0x7fb8080fca50 2024-03-24T10:38:40.107 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:38:40.107 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":1,"fsid":"386110d4-e9ca-11ee-95cd-87774f69a715","modified":"2024-03-24T10:36:06.632144Z","created":"2024-03-24T10:36:06.632144Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi094","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-24T10:38:40.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:40.108+0000 7fb80e136700 1 -- 172.21.15.96:0/1586464824 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fb7f00368d0 msgr2=0x7fb7f0038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:38:40.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:40.108+0000 7fb80e136700 1 --2- 172.21.15.96:0/1586464824 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fb7f00368d0 0x7fb7f0038d80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc009990 tx=0x7fb7fc006e40 comp rx=0 tx=0).stop 2024-03-24T10:38:40.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:40.108+0000 7fb80e136700 1 -- 172.21.15.96:0/1586464824 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8080fca50 msgr2=0x7fb80810d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:38:40.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:40.108+0000 7fb80e136700 1 --2- 172.21.15.96:0/1586464824 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8080fca50 0x7fb80810d0b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb7f80096b0 tx=0x7fb7f8009e30 comp rx=0 tx=0).stop 2024-03-24T10:38:40.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:40.108+0000 7fb80e136700 1 -- 172.21.15.96:0/1586464824 shutdown_connections 2024-03-24T10:38:40.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:40.108+0000 7fb80e136700 1 --2- 172.21.15.96:0/1586464824 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fb7f00368d0 0x7fb7f0038d80 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:40.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:40.108+0000 7fb80e136700 1 --2- 172.21.15.96:0/1586464824 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8080fca50 0x7fb80810d0b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:40.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:40.108+0000 7fb80e136700 1 -- 172.21.15.96:0/1586464824 >> 172.21.15.96:0/1586464824 conn(0x7fb8080f8190 msgr2=0x7fb808103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:38:40.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:40.108+0000 7fb80e136700 1 -- 172.21.15.96:0/1586464824 shutdown_connections 2024-03-24T10:38:40.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:40.108+0000 7fb80e136700 1 -- 172.21.15.96:0/1586464824 wait complete. 2024-03-24T10:38:40.110 INFO:teuthology.orchestra.run.smithi096.stderr:dumped monmap epoch 1 2024-03-24T10:38:40.740 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:40 smithi094 ceph-mon[29446]: from='client.? 172.21.15.96:0/1586464824' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T10:38:41.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:41 smithi094 ceph-mon[29446]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:38:41.671 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-24T10:38:41.672 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T10:38:41.921 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T10:38:41.967 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T10:38:43.312 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:43 smithi094 ceph-mon[29446]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:38:43.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:43.461+0000 7f3a88308700 1 -- 172.21.15.96:0/1393488224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a800fb0f0 msgr2=0x7f3a800fb500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:38:43.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:43.461+0000 7f3a88308700 1 --2- 172.21.15.96:0/1393488224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a800fb0f0 0x7f3a800fb500 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3a700048c0 tx=0x7f3a70009e30 comp rx=0 tx=0).stop 2024-03-24T10:38:43.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:43.461+0000 7f3a88308700 1 -- 172.21.15.96:0/1393488224 shutdown_connections 2024-03-24T10:38:43.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:43.461+0000 7f3a88308700 1 --2- 172.21.15.96:0/1393488224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a800fb0f0 0x7f3a800fb500 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:43.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:43.461+0000 7f3a88308700 1 -- 172.21.15.96:0/1393488224 >> 172.21.15.96:0/1393488224 conn(0x7f3a800f6850 msgr2=0x7f3a800f8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:38:43.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:43.461+0000 7f3a88308700 1 -- 172.21.15.96:0/1393488224 shutdown_connections 2024-03-24T10:38:43.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:43.461+0000 7f3a88308700 1 -- 172.21.15.96:0/1393488224 wait complete. 2024-03-24T10:38:43.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:43.462+0000 7f3a88308700 1 Processor -- start 2024-03-24T10:38:43.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:43.462+0000 7f3a88308700 1 -- start start 2024-03-24T10:38:43.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:43.462+0000 7f3a88308700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a800fb0f0 0x7f3a8010bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:38:43.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:43.462+0000 7f3a88308700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8010c500 con 0x7f3a800fb0f0 2024-03-24T10:38:43.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:43.462+0000 7f3a860a4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a800fb0f0 0x7f3a8010bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:38:43.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:43.462+0000 7f3a860a4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a800fb0f0 0x7f3a8010bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.96:54604/0 (socket says 172.21.15.96:54604) 2024-03-24T10:38:43.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:43.462+0000 7f3a860a4700 1 -- 172.21.15.96:0/1390786542 learned_addr learned my addr 172.21.15.96:0/1390786542 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-24T10:38:43.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:43.463+0000 7f3a860a4700 1 -- 172.21.15.96:0/1390786542 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a70005040 con 0x7f3a800fb0f0 2024-03-24T10:38:43.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:43.463+0000 7f3a860a4700 1 --2- 172.21.15.96:0/1390786542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a800fb0f0 0x7f3a8010bfc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3a70009680 tx=0x7f3a70009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:38:43.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:43.463+0000 7f3a76ffd700 1 -- 172.21.15.96:0/1390786542 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3a7000b5c0 con 0x7f3a800fb0f0 2024-03-24T10:38:43.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:43.463+0000 7f3a76ffd700 1 -- 172.21.15.96:0/1390786542 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3a7000be70 con 0x7f3a800fb0f0 2024-03-24T10:38:43.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:43.463+0000 7f3a88308700 1 -- 172.21.15.96:0/1390786542 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a8010c700 con 0x7f3a800fb0f0 2024-03-24T10:38:43.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:43.463+0000 7f3a76ffd700 1 -- 172.21.15.96:0/1390786542 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3a700135a0 con 0x7f3a800fb0f0 2024-03-24T10:38:43.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:43.463+0000 7f3a88308700 1 -- 172.21.15.96:0/1390786542 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a8010d270 con 0x7f3a800fb0f0 2024-03-24T10:38:43.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:43.464+0000 7f3a76ffd700 1 -- 172.21.15.96:0/1390786542 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f3a70013860 con 0x7f3a800fb0f0 2024-03-24T10:38:43.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:43.464+0000 7f3a88308700 1 -- 172.21.15.96:0/1390786542 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a80101600 con 0x7f3a800fb0f0 2024-03-24T10:38:43.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:43.464+0000 7f3a76ffd700 1 --2- 172.21.15.96:0/1390786542 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f3a6c036880 0x7f3a6c038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:38:43.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:43.464+0000 7f3a76ffd700 1 -- 172.21.15.96:0/1390786542 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f3a70049540 con 0x7f3a800fb0f0 2024-03-24T10:38:43.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:43.464+0000 7f3a858a3700 1 --2- 172.21.15.96:0/1390786542 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f3a6c036880 0x7f3a6c038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:38:43.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:43.465+0000 7f3a858a3700 1 --2- 172.21.15.96:0/1390786542 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f3a6c036880 0x7f3a6c038d30 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c009990 tx=0x7f3a7c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:38:43.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:43.469+0000 7f3a76ffd700 1 -- 172.21.15.96:0/1390786542 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f3a70011370 con 0x7f3a800fb0f0 2024-03-24T10:38:43.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:43.731+0000 7f3a88308700 1 -- 172.21.15.96:0/1390786542 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3a8010dc80 con 0x7f3a800fb0f0 2024-03-24T10:38:43.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:43.732+0000 7f3a76ffd700 1 -- 172.21.15.96:0/1390786542 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f3a70024b50 con 0x7f3a800fb0f0 2024-03-24T10:38:43.733 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:38:43.733 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":1,"fsid":"386110d4-e9ca-11ee-95cd-87774f69a715","modified":"2024-03-24T10:36:06.632144Z","created":"2024-03-24T10:36:06.632144Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi094","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-24T10:38:43.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:43.734+0000 7f3a88308700 1 -- 172.21.15.96:0/1390786542 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f3a6c036880 msgr2=0x7f3a6c038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:38:43.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:43.734+0000 7f3a88308700 1 --2- 172.21.15.96:0/1390786542 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f3a6c036880 0x7f3a6c038d30 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c009990 tx=0x7f3a7c006e40 comp rx=0 tx=0).stop 2024-03-24T10:38:43.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:43.734+0000 7f3a88308700 1 -- 172.21.15.96:0/1390786542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a800fb0f0 msgr2=0x7f3a8010bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:38:43.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:43.734+0000 7f3a88308700 1 --2- 172.21.15.96:0/1390786542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a800fb0f0 0x7f3a8010bfc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3a70009680 tx=0x7f3a70009e30 comp rx=0 tx=0).stop 2024-03-24T10:38:43.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:43.734+0000 7f3a88308700 1 -- 172.21.15.96:0/1390786542 shutdown_connections 2024-03-24T10:38:43.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:43.734+0000 7f3a88308700 1 --2- 172.21.15.96:0/1390786542 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f3a6c036880 0x7f3a6c038d30 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:43.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:43.734+0000 7f3a88308700 1 --2- 172.21.15.96:0/1390786542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a800fb0f0 0x7f3a8010bfc0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:43.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:43.734+0000 7f3a88308700 1 -- 172.21.15.96:0/1390786542 >> 172.21.15.96:0/1390786542 conn(0x7f3a800f6850 msgr2=0x7f3a800feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:38:43.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:43.734+0000 7f3a88308700 1 -- 172.21.15.96:0/1390786542 shutdown_connections 2024-03-24T10:38:43.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:43.734+0000 7f3a88308700 1 -- 172.21.15.96:0/1390786542 wait complete. 2024-03-24T10:38:43.736 INFO:teuthology.orchestra.run.smithi096.stderr:dumped monmap epoch 1 2024-03-24T10:38:44.909 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:44 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' 2024-03-24T10:38:44.909 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:44 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' 2024-03-24T10:38:44.909 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:44 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' 2024-03-24T10:38:44.909 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:44 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' 2024-03-24T10:38:44.909 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:44 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' 2024-03-24T10:38:44.909 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:44 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' 2024-03-24T10:38:44.909 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:44 smithi094 ceph-mon[29446]: Deploying daemon prometheus.smithi094 on smithi094 2024-03-24T10:38:44.909 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:44 smithi094 ceph-mon[29446]: from='client.? 172.21.15.96:0/1390786542' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T10:38:45.343 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-24T10:38:45.343 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T10:38:45.589 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T10:38:45.635 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T10:38:45.897 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:45 smithi094 ceph-mon[29446]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:38:47.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:47.190+0000 7fcedb015700 1 -- 172.21.15.96:0/3785317735 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fced40fd050 msgr2=0x7fced40fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:38:47.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:47.190+0000 7fcedb015700 1 --2- 172.21.15.96:0/3785317735 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fced40fd050 0x7fced40fd460 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcec80048c0 tx=0x7fcec8009e30 comp rx=0 tx=0).stop 2024-03-24T10:38:47.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:47.190+0000 7fcedb015700 1 -- 172.21.15.96:0/3785317735 shutdown_connections 2024-03-24T10:38:47.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:47.190+0000 7fcedb015700 1 --2- 172.21.15.96:0/3785317735 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fced40fd050 0x7fced40fd460 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:47.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:47.190+0000 7fcedb015700 1 -- 172.21.15.96:0/3785317735 >> 172.21.15.96:0/3785317735 conn(0x7fced40f70d0 msgr2=0x7fced40f9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:38:47.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:47.190+0000 7fcedb015700 1 -- 172.21.15.96:0/3785317735 shutdown_connections 2024-03-24T10:38:47.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:47.190+0000 7fcedb015700 1 -- 172.21.15.96:0/3785317735 wait complete. 2024-03-24T10:38:47.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:47.191+0000 7fcedb015700 1 Processor -- start 2024-03-24T10:38:47.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:47.191+0000 7fcedb015700 1 -- start start 2024-03-24T10:38:47.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:47.191+0000 7fcedb015700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fced40fd050 0x7fced4065bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:38:47.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:47.191+0000 7fcedb015700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced410dbd0 con 0x7fced40fd050 2024-03-24T10:38:47.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:47.191+0000 7fced8db1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fced40fd050 0x7fced4065bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:38:47.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:47.191+0000 7fced8db1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fced40fd050 0x7fced4065bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.96:59140/0 (socket says 172.21.15.96:59140) 2024-03-24T10:38:47.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:47.191+0000 7fced8db1700 1 -- 172.21.15.96:0/1022226171 learned_addr learned my addr 172.21.15.96:0/1022226171 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-24T10:38:47.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:47.192+0000 7fced8db1700 1 -- 172.21.15.96:0/1022226171 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec8005040 con 0x7fced40fd050 2024-03-24T10:38:47.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:47.192+0000 7fced8db1700 1 --2- 172.21.15.96:0/1022226171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fced40fd050 0x7fced4065bd0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fcec80096b0 tx=0x7fcec8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:38:47.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:47.192+0000 7fced17fa700 1 -- 172.21.15.96:0/1022226171 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcec800bed0 con 0x7fced40fd050 2024-03-24T10:38:47.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:47.192+0000 7fcedb015700 1 -- 172.21.15.96:0/1022226171 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fced41121e0 con 0x7fced40fd050 2024-03-24T10:38:47.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:47.192+0000 7fced17fa700 1 -- 172.21.15.96:0/1022226171 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcec800b5c0 con 0x7fced40fd050 2024-03-24T10:38:47.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:47.192+0000 7fcedb015700 1 -- 172.21.15.96:0/1022226171 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fced4112610 con 0x7fced40fd050 2024-03-24T10:38:47.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:47.192+0000 7fced17fa700 1 -- 172.21.15.96:0/1022226171 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcec8025560 con 0x7fced40fd050 2024-03-24T10:38:47.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:47.193+0000 7fced17fa700 1 -- 172.21.15.96:0/1022226171 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fcec80256c0 con 0x7fced40fd050 2024-03-24T10:38:47.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:47.193+0000 7fcedb015700 1 -- 172.21.15.96:0/1022226171 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fced4105920 con 0x7fced40fd050 2024-03-24T10:38:47.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:47.193+0000 7fced17fa700 1 --2- 172.21.15.96:0/1022226171 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fcec4036880 0x7fcec4038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:38:47.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:47.193+0000 7fced17fa700 1 -- 172.21.15.96:0/1022226171 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fcec8049aa0 con 0x7fced40fd050 2024-03-24T10:38:47.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:47.193+0000 7fced3fff700 1 --2- 172.21.15.96:0/1022226171 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fcec4036880 0x7fcec4038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:38:47.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:47.193+0000 7fced3fff700 1 --2- 172.21.15.96:0/1022226171 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fcec4036880 0x7fcec4038d30 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fcec0009990 tx=0x7fcec0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:38:47.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:47.198+0000 7fced17fa700 1 -- 172.21.15.96:0/1022226171 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fcec8013420 con 0x7fced40fd050 2024-03-24T10:38:47.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:47.460+0000 7fcedb015700 1 -- 172.21.15.96:0/1022226171 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fced405f5f0 con 0x7fced40fd050 2024-03-24T10:38:47.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:47.460+0000 7fced17fa700 1 -- 172.21.15.96:0/1022226171 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fcec8011370 con 0x7fced40fd050 2024-03-24T10:38:47.461 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:38:47.461 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":1,"fsid":"386110d4-e9ca-11ee-95cd-87774f69a715","modified":"2024-03-24T10:36:06.632144Z","created":"2024-03-24T10:36:06.632144Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi094","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-24T10:38:47.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:47.462+0000 7fcedb015700 1 -- 172.21.15.96:0/1022226171 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fcec4036880 msgr2=0x7fcec4038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:38:47.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:47.462+0000 7fcedb015700 1 --2- 172.21.15.96:0/1022226171 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fcec4036880 0x7fcec4038d30 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fcec0009990 tx=0x7fcec0006e40 comp rx=0 tx=0).stop 2024-03-24T10:38:47.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:47.462+0000 7fcedb015700 1 -- 172.21.15.96:0/1022226171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fced40fd050 msgr2=0x7fced4065bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:38:47.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:47.462+0000 7fcedb015700 1 --2- 172.21.15.96:0/1022226171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fced40fd050 0x7fced4065bd0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fcec80096b0 tx=0x7fcec8009e30 comp rx=0 tx=0).stop 2024-03-24T10:38:47.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:47.462+0000 7fcedb015700 1 -- 172.21.15.96:0/1022226171 shutdown_connections 2024-03-24T10:38:47.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:47.462+0000 7fcedb015700 1 --2- 172.21.15.96:0/1022226171 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fcec4036880 0x7fcec4038d30 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:47.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:47.462+0000 7fcedb015700 1 --2- 172.21.15.96:0/1022226171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fced40fd050 0x7fced4065bd0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:47.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:47.462+0000 7fcedb015700 1 -- 172.21.15.96:0/1022226171 >> 172.21.15.96:0/1022226171 conn(0x7fced40f70d0 msgr2=0x7fced4102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:38:47.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:47.462+0000 7fcedb015700 1 -- 172.21.15.96:0/1022226171 shutdown_connections 2024-03-24T10:38:47.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:47.462+0000 7fcedb015700 1 -- 172.21.15.96:0/1022226171 wait complete. 2024-03-24T10:38:47.464 INFO:teuthology.orchestra.run.smithi096.stderr:dumped monmap epoch 1 2024-03-24T10:38:47.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:47 smithi094 ceph-mon[29446]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:38:47.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:47 smithi094 ceph-mon[29446]: from='client.? 172.21.15.96:0/1022226171' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T10:38:49.028 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-24T10:38:49.028 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T10:38:49.274 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T10:38:49.320 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T10:38:49.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:49 smithi094 ceph-mon[29446]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:38:49.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:49 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' 2024-03-24T10:38:50.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:50.807+0000 7f3ad3082700 1 -- 172.21.15.96:0/3529999753 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3acc0fd6d0 msgr2=0x7f3acc0ffab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:38:50.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:50.807+0000 7f3ad3082700 1 --2- 172.21.15.96:0/3529999753 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3acc0fd6d0 0x7f3acc0ffab0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4009c60 tx=0x7f3ac4004d10 comp rx=0 tx=0).stop 2024-03-24T10:38:50.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:50.807+0000 7f3ad3082700 1 -- 172.21.15.96:0/3529999753 shutdown_connections 2024-03-24T10:38:50.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:50.807+0000 7f3ad3082700 1 --2- 172.21.15.96:0/3529999753 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3acc0fd6d0 0x7f3acc0ffab0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:50.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:50.807+0000 7f3ad3082700 1 -- 172.21.15.96:0/3529999753 >> 172.21.15.96:0/3529999753 conn(0x7f3acc0f70d0 msgr2=0x7f3acc0f9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:38:50.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:50.807+0000 7f3ad3082700 1 -- 172.21.15.96:0/3529999753 shutdown_connections 2024-03-24T10:38:50.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:50.807+0000 7f3ad3082700 1 -- 172.21.15.96:0/3529999753 wait complete. 2024-03-24T10:38:50.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:50.808+0000 7f3ad3082700 1 Processor -- start 2024-03-24T10:38:50.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:50.808+0000 7f3ad3082700 1 -- start start 2024-03-24T10:38:50.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:50.808+0000 7f3ad3082700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3acc0fd6d0 0x7f3acc10d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:38:50.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:50.808+0000 7f3ad3082700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3acc10d5f0 con 0x7f3acc0fd6d0 2024-03-24T10:38:50.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:50.808+0000 7f3ad0e1e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3acc0fd6d0 0x7f3acc10d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:38:50.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:50.808+0000 7f3ad0e1e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3acc0fd6d0 0x7f3acc10d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.96:59166/0 (socket says 172.21.15.96:59166) 2024-03-24T10:38:50.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:50.808+0000 7f3ad0e1e700 1 -- 172.21.15.96:0/3632536073 learned_addr learned my addr 172.21.15.96:0/3632536073 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-24T10:38:50.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:50.809+0000 7f3ad0e1e700 1 -- 172.21.15.96:0/3632536073 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac4009940 con 0x7f3acc0fd6d0 2024-03-24T10:38:50.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:50.809+0000 7f3ad0e1e700 1 --2- 172.21.15.96:0/3632536073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3acc0fd6d0 0x7f3acc10d0b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3ac40097b0 tx=0x7f3ac4004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:38:50.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:50.809+0000 7f3ac97fa700 1 -- 172.21.15.96:0/3632536073 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3ac4012410 con 0x7f3acc0fd6d0 2024-03-24T10:38:50.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:50.809+0000 7f3ad3082700 1 -- 172.21.15.96:0/3632536073 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3acc10d7f0 con 0x7f3acc0fd6d0 2024-03-24T10:38:50.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:50.809+0000 7f3ac97fa700 1 -- 172.21.15.96:0/3632536073 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ac4012a40 con 0x7f3acc0fd6d0 2024-03-24T10:38:50.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:50.809+0000 7f3ac97fa700 1 -- 172.21.15.96:0/3632536073 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3ac401b980 con 0x7f3acc0fd6d0 2024-03-24T10:38:50.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:50.809+0000 7f3ad3082700 1 -- 172.21.15.96:0/3632536073 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3acc10e360 con 0x7f3acc0fd6d0 2024-03-24T10:38:50.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:50.810+0000 7f3ac97fa700 1 -- 172.21.15.96:0/3632536073 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f3ac4019040 con 0x7f3acc0fd6d0 2024-03-24T10:38:50.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:50.810+0000 7f3ad3082700 1 -- 172.21.15.96:0/3632536073 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3acc102f60 con 0x7f3acc0fd6d0 2024-03-24T10:38:50.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:50.810+0000 7f3ac97fa700 1 --2- 172.21.15.96:0/3632536073 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f3ab4036880 0x7f3ab4038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:38:50.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:50.810+0000 7f3ac97fa700 1 -- 172.21.15.96:0/3632536073 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f3ac4049cf0 con 0x7f3acc0fd6d0 2024-03-24T10:38:50.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:50.810+0000 7f3acbfff700 1 --2- 172.21.15.96:0/3632536073 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f3ab4036880 0x7f3ab4038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:38:50.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:50.811+0000 7f3acbfff700 1 --2- 172.21.15.96:0/3632536073 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f3ab4036880 0x7f3ab4038d30 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0009990 tx=0x7f3ac0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:38:50.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:50.815+0000 7f3ac97fa700 1 -- 172.21.15.96:0/3632536073 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f3ac4012bb0 con 0x7f3acc0fd6d0 2024-03-24T10:38:51.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:51.076+0000 7f3ad3082700 1 -- 172.21.15.96:0/3632536073 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3acc10ed70 con 0x7f3acc0fd6d0 2024-03-24T10:38:51.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:51.077+0000 7f3ac97fa700 1 -- 172.21.15.96:0/3632536073 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f3ac4017090 con 0x7f3acc0fd6d0 2024-03-24T10:38:51.077 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:38:51.077 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":1,"fsid":"386110d4-e9ca-11ee-95cd-87774f69a715","modified":"2024-03-24T10:36:06.632144Z","created":"2024-03-24T10:36:06.632144Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi094","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-24T10:38:51.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:51.079+0000 7f3ad3082700 1 -- 172.21.15.96:0/3632536073 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f3ab4036880 msgr2=0x7f3ab4038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:38:51.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:51.079+0000 7f3ad3082700 1 --2- 172.21.15.96:0/3632536073 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f3ab4036880 0x7f3ab4038d30 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0009990 tx=0x7f3ac0006e40 comp rx=0 tx=0).stop 2024-03-24T10:38:51.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:51.079+0000 7f3ad3082700 1 -- 172.21.15.96:0/3632536073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3acc0fd6d0 msgr2=0x7f3acc10d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:38:51.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:51.079+0000 7f3ad3082700 1 --2- 172.21.15.96:0/3632536073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3acc0fd6d0 0x7f3acc10d0b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3ac40097b0 tx=0x7f3ac4004d10 comp rx=0 tx=0).stop 2024-03-24T10:38:51.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:51.079+0000 7f3ad3082700 1 -- 172.21.15.96:0/3632536073 shutdown_connections 2024-03-24T10:38:51.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:51.079+0000 7f3ad3082700 1 --2- 172.21.15.96:0/3632536073 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f3ab4036880 0x7f3ab4038d30 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:51.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:51.079+0000 7f3ad3082700 1 --2- 172.21.15.96:0/3632536073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3acc0fd6d0 0x7f3acc10d0b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:51.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:51.079+0000 7f3ad3082700 1 -- 172.21.15.96:0/3632536073 >> 172.21.15.96:0/3632536073 conn(0x7f3acc0f70d0 msgr2=0x7f3acc103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:38:51.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:51.079+0000 7f3ad3082700 1 -- 172.21.15.96:0/3632536073 shutdown_connections 2024-03-24T10:38:51.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:51.079+0000 7f3ad3082700 1 -- 172.21.15.96:0/3632536073 wait complete. 2024-03-24T10:38:51.080 INFO:teuthology.orchestra.run.smithi096.stderr:dumped monmap epoch 1 2024-03-24T10:38:51.434 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:51 smithi094 ceph-mon[29446]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:38:51.434 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:51 smithi094 ceph-mon[29446]: from='client.? 172.21.15.96:0/3632536073' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T10:38:52.621 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-24T10:38:52.622 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T10:38:52.872 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T10:38:52.918 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T10:38:53.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:53 smithi094 ceph-mon[29446]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:38:54.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:54.310+0000 7fe06c218700 1 -- 172.21.15.96:0/1686268916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0640fe0b0 msgr2=0x7fe0640fe4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:38:54.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:54.310+0000 7fe06c218700 1 --2- 172.21.15.96:0/1686268916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0640fe0b0 0x7fe0640fe4c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe0540048c0 tx=0x7fe054009e30 comp rx=0 tx=0).stop 2024-03-24T10:38:54.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:54.310+0000 7fe06c218700 1 -- 172.21.15.96:0/1686268916 shutdown_connections 2024-03-24T10:38:54.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:54.310+0000 7fe06c218700 1 --2- 172.21.15.96:0/1686268916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0640fe0b0 0x7fe0640fe4c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:54.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:54.310+0000 7fe06c218700 1 -- 172.21.15.96:0/1686268916 >> 172.21.15.96:0/1686268916 conn(0x7fe0640f6850 msgr2=0x7fe0640f8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:38:54.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:54.310+0000 7fe06c218700 1 -- 172.21.15.96:0/1686268916 shutdown_connections 2024-03-24T10:38:54.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:54.310+0000 7fe06c218700 1 -- 172.21.15.96:0/1686268916 wait complete. 2024-03-24T10:38:54.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:54.311+0000 7fe06c218700 1 Processor -- start 2024-03-24T10:38:54.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:54.311+0000 7fe06c218700 1 -- start start 2024-03-24T10:38:54.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:54.311+0000 7fe06c218700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0640fe0b0 0x7fe0640fd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:38:54.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:54.311+0000 7fe06c218700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0640fdd00 con 0x7fe0640fe0b0 2024-03-24T10:38:54.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:54.311+0000 7fe069fb4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0640fe0b0 0x7fe0640fd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:38:54.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:54.311+0000 7fe069fb4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0640fe0b0 0x7fe0640fd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.96:59194/0 (socket says 172.21.15.96:59194) 2024-03-24T10:38:54.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:54.311+0000 7fe069fb4700 1 -- 172.21.15.96:0/531651643 learned_addr learned my addr 172.21.15.96:0/531651643 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-24T10:38:54.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:54.311+0000 7fe069fb4700 1 -- 172.21.15.96:0/531651643 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe054005040 con 0x7fe0640fe0b0 2024-03-24T10:38:54.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:54.311+0000 7fe069fb4700 1 --2- 172.21.15.96:0/531651643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0640fe0b0 0x7fe0640fd7c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe0540096b0 tx=0x7fe054009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:38:54.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:54.312+0000 7fe05a7fc700 1 -- 172.21.15.96:0/531651643 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe05400bed0 con 0x7fe0640fe0b0 2024-03-24T10:38:54.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:54.312+0000 7fe06c218700 1 -- 172.21.15.96:0/531651643 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0640fbdd0 con 0x7fe0640fe0b0 2024-03-24T10:38:54.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:54.312+0000 7fe05a7fc700 1 -- 172.21.15.96:0/531651643 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe05400b5c0 con 0x7fe0640fe0b0 2024-03-24T10:38:54.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:54.312+0000 7fe05a7fc700 1 -- 172.21.15.96:0/531651643 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe05401c5b0 con 0x7fe0640fe0b0 2024-03-24T10:38:54.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:54.312+0000 7fe06c218700 1 -- 172.21.15.96:0/531651643 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0640fc2c0 con 0x7fe0640fe0b0 2024-03-24T10:38:54.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:54.312+0000 7fe05a7fc700 1 -- 172.21.15.96:0/531651643 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fe054004d30 con 0x7fe0640fe0b0 2024-03-24T10:38:54.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:54.313+0000 7fe06c218700 1 -- 172.21.15.96:0/531651643 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe06405f5f0 con 0x7fe0640fe0b0 2024-03-24T10:38:54.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:54.313+0000 7fe05a7fc700 1 --2- 172.21.15.96:0/531651643 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fe0500368d0 0x7fe050038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:38:54.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:54.313+0000 7fe05a7fc700 1 -- 172.21.15.96:0/531651643 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fe054049ab0 con 0x7fe0640fe0b0 2024-03-24T10:38:54.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:54.313+0000 7fe0697b3700 1 --2- 172.21.15.96:0/531651643 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fe0500368d0 0x7fe050038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:38:54.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:54.313+0000 7fe0697b3700 1 --2- 172.21.15.96:0/531651643 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fe0500368d0 0x7fe050038d80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe060009990 tx=0x7fe060006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:38:54.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:54.318+0000 7fe05a7fc700 1 -- 172.21.15.96:0/531651643 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fe054011550 con 0x7fe0640fe0b0 2024-03-24T10:38:54.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:54.321+0000 7fe05a7fc700 1 -- 172.21.15.96:0/531651643 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 14) v1 ==== 43438+0+0 (secure 0 0 0) 0x7fe054025400 con 0x7fe0640fe0b0 2024-03-24T10:38:54.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:54.345+0000 7fe0697b3700 1 -- 172.21.15.96:0/531651643 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fe0500368d0 msgr2=0x7fe050038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-03-24T10:38:54.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:54.345+0000 7fe0697b3700 1 -- 172.21.15.96:0/531651643 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fe0500368d0 msgr2=0x7fe050038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-24T10:38:54.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:54.345+0000 7fe0697b3700 1 --2- 172.21.15.96:0/531651643 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fe0500368d0 0x7fe050038d80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe060009990 tx=0x7fe060006e40 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-24T10:38:54.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:54.345+0000 7fe0697b3700 1 --2- 172.21.15.96:0/531651643 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fe0500368d0 0x7fe050038d80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe060009990 tx=0x7fe060006e40 comp rx=0 tx=0).stop 2024-03-24T10:38:54.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:54.345+0000 7fe05a7fc700 1 -- 172.21.15.96:0/531651643 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fe0500368d0 msgr2=0x7fe050038d80 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-24T10:38:54.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:54.345+0000 7fe05a7fc700 1 --2- 172.21.15.96:0/531651643 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fe0500368d0 0x7fe050038d80 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:54.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:54.586+0000 7fe06c218700 1 -- 172.21.15.96:0/531651643 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe04c005160 con 0x7fe0640fe0b0 2024-03-24T10:38:54.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:54.587+0000 7fe05a7fc700 1 -- 172.21.15.96:0/531651643 <== mon.0 v2:172.21.15.94:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fe054011730 con 0x7fe0640fe0b0 2024-03-24T10:38:54.588 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:38:54.588 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":1,"fsid":"386110d4-e9ca-11ee-95cd-87774f69a715","modified":"2024-03-24T10:36:06.632144Z","created":"2024-03-24T10:36:06.632144Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi094","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-24T10:38:54.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:54.589+0000 7fe06c218700 1 -- 172.21.15.96:0/531651643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0640fe0b0 msgr2=0x7fe0640fd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:38:54.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:54.589+0000 7fe06c218700 1 --2- 172.21.15.96:0/531651643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0640fe0b0 0x7fe0640fd7c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe0540096b0 tx=0x7fe054009e30 comp rx=0 tx=0).stop 2024-03-24T10:38:54.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:54.589+0000 7fe06c218700 1 -- 172.21.15.96:0/531651643 shutdown_connections 2024-03-24T10:38:54.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:54.589+0000 7fe06c218700 1 --2- 172.21.15.96:0/531651643 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7fe0500368d0 0x7fe050038d80 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:54.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:54.589+0000 7fe06c218700 1 --2- 172.21.15.96:0/531651643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0640fe0b0 0x7fe0640fd7c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:54.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:54.589+0000 7fe06c218700 1 -- 172.21.15.96:0/531651643 >> 172.21.15.96:0/531651643 conn(0x7fe0640f6850 msgr2=0x7fe064103bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:38:54.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:54.589+0000 7fe06c218700 1 -- 172.21.15.96:0/531651643 shutdown_connections 2024-03-24T10:38:54.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:54.589+0000 7fe06c218700 1 -- 172.21.15.96:0/531651643 wait complete. 2024-03-24T10:38:54.590 INFO:teuthology.orchestra.run.smithi096.stderr:dumped monmap epoch 1 2024-03-24T10:38:54.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:54 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' 2024-03-24T10:38:54.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:54 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' 2024-03-24T10:38:54.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:54 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' 2024-03-24T10:38:54.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:54 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-03-24T10:38:55.637 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:55 smithi094 ceph-mon[29446]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-03-24T10:38:55.637 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:55 smithi094 ceph-mon[29446]: mgrmap e14: smithi094.yaethu(active, since 96s) 2024-03-24T10:38:55.637 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:55 smithi094 ceph-mon[29446]: from='client.? 172.21.15.96:0/531651643' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T10:38:56.161 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-24T10:38:56.161 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T10:38:56.412 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T10:38:56.458 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T10:38:57.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:57.913+0000 7f54b4a4a700 1 -- 172.21.15.96:0/146978431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54b00fb960 msgr2=0x7f54b00fbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:38:57.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:57.913+0000 7f54b4a4a700 1 --2- 172.21.15.96:0/146978431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54b00fb960 0x7f54b00fbd70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f54a80048c0 tx=0x7f54a8009e30 comp rx=0 tx=0).stop 2024-03-24T10:38:57.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:57.914+0000 7f54b4a4a700 1 -- 172.21.15.96:0/146978431 shutdown_connections 2024-03-24T10:38:57.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:57.914+0000 7f54b4a4a700 1 --2- 172.21.15.96:0/146978431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54b00fb960 0x7f54b00fbd70 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:57.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:57.914+0000 7f54b4a4a700 1 -- 172.21.15.96:0/146978431 >> 172.21.15.96:0/146978431 conn(0x7f54b00f70c0 msgr2=0x7f54b00f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:38:57.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:57.914+0000 7f54b4a4a700 1 -- 172.21.15.96:0/146978431 shutdown_connections 2024-03-24T10:38:57.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:57.914+0000 7f54b4a4a700 1 -- 172.21.15.96:0/146978431 wait complete. 2024-03-24T10:38:57.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:57.914+0000 7f54b4a4a700 1 Processor -- start 2024-03-24T10:38:57.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:57.914+0000 7f54b4a4a700 1 -- start start 2024-03-24T10:38:57.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:57.914+0000 7f54b4a4a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54b00fb960 0x7f54b0066860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:38:57.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:57.914+0000 7f54b4a4a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b010dba0 con 0x7f54b00fb960 2024-03-24T10:38:57.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:57.914+0000 7f54ae59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54b00fb960 0x7f54b0066860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:38:57.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:57.915+0000 7f54ae59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54b00fb960 0x7f54b0066860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.96:33516/0 (socket says 172.21.15.96:33516) 2024-03-24T10:38:57.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:57.915+0000 7f54ae59c700 1 -- 172.21.15.96:0/3687381999 learned_addr learned my addr 172.21.15.96:0/3687381999 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-24T10:38:57.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:57.915+0000 7f54ae59c700 1 -- 172.21.15.96:0/3687381999 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a8005040 con 0x7f54b00fb960 2024-03-24T10:38:57.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:57.915+0000 7f54ae59c700 1 --2- 172.21.15.96:0/3687381999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54b00fb960 0x7f54b0066860 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f54a80096b0 tx=0x7f54a8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:38:57.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:57.915+0000 7f549effd700 1 -- 172.21.15.96:0/3687381999 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f54a800bed0 con 0x7f54b00fb960 2024-03-24T10:38:57.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:57.915+0000 7f549effd700 1 -- 172.21.15.96:0/3687381999 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f54a800b5c0 con 0x7f54b00fb960 2024-03-24T10:38:57.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:57.915+0000 7f549effd700 1 -- 172.21.15.96:0/3687381999 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f54a801c5b0 con 0x7f54b00fb960 2024-03-24T10:38:57.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:57.915+0000 7f54b4a4a700 1 -- 172.21.15.96:0/3687381999 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54b01121b0 con 0x7f54b00fb960 2024-03-24T10:38:57.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:57.915+0000 7f54b4a4a700 1 -- 172.21.15.96:0/3687381999 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54b01125e0 con 0x7f54b00fb960 2024-03-24T10:38:57.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:57.916+0000 7f54b4a4a700 1 -- 172.21.15.96:0/3687381999 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54b0105920 con 0x7f54b00fb960 2024-03-24T10:38:57.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:57.916+0000 7f549effd700 1 -- 172.21.15.96:0/3687381999 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 14) v1 ==== 43438+0+0 (secure 0 0 0) 0x7f54a8004d30 con 0x7f54b00fb960 2024-03-24T10:38:57.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:57.916+0000 7f549effd700 1 --2- 172.21.15.96:0/3687381999 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f54980368d0 0x7f5498038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:38:57.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:57.916+0000 7f549effd700 1 -- 172.21.15.96:0/3687381999 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f54a8049a90 con 0x7f54b00fb960 2024-03-24T10:38:57.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:57.916+0000 7f54add9b700 1 -- 172.21.15.96:0/3687381999 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f54980368d0 msgr2=0x7f5498038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/2406899470 2024-03-24T10:38:57.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:57.917+0000 7f54add9b700 1 --2- 172.21.15.96:0/3687381999 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f54980368d0 0x7f5498038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-24T10:38:57.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:57.921+0000 7f549effd700 1 -- 172.21.15.96:0/3687381999 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f54a8011960 con 0x7f54b00fb960 2024-03-24T10:38:58.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:58.118+0000 7f54add9b700 1 -- 172.21.15.96:0/3687381999 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f54980368d0 msgr2=0x7f5498038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/2406899470 2024-03-24T10:38:58.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:58.118+0000 7f54add9b700 1 --2- 172.21.15.96:0/3687381999 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f54980368d0 0x7f5498038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-24T10:38:58.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:58.184+0000 7f54b4a4a700 1 -- 172.21.15.96:0/3687381999 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f54b005f5f0 con 0x7f54b00fb960 2024-03-24T10:38:58.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:58.185+0000 7f549effd700 1 -- 172.21.15.96:0/3687381999 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f54a8011370 con 0x7f54b00fb960 2024-03-24T10:38:58.186 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:38:58.187 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":1,"fsid":"386110d4-e9ca-11ee-95cd-87774f69a715","modified":"2024-03-24T10:36:06.632144Z","created":"2024-03-24T10:36:06.632144Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi094","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-24T10:38:58.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:58.187+0000 7f54b4a4a700 1 -- 172.21.15.96:0/3687381999 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f54980368d0 msgr2=0x7f5498038d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-24T10:38:58.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:58.187+0000 7f54b4a4a700 1 --2- 172.21.15.96:0/3687381999 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f54980368d0 0x7f5498038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:58.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:58.187+0000 7f54b4a4a700 1 -- 172.21.15.96:0/3687381999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54b00fb960 msgr2=0x7f54b0066860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:38:58.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:58.187+0000 7f54b4a4a700 1 --2- 172.21.15.96:0/3687381999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54b00fb960 0x7f54b0066860 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f54a80096b0 tx=0x7f54a8009e30 comp rx=0 tx=0).stop 2024-03-24T10:38:58.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:58.187+0000 7f54b4a4a700 1 -- 172.21.15.96:0/3687381999 shutdown_connections 2024-03-24T10:38:58.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:58.187+0000 7f54b4a4a700 1 --2- 172.21.15.96:0/3687381999 >> [v2:172.21.15.94:6800/2406899470,v1:172.21.15.94:6801/2406899470] conn(0x7f54980368d0 0x7f5498038d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:58.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:58.187+0000 7f54b4a4a700 1 --2- 172.21.15.96:0/3687381999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54b00fb960 0x7f54b0066860 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:38:58.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:58.187+0000 7f54b4a4a700 1 -- 172.21.15.96:0/3687381999 >> 172.21.15.96:0/3687381999 conn(0x7f54b00f70c0 msgr2=0x7f54b0102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:38:58.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:58.187+0000 7f54b4a4a700 1 -- 172.21.15.96:0/3687381999 shutdown_connections 2024-03-24T10:38:58.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:38:58.187+0000 7f54b4a4a700 1 -- 172.21.15.96:0/3687381999 wait complete. 2024-03-24T10:38:58.189 INFO:teuthology.orchestra.run.smithi096.stderr:dumped monmap epoch 1 2024-03-24T10:38:58.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:38:58 smithi094 ceph-mon[29446]: from='client.? 172.21.15.96:0/3687381999' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T10:38:59.796 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-24T10:38:59.796 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T10:39:00.037 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T10:39:00.084 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T10:39:01.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:00 smithi094 ceph-mon[29446]: Active manager daemon smithi094.yaethu restarted 2024-03-24T10:39:01.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:00 smithi094 ceph-mon[29446]: Activating manager daemon smithi094.yaethu 2024-03-24T10:39:01.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:00 smithi094 ceph-mon[29446]: osdmap e5: 0 total, 0 up, 0 in 2024-03-24T10:39:01.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:00 smithi094 ceph-mon[29446]: mgrmap e15: smithi094.yaethu(active, starting, since 0.002817s) 2024-03-24T10:39:01.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:00 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata", "id": "smithi094"}]: dispatch 2024-03-24T10:39:01.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:00 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mgr metadata", "who": "smithi094.yaethu", "id": "smithi094.yaethu"}]: dispatch 2024-03-24T10:39:01.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:00 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-24T10:39:01.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:00 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-24T10:39:01.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:00 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-24T10:39:01.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:00 smithi094 ceph-mon[29446]: Manager daemon smithi094.yaethu is now available 2024-03-24T10:39:01.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:00 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:39:01.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:00 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:39:01.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:01.587+0000 7f73f37b8700 1 -- 172.21.15.96:0/230864028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73ec0fca50 msgr2=0x7f73ec0fce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:39:01.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:01.587+0000 7f73f37b8700 1 --2- 172.21.15.96:0/230864028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73ec0fca50 0x7f73ec0fce60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f73e80048c0 tx=0x7f73e8009e30 comp rx=0 tx=0).stop 2024-03-24T10:39:01.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:01.587+0000 7f73f37b8700 1 -- 172.21.15.96:0/230864028 shutdown_connections 2024-03-24T10:39:01.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:01.587+0000 7f73f37b8700 1 --2- 172.21.15.96:0/230864028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73ec0fca50 0x7f73ec0fce60 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:39:01.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:01.587+0000 7f73f37b8700 1 -- 172.21.15.96:0/230864028 >> 172.21.15.96:0/230864028 conn(0x7f73ec0f8190 msgr2=0x7f73ec0fa5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:39:01.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:01.587+0000 7f73f37b8700 1 -- 172.21.15.96:0/230864028 shutdown_connections 2024-03-24T10:39:01.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:01.587+0000 7f73f37b8700 1 -- 172.21.15.96:0/230864028 wait complete. 2024-03-24T10:39:01.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:01.588+0000 7f73f37b8700 1 Processor -- start 2024-03-24T10:39:01.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:01.588+0000 7f73f37b8700 1 -- start start 2024-03-24T10:39:01.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:01.588+0000 7f73f37b8700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73ec0fca50 0x7f73ec10d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:39:01.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:01.588+0000 7f73f37b8700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73ec10d5f0 con 0x7f73ec0fca50 2024-03-24T10:39:01.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:01.588+0000 7f73f1554700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73ec0fca50 0x7f73ec10d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:39:01.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:01.588+0000 7f73f1554700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73ec0fca50 0x7f73ec10d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.96:33530/0 (socket says 172.21.15.96:33530) 2024-03-24T10:39:01.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:01.588+0000 7f73f1554700 1 -- 172.21.15.96:0/3134757630 learned_addr learned my addr 172.21.15.96:0/3134757630 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-24T10:39:01.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:01.588+0000 7f73f1554700 1 -- 172.21.15.96:0/3134757630 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e8005040 con 0x7f73ec0fca50 2024-03-24T10:39:01.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:01.589+0000 7f73f1554700 1 --2- 172.21.15.96:0/3134757630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73ec0fca50 0x7f73ec10d0b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f73e80096b0 tx=0x7f73e8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:39:01.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:01.589+0000 7f73e1ffb700 1 -- 172.21.15.96:0/3134757630 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f73e800bed0 con 0x7f73ec0fca50 2024-03-24T10:39:01.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:01.589+0000 7f73e1ffb700 1 -- 172.21.15.96:0/3134757630 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f73e800b5c0 con 0x7f73ec0fca50 2024-03-24T10:39:01.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:01.589+0000 7f73f37b8700 1 -- 172.21.15.96:0/3134757630 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73ec10d7f0 con 0x7f73ec0fca50 2024-03-24T10:39:01.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:01.589+0000 7f73e1ffb700 1 -- 172.21.15.96:0/3134757630 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f73e801c5b0 con 0x7f73ec0fca50 2024-03-24T10:39:01.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:01.589+0000 7f73f37b8700 1 -- 172.21.15.96:0/3134757630 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73ec10e360 con 0x7f73ec0fca50 2024-03-24T10:39:01.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:01.590+0000 7f73e1ffb700 1 -- 172.21.15.96:0/3134757630 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 15) v1 ==== 43162+0+0 (secure 0 0 0) 0x7f73e801c820 con 0x7f73ec0fca50 2024-03-24T10:39:01.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:01.590+0000 7f73f37b8700 1 -- 172.21.15.96:0/3134757630 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73ec102f60 con 0x7f73ec0fca50 2024-03-24T10:39:01.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:01.590+0000 7f73e1ffb700 1 -- 172.21.15.96:0/3134757630 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f73e80497c0 con 0x7f73ec0fca50 2024-03-24T10:39:01.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:01.595+0000 7f73e1ffb700 1 -- 172.21.15.96:0/3134757630 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f73e8023300 con 0x7f73ec0fca50 2024-03-24T10:39:01.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:01.833+0000 7f73e1ffb700 1 -- 172.21.15.96:0/3134757630 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 16) v1 ==== 43289+0+0 (secure 0 0 0) 0x7f73e802bd30 con 0x7f73ec0fca50 2024-03-24T10:39:01.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:01.833+0000 7f73e1ffb700 1 --2- 172.21.15.96:0/3134757630 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f73d8037920 0x7f73d8039d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:39:01.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:01.833+0000 7f73f0d53700 1 --2- 172.21.15.96:0/3134757630 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f73d8037920 0x7f73d8039d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:39:01.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:01.833+0000 7f73f0d53700 1 --2- 172.21.15.96:0/3134757630 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f73d8037920 0x7f73d8039d00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f73dc009990 tx=0x7f73dc006e40 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:39:01.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:01.875+0000 7f73f37b8700 1 -- 172.21.15.96:0/3134757630 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f73ec10ed70 con 0x7f73ec0fca50 2024-03-24T10:39:01.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:01.876+0000 7f73e1ffb700 1 -- 172.21.15.96:0/3134757630 <== mon.0 v2:172.21.15.94:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f73e8011370 con 0x7f73ec0fca50 2024-03-24T10:39:01.876 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:39:01.877 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":1,"fsid":"386110d4-e9ca-11ee-95cd-87774f69a715","modified":"2024-03-24T10:36:06.632144Z","created":"2024-03-24T10:36:06.632144Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi094","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-24T10:39:01.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:01.878+0000 7f73f37b8700 1 -- 172.21.15.96:0/3134757630 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f73d8037920 msgr2=0x7f73d8039d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:39:01.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:01.878+0000 7f73f37b8700 1 --2- 172.21.15.96:0/3134757630 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f73d8037920 0x7f73d8039d00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f73dc009990 tx=0x7f73dc006e40 comp rx=0 tx=0).stop 2024-03-24T10:39:01.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:01.878+0000 7f73f37b8700 1 -- 172.21.15.96:0/3134757630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73ec0fca50 msgr2=0x7f73ec10d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:39:01.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:01.878+0000 7f73f37b8700 1 --2- 172.21.15.96:0/3134757630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73ec0fca50 0x7f73ec10d0b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f73e80096b0 tx=0x7f73e8009e30 comp rx=0 tx=0).stop 2024-03-24T10:39:01.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:01.878+0000 7f73f37b8700 1 -- 172.21.15.96:0/3134757630 shutdown_connections 2024-03-24T10:39:01.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:01.878+0000 7f73f37b8700 1 --2- 172.21.15.96:0/3134757630 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f73d8037920 0x7f73d8039d00 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:39:01.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:01.878+0000 7f73f37b8700 1 --2- 172.21.15.96:0/3134757630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73ec0fca50 0x7f73ec10d0b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:39:01.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:01.878+0000 7f73f37b8700 1 -- 172.21.15.96:0/3134757630 >> 172.21.15.96:0/3134757630 conn(0x7f73ec0f8190 msgr2=0x7f73ec103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:39:01.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:01.878+0000 7f73f37b8700 1 -- 172.21.15.96:0/3134757630 shutdown_connections 2024-03-24T10:39:01.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:01.878+0000 7f73f37b8700 1 -- 172.21.15.96:0/3134757630 wait complete. 2024-03-24T10:39:01.880 INFO:teuthology.orchestra.run.smithi096.stderr:dumped monmap epoch 1 2024-03-24T10:39:01.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:01 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:01.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:01 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:39:01.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:01 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi094.yaethu/mirror_snapshot_schedule"}]: dispatch 2024-03-24T10:39:01.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:01 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi094.yaethu/trash_purge_schedule"}]: dispatch 2024-03-24T10:39:01.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:01 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:01.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:01 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:01.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:01 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:01.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:01 smithi094 ceph-mon[29446]: mgrmap e16: smithi094.yaethu(active, since 1.005s) 2024-03-24T10:39:01.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:01 smithi094 ceph-mon[29446]: from='client.? 172.21.15.96:0/3134757630' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T10:39:03.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:02 smithi094 ceph-mon[29446]: [24/Mar/2024:10:39:01] ENGINE Bus STARTING 2024-03-24T10:39:03.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:02 smithi094 ceph-mon[29446]: [24/Mar/2024:10:39:01] ENGINE Serving on https://172.21.15.94:7150 2024-03-24T10:39:03.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:02 smithi094 ceph-mon[29446]: [24/Mar/2024:10:39:01] ENGINE Bus STARTED 2024-03-24T10:39:03.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:02 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:03.500 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-24T10:39:03.501 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T10:39:03.923 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T10:39:04.308 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T10:39:04.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:04 smithi094 ceph-mon[29446]: mgrmap e17: smithi094.yaethu(active, since 2s) 2024-03-24T10:39:06.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:05 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:06.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:05 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:06.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:05 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:06.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:05 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:06.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:05 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "who": "osd/host:smithi094", "name": "osd_memory_target"}]: dispatch 2024-03-24T10:39:06.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:05 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:06.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:05 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:06.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:05 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:06.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:05 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:06.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:05 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "who": "osd/host:smithi096", "name": "osd_memory_target"}]: dispatch 2024-03-24T10:39:06.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:05 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:39:06.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:05 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:39:06.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:06.647+0000 7f5ac9e06700 1 -- 172.21.15.96:0/946366966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ac4102ae0 msgr2=0x7f5ac4102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:39:06.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:06.647+0000 7f5ac9e06700 1 --2- 172.21.15.96:0/946366966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ac4102ae0 0x7f5ac4102eb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5ab80048c0 tx=0x7f5ab8009e30 comp rx=0 tx=0).stop 2024-03-24T10:39:06.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:06.647+0000 7f5ac9e06700 1 -- 172.21.15.96:0/946366966 shutdown_connections 2024-03-24T10:39:06.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:06.647+0000 7f5ac9e06700 1 --2- 172.21.15.96:0/946366966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ac4102ae0 0x7f5ac4102eb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:39:06.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:06.647+0000 7f5ac9e06700 1 -- 172.21.15.96:0/946366966 >> 172.21.15.96:0/946366966 conn(0x7f5ac40f85e0 msgr2=0x7f5ac40fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:39:06.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:06.647+0000 7f5ac9e06700 1 -- 172.21.15.96:0/946366966 shutdown_connections 2024-03-24T10:39:06.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:06.647+0000 7f5ac9e06700 1 -- 172.21.15.96:0/946366966 wait complete. 2024-03-24T10:39:06.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:06.648+0000 7f5ac9e06700 1 Processor -- start 2024-03-24T10:39:06.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:06.648+0000 7f5ac9e06700 1 -- start start 2024-03-24T10:39:06.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:06.648+0000 7f5ac9e06700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ac4102ae0 0x7f5ac41131e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:39:06.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:06.648+0000 7f5ac9e06700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac4113720 con 0x7f5ac4102ae0 2024-03-24T10:39:06.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:06.649+0000 7f5ac37fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ac4102ae0 0x7f5ac41131e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:39:06.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:06.649+0000 7f5ac37fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ac4102ae0 0x7f5ac41131e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.96:49336/0 (socket says 172.21.15.96:49336) 2024-03-24T10:39:06.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:06.649+0000 7f5ac37fe700 1 -- 172.21.15.96:0/2131055590 learned_addr learned my addr 172.21.15.96:0/2131055590 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-24T10:39:06.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:06.649+0000 7f5ac37fe700 1 -- 172.21.15.96:0/2131055590 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ab8005040 con 0x7f5ac4102ae0 2024-03-24T10:39:06.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:06.649+0000 7f5ac37fe700 1 --2- 172.21.15.96:0/2131055590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ac4102ae0 0x7f5ac41131e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5ab800b820 tx=0x7f5ab8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:39:06.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:06.649+0000 7f5aabfff700 1 -- 172.21.15.96:0/2131055590 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5ab80093c0 con 0x7f5ac4102ae0 2024-03-24T10:39:06.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:06.649+0000 7f5aabfff700 1 -- 172.21.15.96:0/2131055590 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5ab8013420 con 0x7f5ac4102ae0 2024-03-24T10:39:06.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:06.649+0000 7f5ac9e06700 1 -- 172.21.15.96:0/2131055590 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ac4113920 con 0x7f5ac4102ae0 2024-03-24T10:39:06.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:06.649+0000 7f5aabfff700 1 -- 172.21.15.96:0/2131055590 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5ab801c5b0 con 0x7f5ac4102ae0 2024-03-24T10:39:06.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:06.649+0000 7f5ac9e06700 1 -- 172.21.15.96:0/2131055590 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ac4113d90 con 0x7f5ac4102ae0 2024-03-24T10:39:06.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:06.650+0000 7f5aabfff700 1 -- 172.21.15.96:0/2131055590 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7f5ab801c7f0 con 0x7f5ac4102ae0 2024-03-24T10:39:06.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:06.650+0000 7f5ac9e06700 1 -- 172.21.15.96:0/2131055590 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ab00052b0 con 0x7f5ac4102ae0 2024-03-24T10:39:06.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:06.650+0000 7f5aabfff700 1 --2- 172.21.15.96:0/2131055590 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f5aac036970 0x7f5aac038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:39:06.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:06.650+0000 7f5aabfff700 1 -- 172.21.15.96:0/2131055590 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f5ab804abf0 con 0x7f5ac4102ae0 2024-03-24T10:39:06.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:06.650+0000 7f5ac2ffd700 1 --2- 172.21.15.96:0/2131055590 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f5aac036970 0x7f5aac038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:39:06.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:06.651+0000 7f5ac2ffd700 1 --2- 172.21.15.96:0/2131055590 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f5aac036970 0x7f5aac038e20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4009990 tx=0x7f5ab4006e40 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:39:06.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:06.655+0000 7f5aabfff700 1 -- 172.21.15.96:0/2131055590 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f5ab8023300 con 0x7f5ac4102ae0 2024-03-24T10:39:06.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:06.931+0000 7f5ac9e06700 1 -- 172.21.15.96:0/2131055590 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f5ab0005560 con 0x7f5ac4102ae0 2024-03-24T10:39:06.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:06.931+0000 7f5aabfff700 1 -- 172.21.15.96:0/2131055590 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f5ab8011b60 con 0x7f5ac4102ae0 2024-03-24T10:39:06.932 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:39:06.932 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":1,"fsid":"386110d4-e9ca-11ee-95cd-87774f69a715","modified":"2024-03-24T10:36:06.632144Z","created":"2024-03-24T10:36:06.632144Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi094","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-24T10:39:06.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:06.933+0000 7f5ac9e06700 1 -- 172.21.15.96:0/2131055590 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f5aac036970 msgr2=0x7f5aac038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:39:06.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:06.933+0000 7f5ac9e06700 1 --2- 172.21.15.96:0/2131055590 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f5aac036970 0x7f5aac038e20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4009990 tx=0x7f5ab4006e40 comp rx=0 tx=0).stop 2024-03-24T10:39:06.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:06.933+0000 7f5ac9e06700 1 -- 172.21.15.96:0/2131055590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ac4102ae0 msgr2=0x7f5ac41131e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:39:06.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:06.933+0000 7f5ac9e06700 1 --2- 172.21.15.96:0/2131055590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ac4102ae0 0x7f5ac41131e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5ab800b820 tx=0x7f5ab8009e30 comp rx=0 tx=0).stop 2024-03-24T10:39:06.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:06.933+0000 7f5ac9e06700 1 -- 172.21.15.96:0/2131055590 shutdown_connections 2024-03-24T10:39:06.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:06.933+0000 7f5ac9e06700 1 --2- 172.21.15.96:0/2131055590 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f5aac036970 0x7f5aac038e20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:39:06.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:06.933+0000 7f5ac9e06700 1 --2- 172.21.15.96:0/2131055590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ac4102ae0 0x7f5ac41131e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:39:06.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:06.933+0000 7f5ac9e06700 1 -- 172.21.15.96:0/2131055590 >> 172.21.15.96:0/2131055590 conn(0x7f5ac40f85e0 msgr2=0x7f5ac40f9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:39:06.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:06.934+0000 7f5ac9e06700 1 -- 172.21.15.96:0/2131055590 shutdown_connections 2024-03-24T10:39:06.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:06.934+0000 7f5ac9e06700 1 -- 172.21.15.96:0/2131055590 wait complete. 2024-03-24T10:39:06.935 INFO:teuthology.orchestra.run.smithi096.stderr:dumped monmap epoch 1 2024-03-24T10:39:07.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:06 smithi094 ceph-mon[29446]: Updating smithi094:/etc/ceph/ceph.conf 2024-03-24T10:39:07.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:06 smithi094 ceph-mon[29446]: Updating smithi096:/etc/ceph/ceph.conf 2024-03-24T10:39:07.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:06 smithi094 ceph-mon[29446]: Updating smithi096:/var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T10:39:07.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:06 smithi094 ceph-mon[29446]: Updating smithi094:/var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T10:39:07.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:06 smithi094 ceph-mon[29446]: Updating smithi096:/etc/ceph/ceph.client.admin.keyring 2024-03-24T10:39:07.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:06 smithi094 ceph-mon[29446]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-03-24T10:39:07.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:06 smithi094 ceph-mon[29446]: Updating smithi096:/var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T10:39:07.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:06 smithi094 ceph-mon[29446]: Updating smithi094:/var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T10:39:07.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:06 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:07.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:06 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:07.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:06 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:07.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:06 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:07.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:06 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:07.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:06 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi096", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-24T10:39:07.249 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:06 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi096", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-03-24T10:39:07.249 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:06 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:39:08.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:07 smithi094 ceph-mon[29446]: Deploying daemon crash.smithi096 on smithi096 2024-03-24T10:39:08.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:07 smithi094 ceph-mon[29446]: from='client.? 172.21.15.96:0/2131055590' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T10:39:08.586 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-24T10:39:08.587 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T10:39:08.849 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T10:39:10.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:10.383+0000 7f673f101700 1 -- 172.21.15.96:0/4190985397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6738101960 msgr2=0x7f6738101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:39:10.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:10.383+0000 7f673f101700 1 --2- 172.21.15.96:0/4190985397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6738101960 0x7f6738101d30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6730009c60 tx=0x7f6730004d10 comp rx=0 tx=0).stop 2024-03-24T10:39:10.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:10.383+0000 7f673f101700 1 -- 172.21.15.96:0/4190985397 shutdown_connections 2024-03-24T10:39:10.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:10.383+0000 7f673f101700 1 --2- 172.21.15.96:0/4190985397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6738101960 0x7f6738101d30 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:39:10.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:10.383+0000 7f673f101700 1 -- 172.21.15.96:0/4190985397 >> 172.21.15.96:0/4190985397 conn(0x7f67380f6c30 msgr2=0x7f67380f9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:39:10.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:10.383+0000 7f673f101700 1 -- 172.21.15.96:0/4190985397 shutdown_connections 2024-03-24T10:39:10.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:10.383+0000 7f673f101700 1 -- 172.21.15.96:0/4190985397 wait complete. 2024-03-24T10:39:10.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:10.383+0000 7f673f101700 1 Processor -- start 2024-03-24T10:39:10.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:10.384+0000 7f673f101700 1 -- start start 2024-03-24T10:39:10.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:10.384+0000 7f673f101700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6738101960 0x7f673810e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:39:10.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:10.384+0000 7f673f101700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f673810e680 con 0x7f6738101960 2024-03-24T10:39:10.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:10.384+0000 7f673ce9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6738101960 0x7f673810e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:39:10.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:10.384+0000 7f673ce9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6738101960 0x7f673810e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.96:49356/0 (socket says 172.21.15.96:49356) 2024-03-24T10:39:10.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:10.384+0000 7f673ce9d700 1 -- 172.21.15.96:0/4135252473 learned_addr learned my addr 172.21.15.96:0/4135252473 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-24T10:39:10.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:10.385+0000 7f673ce9d700 1 -- 172.21.15.96:0/4135252473 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6730009940 con 0x7f6738101960 2024-03-24T10:39:10.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:10.385+0000 7f673ce9d700 1 --2- 172.21.15.96:0/4135252473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6738101960 0x7f673810e140 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f67300096b0 tx=0x7f6730004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:39:10.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:10.385+0000 7f67357fa700 1 -- 172.21.15.96:0/4135252473 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6730007b30 con 0x7f6738101960 2024-03-24T10:39:10.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:10.385+0000 7f673f101700 1 -- 172.21.15.96:0/4135252473 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f673810e880 con 0x7f6738101960 2024-03-24T10:39:10.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:10.385+0000 7f67357fa700 1 -- 172.21.15.96:0/4135252473 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f673001d910 con 0x7f6738101960 2024-03-24T10:39:10.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:10.385+0000 7f67357fa700 1 -- 172.21.15.96:0/4135252473 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f67300128b0 con 0x7f6738101960 2024-03-24T10:39:10.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:10.385+0000 7f673f101700 1 -- 172.21.15.96:0/4135252473 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f673810ed70 con 0x7f6738101960 2024-03-24T10:39:10.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:10.386+0000 7f67357fa700 1 -- 172.21.15.96:0/4135252473 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7f6730019040 con 0x7f6738101960 2024-03-24T10:39:10.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:10.386+0000 7f673f101700 1 -- 172.21.15.96:0/4135252473 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6738100cd0 con 0x7f6738101960 2024-03-24T10:39:10.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:10.386+0000 7f67357fa700 1 --2- 172.21.15.96:0/4135252473 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f6724036970 0x7f6724038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:39:10.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:10.386+0000 7f67357fa700 1 -- 172.21.15.96:0/4135252473 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f673004a3c0 con 0x7f6738101960 2024-03-24T10:39:10.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:10.386+0000 7f6737fff700 1 --2- 172.21.15.96:0/4135252473 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f6724036970 0x7f6724038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:39:10.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:10.387+0000 7f6737fff700 1 --2- 172.21.15.96:0/4135252473 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f6724036970 0x7f6724038e20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6720009990 tx=0x7f6720006e40 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:39:10.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:10.391+0000 7f67357fa700 1 -- 172.21.15.96:0/4135252473 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f673002eb40 con 0x7f6738101960 2024-03-24T10:39:10.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:10.672+0000 7f673f101700 1 -- 172.21.15.96:0/4135252473 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6738105cc0 con 0x7f6738101960 2024-03-24T10:39:10.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:10.673+0000 7f67357fa700 1 -- 172.21.15.96:0/4135252473 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f6730017070 con 0x7f6738101960 2024-03-24T10:39:10.674 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:39:10.674 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":1,"fsid":"386110d4-e9ca-11ee-95cd-87774f69a715","modified":"2024-03-24T10:36:06.632144Z","created":"2024-03-24T10:36:06.632144Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi094","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-24T10:39:10.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:10.675+0000 7f673f101700 1 -- 172.21.15.96:0/4135252473 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f6724036970 msgr2=0x7f6724038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:39:10.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:10.675+0000 7f673f101700 1 --2- 172.21.15.96:0/4135252473 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f6724036970 0x7f6724038e20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6720009990 tx=0x7f6720006e40 comp rx=0 tx=0).stop 2024-03-24T10:39:10.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:10.675+0000 7f673f101700 1 -- 172.21.15.96:0/4135252473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6738101960 msgr2=0x7f673810e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:39:10.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:10.675+0000 7f673f101700 1 --2- 172.21.15.96:0/4135252473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6738101960 0x7f673810e140 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f67300096b0 tx=0x7f6730004d10 comp rx=0 tx=0).stop 2024-03-24T10:39:10.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:10.675+0000 7f673f101700 1 -- 172.21.15.96:0/4135252473 shutdown_connections 2024-03-24T10:39:10.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:10.675+0000 7f673f101700 1 --2- 172.21.15.96:0/4135252473 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f6724036970 0x7f6724038e20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:39:10.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:10.675+0000 7f673f101700 1 --2- 172.21.15.96:0/4135252473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6738101960 0x7f673810e140 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:39:10.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:10.675+0000 7f673f101700 1 -- 172.21.15.96:0/4135252473 >> 172.21.15.96:0/4135252473 conn(0x7f67380f6c30 msgr2=0x7f67380f83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:39:10.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:10.675+0000 7f673f101700 1 -- 172.21.15.96:0/4135252473 shutdown_connections 2024-03-24T10:39:10.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:10.675+0000 7f673f101700 1 -- 172.21.15.96:0/4135252473 wait complete. 2024-03-24T10:39:10.678 INFO:teuthology.orchestra.run.smithi096.stderr:dumped monmap epoch 1 2024-03-24T10:39:10.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:10 smithi094 ceph-mon[29446]: from='client.? 172.21.15.96:0/4135252473' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T10:39:12.265 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-24T10:39:12.265 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T10:39:12.537 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T10:39:12.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:12 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:12.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:12 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:12.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:12 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:12.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:12 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:12.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:12 smithi094 ceph-mon[29446]: Deploying daemon node-exporter.smithi096 on smithi096 2024-03-24T10:39:14.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:14.598+0000 7f1c33a43700 1 -- 172.21.15.96:0/1151516995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c2c102bd0 msgr2=0x7f1c2c102fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:39:14.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:14.598+0000 7f1c33a43700 1 --2- 172.21.15.96:0/1151516995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c2c102bd0 0x7f1c2c102fa0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1c280048c0 tx=0x7f1c28009e30 comp rx=0 tx=0).stop 2024-03-24T10:39:14.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:14.598+0000 7f1c33a43700 1 -- 172.21.15.96:0/1151516995 shutdown_connections 2024-03-24T10:39:14.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:14.598+0000 7f1c33a43700 1 --2- 172.21.15.96:0/1151516995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c2c102bd0 0x7f1c2c102fa0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:39:14.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:14.598+0000 7f1c33a43700 1 -- 172.21.15.96:0/1151516995 >> 172.21.15.96:0/1151516995 conn(0x7f1c2c0f85e0 msgr2=0x7f1c2c0fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:39:14.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:14.598+0000 7f1c33a43700 1 -- 172.21.15.96:0/1151516995 shutdown_connections 2024-03-24T10:39:14.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:14.598+0000 7f1c33a43700 1 -- 172.21.15.96:0/1151516995 wait complete. 2024-03-24T10:39:14.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:14.599+0000 7f1c33a43700 1 Processor -- start 2024-03-24T10:39:14.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:14.599+0000 7f1c33a43700 1 -- start start 2024-03-24T10:39:14.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:14.599+0000 7f1c33a43700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c2c102bd0 0x7f1c2c113230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:39:14.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:14.599+0000 7f1c33a43700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2c113770 con 0x7f1c2c102bd0 2024-03-24T10:39:14.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:14.599+0000 7f1c317df700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c2c102bd0 0x7f1c2c113230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:39:14.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:14.599+0000 7f1c317df700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c2c102bd0 0x7f1c2c113230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.96:49382/0 (socket says 172.21.15.96:49382) 2024-03-24T10:39:14.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:14.599+0000 7f1c317df700 1 -- 172.21.15.96:0/3144791262 learned_addr learned my addr 172.21.15.96:0/3144791262 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-24T10:39:14.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:14.599+0000 7f1c317df700 1 -- 172.21.15.96:0/3144791262 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c28005040 con 0x7f1c2c102bd0 2024-03-24T10:39:14.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:14.600+0000 7f1c317df700 1 --2- 172.21.15.96:0/3144791262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c2c102bd0 0x7f1c2c113230 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1c2800b820 tx=0x7f1c28009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:39:14.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:14.600+0000 7f1c21ffb700 1 -- 172.21.15.96:0/3144791262 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1c280093c0 con 0x7f1c2c102bd0 2024-03-24T10:39:14.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:14.600+0000 7f1c33a43700 1 -- 172.21.15.96:0/3144791262 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c2c113970 con 0x7f1c2c102bd0 2024-03-24T10:39:14.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:14.600+0000 7f1c21ffb700 1 -- 172.21.15.96:0/3144791262 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1c28013420 con 0x7f1c2c102bd0 2024-03-24T10:39:14.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:14.600+0000 7f1c21ffb700 1 -- 172.21.15.96:0/3144791262 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1c2801c5b0 con 0x7f1c2c102bd0 2024-03-24T10:39:14.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:14.600+0000 7f1c33a43700 1 -- 172.21.15.96:0/3144791262 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c2c113da0 con 0x7f1c2c102bd0 2024-03-24T10:39:14.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:14.601+0000 7f1c21ffb700 1 -- 172.21.15.96:0/3144791262 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7f1c2801c7a0 con 0x7f1c2c102bd0 2024-03-24T10:39:14.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:14.601+0000 7f1c21ffb700 1 --2- 172.21.15.96:0/3144791262 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f1c180369c0 0x7f1c18038e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:39:14.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:14.601+0000 7f1c21ffb700 1 -- 172.21.15.96:0/3144791262 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f1c2804aad0 con 0x7f1c2c102bd0 2024-03-24T10:39:14.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:14.601+0000 7f1c33a43700 1 -- 172.21.15.96:0/3144791262 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c2c05f5f0 con 0x7f1c2c102bd0 2024-03-24T10:39:14.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:14.602+0000 7f1c30fde700 1 --2- 172.21.15.96:0/3144791262 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f1c180369c0 0x7f1c18038e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:39:14.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:14.602+0000 7f1c30fde700 1 --2- 172.21.15.96:0/3144791262 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f1c180369c0 0x7f1c18038e70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c009990 tx=0x7f1c1c006e40 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:39:14.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:14.607+0000 7f1c21ffb700 1 -- 172.21.15.96:0/3144791262 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f1c28023300 con 0x7f1c2c102bd0 2024-03-24T10:39:14.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:14.870+0000 7f1c33a43700 1 -- 172.21.15.96:0/3144791262 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f1c2c04bed0 con 0x7f1c2c102bd0 2024-03-24T10:39:14.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:14.870+0000 7f1c21ffb700 1 -- 172.21.15.96:0/3144791262 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f1c28011b60 con 0x7f1c2c102bd0 2024-03-24T10:39:14.871 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:39:14.871 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":1,"fsid":"386110d4-e9ca-11ee-95cd-87774f69a715","modified":"2024-03-24T10:36:06.632144Z","created":"2024-03-24T10:36:06.632144Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi094","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-24T10:39:14.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:14.872+0000 7f1c33a43700 1 -- 172.21.15.96:0/3144791262 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f1c180369c0 msgr2=0x7f1c18038e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:39:14.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:14.872+0000 7f1c33a43700 1 --2- 172.21.15.96:0/3144791262 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f1c180369c0 0x7f1c18038e70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c009990 tx=0x7f1c1c006e40 comp rx=0 tx=0).stop 2024-03-24T10:39:14.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:14.872+0000 7f1c33a43700 1 -- 172.21.15.96:0/3144791262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c2c102bd0 msgr2=0x7f1c2c113230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:39:14.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:14.872+0000 7f1c33a43700 1 --2- 172.21.15.96:0/3144791262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c2c102bd0 0x7f1c2c113230 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1c2800b820 tx=0x7f1c28009e30 comp rx=0 tx=0).stop 2024-03-24T10:39:14.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:14.873+0000 7f1c33a43700 1 -- 172.21.15.96:0/3144791262 shutdown_connections 2024-03-24T10:39:14.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:14.873+0000 7f1c33a43700 1 --2- 172.21.15.96:0/3144791262 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f1c180369c0 0x7f1c18038e70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:39:14.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:14.873+0000 7f1c33a43700 1 --2- 172.21.15.96:0/3144791262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c2c102bd0 0x7f1c2c113230 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:39:14.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:14.873+0000 7f1c33a43700 1 -- 172.21.15.96:0/3144791262 >> 172.21.15.96:0/3144791262 conn(0x7f1c2c0f85e0 msgr2=0x7f1c2c0f9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:39:14.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:14.873+0000 7f1c33a43700 1 -- 172.21.15.96:0/3144791262 shutdown_connections 2024-03-24T10:39:14.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:14.873+0000 7f1c33a43700 1 -- 172.21.15.96:0/3144791262 wait complete. 2024-03-24T10:39:14.874 INFO:teuthology.orchestra.run.smithi096.stderr:dumped monmap epoch 1 2024-03-24T10:39:15.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:14 smithi094 ceph-mon[29446]: from='client.? 172.21.15.96:0/3144791262' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T10:39:16.686 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-24T10:39:16.686 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T10:39:17.168 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T10:39:17.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:16 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:17.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:16 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:39:17.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:16 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:17.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:16 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:17.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:16 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:17.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:16 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:17.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:16 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi096.gfonnu", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T10:39:17.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:16 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi096.gfonnu", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-24T10:39:17.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:16 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T10:39:17.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:16 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:39:17.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:16 smithi094 ceph-mon[29446]: Deploying daemon mgr.smithi096.gfonnu on smithi096 2024-03-24T10:39:19.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:19.767+0000 7fc58b29f700 1 -- 172.21.15.96:0/323034446 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc584105210 msgr2=0x7fc5841055e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:39:19.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:19.767+0000 7fc58b29f700 1 --2- 172.21.15.96:0/323034446 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc584105210 0x7fc5841055e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc5800048c0 tx=0x7fc580009e30 comp rx=0 tx=0).stop 2024-03-24T10:39:19.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:19.767+0000 7fc58b29f700 1 -- 172.21.15.96:0/323034446 shutdown_connections 2024-03-24T10:39:19.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:19.767+0000 7fc58b29f700 1 --2- 172.21.15.96:0/323034446 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc584105210 0x7fc5841055e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:39:19.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:19.767+0000 7fc58b29f700 1 -- 172.21.15.96:0/323034446 >> 172.21.15.96:0/323034446 conn(0x7fc584067f20 msgr2=0x7fc584068320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:39:19.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:19.767+0000 7fc58b29f700 1 -- 172.21.15.96:0/323034446 shutdown_connections 2024-03-24T10:39:19.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:19.767+0000 7fc58b29f700 1 -- 172.21.15.96:0/323034446 wait complete. 2024-03-24T10:39:19.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:19.768+0000 7fc58b29f700 1 Processor -- start 2024-03-24T10:39:19.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:19.768+0000 7fc58b29f700 1 -- start start 2024-03-24T10:39:19.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:19.768+0000 7fc58b29f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc584105210 0x7fc58411a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:39:19.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:19.768+0000 7fc58b29f700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58411ad60 con 0x7fc584105210 2024-03-24T10:39:19.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:19.768+0000 7fc58903b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc584105210 0x7fc58411a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:39:19.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:19.768+0000 7fc58903b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc584105210 0x7fc58411a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.96:47664/0 (socket says 172.21.15.96:47664) 2024-03-24T10:39:19.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:19.768+0000 7fc58903b700 1 -- 172.21.15.96:0/3923570479 learned_addr learned my addr 172.21.15.96:0/3923570479 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-24T10:39:19.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:19.768+0000 7fc58903b700 1 -- 172.21.15.96:0/3923570479 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc580005040 con 0x7fc584105210 2024-03-24T10:39:19.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:19.769+0000 7fc58903b700 1 --2- 172.21.15.96:0/3923570479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc584105210 0x7fc58411a820 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc58000b890 tx=0x7fc580009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:39:19.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:19.769+0000 7fc579ffb700 1 -- 172.21.15.96:0/3923570479 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc580009950 con 0x7fc584105210 2024-03-24T10:39:19.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:19.769+0000 7fc58b29f700 1 -- 172.21.15.96:0/3923570479 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc58411af60 con 0x7fc584105210 2024-03-24T10:39:19.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:19.769+0000 7fc579ffb700 1 -- 172.21.15.96:0/3923570479 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc580004e60 con 0x7fc584105210 2024-03-24T10:39:19.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:19.769+0000 7fc579ffb700 1 -- 172.21.15.96:0/3923570479 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc580013630 con 0x7fc584105210 2024-03-24T10:39:19.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:19.769+0000 7fc58b29f700 1 -- 172.21.15.96:0/3923570479 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc58411b3f0 con 0x7fc584105210 2024-03-24T10:39:19.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:19.770+0000 7fc58b29f700 1 -- 172.21.15.96:0/3923570479 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc58406e7b0 con 0x7fc584105210 2024-03-24T10:39:19.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:19.770+0000 7fc579ffb700 1 -- 172.21.15.96:0/3923570479 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7fc580013860 con 0x7fc584105210 2024-03-24T10:39:19.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:19.770+0000 7fc579ffb700 1 --2- 172.21.15.96:0/3923570479 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fc5700369c0 0x7fc570038e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:39:19.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:19.770+0000 7fc579ffb700 1 -- 172.21.15.96:0/3923570479 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fc58004c780 con 0x7fc584105210 2024-03-24T10:39:19.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:19.771+0000 7fc58883a700 1 --2- 172.21.15.96:0/3923570479 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fc5700369c0 0x7fc570038e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:39:19.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:19.771+0000 7fc58883a700 1 --2- 172.21.15.96:0/3923570479 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fc5700369c0 0x7fc570038e70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc574009990 tx=0x7fc574006e40 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:39:19.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:19.775+0000 7fc579ffb700 1 -- 172.21.15.96:0/3923570479 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fc580026cf0 con 0x7fc584105210 2024-03-24T10:39:20.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:20.066+0000 7fc58b29f700 1 -- 172.21.15.96:0/3923570479 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc58404bed0 con 0x7fc584105210 2024-03-24T10:39:20.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:20.066+0000 7fc579ffb700 1 -- 172.21.15.96:0/3923570479 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fc580027e40 con 0x7fc584105210 2024-03-24T10:39:20.067 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:39:20.067 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":1,"fsid":"386110d4-e9ca-11ee-95cd-87774f69a715","modified":"2024-03-24T10:36:06.632144Z","created":"2024-03-24T10:36:06.632144Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi094","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-24T10:39:20.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:20.068+0000 7fc58b29f700 1 -- 172.21.15.96:0/3923570479 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fc5700369c0 msgr2=0x7fc570038e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:39:20.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:20.068+0000 7fc58b29f700 1 --2- 172.21.15.96:0/3923570479 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fc5700369c0 0x7fc570038e70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc574009990 tx=0x7fc574006e40 comp rx=0 tx=0).stop 2024-03-24T10:39:20.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:20.068+0000 7fc58b29f700 1 -- 172.21.15.96:0/3923570479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc584105210 msgr2=0x7fc58411a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:39:20.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:20.068+0000 7fc58b29f700 1 --2- 172.21.15.96:0/3923570479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc584105210 0x7fc58411a820 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc58000b890 tx=0x7fc580009e30 comp rx=0 tx=0).stop 2024-03-24T10:39:20.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:20.068+0000 7fc58b29f700 1 -- 172.21.15.96:0/3923570479 shutdown_connections 2024-03-24T10:39:20.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:20.068+0000 7fc58b29f700 1 --2- 172.21.15.96:0/3923570479 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fc5700369c0 0x7fc570038e70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:39:20.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:20.068+0000 7fc58b29f700 1 --2- 172.21.15.96:0/3923570479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc584105210 0x7fc58411a820 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:39:20.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:20.068+0000 7fc58b29f700 1 -- 172.21.15.96:0/3923570479 >> 172.21.15.96:0/3923570479 conn(0x7fc584067f20 msgr2=0x7fc58410ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:39:20.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:20.068+0000 7fc58b29f700 1 -- 172.21.15.96:0/3923570479 shutdown_connections 2024-03-24T10:39:20.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:20.068+0000 7fc58b29f700 1 -- 172.21.15.96:0/3923570479 wait complete. 2024-03-24T10:39:20.070 INFO:teuthology.orchestra.run.smithi096.stderr:dumped monmap epoch 1 2024-03-24T10:39:20.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:20 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:20.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:20 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:20.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:20 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:20.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:20 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:20.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:20 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T10:39:20.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:20 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:39:20.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:20 smithi094 ceph-mon[29446]: from='client.? 172.21.15.96:0/3923570479' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T10:39:21.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:21 smithi094 ceph-mon[29446]: Deploying daemon mon.smithi096 on smithi096 2024-03-24T10:39:21.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:21 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:22.119 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-24T10:39:22.120 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T10:39:22.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:22 smithi094 ceph-mon[29446]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:39:22.813 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi096/config 2024-03-24T10:39:24.420 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 podman[38049]: 2024-03-24 10:39:24.25823019 +0000 UTC m=+0.018488590 image pull 837124da804ffab28571a3e286aff1c534a701f228c0d3d7a309fcb128aca1de quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232 2024-03-24T10:39:24.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:24.437+0000 7f490b8f0700 1 -- 172.21.15.96:0/2114465798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4904102b80 msgr2=0x7f4904102f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:39:24.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:24.437+0000 7f490b8f0700 1 --2- 172.21.15.96:0/2114465798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4904102b80 0x7f4904102f50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f48f40048c0 tx=0x7f48f4009e30 comp rx=0 tx=0).stop 2024-03-24T10:39:24.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:24.437+0000 7f490b8f0700 1 -- 172.21.15.96:0/2114465798 shutdown_connections 2024-03-24T10:39:24.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:24.437+0000 7f490b8f0700 1 --2- 172.21.15.96:0/2114465798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4904102b80 0x7f4904102f50 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:39:24.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:24.437+0000 7f490b8f0700 1 -- 172.21.15.96:0/2114465798 >> 172.21.15.96:0/2114465798 conn(0x7f49040f86f0 msgr2=0x7f49040fab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:39:24.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:24.437+0000 7f490b8f0700 1 -- 172.21.15.96:0/2114465798 shutdown_connections 2024-03-24T10:39:24.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:24.437+0000 7f490b8f0700 1 -- 172.21.15.96:0/2114465798 wait complete. 2024-03-24T10:39:24.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:24.438+0000 7f490b8f0700 1 Processor -- start 2024-03-24T10:39:24.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:24.438+0000 7f490b8f0700 1 -- start start 2024-03-24T10:39:24.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:24.438+0000 7f490b8f0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4904102b80 0x7f4904113280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:39:24.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:24.438+0000 7f490b8f0700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49041137c0 con 0x7f4904102b80 2024-03-24T10:39:24.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:24.438+0000 7f490968c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4904102b80 0x7f4904113280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:39:24.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:24.438+0000 7f490968c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4904102b80 0x7f4904113280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.96:47684/0 (socket says 172.21.15.96:47684) 2024-03-24T10:39:24.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:24.438+0000 7f490968c700 1 -- 172.21.15.96:0/74165748 learned_addr learned my addr 172.21.15.96:0/74165748 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-24T10:39:24.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:24.438+0000 7f490968c700 1 -- 172.21.15.96:0/74165748 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f4005040 con 0x7f4904102b80 2024-03-24T10:39:24.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:24.439+0000 7f490968c700 1 --2- 172.21.15.96:0/74165748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4904102b80 0x7f4904113280 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f48f400b820 tx=0x7f48f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:39:24.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:24.439+0000 7f48f9ffb700 1 -- 172.21.15.96:0/74165748 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f48f40093c0 con 0x7f4904102b80 2024-03-24T10:39:24.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:24.439+0000 7f48f9ffb700 1 -- 172.21.15.96:0/74165748 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f48f4013420 con 0x7f4904102b80 2024-03-24T10:39:24.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:24.439+0000 7f490b8f0700 1 -- 172.21.15.96:0/74165748 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49041139c0 con 0x7f4904102b80 2024-03-24T10:39:24.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:24.439+0000 7f48f9ffb700 1 -- 172.21.15.96:0/74165748 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f48f401c5b0 con 0x7f4904102b80 2024-03-24T10:39:24.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:24.439+0000 7f490b8f0700 1 -- 172.21.15.96:0/74165748 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4904113eb0 con 0x7f4904102b80 2024-03-24T10:39:24.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:24.440+0000 7f48f9ffb700 1 -- 172.21.15.96:0/74165748 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7f48f401c7f0 con 0x7f4904102b80 2024-03-24T10:39:24.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:24.440+0000 7f48f9ffb700 1 --2- 172.21.15.96:0/74165748 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f48f0036970 0x7f48f0038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:39:24.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:24.440+0000 7f48f9ffb700 1 -- 172.21.15.96:0/74165748 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f48f4049bd0 con 0x7f4904102b80 2024-03-24T10:39:24.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:24.440+0000 7f490b8f0700 1 -- 172.21.15.96:0/74165748 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49040fe2d0 con 0x7f4904102b80 2024-03-24T10:39:24.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:24.440+0000 7f4908e8b700 1 --2- 172.21.15.96:0/74165748 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f48f0036970 0x7f48f0038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:39:24.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:24.441+0000 7f4908e8b700 1 --2- 172.21.15.96:0/74165748 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f48f0036970 0x7f48f0038e20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4900009990 tx=0x7f4900006e40 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:39:24.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:24.445+0000 7f48f9ffb700 1 -- 172.21.15.96:0/74165748 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f48f4023300 con 0x7f4904102b80 2024-03-24T10:39:24.722 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 podman[38049]: 2024-03-24 10:39:24.511686744 +0000 UTC m=+0.271945137 container create 710030b222cb7d230d798028067f9c7c15263afea0d120799e9f9958789d8e56 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-mon-smithi096, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=quincy-9c4e0c3, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319) 2024-03-24T10:39:24.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:24.721+0000 7f490b8f0700 1 -- 172.21.15.96:0/74165748 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f490404bed0 con 0x7f4904102b80 2024-03-24T10:39:24.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:24.722+0000 7f48f9ffb700 1 -- 172.21.15.96:0/74165748 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f48f4012e40 con 0x7f4904102b80 2024-03-24T10:39:24.724 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:39:24.724 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":1,"fsid":"386110d4-e9ca-11ee-95cd-87774f69a715","modified":"2024-03-24T10:36:06.632144Z","created":"2024-03-24T10:36:06.632144Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi094","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-24T10:39:24.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:24.724+0000 7f490b8f0700 1 -- 172.21.15.96:0/74165748 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f48f0036970 msgr2=0x7f48f0038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:39:24.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:24.724+0000 7f490b8f0700 1 --2- 172.21.15.96:0/74165748 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f48f0036970 0x7f48f0038e20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4900009990 tx=0x7f4900006e40 comp rx=0 tx=0).stop 2024-03-24T10:39:24.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:24.724+0000 7f490b8f0700 1 -- 172.21.15.96:0/74165748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4904102b80 msgr2=0x7f4904113280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:39:24.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:24.724+0000 7f490b8f0700 1 --2- 172.21.15.96:0/74165748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4904102b80 0x7f4904113280 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f48f400b820 tx=0x7f48f4009e30 comp rx=0 tx=0).stop 2024-03-24T10:39:24.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:24.724+0000 7f490b8f0700 1 -- 172.21.15.96:0/74165748 shutdown_connections 2024-03-24T10:39:24.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:24.724+0000 7f490b8f0700 1 --2- 172.21.15.96:0/74165748 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f48f0036970 0x7f48f0038e20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:39:24.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:24.724+0000 7f490b8f0700 1 --2- 172.21.15.96:0/74165748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4904102b80 0x7f4904113280 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:39:24.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:24.724+0000 7f490b8f0700 1 -- 172.21.15.96:0/74165748 >> 172.21.15.96:0/74165748 conn(0x7f49040f86f0 msgr2=0x7f49040f9dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:39:24.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:24.724+0000 7f490b8f0700 1 -- 172.21.15.96:0/74165748 shutdown_connections 2024-03-24T10:39:24.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:24.724+0000 7f490b8f0700 1 -- 172.21.15.96:0/74165748 wait complete. 2024-03-24T10:39:24.726 INFO:teuthology.orchestra.run.smithi096.stderr:dumped monmap epoch 1 2024-03-24T10:39:24.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:24 smithi094 ceph-mon[29446]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:39:25.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 podman[38049]: 2024-03-24 10:39:24.730658396 +0000 UTC m=+0.490916799 container init 710030b222cb7d230d798028067f9c7c15263afea0d120799e9f9958789d8e56 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-mon-smithi096, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, RELEASE=quincy-9c4e0c3, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-24T10:39:25.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 podman[38049]: 2024-03-24 10:39:24.734746218 +0000 UTC m=+0.495004629 container start 710030b222cb7d230d798028067f9c7c15263afea0d120799e9f9958789d8e56 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-mon-smithi096, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-9c4e0c3, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240319) 2024-03-24T10:39:25.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: set uid:gid to 167:167 (ceph:ceph) 2024-03-24T10:39:25.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable), process ceph-mon, pid 2 2024-03-24T10:39:25.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: pidfile_write: ignore empty --pid-file 2024-03-24T10:39:25.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: load: jerasure load: lrc 2024-03-24T10:39:25.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: RocksDB version: 6.15.5 2024-03-24T10:39:25.077 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-03-24T10:39:25.077 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Compile date Mar 21 2024 2024-03-24T10:39:25.077 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: DB SUMMARY 2024-03-24T10:39:25.077 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: DB Session ID: 6RWRFJ6EK8JAT7OB3PLG 2024-03-24T10:39:25.077 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: CURRENT file: CURRENT 2024-03-24T10:39:25.077 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: IDENTITY file: IDENTITY 2024-03-24T10:39:25.077 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: MANIFEST file: MANIFEST-000003 size: 57 Bytes 2024-03-24T10:39:25.077 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi096/store.db dir, Total Num: 0, files: 2024-03-24T10:39:25.077 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi096/store.db: 000004.log size: 511 ; 2024-03-24T10:39:25.077 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.error_if_exists: 0 2024-03-24T10:39:25.077 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.create_if_missing: 0 2024-03-24T10:39:25.077 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.paranoid_checks: 1 2024-03-24T10:39:25.077 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-24T10:39:25.077 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.env: 0x55cd280ffa80 2024-03-24T10:39:25.077 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.fs: Posix File System 2024-03-24T10:39:25.078 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.info_log: 0x55cd28e4fc00 2024-03-24T10:39:25.078 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.max_file_opening_threads: 16 2024-03-24T10:39:25.078 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.statistics: (nil) 2024-03-24T10:39:25.078 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.use_fsync: 0 2024-03-24T10:39:25.078 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.max_log_file_size: 0 2024-03-24T10:39:25.078 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-24T10:39:25.078 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-24T10:39:25.078 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.keep_log_file_num: 1000 2024-03-24T10:39:25.078 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.recycle_log_file_num: 0 2024-03-24T10:39:25.078 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.allow_fallocate: 1 2024-03-24T10:39:25.078 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.allow_mmap_reads: 0 2024-03-24T10:39:25.078 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.allow_mmap_writes: 0 2024-03-24T10:39:25.078 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.use_direct_reads: 0 2024-03-24T10:39:25.078 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-24T10:39:25.078 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.create_missing_column_families: 0 2024-03-24T10:39:25.079 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.db_log_dir: 2024-03-24T10:39:25.079 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi096/store.db 2024-03-24T10:39:25.079 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-24T10:39:25.079 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-24T10:39:25.079 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-24T10:39:25.079 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-24T10:39:25.079 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-24T10:39:25.079 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-24T10:39:25.079 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.advise_random_on_open: 1 2024-03-24T10:39:25.079 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.db_write_buffer_size: 0 2024-03-24T10:39:25.079 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.write_buffer_manager: 0x55cd29bc4ab0 2024-03-24T10:39:25.079 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-24T10:39:25.079 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-03-24T10:39:25.080 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-24T10:39:25.080 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-24T10:39:25.080 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.rate_limiter: (nil) 2024-03-24T10:39:25.080 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-24T10:39:25.080 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.wal_recovery_mode: 2 2024-03-24T10:39:25.080 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.enable_thread_tracking: 0 2024-03-24T10:39:25.080 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.enable_pipelined_write: 0 2024-03-24T10:39:25.080 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.unordered_write: 0 2024-03-24T10:39:25.080 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-24T10:39:25.080 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-24T10:39:25.080 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-24T10:39:25.080 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-24T10:39:25.080 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.row_cache: None 2024-03-24T10:39:25.081 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.wal_filter: None 2024-03-24T10:39:25.081 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-24T10:39:25.082 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.allow_ingest_behind: 0 2024-03-24T10:39:25.083 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.preserve_deletes: 0 2024-03-24T10:39:25.083 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.two_write_queues: 0 2024-03-24T10:39:25.083 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.manual_wal_flush: 0 2024-03-24T10:39:25.083 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.atomic_flush: 0 2024-03-24T10:39:25.083 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-24T10:39:25.083 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-24T10:39:25.083 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-24T10:39:25.083 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.log_readahead_size: 0 2024-03-24T10:39:25.083 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-24T10:39:25.083 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.best_efforts_recovery: 0 2024-03-24T10:39:25.083 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-24T10:39:25.083 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-24T10:39:25.083 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.allow_data_in_errors: 0 2024-03-24T10:39:25.084 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.db_host_id: __hostname__ 2024-03-24T10:39:25.084 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.max_background_jobs: 2 2024-03-24T10:39:25.084 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.max_background_compactions: -1 2024-03-24T10:39:25.084 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.max_subcompactions: 1 2024-03-24T10:39:25.084 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-24T10:39:25.084 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-24T10:39:25.084 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-24T10:39:25.084 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.max_total_wal_size: 0 2024-03-24T10:39:25.084 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-24T10:39:25.084 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-24T10:39:25.084 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-24T10:39:25.084 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-24T10:39:25.084 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.max_open_files: -1 2024-03-24T10:39:25.084 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.bytes_per_sync: 0 2024-03-24T10:39:25.084 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-24T10:39:25.085 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-24T10:39:25.085 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.compaction_readahead_size: 0 2024-03-24T10:39:25.085 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.max_background_flushes: -1 2024-03-24T10:39:25.085 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Compression algorithms supported: 2024-03-24T10:39:25.085 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-24T10:39:25.085 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: kZSTD supported: 0 2024-03-24T10:39:25.085 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: kXpressCompression supported: 0 2024-03-24T10:39:25.085 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: kLZ4HCCompression supported: 1 2024-03-24T10:39:25.085 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: kZlibCompression supported: 1 2024-03-24T10:39:25.085 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: kSnappyCompression supported: 1 2024-03-24T10:39:25.085 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: kLZ4Compression supported: 1 2024-03-24T10:39:25.085 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: kBZip2Compression supported: 0 2024-03-24T10:39:25.085 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-24T10:39:25.085 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi096/store.db/MANIFEST-000003 2024-03-24T10:39:25.085 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-03-24T10:39:25.085 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-24T10:39:25.086 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.merge_operator: 2024-03-24T10:39:25.086 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.compaction_filter: None 2024-03-24T10:39:25.086 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.compaction_filter_factory: None 2024-03-24T10:39:25.086 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.sst_partitioner_factory: None 2024-03-24T10:39:25.086 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-24T10:39:25.086 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-24T10:39:25.086 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55cd28ea4f58) 2024-03-24T10:39:25.086 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: cache_index_and_filter_blocks: 1 2024-03-24T10:39:25.086 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-24T10:39:25.086 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-24T10:39:25.086 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: pin_top_level_index_and_filter: 1 2024-03-24T10:39:25.086 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: index_type: 0 2024-03-24T10:39:25.086 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: data_block_index_type: 0 2024-03-24T10:39:25.086 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: index_shortening: 1 2024-03-24T10:39:25.087 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-24T10:39:25.087 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: hash_index_allow_collision: 1 2024-03-24T10:39:25.087 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: checksum: 1 2024-03-24T10:39:25.087 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: no_block_cache: 0 2024-03-24T10:39:25.087 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: block_cache: 0x55cd28ed7090 2024-03-24T10:39:25.087 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: block_cache_name: BinnedLRUCache 2024-03-24T10:39:25.087 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: block_cache_options: 2024-03-24T10:39:25.087 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: capacity : 536870912 2024-03-24T10:39:25.087 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: num_shard_bits : 4 2024-03-24T10:39:25.087 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: strict_capacity_limit : 0 2024-03-24T10:39:25.087 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: high_pri_pool_ratio: 0.000 2024-03-24T10:39:25.087 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: block_cache_compressed: (nil) 2024-03-24T10:39:25.087 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: persistent_cache: (nil) 2024-03-24T10:39:25.087 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: block_size: 4096 2024-03-24T10:39:25.087 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: block_size_deviation: 10 2024-03-24T10:39:25.087 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: block_restart_interval: 16 2024-03-24T10:39:25.087 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: index_block_restart_interval: 1 2024-03-24T10:39:25.088 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: metadata_block_size: 4096 2024-03-24T10:39:25.088 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: partition_filters: 0 2024-03-24T10:39:25.088 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: use_delta_encoding: 1 2024-03-24T10:39:25.088 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-03-24T10:39:25.088 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: whole_key_filtering: 1 2024-03-24T10:39:25.088 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: verify_compression: 0 2024-03-24T10:39:25.088 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: read_amp_bytes_per_bit: 0 2024-03-24T10:39:25.088 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: format_version: 4 2024-03-24T10:39:25.088 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: enable_index_compression: 1 2024-03-24T10:39:25.088 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: block_align: 0 2024-03-24T10:39:25.088 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.write_buffer_size: 33554432 2024-03-24T10:39:25.088 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.max_write_buffer_number: 2 2024-03-24T10:39:25.088 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.compression: NoCompression 2024-03-24T10:39:25.088 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.bottommost_compression: Disabled 2024-03-24T10:39:25.088 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.prefix_extractor: nullptr 2024-03-24T10:39:25.089 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-24T10:39:25.089 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.num_levels: 7 2024-03-24T10:39:25.089 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-24T10:39:25.089 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-24T10:39:25.089 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-24T10:39:25.089 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-24T10:39:25.089 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-24T10:39:25.089 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-24T10:39:25.089 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-24T10:39:25.089 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-24T10:39:25.089 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-24T10:39:25.089 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-24T10:39:25.089 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-24T10:39:25.089 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.compression_opts.level: 32767 2024-03-24T10:39:25.089 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.compression_opts.strategy: 0 2024-03-24T10:39:25.089 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-24T10:39:25.089 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-24T10:39:25.090 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-24T10:39:25.090 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.compression_opts.enabled: false 2024-03-24T10:39:25.090 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-24T10:39:25.090 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-24T10:39:25.090 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-24T10:39:25.090 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.target_file_size_base: 67108864 2024-03-24T10:39:25.090 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-24T10:39:25.090 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-24T10:39:25.090 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-24T10:39:25.090 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-24T10:39:25.090 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-24T10:39:25.090 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-24T10:39:25.090 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-24T10:39:25.090 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-24T10:39:25.090 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-24T10:39:25.091 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-24T10:39:25.091 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-24T10:39:25.091 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-24T10:39:25.091 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-24T10:39:25.091 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.arena_block_size: 4194304 2024-03-24T10:39:25.091 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-24T10:39:25.091 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-24T10:39:25.091 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-03-24T10:39:25.091 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.disable_auto_compactions: 0 2024-03-24T10:39:25.091 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-24T10:39:25.091 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-24T10:39:25.091 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-24T10:39:25.091 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-24T10:39:25.091 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-24T10:39:25.091 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-24T10:39:25.091 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-24T10:39:25.091 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-24T10:39:25.092 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-24T10:39:25.092 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-24T10:39:25.092 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.table_properties_collectors: 2024-03-24T10:39:25.092 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.inplace_update_support: 0 2024-03-24T10:39:25.092 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-24T10:39:25.092 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-24T10:39:25.092 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-24T10:39:25.092 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-24T10:39:25.092 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.bloom_locality: 0 2024-03-24T10:39:25.092 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.max_successive_merges: 0 2024-03-24T10:39:25.092 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-24T10:39:25.092 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.paranoid_file_checks: 0 2024-03-24T10:39:25.092 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.force_consistency_checks: 1 2024-03-24T10:39:25.092 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.report_bg_io_stats: 0 2024-03-24T10:39:25.092 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.ttl: 2592000 2024-03-24T10:39:25.092 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-24T10:39:25.093 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.enable_blob_files: false 2024-03-24T10:39:25.093 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.min_blob_size: 0 2024-03-24T10:39:25.093 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.blob_file_size: 268435456 2024-03-24T10:39:25.093 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-24T10:39:25.093 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-24T10:39:25.093 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-24T10:39:25.093 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi096/store.db/MANIFEST-000003 succeeded,manifest_file_number is 3, next_file_number is 5, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-24T10:39:25.093 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 0 2024-03-24T10:39:25.093 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: [db/version_set.cc:4082] Creating manifest 7 2024-03-24T10:39:25.093 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711276764766911, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-24T10:39:25.093 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #4 mode 2 2024-03-24T10:39:25.093 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-03-24T10:39:25.093 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711276764767227, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1545, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1711276764, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "a175449e-27d4-4789-b758-06edc3bb0eea", "db_session_id": "6RWRFJ6EK8JAT7OB3PLG"}} 2024-03-24T10:39:25.093 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: [db/version_set.cc:4082] Creating manifest 9 2024-03-24T10:39:25.093 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711276764767536, "job": 1, "event": "recovery_finished"} 2024-03-24T10:39:25.093 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-smithi096/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-24T10:39:25.094 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x55cd28e6b180 2024-03-24T10:39:25.094 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: DB pointer 0x55cd28f40000 2024-03-24T10:39:25.094 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-03-24T10:39:25.094 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-03-24T10:39:25.094 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: ** DB Stats ** 2024-03-24T10:39:25.094 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-24T10:39:25.094 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-24T10:39:25.094 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-24T10:39:25.094 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-24T10:39:25.094 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-24T10:39:25.094 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-03-24T10:39:25.094 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-24T10:39:25.094 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: 2024-03-24T10:39:25.094 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: ** Compaction Stats [default] ** 2024-03-24T10:39:25.095 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-24T10:39:25.095 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-24T10:39:25.095 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: L0 1/0 1.51 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.9 0.00 0.00 1 0.000 0 0 2024-03-24T10:39:25.095 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: Sum 1/0 1.51 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.9 0.00 0.00 1 0.000 0 0 2024-03-24T10:39:25.095 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.9 0.00 0.00 1 0.000 0 0 2024-03-24T10:39:25.095 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: 2024-03-24T10:39:25.095 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: ** Compaction Stats [default] ** 2024-03-24T10:39:25.095 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-24T10:39:25.095 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-24T10:39:25.095 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.9 0.00 0.00 1 0.000 0 0 2024-03-24T10:39:25.095 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-24T10:39:25.095 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-24T10:39:25.095 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-24T10:39:25.095 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-24T10:39:25.096 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-24T10:39:25.096 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-24T10:39:25.096 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: Cumulative compaction: 0.00 GB write, 0.55 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-24T10:39:25.096 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: Interval compaction: 0.00 GB write, 0.55 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-24T10:39:25.096 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-24T10:39:25.096 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: 2024-03-24T10:39:25.096 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-24T10:39:25.096 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: 2024-03-24T10:39:25.096 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: ** Compaction Stats [default] ** 2024-03-24T10:39:25.096 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-24T10:39:25.096 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-24T10:39:25.096 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: L0 1/0 1.51 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.9 0.00 0.00 1 0.000 0 0 2024-03-24T10:39:25.096 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: Sum 1/0 1.51 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.9 0.00 0.00 1 0.000 0 0 2024-03-24T10:39:25.096 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-03-24T10:39:25.096 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: 2024-03-24T10:39:25.097 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: ** Compaction Stats [default] ** 2024-03-24T10:39:25.097 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-24T10:39:25.097 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-24T10:39:25.097 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.9 0.00 0.00 1 0.000 0 0 2024-03-24T10:39:25.097 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-24T10:39:25.097 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-24T10:39:25.097 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-24T10:39:25.097 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-24T10:39:25.097 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-24T10:39:25.097 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-24T10:39:25.097 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: Cumulative compaction: 0.00 GB write, 0.55 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-24T10:39:25.097 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-24T10:39:25.097 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-24T10:39:25.097 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: 2024-03-24T10:39:25.097 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-24T10:39:25.098 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: mon.smithi096 does not exist in monmap, will attempt to join an existing cluster 2024-03-24T10:39:25.098 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: using public_addr v2:172.21.15.96:0/0 -> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] 2024-03-24T10:39:25.098 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: starting mon.smithi096 rank -1 at public addrs [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] at bind addrs [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi096 fsid 386110d4-e9ca-11ee-95cd-87774f69a715 2024-03-24T10:39:25.098 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: mon.smithi096@-1(???) e0 preinit fsid 386110d4-e9ca-11ee-95cd-87774f69a715 2024-03-24T10:39:25.098 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: mon.smithi096@-1(synchronizing).mds e1 new map 2024-03-24T10:39:25.098 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: mon.smithi096@-1(synchronizing).mds e1 print_map 2024-03-24T10:39:25.098 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: e1 2024-03-24T10:39:25.098 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T10:39:25.098 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:39:25.098 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: legacy client fscid: -1 2024-03-24T10:39:25.098 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: 2024-03-24T10:39:25.098 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: No filesystems configured 2024-03-24T10:39:25.098 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: mon.smithi096@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-24T10:39:25.099 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: mon.smithi096@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-24T10:39:25.099 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: mon.smithi096@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-24T10:39:25.099 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: mon.smithi096@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-24T10:39:25.099 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: mon.smithi096@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-24T10:39:25.099 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: mon.smithi096@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-24T10:39:25.099 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: mon.smithi096@-1(synchronizing).osd e5 e5: 0 total, 0 up, 0 in 2024-03-24T10:39:25.099 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: mon.smithi096@-1(synchronizing).osd e5 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-24T10:39:25.099 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: mon.smithi096@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-03-24T10:39:25.099 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: mon.smithi096@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-03-24T10:39:25.099 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: mon.smithi096@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-03-24T10:39:25.099 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:39:25.099 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:25.099 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:25.099 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:25.099 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-03-24T10:39:25.100 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14162 172.21.15.94:0/1958186361' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-03-24T10:39:25.100 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: mgrmap e14: smithi094.yaethu(active, since 96s) 2024-03-24T10:39:25.100 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='client.? 172.21.15.96:0/531651643' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T10:39:25.100 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='client.? 172.21.15.96:0/3687381999' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T10:39:25.100 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: Active manager daemon smithi094.yaethu restarted 2024-03-24T10:39:25.100 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: Activating manager daemon smithi094.yaethu 2024-03-24T10:39:25.100 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: osdmap e5: 0 total, 0 up, 0 in 2024-03-24T10:39:25.100 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: mgrmap e15: smithi094.yaethu(active, starting, since 0.002817s) 2024-03-24T10:39:25.100 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata", "id": "smithi094"}]: dispatch 2024-03-24T10:39:25.100 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mgr metadata", "who": "smithi094.yaethu", "id": "smithi094.yaethu"}]: dispatch 2024-03-24T10:39:25.100 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-24T10:39:25.100 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-24T10:39:25.100 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-24T10:39:25.100 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: Manager daemon smithi094.yaethu is now available 2024-03-24T10:39:25.100 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:39:25.100 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:39:25.101 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:25.101 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:39:25.101 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi094.yaethu/mirror_snapshot_schedule"}]: dispatch 2024-03-24T10:39:25.101 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi094.yaethu/trash_purge_schedule"}]: dispatch 2024-03-24T10:39:25.101 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:25.101 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:25.101 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:25.101 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: mgrmap e16: smithi094.yaethu(active, since 1.005s) 2024-03-24T10:39:25.101 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='client.? 172.21.15.96:0/3134757630' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T10:39:25.101 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: [24/Mar/2024:10:39:01] ENGINE Bus STARTING 2024-03-24T10:39:25.101 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: [24/Mar/2024:10:39:01] ENGINE Serving on https://172.21.15.94:7150 2024-03-24T10:39:25.101 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: [24/Mar/2024:10:39:01] ENGINE Bus STARTED 2024-03-24T10:39:25.101 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:25.101 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: mgrmap e17: smithi094.yaethu(active, since 2s) 2024-03-24T10:39:25.101 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:25.101 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:25.102 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:25.102 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:25.102 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "who": "osd/host:smithi094", "name": "osd_memory_target"}]: dispatch 2024-03-24T10:39:25.102 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:25.102 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:25.102 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:25.102 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:25.102 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "who": "osd/host:smithi096", "name": "osd_memory_target"}]: dispatch 2024-03-24T10:39:25.102 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:39:25.102 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:39:25.102 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: Updating smithi094:/etc/ceph/ceph.conf 2024-03-24T10:39:25.102 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: Updating smithi096:/etc/ceph/ceph.conf 2024-03-24T10:39:25.102 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: Updating smithi096:/var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T10:39:25.102 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: Updating smithi094:/var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T10:39:25.102 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: Updating smithi096:/etc/ceph/ceph.client.admin.keyring 2024-03-24T10:39:25.103 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-03-24T10:39:25.103 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: Updating smithi096:/var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T10:39:25.103 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: Updating smithi094:/var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T10:39:25.103 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:25.103 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:25.103 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:25.103 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:25.103 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:25.103 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi096", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-24T10:39:25.103 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi096", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-03-24T10:39:25.103 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:39:25.103 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: Deploying daemon crash.smithi096 on smithi096 2024-03-24T10:39:25.103 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='client.? 172.21.15.96:0/2131055590' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T10:39:25.103 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='client.? 172.21.15.96:0/4135252473' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T10:39:25.103 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:25.104 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:25.104 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:25.104 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:25.104 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: Deploying daemon node-exporter.smithi096 on smithi096 2024-03-24T10:39:25.104 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='client.? 172.21.15.96:0/3144791262' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T10:39:25.104 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:25.104 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:39:25.104 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:25.104 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:25.104 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:25.104 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:25.105 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi096.gfonnu", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T10:39:25.105 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi096.gfonnu", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-24T10:39:25.105 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T10:39:25.105 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:39:25.105 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: Deploying daemon mgr.smithi096.gfonnu on smithi096 2024-03-24T10:39:25.105 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:25.105 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:25.105 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:25.105 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:25.105 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T10:39:25.105 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:39:25.105 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='client.? 172.21.15.96:0/3923570479' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T10:39:25.105 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: Deploying daemon mon.smithi096 on smithi096 2024-03-24T10:39:25.106 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:25.106 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:39:25.106 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:39:25.106 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 ceph-mon[38097]: mon.smithi096@-1(synchronizing).paxosservice(auth 1..6) refresh upgraded, format 0 -> 3 2024-03-24T10:39:25.106 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 bash[38049]: 710030b222cb7d230d798028067f9c7c15263afea0d120799e9f9958789d8e56 2024-03-24T10:39:25.106 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:24 smithi096 systemd[1]: Started Ceph mon.smithi096 for 386110d4-e9ca-11ee-95cd-87774f69a715. 2024-03-24T10:39:26.545 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-24T10:39:26.545 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T10:39:26.794 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi096/config 2024-03-24T10:39:30.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:29 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata", "id": "smithi094"}]: dispatch 2024-03-24T10:39:30.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:29 smithi094 ceph-mon[29446]: mon.smithi094 calling monitor election 2024-03-24T10:39:30.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:29 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata", "id": "smithi096"}]: dispatch 2024-03-24T10:39:30.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:29 smithi094 ceph-mon[29446]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:39:30.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:29 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata", "id": "smithi096"}]: dispatch 2024-03-24T10:39:30.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:29 smithi094 ceph-mon[29446]: from='mgr.? 172.21.15.96:0/2331785266' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi096.gfonnu/crt"}]: dispatch 2024-03-24T10:39:30.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:29 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata", "id": "smithi096"}]: dispatch 2024-03-24T10:39:30.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:29 smithi094 ceph-mon[29446]: mon.smithi096 calling monitor election 2024-03-24T10:39:30.249 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:29 smithi094 ceph-mon[29446]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:39:30.249 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:29 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata", "id": "smithi096"}]: dispatch 2024-03-24T10:39:30.249 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:29 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata", "id": "smithi096"}]: dispatch 2024-03-24T10:39:30.249 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:29 smithi094 ceph-mon[29446]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:39:30.249 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:29 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata", "id": "smithi096"}]: dispatch 2024-03-24T10:39:30.249 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:29 smithi094 ceph-mon[29446]: mon.smithi094 is new leader, mons smithi094,smithi096 in quorum (ranks 0,1) 2024-03-24T10:39:30.249 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:29 smithi094 ceph-mon[29446]: monmap e2: 2 mons at {smithi094=[v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0],smithi096=[v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-24T10:39:30.249 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:29 smithi094 ceph-mon[29446]: fsmap 2024-03-24T10:39:30.249 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:29 smithi094 ceph-mon[29446]: osdmap e5: 0 total, 0 up, 0 in 2024-03-24T10:39:30.249 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:29 smithi094 ceph-mon[29446]: mgrmap e17: smithi094.yaethu(active, since 29s) 2024-03-24T10:39:30.249 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:29 smithi094 ceph-mon[29446]: overall HEALTH_OK 2024-03-24T10:39:30.249 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:29 smithi094 ceph-mon[29446]: Standby manager daemon smithi096.gfonnu started 2024-03-24T10:39:30.249 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:29 smithi094 ceph-mon[29446]: from='mgr.? 172.21.15.96:0/2331785266' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T10:39:30.249 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:29 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:30.249 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:29 smithi094 ceph-mon[29446]: from='mgr.? 172.21.15.96:0/2331785266' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi096.gfonnu/key"}]: dispatch 2024-03-24T10:39:30.249 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:29 smithi094 ceph-mon[29446]: from='mgr.? 172.21.15.96:0/2331785266' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T10:39:30.249 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:29 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:30.250 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:29 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:30.250 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:29 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:30.250 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:29 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:30.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:29 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata", "id": "smithi094"}]: dispatch 2024-03-24T10:39:30.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:29 smithi096 ceph-mon[38097]: mon.smithi094 calling monitor election 2024-03-24T10:39:30.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:29 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata", "id": "smithi096"}]: dispatch 2024-03-24T10:39:30.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:29 smithi096 ceph-mon[38097]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:39:30.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:29 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata", "id": "smithi096"}]: dispatch 2024-03-24T10:39:30.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:29 smithi096 ceph-mon[38097]: from='mgr.? 172.21.15.96:0/2331785266' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi096.gfonnu/crt"}]: dispatch 2024-03-24T10:39:30.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:29 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata", "id": "smithi096"}]: dispatch 2024-03-24T10:39:30.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:29 smithi096 ceph-mon[38097]: mon.smithi096 calling monitor election 2024-03-24T10:39:30.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:29 smithi096 ceph-mon[38097]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:39:30.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:29 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata", "id": "smithi096"}]: dispatch 2024-03-24T10:39:30.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:29 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata", "id": "smithi096"}]: dispatch 2024-03-24T10:39:30.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:29 smithi096 ceph-mon[38097]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:39:30.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:29 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata", "id": "smithi096"}]: dispatch 2024-03-24T10:39:30.327 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:29 smithi096 ceph-mon[38097]: mon.smithi094 is new leader, mons smithi094,smithi096 in quorum (ranks 0,1) 2024-03-24T10:39:30.327 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:29 smithi096 ceph-mon[38097]: monmap e2: 2 mons at {smithi094=[v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0],smithi096=[v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-24T10:39:30.327 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:29 smithi096 ceph-mon[38097]: fsmap 2024-03-24T10:39:30.327 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:29 smithi096 ceph-mon[38097]: osdmap e5: 0 total, 0 up, 0 in 2024-03-24T10:39:30.327 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:29 smithi096 ceph-mon[38097]: mgrmap e17: smithi094.yaethu(active, since 29s) 2024-03-24T10:39:30.327 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:29 smithi096 ceph-mon[38097]: overall HEALTH_OK 2024-03-24T10:39:30.327 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:29 smithi096 ceph-mon[38097]: Standby manager daemon smithi096.gfonnu started 2024-03-24T10:39:30.327 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:29 smithi096 ceph-mon[38097]: from='mgr.? 172.21.15.96:0/2331785266' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T10:39:30.327 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:29 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:30.327 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:29 smithi096 ceph-mon[38097]: from='mgr.? 172.21.15.96:0/2331785266' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi096.gfonnu/key"}]: dispatch 2024-03-24T10:39:30.327 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:29 smithi096 ceph-mon[38097]: from='mgr.? 172.21.15.96:0/2331785266' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T10:39:30.327 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:29 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:30.327 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:29 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:30.327 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:29 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:30.327 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:29 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:31.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:30 smithi096 ceph-mon[38097]: mgrmap e18: smithi094.yaethu(active, since 29s), standbys: smithi096.gfonnu 2024-03-24T10:39:31.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:30 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mgr metadata", "who": "smithi096.gfonnu", "id": "smithi096.gfonnu"}]: dispatch 2024-03-24T10:39:31.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:30 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:39:31.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:30 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata", "id": "smithi096"}]: dispatch 2024-03-24T10:39:31.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:30 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:39:31.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:30 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:31.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:30 smithi094 ceph-mon[29446]: mgrmap e18: smithi094.yaethu(active, since 29s), standbys: smithi096.gfonnu 2024-03-24T10:39:31.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:30 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mgr metadata", "who": "smithi096.gfonnu", "id": "smithi096.gfonnu"}]: dispatch 2024-03-24T10:39:31.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:30 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:39:31.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:30 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata", "id": "smithi096"}]: dispatch 2024-03-24T10:39:31.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:30 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:39:31.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:30 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:31.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.389+0000 7f99a2b87700 1 -- 172.21.15.96:0/3750370458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f998c003580 msgr2=0x7f998c005a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:39:31.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.389+0000 7f99a2b87700 1 --2- 172.21.15.96:0/3750370458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f998c003580 0x7f998c005a00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f999c0fb9c0 tx=0x7f999000b040 comp rx=0 tx=0).stop 2024-03-24T10:39:31.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.390+0000 7f99a2b87700 1 -- 172.21.15.96:0/3750370458 shutdown_connections 2024-03-24T10:39:31.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.390+0000 7f99a2b87700 1 --2- 172.21.15.96:0/3750370458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f998c003580 0x7f998c005a00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:39:31.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.390+0000 7f99a2b87700 1 -- 172.21.15.96:0/3750370458 >> 172.21.15.96:0/3750370458 conn(0x7f999c0f75f0 msgr2=0x7f999c0f9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:39:31.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.390+0000 7f99a2b87700 1 -- 172.21.15.96:0/3750370458 shutdown_connections 2024-03-24T10:39:31.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.390+0000 7f99a2b87700 1 -- 172.21.15.96:0/3750370458 wait complete. 2024-03-24T10:39:31.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.390+0000 7f99a2b87700 1 Processor -- start 2024-03-24T10:39:31.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.390+0000 7f99a2b87700 1 -- start start 2024-03-24T10:39:31.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.391+0000 7f99a2b87700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f998c003580 0x7f999c113250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:39:31.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.391+0000 7f99a2b87700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f999c113790 0x7f999c10d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:39:31.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.391+0000 7f99a2b87700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f999c113da0 con 0x7f998c003580 2024-03-24T10:39:31.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.391+0000 7f99a2b87700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f999c113f10 con 0x7f999c113790 2024-03-24T10:39:31.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.391+0000 7f999bfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f999c113790 0x7f999c10d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:39:31.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.391+0000 7f999bfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f999c113790 0x7f999c10d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52848/0 (socket says 172.21.15.96:52848) 2024-03-24T10:39:31.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.391+0000 7f999bfff700 1 -- 172.21.15.96:0/2254434488 learned_addr learned my addr 172.21.15.96:0/2254434488 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-24T10:39:31.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.391+0000 7f999bfff700 1 -- 172.21.15.96:0/2254434488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f999c113790 msgr2=0x7f999c10d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-24T10:39:31.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.391+0000 7f999bfff700 1 -- 172.21.15.96:0/2254434488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f999c113790 msgr2=0x7f999c10d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-24T10:39:31.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.391+0000 7f999bfff700 1 --2- 172.21.15.96:0/2254434488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f999c113790 0x7f999c10d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-24T10:39:31.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.391+0000 7f999bfff700 1 --2- 172.21.15.96:0/2254434488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f999c113790 0x7f999c10d1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-24T10:39:31.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.391+0000 7f99a0923700 1 --2- 172.21.15.96:0/2254434488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f998c003580 0x7f999c113250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:39:31.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.391+0000 7f99a0923700 1 -- 172.21.15.96:0/2254434488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f999c113790 msgr2=0x7f999c10d1f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-24T10:39:31.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.391+0000 7f99a0923700 1 --2- 172.21.15.96:0/2254434488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f999c113790 0x7f999c10d1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:39:31.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.391+0000 7f99a0923700 1 -- 172.21.15.96:0/2254434488 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9990005040 con 0x7f998c003580 2024-03-24T10:39:31.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.392+0000 7f99a0923700 1 --2- 172.21.15.96:0/2254434488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f998c003580 0x7f999c113250 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9990009ce0 tx=0x7f999000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:39:31.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.392+0000 7f99997fa700 1 -- 172.21.15.96:0/2254434488 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f999001b070 con 0x7f998c003580 2024-03-24T10:39:31.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.392+0000 7f99a2b87700 1 -- 172.21.15.96:0/2254434488 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f999c10d7f0 con 0x7f998c003580 2024-03-24T10:39:31.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.392+0000 7f99997fa700 1 -- 172.21.15.96:0/2254434488 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9990016d10 con 0x7f998c003580 2024-03-24T10:39:31.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.392+0000 7f99997fa700 1 -- 172.21.15.96:0/2254434488 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f999001f620 con 0x7f998c003580 2024-03-24T10:39:31.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.392+0000 7f99a2b87700 1 -- 172.21.15.96:0/2254434488 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f999c10dd30 con 0x7f998c003580 2024-03-24T10:39:31.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.393+0000 7f99a2b87700 1 -- 172.21.15.96:0/2254434488 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f999c0fd1d0 con 0x7f998c003580 2024-03-24T10:39:31.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.393+0000 7f99997fa700 1 -- 172.21.15.96:0/2254434488 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7f9990020440 con 0x7f998c003580 2024-03-24T10:39:31.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.394+0000 7f99997fa700 1 --2- 172.21.15.96:0/2254434488 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f9980069320 0x7f998006b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:39:31.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.394+0000 7f99997fa700 1 -- 172.21.15.96:0/2254434488 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f9990087f70 con 0x7f998c003580 2024-03-24T10:39:31.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.394+0000 7f999bfff700 1 --2- 172.21.15.96:0/2254434488 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f9980069320 0x7f998006b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:39:31.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.394+0000 7f999bfff700 1 --2- 172.21.15.96:0/2254434488 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f9980069320 0x7f998006b7d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9988006fd0 tx=0x7f9988008040 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:39:31.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.398+0000 7f99997fa700 1 -- 172.21.15.96:0/2254434488 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f9990055380 con 0x7f998c003580 2024-03-24T10:39:31.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.667+0000 7f99a2b87700 1 -- 172.21.15.96:0/2254434488 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f999c05f5f0 con 0x7f998c003580 2024-03-24T10:39:31.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.667+0000 7f99997fa700 1 -- 172.21.15.96:0/2254434488 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1013 (secure 0 0 0) 0x7f9990024410 con 0x7f998c003580 2024-03-24T10:39:31.668 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-24T10:39:31.668 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":2,"fsid":"386110d4-e9ca-11ee-95cd-87774f69a715","modified":"2024-03-24T10:39:24.793140Z","created":"2024-03-24T10:36:06.632144Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi094","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi096","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:3300","nonce":0},{"type":"v1","addr":"172.21.15.96:6789","nonce":0}]},"addr":"172.21.15.96:6789/0","public_addr":"172.21.15.96:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-24T10:39:31.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.669+0000 7f99a2b87700 1 -- 172.21.15.96:0/2254434488 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f9980069320 msgr2=0x7f998006b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:39:31.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.669+0000 7f99a2b87700 1 --2- 172.21.15.96:0/2254434488 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f9980069320 0x7f998006b7d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9988006fd0 tx=0x7f9988008040 comp rx=0 tx=0).stop 2024-03-24T10:39:31.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.669+0000 7f99a2b87700 1 -- 172.21.15.96:0/2254434488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f998c003580 msgr2=0x7f999c113250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:39:31.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.669+0000 7f99a2b87700 1 --2- 172.21.15.96:0/2254434488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f998c003580 0x7f999c113250 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9990009ce0 tx=0x7f999000b040 comp rx=0 tx=0).stop 2024-03-24T10:39:31.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.669+0000 7f99a2b87700 1 -- 172.21.15.96:0/2254434488 shutdown_connections 2024-03-24T10:39:31.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.669+0000 7f99a2b87700 1 --2- 172.21.15.96:0/2254434488 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f9980069320 0x7f998006b7d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:39:31.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.669+0000 7f99a2b87700 1 --2- 172.21.15.96:0/2254434488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f998c003580 0x7f999c113250 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:39:31.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.669+0000 7f99a2b87700 1 --2- 172.21.15.96:0/2254434488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f999c113790 0x7f999c10d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:39:31.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.669+0000 7f99a2b87700 1 -- 172.21.15.96:0/2254434488 >> 172.21.15.96:0/2254434488 conn(0x7f999c0f75f0 msgr2=0x7f999c0663a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:39:31.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.670+0000 7f99a2b87700 1 -- 172.21.15.96:0/2254434488 shutdown_connections 2024-03-24T10:39:31.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:39:31.670+0000 7f99a2b87700 1 -- 172.21.15.96:0/2254434488 wait complete. 2024-03-24T10:39:31.671 INFO:teuthology.orchestra.run.smithi096.stderr:dumped monmap epoch 2 2024-03-24T10:39:32.052 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:31 smithi096 ceph-mon[38097]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:39:32.052 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:31 smithi096 ceph-mon[38097]: from='client.? 172.21.15.96:0/2254434488' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T10:39:32.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:31 smithi094 ceph-mon[29446]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:39:32.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:31 smithi094 ceph-mon[29446]: from='client.? 172.21.15.96:0/2254434488' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T10:39:32.341 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-24T10:39:32.342 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph config generate-minimal-conf 2024-03-24T10:39:32.614 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:39:33.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:33 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:33.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:33 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:33.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:33 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:39:33.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:33 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:39:33.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:33 smithi094 ceph-mon[29446]: Updating smithi094:/etc/ceph/ceph.conf 2024-03-24T10:39:33.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:33 smithi094 ceph-mon[29446]: Updating smithi096:/etc/ceph/ceph.conf 2024-03-24T10:39:33.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:33 smithi094 ceph-mon[29446]: Updating smithi096:/var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T10:39:33.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:33 smithi094 ceph-mon[29446]: Updating smithi094:/var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T10:39:33.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:33 smithi094 ceph-mon[29446]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:39:33.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:33 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:33.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:33 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:33.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:33 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:33.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:33 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:33.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:33 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:33.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:33 smithi094 ceph-mon[29446]: Reconfiguring mon.smithi094 (unknown last config time)... 2024-03-24T10:39:33.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:33 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T10:39:33.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:33 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T10:39:33.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:33 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:39:33.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:33 smithi094 ceph-mon[29446]: Reconfiguring daemon mon.smithi094 on smithi094 2024-03-24T10:39:33.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:33 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:33.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:33 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:33.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:33 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:39:33.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:33 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:39:33.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:33 smithi096 ceph-mon[38097]: Updating smithi094:/etc/ceph/ceph.conf 2024-03-24T10:39:33.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:33 smithi096 ceph-mon[38097]: Updating smithi096:/etc/ceph/ceph.conf 2024-03-24T10:39:33.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:33 smithi096 ceph-mon[38097]: Updating smithi096:/var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T10:39:33.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:33 smithi096 ceph-mon[38097]: Updating smithi094:/var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T10:39:33.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:33 smithi096 ceph-mon[38097]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:39:33.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:33 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:33.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:33 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:33.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:33 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:33.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:33 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:33.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:33 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:33.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:33 smithi096 ceph-mon[38097]: Reconfiguring mon.smithi094 (unknown last config time)... 2024-03-24T10:39:33.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:33 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T10:39:33.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:33 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T10:39:33.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:33 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:39:33.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:33 smithi096 ceph-mon[38097]: Reconfiguring daemon mon.smithi094 on smithi094 2024-03-24T10:39:34.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:34.161+0000 7f0b4b8bc700 1 -- 172.21.15.94:0/2355221395 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b441019d0 msgr2=0x7f0b44101da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:39:34.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:34.161+0000 7f0b4b8bc700 1 --2- 172.21.15.94:0/2355221395 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b441019d0 0x7f0b44101da0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0b340048c0 tx=0x7f0b34009e30 comp rx=0 tx=0).stop 2024-03-24T10:39:34.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:34.161+0000 7f0b4b8bc700 1 -- 172.21.15.94:0/2355221395 shutdown_connections 2024-03-24T10:39:34.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:34.161+0000 7f0b4b8bc700 1 --2- 172.21.15.94:0/2355221395 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b441019d0 0x7f0b44101da0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:39:34.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:34.161+0000 7f0b4b8bc700 1 -- 172.21.15.94:0/2355221395 >> 172.21.15.94:0/2355221395 conn(0x7f0b44071800 msgr2=0x7f0b44071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:39:34.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:34.161+0000 7f0b4b8bc700 1 -- 172.21.15.94:0/2355221395 shutdown_connections 2024-03-24T10:39:34.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:34.161+0000 7f0b4b8bc700 1 -- 172.21.15.94:0/2355221395 wait complete. 2024-03-24T10:39:34.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:34.162+0000 7f0b4b8bc700 1 Processor -- start 2024-03-24T10:39:34.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:34.162+0000 7f0b4b8bc700 1 -- start start 2024-03-24T10:39:34.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:34.162+0000 7f0b4b8bc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b441019d0 0x7f0b440fd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:39:34.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:34.162+0000 7f0b4b8bc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b440fdae0 0x7f0b440fdf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:39:34.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:34.162+0000 7f0b4b8bc700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b44119570 con 0x7f0b441019d0 2024-03-24T10:39:34.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:34.162+0000 7f0b4b8bc700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b441196b0 con 0x7f0b440fdae0 2024-03-24T10:39:34.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:34.162+0000 7f0b49658700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b441019d0 0x7f0b440fd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:39:34.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:34.162+0000 7f0b49658700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b441019d0 0x7f0b440fd5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48594/0 (socket says 172.21.15.94:48594) 2024-03-24T10:39:34.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:34.162+0000 7f0b49658700 1 -- 172.21.15.94:0/3646324331 learned_addr learned my addr 172.21.15.94:0/3646324331 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:39:34.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:34.162+0000 7f0b49658700 1 -- 172.21.15.94:0/3646324331 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b440fdae0 msgr2=0x7f0b440fdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:39:34.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:34.162+0000 7f0b49658700 1 --2- 172.21.15.94:0/3646324331 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b440fdae0 0x7f0b440fdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:39:34.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:34.162+0000 7f0b49658700 1 -- 172.21.15.94:0/3646324331 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b34005040 con 0x7f0b441019d0 2024-03-24T10:39:34.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:34.162+0000 7f0b49658700 1 --2- 172.21.15.94:0/3646324331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b441019d0 0x7f0b440fd5a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0b34000c00 tx=0x7f0b34009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:39:34.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:34.163+0000 7f0b39ffb700 1 -- 172.21.15.94:0/3646324331 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b3400e040 con 0x7f0b441019d0 2024-03-24T10:39:34.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:34.163+0000 7f0b39ffb700 1 -- 172.21.15.94:0/3646324331 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0b34013420 con 0x7f0b441019d0 2024-03-24T10:39:34.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:34.163+0000 7f0b39ffb700 1 -- 172.21.15.94:0/3646324331 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b3401c630 con 0x7f0b441019d0 2024-03-24T10:39:34.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:34.163+0000 7f0b4b8bc700 1 -- 172.21.15.94:0/3646324331 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b44119850 con 0x7f0b441019d0 2024-03-24T10:39:34.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:34.163+0000 7f0b4b8bc700 1 -- 172.21.15.94:0/3646324331 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b44119e10 con 0x7f0b441019d0 2024-03-24T10:39:34.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:34.164+0000 7f0b4b8bc700 1 -- 172.21.15.94:0/3646324331 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b440fd0d0 con 0x7f0b441019d0 2024-03-24T10:39:34.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:34.164+0000 7f0b39ffb700 1 -- 172.21.15.94:0/3646324331 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7f0b34013590 con 0x7f0b441019d0 2024-03-24T10:39:34.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:34.165+0000 7f0b39ffb700 1 --2- 172.21.15.94:0/3646324331 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f0b30069320 0x7f0b3006b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:39:34.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:34.165+0000 7f0b39ffb700 1 -- 172.21.15.94:0/3646324331 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f0b340873e0 con 0x7f0b441019d0 2024-03-24T10:39:34.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:34.165+0000 7f0b48e57700 1 --2- 172.21.15.94:0/3646324331 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f0b30069320 0x7f0b3006b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:39:34.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:34.165+0000 7f0b48e57700 1 --2- 172.21.15.94:0/3646324331 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f0b30069320 0x7f0b3006b7d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0b40009930 tx=0x7f0b40006de0 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:39:34.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:34.169+0000 7f0b39ffb700 1 -- 172.21.15.94:0/3646324331 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f0b340547f0 con 0x7f0b441019d0 2024-03-24T10:39:34.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:34.359+0000 7f0b4b8bc700 1 -- 172.21.15.94:0/3646324331 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f0b4405f5f0 con 0x7f0b441019d0 2024-03-24T10:39:34.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:34.359+0000 7f0b39ffb700 1 -- 172.21.15.94:0/3646324331 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+223 (secure 0 0 0) 0x7f0b34086b90 con 0x7f0b441019d0 2024-03-24T10:39:34.361 INFO:teuthology.orchestra.run.smithi094.stdout:# minimal ceph.conf for 386110d4-e9ca-11ee-95cd-87774f69a715 2024-03-24T10:39:34.361 INFO:teuthology.orchestra.run.smithi094.stdout:[global] 2024-03-24T10:39:34.361 INFO:teuthology.orchestra.run.smithi094.stdout: fsid = 386110d4-e9ca-11ee-95cd-87774f69a715 2024-03-24T10:39:34.361 INFO:teuthology.orchestra.run.smithi094.stdout: mon_host = [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] 2024-03-24T10:39:34.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:34.361+0000 7f0b4b8bc700 1 -- 172.21.15.94:0/3646324331 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f0b30069320 msgr2=0x7f0b3006b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:39:34.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:34.361+0000 7f0b4b8bc700 1 --2- 172.21.15.94:0/3646324331 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f0b30069320 0x7f0b3006b7d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0b40009930 tx=0x7f0b40006de0 comp rx=0 tx=0).stop 2024-03-24T10:39:34.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:34.362+0000 7f0b4b8bc700 1 -- 172.21.15.94:0/3646324331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b441019d0 msgr2=0x7f0b440fd5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:39:34.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:34.362+0000 7f0b4b8bc700 1 --2- 172.21.15.94:0/3646324331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b441019d0 0x7f0b440fd5a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0b34000c00 tx=0x7f0b34009e30 comp rx=0 tx=0).stop 2024-03-24T10:39:34.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:34.362+0000 7f0b4b8bc700 1 -- 172.21.15.94:0/3646324331 shutdown_connections 2024-03-24T10:39:34.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:34.362+0000 7f0b4b8bc700 1 --2- 172.21.15.94:0/3646324331 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f0b30069320 0x7f0b3006b7d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:39:34.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:34.362+0000 7f0b4b8bc700 1 --2- 172.21.15.94:0/3646324331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b441019d0 0x7f0b440fd5a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:39:34.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:34.362+0000 7f0b4b8bc700 1 --2- 172.21.15.94:0/3646324331 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b440fdae0 0x7f0b440fdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:39:34.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:34.362+0000 7f0b4b8bc700 1 -- 172.21.15.94:0/3646324331 >> 172.21.15.94:0/3646324331 conn(0x7f0b44071800 msgr2=0x7f0b44065270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:39:34.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:34.362+0000 7f0b4b8bc700 1 -- 172.21.15.94:0/3646324331 shutdown_connections 2024-03-24T10:39:34.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:34.362+0000 7f0b4b8bc700 1 -- 172.21.15.94:0/3646324331 wait complete. 2024-03-24T10:39:34.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:34 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/3646324331' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:39:34.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:34 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/3646324331' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:39:35.643 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-24T10:39:35.643 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-03-24T10:39:35.643 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/ceph/ceph.conf 2024-03-24T10:39:35.675 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-03-24T10:39:35.675 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-24T10:39:35.742 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-24T10:39:35.742 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/ceph/ceph.conf 2024-03-24T10:39:35.777 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-24T10:39:35.777 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-24T10:39:35.848 INFO:tasks.cephadm:Deploying OSDs... 2024-03-24T10:39:35.848 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-03-24T10:39:35.848 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/scratch_devs of=/dev/stdout 2024-03-24T10:39:35.863 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-24T10:39:35.863 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_1 2024-03-24T10:39:35.920 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-24T10:39:35.920 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-24T10:39:35.920 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-03-24T10:39:35.920 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-24T10:39:35.920 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-03-24T10:39:35.920 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-03-24 10:38:52.681209895 +0000 2024-03-24T10:39:35.920 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-03-24 10:32:46.382023022 +0000 2024-03-24T10:39:35.920 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-03-24 10:32:46.382023022 +0000 2024-03-24T10:39:35.920 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-03-24 10:32:46.382023022 +0000 2024-03-24T10:39:35.920 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-24T10:39:35.986 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-03-24T10:39:35.986 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-03-24T10:39:35.986 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 8.5176e-05 s, 6.0 MB/s 2024-03-24T10:39:35.987 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-24T10:39:36.043 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_2 2024-03-24T10:39:36.098 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-24T10:39:36.099 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-24T10:39:36.099 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-03-24T10:39:36.099 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-24T10:39:36.099 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-03-24T10:39:36.099 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-03-24 10:38:52.682209878 +0000 2024-03-24T10:39:36.099 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-03-24 10:32:46.725017216 +0000 2024-03-24T10:39:36.099 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-03-24 10:32:46.725017216 +0000 2024-03-24T10:39:36.099 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-03-24 10:32:46.725017216 +0000 2024-03-24T10:39:36.099 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-24T10:39:36.162 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:36 smithi094 ceph-mon[29446]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:39:36.162 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:36 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:36.162 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:36 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:36.162 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:36 smithi094 ceph-mon[29446]: Reconfiguring mgr.smithi094.yaethu (unknown last config time)... 2024-03-24T10:39:36.162 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:36 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi094.yaethu", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T10:39:36.162 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:36 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T10:39:36.162 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:36 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:39:36.163 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:36 smithi094 ceph-mon[29446]: Reconfiguring daemon mgr.smithi094.yaethu on smithi094 2024-03-24T10:39:36.164 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-03-24T10:39:36.164 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-03-24T10:39:36.164 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 8.1641e-05 s, 6.3 MB/s 2024-03-24T10:39:36.165 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-24T10:39:36.221 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_3 2024-03-24T10:39:36.276 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-24T10:39:36.276 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-24T10:39:36.276 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-03-24T10:39:36.277 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-24T10:39:36.277 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-03-24T10:39:36.277 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-03-24 10:38:52.682209878 +0000 2024-03-24T10:39:36.277 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-03-24 10:32:47.060011545 +0000 2024-03-24T10:39:36.277 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-03-24 10:32:47.060011545 +0000 2024-03-24T10:39:36.277 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-03-24 10:32:47.060011545 +0000 2024-03-24T10:39:36.277 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-24T10:39:36.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:36 smithi096 ceph-mon[38097]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:39:36.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:36 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:36.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:36 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:36.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:36 smithi096 ceph-mon[38097]: Reconfiguring mgr.smithi094.yaethu (unknown last config time)... 2024-03-24T10:39:36.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:36 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi094.yaethu", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T10:39:36.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:36 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T10:39:36.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:36 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:39:36.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:36 smithi096 ceph-mon[38097]: Reconfiguring daemon mgr.smithi094.yaethu on smithi094 2024-03-24T10:39:36.342 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-03-24T10:39:36.343 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-03-24T10:39:36.343 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000128713 s, 4.0 MB/s 2024-03-24T10:39:36.344 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-24T10:39:36.400 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_4 2024-03-24T10:39:36.454 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-24T10:39:36.455 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-24T10:39:36.455 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-03-24T10:39:36.455 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-24T10:39:36.455 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-03-24T10:39:36.455 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-03-24 10:38:52.683209861 +0000 2024-03-24T10:39:36.455 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-03-24 10:32:47.415005536 +0000 2024-03-24T10:39:36.455 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-03-24 10:32:47.415005536 +0000 2024-03-24T10:39:36.455 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-03-24 10:32:47.415005536 +0000 2024-03-24T10:39:36.455 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-24T10:39:36.521 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-03-24T10:39:36.521 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-03-24T10:39:36.521 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 9.1247e-05 s, 5.6 MB/s 2024-03-24T10:39:36.522 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-24T10:39:36.578 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-24T10:39:36.579 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/scratch_devs of=/dev/stdout 2024-03-24T10:39:36.598 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-24T10:39:36.598 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_1 2024-03-24T10:39:36.656 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-24T10:39:36.656 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-24T10:39:36.656 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-03-24T10:39:36.656 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-24T10:39:36.656 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-03-24T10:39:36.656 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-03-24 10:39:22.995217437 +0000 2024-03-24T10:39:36.656 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-03-24 10:32:46.772708002 +0000 2024-03-24T10:39:36.656 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-03-24 10:32:46.772708002 +0000 2024-03-24T10:39:36.656 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-03-24 10:32:46.772708002 +0000 2024-03-24T10:39:36.657 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-24T10:39:36.726 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-03-24T10:39:36.726 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-03-24T10:39:36.726 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.00029182 s, 1.8 MB/s 2024-03-24T10:39:36.727 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-24T10:39:36.786 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_2 2024-03-24T10:39:36.843 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-24T10:39:36.843 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-24T10:39:36.844 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-03-24T10:39:36.844 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-24T10:39:36.844 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-03-24T10:39:36.844 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-03-24 10:39:22.995217437 +0000 2024-03-24T10:39:36.844 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-03-24 10:32:47.102700431 +0000 2024-03-24T10:39:36.844 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-03-24 10:32:47.102700431 +0000 2024-03-24T10:39:36.844 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-03-24 10:32:47.102700431 +0000 2024-03-24T10:39:36.844 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-24T10:39:36.911 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-03-24T10:39:36.911 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-03-24T10:39:36.911 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000205268 s, 2.5 MB/s 2024-03-24T10:39:36.912 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-24T10:39:36.969 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_3 2024-03-24T10:39:37.025 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-24T10:39:37.025 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-24T10:39:37.025 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-03-24T10:39:37.026 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-24T10:39:37.026 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-03-24T10:39:37.026 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-03-24 10:39:22.996217415 +0000 2024-03-24T10:39:37.026 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-03-24 10:32:47.434692813 +0000 2024-03-24T10:39:37.026 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-03-24 10:32:47.434692813 +0000 2024-03-24T10:39:37.026 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-03-24 10:32:47.434692813 +0000 2024-03-24T10:39:37.026 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-24T10:39:37.094 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-03-24T10:39:37.094 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-03-24T10:39:37.094 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000167245 s, 3.1 MB/s 2024-03-24T10:39:37.094 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-24T10:39:37.150 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_4 2024-03-24T10:39:37.205 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-24T10:39:37.205 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-24T10:39:37.205 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-03-24T10:39:37.205 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-24T10:39:37.205 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-03-24T10:39:37.206 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-03-24 10:39:22.996217415 +0000 2024-03-24T10:39:37.206 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-03-24 10:32:47.785684760 +0000 2024-03-24T10:39:37.206 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-03-24 10:32:47.785684760 +0000 2024-03-24T10:39:37.206 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-03-24 10:32:47.785684760 +0000 2024-03-24T10:39:37.206 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-24T10:39:37.271 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-03-24T10:39:37.271 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-03-24T10:39:37.271 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000193519 s, 2.6 MB/s 2024-03-24T10:39:37.272 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-24T10:39:37.328 INFO:tasks.cephadm:Deploying osd.0 on smithi094 with /dev/vg_nvme/lv_4... 2024-03-24T10:39:37.329 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-24T10:39:38.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:37 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:38.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:37 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:38.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:37 smithi096 ceph-mon[38097]: Reconfiguring crash.smithi094 (monmap changed)... 2024-03-24T10:39:38.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:37 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi094", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-24T10:39:38.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:37 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:39:38.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:37 smithi096 ceph-mon[38097]: Reconfiguring daemon crash.smithi094 on smithi094 2024-03-24T10:39:38.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:37 smithi096 ceph-mon[38097]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:39:38.087 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:37 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:38.087 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:37 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:38.087 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:37 smithi094 ceph-mon[29446]: Reconfiguring crash.smithi094 (monmap changed)... 2024-03-24T10:39:38.087 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:37 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi094", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-24T10:39:38.087 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:37 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:39:38.087 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:37 smithi094 ceph-mon[29446]: Reconfiguring daemon crash.smithi094 on smithi094 2024-03-24T10:39:38.087 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:37 smithi094 ceph-mon[29446]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:39:38.095 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:39:39.402 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:39 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:39.402 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:39 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:39.402 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:39 smithi094 ceph-mon[29446]: Reconfiguring alertmanager.smithi094 (dependencies changed)... 2024-03-24T10:39:39.402 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:39 smithi094 ceph-mon[29446]: Reconfiguring daemon alertmanager.smithi094 on smithi094 2024-03-24T10:39:39.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:39 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:39.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:39 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:39.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:39 smithi096 ceph-mon[38097]: Reconfiguring alertmanager.smithi094 (dependencies changed)... 2024-03-24T10:39:39.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:39 smithi096 ceph-mon[38097]: Reconfiguring daemon alertmanager.smithi094 on smithi094 2024-03-24T10:39:40.206 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:40 smithi094 ceph-mon[29446]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:39:40.206 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-24T10:39:40.206 INFO:teuthology.orchestra.run.smithi094.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-24T10:39:40.206 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10+0 records in 2024-03-24T10:39:40.207 INFO:teuthology.orchestra.run.smithi094.stderr:10+0 records out 2024-03-24T10:39:40.207 INFO:teuthology.orchestra.run.smithi094.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00889047 s, 1.2 GB/s 2024-03-24T10:39:40.207 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping successful for: 2024-03-24T10:39:40.459 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:40 smithi096 ceph-mon[38097]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:39:42.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:42 smithi094 ceph-mon[29446]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:39:42.554 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi094:vg_nvme/lv_4 2024-03-24T10:39:42.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:42 smithi096 ceph-mon[38097]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:39:43.602 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:39:44.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:44 smithi094 ceph-mon[29446]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:39:44.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:44 smithi096 ceph-mon[38097]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:39:45.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:45.910+0000 7f117d048700 1 -- 172.21.15.94:0/558674693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11780fce40 msgr2=0x7f11780fd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:39:45.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:45.910+0000 7f117d048700 1 --2- 172.21.15.94:0/558674693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11780fce40 0x7f11780fd250 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1168004940 tx=0x7f1168009e30 comp rx=0 tx=0).stop 2024-03-24T10:39:45.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:45.910+0000 7f117d048700 1 -- 172.21.15.94:0/558674693 shutdown_connections 2024-03-24T10:39:45.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:45.910+0000 7f117d048700 1 --2- 172.21.15.94:0/558674693 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11780fe040 0x7f11780fe4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:39:45.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:45.910+0000 7f117d048700 1 --2- 172.21.15.94:0/558674693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11780fce40 0x7f11780fd250 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:39:45.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:45.910+0000 7f117d048700 1 -- 172.21.15.94:0/558674693 >> 172.21.15.94:0/558674693 conn(0x7f11780f8570 msgr2=0x7f11780fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:39:45.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:45.910+0000 7f117d048700 1 -- 172.21.15.94:0/558674693 shutdown_connections 2024-03-24T10:39:45.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:45.910+0000 7f117d048700 1 -- 172.21.15.94:0/558674693 wait complete. 2024-03-24T10:39:45.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:45.910+0000 7f117d048700 1 Processor -- start 2024-03-24T10:39:45.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:45.910+0000 7f117d048700 1 -- start start 2024-03-24T10:39:45.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:45.911+0000 7f117d048700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11780fce40 0x7f117810da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:39:45.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:45.911+0000 7f117d048700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11780fe040 0x7f117810df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:39:45.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:45.911+0000 7f117d048700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f117810e600 con 0x7f11780fce40 2024-03-24T10:39:45.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:45.911+0000 7f117d048700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f117810edf0 con 0x7f11780fe040 2024-03-24T10:39:45.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:45.911+0000 7f1176d9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11780fce40 0x7f117810da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:39:45.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:45.911+0000 7f1176d9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11780fce40 0x7f117810da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60544/0 (socket says 172.21.15.94:60544) 2024-03-24T10:39:45.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:45.911+0000 7f1176d9d700 1 -- 172.21.15.94:0/1017662969 learned_addr learned my addr 172.21.15.94:0/1017662969 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:39:45.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:45.911+0000 7f1176d9d700 1 -- 172.21.15.94:0/1017662969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11780fe040 msgr2=0x7f117810df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:39:45.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:45.911+0000 7f117659c700 1 --2- 172.21.15.94:0/1017662969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11780fe040 0x7f117810df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:39:45.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:45.911+0000 7f1176d9d700 1 --2- 172.21.15.94:0/1017662969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11780fe040 0x7f117810df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:39:45.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:45.911+0000 7f1176d9d700 1 -- 172.21.15.94:0/1017662969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1168005040 con 0x7f11780fce40 2024-03-24T10:39:45.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:45.911+0000 7f1176d9d700 1 --2- 172.21.15.94:0/1017662969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11780fce40 0x7f117810da30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f116800b3b0 tx=0x7f1168009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:39:45.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:45.911+0000 7f115f7fe700 1 -- 172.21.15.94:0/1017662969 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f116800e040 con 0x7f11780fce40 2024-03-24T10:39:45.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:45.911+0000 7f115f7fe700 1 -- 172.21.15.94:0/1017662969 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f116800bcc0 con 0x7f11780fce40 2024-03-24T10:39:45.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:45.911+0000 7f115f7fe700 1 -- 172.21.15.94:0/1017662969 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f116801c630 con 0x7f11780fce40 2024-03-24T10:39:45.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:45.911+0000 7f117d048700 1 -- 172.21.15.94:0/1017662969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f117810f070 con 0x7f11780fce40 2024-03-24T10:39:45.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:45.912+0000 7f117d048700 1 -- 172.21.15.94:0/1017662969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f117810f630 con 0x7f11780fce40 2024-03-24T10:39:45.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:45.913+0000 7f117d048700 1 -- 172.21.15.94:0/1017662969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f117805f5f0 con 0x7f11780fce40 2024-03-24T10:39:45.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:45.913+0000 7f115f7fe700 1 -- 172.21.15.94:0/1017662969 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7f1168013420 con 0x7f11780fce40 2024-03-24T10:39:45.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:45.913+0000 7f115f7fe700 1 --2- 172.21.15.94:0/1017662969 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f1160069210 0x7f116006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:39:45.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:45.913+0000 7f115f7fe700 1 -- 172.21.15.94:0/1017662969 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f1168086e90 con 0x7f11780fce40 2024-03-24T10:39:45.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:45.913+0000 7f117659c700 1 --2- 172.21.15.94:0/1017662969 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f1160069210 0x7f116006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:39:45.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:45.914+0000 7f117659c700 1 --2- 172.21.15.94:0/1017662969 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f1160069210 0x7f116006b6c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f116c006fd0 tx=0x7f116c008040 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:39:45.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:45.918+0000 7f115f7fe700 1 -- 172.21.15.94:0/1017662969 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f1168057d30 con 0x7f11780fce40 2024-03-24T10:39:46.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:39:46.122+0000 7f117d048700 1 -- 172.21.15.94:0/1017662969 --> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f117810f960 con 0x7f1160069210 2024-03-24T10:39:46.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:46 smithi094 ceph-mon[29446]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:39:46.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:46 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:46.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:46 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:46.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:46 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:39:46.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:46 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T10:39:46.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:46 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T10:39:46.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:46 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:39:46.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:46 smithi096 ceph-mon[38097]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:39:46.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:46 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:46.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:46 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:46.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:46 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:39:46.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:46 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T10:39:46.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:46 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T10:39:46.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:46 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:39:47.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:47 smithi094 ceph-mon[29446]: Reconfiguring grafana.smithi094 (dependencies changed)... 2024-03-24T10:39:47.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:47 smithi094 ceph-mon[29446]: Reconfiguring daemon grafana.smithi094 on smithi094 2024-03-24T10:39:47.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:47 smithi094 ceph-mon[29446]: from='client.14266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:39:47.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:47 smithi096 ceph-mon[38097]: Reconfiguring grafana.smithi094 (dependencies changed)... 2024-03-24T10:39:47.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:47 smithi096 ceph-mon[38097]: Reconfiguring daemon grafana.smithi094 on smithi094 2024-03-24T10:39:47.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:47 smithi096 ceph-mon[38097]: from='client.14266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:39:48.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:48 smithi094 ceph-mon[29446]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:39:48.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:48 smithi096 ceph-mon[38097]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:39:50.342 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:50 smithi094 ceph-mon[29446]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:39:50.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:50 smithi096 ceph-mon[38097]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:39:51.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:51 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:51.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:51 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:51.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:51 smithi094 ceph-mon[29446]: Reconfiguring prometheus.smithi094 (dependencies changed)... 2024-03-24T10:39:51.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:51 smithi094 ceph-mon[29446]: Reconfiguring daemon prometheus.smithi094 on smithi094 2024-03-24T10:39:51.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:51 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:51.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:51 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:39:51.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:51 smithi096 ceph-mon[38097]: Reconfiguring prometheus.smithi094 (dependencies changed)... 2024-03-24T10:39:51.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:51 smithi096 ceph-mon[38097]: Reconfiguring daemon prometheus.smithi094 on smithi094 2024-03-24T10:39:52.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:52 smithi094 ceph-mon[29446]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:39:52.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:52 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/3621445311' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58fd53f9-8ad5-4296-8804-96a98647bba7"}]: dispatch 2024-03-24T10:39:52.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:52 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/3621445311' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "58fd53f9-8ad5-4296-8804-96a98647bba7"}]': finished 2024-03-24T10:39:52.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:52 smithi094 ceph-mon[29446]: osdmap e6: 1 total, 0 up, 1 in 2024-03-24T10:39:52.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:52 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T10:39:52.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:52 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/2098104909' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T10:39:52.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:52 smithi096 ceph-mon[38097]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:39:52.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:52 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/3621445311' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58fd53f9-8ad5-4296-8804-96a98647bba7"}]: dispatch 2024-03-24T10:39:52.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:52 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/3621445311' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "58fd53f9-8ad5-4296-8804-96a98647bba7"}]': finished 2024-03-24T10:39:52.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:52 smithi096 ceph-mon[38097]: osdmap e6: 1 total, 0 up, 1 in 2024-03-24T10:39:52.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:52 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T10:39:52.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:52 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/2098104909' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T10:39:54.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:54 smithi094 ceph-mon[29446]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:39:54.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:54 smithi096 ceph-mon[38097]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:39:56.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:56 smithi094 ceph-mon[29446]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:39:56.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:56 smithi096 ceph-mon[38097]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:39:58.373 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:39:58 smithi094 ceph-mon[29446]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:39:58.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:39:58 smithi096 ceph-mon[38097]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:40:00.474 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:00 smithi094 ceph-mon[29446]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:40:00.474 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:00 smithi094 ceph-mon[29446]: overall HEALTH_OK 2024-03-24T10:40:00.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:00 smithi096 ceph-mon[38097]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:40:00.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:00 smithi096 ceph-mon[38097]: overall HEALTH_OK 2024-03-24T10:40:01.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:01 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:01.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:01 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:01.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:01 smithi096 ceph-mon[38097]: Reconfiguring crash.smithi096 (monmap changed)... 2024-03-24T10:40:01.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:01 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi096", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-24T10:40:01.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:01 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:40:01.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:01 smithi096 ceph-mon[38097]: Reconfiguring daemon crash.smithi096 on smithi096 2024-03-24T10:40:01.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:01 smithi096 ceph-mon[38097]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:40:01.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:01 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:40:01.840 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:01 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:01.841 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:01 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:01.841 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:01 smithi094 ceph-mon[29446]: Reconfiguring crash.smithi096 (monmap changed)... 2024-03-24T10:40:01.841 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:01 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi096", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-24T10:40:01.841 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:01 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:40:01.841 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:01 smithi094 ceph-mon[29446]: Reconfiguring daemon crash.smithi096 on smithi096 2024-03-24T10:40:01.841 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:01 smithi094 ceph-mon[29446]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:40:01.841 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:01 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:40:02.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:02 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-24T10:40:02.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:02 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:40:02.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:02 smithi096 ceph-mon[38097]: Deploying daemon osd.0 on smithi094 2024-03-24T10:40:02.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:02 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:02.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:02 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:02.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:02 smithi096 ceph-mon[38097]: Reconfiguring mgr.smithi096.gfonnu (monmap changed)... 2024-03-24T10:40:02.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:02 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi096.gfonnu", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T10:40:02.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:02 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T10:40:02.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:02 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:40:02.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:02 smithi096 ceph-mon[38097]: Reconfiguring daemon mgr.smithi096.gfonnu on smithi096 2024-03-24T10:40:02.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:02 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-24T10:40:02.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:02 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:40:02.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:02 smithi094 ceph-mon[29446]: Deploying daemon osd.0 on smithi094 2024-03-24T10:40:02.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:02 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:02.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:02 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:02.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:02 smithi094 ceph-mon[29446]: Reconfiguring mgr.smithi096.gfonnu (monmap changed)... 2024-03-24T10:40:02.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:02 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi096.gfonnu", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T10:40:02.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:02 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T10:40:02.999 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:02 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:40:02.999 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:02 smithi094 ceph-mon[29446]: Reconfiguring daemon mgr.smithi096.gfonnu on smithi096 2024-03-24T10:40:04.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:04 smithi094 ceph-mon[29446]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:40:04.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:04 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:04.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:04 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:04.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:04 smithi094 ceph-mon[29446]: Reconfiguring mon.smithi096 (monmap changed)... 2024-03-24T10:40:04.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:04 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T10:40:04.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:04 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T10:40:04.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:04 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:40:04.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:04 smithi094 ceph-mon[29446]: Reconfiguring daemon mon.smithi096 on smithi096 2024-03-24T10:40:04.549 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:04 smithi096 ceph-mon[38097]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:40:04.549 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:04 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:04.549 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:04 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:04.549 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:04 smithi096 ceph-mon[38097]: Reconfiguring mon.smithi096 (monmap changed)... 2024-03-24T10:40:04.549 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:04 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T10:40:04.549 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:04 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T10:40:04.549 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:04 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:40:04.549 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:04 smithi096 ceph-mon[38097]: Reconfiguring daemon mon.smithi096 on smithi096 2024-03-24T10:40:05.860 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:05 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:05.861 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:05 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:05.861 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:05 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-24T10:40:05.861 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:05 smithi096 ceph-mon[38097]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-24T10:40:05.861 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:05 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi094.front.sepia.ceph.com:9093"}]: dispatch 2024-03-24T10:40:05.861 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:05 smithi096 ceph-mon[38097]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi094.front.sepia.ceph.com:9093"}]: dispatch 2024-03-24T10:40:05.861 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:05 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:05.861 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:05 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-24T10:40:05.861 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:05 smithi096 ceph-mon[38097]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-24T10:40:05.861 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:05 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi094.front.sepia.ceph.com:3000"}]: dispatch 2024-03-24T10:40:05.861 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:05 smithi096 ceph-mon[38097]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi094.front.sepia.ceph.com:3000"}]: dispatch 2024-03-24T10:40:05.861 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:05 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:05.861 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:05 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T10:40:05.861 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:05 smithi096 ceph-mon[38097]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T10:40:05.862 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:05 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi094.front.sepia.ceph.com:9095"}]: dispatch 2024-03-24T10:40:05.862 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:05 smithi096 ceph-mon[38097]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi094.front.sepia.ceph.com:9095"}]: dispatch 2024-03-24T10:40:05.862 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:05 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:05.862 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:05 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:40:05.862 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:05 smithi096 ceph-mon[38097]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:40:05.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:05 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:05.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:05 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:05.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:05 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-24T10:40:05.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:05 smithi094 ceph-mon[29446]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-24T10:40:05.999 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:05 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi094.front.sepia.ceph.com:9093"}]: dispatch 2024-03-24T10:40:05.999 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:05 smithi094 ceph-mon[29446]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi094.front.sepia.ceph.com:9093"}]: dispatch 2024-03-24T10:40:05.999 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:05 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:05.999 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:05 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-24T10:40:05.999 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:05 smithi094 ceph-mon[29446]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-24T10:40:05.999 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:05 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi094.front.sepia.ceph.com:3000"}]: dispatch 2024-03-24T10:40:05.999 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:05 smithi094 ceph-mon[29446]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi094.front.sepia.ceph.com:3000"}]: dispatch 2024-03-24T10:40:05.999 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:05 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:05.999 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:05 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T10:40:05.999 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:05 smithi094 ceph-mon[29446]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T10:40:05.999 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:05 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi094.front.sepia.ceph.com:9095"}]: dispatch 2024-03-24T10:40:05.999 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:05 smithi094 ceph-mon[29446]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi094.front.sepia.ceph.com:9095"}]: dispatch 2024-03-24T10:40:05.999 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:05 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:05.999 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:05 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:40:05.999 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:05 smithi094 ceph-mon[29446]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:40:07.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:07 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:07.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:07 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:07.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:07 smithi094 ceph-mon[29446]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:40:07.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:07 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:07.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:07 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:07.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:07 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:07.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:07 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:07.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:07 smithi096 ceph-mon[38097]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:40:07.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:07 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:07.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:07 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:10.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:09 smithi096 ceph-mon[38097]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:40:10.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:09 smithi094 ceph-mon[29446]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:40:11.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:11.538+0000 7f115f7fe700 1 -- 172.21.15.94:0/1017662969 <== mgr.14232 v2:172.21.15.94:6800/2184357820 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f117810f960 con 0x7f1160069210 2024-03-24T10:40:11.540 INFO:teuthology.orchestra.run.smithi094.stdout:Created osd(s) 0 on host 'smithi094' 2024-03-24T10:40:11.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:11.541+0000 7f117d048700 1 -- 172.21.15.94:0/1017662969 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f1160069210 msgr2=0x7f116006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:40:11.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:11.541+0000 7f117d048700 1 --2- 172.21.15.94:0/1017662969 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f1160069210 0x7f116006b6c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f116c006fd0 tx=0x7f116c008040 comp rx=0 tx=0).stop 2024-03-24T10:40:11.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:11.541+0000 7f117d048700 1 -- 172.21.15.94:0/1017662969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11780fce40 msgr2=0x7f117810da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:40:11.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:11.541+0000 7f117d048700 1 --2- 172.21.15.94:0/1017662969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11780fce40 0x7f117810da30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f116800b3b0 tx=0x7f1168009e30 comp rx=0 tx=0).stop 2024-03-24T10:40:11.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:11.541+0000 7f117d048700 1 -- 172.21.15.94:0/1017662969 shutdown_connections 2024-03-24T10:40:11.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:11.541+0000 7f117d048700 1 --2- 172.21.15.94:0/1017662969 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f1160069210 0x7f116006b6c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:40:11.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:11.541+0000 7f117d048700 1 --2- 172.21.15.94:0/1017662969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11780fce40 0x7f117810da30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:40:11.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:11.541+0000 7f117d048700 1 --2- 172.21.15.94:0/1017662969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11780fe040 0x7f117810df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:40:11.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:11.541+0000 7f117d048700 1 -- 172.21.15.94:0/1017662969 >> 172.21.15.94:0/1017662969 conn(0x7f11780f8570 msgr2=0x7f11780f9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:40:11.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:11.541+0000 7f117d048700 1 -- 172.21.15.94:0/1017662969 shutdown_connections 2024-03-24T10:40:11.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:11.541+0000 7f117d048700 1 -- 172.21.15.94:0/1017662969 wait complete. 2024-03-24T10:40:12.188 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:11 smithi094 ceph-mon[29446]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:40:12.188 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:11 smithi094 ceph-mon[29446]: from='osd.0 [v2:172.21.15.94:6802/3857839890,v1:172.21.15.94:6803/3857839890]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-24T10:40:12.188 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:11 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:12.188 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:11 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:12.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:11 smithi096 ceph-mon[38097]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:40:12.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:11 smithi096 ceph-mon[38097]: from='osd.0 [v2:172.21.15.94:6802/3857839890,v1:172.21.15.94:6803/3857839890]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-24T10:40:12.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:11 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:12.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:11 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:12.451 DEBUG:teuthology.orchestra.run.smithi094:osd.0> sudo journalctl -f -n 0 -u ceph-386110d4-e9ca-11ee-95cd-87774f69a715@osd.0.service 2024-03-24T10:40:12.454 INFO:tasks.cephadm:Deploying osd.1 on smithi094 with /dev/vg_nvme/lv_3... 2024-03-24T10:40:12.454 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-24T10:40:12.754 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:40:13.175 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:12 smithi094 ceph-mon[29446]: from='osd.0 [v2:172.21.15.94:6802/3857839890,v1:172.21.15.94:6803/3857839890]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-24T10:40:13.175 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:12 smithi094 ceph-mon[29446]: osdmap e7: 1 total, 0 up, 1 in 2024-03-24T10:40:13.175 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:12 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T10:40:13.175 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:12 smithi094 ceph-mon[29446]: from='osd.0 [v2:172.21.15.94:6802/3857839890,v1:172.21.15.94:6803/3857839890]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-03-24T10:40:13.175 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:12 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:13.175 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:12 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:13.176 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:40:12 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-0[47727]: 2024-03-24T10:40:12.909+0000 7fcc87170700 -1 osd.0 0 waiting for initial osdmap 2024-03-24T10:40:13.176 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:40:12 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-0[47727]: 2024-03-24T10:40:12.912+0000 7fcc80768700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T10:40:13.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:12 smithi096 ceph-mon[38097]: from='osd.0 [v2:172.21.15.94:6802/3857839890,v1:172.21.15.94:6803/3857839890]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-24T10:40:13.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:12 smithi096 ceph-mon[38097]: osdmap e7: 1 total, 0 up, 1 in 2024-03-24T10:40:13.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:12 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T10:40:13.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:12 smithi096 ceph-mon[38097]: from='osd.0 [v2:172.21.15.94:6802/3857839890,v1:172.21.15.94:6803/3857839890]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-03-24T10:40:13.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:12 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:13.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:12 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:14.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:13 smithi094 ceph-mon[29446]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:40:14.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:13 smithi094 ceph-mon[29446]: from='osd.0 [v2:172.21.15.94:6802/3857839890,v1:172.21.15.94:6803/3857839890]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-03-24T10:40:14.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:13 smithi094 ceph-mon[29446]: osdmap e8: 1 total, 0 up, 1 in 2024-03-24T10:40:14.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:13 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T10:40:14.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:13 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T10:40:14.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:13 smithi096 ceph-mon[38097]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:40:14.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:13 smithi096 ceph-mon[38097]: from='osd.0 [v2:172.21.15.94:6802/3857839890,v1:172.21.15.94:6803/3857839890]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-03-24T10:40:14.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:13 smithi096 ceph-mon[38097]: osdmap e8: 1 total, 0 up, 1 in 2024-03-24T10:40:14.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:13 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T10:40:14.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:13 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T10:40:14.907 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-24T10:40:14.907 INFO:teuthology.orchestra.run.smithi094.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-24T10:40:14.907 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10+0 records in 2024-03-24T10:40:14.908 INFO:teuthology.orchestra.run.smithi094.stderr:10+0 records out 2024-03-24T10:40:14.908 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111254 s, 943 MB/s 2024-03-24T10:40:14.908 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping successful for: 2024-03-24T10:40:15.215 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:14 smithi094 ceph-mon[29446]: purged_snaps scrub starts 2024-03-24T10:40:15.215 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:14 smithi094 ceph-mon[29446]: purged_snaps scrub ok 2024-03-24T10:40:15.215 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:14 smithi094 ceph-mon[29446]: osd.0 [v2:172.21.15.94:6802/3857839890,v1:172.21.15.94:6803/3857839890] boot 2024-03-24T10:40:15.215 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:14 smithi094 ceph-mon[29446]: osdmap e9: 1 total, 1 up, 1 in 2024-03-24T10:40:15.215 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:14 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T10:40:15.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:14 smithi096 ceph-mon[38097]: purged_snaps scrub starts 2024-03-24T10:40:15.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:14 smithi096 ceph-mon[38097]: purged_snaps scrub ok 2024-03-24T10:40:15.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:14 smithi096 ceph-mon[38097]: osd.0 [v2:172.21.15.94:6802/3857839890,v1:172.21.15.94:6803/3857839890] boot 2024-03-24T10:40:15.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:14 smithi096 ceph-mon[38097]: osdmap e9: 1 total, 1 up, 1 in 2024-03-24T10:40:15.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:14 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T10:40:16.197 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:15 smithi094 ceph-mon[29446]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:40:16.197 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:15 smithi094 ceph-mon[29446]: osdmap e10: 1 total, 1 up, 1 in 2024-03-24T10:40:16.197 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:15 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:40:16.224 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi094:vg_nvme/lv_3 2024-03-24T10:40:16.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:15 smithi096 ceph-mon[38097]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T10:40:16.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:15 smithi096 ceph-mon[38097]: osdmap e10: 1 total, 1 up, 1 in 2024-03-24T10:40:16.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:15 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:40:16.498 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:40:17.962 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:17 smithi094 ceph-mon[29446]: pgmap v36: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-24T10:40:17.963 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:17 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:17.963 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:17 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:17.963 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:17 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T10:40:17.963 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:17 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:17.963 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:17 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:40:17.963 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:17 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:40:17.963 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:17 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:17.963 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:17 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:40:18.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:17 smithi096 ceph-mon[38097]: pgmap v36: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-24T10:40:18.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:17 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:18.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:17 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:18.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:17 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T10:40:18.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:17 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:18.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:17 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:40:18.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:17 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:40:18.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:17 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:18.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:17 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:40:18.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:18.681+0000 7efe6d260700 1 -- 172.21.15.94:0/1785821611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe680fcf80 msgr2=0x7efe680fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:40:18.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:18.681+0000 7efe6d260700 1 --2- 172.21.15.94:0/1785821611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe680fcf80 0x7efe680fd3f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7efe58004940 tx=0x7efe58009e30 comp rx=0 tx=0).stop 2024-03-24T10:40:18.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:18.681+0000 7efe6d260700 1 -- 172.21.15.94:0/1785821611 shutdown_connections 2024-03-24T10:40:18.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:18.681+0000 7efe6d260700 1 --2- 172.21.15.94:0/1785821611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe680fcf80 0x7efe680fd3f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:40:18.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:18.681+0000 7efe6d260700 1 --2- 172.21.15.94:0/1785821611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe680fbcd0 0x7efe680fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:40:18.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:18.681+0000 7efe6d260700 1 -- 172.21.15.94:0/1785821611 >> 172.21.15.94:0/1785821611 conn(0x7efe680f7450 msgr2=0x7efe680f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:40:18.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:18.681+0000 7efe6d260700 1 -- 172.21.15.94:0/1785821611 shutdown_connections 2024-03-24T10:40:18.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:18.681+0000 7efe6d260700 1 -- 172.21.15.94:0/1785821611 wait complete. 2024-03-24T10:40:18.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:18.682+0000 7efe6d260700 1 Processor -- start 2024-03-24T10:40:18.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:18.682+0000 7efe6d260700 1 -- start start 2024-03-24T10:40:18.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:18.682+0000 7efe6d260700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe680fbcd0 0x7efe68066cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:40:18.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:18.682+0000 7efe6d260700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe680fcf80 0x7efe68112700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:40:18.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:18.682+0000 7efe6d260700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe68112c40 con 0x7efe680fbcd0 2024-03-24T10:40:18.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:18.682+0000 7efe6d260700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe68112d80 con 0x7efe680fcf80 2024-03-24T10:40:18.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:18.682+0000 7efe66d9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe680fbcd0 0x7efe68066cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:40:18.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:18.682+0000 7efe6659c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe680fcf80 0x7efe68112700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:40:18.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:18.682+0000 7efe66d9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe680fbcd0 0x7efe68066cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47646/0 (socket says 172.21.15.94:47646) 2024-03-24T10:40:18.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:18.682+0000 7efe66d9d700 1 -- 172.21.15.94:0/4028136301 learned_addr learned my addr 172.21.15.94:0/4028136301 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:40:18.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:18.682+0000 7efe66d9d700 1 -- 172.21.15.94:0/4028136301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe680fcf80 msgr2=0x7efe68112700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:40:18.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:18.682+0000 7efe66d9d700 1 --2- 172.21.15.94:0/4028136301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe680fcf80 0x7efe68112700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:40:18.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:18.682+0000 7efe66d9d700 1 -- 172.21.15.94:0/4028136301 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe58005040 con 0x7efe680fbcd0 2024-03-24T10:40:18.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:18.683+0000 7efe6659c700 1 --2- 172.21.15.94:0/4028136301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe680fcf80 0x7efe68112700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T10:40:18.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:18.683+0000 7efe66d9d700 1 --2- 172.21.15.94:0/4028136301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe680fbcd0 0x7efe68066cf0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efe5c002a50 tx=0x7efe5c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:40:18.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:18.683+0000 7efe4f7fe700 1 -- 172.21.15.94:0/4028136301 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe5c007cc0 con 0x7efe680fbcd0 2024-03-24T10:40:18.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:18.683+0000 7efe6d260700 1 -- 172.21.15.94:0/4028136301 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe68112fb0 con 0x7efe680fbcd0 2024-03-24T10:40:18.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:18.683+0000 7efe4f7fe700 1 -- 172.21.15.94:0/4028136301 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efe5c00f040 con 0x7efe680fbcd0 2024-03-24T10:40:18.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:18.683+0000 7efe6d260700 1 -- 172.21.15.94:0/4028136301 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe68113540 con 0x7efe680fbcd0 2024-03-24T10:40:18.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:18.683+0000 7efe4f7fe700 1 -- 172.21.15.94:0/4028136301 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe5c00d9d0 con 0x7efe680fbcd0 2024-03-24T10:40:18.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:18.684+0000 7efe6d260700 1 -- 172.21.15.94:0/4028136301 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe6804c710 con 0x7efe680fbcd0 2024-03-24T10:40:18.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:18.684+0000 7efe4f7fe700 1 -- 172.21.15.94:0/4028136301 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7efe5c00dc60 con 0x7efe680fbcd0 2024-03-24T10:40:18.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:18.685+0000 7efe4f7fe700 1 --2- 172.21.15.94:0/4028136301 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7efe50069210 0x7efe5006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:40:18.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:18.685+0000 7efe4f7fe700 1 -- 172.21.15.94:0/4028136301 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1978+0+0 (secure 0 0 0) 0x7efe5c086070 con 0x7efe680fbcd0 2024-03-24T10:40:18.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:18.685+0000 7efe6659c700 1 --2- 172.21.15.94:0/4028136301 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7efe50069210 0x7efe5006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:40:18.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:18.685+0000 7efe6659c700 1 --2- 172.21.15.94:0/4028136301 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7efe50069210 0x7efe5006b6c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7efe5800b3b0 tx=0x7efe58009e30 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:40:18.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:18.689+0000 7efe4f7fe700 1 -- 172.21.15.94:0/4028136301 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7efe5c056c30 con 0x7efe680fbcd0 2024-03-24T10:40:18.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:18.893+0000 7efe6d260700 1 -- 172.21.15.94:0/4028136301 --> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7efe68101310 con 0x7efe50069210 2024-03-24T10:40:19.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:18 smithi094 ceph-mon[29446]: Detected new or changed devices on smithi094 2024-03-24T10:40:19.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:18 smithi094 ceph-mon[29446]: Adjusting osd_memory_target on smithi094 to 12864M 2024-03-24T10:40:19.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:18 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T10:40:19.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:18 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T10:40:19.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:18 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:40:19.249 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:18 smithi096 ceph-mon[38097]: Detected new or changed devices on smithi094 2024-03-24T10:40:19.249 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:18 smithi096 ceph-mon[38097]: Adjusting osd_memory_target on smithi094 to 12864M 2024-03-24T10:40:19.249 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:18 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T10:40:19.249 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:18 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T10:40:19.250 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:18 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:40:20.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:19 smithi094 ceph-mon[29446]: pgmap v37: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-24T10:40:20.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:19 smithi094 ceph-mon[29446]: from='client.14284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:40:20.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:19 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:20.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:19 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:20.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:19 smithi096 ceph-mon[38097]: pgmap v37: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-24T10:40:20.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:19 smithi096 ceph-mon[38097]: from='client.14284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:40:20.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:19 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:20.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:19 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:22.206 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:21 smithi094 ceph-mon[29446]: pgmap v38: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-24T10:40:22.206 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:21 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/335292230' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3087b482-aede-4e88-848c-3e7438eaccef"}]: dispatch 2024-03-24T10:40:22.206 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:21 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/335292230' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3087b482-aede-4e88-848c-3e7438eaccef"}]': finished 2024-03-24T10:40:22.206 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:21 smithi094 ceph-mon[29446]: osdmap e11: 2 total, 1 up, 2 in 2024-03-24T10:40:22.206 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:21 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T10:40:22.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:21 smithi096 ceph-mon[38097]: pgmap v38: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-24T10:40:22.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:21 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/335292230' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3087b482-aede-4e88-848c-3e7438eaccef"}]: dispatch 2024-03-24T10:40:22.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:21 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/335292230' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3087b482-aede-4e88-848c-3e7438eaccef"}]': finished 2024-03-24T10:40:22.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:21 smithi096 ceph-mon[38097]: osdmap e11: 2 total, 1 up, 2 in 2024-03-24T10:40:22.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:21 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T10:40:22.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:22 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/3702900524' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T10:40:23.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:22 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/3702900524' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T10:40:24.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:23 smithi094 ceph-mon[29446]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-24T10:40:24.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:23 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:24.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:23 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:24.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:23 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:40:24.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:23 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:40:24.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:23 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:24.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:23 smithi096 ceph-mon[38097]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-24T10:40:24.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:23 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:24.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:23 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:24.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:23 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:40:24.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:23 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:40:24.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:23 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:26.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:25 smithi094 ceph-mon[29446]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-24T10:40:26.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:25 smithi096 ceph-mon[38097]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-24T10:40:28.108 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:27 smithi094 ceph-mon[29446]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-24T10:40:28.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:27 smithi096 ceph-mon[38097]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-24T10:40:29.137 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:28 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-24T10:40:29.137 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:28 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:40:29.316 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:28 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-24T10:40:29.316 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:28 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:40:29.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:29 smithi094 ceph-mon[29446]: Deploying daemon osd.1 on smithi094 2024-03-24T10:40:29.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:29 smithi094 ceph-mon[29446]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-24T10:40:30.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:29 smithi096 ceph-mon[38097]: Deploying daemon osd.1 on smithi094 2024-03-24T10:40:30.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:29 smithi096 ceph-mon[38097]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-24T10:40:31.243 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:30 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:40:31.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:30 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:40:32.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:31 smithi094 ceph-mon[29446]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-24T10:40:32.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:31 smithi096 ceph-mon[38097]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-24T10:40:34.089 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:33 smithi094 ceph-mon[29446]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-24T10:40:34.089 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:33 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:34.089 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:33 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:40:34.089 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:33 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:34.276 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:33 smithi096 ceph-mon[38097]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-24T10:40:34.276 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:33 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:34.276 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:33 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:40:34.276 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:33 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:36.391 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:36 smithi094 ceph-mon[29446]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-24T10:40:36.392 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:36 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:36.392 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:36 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:36.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:36 smithi096 ceph-mon[38097]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-24T10:40:36.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:36 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:36.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:36 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:37.247 INFO:teuthology.orchestra.run.smithi094.stdout:Created osd(s) 1 on host 'smithi094' 2024-03-24T10:40:37.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:37.245+0000 7efe4f7fe700 1 -- 172.21.15.94:0/4028136301 <== mgr.14232 v2:172.21.15.94:6800/2184357820 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7efe68101310 con 0x7efe50069210 2024-03-24T10:40:37.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:37.248+0000 7efe6d260700 1 -- 172.21.15.94:0/4028136301 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7efe50069210 msgr2=0x7efe5006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:40:37.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:37.248+0000 7efe6d260700 1 --2- 172.21.15.94:0/4028136301 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7efe50069210 0x7efe5006b6c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7efe5800b3b0 tx=0x7efe58009e30 comp rx=0 tx=0).stop 2024-03-24T10:40:37.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:37.248+0000 7efe6d260700 1 -- 172.21.15.94:0/4028136301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe680fbcd0 msgr2=0x7efe68066cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:40:37.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:37.248+0000 7efe6d260700 1 --2- 172.21.15.94:0/4028136301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe680fbcd0 0x7efe68066cf0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efe5c002a50 tx=0x7efe5c0046b0 comp rx=0 tx=0).stop 2024-03-24T10:40:37.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:37.248+0000 7efe6d260700 1 -- 172.21.15.94:0/4028136301 shutdown_connections 2024-03-24T10:40:37.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:37.248+0000 7efe6d260700 1 --2- 172.21.15.94:0/4028136301 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7efe50069210 0x7efe5006b6c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:40:37.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:37.248+0000 7efe6d260700 1 --2- 172.21.15.94:0/4028136301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe680fbcd0 0x7efe68066cf0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:40:37.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:37.248+0000 7efe6d260700 1 --2- 172.21.15.94:0/4028136301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe680fcf80 0x7efe68112700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:40:37.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:37.248+0000 7efe6d260700 1 -- 172.21.15.94:0/4028136301 >> 172.21.15.94:0/4028136301 conn(0x7efe680f7450 msgr2=0x7efe680f8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:40:37.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:37.248+0000 7efe6d260700 1 -- 172.21.15.94:0/4028136301 shutdown_connections 2024-03-24T10:40:37.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:37.248+0000 7efe6d260700 1 -- 172.21.15.94:0/4028136301 wait complete. 2024-03-24T10:40:38.480 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:38 smithi094 ceph-mon[29446]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-24T10:40:38.480 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:38 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:38.480 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:38 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:38.480 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:38 smithi094 ceph-mon[29446]: from='osd.1 [v2:172.21.15.94:6810/2028968832,v1:172.21.15.94:6811/2028968832]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-24T10:40:38.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:38 smithi096 ceph-mon[38097]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-24T10:40:38.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:38 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:38.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:38 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:38.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:38 smithi096 ceph-mon[38097]: from='osd.1 [v2:172.21.15.94:6810/2028968832,v1:172.21.15.94:6811/2028968832]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-24T10:40:38.800 DEBUG:teuthology.orchestra.run.smithi094:osd.1> sudo journalctl -f -n 0 -u ceph-386110d4-e9ca-11ee-95cd-87774f69a715@osd.1.service 2024-03-24T10:40:38.802 INFO:tasks.cephadm:Deploying osd.2 on smithi094 with /dev/vg_nvme/lv_2... 2024-03-24T10:40:38.802 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-24T10:40:39.190 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:40:39.514 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:39 smithi094 ceph-mon[29446]: from='osd.1 [v2:172.21.15.94:6810/2028968832,v1:172.21.15.94:6811/2028968832]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-24T10:40:39.514 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:39 smithi094 ceph-mon[29446]: osdmap e12: 2 total, 1 up, 2 in 2024-03-24T10:40:39.514 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:39 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T10:40:39.515 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:39 smithi094 ceph-mon[29446]: from='osd.1 [v2:172.21.15.94:6810/2028968832,v1:172.21.15.94:6811/2028968832]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-03-24T10:40:39.515 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:40:39 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-1[55045]: 2024-03-24T10:40:39.269+0000 7f4d1d321700 -1 osd.1 0 waiting for initial osdmap 2024-03-24T10:40:39.515 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:40:39 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-1[55045]: 2024-03-24T10:40:39.272+0000 7f4d1791b700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T10:40:39.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:39 smithi096 ceph-mon[38097]: from='osd.1 [v2:172.21.15.94:6810/2028968832,v1:172.21.15.94:6811/2028968832]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-24T10:40:39.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:39 smithi096 ceph-mon[38097]: osdmap e12: 2 total, 1 up, 2 in 2024-03-24T10:40:39.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:39 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T10:40:39.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:39 smithi096 ceph-mon[38097]: from='osd.1 [v2:172.21.15.94:6810/2028968832,v1:172.21.15.94:6811/2028968832]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-03-24T10:40:40.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:40 smithi094 ceph-mon[29446]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-24T10:40:40.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:40 smithi094 ceph-mon[29446]: from='osd.1 [v2:172.21.15.94:6810/2028968832,v1:172.21.15.94:6811/2028968832]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-03-24T10:40:40.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:40 smithi094 ceph-mon[29446]: osdmap e13: 2 total, 1 up, 2 in 2024-03-24T10:40:40.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:40 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T10:40:40.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:40 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T10:40:40.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:40 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:40.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:40 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:40.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:40 smithi096 ceph-mon[38097]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-24T10:40:40.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:40 smithi096 ceph-mon[38097]: from='osd.1 [v2:172.21.15.94:6810/2028968832,v1:172.21.15.94:6811/2028968832]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-03-24T10:40:40.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:40 smithi096 ceph-mon[38097]: osdmap e13: 2 total, 1 up, 2 in 2024-03-24T10:40:40.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:40 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T10:40:40.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:40 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T10:40:40.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:40 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:40.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:40 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:41.340 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:41 smithi094 ceph-mon[29446]: purged_snaps scrub starts 2024-03-24T10:40:41.340 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:41 smithi094 ceph-mon[29446]: purged_snaps scrub ok 2024-03-24T10:40:41.340 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:41 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T10:40:41.340 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:41 smithi094 ceph-mon[29446]: osd.1 [v2:172.21.15.94:6810/2028968832,v1:172.21.15.94:6811/2028968832] boot 2024-03-24T10:40:41.340 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:41 smithi094 ceph-mon[29446]: osdmap e14: 2 total, 2 up, 2 in 2024-03-24T10:40:41.341 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:41 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T10:40:41.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:41 smithi096 ceph-mon[38097]: purged_snaps scrub starts 2024-03-24T10:40:41.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:41 smithi096 ceph-mon[38097]: purged_snaps scrub ok 2024-03-24T10:40:41.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:41 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T10:40:41.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:41 smithi096 ceph-mon[38097]: osd.1 [v2:172.21.15.94:6810/2028968832,v1:172.21.15.94:6811/2028968832] boot 2024-03-24T10:40:41.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:41 smithi096 ceph-mon[38097]: osdmap e14: 2 total, 2 up, 2 in 2024-03-24T10:40:41.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:41 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T10:40:42.476 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:42 smithi094 ceph-mon[29446]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-24T10:40:42.476 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:42 smithi094 ceph-mon[29446]: osdmap e15: 2 total, 2 up, 2 in 2024-03-24T10:40:42.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:42 smithi096 ceph-mon[38097]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-24T10:40:42.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:42 smithi096 ceph-mon[38097]: osdmap e15: 2 total, 2 up, 2 in 2024-03-24T10:40:43.059 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-24T10:40:43.059 INFO:teuthology.orchestra.run.smithi094.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-24T10:40:43.059 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10+0 records in 2024-03-24T10:40:43.059 INFO:teuthology.orchestra.run.smithi094.stderr:10+0 records out 2024-03-24T10:40:43.059 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00967422 s, 1.1 GB/s 2024-03-24T10:40:43.059 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping successful for: 2024-03-24T10:40:43.881 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi094:vg_nvme/lv_2 2024-03-24T10:40:44.329 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:40:44.495 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:44 smithi094 ceph-mon[29446]: pgmap v54: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-24T10:40:44.495 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:44 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:44.495 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:44 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:44.495 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:44 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T10:40:44.496 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:44 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T10:40:44.496 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:44 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:44.496 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:44 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:40:44.496 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:44 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:40:44.496 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:44 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:44.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:44 smithi096 ceph-mon[38097]: pgmap v54: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-24T10:40:44.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:44 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:44.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:44 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:44.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:44 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T10:40:44.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:44 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T10:40:44.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:44 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:44.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:44 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:40:44.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:44 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:40:44.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:44 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:45.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:45 smithi096 ceph-mon[38097]: Detected new or changed devices on smithi094 2024-03-24T10:40:45.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:45 smithi096 ceph-mon[38097]: Adjusting osd_memory_target on smithi094 to 6432M 2024-03-24T10:40:45.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:45 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:40:45.619 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:45 smithi094 ceph-mon[29446]: Detected new or changed devices on smithi094 2024-03-24T10:40:45.619 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:45 smithi094 ceph-mon[29446]: Adjusting osd_memory_target on smithi094 to 6432M 2024-03-24T10:40:45.619 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:45 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:40:45.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:45.930+0000 7f4d3d242700 1 -- 172.21.15.94:0/1645199274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d380fbcd0 msgr2=0x7f4d380fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:40:45.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:45.930+0000 7f4d3d242700 1 --2- 172.21.15.94:0/1645199274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d380fbcd0 0x7f4d380fc0e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4d28004940 tx=0x7f4d28009e30 comp rx=0 tx=0).stop 2024-03-24T10:40:45.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:45.930+0000 7f4d3d242700 1 -- 172.21.15.94:0/1645199274 shutdown_connections 2024-03-24T10:40:45.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:45.930+0000 7f4d3d242700 1 --2- 172.21.15.94:0/1645199274 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d380fcf80 0x7f4d380fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:40:45.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:45.930+0000 7f4d3d242700 1 --2- 172.21.15.94:0/1645199274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d380fbcd0 0x7f4d380fc0e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:40:45.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:45.930+0000 7f4d3d242700 1 -- 172.21.15.94:0/1645199274 >> 172.21.15.94:0/1645199274 conn(0x7f4d380f7450 msgr2=0x7f4d380f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:40:45.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:45.930+0000 7f4d3d242700 1 -- 172.21.15.94:0/1645199274 shutdown_connections 2024-03-24T10:40:45.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:45.930+0000 7f4d3d242700 1 -- 172.21.15.94:0/1645199274 wait complete. 2024-03-24T10:40:45.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:45.931+0000 7f4d3d242700 1 Processor -- start 2024-03-24T10:40:45.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:45.931+0000 7f4d3d242700 1 -- start start 2024-03-24T10:40:45.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:45.931+0000 7f4d3d242700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d380fbcd0 0x7f4d3810c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:40:45.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:45.931+0000 7f4d3d242700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d380fcf80 0x7f4d3810cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:40:45.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:45.931+0000 7f4d3d242700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3810d400 con 0x7f4d380fbcd0 2024-03-24T10:40:45.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:45.931+0000 7f4d3d242700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d38111da0 con 0x7f4d380fcf80 2024-03-24T10:40:45.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:45.932+0000 7f4d36d9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d380fbcd0 0x7f4d3810c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:40:45.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:45.932+0000 7f4d36d9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d380fbcd0 0x7f4d3810c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48706/0 (socket says 172.21.15.94:48706) 2024-03-24T10:40:45.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:45.932+0000 7f4d36d9d700 1 -- 172.21.15.94:0/2612558807 learned_addr learned my addr 172.21.15.94:0/2612558807 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:40:45.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:45.932+0000 7f4d3659c700 1 --2- 172.21.15.94:0/2612558807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d380fcf80 0x7f4d3810cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:40:45.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:45.932+0000 7f4d36d9d700 1 -- 172.21.15.94:0/2612558807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d380fcf80 msgr2=0x7f4d3810cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:40:45.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:45.932+0000 7f4d36d9d700 1 --2- 172.21.15.94:0/2612558807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d380fcf80 0x7f4d3810cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:40:45.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:45.932+0000 7f4d36d9d700 1 -- 172.21.15.94:0/2612558807 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d28005040 con 0x7f4d380fbcd0 2024-03-24T10:40:45.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:45.932+0000 7f4d36d9d700 1 --2- 172.21.15.94:0/2612558807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d380fbcd0 0x7f4d3810c830 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4d2800b3b0 tx=0x7f4d28009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:40:45.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:45.932+0000 7f4d1f7fe700 1 -- 172.21.15.94:0/2612558807 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d2800e040 con 0x7f4d380fbcd0 2024-03-24T10:40:45.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:45.932+0000 7f4d1f7fe700 1 -- 172.21.15.94:0/2612558807 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4d2800bcc0 con 0x7f4d380fbcd0 2024-03-24T10:40:45.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:45.932+0000 7f4d3d242700 1 -- 172.21.15.94:0/2612558807 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d38111fa0 con 0x7f4d380fbcd0 2024-03-24T10:40:45.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:45.932+0000 7f4d1f7fe700 1 -- 172.21.15.94:0/2612558807 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d2801c630 con 0x7f4d380fbcd0 2024-03-24T10:40:45.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:45.932+0000 7f4d3d242700 1 -- 172.21.15.94:0/2612558807 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d38112480 con 0x7f4d380fbcd0 2024-03-24T10:40:45.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:45.933+0000 7f4d3d242700 1 -- 172.21.15.94:0/2612558807 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d3804c710 con 0x7f4d380fbcd0 2024-03-24T10:40:45.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:45.934+0000 7f4d1f7fe700 1 -- 172.21.15.94:0/2612558807 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7f4d28013980 con 0x7f4d380fbcd0 2024-03-24T10:40:45.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:45.934+0000 7f4d1f7fe700 1 --2- 172.21.15.94:0/2612558807 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f4d20069210 0x7f4d2006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:40:45.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:45.934+0000 7f4d1f7fe700 1 -- 172.21.15.94:0/2612558807 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2410+0+0 (secure 0 0 0) 0x7f4d280872a0 con 0x7f4d380fbcd0 2024-03-24T10:40:45.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:45.934+0000 7f4d3659c700 1 --2- 172.21.15.94:0/2612558807 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f4d20069210 0x7f4d2006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:40:45.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:45.934+0000 7f4d3659c700 1 --2- 172.21.15.94:0/2612558807 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f4d20069210 0x7f4d2006b6c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c006fd0 tx=0x7f4d2c008040 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:40:45.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:45.938+0000 7f4d1f7fe700 1 -- 172.21.15.94:0/2612558807 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f4d28057cb0 con 0x7f4d380fbcd0 2024-03-24T10:40:46.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:40:46.146+0000 7f4d3d242700 1 -- 172.21.15.94:0/2612558807 --> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f4d38101260 con 0x7f4d20069210 2024-03-24T10:40:46.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:46 smithi096 ceph-mon[38097]: pgmap v55: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-24T10:40:46.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:46 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:40:46.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:46 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T10:40:46.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:46 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T10:40:46.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:46 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:40:46.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:46 smithi094 ceph-mon[29446]: pgmap v55: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-24T10:40:46.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:46 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:40:46.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:46 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T10:40:46.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:46 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T10:40:46.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:46 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:40:47.446 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:47 smithi094 ceph-mon[29446]: from='client.14302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:40:47.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:47 smithi096 ceph-mon[38097]: from='client.14302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:40:48.336 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:48 smithi094 ceph-mon[29446]: pgmap v56: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-24T10:40:48.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:48 smithi096 ceph-mon[38097]: pgmap v56: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-24T10:40:50.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:50 smithi096 ceph-mon[38097]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-24T10:40:50.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:50 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/2188963353' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48c00627-22a6-480c-9aef-38bc326fc4dd"}]: dispatch 2024-03-24T10:40:50.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:50 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/2188963353' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "48c00627-22a6-480c-9aef-38bc326fc4dd"}]': finished 2024-03-24T10:40:50.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:50 smithi096 ceph-mon[38097]: osdmap e16: 3 total, 2 up, 3 in 2024-03-24T10:40:50.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:50 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T10:40:50.614 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:50 smithi094 ceph-mon[29446]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-24T10:40:50.614 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:50 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/2188963353' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48c00627-22a6-480c-9aef-38bc326fc4dd"}]: dispatch 2024-03-24T10:40:50.614 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:50 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/2188963353' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "48c00627-22a6-480c-9aef-38bc326fc4dd"}]': finished 2024-03-24T10:40:50.614 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:50 smithi094 ceph-mon[29446]: osdmap e16: 3 total, 2 up, 3 in 2024-03-24T10:40:50.614 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:50 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T10:40:51.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:51 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/3327468176' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T10:40:51.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:51 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:51.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:51 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:51.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:51 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:40:51.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:51 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:40:51.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:51 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:51.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:51 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/3327468176' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T10:40:51.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:51 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:51.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:51 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:51.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:51 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:40:51.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:51 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:40:51.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:51 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:40:52.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:52 smithi096 ceph-mon[38097]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-24T10:40:52.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:52 smithi094 ceph-mon[29446]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-24T10:40:54.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:54 smithi094 ceph-mon[29446]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-24T10:40:54.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:54 smithi096 ceph-mon[38097]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-24T10:40:56.437 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:56 smithi094 ceph-mon[29446]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-24T10:40:56.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:56 smithi096 ceph-mon[38097]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-24T10:40:57.483 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:57 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-24T10:40:57.483 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:57 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:40:57.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:57 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-24T10:40:57.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:57 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:40:58.423 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:58 smithi094 ceph-mon[29446]: Deploying daemon osd.2 on smithi094 2024-03-24T10:40:58.423 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:40:58 smithi094 ceph-mon[29446]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-24T10:40:58.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:58 smithi096 ceph-mon[38097]: Deploying daemon osd.2 on smithi094 2024-03-24T10:40:58.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:40:58 smithi096 ceph-mon[38097]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-24T10:41:00.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:00 smithi094 ceph-mon[29446]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-24T10:41:00.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:00 smithi096 ceph-mon[38097]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-24T10:41:01.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:01 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:41:01.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:01 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:41:02.434 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:02 smithi094 ceph-mon[29446]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-24T10:41:02.434 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:02 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:02.434 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:02 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:02.434 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:02 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:41:02.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:02 smithi096 ceph-mon[38097]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-24T10:41:02.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:02 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:02.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:02 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:02.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:02 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:41:04.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:04 smithi096 ceph-mon[38097]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-24T10:41:04.634 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:04 smithi094 ceph-mon[29446]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-24T10:41:05.060 INFO:teuthology.orchestra.run.smithi094.stdout:Created osd(s) 2 on host 'smithi094' 2024-03-24T10:41:05.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:41:05.060+0000 7f4d1f7fe700 1 -- 172.21.15.94:0/2612558807 <== mgr.14232 v2:172.21.15.94:6800/2184357820 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f4d38101260 con 0x7f4d20069210 2024-03-24T10:41:05.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:41:05.062+0000 7f4d3d242700 1 -- 172.21.15.94:0/2612558807 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f4d20069210 msgr2=0x7f4d2006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:41:05.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:41:05.062+0000 7f4d3d242700 1 --2- 172.21.15.94:0/2612558807 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f4d20069210 0x7f4d2006b6c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c006fd0 tx=0x7f4d2c008040 comp rx=0 tx=0).stop 2024-03-24T10:41:05.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:41:05.062+0000 7f4d3d242700 1 -- 172.21.15.94:0/2612558807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d380fbcd0 msgr2=0x7f4d3810c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:41:05.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:41:05.062+0000 7f4d3d242700 1 --2- 172.21.15.94:0/2612558807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d380fbcd0 0x7f4d3810c830 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4d2800b3b0 tx=0x7f4d28009e30 comp rx=0 tx=0).stop 2024-03-24T10:41:05.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:41:05.062+0000 7f4d3d242700 1 -- 172.21.15.94:0/2612558807 shutdown_connections 2024-03-24T10:41:05.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:41:05.062+0000 7f4d3d242700 1 --2- 172.21.15.94:0/2612558807 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f4d20069210 0x7f4d2006b6c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:41:05.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:41:05.062+0000 7f4d3d242700 1 --2- 172.21.15.94:0/2612558807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d380fbcd0 0x7f4d3810c830 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:41:05.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:41:05.062+0000 7f4d3d242700 1 --2- 172.21.15.94:0/2612558807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d380fcf80 0x7f4d3810cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:41:05.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:41:05.062+0000 7f4d3d242700 1 -- 172.21.15.94:0/2612558807 >> 172.21.15.94:0/2612558807 conn(0x7f4d380f7450 msgr2=0x7f4d380f8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:41:05.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:41:05.062+0000 7f4d3d242700 1 -- 172.21.15.94:0/2612558807 shutdown_connections 2024-03-24T10:41:05.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:41:05.062+0000 7f4d3d242700 1 -- 172.21.15.94:0/2612558807 wait complete. 2024-03-24T10:41:05.851 DEBUG:teuthology.orchestra.run.smithi094:osd.2> sudo journalctl -f -n 0 -u ceph-386110d4-e9ca-11ee-95cd-87774f69a715@osd.2.service 2024-03-24T10:41:05.853 INFO:tasks.cephadm:Deploying osd.3 on smithi096 with /dev/vg_nvme/lv_4... 2024-03-24T10:41:05.854 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-24T10:41:06.110 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:06 smithi094 ceph-mon[29446]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-24T10:41:06.110 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:06 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:06.110 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:06 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:06.110 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:06 smithi094 ceph-mon[29446]: from='osd.2 [v2:172.21.15.94:6818/2122131371,v1:172.21.15.94:6819/2122131371]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-24T10:41:06.114 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi096/config 2024-03-24T10:41:06.236 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:06 smithi096 ceph-mon[38097]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-24T10:41:06.236 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:06 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:06.236 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:06 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:06.236 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:06 smithi096 ceph-mon[38097]: from='osd.2 [v2:172.21.15.94:6818/2122131371,v1:172.21.15.94:6819/2122131371]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-24T10:41:07.395 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:07 smithi094 ceph-mon[29446]: from='osd.2 [v2:172.21.15.94:6818/2122131371,v1:172.21.15.94:6819/2122131371]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-24T10:41:07.395 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:07 smithi094 ceph-mon[29446]: osdmap e17: 3 total, 2 up, 3 in 2024-03-24T10:41:07.395 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:07 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T10:41:07.395 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:07 smithi094 ceph-mon[29446]: from='osd.2 [v2:172.21.15.94:6818/2122131371,v1:172.21.15.94:6819/2122131371]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-03-24T10:41:07.396 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:41:07 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-2[62439]: 2024-03-24T10:41:07.326+0000 7fc7af4a4700 -1 osd.2 0 waiting for initial osdmap 2024-03-24T10:41:07.396 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:41:07 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-2[62439]: 2024-03-24T10:41:07.331+0000 7fc7aa29f700 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T10:41:07.434 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:07 smithi096 ceph-mon[38097]: from='osd.2 [v2:172.21.15.94:6818/2122131371,v1:172.21.15.94:6819/2122131371]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-24T10:41:07.434 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:07 smithi096 ceph-mon[38097]: osdmap e17: 3 total, 2 up, 3 in 2024-03-24T10:41:07.434 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:07 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T10:41:07.434 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:07 smithi096 ceph-mon[38097]: from='osd.2 [v2:172.21.15.94:6818/2122131371,v1:172.21.15.94:6819/2122131371]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-03-24T10:41:07.998 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-24T10:41:07.999 INFO:teuthology.orchestra.run.smithi096.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-24T10:41:07.999 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10+0 records in 2024-03-24T10:41:07.999 INFO:teuthology.orchestra.run.smithi096.stderr:10+0 records out 2024-03-24T10:41:07.999 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0121629 s, 862 MB/s 2024-03-24T10:41:07.999 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping successful for: 2024-03-24T10:41:08.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:08 smithi094 ceph-mon[29446]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-24T10:41:08.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:08 smithi094 ceph-mon[29446]: from='osd.2 [v2:172.21.15.94:6818/2122131371,v1:172.21.15.94:6819/2122131371]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-03-24T10:41:08.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:08 smithi094 ceph-mon[29446]: osdmap e18: 3 total, 2 up, 3 in 2024-03-24T10:41:08.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:08 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T10:41:08.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:08 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T10:41:08.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:08 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T10:41:08.562 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:08 smithi096 ceph-mon[38097]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-24T10:41:08.562 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:08 smithi096 ceph-mon[38097]: from='osd.2 [v2:172.21.15.94:6818/2122131371,v1:172.21.15.94:6819/2122131371]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-03-24T10:41:08.562 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:08 smithi096 ceph-mon[38097]: osdmap e18: 3 total, 2 up, 3 in 2024-03-24T10:41:08.562 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:08 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T10:41:08.562 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:08 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T10:41:08.562 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:08 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T10:41:08.586 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi096:vg_nvme/lv_4 2024-03-24T10:41:08.835 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi096/config 2024-03-24T10:41:09.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:09 smithi094 ceph-mon[29446]: purged_snaps scrub starts 2024-03-24T10:41:09.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:09 smithi094 ceph-mon[29446]: purged_snaps scrub ok 2024-03-24T10:41:09.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:09 smithi094 ceph-mon[29446]: osd.2 [v2:172.21.15.94:6818/2122131371,v1:172.21.15.94:6819/2122131371] boot 2024-03-24T10:41:09.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:09 smithi094 ceph-mon[29446]: osdmap e19: 3 total, 3 up, 3 in 2024-03-24T10:41:09.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:09 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T10:41:09.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:09 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:09.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:09 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:09.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:09 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-03-24T10:41:09.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:09 smithi096 ceph-mon[38097]: purged_snaps scrub starts 2024-03-24T10:41:09.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:09 smithi096 ceph-mon[38097]: purged_snaps scrub ok 2024-03-24T10:41:09.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:09 smithi096 ceph-mon[38097]: osd.2 [v2:172.21.15.94:6818/2122131371,v1:172.21.15.94:6819/2122131371] boot 2024-03-24T10:41:09.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:09 smithi096 ceph-mon[38097]: osdmap e19: 3 total, 3 up, 3 in 2024-03-24T10:41:09.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:09 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T10:41:09.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:09 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:09.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:09 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:09.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:09 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-03-24T10:41:10.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:10.404+0000 7fa7e30b6700 1 -- 172.21.15.96:0/234853970 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7dc0fcf80 msgr2=0x7fa7dc0fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:41:10.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:10.404+0000 7fa7e30b6700 1 --2- 172.21.15.96:0/234853970 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7dc0fcf80 0x7fa7dc0fd3f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8004940 tx=0x7fa7c8009e30 comp rx=0 tx=0).stop 2024-03-24T10:41:10.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:10.404+0000 7fa7e30b6700 1 -- 172.21.15.96:0/234853970 shutdown_connections 2024-03-24T10:41:10.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:10.404+0000 7fa7e30b6700 1 --2- 172.21.15.96:0/234853970 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7dc0fcf80 0x7fa7dc0fd3f0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:41:10.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:10.404+0000 7fa7e30b6700 1 --2- 172.21.15.96:0/234853970 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7dc0fbcd0 0x7fa7dc0fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:41:10.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:10.404+0000 7fa7e30b6700 1 -- 172.21.15.96:0/234853970 >> 172.21.15.96:0/234853970 conn(0x7fa7dc0f7450 msgr2=0x7fa7dc0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:41:10.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:10.404+0000 7fa7e30b6700 1 -- 172.21.15.96:0/234853970 shutdown_connections 2024-03-24T10:41:10.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:10.404+0000 7fa7e30b6700 1 -- 172.21.15.96:0/234853970 wait complete. 2024-03-24T10:41:10.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:10.404+0000 7fa7e30b6700 1 Processor -- start 2024-03-24T10:41:10.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:10.404+0000 7fa7e30b6700 1 -- start start 2024-03-24T10:41:10.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:10.404+0000 7fa7e30b6700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7dc0fbcd0 0x7fa7dc066cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:41:10.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:10.404+0000 7fa7e30b6700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7dc0fcf80 0x7fa7dc112740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:41:10.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:10.404+0000 7fa7e30b6700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7dc112c80 con 0x7fa7dc0fbcd0 2024-03-24T10:41:10.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:10.404+0000 7fa7e30b6700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7dc112dc0 con 0x7fa7dc0fcf80 2024-03-24T10:41:10.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:10.405+0000 7fa7dbfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7dc0fcf80 0x7fa7dc112740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:41:10.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:10.405+0000 7fa7dbfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7dc0fcf80 0x7fa7dc112740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:46812/0 (socket says 172.21.15.96:46812) 2024-03-24T10:41:10.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:10.405+0000 7fa7dbfff700 1 -- 172.21.15.96:0/2441690491 learned_addr learned my addr 172.21.15.96:0/2441690491 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-24T10:41:10.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:10.405+0000 7fa7e0e52700 1 --2- 172.21.15.96:0/2441690491 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7dc0fbcd0 0x7fa7dc066cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:41:10.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:10.405+0000 7fa7dbfff700 1 -- 172.21.15.96:0/2441690491 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7dc0fbcd0 msgr2=0x7fa7dc066cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:41:10.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:10.405+0000 7fa7dbfff700 1 --2- 172.21.15.96:0/2441690491 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7dc0fbcd0 0x7fa7dc066cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:41:10.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:10.405+0000 7fa7dbfff700 1 -- 172.21.15.96:0/2441690491 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c8005040 con 0x7fa7dc0fcf80 2024-03-24T10:41:10.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:10.405+0000 7fa7dbfff700 1 --2- 172.21.15.96:0/2441690491 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7dc0fcf80 0x7fa7dc112740 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8009150 tx=0x7fa7c8009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:41:10.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:10.405+0000 7fa7d97fa700 1 -- 172.21.15.96:0/2441690491 <== mon.1 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7c800e040 con 0x7fa7dc0fcf80 2024-03-24T10:41:10.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:10.405+0000 7fa7d97fa700 1 -- 172.21.15.96:0/2441690491 <== mon.1 v2:172.21.15.96:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa7c800bcc0 con 0x7fa7dc0fcf80 2024-03-24T10:41:10.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:10.405+0000 7fa7e30b6700 1 -- 172.21.15.96:0/2441690491 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7dc112fc0 con 0x7fa7dc0fcf80 2024-03-24T10:41:10.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:10.405+0000 7fa7d97fa700 1 -- 172.21.15.96:0/2441690491 <== mon.1 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7c801c630 con 0x7fa7dc0fcf80 2024-03-24T10:41:10.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:10.405+0000 7fa7e30b6700 1 -- 172.21.15.96:0/2441690491 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7dc113550 con 0x7fa7dc0fcf80 2024-03-24T10:41:10.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:10.406+0000 7fa7e30b6700 1 -- 172.21.15.96:0/2441690491 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7dc04c710 con 0x7fa7dc0fcf80 2024-03-24T10:41:10.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:10.407+0000 7fa7d97fa700 1 -- 172.21.15.96:0/2441690491 <== mon.1 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7fa7c8013420 con 0x7fa7dc0fcf80 2024-03-24T10:41:10.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:10.407+0000 7fa7d97fa700 1 --2- 172.21.15.96:0/2441690491 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fa7cc069210 0x7fa7cc06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:41:10.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:10.407+0000 7fa7d97fa700 1 -- 172.21.15.96:0/2441690491 <== mon.1 v2:172.21.15.96:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3228+0+0 (secure 0 0 0) 0x7fa7c8087440 con 0x7fa7dc0fcf80 2024-03-24T10:41:10.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:10.407+0000 7fa7e0e52700 1 --2- 172.21.15.96:0/2441690491 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fa7cc069210 0x7fa7cc06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:41:10.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:10.408+0000 7fa7e0e52700 1 --2- 172.21.15.96:0/2441690491 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fa7cc069210 0x7fa7cc06b6c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0007fd0 tx=0x7fa7d0009040 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:41:10.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:10.412+0000 7fa7d97fa700 1 -- 172.21.15.96:0/2441690491 <== mon.1 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fa7c8057c40 con 0x7fa7dc0fcf80 2024-03-24T10:41:10.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:10 smithi094 ceph-mon[29446]: pgmap v71: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-24T10:41:10.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:10 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-03-24T10:41:10.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:10 smithi094 ceph-mon[29446]: osdmap e20: 3 total, 3 up, 3 in 2024-03-24T10:41:10.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:10 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-24T10:41:10.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:10 smithi096 ceph-mon[38097]: pgmap v71: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-24T10:41:10.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:10 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-03-24T10:41:10.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:10 smithi096 ceph-mon[38097]: osdmap e20: 3 total, 3 up, 3 in 2024-03-24T10:41:10.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:10 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-24T10:41:10.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:10.604+0000 7fa7e30b6700 1 -- 172.21.15.96:0/2441690491 --> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7dc101340 con 0x7fa7cc069210 2024-03-24T10:41:11.710 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:11 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-24T10:41:11.710 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:11 smithi096 ceph-mon[38097]: osdmap e21: 3 total, 3 up, 3 in 2024-03-24T10:41:11.710 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:11 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T10:41:11.710 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:11 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T10:41:11.710 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:11 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:41:11.719 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:11 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-24T10:41:11.719 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:11 smithi094 ceph-mon[29446]: osdmap e21: 3 total, 3 up, 3 in 2024-03-24T10:41:11.720 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:11 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T10:41:11.720 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:11 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T10:41:11.720 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:11 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:41:11.720 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:41:11 smithi094 sudo[66991]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-24T10:41:11.720 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:41:11 smithi094 sudo[66991]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-24T10:41:11.720 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:41:11 smithi094 sudo[66991]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-24T10:41:11.720 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:41:11 smithi094 sudo[66991]: pam_unix(sudo:session): session closed for user root 2024-03-24T10:41:11.995 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:41:11 smithi094 sudo[66997]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-24T10:41:11.995 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:41:11 smithi094 sudo[66997]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-24T10:41:11.995 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:41:11 smithi094 sudo[66997]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-24T10:41:11.995 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:41:11 smithi094 sudo[66997]: pam_unix(sudo:session): session closed for user root 2024-03-24T10:41:12.247 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:41:12 smithi094 sudo[67000]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-24T10:41:12.247 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:41:12 smithi094 sudo[67000]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-24T10:41:12.248 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:41:12 smithi094 sudo[67000]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-24T10:41:12.248 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:41:12 smithi094 sudo[67000]: pam_unix(sudo:session): session closed for user root 2024-03-24T10:41:12.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:12.352+0000 7fa7d97fa700 1 -- 172.21.15.96:0/2441690491 <== mon.1 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fa7c8054580 con 0x7fa7dc0fcf80 2024-03-24T10:41:12.431 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:12 smithi096 ceph-mon[38097]: from='client.24129 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:41:12.431 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:12 smithi096 ceph-mon[38097]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-03-24T10:41:12.431 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:12 smithi096 ceph-mon[38097]: osdmap e22: 3 total, 3 up, 3 in 2024-03-24T10:41:12.431 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:12 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:12.431 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:12 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:12.431 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:12 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:12.431 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:12 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T10:41:12.431 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:12 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T10:41:12.431 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:12 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T10:41:12.431 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:12 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:12.431 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:12 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:41:12.431 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:12 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:41:12.431 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:12 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:12.431 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:12 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:41:12.581 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:41:12 smithi094 sudo[67101]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-24T10:41:12.581 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:41:12 smithi094 sudo[67101]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-24T10:41:12.581 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:41:12 smithi094 sudo[67101]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-24T10:41:12.582 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:41:12 smithi094 sudo[67101]: pam_unix(sudo:session): session closed for user root 2024-03-24T10:41:12.582 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:12 smithi094 ceph-mon[29446]: from='client.24129 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:41:12.582 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:12 smithi094 ceph-mon[29446]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-03-24T10:41:12.582 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:12 smithi094 ceph-mon[29446]: osdmap e22: 3 total, 3 up, 3 in 2024-03-24T10:41:12.582 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:12 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:12.582 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:12 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:12.582 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:12 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:12.582 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:12 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T10:41:12.582 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:12 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T10:41:12.582 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:12 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T10:41:12.582 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:12 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:12.582 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:12 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:41:12.583 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:12 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:41:12.583 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:12 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:12.583 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:12 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:41:12.842 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:41:12 smithi094 sudo[67190]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-24T10:41:12.842 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:41:12 smithi094 sudo[67190]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-24T10:41:12.843 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:41:12 smithi094 sudo[67190]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-24T10:41:12.843 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:41:12 smithi094 sudo[67190]: pam_unix(sudo:session): session closed for user root 2024-03-24T10:41:13.092 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:41:12 smithi094 sudo[67199]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-24T10:41:13.092 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:41:12 smithi094 sudo[67199]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-24T10:41:13.092 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:41:12 smithi094 sudo[67199]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-24T10:41:13.093 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:41:12 smithi094 sudo[67199]: pam_unix(sudo:session): session closed for user root 2024-03-24T10:41:13.360 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:13 smithi094 sudo[67215]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-24T10:41:13.360 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:13 smithi094 sudo[67215]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-24T10:41:13.360 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:13 smithi094 sudo[67215]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-24T10:41:13.360 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:13 smithi094 sudo[67215]: pam_unix(sudo:session): session closed for user root 2024-03-24T10:41:13.667 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:13 smithi096 ceph-mon[38097]: Detected new or changed devices on smithi094 2024-03-24T10:41:13.667 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:13 smithi096 ceph-mon[38097]: Adjusting osd_memory_target on smithi094 to 4288M 2024-03-24T10:41:13.667 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:13 smithi096 ceph-mon[38097]: mgrmap e19: smithi094.yaethu(active, since 2m), standbys: smithi096.gfonnu 2024-03-24T10:41:13.733 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:13 smithi094 sudo[67300]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-24T10:41:13.733 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:13 smithi094 sudo[67300]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-24T10:41:13.733 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:13 smithi094 sudo[67300]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-24T10:41:13.733 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:13 smithi094 sudo[67300]: pam_unix(sudo:session): session closed for user root 2024-03-24T10:41:13.733 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:13 smithi094 ceph-mon[29446]: Detected new or changed devices on smithi094 2024-03-24T10:41:13.733 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:13 smithi094 ceph-mon[29446]: Adjusting osd_memory_target on smithi094 to 4288M 2024-03-24T10:41:13.733 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:13 smithi094 ceph-mon[29446]: mgrmap e19: smithi094.yaethu(active, since 2m), standbys: smithi096.gfonnu 2024-03-24T10:41:13.924 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:13 smithi096 sudo[41587]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-24T10:41:13.924 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:13 smithi096 sudo[41587]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-24T10:41:13.924 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:13 smithi096 sudo[41587]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-24T10:41:13.924 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:13 smithi096 sudo[41587]: pam_unix(sudo:session): session closed for user root 2024-03-24T10:41:14.193 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:13 smithi096 sudo[41615]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-24T10:41:14.193 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:13 smithi096 sudo[41615]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-24T10:41:14.193 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:13 smithi096 sudo[41615]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-24T10:41:14.194 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:13 smithi096 sudo[41615]: pam_unix(sudo:session): session closed for user root 2024-03-24T10:41:14.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:14 smithi096 ceph-mon[38097]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-24T10:41:14.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:14 smithi096 ceph-mon[38097]: pgmap v76: 1 pgs: 1 active+clean; 449 KiB data, 871 MiB used, 267 GiB / 268 GiB avail 2024-03-24T10:41:14.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:14 smithi096 ceph-mon[38097]: from='client.? 172.21.15.96:0/1564834659' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "910859cd-d028-45f8-8e28-af4e46be460b"}]: dispatch 2024-03-24T10:41:14.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:14 smithi096 ceph-mon[38097]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-24T10:41:14.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:14 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata", "id": "smithi094"}]: dispatch 2024-03-24T10:41:14.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:14 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata", "id": "smithi096"}]: dispatch 2024-03-24T10:41:14.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:14 smithi096 ceph-mon[38097]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "910859cd-d028-45f8-8e28-af4e46be460b"}]: dispatch 2024-03-24T10:41:14.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:14 smithi096 ceph-mon[38097]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "910859cd-d028-45f8-8e28-af4e46be460b"}]': finished 2024-03-24T10:41:14.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:14 smithi096 ceph-mon[38097]: osdmap e23: 4 total, 3 up, 4 in 2024-03-24T10:41:14.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:14 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T10:41:14.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:14 smithi096 ceph-mon[38097]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-24T10:41:14.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:14 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata", "id": "smithi094"}]: dispatch 2024-03-24T10:41:14.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:14 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata", "id": "smithi096"}]: dispatch 2024-03-24T10:41:14.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:14 smithi096 ceph-mon[38097]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-24T10:41:14.577 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:14 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:14.577 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:14 smithi096 ceph-mon[38097]: from='client.? 172.21.15.96:0/1723400215' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T10:41:14.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:14 smithi094 ceph-mon[29446]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-24T10:41:14.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:14 smithi094 ceph-mon[29446]: pgmap v76: 1 pgs: 1 active+clean; 449 KiB data, 871 MiB used, 267 GiB / 268 GiB avail 2024-03-24T10:41:14.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:14 smithi094 ceph-mon[29446]: from='client.? 172.21.15.96:0/1564834659' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "910859cd-d028-45f8-8e28-af4e46be460b"}]: dispatch 2024-03-24T10:41:14.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:14 smithi094 ceph-mon[29446]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-24T10:41:14.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:14 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata", "id": "smithi094"}]: dispatch 2024-03-24T10:41:14.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:14 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata", "id": "smithi096"}]: dispatch 2024-03-24T10:41:14.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:14 smithi094 ceph-mon[29446]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "910859cd-d028-45f8-8e28-af4e46be460b"}]: dispatch 2024-03-24T10:41:14.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:14 smithi094 ceph-mon[29446]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "910859cd-d028-45f8-8e28-af4e46be460b"}]': finished 2024-03-24T10:41:14.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:14 smithi094 ceph-mon[29446]: osdmap e23: 4 total, 3 up, 4 in 2024-03-24T10:41:14.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:14 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T10:41:14.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:14 smithi094 ceph-mon[29446]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-24T10:41:14.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:14 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata", "id": "smithi094"}]: dispatch 2024-03-24T10:41:14.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:14 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata", "id": "smithi096"}]: dispatch 2024-03-24T10:41:14.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:14 smithi094 ceph-mon[29446]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-24T10:41:14.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:14 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:14.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:14 smithi094 ceph-mon[29446]: from='client.? 172.21.15.96:0/1723400215' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T10:41:15.688 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:15 smithi094 ceph-mon[29446]: pgmap v78: 1 pgs: 1 active+clean; 449 KiB data, 872 MiB used, 267 GiB / 268 GiB avail 2024-03-24T10:41:15.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:15 smithi096 ceph-mon[38097]: pgmap v78: 1 pgs: 1 active+clean; 449 KiB data, 872 MiB used, 267 GiB / 268 GiB avail 2024-03-24T10:41:16.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:16 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:41:16.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:16 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:16.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:16 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:16.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:16 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:41:16.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:16 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:16.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:16 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:17.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:17 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:41:17.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:17 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:41:17.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:17 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:17.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:17 smithi094 ceph-mon[29446]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 4.9 KiB/s rd, 237 KiB/s wr, 16 op/s 2024-03-24T10:41:17.760 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:17 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:41:17.760 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:17 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:41:17.760 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:17 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:17.760 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:17 smithi096 ceph-mon[38097]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 4.9 KiB/s rd, 237 KiB/s wr, 16 op/s 2024-03-24T10:41:20.068 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:19 smithi096 ceph-mon[38097]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.6 KiB/s rd, 174 KiB/s wr, 11 op/s 2024-03-24T10:41:20.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:19 smithi094 ceph-mon[29446]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.6 KiB/s rd, 174 KiB/s wr, 11 op/s 2024-03-24T10:41:21.206 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:20 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-24T10:41:21.207 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:20 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:41:21.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:20 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-24T10:41:21.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:20 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:41:22.192 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:21 smithi096 ceph-mon[38097]: Deploying daemon osd.3 on smithi096 2024-03-24T10:41:22.192 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:21 smithi096 ceph-mon[38097]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.8 KiB/s rd, 137 KiB/s wr, 9 op/s 2024-03-24T10:41:22.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:21 smithi094 ceph-mon[29446]: Deploying daemon osd.3 on smithi096 2024-03-24T10:41:22.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:21 smithi094 ceph-mon[29446]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.8 KiB/s rd, 137 KiB/s wr, 9 op/s 2024-03-24T10:41:24.012 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:23 smithi096 ceph-mon[38097]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-03-24T10:41:24.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:23 smithi094 ceph-mon[29446]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-03-24T10:41:26.173 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:25 smithi096 ceph-mon[38097]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.4 KiB/s rd, 114 KiB/s wr, 7 op/s 2024-03-24T10:41:26.174 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:25 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:26.174 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:25 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:26.174 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:25 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:41:26.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:25 smithi094 ceph-mon[29446]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.4 KiB/s rd, 114 KiB/s wr, 7 op/s 2024-03-24T10:41:26.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:25 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:26.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:25 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:26.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:25 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:41:28.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:27 smithi094 ceph-mon[29446]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-03-24T10:41:28.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:27 smithi096 ceph-mon[38097]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-03-24T10:41:29.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:29.575+0000 7fa7d97fa700 1 -- 172.21.15.96:0/2441690491 <== mgr.14232 v2:172.21.15.94:6800/2184357820 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa7dc101340 con 0x7fa7cc069210 2024-03-24T10:41:29.576 INFO:teuthology.orchestra.run.smithi096.stdout:Created osd(s) 3 on host 'smithi096' 2024-03-24T10:41:29.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:29.577+0000 7fa7e30b6700 1 -- 172.21.15.96:0/2441690491 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fa7cc069210 msgr2=0x7fa7cc06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:41:29.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:29.577+0000 7fa7e30b6700 1 --2- 172.21.15.96:0/2441690491 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fa7cc069210 0x7fa7cc06b6c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0007fd0 tx=0x7fa7d0009040 comp rx=0 tx=0).stop 2024-03-24T10:41:29.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:29.577+0000 7fa7e30b6700 1 -- 172.21.15.96:0/2441690491 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7dc0fcf80 msgr2=0x7fa7dc112740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:41:29.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:29.577+0000 7fa7e30b6700 1 --2- 172.21.15.96:0/2441690491 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7dc0fcf80 0x7fa7dc112740 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8009150 tx=0x7fa7c8009e30 comp rx=0 tx=0).stop 2024-03-24T10:41:29.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:29.577+0000 7fa7e30b6700 1 -- 172.21.15.96:0/2441690491 shutdown_connections 2024-03-24T10:41:29.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:29.577+0000 7fa7e30b6700 1 --2- 172.21.15.96:0/2441690491 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fa7cc069210 0x7fa7cc06b6c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:41:29.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:29.577+0000 7fa7e30b6700 1 --2- 172.21.15.96:0/2441690491 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7dc0fbcd0 0x7fa7dc066cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:41:29.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:29.577+0000 7fa7e30b6700 1 --2- 172.21.15.96:0/2441690491 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7dc0fcf80 0x7fa7dc112740 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:41:29.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:29.577+0000 7fa7e30b6700 1 -- 172.21.15.96:0/2441690491 >> 172.21.15.96:0/2441690491 conn(0x7fa7dc0f7450 msgr2=0x7fa7dc0f8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:41:29.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:29.577+0000 7fa7e30b6700 1 -- 172.21.15.96:0/2441690491 shutdown_connections 2024-03-24T10:41:29.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:29.577+0000 7fa7e30b6700 1 -- 172.21.15.96:0/2441690491 wait complete. 2024-03-24T10:41:29.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:29 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:29.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:29 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:29.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:29 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:41:29.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:29 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:41:29.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:29 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:29.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:29 smithi096 ceph-mon[38097]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-24T10:41:29.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:29 smithi096 ceph-mon[38097]: from='osd.3 [v2:172.21.15.96:6800/3684617676,v1:172.21.15.96:6801/3684617676]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-24T10:41:29.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:29 smithi096 ceph-mon[38097]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-24T10:41:29.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:29 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:29.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:29 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:41:29.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:29 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:29.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:29 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:29.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:29 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:29.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:29 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:41:29.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:29 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:41:29.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:29 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:29.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:29 smithi094 ceph-mon[29446]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-24T10:41:29.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:29 smithi094 ceph-mon[29446]: from='osd.3 [v2:172.21.15.96:6800/3684617676,v1:172.21.15.96:6801/3684617676]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-24T10:41:29.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:29 smithi094 ceph-mon[29446]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-24T10:41:29.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:29 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:29.999 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:29 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:41:29.999 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:29 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:30.231 DEBUG:teuthology.orchestra.run.smithi096:osd.3> sudo journalctl -f -n 0 -u ceph-386110d4-e9ca-11ee-95cd-87774f69a715@osd.3.service 2024-03-24T10:41:30.234 INFO:tasks.cephadm:Deploying osd.4 on smithi096 with /dev/vg_nvme/lv_3... 2024-03-24T10:41:30.234 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-24T10:41:30.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:30 smithi096 ceph-mon[38097]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-24T10:41:30.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:30 smithi096 ceph-mon[38097]: osdmap e24: 4 total, 3 up, 4 in 2024-03-24T10:41:30.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:30 smithi096 ceph-mon[38097]: from='osd.3 [v2:172.21.15.96:6800/3684617676,v1:172.21.15.96:6801/3684617676]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-03-24T10:41:30.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:30 smithi096 ceph-mon[38097]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-03-24T10:41:30.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:30 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T10:41:30.826 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:41:30 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3[43669]: 2024-03-24T10:41:30.731+0000 7ff8d7071700 -1 osd.3 0 waiting for initial osdmap 2024-03-24T10:41:30.826 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:41:30 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3[43669]: 2024-03-24T10:41:30.736+0000 7ff8d266d700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T10:41:30.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:30 smithi094 ceph-mon[29446]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-24T10:41:30.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:30 smithi094 ceph-mon[29446]: osdmap e24: 4 total, 3 up, 4 in 2024-03-24T10:41:30.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:30 smithi094 ceph-mon[29446]: from='osd.3 [v2:172.21.15.96:6800/3684617676,v1:172.21.15.96:6801/3684617676]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-03-24T10:41:30.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:30 smithi094 ceph-mon[29446]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-03-24T10:41:30.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:30 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T10:41:31.319 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi096/config 2024-03-24T10:41:31.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:31 smithi094 ceph-mon[29446]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-03-24T10:41:31.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:31 smithi094 ceph-mon[29446]: osdmap e25: 4 total, 3 up, 4 in 2024-03-24T10:41:31.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:31 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T10:41:31.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:31 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T10:41:31.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:31 smithi094 ceph-mon[29446]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-24T10:41:31.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:31 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:41:31.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:31 smithi094 ceph-mon[29446]: osd.3 [v2:172.21.15.96:6800/3684617676,v1:172.21.15.96:6801/3684617676] boot 2024-03-24T10:41:31.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:31 smithi094 ceph-mon[29446]: osdmap e26: 4 total, 4 up, 4 in 2024-03-24T10:41:31.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:31 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T10:41:32.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:31 smithi096 ceph-mon[38097]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-03-24T10:41:32.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:31 smithi096 ceph-mon[38097]: osdmap e25: 4 total, 3 up, 4 in 2024-03-24T10:41:32.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:31 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T10:41:32.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:31 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T10:41:32.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:31 smithi096 ceph-mon[38097]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-24T10:41:32.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:31 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:41:32.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:31 smithi096 ceph-mon[38097]: osd.3 [v2:172.21.15.96:6800/3684617676,v1:172.21.15.96:6801/3684617676] boot 2024-03-24T10:41:32.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:31 smithi096 ceph-mon[38097]: osdmap e26: 4 total, 4 up, 4 in 2024-03-24T10:41:32.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:31 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T10:41:32.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:32 smithi094 ceph-mon[29446]: purged_snaps scrub starts 2024-03-24T10:41:32.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:32 smithi094 ceph-mon[29446]: purged_snaps scrub ok 2024-03-24T10:41:32.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:32 smithi094 ceph-mon[29446]: osdmap e27: 4 total, 4 up, 4 in 2024-03-24T10:41:33.002 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:32 smithi096 ceph-mon[38097]: purged_snaps scrub starts 2024-03-24T10:41:33.003 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:32 smithi096 ceph-mon[38097]: purged_snaps scrub ok 2024-03-24T10:41:33.003 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:32 smithi096 ceph-mon[38097]: osdmap e27: 4 total, 4 up, 4 in 2024-03-24T10:41:33.579 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-24T10:41:33.579 INFO:teuthology.orchestra.run.smithi096.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-24T10:41:33.579 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10+0 records in 2024-03-24T10:41:33.579 INFO:teuthology.orchestra.run.smithi096.stderr:10+0 records out 2024-03-24T10:41:33.579 INFO:teuthology.orchestra.run.smithi096.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0124038 s, 845 MB/s 2024-03-24T10:41:33.579 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping successful for: 2024-03-24T10:41:33.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:33 smithi094 ceph-mon[29446]: pgmap v91: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-24T10:41:33.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:33 smithi094 ceph-mon[29446]: osdmap e28: 4 total, 4 up, 4 in 2024-03-24T10:41:34.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:33 smithi096 ceph-mon[38097]: pgmap v91: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-24T10:41:34.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:33 smithi096 ceph-mon[38097]: osdmap e28: 4 total, 4 up, 4 in 2024-03-24T10:41:34.909 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi096:vg_nvme/lv_3 2024-03-24T10:41:35.158 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi096/config 2024-03-24T10:41:35.537 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:35 smithi096 ceph-mon[38097]: Detected new or changed devices on smithi096 2024-03-24T10:41:35.537 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:35 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:35.537 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:35 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:35.538 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:35 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T10:41:35.538 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:35 smithi096 ceph-mon[38097]: Adjusting osd_memory_target on smithi096 to 15936M 2024-03-24T10:41:35.538 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:35 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:35.538 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:35 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:41:35.538 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:35 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:41:35.538 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:35 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:35.538 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:35 smithi096 ceph-mon[38097]: osdmap e29: 4 total, 4 up, 4 in 2024-03-24T10:41:35.538 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:35 smithi096 ceph-mon[38097]: pgmap v94: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-24T10:41:35.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:35 smithi094 ceph-mon[29446]: Detected new or changed devices on smithi096 2024-03-24T10:41:35.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:35 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:35.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:35 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:35.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:35 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T10:41:35.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:35 smithi094 ceph-mon[29446]: Adjusting osd_memory_target on smithi096 to 15936M 2024-03-24T10:41:35.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:35 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:35.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:35 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:41:35.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:35 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:41:35.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:35 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:35.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:35 smithi094 ceph-mon[29446]: osdmap e29: 4 total, 4 up, 4 in 2024-03-24T10:41:35.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:35 smithi094 ceph-mon[29446]: pgmap v94: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-24T10:41:36.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:36.772+0000 7fd839227700 1 -- 172.21.15.96:0/3577486309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd8340fce70 msgr2=0x7fd8340fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:41:36.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:36.772+0000 7fd839227700 1 --2- 172.21.15.96:0/3577486309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd8340fce70 0x7fd8340fd280 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd81c004940 tx=0x7fd81c009e30 comp rx=0 tx=0).stop 2024-03-24T10:41:36.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:36.772+0000 7fd839227700 1 -- 172.21.15.96:0/3577486309 shutdown_connections 2024-03-24T10:41:36.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:36.772+0000 7fd839227700 1 --2- 172.21.15.96:0/3577486309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8340fdfe0 0x7fd8340fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:41:36.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:36.772+0000 7fd839227700 1 --2- 172.21.15.96:0/3577486309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd8340fce70 0x7fd8340fd280 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:41:36.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:36.772+0000 7fd839227700 1 -- 172.21.15.96:0/3577486309 >> 172.21.15.96:0/3577486309 conn(0x7fd8340f8550 msgr2=0x7fd8340fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:41:36.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:36.772+0000 7fd839227700 1 -- 172.21.15.96:0/3577486309 shutdown_connections 2024-03-24T10:41:36.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:36.772+0000 7fd839227700 1 -- 172.21.15.96:0/3577486309 wait complete. 2024-03-24T10:41:36.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:36.773+0000 7fd839227700 1 Processor -- start 2024-03-24T10:41:36.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:36.773+0000 7fd839227700 1 -- start start 2024-03-24T10:41:36.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:36.773+0000 7fd839227700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd8340fce70 0x7fd83410d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:41:36.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:36.773+0000 7fd839227700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8340fdfe0 0x7fd83410de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:41:36.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:36.773+0000 7fd839227700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83410e510 con 0x7fd8340fdfe0 2024-03-24T10:41:36.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:36.773+0000 7fd839227700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd834112eb0 con 0x7fd8340fce70 2024-03-24T10:41:36.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:36.773+0000 7fd832d9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd8340fce70 0x7fd83410d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:41:36.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:36.773+0000 7fd832d9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd8340fce70 0x7fd83410d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34626/0 (socket says 172.21.15.96:34626) 2024-03-24T10:41:36.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:36.773+0000 7fd832d9d700 1 -- 172.21.15.96:0/1774219576 learned_addr learned my addr 172.21.15.96:0/1774219576 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-24T10:41:36.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:36.773+0000 7fd83259c700 1 --2- 172.21.15.96:0/1774219576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8340fdfe0 0x7fd83410de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:41:36.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:36.773+0000 7fd832d9d700 1 -- 172.21.15.96:0/1774219576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8340fdfe0 msgr2=0x7fd83410de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:41:36.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:36.773+0000 7fd832d9d700 1 --2- 172.21.15.96:0/1774219576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8340fdfe0 0x7fd83410de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:41:36.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:36.773+0000 7fd832d9d700 1 -- 172.21.15.96:0/1774219576 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd81c005040 con 0x7fd8340fce70 2024-03-24T10:41:36.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:36.773+0000 7fd83259c700 1 --2- 172.21.15.96:0/1774219576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8340fdfe0 0x7fd83410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T10:41:36.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:36.773+0000 7fd832d9d700 1 --2- 172.21.15.96:0/1774219576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd8340fce70 0x7fd83410d940 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd81c00b3b0 tx=0x7fd81c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:41:36.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:36.773+0000 7fd82b7fe700 1 -- 172.21.15.96:0/1774219576 <== mon.1 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd81c00e040 con 0x7fd8340fce70 2024-03-24T10:41:36.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:36.774+0000 7fd82b7fe700 1 -- 172.21.15.96:0/1774219576 <== mon.1 v2:172.21.15.96:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd81c00bcc0 con 0x7fd8340fce70 2024-03-24T10:41:36.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:36.774+0000 7fd839227700 1 -- 172.21.15.96:0/1774219576 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8341130b0 con 0x7fd8340fce70 2024-03-24T10:41:36.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:36.774+0000 7fd82b7fe700 1 -- 172.21.15.96:0/1774219576 <== mon.1 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd81c01c630 con 0x7fd8340fce70 2024-03-24T10:41:36.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:36.774+0000 7fd839227700 1 -- 172.21.15.96:0/1774219576 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd834113620 con 0x7fd8340fce70 2024-03-24T10:41:36.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:36.774+0000 7fd839227700 1 -- 172.21.15.96:0/1774219576 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd83404c710 con 0x7fd8340fce70 2024-03-24T10:41:36.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:36.775+0000 7fd82b7fe700 1 -- 172.21.15.96:0/1774219576 <== mon.1 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fd81c013980 con 0x7fd8340fce70 2024-03-24T10:41:36.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:36.775+0000 7fd82b7fe700 1 --2- 172.21.15.96:0/1774219576 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fd8200692a0 0x7fd82006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:41:36.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:36.775+0000 7fd82b7fe700 1 -- 172.21.15.96:0/1774219576 <== mon.1 v2:172.21.15.96:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3770+0+0 (secure 0 0 0) 0x7fd81c087910 con 0x7fd8340fce70 2024-03-24T10:41:36.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:36.775+0000 7fd83259c700 1 --2- 172.21.15.96:0/1774219576 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fd8200692a0 0x7fd82006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:41:36.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:36.776+0000 7fd83259c700 1 --2- 172.21.15.96:0/1774219576 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fd8200692a0 0x7fd82006b750 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd824006fd0 tx=0x7fd824008040 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:41:36.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:36.780+0000 7fd82b7fe700 1 -- 172.21.15.96:0/1774219576 <== mon.1 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fd81c057cf0 con 0x7fd8340fce70 2024-03-24T10:41:36.798 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:36 smithi096 ceph-mon[38097]: osdmap e30: 4 total, 4 up, 4 in 2024-03-24T10:41:36.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:36.972+0000 7fd839227700 1 -- 172.21.15.96:0/1774219576 --> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fd834102280 con 0x7fd8200692a0 2024-03-24T10:41:36.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:36 smithi094 ceph-mon[29446]: osdmap e30: 4 total, 4 up, 4 in 2024-03-24T10:41:37.696 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:37 smithi096 ceph-mon[38097]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-03-24T10:41:37.696 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:37 smithi096 ceph-mon[38097]: from='client.24153 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:41:37.696 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:37 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T10:41:37.696 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:37 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T10:41:37.696 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:37 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:41:37.930 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:37 smithi094 ceph-mon[29446]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-03-24T10:41:37.930 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:37 smithi094 ceph-mon[29446]: from='client.24153 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:41:37.930 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:37 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T10:41:37.930 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:37 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T10:41:37.930 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:37 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:41:40.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:39 smithi094 ceph-mon[29446]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-24T10:41:40.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:39 smithi094 ceph-mon[29446]: from='client.? 172.21.15.96:0/1551781444' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9d37e21d-38dc-4e5b-b5ef-3828d7b25096"}]: dispatch 2024-03-24T10:41:40.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:39 smithi094 ceph-mon[29446]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9d37e21d-38dc-4e5b-b5ef-3828d7b25096"}]: dispatch 2024-03-24T10:41:40.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:39 smithi094 ceph-mon[29446]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9d37e21d-38dc-4e5b-b5ef-3828d7b25096"}]': finished 2024-03-24T10:41:40.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:39 smithi094 ceph-mon[29446]: osdmap e31: 5 total, 4 up, 5 in 2024-03-24T10:41:40.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:39 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T10:41:40.266 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:39 smithi096 ceph-mon[38097]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-24T10:41:40.266 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:39 smithi096 ceph-mon[38097]: from='client.? 172.21.15.96:0/1551781444' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9d37e21d-38dc-4e5b-b5ef-3828d7b25096"}]: dispatch 2024-03-24T10:41:40.266 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:39 smithi096 ceph-mon[38097]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9d37e21d-38dc-4e5b-b5ef-3828d7b25096"}]: dispatch 2024-03-24T10:41:40.266 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:39 smithi096 ceph-mon[38097]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9d37e21d-38dc-4e5b-b5ef-3828d7b25096"}]': finished 2024-03-24T10:41:40.266 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:39 smithi096 ceph-mon[38097]: osdmap e31: 5 total, 4 up, 5 in 2024-03-24T10:41:40.267 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:39 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T10:41:41.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:40 smithi094 ceph-mon[29446]: from='client.? 172.21.15.96:0/281058211' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T10:41:41.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:40 smithi096 ceph-mon[38097]: from='client.? 172.21.15.96:0/281058211' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T10:41:42.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:41 smithi094 ceph-mon[29446]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-03-24T10:41:42.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:41 smithi096 ceph-mon[38097]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-03-24T10:41:44.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:43 smithi096 ceph-mon[38097]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-24T10:41:44.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:43 smithi094 ceph-mon[29446]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-24T10:41:46.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:45 smithi096 ceph-mon[38097]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-03-24T10:41:46.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:45 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:41:46.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:45 smithi094 ceph-mon[29446]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-03-24T10:41:46.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:45 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:41:47.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:46 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-24T10:41:47.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:46 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:41:47.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:46 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-24T10:41:47.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:46 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:41:48.223 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:47 smithi096 ceph-mon[38097]: Deploying daemon osd.4 on smithi096 2024-03-24T10:41:48.223 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:47 smithi096 ceph-mon[38097]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-24T10:41:48.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:47 smithi094 ceph-mon[29446]: Deploying daemon osd.4 on smithi096 2024-03-24T10:41:48.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:47 smithi094 ceph-mon[29446]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-24T10:41:50.227 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:49 smithi096 ceph-mon[38097]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-24T10:41:50.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:49 smithi094 ceph-mon[29446]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-24T10:41:52.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:51 smithi094 ceph-mon[29446]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-24T10:41:52.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:51 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:52.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:51 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:52.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:51 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:41:52.257 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:51 smithi096 ceph-mon[38097]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-24T10:41:52.257 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:51 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:52.257 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:51 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:52.257 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:51 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:41:54.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:53 smithi094 ceph-mon[29446]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-24T10:41:54.306 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:53 smithi096 ceph-mon[38097]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-24T10:41:54.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:54.934+0000 7fd82b7fe700 1 -- 172.21.15.96:0/1774219576 <== mgr.14232 v2:172.21.15.94:6800/2184357820 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd834102280 con 0x7fd8200692a0 2024-03-24T10:41:54.935 INFO:teuthology.orchestra.run.smithi096.stdout:Created osd(s) 4 on host 'smithi096' 2024-03-24T10:41:54.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:54.937+0000 7fd839227700 1 -- 172.21.15.96:0/1774219576 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fd8200692a0 msgr2=0x7fd82006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:41:54.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:54.937+0000 7fd839227700 1 --2- 172.21.15.96:0/1774219576 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fd8200692a0 0x7fd82006b750 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd824006fd0 tx=0x7fd824008040 comp rx=0 tx=0).stop 2024-03-24T10:41:54.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:54.937+0000 7fd839227700 1 -- 172.21.15.96:0/1774219576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd8340fce70 msgr2=0x7fd83410d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:41:54.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:54.937+0000 7fd839227700 1 --2- 172.21.15.96:0/1774219576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd8340fce70 0x7fd83410d940 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd81c00b3b0 tx=0x7fd81c009e30 comp rx=0 tx=0).stop 2024-03-24T10:41:54.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:54.937+0000 7fd839227700 1 -- 172.21.15.96:0/1774219576 shutdown_connections 2024-03-24T10:41:54.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:54.937+0000 7fd839227700 1 --2- 172.21.15.96:0/1774219576 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fd8200692a0 0x7fd82006b750 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:41:54.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:54.937+0000 7fd839227700 1 --2- 172.21.15.96:0/1774219576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd8340fce70 0x7fd83410d940 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:41:54.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:54.937+0000 7fd839227700 1 --2- 172.21.15.96:0/1774219576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8340fdfe0 0x7fd83410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:41:54.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:54.937+0000 7fd839227700 1 -- 172.21.15.96:0/1774219576 >> 172.21.15.96:0/1774219576 conn(0x7fd8340f8550 msgr2=0x7fd8340f9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:41:54.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:54.937+0000 7fd839227700 1 -- 172.21.15.96:0/1774219576 shutdown_connections 2024-03-24T10:41:54.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:41:54.937+0000 7fd839227700 1 -- 172.21.15.96:0/1774219576 wait complete. 2024-03-24T10:41:55.593 DEBUG:teuthology.orchestra.run.smithi096:osd.4> sudo journalctl -f -n 0 -u ceph-386110d4-e9ca-11ee-95cd-87774f69a715@osd.4.service 2024-03-24T10:41:55.595 INFO:tasks.cephadm:Deploying osd.5 on smithi096 with /dev/vg_nvme/lv_2... 2024-03-24T10:41:55.595 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-24T10:41:55.825 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:41:55 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4[50398]: 2024-03-24T10:41:55.751+0000 7f519c49f540 -1 osd.4 0 log_to_monitors true 2024-03-24T10:41:56.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:55 smithi094 ceph-mon[29446]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-24T10:41:56.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:55 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:56.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:55 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:56.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:55 smithi094 ceph-mon[29446]: from='osd.4 [v2:172.21.15.96:6808/135461738,v1:172.21.15.96:6809/135461738]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-24T10:41:56.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:55 smithi094 ceph-mon[29446]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-24T10:41:56.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:55 smithi096 ceph-mon[38097]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-24T10:41:56.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:55 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:56.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:55 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:56.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:55 smithi096 ceph-mon[38097]: from='osd.4 [v2:172.21.15.96:6808/135461738,v1:172.21.15.96:6809/135461738]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-24T10:41:56.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:55 smithi096 ceph-mon[38097]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-24T10:41:56.370 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi096/config 2024-03-24T10:41:57.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:56 smithi096 ceph-mon[38097]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-24T10:41:57.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:56 smithi096 ceph-mon[38097]: osdmap e32: 5 total, 4 up, 5 in 2024-03-24T10:41:57.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:56 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T10:41:57.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:56 smithi096 ceph-mon[38097]: from='osd.4 [v2:172.21.15.96:6808/135461738,v1:172.21.15.96:6809/135461738]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-03-24T10:41:57.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:56 smithi096 ceph-mon[38097]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-03-24T10:41:57.076 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:41:56 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4[50398]: 2024-03-24T10:41:56.945+0000 7f5192c71700 -1 osd.4 0 waiting for initial osdmap 2024-03-24T10:41:57.076 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:41:56 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4[50398]: 2024-03-24T10:41:56.952+0000 7f518e26d700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T10:41:57.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:56 smithi094 ceph-mon[29446]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-24T10:41:57.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:56 smithi094 ceph-mon[29446]: osdmap e32: 5 total, 4 up, 5 in 2024-03-24T10:41:57.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:56 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T10:41:57.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:56 smithi094 ceph-mon[29446]: from='osd.4 [v2:172.21.15.96:6808/135461738,v1:172.21.15.96:6809/135461738]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-03-24T10:41:57.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:56 smithi094 ceph-mon[29446]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-03-24T10:41:58.175 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:57 smithi096 ceph-mon[38097]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-24T10:41:58.175 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:57 smithi096 ceph-mon[38097]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-03-24T10:41:58.175 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:57 smithi096 ceph-mon[38097]: osdmap e33: 5 total, 4 up, 5 in 2024-03-24T10:41:58.175 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:57 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T10:41:58.175 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:57 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T10:41:58.175 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:57 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:58.176 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:57 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:58.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:57 smithi094 ceph-mon[29446]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-24T10:41:58.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:57 smithi094 ceph-mon[29446]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-03-24T10:41:58.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:57 smithi094 ceph-mon[29446]: osdmap e33: 5 total, 4 up, 5 in 2024-03-24T10:41:58.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:57 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T10:41:58.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:57 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T10:41:58.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:57 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:58.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:57 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:41:58.920 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-24T10:41:58.921 INFO:teuthology.orchestra.run.smithi096.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-24T10:41:58.921 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10+0 records in 2024-03-24T10:41:58.921 INFO:teuthology.orchestra.run.smithi096.stderr:10+0 records out 2024-03-24T10:41:58.921 INFO:teuthology.orchestra.run.smithi096.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0123575 s, 849 MB/s 2024-03-24T10:41:58.921 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 2024-03-24T10:41:58.921 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping successful for: 2024-03-24T10:41:59.246 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:58 smithi096 ceph-mon[38097]: purged_snaps scrub starts 2024-03-24T10:41:59.246 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:58 smithi096 ceph-mon[38097]: purged_snaps scrub ok 2024-03-24T10:41:59.246 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:58 smithi096 ceph-mon[38097]: osd.4 [v2:172.21.15.96:6808/135461738,v1:172.21.15.96:6809/135461738] boot 2024-03-24T10:41:59.247 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:58 smithi096 ceph-mon[38097]: osdmap e34: 5 total, 5 up, 5 in 2024-03-24T10:41:59.247 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:58 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T10:41:59.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:58 smithi094 ceph-mon[29446]: purged_snaps scrub starts 2024-03-24T10:41:59.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:58 smithi094 ceph-mon[29446]: purged_snaps scrub ok 2024-03-24T10:41:59.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:58 smithi094 ceph-mon[29446]: osd.4 [v2:172.21.15.96:6808/135461738,v1:172.21.15.96:6809/135461738] boot 2024-03-24T10:41:59.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:58 smithi094 ceph-mon[29446]: osdmap e34: 5 total, 5 up, 5 in 2024-03-24T10:41:59.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:58 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T10:41:59.607 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi096:vg_nvme/lv_2 2024-03-24T10:42:00.104 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi096/config 2024-03-24T10:42:00.227 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:59 smithi096 ceph-mon[38097]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-24T10:42:00.227 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:41:59 smithi096 ceph-mon[38097]: osdmap e35: 5 total, 5 up, 5 in 2024-03-24T10:42:00.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:59 smithi094 ceph-mon[29446]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-24T10:42:00.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:41:59 smithi094 ceph-mon[29446]: osdmap e35: 5 total, 5 up, 5 in 2024-03-24T10:42:01.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:00 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:42:01.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:00 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:42:02.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:01 smithi094 ceph-mon[29446]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-24T10:42:02.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:01 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:42:02.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:01 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:42:02.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:01 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T10:42:02.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:01 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T10:42:02.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:01 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:42:02.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:01 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:42:02.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:01 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:42:02.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:01 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:42:02.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:01 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:42:02.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:01 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:42:02.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:01 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:42:02.249 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:01 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:42:02.283 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:01 smithi096 ceph-mon[38097]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-24T10:42:02.283 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:01 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:42:02.283 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:01 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:42:02.283 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:01 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T10:42:02.283 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:01 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T10:42:02.284 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:01 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:42:02.284 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:01 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:42:02.284 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:01 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:42:02.284 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:01 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:42:02.284 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:01 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:42:02.284 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:01 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:42:02.284 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:01 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:42:02.284 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:01 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:42:02.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:02.381+0000 7f0285cb4700 1 -- 172.21.15.96:0/2748789264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02800fcf80 msgr2=0x7f02800fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:02.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:02.381+0000 7f0285cb4700 1 --2- 172.21.15.96:0/2748789264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02800fcf80 0x7f02800fd3f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0274004940 tx=0x7f0274009e30 comp rx=0 tx=0).stop 2024-03-24T10:42:02.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:02.381+0000 7f0285cb4700 1 -- 172.21.15.96:0/2748789264 shutdown_connections 2024-03-24T10:42:02.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:02.381+0000 7f0285cb4700 1 --2- 172.21.15.96:0/2748789264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02800fcf80 0x7f02800fd3f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:02.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:02.381+0000 7f0285cb4700 1 --2- 172.21.15.96:0/2748789264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02800fbcd0 0x7f02800fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:02.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:02.381+0000 7f0285cb4700 1 -- 172.21.15.96:0/2748789264 >> 172.21.15.96:0/2748789264 conn(0x7f02800f7430 msgr2=0x7f02800f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:42:02.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:02.381+0000 7f0285cb4700 1 -- 172.21.15.96:0/2748789264 shutdown_connections 2024-03-24T10:42:02.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:02.381+0000 7f0285cb4700 1 -- 172.21.15.96:0/2748789264 wait complete. 2024-03-24T10:42:02.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:02.381+0000 7f0285cb4700 1 Processor -- start 2024-03-24T10:42:02.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:02.382+0000 7f0285cb4700 1 -- start start 2024-03-24T10:42:02.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:02.382+0000 7f0285cb4700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02800fbcd0 0x7f02801106d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:02.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:02.382+0000 7f0285cb4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02800fcf80 0x7f0280112c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:02.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:02.382+0000 7f0285cb4700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0280066440 con 0x7f02800fcf80 2024-03-24T10:42:02.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:02.382+0000 7f0285cb4700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0280066580 con 0x7f02800fbcd0 2024-03-24T10:42:02.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:02.382+0000 7f027f7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02800fbcd0 0x7f02801106d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:02.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:02.382+0000 7f027f7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02800fbcd0 0x7f02801106d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44716/0 (socket says 172.21.15.96:44716) 2024-03-24T10:42:02.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:02.382+0000 7f027f7fe700 1 -- 172.21.15.96:0/2585897346 learned_addr learned my addr 172.21.15.96:0/2585897346 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-24T10:42:02.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:02.382+0000 7f027effd700 1 --2- 172.21.15.96:0/2585897346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02800fcf80 0x7f0280112c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:02.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:02.382+0000 7f027f7fe700 1 -- 172.21.15.96:0/2585897346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02800fcf80 msgr2=0x7f0280112c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:02.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:02.382+0000 7f027f7fe700 1 --2- 172.21.15.96:0/2585897346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02800fcf80 0x7f0280112c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:02.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:02.382+0000 7f027f7fe700 1 -- 172.21.15.96:0/2585897346 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0274005040 con 0x7f02800fbcd0 2024-03-24T10:42:02.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:02.382+0000 7f027effd700 1 --2- 172.21.15.96:0/2585897346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02800fcf80 0x7f0280112c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T10:42:02.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:02.382+0000 7f027f7fe700 1 --2- 172.21.15.96:0/2585897346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02800fbcd0 0x7f02801106d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f027800bbb0 tx=0x7f0278009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:42:02.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:02.382+0000 7f0267fff700 1 -- 172.21.15.96:0/2585897346 <== mon.1 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02780092e0 con 0x7f02800fbcd0 2024-03-24T10:42:02.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:02.382+0000 7f0285cb4700 1 -- 172.21.15.96:0/2585897346 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02801132b0 con 0x7f02800fbcd0 2024-03-24T10:42:02.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:02.382+0000 7f0267fff700 1 -- 172.21.15.96:0/2585897346 <== mon.1 v2:172.21.15.96:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f027800cbd0 con 0x7f02800fbcd0 2024-03-24T10:42:02.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:02.382+0000 7f0267fff700 1 -- 172.21.15.96:0/2585897346 <== mon.1 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0278007400 con 0x7f02800fbcd0 2024-03-24T10:42:02.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:02.383+0000 7f0285cb4700 1 -- 172.21.15.96:0/2585897346 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0280113870 con 0x7f02800fbcd0 2024-03-24T10:42:02.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:02.383+0000 7f0285cb4700 1 -- 172.21.15.96:0/2585897346 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f028005f5f0 con 0x7f02800fbcd0 2024-03-24T10:42:02.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:02.384+0000 7f0267fff700 1 -- 172.21.15.96:0/2585897346 <== mon.1 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f0278012070 con 0x7f02800fbcd0 2024-03-24T10:42:02.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:02.384+0000 7f0267fff700 1 --2- 172.21.15.96:0/2585897346 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f02680692b0 0x7f026806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:02.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:02.384+0000 7f0267fff700 1 -- 172.21.15.96:0/2585897346 <== mon.1 v2:172.21.15.96:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4202+0+0 (secure 0 0 0) 0x7f0278086ab0 con 0x7f02800fbcd0 2024-03-24T10:42:02.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:02.385+0000 7f027effd700 1 --2- 172.21.15.96:0/2585897346 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f02680692b0 0x7f026806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:02.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:02.385+0000 7f027effd700 1 --2- 172.21.15.96:0/2585897346 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f02680692b0 0x7f026806b760 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f027400b3b0 tx=0x7f0274009e30 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:42:02.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:02.389+0000 7f0267fff700 1 -- 172.21.15.96:0/2585897346 <== mon.1 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f0278053250 con 0x7f02800fbcd0 2024-03-24T10:42:02.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:02.582+0000 7f0285cb4700 1 -- 172.21.15.96:0/2585897346 --> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f0280101220 con 0x7f02680692b0 2024-03-24T10:42:03.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:02 smithi094 ceph-mon[29446]: Detected new or changed devices on smithi096 2024-03-24T10:42:03.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:02 smithi094 ceph-mon[29446]: Adjusting osd_memory_target on smithi096 to 7968M 2024-03-24T10:42:03.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:02 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T10:42:03.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:02 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T10:42:03.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:02 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:42:03.277 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:02 smithi096 ceph-mon[38097]: Detected new or changed devices on smithi096 2024-03-24T10:42:03.277 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:02 smithi096 ceph-mon[38097]: Adjusting osd_memory_target on smithi096 to 7968M 2024-03-24T10:42:03.277 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:02 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T10:42:03.277 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:02 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T10:42:03.277 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:02 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:42:04.133 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:03 smithi096 ceph-mon[38097]: from='client.24171 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:42:04.134 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:03 smithi096 ceph-mon[38097]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-24T10:42:04.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:03 smithi094 ceph-mon[29446]: from='client.24171 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:42:04.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:03 smithi094 ceph-mon[29446]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-24T10:42:05.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:04 smithi094 ceph-mon[29446]: from='client.? 172.21.15.96:0/4248109317' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "838cf934-b936-4257-8510-3d5bbf4ef034"}]: dispatch 2024-03-24T10:42:05.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:04 smithi094 ceph-mon[29446]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "838cf934-b936-4257-8510-3d5bbf4ef034"}]: dispatch 2024-03-24T10:42:05.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:04 smithi094 ceph-mon[29446]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "838cf934-b936-4257-8510-3d5bbf4ef034"}]': finished 2024-03-24T10:42:05.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:04 smithi094 ceph-mon[29446]: osdmap e36: 6 total, 5 up, 6 in 2024-03-24T10:42:05.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:04 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T10:42:05.256 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:04 smithi096 ceph-mon[38097]: from='client.? 172.21.15.96:0/4248109317' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "838cf934-b936-4257-8510-3d5bbf4ef034"}]: dispatch 2024-03-24T10:42:05.256 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:04 smithi096 ceph-mon[38097]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "838cf934-b936-4257-8510-3d5bbf4ef034"}]: dispatch 2024-03-24T10:42:05.257 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:04 smithi096 ceph-mon[38097]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "838cf934-b936-4257-8510-3d5bbf4ef034"}]': finished 2024-03-24T10:42:05.257 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:04 smithi096 ceph-mon[38097]: osdmap e36: 6 total, 5 up, 6 in 2024-03-24T10:42:05.257 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:04 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T10:42:06.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:05 smithi094 ceph-mon[29446]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-24T10:42:06.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:05 smithi094 ceph-mon[29446]: from='client.? 172.21.15.96:0/2492185896' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T10:42:06.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:05 smithi096 ceph-mon[38097]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-24T10:42:06.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:05 smithi096 ceph-mon[38097]: from='client.? 172.21.15.96:0/2492185896' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T10:42:08.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:07 smithi094 ceph-mon[29446]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-24T10:42:08.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:07 smithi096 ceph-mon[38097]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-24T10:42:10.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:09 smithi096 ceph-mon[38097]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-24T10:42:10.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:09 smithi094 ceph-mon[29446]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-24T10:42:12.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:11 smithi096 ceph-mon[38097]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-24T10:42:12.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:11 smithi094 ceph-mon[29446]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-24T10:42:13.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:12 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-24T10:42:13.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:12 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:42:13.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:12 smithi096 ceph-mon[38097]: Deploying daemon osd.5 on smithi096 2024-03-24T10:42:13.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:12 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-24T10:42:13.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:12 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:42:13.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:12 smithi094 ceph-mon[29446]: Deploying daemon osd.5 on smithi096 2024-03-24T10:42:14.243 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:13 smithi096 ceph-mon[38097]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-24T10:42:14.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:13 smithi094 ceph-mon[29446]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-24T10:42:16.151 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:15 smithi096 ceph-mon[38097]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-24T10:42:16.151 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:15 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:42:16.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:15 smithi094 ceph-mon[29446]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-24T10:42:16.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:15 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:42:18.101 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:17 smithi096 ceph-mon[38097]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-24T10:42:18.102 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:17 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:42:18.102 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:17 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:42:18.102 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:17 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:42:18.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:17 smithi094 ceph-mon[29446]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-24T10:42:18.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:17 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:42:18.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:17 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:42:18.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:17 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:42:20.127 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:19 smithi096 ceph-mon[38097]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-24T10:42:20.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:19 smithi094 ceph-mon[29446]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-24T10:42:20.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:20.896+0000 7f0267fff700 1 -- 172.21.15.96:0/2585897346 <== mgr.14232 v2:172.21.15.94:6800/2184357820 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0280101220 con 0x7f02680692b0 2024-03-24T10:42:20.896 INFO:teuthology.orchestra.run.smithi096.stdout:Created osd(s) 5 on host 'smithi096' 2024-03-24T10:42:20.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:20.898+0000 7f0285cb4700 1 -- 172.21.15.96:0/2585897346 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f02680692b0 msgr2=0x7f026806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:20.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:20.898+0000 7f0285cb4700 1 --2- 172.21.15.96:0/2585897346 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f02680692b0 0x7f026806b760 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f027400b3b0 tx=0x7f0274009e30 comp rx=0 tx=0).stop 2024-03-24T10:42:20.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:20.898+0000 7f0285cb4700 1 -- 172.21.15.96:0/2585897346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02800fbcd0 msgr2=0x7f02801106d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:20.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:20.898+0000 7f0285cb4700 1 --2- 172.21.15.96:0/2585897346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02800fbcd0 0x7f02801106d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f027800bbb0 tx=0x7f0278009840 comp rx=0 tx=0).stop 2024-03-24T10:42:20.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:20.898+0000 7f0285cb4700 1 -- 172.21.15.96:0/2585897346 shutdown_connections 2024-03-24T10:42:20.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:20.898+0000 7f0285cb4700 1 --2- 172.21.15.96:0/2585897346 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f02680692b0 0x7f026806b760 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:20.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:20.898+0000 7f0285cb4700 1 --2- 172.21.15.96:0/2585897346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02800fbcd0 0x7f02801106d0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:20.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:20.898+0000 7f0285cb4700 1 --2- 172.21.15.96:0/2585897346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02800fcf80 0x7f0280112c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:20.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:20.898+0000 7f0285cb4700 1 -- 172.21.15.96:0/2585897346 >> 172.21.15.96:0/2585897346 conn(0x7f02800f7430 msgr2=0x7f02800f8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:42:20.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:20.898+0000 7f0285cb4700 1 -- 172.21.15.96:0/2585897346 shutdown_connections 2024-03-24T10:42:20.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:20.898+0000 7f0285cb4700 1 -- 172.21.15.96:0/2585897346 wait complete. 2024-03-24T10:42:21.861 DEBUG:teuthology.orchestra.run.smithi096:osd.5> sudo journalctl -f -n 0 -u ceph-386110d4-e9ca-11ee-95cd-87774f69a715@osd.5.service 2024-03-24T10:42:21.863 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-03-24T10:42:21.864 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph osd stat -f json 2024-03-24T10:42:22.138 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:42:22.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:21 smithi094 ceph-mon[29446]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-24T10:42:22.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:21 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:42:22.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:21 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:42:22.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:21 smithi094 ceph-mon[29446]: from='osd.5 [v2:172.21.15.96:6816/3738830217,v1:172.21.15.96:6817/3738830217]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-24T10:42:22.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:21 smithi094 ceph-mon[29446]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-24T10:42:22.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:21 smithi096 ceph-mon[38097]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-24T10:42:22.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:21 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:42:22.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:21 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:42:22.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:21 smithi096 ceph-mon[38097]: from='osd.5 [v2:172.21.15.96:6816/3738830217,v1:172.21.15.96:6817/3738830217]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-24T10:42:22.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:21 smithi096 ceph-mon[38097]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-24T10:42:23.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:22 smithi096 ceph-mon[38097]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-24T10:42:23.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:22 smithi096 ceph-mon[38097]: osdmap e37: 6 total, 5 up, 6 in 2024-03-24T10:42:23.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:22 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T10:42:23.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:22 smithi096 ceph-mon[38097]: from='osd.5 [v2:172.21.15.96:6816/3738830217,v1:172.21.15.96:6817/3738830217]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-03-24T10:42:23.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:22 smithi096 ceph-mon[38097]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-03-24T10:42:23.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:22 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:42:23.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:22 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:42:23.076 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:42:23 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5[57301]: 2024-03-24T10:42:23.000+0000 7f2d16191700 -1 osd.5 0 waiting for initial osdmap 2024-03-24T10:42:23.076 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:42:23 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5[57301]: 2024-03-24T10:42:23.007+0000 7f2d0ef88700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T10:42:23.282 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:22 smithi094 ceph-mon[29446]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-24T10:42:23.282 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:22 smithi094 ceph-mon[29446]: osdmap e37: 6 total, 5 up, 6 in 2024-03-24T10:42:23.282 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:22 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T10:42:23.282 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:22 smithi094 ceph-mon[29446]: from='osd.5 [v2:172.21.15.96:6816/3738830217,v1:172.21.15.96:6817/3738830217]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-03-24T10:42:23.282 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:22 smithi094 ceph-mon[29446]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-03-24T10:42:23.282 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:22 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:42:23.282 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:22 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:42:23.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:23.719+0000 7fa566f59700 1 -- 172.21.15.94:0/2304081693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5600fe050 msgr2=0x7fa5600fe4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:23.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:23.719+0000 7fa566f59700 1 --2- 172.21.15.94:0/2304081693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5600fe050 0x7fa5600fe4c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa550004940 tx=0x7fa550009e30 comp rx=0 tx=0).stop 2024-03-24T10:42:23.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:23.719+0000 7fa566f59700 1 -- 172.21.15.94:0/2304081693 shutdown_connections 2024-03-24T10:42:23.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:23.719+0000 7fa566f59700 1 --2- 172.21.15.94:0/2304081693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5600fe050 0x7fa5600fe4c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:23.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:23.719+0000 7fa566f59700 1 --2- 172.21.15.94:0/2304081693 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5600fce50 0x7fa5600fd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:23.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:23.719+0000 7fa566f59700 1 -- 172.21.15.94:0/2304081693 >> 172.21.15.94:0/2304081693 conn(0x7fa5600f8580 msgr2=0x7fa5600fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:42:23.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:23.719+0000 7fa566f59700 1 -- 172.21.15.94:0/2304081693 shutdown_connections 2024-03-24T10:42:23.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:23.719+0000 7fa566f59700 1 -- 172.21.15.94:0/2304081693 wait complete. 2024-03-24T10:42:23.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:23.719+0000 7fa566f59700 1 Processor -- start 2024-03-24T10:42:23.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:23.719+0000 7fa566f59700 1 -- start start 2024-03-24T10:42:23.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:23.720+0000 7fa566f59700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5600fce50 0x7fa56010d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:23.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:23.720+0000 7fa566f59700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5600fe050 0x7fa56010dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:23.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:23.720+0000 7fa566f59700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56010e3e0 con 0x7fa5600fce50 2024-03-24T10:42:23.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:23.720+0000 7fa566f59700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56010e550 con 0x7fa5600fe050 2024-03-24T10:42:23.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:23.720+0000 7fa564cf5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5600fce50 0x7fa56010d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:23.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:23.720+0000 7fa564cf5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5600fce50 0x7fa56010d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36608/0 (socket says 172.21.15.94:36608) 2024-03-24T10:42:23.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:23.720+0000 7fa564cf5700 1 -- 172.21.15.94:0/3632004938 learned_addr learned my addr 172.21.15.94:0/3632004938 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:42:23.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:23.720+0000 7fa557fff700 1 --2- 172.21.15.94:0/3632004938 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5600fe050 0x7fa56010dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:23.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:23.720+0000 7fa564cf5700 1 -- 172.21.15.94:0/3632004938 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5600fe050 msgr2=0x7fa56010dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:23.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:23.720+0000 7fa564cf5700 1 --2- 172.21.15.94:0/3632004938 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5600fe050 0x7fa56010dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:23.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:23.720+0000 7fa564cf5700 1 -- 172.21.15.94:0/3632004938 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa550005040 con 0x7fa5600fce50 2024-03-24T10:42:23.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:23.720+0000 7fa564cf5700 1 --2- 172.21.15.94:0/3632004938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5600fce50 0x7fa56010d8a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa55c002a80 tx=0x7fa55c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:42:23.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:23.720+0000 7fa5557fa700 1 -- 172.21.15.94:0/3632004938 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa55c007cc0 con 0x7fa5600fce50 2024-03-24T10:42:23.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:23.721+0000 7fa566f59700 1 -- 172.21.15.94:0/3632004938 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa560112fb0 con 0x7fa5600fce50 2024-03-24T10:42:23.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:23.720+0000 7fa5557fa700 1 -- 172.21.15.94:0/3632004938 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa55c00f040 con 0x7fa5600fce50 2024-03-24T10:42:23.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:23.721+0000 7fa5557fa700 1 -- 172.21.15.94:0/3632004938 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa55c00db30 con 0x7fa5600fce50 2024-03-24T10:42:23.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:23.721+0000 7fa566f59700 1 -- 172.21.15.94:0/3632004938 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5601134e0 con 0x7fa5600fce50 2024-03-24T10:42:23.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:23.721+0000 7fa566f59700 1 -- 172.21.15.94:0/3632004938 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa56004c710 con 0x7fa5600fce50 2024-03-24T10:42:23.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:23.722+0000 7fa5557fa700 1 -- 172.21.15.94:0/3632004938 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fa55c00dc90 con 0x7fa5600fce50 2024-03-24T10:42:23.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:23.722+0000 7fa5557fa700 1 --2- 172.21.15.94:0/3632004938 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fa5480692a0 0x7fa54806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:23.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:23.723+0000 7fa557fff700 1 --2- 172.21.15.94:0/3632004938 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fa5480692a0 0x7fa54806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:23.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:23.723+0000 7fa5557fa700 1 -- 172.21.15.94:0/3632004938 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4354+0+0 (secure 0 0 0) 0x7fa55c086500 con 0x7fa5600fce50 2024-03-24T10:42:23.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:23.723+0000 7fa557fff700 1 --2- 172.21.15.94:0/3632004938 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fa5480692a0 0x7fa54806b750 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa55000b970 tx=0x7fa550009e30 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:42:23.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:23.727+0000 7fa5557fa700 1 -- 172.21.15.94:0/3632004938 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fa55c04d190 con 0x7fa5600fce50 2024-03-24T10:42:23.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:23.914+0000 7fa566f59700 1 -- 172.21.15.94:0/3632004938 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fa5601136d0 con 0x7fa5600fce50 2024-03-24T10:42:23.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:23.915+0000 7fa5557fa700 1 -- 172.21.15.94:0/3632004938 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v38) v1 ==== 74+0+130 (secure 0 0 0) 0x7fa55c056690 con 0x7fa5600fce50 2024-03-24T10:42:23.916 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:42:23.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:23.917+0000 7fa566f59700 1 -- 172.21.15.94:0/3632004938 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fa5480692a0 msgr2=0x7fa54806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:23.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:23.917+0000 7fa566f59700 1 --2- 172.21.15.94:0/3632004938 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fa5480692a0 0x7fa54806b750 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa55000b970 tx=0x7fa550009e30 comp rx=0 tx=0).stop 2024-03-24T10:42:23.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:23.917+0000 7fa566f59700 1 -- 172.21.15.94:0/3632004938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5600fce50 msgr2=0x7fa56010d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:23.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:23.917+0000 7fa566f59700 1 --2- 172.21.15.94:0/3632004938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5600fce50 0x7fa56010d8a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa55c002a80 tx=0x7fa55c0046b0 comp rx=0 tx=0).stop 2024-03-24T10:42:23.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:23.918+0000 7fa566f59700 1 -- 172.21.15.94:0/3632004938 shutdown_connections 2024-03-24T10:42:23.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:23.918+0000 7fa566f59700 1 --2- 172.21.15.94:0/3632004938 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fa5480692a0 0x7fa54806b750 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:23.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:23.918+0000 7fa566f59700 1 --2- 172.21.15.94:0/3632004938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5600fce50 0x7fa56010d8a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:23.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:23.918+0000 7fa566f59700 1 --2- 172.21.15.94:0/3632004938 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5600fe050 0x7fa56010dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:23.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:23.918+0000 7fa566f59700 1 -- 172.21.15.94:0/3632004938 >> 172.21.15.94:0/3632004938 conn(0x7fa5600f8580 msgr2=0x7fa5600f9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:42:23.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:23.918+0000 7fa566f59700 1 -- 172.21.15.94:0/3632004938 shutdown_connections 2024-03-24T10:42:23.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:23.918+0000 7fa566f59700 1 -- 172.21.15.94:0/3632004938 wait complete. 2024-03-24T10:42:24.040 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:23 smithi096 ceph-mon[38097]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-24T10:42:24.041 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:23 smithi096 ceph-mon[38097]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-03-24T10:42:24.041 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:23 smithi096 ceph-mon[38097]: osdmap e38: 6 total, 5 up, 6 in 2024-03-24T10:42:24.041 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:23 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T10:42:24.041 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:23 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T10:42:24.041 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:23 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/3632004938' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-24T10:42:24.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:23 smithi094 ceph-mon[29446]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-24T10:42:24.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:23 smithi094 ceph-mon[29446]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-03-24T10:42:24.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:23 smithi094 ceph-mon[29446]: osdmap e38: 6 total, 5 up, 6 in 2024-03-24T10:42:24.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:23 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T10:42:24.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:23 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T10:42:24.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:23 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/3632004938' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-24T10:42:24.542 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":38,"num_osds":6,"num_up_osds":5,"osd_up_since":1711276917,"num_in_osds":6,"osd_in_since":1711276924,"num_remapped_pgs":0} 2024-03-24T10:42:25.166 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:25 smithi096 ceph-mon[38097]: purged_snaps scrub starts 2024-03-24T10:42:25.166 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:25 smithi096 ceph-mon[38097]: purged_snaps scrub ok 2024-03-24T10:42:25.166 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:25 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T10:42:25.166 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:25 smithi096 ceph-mon[38097]: osd.5 [v2:172.21.15.96:6816/3738830217,v1:172.21.15.96:6817/3738830217] boot 2024-03-24T10:42:25.166 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:25 smithi096 ceph-mon[38097]: osdmap e39: 6 total, 6 up, 6 in 2024-03-24T10:42:25.166 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:25 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T10:42:25.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:25 smithi094 ceph-mon[29446]: purged_snaps scrub starts 2024-03-24T10:42:25.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:25 smithi094 ceph-mon[29446]: purged_snaps scrub ok 2024-03-24T10:42:25.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:25 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T10:42:25.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:25 smithi094 ceph-mon[29446]: osd.5 [v2:172.21.15.96:6816/3738830217,v1:172.21.15.96:6817/3738830217] boot 2024-03-24T10:42:25.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:25 smithi094 ceph-mon[29446]: osdmap e39: 6 total, 6 up, 6 in 2024-03-24T10:42:25.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:25 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T10:42:25.543 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph osd stat -f json 2024-03-24T10:42:25.813 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:42:26.214 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:26 smithi094 ceph-mon[29446]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-24T10:42:26.214 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:26 smithi094 ceph-mon[29446]: osdmap e40: 6 total, 6 up, 6 in 2024-03-24T10:42:26.214 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:26 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:42:26.214 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:26 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:42:26.214 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:26 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T10:42:26.214 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:26 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T10:42:26.214 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:26 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T10:42:26.214 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:26 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:42:26.214 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:26 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:42:26.215 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:26 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:42:26.215 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:26 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:42:26.215 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:26 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:42:26.215 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:26 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:42:26.215 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:26 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:42:26.215 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:26 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:42:26.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:26 smithi096 ceph-mon[38097]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-24T10:42:26.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:26 smithi096 ceph-mon[38097]: osdmap e40: 6 total, 6 up, 6 in 2024-03-24T10:42:26.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:26 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:42:26.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:26 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:42:26.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:26 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T10:42:26.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:26 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T10:42:26.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:26 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T10:42:26.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:26 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:42:26.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:26 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:42:26.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:26 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:42:26.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:26 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:42:26.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:26 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:42:26.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:26 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:42:26.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:26 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:42:26.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:26 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:42:27.235 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:27 smithi094 ceph-mon[29446]: Detected new or changed devices on smithi096 2024-03-24T10:42:27.235 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:27 smithi094 ceph-mon[29446]: Adjusting osd_memory_target on smithi096 to 5312M 2024-03-24T10:42:27.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:27 smithi096 ceph-mon[38097]: Detected new or changed devices on smithi096 2024-03-24T10:42:27.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:27 smithi096 ceph-mon[38097]: Adjusting osd_memory_target on smithi096 to 5312M 2024-03-24T10:42:27.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:27.460+0000 7fded89f7700 1 -- 172.21.15.94:0/3790107792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fded40fbcd0 msgr2=0x7fded40fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:27.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:27.460+0000 7fded89f7700 1 --2- 172.21.15.94:0/3790107792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fded40fbcd0 0x7fded40fc0e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdec8004940 tx=0x7fdec8009e30 comp rx=0 tx=0).stop 2024-03-24T10:42:27.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:27.460+0000 7fded89f7700 1 -- 172.21.15.94:0/3790107792 shutdown_connections 2024-03-24T10:42:27.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:27.460+0000 7fded89f7700 1 --2- 172.21.15.94:0/3790107792 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fded40fcf80 0x7fded40fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:27.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:27.460+0000 7fded89f7700 1 --2- 172.21.15.94:0/3790107792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fded40fbcd0 0x7fded40fc0e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:27.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:27.460+0000 7fded89f7700 1 -- 172.21.15.94:0/3790107792 >> 172.21.15.94:0/3790107792 conn(0x7fded40f7430 msgr2=0x7fded40f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:42:27.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:27.460+0000 7fded89f7700 1 -- 172.21.15.94:0/3790107792 shutdown_connections 2024-03-24T10:42:27.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:27.460+0000 7fded89f7700 1 -- 172.21.15.94:0/3790107792 wait complete. 2024-03-24T10:42:27.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:27.461+0000 7fded89f7700 1 Processor -- start 2024-03-24T10:42:27.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:27.461+0000 7fded89f7700 1 -- start start 2024-03-24T10:42:27.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:27.461+0000 7fded89f7700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fded40fbcd0 0x7fded40ffd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:27.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:27.461+0000 7fded89f7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fded40fcf80 0x7fded40fe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:27.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:27.461+0000 7fded89f7700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded40fe800 con 0x7fded40fbcd0 2024-03-24T10:42:27.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:27.461+0000 7fded89f7700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded40fe940 con 0x7fded40fcf80 2024-03-24T10:42:27.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:27.461+0000 7fded259c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fded40fbcd0 0x7fded40ffd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:27.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:27.461+0000 7fded259c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fded40fbcd0 0x7fded40ffd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36622/0 (socket says 172.21.15.94:36622) 2024-03-24T10:42:27.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:27.461+0000 7fded259c700 1 -- 172.21.15.94:0/656366973 learned_addr learned my addr 172.21.15.94:0/656366973 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:42:27.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:27.461+0000 7fded259c700 1 -- 172.21.15.94:0/656366973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fded40fcf80 msgr2=0x7fded40fe2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:27.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:27.461+0000 7fded259c700 1 --2- 172.21.15.94:0/656366973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fded40fcf80 0x7fded40fe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:27.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:27.461+0000 7fded259c700 1 -- 172.21.15.94:0/656366973 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec8005040 con 0x7fded40fbcd0 2024-03-24T10:42:27.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:27.461+0000 7fded259c700 1 --2- 172.21.15.94:0/656366973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fded40fbcd0 0x7fded40ffd50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdec800b3b0 tx=0x7fdec8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:42:27.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:27.461+0000 7fdec2ffd700 1 -- 172.21.15.94:0/656366973 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdec800e040 con 0x7fded40fbcd0 2024-03-24T10:42:27.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:27.461+0000 7fdec2ffd700 1 -- 172.21.15.94:0/656366973 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdec800bcc0 con 0x7fded40fbcd0 2024-03-24T10:42:27.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:27.461+0000 7fdec2ffd700 1 -- 172.21.15.94:0/656366973 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdec801c630 con 0x7fded40fbcd0 2024-03-24T10:42:27.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:27.461+0000 7fded89f7700 1 -- 172.21.15.94:0/656366973 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fded40febc0 con 0x7fded40fbcd0 2024-03-24T10:42:27.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:27.462+0000 7fded89f7700 1 -- 172.21.15.94:0/656366973 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fded4115ad0 con 0x7fded40fbcd0 2024-03-24T10:42:27.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:27.462+0000 7fded89f7700 1 -- 172.21.15.94:0/656366973 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fded4101b10 con 0x7fded40fbcd0 2024-03-24T10:42:27.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:27.463+0000 7fdec2ffd700 1 -- 172.21.15.94:0/656366973 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fdec8013980 con 0x7fded40fbcd0 2024-03-24T10:42:27.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:27.463+0000 7fdec2ffd700 1 --2- 172.21.15.94:0/656366973 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fdebc0692a0 0x7fdebc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:27.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:27.463+0000 7fdec2ffd700 1 -- 172.21.15.94:0/656366973 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fdec8087ba0 con 0x7fded40fbcd0 2024-03-24T10:42:27.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:27.463+0000 7fded1d9b700 1 --2- 172.21.15.94:0/656366973 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fdebc0692a0 0x7fdebc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:27.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:27.464+0000 7fded1d9b700 1 --2- 172.21.15.94:0/656366973 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fdebc0692a0 0x7fdebc06b750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdec4004ad0 tx=0x7fdec4007d40 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:42:27.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:27.468+0000 7fdec2ffd700 1 -- 172.21.15.94:0/656366973 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fdec8054210 con 0x7fded40fbcd0 2024-03-24T10:42:27.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:27.655+0000 7fded89f7700 1 -- 172.21.15.94:0/656366973 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fded405f5f0 con 0x7fded40fbcd0 2024-03-24T10:42:27.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:27.655+0000 7fdec2ffd700 1 -- 172.21.15.94:0/656366973 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v40) v1 ==== 74+0+130 (secure 0 0 0) 0x7fdec805be60 con 0x7fded40fbcd0 2024-03-24T10:42:27.656 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:42:27.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:27.657+0000 7fded89f7700 1 -- 172.21.15.94:0/656366973 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fdebc0692a0 msgr2=0x7fdebc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:27.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:27.657+0000 7fded89f7700 1 --2- 172.21.15.94:0/656366973 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fdebc0692a0 0x7fdebc06b750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdec4004ad0 tx=0x7fdec4007d40 comp rx=0 tx=0).stop 2024-03-24T10:42:27.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:27.657+0000 7fded89f7700 1 -- 172.21.15.94:0/656366973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fded40fbcd0 msgr2=0x7fded40ffd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:27.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:27.657+0000 7fded89f7700 1 --2- 172.21.15.94:0/656366973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fded40fbcd0 0x7fded40ffd50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdec800b3b0 tx=0x7fdec8009e30 comp rx=0 tx=0).stop 2024-03-24T10:42:27.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:27.657+0000 7fded89f7700 1 -- 172.21.15.94:0/656366973 shutdown_connections 2024-03-24T10:42:27.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:27.657+0000 7fded89f7700 1 --2- 172.21.15.94:0/656366973 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fdebc0692a0 0x7fdebc06b750 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:27.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:27.657+0000 7fded89f7700 1 --2- 172.21.15.94:0/656366973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fded40fbcd0 0x7fded40ffd50 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:27.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:27.657+0000 7fded89f7700 1 --2- 172.21.15.94:0/656366973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fded40fcf80 0x7fded40fe2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:27.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:27.657+0000 7fded89f7700 1 -- 172.21.15.94:0/656366973 >> 172.21.15.94:0/656366973 conn(0x7fded40f7430 msgr2=0x7fded4065b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:42:27.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:27.657+0000 7fded89f7700 1 -- 172.21.15.94:0/656366973 shutdown_connections 2024-03-24T10:42:27.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:27.657+0000 7fded89f7700 1 -- 172.21.15.94:0/656366973 wait complete. 2024-03-24T10:42:28.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:28 smithi094 ceph-mon[29446]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:42:28.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:28 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/656366973' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-24T10:42:28.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:28 smithi096 ceph-mon[38097]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:42:28.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:28 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/656366973' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-24T10:42:28.326 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":40,"num_osds":6,"num_up_osds":6,"osd_up_since":1711276943,"num_in_osds":6,"osd_in_since":1711276924,"num_remapped_pgs":0} 2024-03-24T10:42:28.327 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-24T10:42:28.582 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:42:30.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:30.058+0000 7f5a35653700 1 -- 172.21.15.94:0/3466254129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a30103780 msgr2=0x7f5a300fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:30.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:30.058+0000 7f5a35653700 1 --2- 172.21.15.94:0/3466254129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a30103780 0x7f5a300fcdf0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5a24004940 tx=0x7f5a24009e30 comp rx=0 tx=0).stop 2024-03-24T10:42:30.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:30.059+0000 7f5a35653700 1 -- 172.21.15.94:0/3466254129 shutdown_connections 2024-03-24T10:42:30.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:30.059+0000 7f5a35653700 1 --2- 172.21.15.94:0/3466254129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a30103780 0x7f5a300fcdf0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:30.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:30.059+0000 7f5a35653700 1 --2- 172.21.15.94:0/3466254129 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a30102de0 0x7f5a301031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:30.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:30.059+0000 7f5a35653700 1 -- 172.21.15.94:0/3466254129 >> 172.21.15.94:0/3466254129 conn(0x7f5a300f8970 msgr2=0x7f5a300fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:42:30.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:30.059+0000 7f5a35653700 1 -- 172.21.15.94:0/3466254129 shutdown_connections 2024-03-24T10:42:30.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:30.059+0000 7f5a35653700 1 -- 172.21.15.94:0/3466254129 wait complete. 2024-03-24T10:42:30.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:30.059+0000 7f5a35653700 1 Processor -- start 2024-03-24T10:42:30.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:30.059+0000 7f5a35653700 1 -- start start 2024-03-24T10:42:30.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:30.059+0000 7f5a35653700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a30102de0 0x7f5a30113b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:30.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:30.059+0000 7f5a35653700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a30103780 0x7f5a301140a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:30.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:30.059+0000 7f5a35653700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a301146f0 con 0x7f5a30102de0 2024-03-24T10:42:30.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:30.059+0000 7f5a35653700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3010dae0 con 0x7f5a30103780 2024-03-24T10:42:30.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:30.059+0000 7f5a2effd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a30102de0 0x7f5a30113b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:30.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:30.060+0000 7f5a2effd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a30102de0 0x7f5a30113b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38378/0 (socket says 172.21.15.94:38378) 2024-03-24T10:42:30.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:30.060+0000 7f5a2effd700 1 -- 172.21.15.94:0/1473374144 learned_addr learned my addr 172.21.15.94:0/1473374144 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:42:30.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:30.060+0000 7f5a2e7fc700 1 --2- 172.21.15.94:0/1473374144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a30103780 0x7f5a301140a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:30.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:30.060+0000 7f5a2effd700 1 -- 172.21.15.94:0/1473374144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a30103780 msgr2=0x7f5a301140a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:30.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:30.060+0000 7f5a2effd700 1 --2- 172.21.15.94:0/1473374144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a30103780 0x7f5a301140a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:30.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:30.060+0000 7f5a2effd700 1 -- 172.21.15.94:0/1473374144 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a24005040 con 0x7f5a30102de0 2024-03-24T10:42:30.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:30.060+0000 7f5a2e7fc700 1 --2- 172.21.15.94:0/1473374144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a30103780 0x7f5a301140a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T10:42:30.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:30.060+0000 7f5a2effd700 1 --2- 172.21.15.94:0/1473374144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a30102de0 0x7f5a30113b60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5a2800cb10 tx=0x7f5a28007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:42:30.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:30.060+0000 7f5a177fe700 1 -- 172.21.15.94:0/1473374144 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a28004200 con 0x7f5a30102de0 2024-03-24T10:42:30.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:30.060+0000 7f5a177fe700 1 -- 172.21.15.94:0/1473374144 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5a2800acc0 con 0x7f5a30102de0 2024-03-24T10:42:30.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:30.060+0000 7f5a35653700 1 -- 172.21.15.94:0/1473374144 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a3010ddc0 con 0x7f5a30102de0 2024-03-24T10:42:30.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:30.060+0000 7f5a177fe700 1 -- 172.21.15.94:0/1473374144 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a28003d90 con 0x7f5a30102de0 2024-03-24T10:42:30.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:30.060+0000 7f5a35653700 1 -- 172.21.15.94:0/1473374144 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a3010e380 con 0x7f5a30102de0 2024-03-24T10:42:30.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:30.061+0000 7f5a35653700 1 -- 172.21.15.94:0/1473374144 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a300fe530 con 0x7f5a30102de0 2024-03-24T10:42:30.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:30.061+0000 7f5a177fe700 1 -- 172.21.15.94:0/1473374144 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f5a28003430 con 0x7f5a30102de0 2024-03-24T10:42:30.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:30.062+0000 7f5a177fe700 1 --2- 172.21.15.94:0/1473374144 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f5a180692a0 0x7f5a1806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:30.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:30.062+0000 7f5a2e7fc700 1 --2- 172.21.15.94:0/1473374144 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f5a180692a0 0x7f5a1806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:30.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:30.062+0000 7f5a177fe700 1 -- 172.21.15.94:0/1473374144 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f5a28087450 con 0x7f5a30102de0 2024-03-24T10:42:30.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:30.062+0000 7f5a2e7fc700 1 --2- 172.21.15.94:0/1473374144 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f5a180692a0 0x7f5a1806b750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5a3010eca0 tx=0x7f5a24009e30 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:42:30.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:30.066+0000 7f5a177fe700 1 -- 172.21.15.94:0/1473374144 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f5a280574d0 con 0x7f5a30102de0 2024-03-24T10:42:30.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:30 smithi094 ceph-mon[29446]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:42:30.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:30.254+0000 7f5a35653700 1 -- 172.21.15.94:0/1473374144 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5a30109ec0 con 0x7f5a30102de0 2024-03-24T10:42:30.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:30.255+0000 7f5a177fe700 1 -- 172.21.15.94:0/1473374144 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11048 (secure 0 0 0) 0x7f5a28057060 con 0x7f5a30102de0 2024-03-24T10:42:30.256 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:42:30.256 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":40,"fsid":"386110d4-e9ca-11ee-95cd-87774f69a715","created":"2024-03-24T10:36:12.011908+0000","modified":"2024-03-24T10:42:25.003349+0000","last_up_change":"2024-03-24T10:42:23.998853+0000","last_in_change":"2024-03-24T10:42:04.898193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-24T10:41:09.161003+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"58fd53f9-8ad5-4296-8804-96a98647bba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3857839890},{"type":"v1","addr":"172.21.15.94:6803","nonce":3857839890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3857839890},{"type":"v1","addr":"172.21.15.94:6805","nonce":3857839890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":3857839890},{"type":"v1","addr":"172.21.15.94:6809","nonce":3857839890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3857839890},{"type":"v1","addr":"172.21.15.94:6807","nonce":3857839890}]},"public_addr":"172.21.15.94:6803/3857839890","cluster_addr":"172.21.15.94:6805/3857839890","heartbeat_back_addr":"172.21.15.94:6809/3857839890","heartbeat_front_addr":"172.21.15.94:6807/3857839890","state":["exists","up"]},{"osd":1,"uuid":"3087b482-aede-4e88-848c-3e7438eaccef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":2028968832},{"type":"v1","addr":"172.21.15.94:6811","nonce":2028968832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":2028968832},{"type":"v1","addr":"172.21.15.94:6813","nonce":2028968832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":2028968832},{"type":"v1","addr":"172.21.15.94:6817","nonce":2028968832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":2028968832},{"type":"v1","addr":"172.21.15.94:6815","nonce":2028968832}]},"public_addr":"172.21.15.94:6811/2028968832","cluster_addr":"172.21.15.94:6813/2028968832","heartbeat_back_addr":"172.21.15.94:6817/2028968832","heartbeat_front_addr":"172.21.15.94:6815/2028968832","state":["exists","up"]},{"osd":2,"uuid":"48c00627-22a6-480c-9aef-38bc326fc4dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":2122131371},{"type":"v1","addr":"172.21.15.94:6819","nonce":2122131371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":2122131371},{"type":"v1","addr":"172.21.15.94:6821","nonce":2122131371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2122131371},{"type":"v1","addr":"172.21.15.94:6825","nonce":2122131371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":2122131371},{"type":"v1","addr":"172.21.15.94:6823","nonce":2122131371}]},"public_addr":"172.21.15.94:6819/2122131371","cluster_addr":"172.21.15.94:6821/2122131371","heartbeat_back_addr":"172.21.15.94:6825/2122131371","heartbeat_front_addr":"172.21.15.94:6823/2122131371","state":["exists","up"]},{"osd":3,"uuid":"910859cd-d028-45f8-8e28-af4e46be460b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":3684617676},{"type":"v1","addr":"172.21.15.96:6801","nonce":3684617676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3684617676},{"type":"v1","addr":"172.21.15.96:6803","nonce":3684617676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3684617676},{"type":"v1","addr":"172.21.15.96:6807","nonce":3684617676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3684617676},{"type":"v1","addr":"172.21.15.96:6805","nonce":3684617676}]},"public_addr":"172.21.15.96:6801/3684617676","cluster_addr":"172.21.15.96:6803/3684617676","heartbeat_back_addr":"172.21.15.96:6807/3684617676","heartbeat_front_addr":"172.21.15.96:6805/3684617676","state":["exists","up"]},{"osd":4,"uuid":"9d37e21d-38dc-4e5b-b5ef-3828d7b25096","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":135461738},{"type":"v1","addr":"172.21.15.96:6809","nonce":135461738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":135461738},{"type":"v1","addr":"172.21.15.96:6811","nonce":135461738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":135461738},{"type":"v1","addr":"172.21.15.96:6815","nonce":135461738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":135461738},{"type":"v1","addr":"172.21.15.96:6813","nonce":135461738}]},"public_addr":"172.21.15.96:6809/135461738","cluster_addr":"172.21.15.96:6811/135461738","heartbeat_back_addr":"172.21.15.96:6815/135461738","heartbeat_front_addr":"172.21.15.96:6813/135461738","state":["exists","up"]},{"osd":5,"uuid":"838cf934-b936-4257-8510-3d5bbf4ef034","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3738830217},{"type":"v1","addr":"172.21.15.96:6817","nonce":3738830217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3738830217},{"type":"v1","addr":"172.21.15.96:6819","nonce":3738830217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3738830217},{"type":"v1","addr":"172.21.15.96:6823","nonce":3738830217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3738830217},{"type":"v1","addr":"172.21.15.96:6821","nonce":3738830217}]},"public_addr":"172.21.15.96:6817/3738830217","cluster_addr":"172.21.15.96:6819/3738830217","heartbeat_back_addr":"172.21.15.96:6823/3738830217","heartbeat_front_addr":"172.21.15.96:6821/3738830217","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T10:40:12.038732+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T10:40:38.407784+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T10:41:06.693615+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T10:41:30.382956+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T10:41:56.733747+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T10:42:22.324376+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/2700857461":"2024-03-25T10:39:00.830398+0000","172.21.15.94:6800/2406899470":"2024-03-25T10:39:00.830398+0000","172.21.15.94:0/2661420513":"2024-03-25T10:37:18.183215+0000","172.21.15.94:0/3628417758":"2024-03-25T10:37:18.183215+0000","172.21.15.94:0/629634217":"2024-03-25T10:37:18.183215+0000","172.21.15.94:0/2641430900":"2024-03-25T10:36:48.432382+0000","172.21.15.94:0/2155575468":"2024-03-25T10:39:00.830398+0000","172.21.15.94:6801/2406899470":"2024-03-25T10:39:00.830398+0000","172.21.15.94:6800/1013449112":"2024-03-25T10:36:48.432382+0000","172.21.15.94:0/2130298637":"2024-03-25T10:39:00.830398+0000","172.21.15.94:0/2837872991":"2024-03-25T10:36:48.432382+0000","172.21.15.94:6801/1996608546":"2024-03-25T10:37:18.183215+0000","172.21.15.94:0/4239612108":"2024-03-25T10:36:48.432382+0000","172.21.15.94:6801/1013449112":"2024-03-25T10:36:48.432382+0000","172.21.15.94:6800/1996608546":"2024-03-25T10:37:18.183215+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-24T10:42:30.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:30.257+0000 7f5a35653700 1 -- 172.21.15.94:0/1473374144 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f5a180692a0 msgr2=0x7f5a1806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:30.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:30.257+0000 7f5a35653700 1 --2- 172.21.15.94:0/1473374144 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f5a180692a0 0x7f5a1806b750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5a3010eca0 tx=0x7f5a24009e30 comp rx=0 tx=0).stop 2024-03-24T10:42:30.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:30.257+0000 7f5a35653700 1 -- 172.21.15.94:0/1473374144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a30102de0 msgr2=0x7f5a30113b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:30.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:30.257+0000 7f5a35653700 1 --2- 172.21.15.94:0/1473374144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a30102de0 0x7f5a30113b60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5a2800cb10 tx=0x7f5a28007f90 comp rx=0 tx=0).stop 2024-03-24T10:42:30.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:30.257+0000 7f5a35653700 1 -- 172.21.15.94:0/1473374144 shutdown_connections 2024-03-24T10:42:30.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:30.257+0000 7f5a35653700 1 --2- 172.21.15.94:0/1473374144 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f5a180692a0 0x7f5a1806b750 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:30.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:30.257+0000 7f5a35653700 1 --2- 172.21.15.94:0/1473374144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a30102de0 0x7f5a30113b60 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:30.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:30.257+0000 7f5a35653700 1 --2- 172.21.15.94:0/1473374144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a30103780 0x7f5a301140a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:30.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:30.257+0000 7f5a35653700 1 -- 172.21.15.94:0/1473374144 >> 172.21.15.94:0/1473374144 conn(0x7f5a300f8970 msgr2=0x7f5a300f9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:42:30.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:30.257+0000 7f5a35653700 1 -- 172.21.15.94:0/1473374144 shutdown_connections 2024-03-24T10:42:30.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:30.257+0000 7f5a35653700 1 -- 172.21.15.94:0/1473374144 wait complete. 2024-03-24T10:42:30.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:30 smithi096 ceph-mon[38097]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:42:30.886 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-24T10:41:09.161003+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-03-24T10:42:30.886 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-24T10:42:31.146 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:42:31.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:31 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/1473374144' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T10:42:31.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:31 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:42:31.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:31 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/1473374144' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T10:42:31.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:31 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:42:32.302 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:32 smithi094 ceph-mon[29446]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:42:32.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:32 smithi096 ceph-mon[38097]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:42:32.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:32.524+0000 7f9de9176700 1 -- 172.21.15.94:0/4026443644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9de4102f70 msgr2=0x7f9de4103340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:32.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:32.524+0000 7f9de9176700 1 --2- 172.21.15.94:0/4026443644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9de4102f70 0x7f9de4103340 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4004940 tx=0x7f9dd4009e30 comp rx=0 tx=0).stop 2024-03-24T10:42:32.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:32.524+0000 7f9de9176700 1 -- 172.21.15.94:0/4026443644 shutdown_connections 2024-03-24T10:42:32.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:32.524+0000 7f9de9176700 1 --2- 172.21.15.94:0/4026443644 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9de40fb3b0 0x7f9de40fb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:32.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:32.524+0000 7f9de9176700 1 --2- 172.21.15.94:0/4026443644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9de4102f70 0x7f9de4103340 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:32.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:32.524+0000 7f9de9176700 1 -- 172.21.15.94:0/4026443644 >> 172.21.15.94:0/4026443644 conn(0x7f9de4072310 msgr2=0x7f9de4072710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:42:32.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:32.524+0000 7f9de9176700 1 -- 172.21.15.94:0/4026443644 shutdown_connections 2024-03-24T10:42:32.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:32.524+0000 7f9de9176700 1 -- 172.21.15.94:0/4026443644 wait complete. 2024-03-24T10:42:32.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:32.525+0000 7f9de9176700 1 Processor -- start 2024-03-24T10:42:32.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:32.525+0000 7f9de9176700 1 -- start start 2024-03-24T10:42:32.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:32.525+0000 7f9de9176700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9de40fb3b0 0x7f9de4113900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:32.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:32.525+0000 7f9de9176700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9de4102f70 0x7f9de4113e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:32.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:32.525+0000 7f9de9176700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de4114490 con 0x7f9de4102f70 2024-03-24T10:42:32.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:32.525+0000 7f9de9176700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de410da90 con 0x7f9de40fb3b0 2024-03-24T10:42:32.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:32.525+0000 7f9de259c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9de4102f70 0x7f9de4113e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:32.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:32.525+0000 7f9de259c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9de4102f70 0x7f9de4113e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38404/0 (socket says 172.21.15.94:38404) 2024-03-24T10:42:32.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:32.525+0000 7f9de259c700 1 -- 172.21.15.94:0/3415967534 learned_addr learned my addr 172.21.15.94:0/3415967534 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:42:32.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:32.526+0000 7f9de2d9d700 1 --2- 172.21.15.94:0/3415967534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9de40fb3b0 0x7f9de4113900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:32.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:32.526+0000 7f9de259c700 1 -- 172.21.15.94:0/3415967534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9de40fb3b0 msgr2=0x7f9de4113900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:32.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:32.526+0000 7f9de259c700 1 --2- 172.21.15.94:0/3415967534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9de40fb3b0 0x7f9de4113900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:32.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:32.526+0000 7f9de259c700 1 -- 172.21.15.94:0/3415967534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dd4005040 con 0x7f9de4102f70 2024-03-24T10:42:32.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:32.526+0000 7f9de259c700 1 --2- 172.21.15.94:0/3415967534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9de4102f70 0x7f9de4113e40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9dd800c830 tx=0x7f9dd8004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:42:32.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:32.526+0000 7f9dcb7fe700 1 -- 172.21.15.94:0/3415967534 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9dd800a760 con 0x7f9de4102f70 2024-03-24T10:42:32.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:32.526+0000 7f9de9176700 1 -- 172.21.15.94:0/3415967534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9de410dd70 con 0x7f9de4102f70 2024-03-24T10:42:32.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:32.526+0000 7f9dcb7fe700 1 -- 172.21.15.94:0/3415967534 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9dd800ad90 con 0x7f9de4102f70 2024-03-24T10:42:32.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:32.526+0000 7f9dcb7fe700 1 -- 172.21.15.94:0/3415967534 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9dd80113a0 con 0x7f9de4102f70 2024-03-24T10:42:32.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:32.526+0000 7f9de9176700 1 -- 172.21.15.94:0/3415967534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9de410e330 con 0x7f9de4102f70 2024-03-24T10:42:32.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:32.527+0000 7f9de9176700 1 -- 172.21.15.94:0/3415967534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9de40fe6d0 con 0x7f9de4102f70 2024-03-24T10:42:32.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:32.527+0000 7f9dcb7fe700 1 -- 172.21.15.94:0/3415967534 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f9dd8012400 con 0x7f9de4102f70 2024-03-24T10:42:32.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:32.528+0000 7f9dcb7fe700 1 --2- 172.21.15.94:0/3415967534 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f9dcc0692a0 0x7f9dcc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:32.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:32.528+0000 7f9dcb7fe700 1 -- 172.21.15.94:0/3415967534 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f9dd8086e90 con 0x7f9de4102f70 2024-03-24T10:42:32.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:32.528+0000 7f9de2d9d700 1 --2- 172.21.15.94:0/3415967534 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f9dcc0692a0 0x7f9dcc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:32.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:32.528+0000 7f9de2d9d700 1 --2- 172.21.15.94:0/3415967534 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f9dcc0692a0 0x7f9dcc06b750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4004f90 tx=0x7f9dd4009e30 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:42:32.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:32.532+0000 7f9dcb7fe700 1 -- 172.21.15.94:0/3415967534 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f9dd8056f90 con 0x7f9de4102f70 2024-03-24T10:42:32.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:32.718+0000 7f9de9176700 1 -- 172.21.15.94:0/3415967534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9de4105390 con 0x7f9de4102f70 2024-03-24T10:42:32.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:32.718+0000 7f9dcb7fe700 1 -- 172.21.15.94:0/3415967534 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9dd8056b20 con 0x7f9de4102f70 2024-03-24T10:42:32.719 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-03-24T10:42:32.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:32.721+0000 7f9de9176700 1 -- 172.21.15.94:0/3415967534 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f9dcc0692a0 msgr2=0x7f9dcc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:32.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:32.721+0000 7f9de9176700 1 --2- 172.21.15.94:0/3415967534 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f9dcc0692a0 0x7f9dcc06b750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4004f90 tx=0x7f9dd4009e30 comp rx=0 tx=0).stop 2024-03-24T10:42:32.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:32.721+0000 7f9de9176700 1 -- 172.21.15.94:0/3415967534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9de4102f70 msgr2=0x7f9de4113e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:32.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:32.721+0000 7f9de9176700 1 --2- 172.21.15.94:0/3415967534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9de4102f70 0x7f9de4113e40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9dd800c830 tx=0x7f9dd8004470 comp rx=0 tx=0).stop 2024-03-24T10:42:32.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:32.721+0000 7f9de9176700 1 -- 172.21.15.94:0/3415967534 shutdown_connections 2024-03-24T10:42:32.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:32.721+0000 7f9de9176700 1 --2- 172.21.15.94:0/3415967534 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f9dcc0692a0 0x7f9dcc06b750 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:32.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:32.721+0000 7f9de9176700 1 --2- 172.21.15.94:0/3415967534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9de40fb3b0 0x7f9de4113900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:32.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:32.721+0000 7f9de9176700 1 --2- 172.21.15.94:0/3415967534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9de4102f70 0x7f9de4113e40 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:32.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:32.721+0000 7f9de9176700 1 -- 172.21.15.94:0/3415967534 >> 172.21.15.94:0/3415967534 conn(0x7f9de4072310 msgr2=0x7f9de40f9580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:42:32.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:32.721+0000 7f9de9176700 1 -- 172.21.15.94:0/3415967534 shutdown_connections 2024-03-24T10:42:32.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:32.721+0000 7f9de9176700 1 -- 172.21.15.94:0/3415967534 wait complete. 2024-03-24T10:42:33.305 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:33 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/3415967534' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T10:42:33.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:33 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/3415967534' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T10:42:33.329 INFO:tasks.cephadm:Setting up client nodes... 2024-03-24T10:42:33.329 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-24T10:42:33.582 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:42:34.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:34 smithi096 ceph-mon[38097]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:42:34.361 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:34 smithi094 ceph-mon[29446]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:42:35.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:35.212+0000 7f2a8ac01700 1 -- 172.21.15.94:0/1390682520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a840fffd0 msgr2=0x7f2a841023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:35.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:35.212+0000 7f2a8ac01700 1 --2- 172.21.15.94:0/1390682520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a840fffd0 0x7f2a841023b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2a70004940 tx=0x7f2a70009e30 comp rx=0 tx=0).stop 2024-03-24T10:42:35.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:35.212+0000 7f2a8ac01700 1 -- 172.21.15.94:0/1390682520 shutdown_connections 2024-03-24T10:42:35.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:35.212+0000 7f2a8ac01700 1 --2- 172.21.15.94:0/1390682520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a840fffd0 0x7f2a841023b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:35.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:35.212+0000 7f2a8ac01700 1 --2- 172.21.15.94:0/1390682520 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a84066a90 0x7f2a840ffa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:35.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:35.212+0000 7f2a8ac01700 1 -- 172.21.15.94:0/1390682520 >> 172.21.15.94:0/1390682520 conn(0x7f2a840f7480 msgr2=0x7f2a840f98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:42:35.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:35.212+0000 7f2a8ac01700 1 -- 172.21.15.94:0/1390682520 shutdown_connections 2024-03-24T10:42:35.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:35.212+0000 7f2a8ac01700 1 -- 172.21.15.94:0/1390682520 wait complete. 2024-03-24T10:42:35.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:35.213+0000 7f2a8ac01700 1 Processor -- start 2024-03-24T10:42:35.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:35.213+0000 7f2a8ac01700 1 -- start start 2024-03-24T10:42:35.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:35.213+0000 7f2a8ac01700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a84066a90 0x7f2a840fd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:35.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:35.213+0000 7f2a8ac01700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a840fffd0 0x7f2a840fb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:35.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:35.213+0000 7f2a8ac01700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a840fbed0 con 0x7f2a84066a90 2024-03-24T10:42:35.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:35.213+0000 7f2a8ac01700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a840fc040 con 0x7f2a840fffd0 2024-03-24T10:42:35.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:35.213+0000 7f2a8899d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a84066a90 0x7f2a840fd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:35.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:35.213+0000 7f2a8899d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a84066a90 0x7f2a840fd2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38430/0 (socket says 172.21.15.94:38430) 2024-03-24T10:42:35.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:35.214+0000 7f2a8899d700 1 -- 172.21.15.94:0/706059934 learned_addr learned my addr 172.21.15.94:0/706059934 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:42:35.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:35.214+0000 7f2a83fff700 1 --2- 172.21.15.94:0/706059934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a840fffd0 0x7f2a840fb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:35.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:35.214+0000 7f2a8899d700 1 -- 172.21.15.94:0/706059934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a840fffd0 msgr2=0x7f2a840fb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:35.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:35.214+0000 7f2a8899d700 1 --2- 172.21.15.94:0/706059934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a840fffd0 0x7f2a840fb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:35.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:35.214+0000 7f2a8899d700 1 -- 172.21.15.94:0/706059934 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a70005040 con 0x7f2a84066a90 2024-03-24T10:42:35.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:35.214+0000 7f2a8899d700 1 --2- 172.21.15.94:0/706059934 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a84066a90 0x7f2a840fd2e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2a7800b790 tx=0x7f2a78009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:42:35.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:35.214+0000 7f2a817fa700 1 -- 172.21.15.94:0/706059934 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a7800f6f0 con 0x7f2a84066a90 2024-03-24T10:42:35.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:35.214+0000 7f2a817fa700 1 -- 172.21.15.94:0/706059934 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a7800fd20 con 0x7f2a84066a90 2024-03-24T10:42:35.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:35.214+0000 7f2a817fa700 1 -- 172.21.15.94:0/706059934 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a780103a0 con 0x7f2a84066a90 2024-03-24T10:42:35.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:35.214+0000 7f2a8ac01700 1 -- 172.21.15.94:0/706059934 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a840fc320 con 0x7f2a84066a90 2024-03-24T10:42:35.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:35.214+0000 7f2a8ac01700 1 -- 172.21.15.94:0/706059934 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a84115a70 con 0x7f2a84066a90 2024-03-24T10:42:35.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:35.215+0000 7f2a8ac01700 1 -- 172.21.15.94:0/706059934 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a8404c710 con 0x7f2a84066a90 2024-03-24T10:42:35.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:35.216+0000 7f2a817fa700 1 -- 172.21.15.94:0/706059934 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f2a78010500 con 0x7f2a84066a90 2024-03-24T10:42:35.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:35.216+0000 7f2a817fa700 1 --2- 172.21.15.94:0/706059934 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f2a740692a0 0x7f2a7406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:35.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:35.216+0000 7f2a817fa700 1 -- 172.21.15.94:0/706059934 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f2a78086c30 con 0x7f2a84066a90 2024-03-24T10:42:35.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:35.216+0000 7f2a83fff700 1 --2- 172.21.15.94:0/706059934 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f2a740692a0 0x7f2a7406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:35.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:35.217+0000 7f2a83fff700 1 --2- 172.21.15.94:0/706059934 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f2a740692a0 0x7f2a7406b750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2a7000b3b0 tx=0x7f2a70009e30 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:42:35.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:35.221+0000 7f2a817fa700 1 -- 172.21.15.94:0/706059934 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f2a78056d30 con 0x7f2a84066a90 2024-03-24T10:42:35.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:35.484+0000 7f2a8ac01700 1 -- 172.21.15.94:0/706059934 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f2a841059a0 con 0x7f2a84066a90 2024-03-24T10:42:35.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:35.488+0000 7f2a817fa700 1 -- 172.21.15.94:0/706059934 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v14) v1 ==== 170+0+59 (secure 0 0 0) 0x7f2a841059a0 con 0x7f2a84066a90 2024-03-24T10:42:35.489 INFO:teuthology.orchestra.run.smithi094.stdout:[client.0] 2024-03-24T10:42:35.489 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQCbAwBm4PP6HBAALLQA6minJNS0ik0DsJn3vA== 2024-03-24T10:42:35.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:35.490+0000 7f2a8ac01700 1 -- 172.21.15.94:0/706059934 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f2a740692a0 msgr2=0x7f2a7406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:35.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:35.490+0000 7f2a8ac01700 1 --2- 172.21.15.94:0/706059934 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f2a740692a0 0x7f2a7406b750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2a7000b3b0 tx=0x7f2a70009e30 comp rx=0 tx=0).stop 2024-03-24T10:42:35.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:35.490+0000 7f2a8ac01700 1 -- 172.21.15.94:0/706059934 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a84066a90 msgr2=0x7f2a840fd2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:35.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:35.490+0000 7f2a8ac01700 1 --2- 172.21.15.94:0/706059934 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a84066a90 0x7f2a840fd2e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2a7800b790 tx=0x7f2a78009220 comp rx=0 tx=0).stop 2024-03-24T10:42:35.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:35.490+0000 7f2a8ac01700 1 -- 172.21.15.94:0/706059934 shutdown_connections 2024-03-24T10:42:35.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:35.490+0000 7f2a8ac01700 1 --2- 172.21.15.94:0/706059934 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f2a740692a0 0x7f2a7406b750 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:35.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:35.490+0000 7f2a8ac01700 1 --2- 172.21.15.94:0/706059934 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a84066a90 0x7f2a840fd2e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:35.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:35.490+0000 7f2a8ac01700 1 --2- 172.21.15.94:0/706059934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a840fffd0 0x7f2a840fb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:35.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:35.490+0000 7f2a8ac01700 1 -- 172.21.15.94:0/706059934 >> 172.21.15.94:0/706059934 conn(0x7f2a840f7480 msgr2=0x7f2a840f78a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:42:35.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:35.490+0000 7f2a8ac01700 1 -- 172.21.15.94:0/706059934 shutdown_connections 2024-03-24T10:42:35.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:35.490+0000 7f2a8ac01700 1 -- 172.21.15.94:0/706059934 wait complete. 2024-03-24T10:42:36.220 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:36 smithi094 ceph-mon[29446]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:42:36.220 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:36 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/706059934' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-24T10:42:36.220 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:36 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/706059934' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-24T10:42:36.244 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-03-24T10:42:36.244 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-24T10:42:36.244 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-24T10:42:36.286 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-24T10:42:36.316 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:36 smithi096 ceph-mon[38097]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:42:36.316 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:36 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/706059934' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-24T10:42:36.316 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:36 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/706059934' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-24T10:42:36.549 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi096/config 2024-03-24T10:42:38.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:38.148+0000 7f1e68014700 1 -- 172.21.15.96:0/3253605292 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e600fbce0 msgr2=0x7f1e600fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:38.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:38.148+0000 7f1e68014700 1 --2- 172.21.15.96:0/3253605292 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e600fbce0 0x7f1e600fc0f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c004940 tx=0x7f1e5c009e30 comp rx=0 tx=0).stop 2024-03-24T10:42:38.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:38.148+0000 7f1e68014700 1 -- 172.21.15.96:0/3253605292 shutdown_connections 2024-03-24T10:42:38.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:38.149+0000 7f1e68014700 1 --2- 172.21.15.96:0/3253605292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e600fcf90 0x7f1e600fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:38.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:38.149+0000 7f1e68014700 1 --2- 172.21.15.96:0/3253605292 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e600fbce0 0x7f1e600fc0f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:38.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:38.149+0000 7f1e68014700 1 -- 172.21.15.96:0/3253605292 >> 172.21.15.96:0/3253605292 conn(0x7f1e600f7480 msgr2=0x7f1e600f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:42:38.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:38.149+0000 7f1e68014700 1 -- 172.21.15.96:0/3253605292 shutdown_connections 2024-03-24T10:42:38.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:38.149+0000 7f1e68014700 1 -- 172.21.15.96:0/3253605292 wait complete. 2024-03-24T10:42:38.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:38.149+0000 7f1e68014700 1 Processor -- start 2024-03-24T10:42:38.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:38.149+0000 7f1e68014700 1 -- start start 2024-03-24T10:42:38.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:38.149+0000 7f1e68014700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e600fbce0 0x7f1e60066dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:38.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:38.149+0000 7f1e68014700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e600fcf90 0x7f1e60112670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:38.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:38.149+0000 7f1e68014700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e60112bb0 con 0x7f1e600fcf90 2024-03-24T10:42:38.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:38.149+0000 7f1e68014700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e60112cf0 con 0x7f1e600fbce0 2024-03-24T10:42:38.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:38.149+0000 7f1e65db0700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e600fbce0 0x7f1e60066dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:38.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:38.149+0000 7f1e65db0700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e600fbce0 0x7f1e60066dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48810/0 (socket says 172.21.15.96:48810) 2024-03-24T10:42:38.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:38.149+0000 7f1e65db0700 1 -- 172.21.15.96:0/3114178677 learned_addr learned my addr 172.21.15.96:0/3114178677 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-24T10:42:38.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:38.150+0000 7f1e655af700 1 --2- 172.21.15.96:0/3114178677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e600fcf90 0x7f1e60112670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:38.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:38.150+0000 7f1e65db0700 1 -- 172.21.15.96:0/3114178677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e600fcf90 msgr2=0x7f1e60112670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:38.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:38.150+0000 7f1e65db0700 1 --2- 172.21.15.96:0/3114178677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e600fcf90 0x7f1e60112670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:38.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:38.150+0000 7f1e65db0700 1 -- 172.21.15.96:0/3114178677 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e5c005040 con 0x7f1e600fbce0 2024-03-24T10:42:38.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:38.150+0000 7f1e65db0700 1 --2- 172.21.15.96:0/3114178677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e600fbce0 0x7f1e60066dd0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c013870 tx=0x7f1e5c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:42:38.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:38.150+0000 7f1e567fc700 1 -- 172.21.15.96:0/3114178677 <== mon.1 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e5c021070 con 0x7f1e600fbce0 2024-03-24T10:42:38.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:38.150+0000 7f1e567fc700 1 -- 172.21.15.96:0/3114178677 <== mon.1 v2:172.21.15.96:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1e5c009430 con 0x7f1e600fbce0 2024-03-24T10:42:38.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:38.150+0000 7f1e567fc700 1 -- 172.21.15.96:0/3114178677 <== mon.1 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e5c00e070 con 0x7f1e600fbce0 2024-03-24T10:42:38.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:38.150+0000 7f1e68014700 1 -- 172.21.15.96:0/3114178677 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e60112f70 con 0x7f1e600fbce0 2024-03-24T10:42:38.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:38.150+0000 7f1e68014700 1 -- 172.21.15.96:0/3114178677 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e60113420 con 0x7f1e600fbce0 2024-03-24T10:42:38.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:38.151+0000 7f1e68014700 1 -- 172.21.15.96:0/3114178677 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e6004c710 con 0x7f1e600fbce0 2024-03-24T10:42:38.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:38.151+0000 7f1e567fc700 1 -- 172.21.15.96:0/3114178677 <== mon.1 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f1e5c01c750 con 0x7f1e600fbce0 2024-03-24T10:42:38.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:38.152+0000 7f1e567fc700 1 --2- 172.21.15.96:0/3114178677 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f1e4c069230 0x7f1e4c06b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:38.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:38.152+0000 7f1e567fc700 1 -- 172.21.15.96:0/3114178677 <== mon.1 v2:172.21.15.96:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f1e5c088090 con 0x7f1e600fbce0 2024-03-24T10:42:38.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:38.152+0000 7f1e655af700 1 --2- 172.21.15.96:0/3114178677 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f1e4c069230 0x7f1e4c06b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:38.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:38.152+0000 7f1e655af700 1 --2- 172.21.15.96:0/3114178677 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f1e4c069230 0x7f1e4c06b6e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1e50009b80 tx=0x7f1e5000b040 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:42:38.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:38.156+0000 7f1e567fc700 1 -- 172.21.15.96:0/3114178677 <== mon.1 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f1e5c058190 con 0x7f1e600fbce0 2024-03-24T10:42:38.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:38 smithi096 ceph-mon[38097]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:42:38.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:38.418+0000 7f1e68014700 1 -- 172.21.15.96:0/3114178677 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f1e60113690 con 0x7f1e600fbce0 2024-03-24T10:42:38.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:38.424+0000 7f1e567fc700 1 -- 172.21.15.96:0/3114178677 <== mon.1 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v15) v1 ==== 170+0+59 (secure 0 0 0) 0x7f1e5c057d20 con 0x7f1e600fbce0 2024-03-24T10:42:38.425 INFO:teuthology.orchestra.run.smithi096.stdout:[client.1] 2024-03-24T10:42:38.425 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQCeAwBmgHQKGRAASE3dWEklKzynHlf1cyq57w== 2024-03-24T10:42:38.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:38.426+0000 7f1e68014700 1 -- 172.21.15.96:0/3114178677 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f1e4c069230 msgr2=0x7f1e4c06b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:38.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:38.426+0000 7f1e68014700 1 --2- 172.21.15.96:0/3114178677 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f1e4c069230 0x7f1e4c06b6e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1e50009b80 tx=0x7f1e5000b040 comp rx=0 tx=0).stop 2024-03-24T10:42:38.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:38.426+0000 7f1e68014700 1 -- 172.21.15.96:0/3114178677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e600fbce0 msgr2=0x7f1e60066dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:38.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:38.426+0000 7f1e68014700 1 --2- 172.21.15.96:0/3114178677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e600fbce0 0x7f1e60066dd0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c013870 tx=0x7f1e5c009e30 comp rx=0 tx=0).stop 2024-03-24T10:42:38.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:38.426+0000 7f1e68014700 1 -- 172.21.15.96:0/3114178677 shutdown_connections 2024-03-24T10:42:38.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:38.426+0000 7f1e68014700 1 --2- 172.21.15.96:0/3114178677 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f1e4c069230 0x7f1e4c06b6e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:38.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:38.426+0000 7f1e68014700 1 --2- 172.21.15.96:0/3114178677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e600fbce0 0x7f1e60066dd0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:38.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:38.426+0000 7f1e68014700 1 --2- 172.21.15.96:0/3114178677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e600fcf90 0x7f1e60112670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:38.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:38.426+0000 7f1e68014700 1 -- 172.21.15.96:0/3114178677 >> 172.21.15.96:0/3114178677 conn(0x7f1e600f7480 msgr2=0x7f1e600f8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:42:38.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:38.426+0000 7f1e68014700 1 -- 172.21.15.96:0/3114178677 shutdown_connections 2024-03-24T10:42:38.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:42:38.427+0000 7f1e68014700 1 -- 172.21.15.96:0/3114178677 wait complete. 2024-03-24T10:42:38.492 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:38 smithi094 ceph-mon[29446]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:42:39.059 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-24T10:42:39.060 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-03-24T10:42:39.060 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-24T10:42:39.142 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-24T10:42:39.142 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-24T10:42:39.142 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph mgr dump --format=json 2024-03-24T10:42:39.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:39 smithi096 ceph-mon[38097]: from='client.? 172.21.15.96:0/3114178677' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-24T10:42:39.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:39 smithi096 ceph-mon[38097]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-24T10:42:39.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:39 smithi096 ceph-mon[38097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-24T10:42:39.408 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:42:39.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:39 smithi094 ceph-mon[29446]: from='client.? 172.21.15.96:0/3114178677' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-24T10:42:39.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:39 smithi094 ceph-mon[29446]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-24T10:42:39.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:39 smithi094 ceph-mon[29446]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-24T10:42:40.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:40 smithi094 ceph-mon[29446]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:42:40.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:40 smithi096 ceph-mon[38097]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:42:41.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:41.113+0000 7f02640ff700 1 -- 172.21.15.94:0/1744363189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f025c1037b0 msgr2=0x7f025c0fce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:41.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:41.113+0000 7f02640ff700 1 --2- 172.21.15.94:0/1744363189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f025c1037b0 0x7f025c0fce20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f024c004940 tx=0x7f024c009e30 comp rx=0 tx=0).stop 2024-03-24T10:42:41.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:41.113+0000 7f02640ff700 1 -- 172.21.15.94:0/1744363189 shutdown_connections 2024-03-24T10:42:41.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:41.113+0000 7f02640ff700 1 --2- 172.21.15.94:0/1744363189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f025c1037b0 0x7f025c0fce20 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:41.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:41.113+0000 7f02640ff700 1 --2- 172.21.15.94:0/1744363189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f025c102e10 0x7f025c1031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:41.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:41.113+0000 7f02640ff700 1 -- 172.21.15.94:0/1744363189 >> 172.21.15.94:0/1744363189 conn(0x7f025c0f8980 msgr2=0x7f025c0fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:42:41.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:41.113+0000 7f02640ff700 1 -- 172.21.15.94:0/1744363189 shutdown_connections 2024-03-24T10:42:41.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:41.113+0000 7f02640ff700 1 -- 172.21.15.94:0/1744363189 wait complete. 2024-03-24T10:42:41.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:41.114+0000 7f02640ff700 1 Processor -- start 2024-03-24T10:42:41.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:41.114+0000 7f02640ff700 1 -- start start 2024-03-24T10:42:41.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:41.114+0000 7f02640ff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f025c102e10 0x7f025c113ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:41.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:41.114+0000 7f02640ff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f025c1037b0 0x7f025c1140e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:41.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:41.114+0000 7f02640ff700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025c1147c0 con 0x7f025c1037b0 2024-03-24T10:42:41.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:41.114+0000 7f02640ff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025c10db20 con 0x7f025c102e10 2024-03-24T10:42:41.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:41.114+0000 7f026169a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f025c1037b0 0x7f025c1140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:41.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:41.114+0000 7f026169a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f025c1037b0 0x7f025c1140e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56564/0 (socket says 172.21.15.94:56564) 2024-03-24T10:42:41.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:41.114+0000 7f026169a700 1 -- 172.21.15.94:0/4114296733 learned_addr learned my addr 172.21.15.94:0/4114296733 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:42:41.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:41.114+0000 7f0261e9b700 1 --2- 172.21.15.94:0/4114296733 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f025c102e10 0x7f025c113ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:41.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:41.114+0000 7f026169a700 1 -- 172.21.15.94:0/4114296733 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f025c102e10 msgr2=0x7f025c113ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:41.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:41.114+0000 7f026169a700 1 --2- 172.21.15.94:0/4114296733 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f025c102e10 0x7f025c113ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:41.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:41.114+0000 7f026169a700 1 -- 172.21.15.94:0/4114296733 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f024c005040 con 0x7f025c1037b0 2024-03-24T10:42:41.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:41.114+0000 7f0261e9b700 1 --2- 172.21.15.94:0/4114296733 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f025c102e10 0x7f025c113ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T10:42:41.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:41.114+0000 7f026169a700 1 --2- 172.21.15.94:0/4114296733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f025c1037b0 0x7f025c1140e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f024c009100 tx=0x7f024c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:42:41.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:41.114+0000 7f02527fc700 1 -- 172.21.15.94:0/4114296733 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f024c00e040 con 0x7f025c1037b0 2024-03-24T10:42:41.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:41.114+0000 7f02527fc700 1 -- 172.21.15.94:0/4114296733 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f024c013420 con 0x7f025c1037b0 2024-03-24T10:42:41.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:41.114+0000 7f02640ff700 1 -- 172.21.15.94:0/4114296733 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f025c10dda0 con 0x7f025c1037b0 2024-03-24T10:42:41.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:41.114+0000 7f02527fc700 1 -- 172.21.15.94:0/4114296733 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f024c01c610 con 0x7f025c1037b0 2024-03-24T10:42:41.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:41.115+0000 7f02640ff700 1 -- 172.21.15.94:0/4114296733 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f025c10e360 con 0x7f025c1037b0 2024-03-24T10:42:41.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:41.116+0000 7f02640ff700 1 -- 172.21.15.94:0/4114296733 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f025c0fe560 con 0x7f025c1037b0 2024-03-24T10:42:41.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:41.116+0000 7f02527fc700 1 -- 172.21.15.94:0/4114296733 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f024c013590 con 0x7f025c1037b0 2024-03-24T10:42:41.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:41.116+0000 7f02527fc700 1 --2- 172.21.15.94:0/4114296733 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f02480692a0 0x7f024806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:41.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:41.117+0000 7f02527fc700 1 -- 172.21.15.94:0/4114296733 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f024c087ff0 con 0x7f025c1037b0 2024-03-24T10:42:41.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:41.117+0000 7f0261e9b700 1 --2- 172.21.15.94:0/4114296733 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f02480692a0 0x7f024806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:41.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:41.117+0000 7f0261e9b700 1 --2- 172.21.15.94:0/4114296733 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f02480692a0 0x7f024806b750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0254007f90 tx=0x7f0254007460 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:42:41.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:41.121+0000 7f02527fc700 1 -- 172.21.15.94:0/4114296733 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f024c058240 con 0x7f025c1037b0 2024-03-24T10:42:41.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:41.356+0000 7f02640ff700 1 -- 172.21.15.94:0/4114296733 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f025c05f5f0 con 0x7f025c1037b0 2024-03-24T10:42:41.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:41.360+0000 7f02527fc700 1 -- 172.21.15.94:0/4114296733 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+167166 (secure 0 0 0) 0x7f024c057dd0 con 0x7f025c1037b0 2024-03-24T10:42:41.362 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:42:41.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:41.362+0000 7f02640ff700 1 -- 172.21.15.94:0/4114296733 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f02480692a0 msgr2=0x7f024806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:41.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:41.362+0000 7f02640ff700 1 --2- 172.21.15.94:0/4114296733 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f02480692a0 0x7f024806b750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0254007f90 tx=0x7f0254007460 comp rx=0 tx=0).stop 2024-03-24T10:42:41.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:41.362+0000 7f02640ff700 1 -- 172.21.15.94:0/4114296733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f025c1037b0 msgr2=0x7f025c1140e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:41.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:41.362+0000 7f02640ff700 1 --2- 172.21.15.94:0/4114296733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f025c1037b0 0x7f025c1140e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f024c009100 tx=0x7f024c009e30 comp rx=0 tx=0).stop 2024-03-24T10:42:41.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:41.362+0000 7f02640ff700 1 -- 172.21.15.94:0/4114296733 shutdown_connections 2024-03-24T10:42:41.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:41.362+0000 7f02640ff700 1 --2- 172.21.15.94:0/4114296733 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f02480692a0 0x7f024806b750 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:41.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:41.362+0000 7f02640ff700 1 --2- 172.21.15.94:0/4114296733 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f025c102e10 0x7f025c113ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:41.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:41.362+0000 7f02640ff700 1 --2- 172.21.15.94:0/4114296733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f025c1037b0 0x7f025c1140e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:41.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:41.362+0000 7f02640ff700 1 -- 172.21.15.94:0/4114296733 >> 172.21.15.94:0/4114296733 conn(0x7f025c0f8980 msgr2=0x7f025c0f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:42:41.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:41.362+0000 7f02640ff700 1 -- 172.21.15.94:0/4114296733 shutdown_connections 2024-03-24T10:42:41.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:41.362+0000 7f02640ff700 1 -- 172.21.15.94:0/4114296733 wait complete. 2024-03-24T10:42:42.112 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":19,"active_gid":14232,"active_name":"smithi094.yaethu","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":2184357820},{"type":"v1","addr":"172.21.15.94:6801","nonce":2184357820}]},"active_addr":"172.21.15.94:6801/2184357820","active_change":"2024-03-24T10:39:00.830516+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":14250,"name":"smithi096.gfonnu","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.94:8443/","prometheus":"http://172.21.15.94:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":978791860}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":1027963628}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":265930160}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":206764114}]}]} 2024-03-24T10:42:42.116 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-24T10:42:42.116 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-24T10:42:42.116 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-24T10:42:42.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:42 smithi096 ceph-mon[38097]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:42:42.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:42 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/4114296733' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-24T10:42:42.369 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:42:42.490 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:42 smithi094 ceph-mon[29446]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:42:42.490 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:42 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/4114296733' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-24T10:42:43.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:43.943+0000 7f40b4859700 1 -- 172.21.15.94:0/1992683994 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40b0102710 msgr2=0x7f40b00fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:43.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:43.943+0000 7f40b4859700 1 --2- 172.21.15.94:0/1992683994 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40b0102710 0x7f40b00fbd80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f40a4004940 tx=0x7f40a4009e30 comp rx=0 tx=0).stop 2024-03-24T10:42:43.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:43.943+0000 7f40b4859700 1 -- 172.21.15.94:0/1992683994 shutdown_connections 2024-03-24T10:42:43.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:43.943+0000 7f40b4859700 1 --2- 172.21.15.94:0/1992683994 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40b0102710 0x7f40b00fbd80 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:43.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:43.943+0000 7f40b4859700 1 --2- 172.21.15.94:0/1992683994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40b0101d70 0x7f40b0102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:43.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:43.943+0000 7f40b4859700 1 -- 172.21.15.94:0/1992683994 >> 172.21.15.94:0/1992683994 conn(0x7f40b0072bc0 msgr2=0x7f40b0072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:42:43.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:43.943+0000 7f40b4859700 1 -- 172.21.15.94:0/1992683994 shutdown_connections 2024-03-24T10:42:43.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:43.943+0000 7f40b4859700 1 -- 172.21.15.94:0/1992683994 wait complete. 2024-03-24T10:42:43.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:43.944+0000 7f40b4859700 1 Processor -- start 2024-03-24T10:42:43.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:43.944+0000 7f40b4859700 1 -- start start 2024-03-24T10:42:43.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:43.944+0000 7f40b4859700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40b0101d70 0x7f40b010ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:43.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:43.944+0000 7f40b4859700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40b0102710 0x7f40b010f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:43.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:43.944+0000 7f40b4859700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b010f760 con 0x7f40b0101d70 2024-03-24T10:42:43.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:43.944+0000 7f40b4859700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b010f8a0 con 0x7f40b0102710 2024-03-24T10:42:43.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:43.944+0000 7f40ae59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40b0101d70 0x7f40b010ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:43.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:43.944+0000 7f40ae59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40b0101d70 0x7f40b010ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56578/0 (socket says 172.21.15.94:56578) 2024-03-24T10:42:43.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:43.944+0000 7f40ae59c700 1 -- 172.21.15.94:0/3882747111 learned_addr learned my addr 172.21.15.94:0/3882747111 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:42:43.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:43.944+0000 7f40ae59c700 1 -- 172.21.15.94:0/3882747111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40b0102710 msgr2=0x7f40b010f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:43.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:43.944+0000 7f40add9b700 1 --2- 172.21.15.94:0/3882747111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40b0102710 0x7f40b010f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:43.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:43.944+0000 7f40ae59c700 1 --2- 172.21.15.94:0/3882747111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40b0102710 0x7f40b010f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:43.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:43.944+0000 7f40ae59c700 1 -- 172.21.15.94:0/3882747111 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a4005040 con 0x7f40b0101d70 2024-03-24T10:42:43.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:43.944+0000 7f40ae59c700 1 --2- 172.21.15.94:0/3882747111 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40b0101d70 0x7f40b010ece0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f40a800cb10 tx=0x7f40a8007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:42:43.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:43.945+0000 7f409effd700 1 -- 172.21.15.94:0/3882747111 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40a8004200 con 0x7f40b0101d70 2024-03-24T10:42:43.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:43.945+0000 7f40b4859700 1 -- 172.21.15.94:0/3882747111 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40b0117b90 con 0x7f40b0101d70 2024-03-24T10:42:43.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:43.945+0000 7f409effd700 1 -- 172.21.15.94:0/3882747111 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f40a800acc0 con 0x7f40b0101d70 2024-03-24T10:42:43.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:43.945+0000 7f409effd700 1 -- 172.21.15.94:0/3882747111 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40a8003d90 con 0x7f40b0101d70 2024-03-24T10:42:43.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:43.945+0000 7f40b4859700 1 -- 172.21.15.94:0/3882747111 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40b0118120 con 0x7f40b0101d70 2024-03-24T10:42:43.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:43.946+0000 7f40b4859700 1 -- 172.21.15.94:0/3882747111 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40b0104240 con 0x7f40b0101d70 2024-03-24T10:42:43.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:43.946+0000 7f409effd700 1 -- 172.21.15.94:0/3882747111 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f40a8003430 con 0x7f40b0101d70 2024-03-24T10:42:43.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:43.947+0000 7f409effd700 1 --2- 172.21.15.94:0/3882747111 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f40980692a0 0x7f409806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:43.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:43.947+0000 7f409effd700 1 -- 172.21.15.94:0/3882747111 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f40a8087470 con 0x7f40b0101d70 2024-03-24T10:42:43.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:43.947+0000 7f40add9b700 1 --2- 172.21.15.94:0/3882747111 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f40980692a0 0x7f409806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:43.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:43.947+0000 7f40add9b700 1 --2- 172.21.15.94:0/3882747111 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f40980692a0 0x7f409806b750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f40a400b3b0 tx=0x7f40a4009e30 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:42:43.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:43.951+0000 7f409effd700 1 -- 172.21.15.94:0/3882747111 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f40a8053a60 con 0x7f40b0101d70 2024-03-24T10:42:44.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:44.138+0000 7f40b4859700 1 -- 172.21.15.94:0/3882747111 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f40b004bed0 con 0x7f40b0101d70 2024-03-24T10:42:44.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:44.139+0000 7f409effd700 1 -- 172.21.15.94:0/3882747111 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11048 (secure 0 0 0) 0x7f40a8057080 con 0x7f40b0101d70 2024-03-24T10:42:44.139 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:42:44.139 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":40,"fsid":"386110d4-e9ca-11ee-95cd-87774f69a715","created":"2024-03-24T10:36:12.011908+0000","modified":"2024-03-24T10:42:25.003349+0000","last_up_change":"2024-03-24T10:42:23.998853+0000","last_in_change":"2024-03-24T10:42:04.898193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-24T10:41:09.161003+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"58fd53f9-8ad5-4296-8804-96a98647bba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3857839890},{"type":"v1","addr":"172.21.15.94:6803","nonce":3857839890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3857839890},{"type":"v1","addr":"172.21.15.94:6805","nonce":3857839890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":3857839890},{"type":"v1","addr":"172.21.15.94:6809","nonce":3857839890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3857839890},{"type":"v1","addr":"172.21.15.94:6807","nonce":3857839890}]},"public_addr":"172.21.15.94:6803/3857839890","cluster_addr":"172.21.15.94:6805/3857839890","heartbeat_back_addr":"172.21.15.94:6809/3857839890","heartbeat_front_addr":"172.21.15.94:6807/3857839890","state":["exists","up"]},{"osd":1,"uuid":"3087b482-aede-4e88-848c-3e7438eaccef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":2028968832},{"type":"v1","addr":"172.21.15.94:6811","nonce":2028968832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":2028968832},{"type":"v1","addr":"172.21.15.94:6813","nonce":2028968832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":2028968832},{"type":"v1","addr":"172.21.15.94:6817","nonce":2028968832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":2028968832},{"type":"v1","addr":"172.21.15.94:6815","nonce":2028968832}]},"public_addr":"172.21.15.94:6811/2028968832","cluster_addr":"172.21.15.94:6813/2028968832","heartbeat_back_addr":"172.21.15.94:6817/2028968832","heartbeat_front_addr":"172.21.15.94:6815/2028968832","state":["exists","up"]},{"osd":2,"uuid":"48c00627-22a6-480c-9aef-38bc326fc4dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":2122131371},{"type":"v1","addr":"172.21.15.94:6819","nonce":2122131371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":2122131371},{"type":"v1","addr":"172.21.15.94:6821","nonce":2122131371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2122131371},{"type":"v1","addr":"172.21.15.94:6825","nonce":2122131371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":2122131371},{"type":"v1","addr":"172.21.15.94:6823","nonce":2122131371}]},"public_addr":"172.21.15.94:6819/2122131371","cluster_addr":"172.21.15.94:6821/2122131371","heartbeat_back_addr":"172.21.15.94:6825/2122131371","heartbeat_front_addr":"172.21.15.94:6823/2122131371","state":["exists","up"]},{"osd":3,"uuid":"910859cd-d028-45f8-8e28-af4e46be460b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":3684617676},{"type":"v1","addr":"172.21.15.96:6801","nonce":3684617676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3684617676},{"type":"v1","addr":"172.21.15.96:6803","nonce":3684617676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3684617676},{"type":"v1","addr":"172.21.15.96:6807","nonce":3684617676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3684617676},{"type":"v1","addr":"172.21.15.96:6805","nonce":3684617676}]},"public_addr":"172.21.15.96:6801/3684617676","cluster_addr":"172.21.15.96:6803/3684617676","heartbeat_back_addr":"172.21.15.96:6807/3684617676","heartbeat_front_addr":"172.21.15.96:6805/3684617676","state":["exists","up"]},{"osd":4,"uuid":"9d37e21d-38dc-4e5b-b5ef-3828d7b25096","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":135461738},{"type":"v1","addr":"172.21.15.96:6809","nonce":135461738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":135461738},{"type":"v1","addr":"172.21.15.96:6811","nonce":135461738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":135461738},{"type":"v1","addr":"172.21.15.96:6815","nonce":135461738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":135461738},{"type":"v1","addr":"172.21.15.96:6813","nonce":135461738}]},"public_addr":"172.21.15.96:6809/135461738","cluster_addr":"172.21.15.96:6811/135461738","heartbeat_back_addr":"172.21.15.96:6815/135461738","heartbeat_front_addr":"172.21.15.96:6813/135461738","state":["exists","up"]},{"osd":5,"uuid":"838cf934-b936-4257-8510-3d5bbf4ef034","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3738830217},{"type":"v1","addr":"172.21.15.96:6817","nonce":3738830217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3738830217},{"type":"v1","addr":"172.21.15.96:6819","nonce":3738830217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3738830217},{"type":"v1","addr":"172.21.15.96:6823","nonce":3738830217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3738830217},{"type":"v1","addr":"172.21.15.96:6821","nonce":3738830217}]},"public_addr":"172.21.15.96:6817/3738830217","cluster_addr":"172.21.15.96:6819/3738830217","heartbeat_back_addr":"172.21.15.96:6823/3738830217","heartbeat_front_addr":"172.21.15.96:6821/3738830217","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T10:40:12.038732+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T10:40:38.407784+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T10:41:06.693615+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T10:41:30.382956+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T10:41:56.733747+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T10:42:22.324376+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/2700857461":"2024-03-25T10:39:00.830398+0000","172.21.15.94:6800/2406899470":"2024-03-25T10:39:00.830398+0000","172.21.15.94:0/2661420513":"2024-03-25T10:37:18.183215+0000","172.21.15.94:0/3628417758":"2024-03-25T10:37:18.183215+0000","172.21.15.94:0/629634217":"2024-03-25T10:37:18.183215+0000","172.21.15.94:0/2641430900":"2024-03-25T10:36:48.432382+0000","172.21.15.94:0/2155575468":"2024-03-25T10:39:00.830398+0000","172.21.15.94:6801/2406899470":"2024-03-25T10:39:00.830398+0000","172.21.15.94:6800/1013449112":"2024-03-25T10:36:48.432382+0000","172.21.15.94:0/2130298637":"2024-03-25T10:39:00.830398+0000","172.21.15.94:0/2837872991":"2024-03-25T10:36:48.432382+0000","172.21.15.94:6801/1996608546":"2024-03-25T10:37:18.183215+0000","172.21.15.94:0/4239612108":"2024-03-25T10:36:48.432382+0000","172.21.15.94:6801/1013449112":"2024-03-25T10:36:48.432382+0000","172.21.15.94:6800/1996608546":"2024-03-25T10:37:18.183215+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-24T10:42:44.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:44.140+0000 7f40b4859700 1 -- 172.21.15.94:0/3882747111 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f40980692a0 msgr2=0x7f409806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:44.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:44.140+0000 7f40b4859700 1 --2- 172.21.15.94:0/3882747111 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f40980692a0 0x7f409806b750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f40a400b3b0 tx=0x7f40a4009e30 comp rx=0 tx=0).stop 2024-03-24T10:42:44.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:44.140+0000 7f40b4859700 1 -- 172.21.15.94:0/3882747111 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40b0101d70 msgr2=0x7f40b010ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:44.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:44.140+0000 7f40b4859700 1 --2- 172.21.15.94:0/3882747111 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40b0101d70 0x7f40b010ece0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f40a800cb10 tx=0x7f40a8007f90 comp rx=0 tx=0).stop 2024-03-24T10:42:44.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:44.141+0000 7f40b4859700 1 -- 172.21.15.94:0/3882747111 shutdown_connections 2024-03-24T10:42:44.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:44.141+0000 7f40b4859700 1 --2- 172.21.15.94:0/3882747111 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f40980692a0 0x7f409806b750 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:44.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:44.141+0000 7f40b4859700 1 --2- 172.21.15.94:0/3882747111 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40b0101d70 0x7f40b010ece0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:44.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:44.141+0000 7f40b4859700 1 --2- 172.21.15.94:0/3882747111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40b0102710 0x7f40b010f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:44.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:44.141+0000 7f40b4859700 1 -- 172.21.15.94:0/3882747111 >> 172.21.15.94:0/3882747111 conn(0x7f40b0072bc0 msgr2=0x7f40b00fb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:42:44.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:44.141+0000 7f40b4859700 1 -- 172.21.15.94:0/3882747111 shutdown_connections 2024-03-24T10:42:44.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:44.141+0000 7f40b4859700 1 -- 172.21.15.94:0/3882747111 wait complete. 2024-03-24T10:42:44.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:44 smithi096 ceph-mon[38097]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:42:44.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:44 smithi094 ceph-mon[29446]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:42:44.800 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-24T10:42:44.801 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-24T10:42:45.056 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:42:45.314 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:45 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/3882747111' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T10:42:45.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:45 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/3882747111' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T10:42:46.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:46 smithi096 ceph-mon[38097]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:42:46.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:46 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:42:46.352 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:46 smithi094 ceph-mon[29446]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:42:46.352 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:46 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:42:46.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:46.450+0000 7fbd07b04700 1 -- 172.21.15.94:0/86969835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd001026c0 msgr2=0x7fbd000fbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:46.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:46.450+0000 7fbd07b04700 1 --2- 172.21.15.94:0/86969835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd001026c0 0x7fbd000fbd30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0004940 tx=0x7fbcf0009e30 comp rx=0 tx=0).stop 2024-03-24T10:42:46.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:46.450+0000 7fbd07b04700 1 -- 172.21.15.94:0/86969835 shutdown_connections 2024-03-24T10:42:46.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:46.450+0000 7fbd07b04700 1 --2- 172.21.15.94:0/86969835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd001026c0 0x7fbd000fbd30 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:46.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:46.450+0000 7fbd07b04700 1 --2- 172.21.15.94:0/86969835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd00101d20 0x7fbd001020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:46.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:46.450+0000 7fbd07b04700 1 -- 172.21.15.94:0/86969835 >> 172.21.15.94:0/86969835 conn(0x7fbd000f7870 msgr2=0x7fbd000f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:42:46.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:46.450+0000 7fbd07b04700 1 -- 172.21.15.94:0/86969835 shutdown_connections 2024-03-24T10:42:46.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:46.450+0000 7fbd07b04700 1 -- 172.21.15.94:0/86969835 wait complete. 2024-03-24T10:42:46.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:46.450+0000 7fbd07b04700 1 Processor -- start 2024-03-24T10:42:46.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:46.450+0000 7fbd07b04700 1 -- start start 2024-03-24T10:42:46.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:46.451+0000 7fbd07b04700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd00101d20 0x7fbd0010c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:46.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:46.451+0000 7fbd07b04700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd001026c0 0x7fbd0010ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:46.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:46.451+0000 7fbd07b04700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd00067190 con 0x7fbd00101d20 2024-03-24T10:42:46.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:46.451+0000 7fbd07b04700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0010d3b0 con 0x7fbd001026c0 2024-03-24T10:42:46.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:46.451+0000 7fbd058a0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd00101d20 0x7fbd0010c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:46.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:46.451+0000 7fbd058a0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd00101d20 0x7fbd0010c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56606/0 (socket says 172.21.15.94:56606) 2024-03-24T10:42:46.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:46.451+0000 7fbd058a0700 1 -- 172.21.15.94:0/866444825 learned_addr learned my addr 172.21.15.94:0/866444825 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:42:46.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:46.451+0000 7fbd058a0700 1 -- 172.21.15.94:0/866444825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd001026c0 msgr2=0x7fbd0010ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:46.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:46.451+0000 7fbd058a0700 1 --2- 172.21.15.94:0/866444825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd001026c0 0x7fbd0010ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:46.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:46.451+0000 7fbd058a0700 1 -- 172.21.15.94:0/866444825 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcf0005040 con 0x7fbd00101d20 2024-03-24T10:42:46.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:46.451+0000 7fbd058a0700 1 --2- 172.21.15.94:0/866444825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd00101d20 0x7fbd0010c930 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc00b790 tx=0x7fbcfc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:42:46.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:46.451+0000 7fbcf67fc700 1 -- 172.21.15.94:0/866444825 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbcfc00f770 con 0x7fbd00101d20 2024-03-24T10:42:46.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:46.451+0000 7fbcf67fc700 1 -- 172.21.15.94:0/866444825 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbcfc00fda0 con 0x7fbd00101d20 2024-03-24T10:42:46.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:46.451+0000 7fbcf67fc700 1 -- 172.21.15.94:0/866444825 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbcfc00d310 con 0x7fbd00101d20 2024-03-24T10:42:46.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:46.451+0000 7fbd07b04700 1 -- 172.21.15.94:0/866444825 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd0010d690 con 0x7fbd00101d20 2024-03-24T10:42:46.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:46.451+0000 7fbd07b04700 1 -- 172.21.15.94:0/866444825 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd0011a230 con 0x7fbd00101d20 2024-03-24T10:42:46.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:46.452+0000 7fbd07b04700 1 -- 172.21.15.94:0/866444825 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd000fd470 con 0x7fbd00101d20 2024-03-24T10:42:46.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:46.453+0000 7fbcf67fc700 1 -- 172.21.15.94:0/866444825 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fbcfc010420 con 0x7fbd00101d20 2024-03-24T10:42:46.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:46.453+0000 7fbcf67fc700 1 --2- 172.21.15.94:0/866444825 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fbcec0692a0 0x7fbcec06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:46.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:46.453+0000 7fbd0509f700 1 --2- 172.21.15.94:0/866444825 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fbcec0692a0 0x7fbcec06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:46.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:46.453+0000 7fbcf67fc700 1 -- 172.21.15.94:0/866444825 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fbcfc086d60 con 0x7fbd00101d20 2024-03-24T10:42:46.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:46.454+0000 7fbd0509f700 1 --2- 172.21.15.94:0/866444825 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fbcec0692a0 0x7fbcec06b750 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbcf000b3b0 tx=0x7fbcf0009e30 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:42:46.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:46.457+0000 7fbcf67fc700 1 -- 172.21.15.94:0/866444825 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fbcfc056f00 con 0x7fbd00101d20 2024-03-24T10:42:46.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:46.643+0000 7fbd07b04700 1 -- 172.21.15.94:0/866444825 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbd0004c760 con 0x7fbd00101d20 2024-03-24T10:42:46.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:46.644+0000 7fbcf67fc700 1 -- 172.21.15.94:0/866444825 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11048 (secure 0 0 0) 0x7fbcfc056a90 con 0x7fbd00101d20 2024-03-24T10:42:46.645 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:42:46.645 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":40,"fsid":"386110d4-e9ca-11ee-95cd-87774f69a715","created":"2024-03-24T10:36:12.011908+0000","modified":"2024-03-24T10:42:25.003349+0000","last_up_change":"2024-03-24T10:42:23.998853+0000","last_in_change":"2024-03-24T10:42:04.898193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-24T10:41:09.161003+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"58fd53f9-8ad5-4296-8804-96a98647bba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3857839890},{"type":"v1","addr":"172.21.15.94:6803","nonce":3857839890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3857839890},{"type":"v1","addr":"172.21.15.94:6805","nonce":3857839890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":3857839890},{"type":"v1","addr":"172.21.15.94:6809","nonce":3857839890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3857839890},{"type":"v1","addr":"172.21.15.94:6807","nonce":3857839890}]},"public_addr":"172.21.15.94:6803/3857839890","cluster_addr":"172.21.15.94:6805/3857839890","heartbeat_back_addr":"172.21.15.94:6809/3857839890","heartbeat_front_addr":"172.21.15.94:6807/3857839890","state":["exists","up"]},{"osd":1,"uuid":"3087b482-aede-4e88-848c-3e7438eaccef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":2028968832},{"type":"v1","addr":"172.21.15.94:6811","nonce":2028968832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":2028968832},{"type":"v1","addr":"172.21.15.94:6813","nonce":2028968832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":2028968832},{"type":"v1","addr":"172.21.15.94:6817","nonce":2028968832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":2028968832},{"type":"v1","addr":"172.21.15.94:6815","nonce":2028968832}]},"public_addr":"172.21.15.94:6811/2028968832","cluster_addr":"172.21.15.94:6813/2028968832","heartbeat_back_addr":"172.21.15.94:6817/2028968832","heartbeat_front_addr":"172.21.15.94:6815/2028968832","state":["exists","up"]},{"osd":2,"uuid":"48c00627-22a6-480c-9aef-38bc326fc4dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":2122131371},{"type":"v1","addr":"172.21.15.94:6819","nonce":2122131371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":2122131371},{"type":"v1","addr":"172.21.15.94:6821","nonce":2122131371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2122131371},{"type":"v1","addr":"172.21.15.94:6825","nonce":2122131371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":2122131371},{"type":"v1","addr":"172.21.15.94:6823","nonce":2122131371}]},"public_addr":"172.21.15.94:6819/2122131371","cluster_addr":"172.21.15.94:6821/2122131371","heartbeat_back_addr":"172.21.15.94:6825/2122131371","heartbeat_front_addr":"172.21.15.94:6823/2122131371","state":["exists","up"]},{"osd":3,"uuid":"910859cd-d028-45f8-8e28-af4e46be460b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":3684617676},{"type":"v1","addr":"172.21.15.96:6801","nonce":3684617676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3684617676},{"type":"v1","addr":"172.21.15.96:6803","nonce":3684617676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3684617676},{"type":"v1","addr":"172.21.15.96:6807","nonce":3684617676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3684617676},{"type":"v1","addr":"172.21.15.96:6805","nonce":3684617676}]},"public_addr":"172.21.15.96:6801/3684617676","cluster_addr":"172.21.15.96:6803/3684617676","heartbeat_back_addr":"172.21.15.96:6807/3684617676","heartbeat_front_addr":"172.21.15.96:6805/3684617676","state":["exists","up"]},{"osd":4,"uuid":"9d37e21d-38dc-4e5b-b5ef-3828d7b25096","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":135461738},{"type":"v1","addr":"172.21.15.96:6809","nonce":135461738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":135461738},{"type":"v1","addr":"172.21.15.96:6811","nonce":135461738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":135461738},{"type":"v1","addr":"172.21.15.96:6815","nonce":135461738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":135461738},{"type":"v1","addr":"172.21.15.96:6813","nonce":135461738}]},"public_addr":"172.21.15.96:6809/135461738","cluster_addr":"172.21.15.96:6811/135461738","heartbeat_back_addr":"172.21.15.96:6815/135461738","heartbeat_front_addr":"172.21.15.96:6813/135461738","state":["exists","up"]},{"osd":5,"uuid":"838cf934-b936-4257-8510-3d5bbf4ef034","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3738830217},{"type":"v1","addr":"172.21.15.96:6817","nonce":3738830217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3738830217},{"type":"v1","addr":"172.21.15.96:6819","nonce":3738830217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3738830217},{"type":"v1","addr":"172.21.15.96:6823","nonce":3738830217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3738830217},{"type":"v1","addr":"172.21.15.96:6821","nonce":3738830217}]},"public_addr":"172.21.15.96:6817/3738830217","cluster_addr":"172.21.15.96:6819/3738830217","heartbeat_back_addr":"172.21.15.96:6823/3738830217","heartbeat_front_addr":"172.21.15.96:6821/3738830217","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T10:40:12.038732+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T10:40:38.407784+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T10:41:06.693615+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T10:41:30.382956+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T10:41:56.733747+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T10:42:22.324376+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/2700857461":"2024-03-25T10:39:00.830398+0000","172.21.15.94:6800/2406899470":"2024-03-25T10:39:00.830398+0000","172.21.15.94:0/2661420513":"2024-03-25T10:37:18.183215+0000","172.21.15.94:0/3628417758":"2024-03-25T10:37:18.183215+0000","172.21.15.94:0/629634217":"2024-03-25T10:37:18.183215+0000","172.21.15.94:0/2641430900":"2024-03-25T10:36:48.432382+0000","172.21.15.94:0/2155575468":"2024-03-25T10:39:00.830398+0000","172.21.15.94:6801/2406899470":"2024-03-25T10:39:00.830398+0000","172.21.15.94:6800/1013449112":"2024-03-25T10:36:48.432382+0000","172.21.15.94:0/2130298637":"2024-03-25T10:39:00.830398+0000","172.21.15.94:0/2837872991":"2024-03-25T10:36:48.432382+0000","172.21.15.94:6801/1996608546":"2024-03-25T10:37:18.183215+0000","172.21.15.94:0/4239612108":"2024-03-25T10:36:48.432382+0000","172.21.15.94:6801/1013449112":"2024-03-25T10:36:48.432382+0000","172.21.15.94:6800/1996608546":"2024-03-25T10:37:18.183215+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-24T10:42:46.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:46.646+0000 7fbd07b04700 1 -- 172.21.15.94:0/866444825 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fbcec0692a0 msgr2=0x7fbcec06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:46.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:46.646+0000 7fbd07b04700 1 --2- 172.21.15.94:0/866444825 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fbcec0692a0 0x7fbcec06b750 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbcf000b3b0 tx=0x7fbcf0009e30 comp rx=0 tx=0).stop 2024-03-24T10:42:46.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:46.646+0000 7fbd07b04700 1 -- 172.21.15.94:0/866444825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd00101d20 msgr2=0x7fbd0010c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:46.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:46.646+0000 7fbd07b04700 1 --2- 172.21.15.94:0/866444825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd00101d20 0x7fbd0010c930 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc00b790 tx=0x7fbcfc009250 comp rx=0 tx=0).stop 2024-03-24T10:42:46.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:46.646+0000 7fbd07b04700 1 -- 172.21.15.94:0/866444825 shutdown_connections 2024-03-24T10:42:46.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:46.646+0000 7fbd07b04700 1 --2- 172.21.15.94:0/866444825 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fbcec0692a0 0x7fbcec06b750 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:46.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:46.646+0000 7fbd07b04700 1 --2- 172.21.15.94:0/866444825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd00101d20 0x7fbd0010c930 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:46.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:46.646+0000 7fbd07b04700 1 --2- 172.21.15.94:0/866444825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd001026c0 0x7fbd0010ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:46.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:46.646+0000 7fbd07b04700 1 -- 172.21.15.94:0/866444825 >> 172.21.15.94:0/866444825 conn(0x7fbd000f7870 msgr2=0x7fbd000f88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:42:46.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:46.646+0000 7fbd07b04700 1 -- 172.21.15.94:0/866444825 shutdown_connections 2024-03-24T10:42:46.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:46.646+0000 7fbd07b04700 1 -- 172.21.15.94:0/866444825 wait complete. 2024-03-24T10:42:47.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:47 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/866444825' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T10:42:47.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:47 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/866444825' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T10:42:47.356 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-24T10:42:47.356 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-24T10:42:47.357 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-24T10:42:47.357 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-24T10:42:47.357 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-24T10:42:47.358 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-24T10:42:47.661 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:42:47.665 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:42:48.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:48 smithi096 ceph-mon[38097]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:42:48.354 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:48 smithi094 ceph-mon[29446]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:42:49.318 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:42:49.319 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:42:49.321 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:42:49.324 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:42:50.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:50 smithi094 ceph-mon[29446]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:42:50.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:50 smithi096 ceph-mon[38097]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:42:52.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:52 smithi096 ceph-mon[38097]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:42:52.436 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:52 smithi094 ceph-mon[29446]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:42:53.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.897+0000 7fd7ef631700 1 -- 172.21.15.94:0/581896771 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7e8102e10 msgr2=0x7fd7e81031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:53.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.897+0000 7fd7ef631700 1 --2- 172.21.15.94:0/581896771 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7e8102e10 0x7fd7e81031e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd7e4004940 tx=0x7fd7e4009e30 comp rx=0 tx=0).stop 2024-03-24T10:42:53.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.897+0000 7fd7ef631700 1 -- 172.21.15.94:0/581896771 shutdown_connections 2024-03-24T10:42:53.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.897+0000 7fd7ef631700 1 --2- 172.21.15.94:0/581896771 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7e81037b0 0x7fd7e80fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:53.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.897+0000 7fd7ef631700 1 --2- 172.21.15.94:0/581896771 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7e8102e10 0x7fd7e81031e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:53.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.897+0000 7fd7ef631700 1 -- 172.21.15.94:0/581896771 >> 172.21.15.94:0/581896771 conn(0x7fd7e80f89a0 msgr2=0x7fd7e80fadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:42:53.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.897+0000 7fd7ef631700 1 -- 172.21.15.94:0/581896771 shutdown_connections 2024-03-24T10:42:53.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.897+0000 7fd7ef631700 1 -- 172.21.15.94:0/581896771 wait complete. 2024-03-24T10:42:53.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.897+0000 7fd7ef631700 1 Processor -- start 2024-03-24T10:42:53.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.897+0000 7fd7ef631700 1 -- start start 2024-03-24T10:42:53.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.898+0000 7fd7ef631700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7e8102e10 0x7fd7e80702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:53.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.898+0000 7fd7ef631700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7e81037b0 0x7fd7e8070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:53.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.898+0000 7fd7ef631700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e8070e20 con 0x7fd7e81037b0 2024-03-24T10:42:53.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.898+0000 7fd7ef631700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e8070f60 con 0x7fd7e8102e10 2024-03-24T10:42:53.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.898+0000 7fd7ecbcc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7e81037b0 0x7fd7e8070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:53.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.898+0000 7fd7ecbcc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7e81037b0 0x7fd7e8070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41792/0 (socket says 172.21.15.94:41792) 2024-03-24T10:42:53.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.898+0000 7fd7ecbcc700 1 -- 172.21.15.94:0/1865923848 learned_addr learned my addr 172.21.15.94:0/1865923848 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:42:53.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.898+0000 7fd7ed3cd700 1 --2- 172.21.15.94:0/1865923848 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7e8102e10 0x7fd7e80702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:53.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.898+0000 7fd7ecbcc700 1 -- 172.21.15.94:0/1865923848 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7e8102e10 msgr2=0x7fd7e80702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:53.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.898+0000 7fd7ecbcc700 1 --2- 172.21.15.94:0/1865923848 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7e8102e10 0x7fd7e80702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:53.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.898+0000 7fd7ecbcc700 1 -- 172.21.15.94:0/1865923848 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e4005040 con 0x7fd7e81037b0 2024-03-24T10:42:53.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.898+0000 7fd7ecbcc700 1 --2- 172.21.15.94:0/1865923848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7e81037b0 0x7fd7e8070800 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd7d800b790 tx=0x7fd7d8009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:42:53.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.898+0000 7fd7ddffb700 1 -- 172.21.15.94:0/1865923848 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7d800f6f0 con 0x7fd7e81037b0 2024-03-24T10:42:53.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.899+0000 7fd7ddffb700 1 -- 172.21.15.94:0/1865923848 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd7d800fd20 con 0x7fd7e81037b0 2024-03-24T10:42:53.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.899+0000 7fd7ef631700 1 -- 172.21.15.94:0/1865923848 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7e8119e90 con 0x7fd7e81037b0 2024-03-24T10:42:53.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.899+0000 7fd7ddffb700 1 -- 172.21.15.94:0/1865923848 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7d80103a0 con 0x7fd7e81037b0 2024-03-24T10:42:53.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.899+0000 7fd7ef631700 1 -- 172.21.15.94:0/1865923848 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7e811a370 con 0x7fd7e81037b0 2024-03-24T10:42:53.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.900+0000 7fd7ef631700 1 -- 172.21.15.94:0/1865923848 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd7e80fa380 con 0x7fd7e81037b0 2024-03-24T10:42:53.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.900+0000 7fd7ddffb700 1 -- 172.21.15.94:0/1865923848 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fd7d800f850 con 0x7fd7e81037b0 2024-03-24T10:42:53.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.900+0000 7fd7ddffb700 1 --2- 172.21.15.94:0/1865923848 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fd7d40692a0 0x7fd7d406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:53.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.901+0000 7fd7ddffb700 1 -- 172.21.15.94:0/1865923848 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fd7d8086c10 con 0x7fd7e81037b0 2024-03-24T10:42:53.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.901+0000 7fd7ed3cd700 1 --2- 172.21.15.94:0/1865923848 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fd7d40692a0 0x7fd7d406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:53.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.901+0000 7fd7ddffb700 1 --2- 172.21.15.94:0/1865923848 >> [v2:172.21.15.96:6800/3684617676,v1:172.21.15.96:6801/3684617676] conn(0x7fd7d406ef70 0x7fd7d4071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:53.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.901+0000 7fd7ddffb700 1 -- 172.21.15.94:0/1865923848 --> [v2:172.21.15.96:6800/3684617676,v1:172.21.15.96:6801/3684617676] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd7d4071a30 con 0x7fd7d406ef70 2024-03-24T10:42:53.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.901+0000 7fd7ddffb700 1 -- 172.21.15.94:0/1865923848 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7d808c070 con 0x7fd7e81037b0 2024-03-24T10:42:53.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.901+0000 7fd7ed3cd700 1 --2- 172.21.15.94:0/1865923848 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fd7d40692a0 0x7fd7d406b750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd7e4004f90 tx=0x7fd7e4009e30 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:42:53.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.901+0000 7fd7edbce700 1 --2- 172.21.15.94:0/1865923848 >> [v2:172.21.15.96:6800/3684617676,v1:172.21.15.96:6801/3684617676] conn(0x7fd7d406ef70 0x7fd7d4071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:53.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.901+0000 7fd7edbce700 1 --2- 172.21.15.94:0/1865923848 >> [v2:172.21.15.96:6800/3684617676,v1:172.21.15.96:6801/3684617676] conn(0x7fd7d406ef70 0x7fd7d4071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:42:53.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.902+0000 7fd7ddffb700 1 -- 172.21.15.94:0/1865923848 <== osd.3 v2:172.21.15.96:6800/3684617676 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fd7d4071a30 con 0x7fd7d406ef70 2024-03-24T10:42:53.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.917+0000 7fd7ef631700 1 -- 172.21.15.94:0/1865923848 --> [v2:172.21.15.96:6800/3684617676,v1:172.21.15.96:6801/3684617676] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd7e80fe560 con 0x7fd7d406ef70 2024-03-24T10:42:53.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.917+0000 7fd7ddffb700 1 -- 172.21.15.94:0/1865923848 <== osd.3 v2:172.21.15.96:6800/3684617676 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fd7e80fe560 con 0x7fd7d406ef70 2024-03-24T10:42:53.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.918+0000 7fd7ef631700 1 -- 172.21.15.94:0/1865923848 >> [v2:172.21.15.96:6800/3684617676,v1:172.21.15.96:6801/3684617676] conn(0x7fd7d406ef70 msgr2=0x7fd7d4071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:53.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.918+0000 7fd7ef631700 1 --2- 172.21.15.94:0/1865923848 >> [v2:172.21.15.96:6800/3684617676,v1:172.21.15.96:6801/3684617676] conn(0x7fd7d406ef70 0x7fd7d4071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:53.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.918+0000 7fd7ef631700 1 -- 172.21.15.94:0/1865923848 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fd7d40692a0 msgr2=0x7fd7d406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:53.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.918+0000 7fd7ef631700 1 --2- 172.21.15.94:0/1865923848 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fd7d40692a0 0x7fd7d406b750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd7e4004f90 tx=0x7fd7e4009e30 comp rx=0 tx=0).stop 2024-03-24T10:42:53.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.918+0000 7fd7ef631700 1 -- 172.21.15.94:0/1865923848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7e81037b0 msgr2=0x7fd7e8070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:53.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.918+0000 7fd7ef631700 1 --2- 172.21.15.94:0/1865923848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7e81037b0 0x7fd7e8070800 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd7d800b790 tx=0x7fd7d8009220 comp rx=0 tx=0).stop 2024-03-24T10:42:53.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.918+0000 7fd7ef631700 1 -- 172.21.15.94:0/1865923848 shutdown_connections 2024-03-24T10:42:53.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.918+0000 7fd7ef631700 1 --2- 172.21.15.94:0/1865923848 >> [v2:172.21.15.96:6800/3684617676,v1:172.21.15.96:6801/3684617676] conn(0x7fd7d406ef70 0x7fd7d4071380 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:53.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.918+0000 7fd7ef631700 1 --2- 172.21.15.94:0/1865923848 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fd7d40692a0 0x7fd7d406b750 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:53.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.918+0000 7fd7ef631700 1 --2- 172.21.15.94:0/1865923848 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7e8102e10 0x7fd7e80702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:53.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.918+0000 7fd7ef631700 1 --2- 172.21.15.94:0/1865923848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7e81037b0 0x7fd7e8070800 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:53.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.918+0000 7fd7ef631700 1 -- 172.21.15.94:0/1865923848 >> 172.21.15.94:0/1865923848 conn(0x7fd7e80f89a0 msgr2=0x7fd7e80f9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:42:53.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.918+0000 7fd7ef631700 1 -- 172.21.15.94:0/1865923848 shutdown_connections 2024-03-24T10:42:53.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:53.918+0000 7fd7ef631700 1 -- 172.21.15.94:0/1865923848 wait complete. 2024-03-24T10:42:54.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:54 smithi094 ceph-mon[29446]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:42:54.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:54 smithi096 ceph-mon[38097]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:42:54.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.481+0000 7f587a6dc700 1 -- 172.21.15.94:0/3906734368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58740fe7b0 msgr2=0x7f5874106e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:54.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.481+0000 7f587a6dc700 1 --2- 172.21.15.94:0/3906734368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58740fe7b0 0x7f5874106e80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5868004940 tx=0x7f5868009e30 comp rx=0 tx=0).stop 2024-03-24T10:42:54.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.481+0000 7f587a6dc700 1 -- 172.21.15.94:0/3906734368 shutdown_connections 2024-03-24T10:42:54.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.481+0000 7f587a6dc700 1 --2- 172.21.15.94:0/3906734368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58740fe7b0 0x7f5874106e80 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:54.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.481+0000 7f587a6dc700 1 --2- 172.21.15.94:0/3906734368 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58740fde10 0x7f58740fe1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:54.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.481+0000 7f587a6dc700 1 -- 172.21.15.94:0/3906734368 >> 172.21.15.94:0/3906734368 conn(0x7f58740f7890 msgr2=0x7f58740f9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:42:54.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.481+0000 7f587a6dc700 1 -- 172.21.15.94:0/3906734368 shutdown_connections 2024-03-24T10:42:54.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.481+0000 7f587a6dc700 1 -- 172.21.15.94:0/3906734368 wait complete. 2024-03-24T10:42:54.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.481+0000 7f587a6dc700 1 Processor -- start 2024-03-24T10:42:54.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.482+0000 7f587a6dc700 1 -- start start 2024-03-24T10:42:54.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.482+0000 7f587a6dc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58740fde10 0x7f5874113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:54.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.482+0000 7f587a6dc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58740fe7b0 0x7f5874113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:54.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.482+0000 7f587a6dc700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58741145a0 con 0x7f58740fe7b0 2024-03-24T10:42:54.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.482+0000 7f587a6dc700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f587410db30 con 0x7f58740fde10 2024-03-24T10:42:54.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.482+0000 7f58737fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58740fe7b0 0x7f5874113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:54.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.482+0000 7f58737fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58740fe7b0 0x7f5874113ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41814/0 (socket says 172.21.15.94:41814) 2024-03-24T10:42:54.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.482+0000 7f58737fe700 1 -- 172.21.15.94:0/3612276382 learned_addr learned my addr 172.21.15.94:0/3612276382 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:42:54.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.482+0000 7f5873fff700 1 --2- 172.21.15.94:0/3612276382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58740fde10 0x7f5874113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:54.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.482+0000 7f58737fe700 1 -- 172.21.15.94:0/3612276382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58740fde10 msgr2=0x7f5874113980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:54.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.482+0000 7f58737fe700 1 --2- 172.21.15.94:0/3612276382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58740fde10 0x7f5874113980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:54.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.482+0000 7f58737fe700 1 -- 172.21.15.94:0/3612276382 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5868005040 con 0x7f58740fe7b0 2024-03-24T10:42:54.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.482+0000 7f58737fe700 1 --2- 172.21.15.94:0/3612276382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58740fe7b0 0x7f5874113ec0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f586800b3b0 tx=0x7f5868009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:42:54.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.482+0000 7f5870ff9700 1 -- 172.21.15.94:0/3612276382 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f586800e040 con 0x7f58740fe7b0 2024-03-24T10:42:54.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.483+0000 7f5870ff9700 1 -- 172.21.15.94:0/3612276382 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5868013420 con 0x7f58740fe7b0 2024-03-24T10:42:54.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.483+0000 7f5870ff9700 1 -- 172.21.15.94:0/3612276382 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f586801c610 con 0x7f58740fe7b0 2024-03-24T10:42:54.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.483+0000 7f587a6dc700 1 -- 172.21.15.94:0/3612276382 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f587410ddb0 con 0x7f58740fe7b0 2024-03-24T10:42:54.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.483+0000 7f587a6dc700 1 -- 172.21.15.94:0/3612276382 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f587410e2f0 con 0x7f58740fe7b0 2024-03-24T10:42:54.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.483+0000 7f587a6dc700 1 -- 172.21.15.94:0/3612276382 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f587410e8d0 con 0x7f58740fe7b0 2024-03-24T10:42:54.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.484+0000 7f5870ff9700 1 -- 172.21.15.94:0/3612276382 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f5868013590 con 0x7f58740fe7b0 2024-03-24T10:42:54.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.484+0000 7f5870ff9700 1 --2- 172.21.15.94:0/3612276382 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f585c0692a0 0x7f585c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:54.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.484+0000 7f5870ff9700 1 -- 172.21.15.94:0/3612276382 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f58680880d0 con 0x7f58740fe7b0 2024-03-24T10:42:54.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.484+0000 7f5873fff700 1 --2- 172.21.15.94:0/3612276382 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f585c0692a0 0x7f585c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:54.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.485+0000 7f5870ff9700 1 --2- 172.21.15.94:0/3612276382 >> [v2:172.21.15.94:6818/2122131371,v1:172.21.15.94:6819/2122131371] conn(0x7f585c06ef70 0x7f585c071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:54.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.485+0000 7f5870ff9700 1 -- 172.21.15.94:0/3612276382 --> [v2:172.21.15.94:6818/2122131371,v1:172.21.15.94:6819/2122131371] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f585c071a30 con 0x7f585c06ef70 2024-03-24T10:42:54.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.485+0000 7f5870ff9700 1 -- 172.21.15.94:0/3612276382 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f586808a6c0 con 0x7f58740fe7b0 2024-03-24T10:42:54.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.485+0000 7f5878c79700 1 --2- 172.21.15.94:0/3612276382 >> [v2:172.21.15.94:6818/2122131371,v1:172.21.15.94:6819/2122131371] conn(0x7f585c06ef70 0x7f585c071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:54.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.485+0000 7f5873fff700 1 --2- 172.21.15.94:0/3612276382 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f585c0692a0 0x7f585c06b750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5864006fd0 tx=0x7f5864008040 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:42:54.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.485+0000 7f5878c79700 1 --2- 172.21.15.94:0/3612276382 >> [v2:172.21.15.94:6818/2122131371,v1:172.21.15.94:6819/2122131371] conn(0x7f585c06ef70 0x7f585c071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:42:54.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.486+0000 7f5870ff9700 1 -- 172.21.15.94:0/3612276382 <== osd.2 v2:172.21.15.94:6818/2122131371 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f585c071a30 con 0x7f585c06ef70 2024-03-24T10:42:54.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.500+0000 7f587a6dc700 1 -- 172.21.15.94:0/3612276382 --> [v2:172.21.15.94:6818/2122131371,v1:172.21.15.94:6819/2122131371] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5874102410 con 0x7f585c06ef70 2024-03-24T10:42:54.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.501+0000 7f5870ff9700 1 -- 172.21.15.94:0/3612276382 <== osd.2 v2:172.21.15.94:6818/2122131371 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5874102410 con 0x7f585c06ef70 2024-03-24T10:42:54.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.501+0000 7f587a6dc700 1 -- 172.21.15.94:0/3612276382 >> [v2:172.21.15.94:6818/2122131371,v1:172.21.15.94:6819/2122131371] conn(0x7f585c06ef70 msgr2=0x7f585c071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:54.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.501+0000 7f587a6dc700 1 --2- 172.21.15.94:0/3612276382 >> [v2:172.21.15.94:6818/2122131371,v1:172.21.15.94:6819/2122131371] conn(0x7f585c06ef70 0x7f585c071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:54.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.501+0000 7f587a6dc700 1 -- 172.21.15.94:0/3612276382 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f585c0692a0 msgr2=0x7f585c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:54.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.501+0000 7f587a6dc700 1 --2- 172.21.15.94:0/3612276382 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f585c0692a0 0x7f585c06b750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5864006fd0 tx=0x7f5864008040 comp rx=0 tx=0).stop 2024-03-24T10:42:54.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.501+0000 7f587a6dc700 1 -- 172.21.15.94:0/3612276382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58740fe7b0 msgr2=0x7f5874113ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:54.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.501+0000 7f587a6dc700 1 --2- 172.21.15.94:0/3612276382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58740fe7b0 0x7f5874113ec0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f586800b3b0 tx=0x7f5868009e30 comp rx=0 tx=0).stop 2024-03-24T10:42:54.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.501+0000 7f587a6dc700 1 -- 172.21.15.94:0/3612276382 shutdown_connections 2024-03-24T10:42:54.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.501+0000 7f587a6dc700 1 --2- 172.21.15.94:0/3612276382 >> [v2:172.21.15.94:6818/2122131371,v1:172.21.15.94:6819/2122131371] conn(0x7f585c06ef70 0x7f585c071380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:54.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.501+0000 7f587a6dc700 1 --2- 172.21.15.94:0/3612276382 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f585c0692a0 0x7f585c06b750 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:54.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.501+0000 7f587a6dc700 1 --2- 172.21.15.94:0/3612276382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58740fde10 0x7f5874113980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:54.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.501+0000 7f587a6dc700 1 --2- 172.21.15.94:0/3612276382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58740fe7b0 0x7f5874113ec0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:54.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.501+0000 7f587a6dc700 1 -- 172.21.15.94:0/3612276382 >> 172.21.15.94:0/3612276382 conn(0x7f58740f7890 msgr2=0x7f5874103740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:42:54.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.501+0000 7f587a6dc700 1 -- 172.21.15.94:0/3612276382 shutdown_connections 2024-03-24T10:42:54.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.501+0000 7f587a6dc700 1 -- 172.21.15.94:0/3612276382 wait complete. 2024-03-24T10:42:54.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.856+0000 7f3dce3bb700 1 -- 172.21.15.94:0/3678299513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dc81026c0 msgr2=0x7f3dc80fbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:54.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.856+0000 7f3dce3bb700 1 --2- 172.21.15.94:0/3678299513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dc81026c0 0x7f3dc80fbd30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc004940 tx=0x7f3dbc009e30 comp rx=0 tx=0).stop 2024-03-24T10:42:54.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.856+0000 7f3dce3bb700 1 -- 172.21.15.94:0/3678299513 shutdown_connections 2024-03-24T10:42:54.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.856+0000 7f3dce3bb700 1 --2- 172.21.15.94:0/3678299513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dc81026c0 0x7f3dc80fbd30 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:54.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.856+0000 7f3dce3bb700 1 --2- 172.21.15.94:0/3678299513 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3dc8101d20 0x7f3dc81020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:54.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.856+0000 7f3dce3bb700 1 -- 172.21.15.94:0/3678299513 >> 172.21.15.94:0/3678299513 conn(0x7f3dc80f78d0 msgr2=0x7f3dc80f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:42:54.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.856+0000 7f3dce3bb700 1 -- 172.21.15.94:0/3678299513 shutdown_connections 2024-03-24T10:42:54.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.856+0000 7f3dce3bb700 1 -- 172.21.15.94:0/3678299513 wait complete. 2024-03-24T10:42:54.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.857+0000 7f3dce3bb700 1 Processor -- start 2024-03-24T10:42:54.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.857+0000 7f3dce3bb700 1 -- start start 2024-03-24T10:42:54.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.857+0000 7f3dce3bb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3dc8101d20 0x7f3dc8113bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:54.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.857+0000 7f3dce3bb700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dc81026c0 0x7f3dc81140f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:54.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.857+0000 7f3dce3bb700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc8114740 con 0x7f3dc81026c0 2024-03-24T10:42:54.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.857+0000 7f3dce3bb700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc810db50 con 0x7f3dc8101d20 2024-03-24T10:42:54.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.857+0000 7f3dc77fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dc81026c0 0x7f3dc81140f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:54.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.857+0000 7f3dc77fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dc81026c0 0x7f3dc81140f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41828/0 (socket says 172.21.15.94:41828) 2024-03-24T10:42:54.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.857+0000 7f3dc77fe700 1 -- 172.21.15.94:0/2349659765 learned_addr learned my addr 172.21.15.94:0/2349659765 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:42:54.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.857+0000 7f3dc7fff700 1 --2- 172.21.15.94:0/2349659765 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3dc8101d20 0x7f3dc8113bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:54.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.857+0000 7f3dc77fe700 1 -- 172.21.15.94:0/2349659765 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3dc8101d20 msgr2=0x7f3dc8113bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:54.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.857+0000 7f3dc77fe700 1 --2- 172.21.15.94:0/2349659765 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3dc8101d20 0x7f3dc8113bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:54.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.857+0000 7f3dc77fe700 1 -- 172.21.15.94:0/2349659765 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dbc005040 con 0x7f3dc81026c0 2024-03-24T10:42:54.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.857+0000 7f3dc77fe700 1 --2- 172.21.15.94:0/2349659765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dc81026c0 0x7f3dc81140f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc009180 tx=0x7f3dbc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:42:54.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.858+0000 7f3dc4ff9700 1 -- 172.21.15.94:0/2349659765 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3dbc00e040 con 0x7f3dc81026c0 2024-03-24T10:42:54.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.858+0000 7f3dc4ff9700 1 -- 172.21.15.94:0/2349659765 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3dbc00bbc0 con 0x7f3dc81026c0 2024-03-24T10:42:54.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.858+0000 7f3dce3bb700 1 -- 172.21.15.94:0/2349659765 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dc810ddd0 con 0x7f3dc81026c0 2024-03-24T10:42:54.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.858+0000 7f3dc4ff9700 1 -- 172.21.15.94:0/2349659765 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3dbc01c3f0 con 0x7f3dc81026c0 2024-03-24T10:42:54.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.858+0000 7f3dce3bb700 1 -- 172.21.15.94:0/2349659765 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dc810e390 con 0x7f3dc81026c0 2024-03-24T10:42:54.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.858+0000 7f3dce3bb700 1 -- 172.21.15.94:0/2349659765 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3dc810e870 con 0x7f3dc81026c0 2024-03-24T10:42:54.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.859+0000 7f3dc4ff9700 1 -- 172.21.15.94:0/2349659765 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f3dbc00bd30 con 0x7f3dc81026c0 2024-03-24T10:42:54.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.859+0000 7f3dc4ff9700 1 --2- 172.21.15.94:0/2349659765 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f3db00692a0 0x7f3db006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:54.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.859+0000 7f3dc4ff9700 1 -- 172.21.15.94:0/2349659765 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f3dbc088c10 con 0x7f3dc81026c0 2024-03-24T10:42:54.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.860+0000 7f3dc7fff700 1 --2- 172.21.15.94:0/2349659765 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f3db00692a0 0x7f3db006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:54.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.860+0000 7f3dc4ff9700 1 --2- 172.21.15.94:0/2349659765 >> [v2:172.21.15.94:6802/3857839890,v1:172.21.15.94:6803/3857839890] conn(0x7f3db006ef70 0x7f3db0071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:54.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.860+0000 7f3dc4ff9700 1 -- 172.21.15.94:0/2349659765 --> [v2:172.21.15.94:6802/3857839890,v1:172.21.15.94:6803/3857839890] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3db0071a30 con 0x7f3db006ef70 2024-03-24T10:42:54.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.860+0000 7f3dc4ff9700 1 -- 172.21.15.94:0/2349659765 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3dbc088ff0 con 0x7f3dc81026c0 2024-03-24T10:42:54.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.860+0000 7f3dcc958700 1 --2- 172.21.15.94:0/2349659765 >> [v2:172.21.15.94:6802/3857839890,v1:172.21.15.94:6803/3857839890] conn(0x7f3db006ef70 0x7f3db0071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:54.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.860+0000 7f3dc7fff700 1 --2- 172.21.15.94:0/2349659765 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f3db00692a0 0x7f3db006b750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3db8004990 tx=0x7f3db8009ee0 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:42:54.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.860+0000 7f3dcc958700 1 --2- 172.21.15.94:0/2349659765 >> [v2:172.21.15.94:6802/3857839890,v1:172.21.15.94:6803/3857839890] conn(0x7f3db006ef70 0x7f3db0071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:42:54.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.861+0000 7f3dc4ff9700 1 -- 172.21.15.94:0/2349659765 <== osd.0 v2:172.21.15.94:6802/3857839890 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f3db0071a30 con 0x7f3db006ef70 2024-03-24T10:42:54.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.875+0000 7f3dce3bb700 1 -- 172.21.15.94:0/2349659765 --> [v2:172.21.15.94:6802/3857839890,v1:172.21.15.94:6803/3857839890] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3dc80f92a0 con 0x7f3db006ef70 2024-03-24T10:42:54.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.876+0000 7f3dc4ff9700 1 -- 172.21.15.94:0/2349659765 <== osd.0 v2:172.21.15.94:6802/3857839890 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3dc80f92a0 con 0x7f3db006ef70 2024-03-24T10:42:54.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.876+0000 7f3dce3bb700 1 -- 172.21.15.94:0/2349659765 >> [v2:172.21.15.94:6802/3857839890,v1:172.21.15.94:6803/3857839890] conn(0x7f3db006ef70 msgr2=0x7f3db0071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:54.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.876+0000 7f3dce3bb700 1 --2- 172.21.15.94:0/2349659765 >> [v2:172.21.15.94:6802/3857839890,v1:172.21.15.94:6803/3857839890] conn(0x7f3db006ef70 0x7f3db0071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:54.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.876+0000 7f3dce3bb700 1 -- 172.21.15.94:0/2349659765 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f3db00692a0 msgr2=0x7f3db006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:54.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.876+0000 7f3dce3bb700 1 --2- 172.21.15.94:0/2349659765 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f3db00692a0 0x7f3db006b750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3db8004990 tx=0x7f3db8009ee0 comp rx=0 tx=0).stop 2024-03-24T10:42:54.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.876+0000 7f3dce3bb700 1 -- 172.21.15.94:0/2349659765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dc81026c0 msgr2=0x7f3dc81140f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:54.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.876+0000 7f3dce3bb700 1 --2- 172.21.15.94:0/2349659765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dc81026c0 0x7f3dc81140f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc009180 tx=0x7f3dbc009e30 comp rx=0 tx=0).stop 2024-03-24T10:42:54.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.876+0000 7f3dce3bb700 1 -- 172.21.15.94:0/2349659765 shutdown_connections 2024-03-24T10:42:54.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.876+0000 7f3dce3bb700 1 --2- 172.21.15.94:0/2349659765 >> [v2:172.21.15.94:6802/3857839890,v1:172.21.15.94:6803/3857839890] conn(0x7f3db006ef70 0x7f3db0071380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:54.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.876+0000 7f3dce3bb700 1 --2- 172.21.15.94:0/2349659765 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f3db00692a0 0x7f3db006b750 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:54.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.876+0000 7f3dce3bb700 1 --2- 172.21.15.94:0/2349659765 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3dc8101d20 0x7f3dc8113bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:54.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.876+0000 7f3dce3bb700 1 --2- 172.21.15.94:0/2349659765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dc81026c0 0x7f3dc81140f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:54.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.876+0000 7f3dce3bb700 1 -- 172.21.15.94:0/2349659765 >> 172.21.15.94:0/2349659765 conn(0x7f3dc80f78d0 msgr2=0x7f3dc80f88d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:42:54.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.876+0000 7f3dce3bb700 1 -- 172.21.15.94:0/2349659765 shutdown_connections 2024-03-24T10:42:54.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:54.876+0000 7f3dce3bb700 1 -- 172.21.15.94:0/2349659765 wait complete. 2024-03-24T10:42:55.422 INFO:teuthology.orchestra.run.smithi094.stdout:81604378649 2024-03-24T10:42:55.422 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-24T10:42:55.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.598+0000 7f44af186700 1 -- 172.21.15.94:0/1436344306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44a8102710 msgr2=0x7f44a80fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:55.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.598+0000 7f44af186700 1 --2- 172.21.15.94:0/1436344306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44a8102710 0x7f44a80fbd80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4494004940 tx=0x7f4494009e30 comp rx=0 tx=0).stop 2024-03-24T10:42:55.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.598+0000 7f44af186700 1 -- 172.21.15.94:0/1436344306 shutdown_connections 2024-03-24T10:42:55.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.598+0000 7f44af186700 1 --2- 172.21.15.94:0/1436344306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44a8102710 0x7f44a80fbd80 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:55.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.598+0000 7f44af186700 1 --2- 172.21.15.94:0/1436344306 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44a8101d70 0x7f44a8102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:55.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.598+0000 7f44af186700 1 -- 172.21.15.94:0/1436344306 >> 172.21.15.94:0/1436344306 conn(0x7f44a8072bc0 msgr2=0x7f44a8072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:42:55.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.598+0000 7f44af186700 1 -- 172.21.15.94:0/1436344306 shutdown_connections 2024-03-24T10:42:55.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.598+0000 7f44af186700 1 -- 172.21.15.94:0/1436344306 wait complete. 2024-03-24T10:42:55.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.598+0000 7f44af186700 1 Processor -- start 2024-03-24T10:42:55.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.598+0000 7f44af186700 1 -- start start 2024-03-24T10:42:55.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.599+0000 7f44af186700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44a8101d70 0x7f44a810eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:55.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.599+0000 7f44af186700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44a8102710 0x7f44a810f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:55.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.599+0000 7f44af186700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a810f6d0 con 0x7f44a8102710 2024-03-24T10:42:55.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.599+0000 7f44af186700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a810f810 con 0x7f44a8101d70 2024-03-24T10:42:55.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.599+0000 7f44a7fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44a8102710 0x7f44a810f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:55.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.599+0000 7f44a7fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44a8102710 0x7f44a810f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41844/0 (socket says 172.21.15.94:41844) 2024-03-24T10:42:55.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.599+0000 7f44a7fff700 1 -- 172.21.15.94:0/778518831 learned_addr learned my addr 172.21.15.94:0/778518831 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:42:55.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.599+0000 7f44acf22700 1 --2- 172.21.15.94:0/778518831 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44a8101d70 0x7f44a810eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:55.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.599+0000 7f44a7fff700 1 -- 172.21.15.94:0/778518831 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44a8101d70 msgr2=0x7f44a810eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:55.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.599+0000 7f44a7fff700 1 --2- 172.21.15.94:0/778518831 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44a8101d70 0x7f44a810eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:55.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.599+0000 7f44a7fff700 1 -- 172.21.15.94:0/778518831 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4494005040 con 0x7f44a8102710 2024-03-24T10:42:55.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.599+0000 7f44a7fff700 1 --2- 172.21.15.94:0/778518831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44a8102710 0x7f44a810f080 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4494009180 tx=0x7f4494009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:42:55.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.600+0000 7f44a57fa700 1 -- 172.21.15.94:0/778518831 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f449400e040 con 0x7f44a8102710 2024-03-24T10:42:55.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.600+0000 7f44af186700 1 -- 172.21.15.94:0/778518831 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44a8117bb0 con 0x7f44a8102710 2024-03-24T10:42:55.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.600+0000 7f44af186700 1 -- 172.21.15.94:0/778518831 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44a81180c0 con 0x7f44a8102710 2024-03-24T10:42:55.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.600+0000 7f44a57fa700 1 -- 172.21.15.94:0/778518831 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f449400bbc0 con 0x7f44a8102710 2024-03-24T10:42:55.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.600+0000 7f44a57fa700 1 -- 172.21.15.94:0/778518831 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f449401c3f0 con 0x7f44a8102710 2024-03-24T10:42:55.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.600+0000 7f44af186700 1 -- 172.21.15.94:0/778518831 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f448c000fd0 con 0x7f44a8102710 2024-03-24T10:42:55.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.602+0000 7f44a57fa700 1 -- 172.21.15.94:0/778518831 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f449400bd30 con 0x7f44a8102710 2024-03-24T10:42:55.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.602+0000 7f44a57fa700 1 --2- 172.21.15.94:0/778518831 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f44980692a0 0x7f449806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:55.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.602+0000 7f44a57fa700 1 -- 172.21.15.94:0/778518831 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f4494088a70 con 0x7f44a8102710 2024-03-24T10:42:55.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.602+0000 7f44acf22700 1 --2- 172.21.15.94:0/778518831 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f44980692a0 0x7f449806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:55.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.602+0000 7f44a57fa700 1 --2- 172.21.15.94:0/778518831 >> [v2:172.21.15.96:6808/135461738,v1:172.21.15.96:6809/135461738] conn(0x7f449806ef70 0x7f4498071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:55.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.602+0000 7f44a57fa700 1 -- 172.21.15.94:0/778518831 --> [v2:172.21.15.96:6808/135461738,v1:172.21.15.96:6809/135461738] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4498071a30 con 0x7f449806ef70 2024-03-24T10:42:55.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.602+0000 7f44a57fa700 1 -- 172.21.15.94:0/778518831 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4494088e70 con 0x7f44a8102710 2024-03-24T10:42:55.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.603+0000 7f44acf22700 1 --2- 172.21.15.94:0/778518831 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f44980692a0 0x7f449806b750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f449c007fd0 tx=0x7f449c009040 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:42:55.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.603+0000 7f44ad723700 1 --2- 172.21.15.94:0/778518831 >> [v2:172.21.15.96:6808/135461738,v1:172.21.15.96:6809/135461738] conn(0x7f449806ef70 0x7f4498071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:55.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.603+0000 7f44ad723700 1 --2- 172.21.15.94:0/778518831 >> [v2:172.21.15.96:6808/135461738,v1:172.21.15.96:6809/135461738] conn(0x7f449806ef70 0x7f4498071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:42:55.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.604+0000 7f44a57fa700 1 -- 172.21.15.94:0/778518831 <== osd.4 v2:172.21.15.96:6808/135461738 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f4498071a30 con 0x7f449806ef70 2024-03-24T10:42:55.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.620+0000 7f44af186700 1 -- 172.21.15.94:0/778518831 --> [v2:172.21.15.96:6808/135461738,v1:172.21.15.96:6809/135461738] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f448c002d30 con 0x7f449806ef70 2024-03-24T10:42:55.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.620+0000 7f44a57fa700 1 -- 172.21.15.94:0/778518831 <== osd.4 v2:172.21.15.96:6808/135461738 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f448c002d30 con 0x7f449806ef70 2024-03-24T10:42:55.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.621+0000 7f44af186700 1 -- 172.21.15.94:0/778518831 >> [v2:172.21.15.96:6808/135461738,v1:172.21.15.96:6809/135461738] conn(0x7f449806ef70 msgr2=0x7f4498071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:55.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.621+0000 7f44af186700 1 --2- 172.21.15.94:0/778518831 >> [v2:172.21.15.96:6808/135461738,v1:172.21.15.96:6809/135461738] conn(0x7f449806ef70 0x7f4498071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:55.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.621+0000 7f44af186700 1 -- 172.21.15.94:0/778518831 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f44980692a0 msgr2=0x7f449806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:55.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.621+0000 7f44af186700 1 --2- 172.21.15.94:0/778518831 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f44980692a0 0x7f449806b750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f449c007fd0 tx=0x7f449c009040 comp rx=0 tx=0).stop 2024-03-24T10:42:55.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.621+0000 7f44af186700 1 -- 172.21.15.94:0/778518831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44a8102710 msgr2=0x7f44a810f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:55.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.621+0000 7f44af186700 1 --2- 172.21.15.94:0/778518831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44a8102710 0x7f44a810f080 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4494009180 tx=0x7f4494009e30 comp rx=0 tx=0).stop 2024-03-24T10:42:55.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.621+0000 7f44af186700 1 -- 172.21.15.94:0/778518831 shutdown_connections 2024-03-24T10:42:55.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.621+0000 7f44af186700 1 --2- 172.21.15.94:0/778518831 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f44980692a0 0x7f449806b750 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:55.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.621+0000 7f44af186700 1 --2- 172.21.15.94:0/778518831 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44a8101d70 0x7f44a810eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:55.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.621+0000 7f44af186700 1 --2- 172.21.15.94:0/778518831 >> [v2:172.21.15.96:6808/135461738,v1:172.21.15.96:6809/135461738] conn(0x7f449806ef70 0x7f4498071380 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:55.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.621+0000 7f44af186700 1 --2- 172.21.15.94:0/778518831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44a8102710 0x7f44a810f080 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:55.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.621+0000 7f44af186700 1 -- 172.21.15.94:0/778518831 >> 172.21.15.94:0/778518831 conn(0x7f44a8072bc0 msgr2=0x7f44a80fb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:42:55.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.621+0000 7f44af186700 1 -- 172.21.15.94:0/778518831 shutdown_connections 2024-03-24T10:42:55.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:55.621+0000 7f44af186700 1 -- 172.21.15.94:0/778518831 wait complete. 2024-03-24T10:42:56.201 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:56 smithi094 ceph-mon[29446]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:42:56.225 INFO:teuthology.orchestra.run.smithi094.stdout:38654705698 2024-03-24T10:42:56.225 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-24T10:42:56.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:56 smithi096 ceph-mon[38097]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:42:56.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.414+0000 7f178b971700 1 -- 172.21.15.94:0/1979952029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1784101d10 msgr2=0x7f17841020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:56.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.414+0000 7f178b971700 1 --2- 172.21.15.94:0/1979952029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1784101d10 0x7f17841020e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1780004940 tx=0x7f1780009e30 comp rx=0 tx=0).stop 2024-03-24T10:42:56.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.414+0000 7f178b971700 1 -- 172.21.15.94:0/1979952029 shutdown_connections 2024-03-24T10:42:56.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.414+0000 7f178b971700 1 --2- 172.21.15.94:0/1979952029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17841026b0 0x7f17840fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:56.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.414+0000 7f178b971700 1 --2- 172.21.15.94:0/1979952029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1784101d10 0x7f17841020e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:56.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.414+0000 7f178b971700 1 -- 172.21.15.94:0/1979952029 >> 172.21.15.94:0/1979952029 conn(0x7f17840f7880 msgr2=0x7f17840f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:42:56.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.415+0000 7f178b971700 1 -- 172.21.15.94:0/1979952029 shutdown_connections 2024-03-24T10:42:56.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.415+0000 7f178b971700 1 -- 172.21.15.94:0/1979952029 wait complete. 2024-03-24T10:42:56.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.415+0000 7f178b971700 1 Processor -- start 2024-03-24T10:42:56.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.415+0000 7f178b971700 1 -- start start 2024-03-24T10:42:56.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.415+0000 7f178b971700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1784101d10 0x7f1784113e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:56.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.415+0000 7f178b971700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17841026b0 0x7f1784114380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:56.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.415+0000 7f178b971700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17841149d0 con 0x7f1784101d10 2024-03-24T10:42:56.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.415+0000 7f178b971700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178410dde0 con 0x7f17841026b0 2024-03-24T10:42:56.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.416+0000 7f178970d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1784101d10 0x7f1784113e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:56.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.416+0000 7f178970d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1784101d10 0x7f1784113e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41858/0 (socket says 172.21.15.94:41858) 2024-03-24T10:42:56.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.416+0000 7f178970d700 1 -- 172.21.15.94:0/1103844221 learned_addr learned my addr 172.21.15.94:0/1103844221 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:42:56.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.416+0000 7f1788f0c700 1 --2- 172.21.15.94:0/1103844221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17841026b0 0x7f1784114380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:56.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.416+0000 7f178970d700 1 -- 172.21.15.94:0/1103844221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17841026b0 msgr2=0x7f1784114380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:56.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.416+0000 7f178970d700 1 --2- 172.21.15.94:0/1103844221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17841026b0 0x7f1784114380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:56.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.416+0000 7f178970d700 1 -- 172.21.15.94:0/1103844221 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1780005040 con 0x7f1784101d10 2024-03-24T10:42:56.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.416+0000 7f1788f0c700 1 --2- 172.21.15.94:0/1103844221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17841026b0 0x7f1784114380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T10:42:56.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.416+0000 7f178970d700 1 --2- 172.21.15.94:0/1103844221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1784101d10 0x7f1784113e40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f178000b3b0 tx=0x7f1780009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:42:56.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.416+0000 7f1779ffb700 1 -- 172.21.15.94:0/1103844221 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f178000e040 con 0x7f1784101d10 2024-03-24T10:42:56.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.416+0000 7f1779ffb700 1 -- 172.21.15.94:0/1103844221 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f178000bbc0 con 0x7f1784101d10 2024-03-24T10:42:56.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.416+0000 7f1779ffb700 1 -- 172.21.15.94:0/1103844221 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f178001c3f0 con 0x7f1784101d10 2024-03-24T10:42:56.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.416+0000 7f178b971700 1 -- 172.21.15.94:0/1103844221 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f178410e060 con 0x7f1784101d10 2024-03-24T10:42:56.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.417+0000 7f178b971700 1 -- 172.21.15.94:0/1103844221 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f178410e620 con 0x7f1784101d10 2024-03-24T10:42:56.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.417+0000 7f178b971700 1 -- 172.21.15.94:0/1103844221 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f17840fd460 con 0x7f1784101d10 2024-03-24T10:42:56.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.418+0000 7f1779ffb700 1 -- 172.21.15.94:0/1103844221 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f178000bd30 con 0x7f1784101d10 2024-03-24T10:42:56.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.419+0000 7f1779ffb700 1 --2- 172.21.15.94:0/1103844221 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f17700692a0 0x7f177006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:56.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.419+0000 7f1779ffb700 1 -- 172.21.15.94:0/1103844221 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f1780088b10 con 0x7f1784101d10 2024-03-24T10:42:56.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.419+0000 7f1788f0c700 1 --2- 172.21.15.94:0/1103844221 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f17700692a0 0x7f177006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:56.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.419+0000 7f1779ffb700 1 --2- 172.21.15.94:0/1103844221 >> [v2:172.21.15.94:6810/2028968832,v1:172.21.15.94:6811/2028968832] conn(0x7f177006ef70 0x7f1770071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:56.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.419+0000 7f1779ffb700 1 -- 172.21.15.94:0/1103844221 --> [v2:172.21.15.94:6810/2028968832,v1:172.21.15.94:6811/2028968832] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1770071a30 con 0x7f177006ef70 2024-03-24T10:42:56.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.419+0000 7f1779ffb700 1 -- 172.21.15.94:0/1103844221 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1780088ef0 con 0x7f1784101d10 2024-03-24T10:42:56.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.419+0000 7f1789f0e700 1 --2- 172.21.15.94:0/1103844221 >> [v2:172.21.15.94:6810/2028968832,v1:172.21.15.94:6811/2028968832] conn(0x7f177006ef70 0x7f1770071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:56.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.419+0000 7f1788f0c700 1 --2- 172.21.15.94:0/1103844221 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f17700692a0 0x7f177006b750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1774007fd0 tx=0x7f1774009040 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:42:56.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.419+0000 7f1789f0e700 1 --2- 172.21.15.94:0/1103844221 >> [v2:172.21.15.94:6810/2028968832,v1:172.21.15.94:6811/2028968832] conn(0x7f177006ef70 0x7f1770071380 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:42:56.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.420+0000 7f1779ffb700 1 -- 172.21.15.94:0/1103844221 <== osd.1 v2:172.21.15.94:6810/2028968832 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f1770071a30 con 0x7f177006ef70 2024-03-24T10:42:56.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.435+0000 7f178b971700 1 -- 172.21.15.94:0/1103844221 --> [v2:172.21.15.94:6810/2028968832,v1:172.21.15.94:6811/2028968832] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f178410e1f0 con 0x7f177006ef70 2024-03-24T10:42:56.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.436+0000 7f1779ffb700 1 -- 172.21.15.94:0/1103844221 <== osd.1 v2:172.21.15.94:6810/2028968832 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f178410e1f0 con 0x7f177006ef70 2024-03-24T10:42:56.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.436+0000 7f178b971700 1 -- 172.21.15.94:0/1103844221 >> [v2:172.21.15.94:6810/2028968832,v1:172.21.15.94:6811/2028968832] conn(0x7f177006ef70 msgr2=0x7f1770071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:56.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.436+0000 7f178b971700 1 --2- 172.21.15.94:0/1103844221 >> [v2:172.21.15.94:6810/2028968832,v1:172.21.15.94:6811/2028968832] conn(0x7f177006ef70 0x7f1770071380 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:56.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.436+0000 7f178b971700 1 -- 172.21.15.94:0/1103844221 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f17700692a0 msgr2=0x7f177006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:56.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.436+0000 7f178b971700 1 --2- 172.21.15.94:0/1103844221 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f17700692a0 0x7f177006b750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1774007fd0 tx=0x7f1774009040 comp rx=0 tx=0).stop 2024-03-24T10:42:56.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.436+0000 7f178b971700 1 -- 172.21.15.94:0/1103844221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1784101d10 msgr2=0x7f1784113e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:56.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.436+0000 7f178b971700 1 --2- 172.21.15.94:0/1103844221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1784101d10 0x7f1784113e40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f178000b3b0 tx=0x7f1780009e30 comp rx=0 tx=0).stop 2024-03-24T10:42:56.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.437+0000 7f178b971700 1 -- 172.21.15.94:0/1103844221 shutdown_connections 2024-03-24T10:42:56.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.437+0000 7f178b971700 1 --2- 172.21.15.94:0/1103844221 >> [v2:172.21.15.94:6810/2028968832,v1:172.21.15.94:6811/2028968832] conn(0x7f177006ef70 0x7f1770071380 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:56.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.437+0000 7f178b971700 1 --2- 172.21.15.94:0/1103844221 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f17700692a0 0x7f177006b750 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:56.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.437+0000 7f178b971700 1 --2- 172.21.15.94:0/1103844221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1784101d10 0x7f1784113e40 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:56.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.437+0000 7f178b971700 1 --2- 172.21.15.94:0/1103844221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17841026b0 0x7f1784114380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:56.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.437+0000 7f178b971700 1 -- 172.21.15.94:0/1103844221 >> 172.21.15.94:0/1103844221 conn(0x7f17840f7880 msgr2=0x7f17840f8f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:42:56.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.437+0000 7f178b971700 1 -- 172.21.15.94:0/1103844221 shutdown_connections 2024-03-24T10:42:56.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:56.437+0000 7f178b971700 1 -- 172.21.15.94:0/1103844221 wait complete. 2024-03-24T10:42:57.424 INFO:teuthology.orchestra.run.smithi094.stdout:111669149715 2024-03-24T10:42:57.425 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-24T10:42:57.793 INFO:teuthology.orchestra.run.smithi094.stdout:60129542175 2024-03-24T10:42:57.793 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-24T10:42:58.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:42:58 smithi096 ceph-mon[38097]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:42:58.476 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:42:58 smithi094 ceph-mon[29446]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:42:58.497 INFO:teuthology.orchestra.run.smithi094.stdout:146028888078 2024-03-24T10:42:58.498 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-24T10:42:58.704 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:42:58.706 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:42:58.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.717+0000 7ff718e51700 1 -- 172.21.15.94:0/3717263350 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff710013b70 con 0x7ff714105990 2024-03-24T10:42:58.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.717+0000 7ff71c0b7700 1 -- 172.21.15.94:0/3717263350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff714105990 msgr2=0x7ff7140ff000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:58.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.717+0000 7ff71c0b7700 1 --2- 172.21.15.94:0/3717263350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff714105990 0x7ff7140ff000 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff710004940 tx=0x7ff710009e30 comp rx=0 tx=0).stop 2024-03-24T10:42:58.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.717+0000 7ff71c0b7700 1 -- 172.21.15.94:0/3717263350 shutdown_connections 2024-03-24T10:42:58.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.717+0000 7ff71c0b7700 1 --2- 172.21.15.94:0/3717263350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff714105990 0x7ff7140ff000 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:58.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.717+0000 7ff71c0b7700 1 --2- 172.21.15.94:0/3717263350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff714104ff0 0x7ff7141053c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:58.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.717+0000 7ff71c0b7700 1 -- 172.21.15.94:0/3717263350 >> 172.21.15.94:0/3717263350 conn(0x7ff7140faba0 msgr2=0x7ff7140fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:42:58.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.717+0000 7ff71c0b7700 1 -- 172.21.15.94:0/3717263350 shutdown_connections 2024-03-24T10:42:58.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.717+0000 7ff71c0b7700 1 -- 172.21.15.94:0/3717263350 wait complete. 2024-03-24T10:42:58.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.718+0000 7ff71c0b7700 1 Processor -- start 2024-03-24T10:42:58.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.718+0000 7ff71c0b7700 1 -- start start 2024-03-24T10:42:58.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.719+0000 7ff71c0b7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff714104ff0 0x7ff714115d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:58.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.719+0000 7ff71c0b7700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff714105990 0x7ff714116260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:58.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.719+0000 7ff71c0b7700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7141168b0 con 0x7ff714105990 2024-03-24T10:42:58.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.719+0000 7ff71c0b7700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff71410fc80 con 0x7ff714104ff0 2024-03-24T10:42:58.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.719+0000 7ff719652700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff714105990 0x7ff714116260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:58.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.719+0000 7ff719e53700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff714104ff0 0x7ff714115d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:58.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.719+0000 7ff719652700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff714105990 0x7ff714116260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41882/0 (socket says 172.21.15.94:41882) 2024-03-24T10:42:58.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.719+0000 7ff719652700 1 -- 172.21.15.94:0/4185187987 learned_addr learned my addr 172.21.15.94:0/4185187987 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:42:58.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.719+0000 7ff719e53700 1 -- 172.21.15.94:0/4185187987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff714105990 msgr2=0x7ff714116260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:58.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.719+0000 7ff719e53700 1 --2- 172.21.15.94:0/4185187987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff714105990 0x7ff714116260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:58.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.719+0000 7ff719e53700 1 -- 172.21.15.94:0/4185187987 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff710005040 con 0x7ff714104ff0 2024-03-24T10:42:58.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.719+0000 7ff719e53700 1 --2- 172.21.15.94:0/4185187987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff714104ff0 0x7ff714115d20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff704009fd0 tx=0x7ff70400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:42:58.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.720+0000 7ff70a7fc700 1 -- 172.21.15.94:0/4185187987 <== mon.1 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff704004e90 con 0x7ff714104ff0 2024-03-24T10:42:58.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.720+0000 7ff71c0b7700 1 -- 172.21.15.94:0/4185187987 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff71410ff00 con 0x7ff714104ff0 2024-03-24T10:42:58.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.720+0000 7ff70a7fc700 1 -- 172.21.15.94:0/4185187987 <== mon.1 v2:172.21.15.96:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff70400f040 con 0x7ff714104ff0 2024-03-24T10:42:58.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.720+0000 7ff71c0b7700 1 -- 172.21.15.94:0/4185187987 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff714110440 con 0x7ff714104ff0 2024-03-24T10:42:58.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.720+0000 7ff70a7fc700 1 -- 172.21.15.94:0/4185187987 <== mon.1 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff70400ca50 con 0x7ff714104ff0 2024-03-24T10:42:58.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.721+0000 7ff71c0b7700 1 -- 172.21.15.94:0/4185187987 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff6f8000fd0 con 0x7ff714104ff0 2024-03-24T10:42:58.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.722+0000 7ff70a7fc700 1 -- 172.21.15.94:0/4185187987 <== mon.1 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7ff70400cbb0 con 0x7ff714104ff0 2024-03-24T10:42:58.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.722+0000 7ff70a7fc700 1 --2- 172.21.15.94:0/4185187987 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7ff7000692a0 0x7ff70006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:58.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.722+0000 7ff70a7fc700 1 -- 172.21.15.94:0/4185187987 <== mon.1 v2:172.21.15.96:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7ff704086780 con 0x7ff714104ff0 2024-03-24T10:42:58.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.722+0000 7ff70a7fc700 1 --2- 172.21.15.94:0/4185187987 >> [v2:172.21.15.96:6816/3738830217,v1:172.21.15.96:6817/3738830217] conn(0x7ff70006ee00 0x7ff700071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:42:58.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.722+0000 7ff719652700 1 --2- 172.21.15.94:0/4185187987 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7ff7000692a0 0x7ff70006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:58.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.722+0000 7ff70a7fc700 1 -- 172.21.15.94:0/4185187987 --> [v2:172.21.15.96:6816/3738830217,v1:172.21.15.96:6817/3738830217] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff7000718c0 con 0x7ff70006ee00 2024-03-24T10:42:58.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.722+0000 7ff70a7fc700 1 -- 172.21.15.94:0/4185187987 <== mon.1 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff70408b220 con 0x7ff714104ff0 2024-03-24T10:42:58.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.723+0000 7ff71a654700 1 --2- 172.21.15.94:0/4185187987 >> [v2:172.21.15.96:6816/3738830217,v1:172.21.15.96:6817/3738830217] conn(0x7ff70006ee00 0x7ff700071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:42:58.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.723+0000 7ff71a654700 1 --2- 172.21.15.94:0/4185187987 >> [v2:172.21.15.96:6816/3738830217,v1:172.21.15.96:6817/3738830217] conn(0x7ff70006ee00 0x7ff700071210 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:42:58.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.723+0000 7ff719652700 1 --2- 172.21.15.94:0/4185187987 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7ff7000692a0 0x7ff70006b750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff710007f50 tx=0x7ff710009e30 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:42:58.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.724+0000 7ff70a7fc700 1 -- 172.21.15.94:0/4185187987 <== osd.5 v2:172.21.15.96:6816/3738830217 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7ff7000718c0 con 0x7ff70006ee00 2024-03-24T10:42:58.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.741+0000 7ff71c0b7700 1 -- 172.21.15.94:0/4185187987 --> [v2:172.21.15.96:6816/3738830217,v1:172.21.15.96:6817/3738830217] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff6f8002d30 con 0x7ff70006ee00 2024-03-24T10:42:58.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.742+0000 7ff70a7fc700 1 -- 172.21.15.94:0/4185187987 <== osd.5 v2:172.21.15.96:6816/3738830217 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ff6f8002d30 con 0x7ff70006ee00 2024-03-24T10:42:58.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.742+0000 7ff71c0b7700 1 -- 172.21.15.94:0/4185187987 >> [v2:172.21.15.96:6816/3738830217,v1:172.21.15.96:6817/3738830217] conn(0x7ff70006ee00 msgr2=0x7ff700071210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:58.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.742+0000 7ff71c0b7700 1 --2- 172.21.15.94:0/4185187987 >> [v2:172.21.15.96:6816/3738830217,v1:172.21.15.96:6817/3738830217] conn(0x7ff70006ee00 0x7ff700071210 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:58.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.742+0000 7ff71c0b7700 1 -- 172.21.15.94:0/4185187987 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7ff7000692a0 msgr2=0x7ff70006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:58.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.742+0000 7ff71c0b7700 1 --2- 172.21.15.94:0/4185187987 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7ff7000692a0 0x7ff70006b750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff710007f50 tx=0x7ff710009e30 comp rx=0 tx=0).stop 2024-03-24T10:42:58.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.742+0000 7ff71c0b7700 1 -- 172.21.15.94:0/4185187987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff714104ff0 msgr2=0x7ff714115d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:42:58.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.742+0000 7ff71c0b7700 1 --2- 172.21.15.94:0/4185187987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff714104ff0 0x7ff714115d20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff704009fd0 tx=0x7ff70400b040 comp rx=0 tx=0).stop 2024-03-24T10:42:58.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.742+0000 7ff71c0b7700 1 -- 172.21.15.94:0/4185187987 shutdown_connections 2024-03-24T10:42:58.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.742+0000 7ff71c0b7700 1 --2- 172.21.15.94:0/4185187987 >> [v2:172.21.15.96:6816/3738830217,v1:172.21.15.96:6817/3738830217] conn(0x7ff70006ee00 0x7ff700071210 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:58.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.742+0000 7ff71c0b7700 1 --2- 172.21.15.94:0/4185187987 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7ff7000692a0 0x7ff70006b750 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:58.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.742+0000 7ff71c0b7700 1 --2- 172.21.15.94:0/4185187987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff714104ff0 0x7ff714115d20 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:58.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.742+0000 7ff71c0b7700 1 --2- 172.21.15.94:0/4185187987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff714105990 0x7ff714116260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:42:58.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.742+0000 7ff71c0b7700 1 -- 172.21.15.94:0/4185187987 >> 172.21.15.94:0/4185187987 conn(0x7ff7140faba0 msgr2=0x7ff7140fbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:42:58.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.743+0000 7ff71c0b7700 1 -- 172.21.15.94:0/4185187987 shutdown_connections 2024-03-24T10:42:58.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:42:58.743+0000 7ff71c0b7700 1 -- 172.21.15.94:0/4185187987 wait complete. 2024-03-24T10:43:00.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:00 smithi094 ceph-mon[29446]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:43:00.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:00 smithi096 ceph-mon[38097]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:43:01.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:01 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:43:01.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:01 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:43:01.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.392+0000 7fa8680a0700 1 -- 172.21.15.94:0/2004763920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa860101d20 msgr2=0x7fa8601020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:01.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.392+0000 7fa8680a0700 1 --2- 172.21.15.94:0/2004763920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa860101d20 0x7fa8601020f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa850004940 tx=0x7fa850009e30 comp rx=0 tx=0).stop 2024-03-24T10:43:01.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.392+0000 7fa8680a0700 1 -- 172.21.15.94:0/2004763920 shutdown_connections 2024-03-24T10:43:01.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.392+0000 7fa8680a0700 1 --2- 172.21.15.94:0/2004763920 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa8601026c0 0x7fa8600fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:01.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.392+0000 7fa8680a0700 1 --2- 172.21.15.94:0/2004763920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa860101d20 0x7fa8601020f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:01.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.392+0000 7fa8680a0700 1 -- 172.21.15.94:0/2004763920 >> 172.21.15.94:0/2004763920 conn(0x7fa8600f7870 msgr2=0x7fa8600f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:43:01.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.392+0000 7fa8680a0700 1 -- 172.21.15.94:0/2004763920 shutdown_connections 2024-03-24T10:43:01.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.392+0000 7fa8680a0700 1 -- 172.21.15.94:0/2004763920 wait complete. 2024-03-24T10:43:01.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.393+0000 7fa8680a0700 1 Processor -- start 2024-03-24T10:43:01.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.393+0000 7fa8680a0700 1 -- start start 2024-03-24T10:43:01.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.393+0000 7fa8680a0700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa860101d20 0x7fa8601139f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:01.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.393+0000 7fa8680a0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8601026c0 0x7fa860113f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:01.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.393+0000 7fa8680a0700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa860114580 con 0x7fa8601026c0 2024-03-24T10:43:01.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.393+0000 7fa8680a0700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86010db80 con 0x7fa860101d20 2024-03-24T10:43:01.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.393+0000 7fa86563b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8601026c0 0x7fa860113f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:01.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.393+0000 7fa86563b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8601026c0 0x7fa860113f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60570/0 (socket says 172.21.15.94:60570) 2024-03-24T10:43:01.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.393+0000 7fa86563b700 1 -- 172.21.15.94:0/922906548 learned_addr learned my addr 172.21.15.94:0/922906548 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:43:01.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.393+0000 7fa865e3c700 1 --2- 172.21.15.94:0/922906548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa860101d20 0x7fa8601139f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:01.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.393+0000 7fa86563b700 1 -- 172.21.15.94:0/922906548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa860101d20 msgr2=0x7fa8601139f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:01.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.393+0000 7fa86563b700 1 --2- 172.21.15.94:0/922906548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa860101d20 0x7fa8601139f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:01.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.393+0000 7fa86563b700 1 -- 172.21.15.94:0/922906548 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa850005040 con 0x7fa8601026c0 2024-03-24T10:43:01.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.394+0000 7fa86563b700 1 --2- 172.21.15.94:0/922906548 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8601026c0 0x7fa860113f30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa85c002a50 tx=0x7fa85c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:43:01.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.394+0000 7fa8567fc700 1 -- 172.21.15.94:0/922906548 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa85c007cc0 con 0x7fa8601026c0 2024-03-24T10:43:01.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.394+0000 7fa8567fc700 1 -- 172.21.15.94:0/922906548 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa85c00f040 con 0x7fa8601026c0 2024-03-24T10:43:01.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.394+0000 7fa8680a0700 1 -- 172.21.15.94:0/922906548 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa86010de60 con 0x7fa8601026c0 2024-03-24T10:43:01.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.394+0000 7fa8567fc700 1 -- 172.21.15.94:0/922906548 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa85c00d9c0 con 0x7fa8601026c0 2024-03-24T10:43:01.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.394+0000 7fa8680a0700 1 -- 172.21.15.94:0/922906548 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa86010e420 con 0x7fa8601026c0 2024-03-24T10:43:01.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.395+0000 7fa8680a0700 1 -- 172.21.15.94:0/922906548 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8600fd470 con 0x7fa8601026c0 2024-03-24T10:43:01.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.395+0000 7fa8567fc700 1 -- 172.21.15.94:0/922906548 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fa85c009730 con 0x7fa8601026c0 2024-03-24T10:43:01.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.395+0000 7fa8567fc700 1 --2- 172.21.15.94:0/922906548 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fa84c0692a0 0x7fa84c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:01.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.396+0000 7fa8567fc700 1 -- 172.21.15.94:0/922906548 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fa85c086bf0 con 0x7fa8601026c0 2024-03-24T10:43:01.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.396+0000 7fa865e3c700 1 --2- 172.21.15.94:0/922906548 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fa84c0692a0 0x7fa84c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:01.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.396+0000 7fa865e3c700 1 --2- 172.21.15.94:0/922906548 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fa84c0692a0 0x7fa84c06b750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa850004f90 tx=0x7fa850009e30 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:43:01.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.400+0000 7fa8567fc700 1 -- 172.21.15.94:0/922906548 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fa85c056c70 con 0x7fa8601026c0 2024-03-24T10:43:01.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.590+0000 7fa8680a0700 1 -- 172.21.15.94:0/922906548 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa860108e20 con 0x7fa8601026c0 2024-03-24T10:43:01.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.590+0000 7fa8567fc700 1 -- 172.21.15.94:0/922906548 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa85c056800 con 0x7fa8601026c0 2024-03-24T10:43:01.591 INFO:teuthology.orchestra.run.smithi094.stdout:38654705700 2024-03-24T10:43:01.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.593+0000 7fa8680a0700 1 -- 172.21.15.94:0/922906548 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fa84c0692a0 msgr2=0x7fa84c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:01.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.593+0000 7fa8680a0700 1 --2- 172.21.15.94:0/922906548 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fa84c0692a0 0x7fa84c06b750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa850004f90 tx=0x7fa850009e30 comp rx=0 tx=0).stop 2024-03-24T10:43:01.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.593+0000 7fa8680a0700 1 -- 172.21.15.94:0/922906548 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8601026c0 msgr2=0x7fa860113f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:01.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.593+0000 7fa8680a0700 1 --2- 172.21.15.94:0/922906548 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8601026c0 0x7fa860113f30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa85c002a50 tx=0x7fa85c004600 comp rx=0 tx=0).stop 2024-03-24T10:43:01.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.593+0000 7fa8680a0700 1 -- 172.21.15.94:0/922906548 shutdown_connections 2024-03-24T10:43:01.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.593+0000 7fa8680a0700 1 --2- 172.21.15.94:0/922906548 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fa84c0692a0 0x7fa84c06b750 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:01.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.593+0000 7fa8680a0700 1 --2- 172.21.15.94:0/922906548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa860101d20 0x7fa8601139f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:01.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.593+0000 7fa8680a0700 1 --2- 172.21.15.94:0/922906548 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8601026c0 0x7fa860113f30 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:01.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.593+0000 7fa8680a0700 1 -- 172.21.15.94:0/922906548 >> 172.21.15.94:0/922906548 conn(0x7fa8600f7870 msgr2=0x7fa8600f8f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:43:01.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.594+0000 7fa8680a0700 1 -- 172.21.15.94:0/922906548 shutdown_connections 2024-03-24T10:43:01.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.594+0000 7fa8680a0700 1 -- 172.21.15.94:0/922906548 wait complete. 2024-03-24T10:43:01.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.811+0000 7ffbe0036700 1 -- 172.21.15.94:0/549272314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffbd8103780 msgr2=0x7ffbd80fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:01.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.811+0000 7ffbe0036700 1 --2- 172.21.15.94:0/549272314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffbd8103780 0x7ffbd80fcdf0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8004940 tx=0x7ffbc8009e30 comp rx=0 tx=0).stop 2024-03-24T10:43:01.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.811+0000 7ffbe0036700 1 -- 172.21.15.94:0/549272314 shutdown_connections 2024-03-24T10:43:01.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.811+0000 7ffbe0036700 1 --2- 172.21.15.94:0/549272314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffbd8103780 0x7ffbd80fcdf0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:01.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.811+0000 7ffbe0036700 1 --2- 172.21.15.94:0/549272314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffbd8102de0 0x7ffbd81031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:01.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.811+0000 7ffbe0036700 1 -- 172.21.15.94:0/549272314 >> 172.21.15.94:0/549272314 conn(0x7ffbd80f8950 msgr2=0x7ffbd80fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:43:01.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.811+0000 7ffbe0036700 1 -- 172.21.15.94:0/549272314 shutdown_connections 2024-03-24T10:43:01.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.811+0000 7ffbe0036700 1 -- 172.21.15.94:0/549272314 wait complete. 2024-03-24T10:43:01.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.812+0000 7ffbe0036700 1 Processor -- start 2024-03-24T10:43:01.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.812+0000 7ffbe0036700 1 -- start start 2024-03-24T10:43:01.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.812+0000 7ffbe0036700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffbd8102de0 0x7ffbd8115c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:01.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.812+0000 7ffbe0036700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffbd8103780 0x7ffbd8116190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:01.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.812+0000 7ffbe0036700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd81167e0 con 0x7ffbd8102de0 2024-03-24T10:43:01.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.812+0000 7ffbe0036700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd810fdc0 con 0x7ffbd8103780 2024-03-24T10:43:01.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.812+0000 7ffbdddd2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffbd8102de0 0x7ffbd8115c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:01.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.812+0000 7ffbdddd2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffbd8102de0 0x7ffbd8115c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60594/0 (socket says 172.21.15.94:60594) 2024-03-24T10:43:01.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.812+0000 7ffbdddd2700 1 -- 172.21.15.94:0/2752152383 learned_addr learned my addr 172.21.15.94:0/2752152383 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:43:01.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.813+0000 7ffbdd5d1700 1 --2- 172.21.15.94:0/2752152383 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffbd8103780 0x7ffbd8116190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:01.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.813+0000 7ffbdddd2700 1 -- 172.21.15.94:0/2752152383 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffbd8103780 msgr2=0x7ffbd8116190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:01.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.813+0000 7ffbdddd2700 1 --2- 172.21.15.94:0/2752152383 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffbd8103780 0x7ffbd8116190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:01.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.813+0000 7ffbdddd2700 1 -- 172.21.15.94:0/2752152383 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc8005040 con 0x7ffbd8102de0 2024-03-24T10:43:01.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.813+0000 7ffbdd5d1700 1 --2- 172.21.15.94:0/2752152383 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffbd8103780 0x7ffbd8116190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T10:43:01.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.813+0000 7ffbdddd2700 1 --2- 172.21.15.94:0/2752152383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffbd8102de0 0x7ffbd8115c50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ffbd400c810 tx=0x7ffbd4004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:43:01.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.813+0000 7ffbce7fc700 1 -- 172.21.15.94:0/2752152383 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffbd4012810 con 0x7ffbd8102de0 2024-03-24T10:43:01.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.813+0000 7ffbce7fc700 1 -- 172.21.15.94:0/2752152383 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffbd4012e40 con 0x7ffbd8102de0 2024-03-24T10:43:01.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.813+0000 7ffbe0036700 1 -- 172.21.15.94:0/2752152383 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbd81100a0 con 0x7ffbd8102de0 2024-03-24T10:43:01.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.813+0000 7ffbce7fc700 1 -- 172.21.15.94:0/2752152383 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffbd400a370 con 0x7ffbd8102de0 2024-03-24T10:43:01.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.813+0000 7ffbe0036700 1 -- 172.21.15.94:0/2752152383 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbd81105e0 con 0x7ffbd8102de0 2024-03-24T10:43:01.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.814+0000 7ffbe0036700 1 -- 172.21.15.94:0/2752152383 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbd8102100 con 0x7ffbd8102de0 2024-03-24T10:43:01.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.815+0000 7ffbce7fc700 1 -- 172.21.15.94:0/2752152383 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7ffbd4011810 con 0x7ffbd8102de0 2024-03-24T10:43:01.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.815+0000 7ffbce7fc700 1 --2- 172.21.15.94:0/2752152383 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7ffbc40692a0 0x7ffbc406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:01.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.815+0000 7ffbce7fc700 1 -- 172.21.15.94:0/2752152383 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7ffbd4087270 con 0x7ffbd8102de0 2024-03-24T10:43:01.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.815+0000 7ffbdd5d1700 1 --2- 172.21.15.94:0/2752152383 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7ffbc40692a0 0x7ffbc406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:01.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.815+0000 7ffbdd5d1700 1 --2- 172.21.15.94:0/2752152383 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7ffbc40692a0 0x7ffbc406b750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8009fd0 tx=0x7ffbc8009e30 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:43:01.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:01.819+0000 7ffbce7fc700 1 -- 172.21.15.94:0/2752152383 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7ffbd4057370 con 0x7ffbd8102de0 2024-03-24T10:43:02.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:02.012+0000 7ffbe0036700 1 -- 172.21.15.94:0/2752152383 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ffbd80fe530 con 0x7ffbd8102de0 2024-03-24T10:43:02.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:02.012+0000 7ffbce7fc700 1 -- 172.21.15.94:0/2752152383 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ffbd80fe530 con 0x7ffbd8102de0 2024-03-24T10:43:02.013 INFO:teuthology.orchestra.run.smithi094.stdout:81604378650 2024-03-24T10:43:02.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:02.014+0000 7ffbe0036700 1 -- 172.21.15.94:0/2752152383 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7ffbc40692a0 msgr2=0x7ffbc406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:02.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:02.014+0000 7ffbe0036700 1 --2- 172.21.15.94:0/2752152383 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7ffbc40692a0 0x7ffbc406b750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8009fd0 tx=0x7ffbc8009e30 comp rx=0 tx=0).stop 2024-03-24T10:43:02.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:02.014+0000 7ffbe0036700 1 -- 172.21.15.94:0/2752152383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffbd8102de0 msgr2=0x7ffbd8115c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:02.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:02.014+0000 7ffbe0036700 1 --2- 172.21.15.94:0/2752152383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffbd8102de0 0x7ffbd8115c50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ffbd400c810 tx=0x7ffbd4004470 comp rx=0 tx=0).stop 2024-03-24T10:43:02.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:02.014+0000 7ffbe0036700 1 -- 172.21.15.94:0/2752152383 shutdown_connections 2024-03-24T10:43:02.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:02.014+0000 7ffbe0036700 1 --2- 172.21.15.94:0/2752152383 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7ffbc40692a0 0x7ffbc406b750 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:02.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:02.014+0000 7ffbe0036700 1 --2- 172.21.15.94:0/2752152383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffbd8102de0 0x7ffbd8115c50 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:02.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:02.014+0000 7ffbe0036700 1 --2- 172.21.15.94:0/2752152383 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffbd8103780 0x7ffbd8116190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:02.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:02.014+0000 7ffbe0036700 1 -- 172.21.15.94:0/2752152383 >> 172.21.15.94:0/2752152383 conn(0x7ffbd80f8950 msgr2=0x7ffbd80fa680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:43:02.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:02.015+0000 7ffbe0036700 1 -- 172.21.15.94:0/2752152383 shutdown_connections 2024-03-24T10:43:02.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:02.015+0000 7ffbe0036700 1 -- 172.21.15.94:0/2752152383 wait complete. 2024-03-24T10:43:02.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:02 smithi096 ceph-mon[38097]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:43:02.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:02 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/922906548' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-24T10:43:02.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:02 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/2752152383' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-24T10:43:02.391 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:02 smithi094 ceph-mon[29446]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:43:02.392 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:02 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/922906548' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-24T10:43:02.392 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:02 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/2752152383' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-24T10:43:02.415 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705698 got 38654705700 for osd.0 2024-03-24T10:43:02.415 DEBUG:teuthology.parallel:result is None 2024-03-24T10:43:02.637 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:43:03.006 INFO:teuthology.orchestra.run.smithi094.stdout:167503724554 2024-03-24T10:43:03.006 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-24T10:43:04.193 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378649 got 81604378650 for osd.2 2024-03-24T10:43:04.193 DEBUG:teuthology.parallel:result is None 2024-03-24T10:43:04.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:04 smithi096 ceph-mon[38097]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:43:04.349 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:04 smithi094 ceph-mon[29446]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:43:04.654 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:43:04.660 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:43:05.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:05.030+0000 7fb1e64bf700 1 -- 172.21.15.94:0/1193382987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1e01037a0 msgr2=0x7fb1e00fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:05.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:05.030+0000 7fb1e64bf700 1 --2- 172.21.15.94:0/1193382987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1e01037a0 0x7fb1e00fce10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4004940 tx=0x7fb1d4009e30 comp rx=0 tx=0).stop 2024-03-24T10:43:05.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:05.030+0000 7fb1e64bf700 1 -- 172.21.15.94:0/1193382987 shutdown_connections 2024-03-24T10:43:05.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:05.030+0000 7fb1e64bf700 1 --2- 172.21.15.94:0/1193382987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1e01037a0 0x7fb1e00fce10 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:05.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:05.030+0000 7fb1e64bf700 1 --2- 172.21.15.94:0/1193382987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1e0102e00 0x7fb1e01031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:05.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:05.030+0000 7fb1e64bf700 1 -- 172.21.15.94:0/1193382987 >> 172.21.15.94:0/1193382987 conn(0x7fb1e00f89b0 msgr2=0x7fb1e00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:43:05.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:05.030+0000 7fb1e64bf700 1 -- 172.21.15.94:0/1193382987 shutdown_connections 2024-03-24T10:43:05.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:05.030+0000 7fb1e64bf700 1 -- 172.21.15.94:0/1193382987 wait complete. 2024-03-24T10:43:05.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:05.031+0000 7fb1e64bf700 1 Processor -- start 2024-03-24T10:43:05.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:05.031+0000 7fb1e64bf700 1 -- start start 2024-03-24T10:43:05.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:05.031+0000 7fb1e64bf700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1e0102e00 0x7fb1e0113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:05.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:05.031+0000 7fb1e64bf700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1e01037a0 0x7fb1e0113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:05.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:05.031+0000 7fb1e64bf700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e01145b0 con 0x7fb1e0102e00 2024-03-24T10:43:05.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:05.031+0000 7fb1e64bf700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e010db40 con 0x7fb1e01037a0 2024-03-24T10:43:05.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:05.032+0000 7fb1dffff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1e0102e00 0x7fb1e0113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:05.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:05.032+0000 7fb1dffff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1e0102e00 0x7fb1e0113990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60606/0 (socket says 172.21.15.94:60606) 2024-03-24T10:43:05.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:05.032+0000 7fb1dffff700 1 -- 172.21.15.94:0/3899254013 learned_addr learned my addr 172.21.15.94:0/3899254013 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:43:05.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:05.032+0000 7fb1df7fe700 1 --2- 172.21.15.94:0/3899254013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1e01037a0 0x7fb1e0113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:05.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:05.032+0000 7fb1dffff700 1 -- 172.21.15.94:0/3899254013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1e01037a0 msgr2=0x7fb1e0113ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:05.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:05.032+0000 7fb1dffff700 1 --2- 172.21.15.94:0/3899254013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1e01037a0 0x7fb1e0113ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:05.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:05.032+0000 7fb1dffff700 1 -- 172.21.15.94:0/3899254013 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d4005040 con 0x7fb1e0102e00 2024-03-24T10:43:05.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:05.032+0000 7fb1df7fe700 1 --2- 172.21.15.94:0/3899254013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1e01037a0 0x7fb1e0113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T10:43:05.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:05.032+0000 7fb1dffff700 1 --2- 172.21.15.94:0/3899254013 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1e0102e00 0x7fb1e0113990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb1d000b790 tx=0x7fb1d0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:43:05.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:05.032+0000 7fb1dcff9700 1 -- 172.21.15.94:0/3899254013 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1d000f770 con 0x7fb1e0102e00 2024-03-24T10:43:05.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:05.032+0000 7fb1dcff9700 1 -- 172.21.15.94:0/3899254013 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb1d000fda0 con 0x7fb1e0102e00 2024-03-24T10:43:05.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:05.032+0000 7fb1e64bf700 1 -- 172.21.15.94:0/3899254013 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1e010de20 con 0x7fb1e0102e00 2024-03-24T10:43:05.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:05.032+0000 7fb1dcff9700 1 -- 172.21.15.94:0/3899254013 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1d000d310 con 0x7fb1e0102e00 2024-03-24T10:43:05.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:05.032+0000 7fb1e64bf700 1 -- 172.21.15.94:0/3899254013 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1e010e3e0 con 0x7fb1e0102e00 2024-03-24T10:43:05.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:05.033+0000 7fb1e64bf700 1 -- 172.21.15.94:0/3899254013 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1e0100110 con 0x7fb1e0102e00 2024-03-24T10:43:05.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:05.034+0000 7fb1dcff9700 1 -- 172.21.15.94:0/3899254013 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fb1d000f8d0 con 0x7fb1e0102e00 2024-03-24T10:43:05.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:05.034+0000 7fb1dcff9700 1 --2- 172.21.15.94:0/3899254013 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fb1c80692a0 0x7fb1c806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:05.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:05.034+0000 7fb1df7fe700 1 --2- 172.21.15.94:0/3899254013 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fb1c80692a0 0x7fb1c806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:05.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:05.034+0000 7fb1dcff9700 1 -- 172.21.15.94:0/3899254013 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fb1d0086dc0 con 0x7fb1e0102e00 2024-03-24T10:43:05.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:05.034+0000 7fb1df7fe700 1 --2- 172.21.15.94:0/3899254013 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fb1c80692a0 0x7fb1c806b750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400b3b0 tx=0x7fb1d4009e30 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:43:05.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:05.038+0000 7fb1dcff9700 1 -- 172.21.15.94:0/3899254013 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fb1d00534d0 con 0x7fb1e0102e00 2024-03-24T10:43:05.054 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:43:05.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:05.237+0000 7fb1e64bf700 1 -- 172.21.15.94:0/3899254013 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fb1e00fe550 con 0x7fb1e0102e00 2024-03-24T10:43:05.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:05.237+0000 7fb1dcff9700 1 -- 172.21.15.94:0/3899254013 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb1d0056af0 con 0x7fb1e0102e00 2024-03-24T10:43:05.238 INFO:teuthology.orchestra.run.smithi094.stdout:146028888080 2024-03-24T10:43:05.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:05.239+0000 7fb1e64bf700 1 -- 172.21.15.94:0/3899254013 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fb1c80692a0 msgr2=0x7fb1c806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:05.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:05.239+0000 7fb1e64bf700 1 --2- 172.21.15.94:0/3899254013 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fb1c80692a0 0x7fb1c806b750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400b3b0 tx=0x7fb1d4009e30 comp rx=0 tx=0).stop 2024-03-24T10:43:05.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:05.239+0000 7fb1e64bf700 1 -- 172.21.15.94:0/3899254013 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1e0102e00 msgr2=0x7fb1e0113990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:05.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:05.239+0000 7fb1e64bf700 1 --2- 172.21.15.94:0/3899254013 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1e0102e00 0x7fb1e0113990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb1d000b790 tx=0x7fb1d0009250 comp rx=0 tx=0).stop 2024-03-24T10:43:05.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:05.239+0000 7fb1e64bf700 1 -- 172.21.15.94:0/3899254013 shutdown_connections 2024-03-24T10:43:05.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:05.239+0000 7fb1e64bf700 1 --2- 172.21.15.94:0/3899254013 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fb1c80692a0 0x7fb1c806b750 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:05.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:05.239+0000 7fb1e64bf700 1 --2- 172.21.15.94:0/3899254013 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1e0102e00 0x7fb1e0113990 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:05.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:05.239+0000 7fb1e64bf700 1 --2- 172.21.15.94:0/3899254013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1e01037a0 0x7fb1e0113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:05.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:05.239+0000 7fb1e64bf700 1 -- 172.21.15.94:0/3899254013 >> 172.21.15.94:0/3899254013 conn(0x7fb1e00f89b0 msgr2=0x7fb1e00fa770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:43:05.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:05.239+0000 7fb1e64bf700 1 -- 172.21.15.94:0/3899254013 shutdown_connections 2024-03-24T10:43:05.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:05.239+0000 7fb1e64bf700 1 -- 172.21.15.94:0/3899254013 wait complete. 2024-03-24T10:43:05.811 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888078 got 146028888080 for osd.4 2024-03-24T10:43:05.811 DEBUG:teuthology.parallel:result is None 2024-03-24T10:43:06.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:06 smithi094 ceph-mon[29446]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:43:06.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:06 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/3899254013' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-24T10:43:06.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:06 smithi096 ceph-mon[38097]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:43:06.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:06 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/3899254013' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-24T10:43:08.430 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:08 smithi094 ceph-mon[29446]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:43:08.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:08.533+0000 7fa87eddc700 1 -- 172.21.15.94:0/4093273972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8780fe830 msgr2=0x7fa878106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:08.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:08.533+0000 7fa87eddc700 1 --2- 172.21.15.94:0/4093273972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8780fe830 0x7fa878106f00 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa864004940 tx=0x7fa864009e30 comp rx=0 tx=0).stop 2024-03-24T10:43:08.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:08.533+0000 7fa87eddc700 1 -- 172.21.15.94:0/4093273972 shutdown_connections 2024-03-24T10:43:08.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:08.533+0000 7fa87eddc700 1 --2- 172.21.15.94:0/4093273972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8780fe830 0x7fa878106f00 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:08.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:08.533+0000 7fa87eddc700 1 --2- 172.21.15.94:0/4093273972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa8780fde90 0x7fa8780fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:08.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:08.533+0000 7fa87eddc700 1 -- 172.21.15.94:0/4093273972 >> 172.21.15.94:0/4093273972 conn(0x7fa878072bc0 msgr2=0x7fa878072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:43:08.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:08.533+0000 7fa87eddc700 1 -- 172.21.15.94:0/4093273972 shutdown_connections 2024-03-24T10:43:08.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:08.533+0000 7fa87eddc700 1 -- 172.21.15.94:0/4093273972 wait complete. 2024-03-24T10:43:08.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:08.533+0000 7fa87eddc700 1 Processor -- start 2024-03-24T10:43:08.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:08.533+0000 7fa87eddc700 1 -- start start 2024-03-24T10:43:08.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:08.534+0000 7fa87eddc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8780fde90 0x7fa878113ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:08.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:08.534+0000 7fa87eddc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa8780fe830 0x7fa8781140e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:08.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:08.534+0000 7fa87eddc700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa878114730 con 0x7fa8780fde90 2024-03-24T10:43:08.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:08.534+0000 7fa87eddc700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87810db20 con 0x7fa8780fe830 2024-03-24T10:43:08.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:08.534+0000 7fa87cb78700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8780fde90 0x7fa878113ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:08.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:08.534+0000 7fa87cb78700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8780fde90 0x7fa878113ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60622/0 (socket says 172.21.15.94:60622) 2024-03-24T10:43:08.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:08.534+0000 7fa87cb78700 1 -- 172.21.15.94:0/2519293172 learned_addr learned my addr 172.21.15.94:0/2519293172 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:43:08.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:08.534+0000 7fa877fff700 1 --2- 172.21.15.94:0/2519293172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa8780fe830 0x7fa8781140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:08.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:08.534+0000 7fa87cb78700 1 -- 172.21.15.94:0/2519293172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa8780fe830 msgr2=0x7fa8781140e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:08.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:08.534+0000 7fa87cb78700 1 --2- 172.21.15.94:0/2519293172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa8780fe830 0x7fa8781140e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:08.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:08.534+0000 7fa87cb78700 1 -- 172.21.15.94:0/2519293172 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa864005040 con 0x7fa8780fde90 2024-03-24T10:43:08.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:08.535+0000 7fa87cb78700 1 --2- 172.21.15.94:0/2519293172 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8780fde90 0x7fa878113ba0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa86c00c830 tx=0x7fa86c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:43:08.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:08.535+0000 7fa8757fa700 1 -- 172.21.15.94:0/2519293172 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa86c00a760 con 0x7fa8780fde90 2024-03-24T10:43:08.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:08.535+0000 7fa87eddc700 1 -- 172.21.15.94:0/2519293172 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa87810de00 con 0x7fa8780fde90 2024-03-24T10:43:08.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:08.535+0000 7fa8757fa700 1 -- 172.21.15.94:0/2519293172 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa86c00ad90 con 0x7fa8780fde90 2024-03-24T10:43:08.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:08.535+0000 7fa8757fa700 1 -- 172.21.15.94:0/2519293172 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa86c0113a0 con 0x7fa8780fde90 2024-03-24T10:43:08.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:08.535+0000 7fa87eddc700 1 -- 172.21.15.94:0/2519293172 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa87810e340 con 0x7fa8780fde90 2024-03-24T10:43:08.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:08.536+0000 7fa87eddc700 1 -- 172.21.15.94:0/2519293172 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa878065e30 con 0x7fa8780fde90 2024-03-24T10:43:08.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:08.537+0000 7fa8757fa700 1 -- 172.21.15.94:0/2519293172 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fa86c011610 con 0x7fa8780fde90 2024-03-24T10:43:08.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:08.537+0000 7fa8757fa700 1 --2- 172.21.15.94:0/2519293172 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fa8680692a0 0x7fa86806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:08.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:08.537+0000 7fa8757fa700 1 -- 172.21.15.94:0/2519293172 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fa86c086ed0 con 0x7fa8780fde90 2024-03-24T10:43:08.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:08.537+0000 7fa877fff700 1 --2- 172.21.15.94:0/2519293172 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fa8680692a0 0x7fa86806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:08.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:08.538+0000 7fa877fff700 1 --2- 172.21.15.94:0/2519293172 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fa8680692a0 0x7fa86806b750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa87810ece0 tx=0x7fa864009e30 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:43:08.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:08.546+0000 7fa8757fa700 1 -- 172.21.15.94:0/2519293172 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fa86c057120 con 0x7fa8780fde90 2024-03-24T10:43:08.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:08 smithi096 ceph-mon[38097]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:43:08.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:08.742+0000 7fa87eddc700 1 -- 172.21.15.94:0/2519293172 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa878109f10 con 0x7fa8780fde90 2024-03-24T10:43:08.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:08.742+0000 7fa8757fa700 1 -- 172.21.15.94:0/2519293172 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa86c056cb0 con 0x7fa8780fde90 2024-03-24T10:43:08.743 INFO:teuthology.orchestra.run.smithi094.stdout:60129542177 2024-03-24T10:43:08.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:08.744+0000 7fa87eddc700 1 -- 172.21.15.94:0/2519293172 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fa8680692a0 msgr2=0x7fa86806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:08.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:08.744+0000 7fa87eddc700 1 --2- 172.21.15.94:0/2519293172 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fa8680692a0 0x7fa86806b750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa87810ece0 tx=0x7fa864009e30 comp rx=0 tx=0).stop 2024-03-24T10:43:08.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:08.744+0000 7fa87eddc700 1 -- 172.21.15.94:0/2519293172 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8780fde90 msgr2=0x7fa878113ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:08.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:08.744+0000 7fa87eddc700 1 --2- 172.21.15.94:0/2519293172 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8780fde90 0x7fa878113ba0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa86c00c830 tx=0x7fa86c004470 comp rx=0 tx=0).stop 2024-03-24T10:43:08.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:08.744+0000 7fa87eddc700 1 -- 172.21.15.94:0/2519293172 shutdown_connections 2024-03-24T10:43:08.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:08.744+0000 7fa87eddc700 1 --2- 172.21.15.94:0/2519293172 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fa8680692a0 0x7fa86806b750 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:08.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:08.744+0000 7fa87eddc700 1 --2- 172.21.15.94:0/2519293172 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8780fde90 0x7fa878113ba0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:08.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:08.744+0000 7fa87eddc700 1 --2- 172.21.15.94:0/2519293172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa8780fe830 0x7fa8781140e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:08.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:08.744+0000 7fa87eddc700 1 -- 172.21.15.94:0/2519293172 >> 172.21.15.94:0/2519293172 conn(0x7fa878072bc0 msgr2=0x7fa8781037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:43:08.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:08.744+0000 7fa87eddc700 1 -- 172.21.15.94:0/2519293172 shutdown_connections 2024-03-24T10:43:08.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:08.744+0000 7fa87eddc700 1 -- 172.21.15.94:0/2519293172 wait complete. 2024-03-24T10:43:09.353 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:09 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/2519293172' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-24T10:43:09.376 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542175 got 60129542177 for osd.1 2024-03-24T10:43:09.377 DEBUG:teuthology.parallel:result is None 2024-03-24T10:43:09.394 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:09 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/2519293172' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-24T10:43:09.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:09.963+0000 7fa9f3645700 1 -- 172.21.15.94:0/2203182618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9ec0fb540 msgr2=0x7fa9ec0fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:09.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:09.963+0000 7fa9f3645700 1 --2- 172.21.15.94:0/2203182618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9ec0fb540 0x7fa9ec0fb990 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc004940 tx=0x7fa9dc009e30 comp rx=0 tx=0).stop 2024-03-24T10:43:09.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:09.963+0000 7fa9f3645700 1 -- 172.21.15.94:0/2203182618 shutdown_connections 2024-03-24T10:43:09.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:09.963+0000 7fa9f3645700 1 --2- 172.21.15.94:0/2203182618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9ec0fb540 0x7fa9ec0fb990 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:09.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:09.963+0000 7fa9f3645700 1 --2- 172.21.15.94:0/2203182618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9ec102fd0 0x7fa9ec1033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:09.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:09.963+0000 7fa9f3645700 1 -- 172.21.15.94:0/2203182618 >> 172.21.15.94:0/2203182618 conn(0x7fa9ec071530 msgr2=0x7fa9ec071930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:43:09.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:09.963+0000 7fa9f3645700 1 -- 172.21.15.94:0/2203182618 shutdown_connections 2024-03-24T10:43:09.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:09.963+0000 7fa9f3645700 1 -- 172.21.15.94:0/2203182618 wait complete. 2024-03-24T10:43:09.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:09.963+0000 7fa9f3645700 1 Processor -- start 2024-03-24T10:43:09.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:09.963+0000 7fa9f3645700 1 -- start start 2024-03-24T10:43:09.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:09.964+0000 7fa9f3645700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9ec0fb540 0x7fa9ec113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:09.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:09.964+0000 7fa9f3645700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9ec102fd0 0x7fa9ec113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:09.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:09.964+0000 7fa9f3645700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ec1145b0 con 0x7fa9ec0fb540 2024-03-24T10:43:09.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:09.964+0000 7fa9f3645700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ec10db40 con 0x7fa9ec102fd0 2024-03-24T10:43:09.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:09.964+0000 7fa9f13e1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9ec0fb540 0x7fa9ec113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:09.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:09.964+0000 7fa9f13e1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9ec0fb540 0x7fa9ec113990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47652/0 (socket says 172.21.15.94:47652) 2024-03-24T10:43:09.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:09.964+0000 7fa9f13e1700 1 -- 172.21.15.94:0/3568196226 learned_addr learned my addr 172.21.15.94:0/3568196226 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:43:09.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:09.964+0000 7fa9f13e1700 1 -- 172.21.15.94:0/3568196226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9ec102fd0 msgr2=0x7fa9ec113ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:09.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:09.964+0000 7fa9f13e1700 1 --2- 172.21.15.94:0/3568196226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9ec102fd0 0x7fa9ec113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:09.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:09.964+0000 7fa9f13e1700 1 -- 172.21.15.94:0/3568196226 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9dc005040 con 0x7fa9ec0fb540 2024-03-24T10:43:09.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:09.964+0000 7fa9f13e1700 1 --2- 172.21.15.94:0/3568196226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9ec0fb540 0x7fa9ec113990 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa9e800b790 tx=0x7fa9e8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:43:09.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:09.964+0000 7fa9e1ffb700 1 -- 172.21.15.94:0/3568196226 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9e800f770 con 0x7fa9ec0fb540 2024-03-24T10:43:09.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:09.964+0000 7fa9e1ffb700 1 -- 172.21.15.94:0/3568196226 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa9e800fda0 con 0x7fa9ec0fb540 2024-03-24T10:43:09.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:09.964+0000 7fa9f3645700 1 -- 172.21.15.94:0/3568196226 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9ec10de20 con 0x7fa9ec0fb540 2024-03-24T10:43:09.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:09.964+0000 7fa9e1ffb700 1 -- 172.21.15.94:0/3568196226 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9e800d310 con 0x7fa9ec0fb540 2024-03-24T10:43:09.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:09.965+0000 7fa9f3645700 1 -- 172.21.15.94:0/3568196226 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9ec10e360 con 0x7fa9ec0fb540 2024-03-24T10:43:09.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:09.965+0000 7fa9f3645700 1 -- 172.21.15.94:0/3568196226 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9ec100270 con 0x7fa9ec0fb540 2024-03-24T10:43:09.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:09.966+0000 7fa9e1ffb700 1 -- 172.21.15.94:0/3568196226 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fa9e8010420 con 0x7fa9ec0fb540 2024-03-24T10:43:09.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:09.966+0000 7fa9e1ffb700 1 --2- 172.21.15.94:0/3568196226 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fa9d80692a0 0x7fa9d806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:09.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:09.966+0000 7fa9e1ffb700 1 -- 172.21.15.94:0/3568196226 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fa9e8086e60 con 0x7fa9ec0fb540 2024-03-24T10:43:09.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:09.966+0000 7fa9f0be0700 1 --2- 172.21.15.94:0/3568196226 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fa9d80692a0 0x7fa9d806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:09.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:09.967+0000 7fa9f0be0700 1 --2- 172.21.15.94:0/3568196226 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fa9d80692a0 0x7fa9d806b750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc004f90 tx=0x7fa9dc009e30 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:43:09.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:09.970+0000 7fa9e1ffb700 1 -- 172.21.15.94:0/3568196226 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fa9e80534d0 con 0x7fa9ec0fb540 2024-03-24T10:43:10.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:10.163+0000 7fa9f3645700 1 -- 172.21.15.94:0/3568196226 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa9ec0fe860 con 0x7fa9ec0fb540 2024-03-24T10:43:10.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:10.163+0000 7fa9e1ffb700 1 -- 172.21.15.94:0/3568196226 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa9e8056af0 con 0x7fa9ec0fb540 2024-03-24T10:43:10.165 INFO:teuthology.orchestra.run.smithi094.stdout:111669149718 2024-03-24T10:43:10.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:10.166+0000 7fa9f3645700 1 -- 172.21.15.94:0/3568196226 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fa9d80692a0 msgr2=0x7fa9d806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:10.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:10.166+0000 7fa9f3645700 1 --2- 172.21.15.94:0/3568196226 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fa9d80692a0 0x7fa9d806b750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc004f90 tx=0x7fa9dc009e30 comp rx=0 tx=0).stop 2024-03-24T10:43:10.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:10.166+0000 7fa9f3645700 1 -- 172.21.15.94:0/3568196226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9ec0fb540 msgr2=0x7fa9ec113990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:10.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:10.166+0000 7fa9f3645700 1 --2- 172.21.15.94:0/3568196226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9ec0fb540 0x7fa9ec113990 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa9e800b790 tx=0x7fa9e8009250 comp rx=0 tx=0).stop 2024-03-24T10:43:10.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:10.166+0000 7fa9f3645700 1 -- 172.21.15.94:0/3568196226 shutdown_connections 2024-03-24T10:43:10.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:10.166+0000 7fa9f3645700 1 --2- 172.21.15.94:0/3568196226 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fa9d80692a0 0x7fa9d806b750 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:10.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:10.166+0000 7fa9f3645700 1 --2- 172.21.15.94:0/3568196226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9ec0fb540 0x7fa9ec113990 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:10.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:10.166+0000 7fa9f3645700 1 --2- 172.21.15.94:0/3568196226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9ec102fd0 0x7fa9ec113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:10.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:10.166+0000 7fa9f3645700 1 -- 172.21.15.94:0/3568196226 >> 172.21.15.94:0/3568196226 conn(0x7fa9ec071530 msgr2=0x7fa9ec0fa580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:43:10.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:10.166+0000 7fa9f3645700 1 -- 172.21.15.94:0/3568196226 shutdown_connections 2024-03-24T10:43:10.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:10.166+0000 7fa9f3645700 1 -- 172.21.15.94:0/3568196226 wait complete. 2024-03-24T10:43:10.486 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:10 smithi094 ceph-mon[29446]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:43:10.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:10 smithi096 ceph-mon[38097]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:43:10.794 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149715 got 111669149718 for osd.3 2024-03-24T10:43:10.794 DEBUG:teuthology.parallel:result is None 2024-03-24T10:43:11.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:11.037+0000 7f00f448c700 1 -- 172.21.15.94:0/884277820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00ec1037a0 msgr2=0x7f00ec0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:11.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:11.037+0000 7f00f448c700 1 --2- 172.21.15.94:0/884277820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00ec1037a0 0x7f00ec0fce10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f00e8004940 tx=0x7f00e8009e30 comp rx=0 tx=0).stop 2024-03-24T10:43:11.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:11.037+0000 7f00f448c700 1 -- 172.21.15.94:0/884277820 shutdown_connections 2024-03-24T10:43:11.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:11.037+0000 7f00f448c700 1 --2- 172.21.15.94:0/884277820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00ec1037a0 0x7f00ec0fce10 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:11.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:11.037+0000 7f00f448c700 1 --2- 172.21.15.94:0/884277820 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f00ec102e00 0x7f00ec1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:11.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:11.037+0000 7f00f448c700 1 -- 172.21.15.94:0/884277820 >> 172.21.15.94:0/884277820 conn(0x7f00ec0f89b0 msgr2=0x7f00ec0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:43:11.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:11.037+0000 7f00f448c700 1 -- 172.21.15.94:0/884277820 shutdown_connections 2024-03-24T10:43:11.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:11.037+0000 7f00f448c700 1 -- 172.21.15.94:0/884277820 wait complete. 2024-03-24T10:43:11.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:11.038+0000 7f00f448c700 1 Processor -- start 2024-03-24T10:43:11.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:11.038+0000 7f00f448c700 1 -- start start 2024-03-24T10:43:11.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:11.038+0000 7f00f448c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f00ec102e00 0x7f00ec113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:11.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:11.038+0000 7f00f448c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00ec1037a0 0x7f00ec113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:11.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:11.038+0000 7f00f448c700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ec1145b0 con 0x7f00ec1037a0 2024-03-24T10:43:11.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:11.038+0000 7f00f448c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ec10db40 con 0x7f00ec102e00 2024-03-24T10:43:11.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:11.038+0000 7f00f1a27700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00ec1037a0 0x7f00ec113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:11.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:11.038+0000 7f00f1a27700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00ec1037a0 0x7f00ec113ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47662/0 (socket says 172.21.15.94:47662) 2024-03-24T10:43:11.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:11.038+0000 7f00f1a27700 1 -- 172.21.15.94:0/3821153073 learned_addr learned my addr 172.21.15.94:0/3821153073 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:43:11.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:11.038+0000 7f00f2228700 1 --2- 172.21.15.94:0/3821153073 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f00ec102e00 0x7f00ec113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:11.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:11.038+0000 7f00f1a27700 1 -- 172.21.15.94:0/3821153073 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f00ec102e00 msgr2=0x7f00ec113990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:11.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:11.038+0000 7f00f1a27700 1 --2- 172.21.15.94:0/3821153073 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f00ec102e00 0x7f00ec113990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:11.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:11.038+0000 7f00f1a27700 1 -- 172.21.15.94:0/3821153073 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00e8005040 con 0x7f00ec1037a0 2024-03-24T10:43:11.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:11.039+0000 7f00f1a27700 1 --2- 172.21.15.94:0/3821153073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00ec1037a0 0x7f00ec113ed0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f00e8009100 tx=0x7f00e8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:43:11.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:11.039+0000 7f00e2ffd700 1 -- 172.21.15.94:0/3821153073 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00e800e040 con 0x7f00ec1037a0 2024-03-24T10:43:11.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:11.039+0000 7f00f448c700 1 -- 172.21.15.94:0/3821153073 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00ec10ddc0 con 0x7f00ec1037a0 2024-03-24T10:43:11.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:11.039+0000 7f00e2ffd700 1 -- 172.21.15.94:0/3821153073 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f00e8009af0 con 0x7f00ec1037a0 2024-03-24T10:43:11.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:11.039+0000 7f00e2ffd700 1 -- 172.21.15.94:0/3821153073 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00e801c3f0 con 0x7f00ec1037a0 2024-03-24T10:43:11.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:11.039+0000 7f00f448c700 1 -- 172.21.15.94:0/3821153073 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00ec10e380 con 0x7f00ec1037a0 2024-03-24T10:43:11.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:11.040+0000 7f00f448c700 1 -- 172.21.15.94:0/3821153073 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00ec100110 con 0x7f00ec1037a0 2024-03-24T10:43:11.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:11.041+0000 7f00e2ffd700 1 -- 172.21.15.94:0/3821153073 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f00e800baf0 con 0x7f00ec1037a0 2024-03-24T10:43:11.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:11.041+0000 7f00e2ffd700 1 --2- 172.21.15.94:0/3821153073 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f00d80692a0 0x7f00d806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:11.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:11.041+0000 7f00e2ffd700 1 -- 172.21.15.94:0/3821153073 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f00e8088550 con 0x7f00ec1037a0 2024-03-24T10:43:11.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:11.041+0000 7f00f2228700 1 --2- 172.21.15.94:0/3821153073 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f00d80692a0 0x7f00d806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:11.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:11.042+0000 7f00f2228700 1 --2- 172.21.15.94:0/3821153073 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f00d80692a0 0x7f00d806b750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f00dc009b60 tx=0x7f00dc0091a0 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:43:11.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:11.045+0000 7f00e2ffd700 1 -- 172.21.15.94:0/3821153073 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f00e8054c60 con 0x7f00ec1037a0 2024-03-24T10:43:11.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:11.229+0000 7f00f448c700 1 -- 172.21.15.94:0/3821153073 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f00ec0fe550 con 0x7f00ec1037a0 2024-03-24T10:43:11.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:11.230+0000 7f00e2ffd700 1 -- 172.21.15.94:0/3821153073 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f00e8058280 con 0x7f00ec1037a0 2024-03-24T10:43:11.231 INFO:teuthology.orchestra.run.smithi094.stdout:167503724557 2024-03-24T10:43:11.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:11.232+0000 7f00f448c700 1 -- 172.21.15.94:0/3821153073 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f00d80692a0 msgr2=0x7f00d806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:11.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:11.232+0000 7f00f448c700 1 --2- 172.21.15.94:0/3821153073 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f00d80692a0 0x7f00d806b750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f00dc009b60 tx=0x7f00dc0091a0 comp rx=0 tx=0).stop 2024-03-24T10:43:11.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:11.232+0000 7f00f448c700 1 -- 172.21.15.94:0/3821153073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00ec1037a0 msgr2=0x7f00ec113ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:11.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:11.232+0000 7f00f448c700 1 --2- 172.21.15.94:0/3821153073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00ec1037a0 0x7f00ec113ed0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f00e8009100 tx=0x7f00e8009e30 comp rx=0 tx=0).stop 2024-03-24T10:43:11.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:11.232+0000 7f00f448c700 1 -- 172.21.15.94:0/3821153073 shutdown_connections 2024-03-24T10:43:11.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:11.232+0000 7f00f448c700 1 --2- 172.21.15.94:0/3821153073 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f00d80692a0 0x7f00d806b750 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:11.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:11.232+0000 7f00f448c700 1 --2- 172.21.15.94:0/3821153073 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f00ec102e00 0x7f00ec113990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:11.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:11.232+0000 7f00f448c700 1 --2- 172.21.15.94:0/3821153073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00ec1037a0 0x7f00ec113ed0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:11.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:11.232+0000 7f00f448c700 1 -- 172.21.15.94:0/3821153073 >> 172.21.15.94:0/3821153073 conn(0x7f00ec0f89b0 msgr2=0x7f00ec0fa770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:43:11.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:11.232+0000 7f00f448c700 1 -- 172.21.15.94:0/3821153073 shutdown_connections 2024-03-24T10:43:11.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:11.232+0000 7f00f448c700 1 -- 172.21.15.94:0/3821153073 wait complete. 2024-03-24T10:43:11.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:11 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/3568196226' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-24T10:43:11.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:11 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/3568196226' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-24T10:43:11.857 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724554 got 167503724557 for osd.5 2024-03-24T10:43:11.857 DEBUG:teuthology.parallel:result is None 2024-03-24T10:43:11.857 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-24T10:43:11.857 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph pg dump --format=json 2024-03-24T10:43:12.112 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:43:12.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:12 smithi094 ceph-mon[29446]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:43:12.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:12 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/3821153073' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-24T10:43:12.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:12 smithi096 ceph-mon[38097]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:43:12.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:12 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/3821153073' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-24T10:43:13.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:13.663+0000 7f74596eb700 1 -- 172.21.15.94:0/2106406854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74540fde10 msgr2=0x7f74540fe1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:13.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:13.663+0000 7f74596eb700 1 --2- 172.21.15.94:0/2106406854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74540fde10 0x7f74540fe1e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7444004940 tx=0x7f7444009e30 comp rx=0 tx=0).stop 2024-03-24T10:43:13.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:13.663+0000 7f74596eb700 1 -- 172.21.15.94:0/2106406854 shutdown_connections 2024-03-24T10:43:13.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:13.663+0000 7f74596eb700 1 --2- 172.21.15.94:0/2106406854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74540fe7b0 0x7f7454106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:13.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:13.663+0000 7f74596eb700 1 --2- 172.21.15.94:0/2106406854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74540fde10 0x7f74540fe1e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:13.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:13.663+0000 7f74596eb700 1 -- 172.21.15.94:0/2106406854 >> 172.21.15.94:0/2106406854 conn(0x7f74540f7880 msgr2=0x7f74540f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:43:13.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:13.664+0000 7f74596eb700 1 -- 172.21.15.94:0/2106406854 shutdown_connections 2024-03-24T10:43:13.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:13.664+0000 7f74596eb700 1 -- 172.21.15.94:0/2106406854 wait complete. 2024-03-24T10:43:13.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:13.664+0000 7f74596eb700 1 Processor -- start 2024-03-24T10:43:13.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:13.664+0000 7f74596eb700 1 -- start start 2024-03-24T10:43:13.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:13.664+0000 7f74596eb700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74540fde10 0x7f7454113e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:13.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:13.664+0000 7f74596eb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74540fe7b0 0x7f7454114380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:13.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:13.664+0000 7f74596eb700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7454114a60 con 0x7f74540fde10 2024-03-24T10:43:13.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:13.664+0000 7f74596eb700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f745410dde0 con 0x7f74540fe7b0 2024-03-24T10:43:13.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:13.665+0000 7f7452ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74540fde10 0x7f7454113e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:13.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:13.665+0000 7f7452ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74540fde10 0x7f7454113e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47678/0 (socket says 172.21.15.94:47678) 2024-03-24T10:43:13.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:13.665+0000 7f7452ffd700 1 -- 172.21.15.94:0/598799561 learned_addr learned my addr 172.21.15.94:0/598799561 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:43:13.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:13.665+0000 7f74527fc700 1 --2- 172.21.15.94:0/598799561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74540fe7b0 0x7f7454114380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:13.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:13.665+0000 7f7452ffd700 1 -- 172.21.15.94:0/598799561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74540fe7b0 msgr2=0x7f7454114380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:13.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:13.665+0000 7f7452ffd700 1 --2- 172.21.15.94:0/598799561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74540fe7b0 0x7f7454114380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:13.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:13.665+0000 7f7452ffd700 1 -- 172.21.15.94:0/598799561 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7444005040 con 0x7f74540fde10 2024-03-24T10:43:13.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:13.665+0000 7f7452ffd700 1 --2- 172.21.15.94:0/598799561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74540fde10 0x7f7454113e40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7444009380 tx=0x7f7444009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:43:13.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:13.665+0000 7f743b7fe700 1 -- 172.21.15.94:0/598799561 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f744400e040 con 0x7f74540fde10 2024-03-24T10:43:13.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:13.665+0000 7f74596eb700 1 -- 172.21.15.94:0/598799561 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f745410e060 con 0x7f74540fde10 2024-03-24T10:43:13.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:13.665+0000 7f743b7fe700 1 -- 172.21.15.94:0/598799561 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7444009af0 con 0x7f74540fde10 2024-03-24T10:43:13.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:13.665+0000 7f743b7fe700 1 -- 172.21.15.94:0/598799561 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f744401c3f0 con 0x7f74540fde10 2024-03-24T10:43:13.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:13.665+0000 7f74596eb700 1 -- 172.21.15.94:0/598799561 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f745410e5a0 con 0x7f74540fde10 2024-03-24T10:43:13.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:13.666+0000 7f74596eb700 1 -- 172.21.15.94:0/598799561 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74541016b0 con 0x7f74540fde10 2024-03-24T10:43:13.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:13.667+0000 7f743b7fe700 1 -- 172.21.15.94:0/598799561 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f7444013420 con 0x7f74540fde10 2024-03-24T10:43:13.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:13.667+0000 7f743b7fe700 1 --2- 172.21.15.94:0/598799561 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f743c0692a0 0x7f743c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:13.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:13.667+0000 7f74527fc700 1 --2- 172.21.15.94:0/598799561 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f743c0692a0 0x7f743c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:13.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:13.667+0000 7f743b7fe700 1 -- 172.21.15.94:0/598799561 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f74440881c0 con 0x7f74540fde10 2024-03-24T10:43:13.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:13.667+0000 7f74527fc700 1 --2- 172.21.15.94:0/598799561 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f743c0692a0 0x7f743c06b750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f745410f720 tx=0x7f74480091a0 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:43:13.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:13.671+0000 7f743b7fe700 1 -- 172.21.15.94:0/598799561 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f7444054860 con 0x7f74540fde10 2024-03-24T10:43:13.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:13.852+0000 7f74596eb700 1 -- 172.21.15.94:0/598799561 --> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f745404bed0 con 0x7f743c0692a0 2024-03-24T10:43:13.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:13.854+0000 7f743b7fe700 1 -- 172.21.15.94:0/598799561 <== mgr.14232 v2:172.21.15.94:6800/2184357820 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27977 (secure 0 0 0) 0x7f745404bed0 con 0x7f743c0692a0 2024-03-24T10:43:13.855 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:43:13.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:13.856+0000 7f74596eb700 1 -- 172.21.15.94:0/598799561 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f743c0692a0 msgr2=0x7f743c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:13.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:13.856+0000 7f74596eb700 1 --2- 172.21.15.94:0/598799561 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f743c0692a0 0x7f743c06b750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f745410f720 tx=0x7f74480091a0 comp rx=0 tx=0).stop 2024-03-24T10:43:13.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:13.856+0000 7f74596eb700 1 -- 172.21.15.94:0/598799561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74540fde10 msgr2=0x7f7454113e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:13.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:13.856+0000 7f74596eb700 1 --2- 172.21.15.94:0/598799561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74540fde10 0x7f7454113e40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7444009380 tx=0x7f7444009e30 comp rx=0 tx=0).stop 2024-03-24T10:43:13.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:13.856+0000 7f74596eb700 1 -- 172.21.15.94:0/598799561 shutdown_connections 2024-03-24T10:43:13.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:13.856+0000 7f74596eb700 1 --2- 172.21.15.94:0/598799561 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f743c0692a0 0x7f743c06b750 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:13.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:13.856+0000 7f74596eb700 1 --2- 172.21.15.94:0/598799561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74540fde10 0x7f7454113e40 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:13.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:13.856+0000 7f74596eb700 1 --2- 172.21.15.94:0/598799561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74540fe7b0 0x7f7454114380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:13.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:13.856+0000 7f74596eb700 1 -- 172.21.15.94:0/598799561 >> 172.21.15.94:0/598799561 conn(0x7f74540f7880 msgr2=0x7f74540ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:43:13.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:13.856+0000 7f74596eb700 1 -- 172.21.15.94:0/598799561 shutdown_connections 2024-03-24T10:43:13.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:13.856+0000 7f74596eb700 1 -- 172.21.15.94:0/598799561 wait complete. 2024-03-24T10:43:13.858 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-03-24T10:43:14.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:14 smithi094 ceph-mon[29446]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:43:14.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:14 smithi096 ceph-mon[38097]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:43:14.604 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":154,"stamp":"2024-03-24T10:43:12.894710+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1787600,"kb_used_data":3872,"kb_used_omap":0,"kb_used_meta":1783680,"kb_avail":560683312,"statfs":{"total":575970213888,"available":574139711488,"internally_reserved":0,"allocated":3964928,"data_stored":2605601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1826488320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.020821"},"pg_stats":[{"pgid":"1.0","version":"23'117","reported_seq":190,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-24T10:42:24.007073+0000","last_change":"2024-03-24T10:41:57.945142+0000","last_active":"2024-03-24T10:42:24.007073+0000","last_peered":"2024-03-24T10:42:24.007073+0000","last_clean":"2024-03-24T10:42:24.007073+0000","last_became_active":"2024-03-24T10:41:57.944820+0000","last_became_peered":"2024-03-24T10:41:57.944820+0000","last_unstale":"2024-03-24T10:42:24.007073+0000","last_undegraded":"2024-03-24T10:42:24.007073+0000","last_fullsized":"2024-03-24T10:42:24.007073+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T10:41:09.335140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T10:41:09.335140+0000","last_clean_scrub_stamp":"2024-03-24T10:41:09.335140+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T16:56:52.835927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447508,"statfs":{"total":95995035648,"available":95690248192,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]}]},{"osd":4,"up_from":34,"seq":146028888081,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447508,"statfs":{"total":95995035648,"available":95690248192,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Mar 24 10:42:58 2024","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":11.426,"5min":11.426,"15min":11.426},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.53400000000000003}]},{"osd":1,"last update":"Sun Mar 24 10:42:58 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.46300000000000002}]},{"osd":2,"last update":"Sun Mar 24 10:42:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.47599999999999998}]},{"osd":3,"last update":"Sun Mar 24 10:42:58 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.32400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34599999999999997}]}]},{"osd":3,"up_from":26,"seq":111669149719,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298224,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446928,"statfs":{"total":95995035648,"available":95689654272,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Mar 24 10:42:35 2024","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.077,"15min":1.077},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":10.772,"5min":10.772,"15min":10.772},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.436}]},{"osd":1,"last update":"Sun Mar 24 10:42:35 2024","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.046,"15min":1.046},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":10.728999999999999,"5min":10.728999999999999,"15min":10.728999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.40899999999999997}]},{"osd":2,"last update":"Sun Mar 24 10:42:35 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.36099999999999999}]},{"osd":4,"last update":"Sun Mar 24 10:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.51500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]}]},{"osd":2,"up_from":19,"seq":81604378652,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297640,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447512,"statfs":{"total":95995035648,"available":95690252288,"internally_reserved":0,"allocated":360448,"data_stored":138532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Mar 24 10:42:09 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.247}]},{"osd":1,"last update":"Sun Mar 24 10:42:09 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.314}]},{"osd":3,"last update":"Sun Mar 24 10:42:32 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.44}]},{"osd":4,"last update":"Sun Mar 24 10:42:58 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.36799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]}]},{"osd":0,"up_from":9,"seq":38654705702,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298224,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446928,"statfs":{"total":95995035648,"available":95689654272,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Mar 24 10:42:45 2024","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.31900000000000001,"15min":0.315},"min":{"1min":0.23100000000000001,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.499,"5min":0.499,"15min":0.499},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.33800000000000002,"15min":0.33400000000000002},"min":{"1min":0.254,"5min":0.186,"15min":0.186},"max":{"1min":0.52100000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.39100000000000001}]},{"osd":2,"last update":"Sun Mar 24 10:42:10 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.433}]},{"osd":3,"last update":"Sun Mar 24 10:42:32 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":9.7699999999999996,"5min":9.7699999999999996,"15min":9.7699999999999996},"last":0.63200000000000001}]},{"osd":4,"last update":"Sun Mar 24 10:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":9.7270000000000003,"5min":9.7270000000000003,"15min":9.7270000000000003},"last":0.65800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542178,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298224,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446928,"statfs":{"total":95995035648,"available":95689654272,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Mar 24 10:42:44 2024","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.32500000000000001,"15min":0.32100000000000001},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.33400000000000002,"15min":0.33100000000000002},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.47799999999999998,"5min":0.498,"15min":0.498},"last":0.315}]},{"osd":2,"last update":"Sun Mar 24 10:42:13 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.27900000000000003}]},{"osd":3,"last update":"Sun Mar 24 10:42:35 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.65000000000000002}]},{"osd":4,"last update":"Sun Mar 24 10:42:58 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.57899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-24T10:43:14.606 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph pg dump --format=json 2024-03-24T10:43:14.858 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:43:15.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:15 smithi094 ceph-mon[29446]: from='client.14440 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-24T10:43:15.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:15 smithi096 ceph-mon[38097]: from='client.14440 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-24T10:43:16.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:16.295+0000 7f17bf745700 1 -- 172.21.15.94:0/1766445302 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17b8101d20 msgr2=0x7f17b81020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:16.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:16.295+0000 7f17bf745700 1 --2- 172.21.15.94:0/1766445302 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17b8101d20 0x7f17b81020f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f17b4004940 tx=0x7f17b4009e30 comp rx=0 tx=0).stop 2024-03-24T10:43:16.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:16.295+0000 7f17bf745700 1 -- 172.21.15.94:0/1766445302 shutdown_connections 2024-03-24T10:43:16.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:16.295+0000 7f17bf745700 1 --2- 172.21.15.94:0/1766445302 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17b81026c0 0x7f17b80fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:16.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:16.295+0000 7f17bf745700 1 --2- 172.21.15.94:0/1766445302 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17b8101d20 0x7f17b81020f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:16.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:16.295+0000 7f17bf745700 1 -- 172.21.15.94:0/1766445302 >> 172.21.15.94:0/1766445302 conn(0x7f17b80f7870 msgr2=0x7f17b80f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:43:16.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:16.295+0000 7f17bf745700 1 -- 172.21.15.94:0/1766445302 shutdown_connections 2024-03-24T10:43:16.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:16.295+0000 7f17bf745700 1 -- 172.21.15.94:0/1766445302 wait complete. 2024-03-24T10:43:16.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:16.295+0000 7f17bf745700 1 Processor -- start 2024-03-24T10:43:16.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:16.295+0000 7f17bf745700 1 -- start start 2024-03-24T10:43:16.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:16.296+0000 7f17bf745700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17b8101d20 0x7f17b810eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:16.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:16.296+0000 7f17bf745700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17b81026c0 0x7f17b810f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:16.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:16.296+0000 7f17bf745700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b810f5c0 con 0x7f17b8101d20 2024-03-24T10:43:16.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:16.296+0000 7f17bf745700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b810f700 con 0x7f17b81026c0 2024-03-24T10:43:16.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:16.296+0000 7f17bd4e1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17b8101d20 0x7f17b810eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:16.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:16.296+0000 7f17bd4e1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17b8101d20 0x7f17b810eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47694/0 (socket says 172.21.15.94:47694) 2024-03-24T10:43:16.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:16.296+0000 7f17bd4e1700 1 -- 172.21.15.94:0/4250167059 learned_addr learned my addr 172.21.15.94:0/4250167059 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:43:16.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:16.296+0000 7f17bcce0700 1 --2- 172.21.15.94:0/4250167059 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17b81026c0 0x7f17b810f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:16.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:16.296+0000 7f17bd4e1700 1 -- 172.21.15.94:0/4250167059 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17b81026c0 msgr2=0x7f17b810f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:16.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:16.296+0000 7f17bd4e1700 1 --2- 172.21.15.94:0/4250167059 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17b81026c0 0x7f17b810f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:16.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:16.296+0000 7f17bd4e1700 1 -- 172.21.15.94:0/4250167059 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b4005040 con 0x7f17b8101d20 2024-03-24T10:43:16.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:16.296+0000 7f17bd4e1700 1 --2- 172.21.15.94:0/4250167059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17b8101d20 0x7f17b810eb40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f17b400b3b0 tx=0x7f17b4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:43:16.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:16.296+0000 7f17adffb700 1 -- 172.21.15.94:0/4250167059 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17b400e040 con 0x7f17b8101d20 2024-03-24T10:43:16.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:16.296+0000 7f17adffb700 1 -- 172.21.15.94:0/4250167059 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f17b400bbc0 con 0x7f17b8101d20 2024-03-24T10:43:16.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:16.296+0000 7f17bf745700 1 -- 172.21.15.94:0/4250167059 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17b810f930 con 0x7f17b8101d20 2024-03-24T10:43:16.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:16.296+0000 7f17adffb700 1 -- 172.21.15.94:0/4250167059 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17b401c3f0 con 0x7f17b8101d20 2024-03-24T10:43:16.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:16.297+0000 7f17bf745700 1 -- 172.21.15.94:0/4250167059 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17b8117f50 con 0x7f17b8101d20 2024-03-24T10:43:16.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:16.297+0000 7f17bf745700 1 -- 172.21.15.94:0/4250167059 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17b80fd470 con 0x7f17b8101d20 2024-03-24T10:43:16.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:16.298+0000 7f17adffb700 1 -- 172.21.15.94:0/4250167059 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f17b400bd30 con 0x7f17b8101d20 2024-03-24T10:43:16.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:16.298+0000 7f17adffb700 1 --2- 172.21.15.94:0/4250167059 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f17a40692a0 0x7f17a406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:16.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:16.298+0000 7f17bcce0700 1 --2- 172.21.15.94:0/4250167059 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f17a40692a0 0x7f17a406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:16.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:16.298+0000 7f17adffb700 1 -- 172.21.15.94:0/4250167059 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f17b40880e0 con 0x7f17b8101d20 2024-03-24T10:43:16.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:16.299+0000 7f17bcce0700 1 --2- 172.21.15.94:0/4250167059 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f17a40692a0 0x7f17a406b750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f17a8006fd0 tx=0x7f17a8008040 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:43:16.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:16.303+0000 7f17adffb700 1 -- 172.21.15.94:0/4250167059 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f17b40546d0 con 0x7f17b8101d20 2024-03-24T10:43:16.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:16.484+0000 7f17bf745700 1 -- 172.21.15.94:0/4250167059 --> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f17b80668f0 con 0x7f17a40692a0 2024-03-24T10:43:16.485 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:16 smithi094 ceph-mon[29446]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:43:16.485 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:16 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:43:16.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:16.486+0000 7f17adffb700 1 -- 172.21.15.94:0/4250167059 <== mgr.14232 v2:172.21.15.94:6800/2184357820 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27922 (secure 0 0 0) 0x7f17b80668f0 con 0x7f17a40692a0 2024-03-24T10:43:16.487 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:43:16.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:16.488+0000 7f17bf745700 1 -- 172.21.15.94:0/4250167059 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f17a40692a0 msgr2=0x7f17a406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:16.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:16.488+0000 7f17bf745700 1 --2- 172.21.15.94:0/4250167059 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f17a40692a0 0x7f17a406b750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f17a8006fd0 tx=0x7f17a8008040 comp rx=0 tx=0).stop 2024-03-24T10:43:16.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:16.488+0000 7f17bf745700 1 -- 172.21.15.94:0/4250167059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17b8101d20 msgr2=0x7f17b810eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:16.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:16.488+0000 7f17bf745700 1 --2- 172.21.15.94:0/4250167059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17b8101d20 0x7f17b810eb40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f17b400b3b0 tx=0x7f17b4009e30 comp rx=0 tx=0).stop 2024-03-24T10:43:16.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:16.488+0000 7f17bf745700 1 -- 172.21.15.94:0/4250167059 shutdown_connections 2024-03-24T10:43:16.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:16.488+0000 7f17bf745700 1 --2- 172.21.15.94:0/4250167059 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f17a40692a0 0x7f17a406b750 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:16.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:16.488+0000 7f17bf745700 1 --2- 172.21.15.94:0/4250167059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17b8101d20 0x7f17b810eb40 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:16.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:16.488+0000 7f17bf745700 1 --2- 172.21.15.94:0/4250167059 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17b81026c0 0x7f17b810f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:16.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:16.488+0000 7f17bf745700 1 -- 172.21.15.94:0/4250167059 >> 172.21.15.94:0/4250167059 conn(0x7f17b80f7870 msgr2=0x7f17b80f8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:43:16.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:16.488+0000 7f17bf745700 1 -- 172.21.15.94:0/4250167059 shutdown_connections 2024-03-24T10:43:16.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:16.488+0000 7f17bf745700 1 -- 172.21.15.94:0/4250167059 wait complete. 2024-03-24T10:43:16.490 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-03-24T10:43:16.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:16 smithi096 ceph-mon[38097]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:43:16.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:16 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:43:17.066 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":155,"stamp":"2024-03-24T10:43:14.895156+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1787600,"kb_used_data":3872,"kb_used_omap":0,"kb_used_meta":1783680,"kb_avail":560683312,"statfs":{"total":575970213888,"available":574139711488,"internally_reserved":0,"allocated":3964928,"data_stored":2605601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1826488320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.020878"},"pg_stats":[{"pgid":"1.0","version":"23'117","reported_seq":190,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-24T10:42:24.007073+0000","last_change":"2024-03-24T10:41:57.945142+0000","last_active":"2024-03-24T10:42:24.007073+0000","last_peered":"2024-03-24T10:42:24.007073+0000","last_clean":"2024-03-24T10:42:24.007073+0000","last_became_active":"2024-03-24T10:41:57.944820+0000","last_became_peered":"2024-03-24T10:41:57.944820+0000","last_unstale":"2024-03-24T10:42:24.007073+0000","last_undegraded":"2024-03-24T10:42:24.007073+0000","last_fullsized":"2024-03-24T10:42:24.007073+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T10:41:09.335140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T10:41:09.335140+0000","last_clean_scrub_stamp":"2024-03-24T10:41:09.335140+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T16:56:52.835927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447508,"statfs":{"total":95995035648,"available":95690248192,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]}]},{"osd":4,"up_from":34,"seq":146028888082,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447508,"statfs":{"total":95995035648,"available":95690248192,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Mar 24 10:42:58 2024","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":11.426,"5min":11.426,"15min":11.426},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.51100000000000001}]},{"osd":1,"last update":"Sun Mar 24 10:42:58 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.495},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.64800000000000002}]},{"osd":2,"last update":"Sun Mar 24 10:42:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.55000000000000004}]},{"osd":3,"last update":"Sun Mar 24 10:42:58 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.53300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]}]},{"osd":3,"up_from":26,"seq":111669149719,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298224,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446928,"statfs":{"total":95995035648,"available":95689654272,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Mar 24 10:42:35 2024","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.077,"15min":1.077},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":10.772,"5min":10.772,"15min":10.772},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.436}]},{"osd":1,"last update":"Sun Mar 24 10:42:35 2024","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.046,"15min":1.046},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":10.728999999999999,"5min":10.728999999999999,"15min":10.728999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.40899999999999997}]},{"osd":2,"last update":"Sun Mar 24 10:42:35 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.36099999999999999}]},{"osd":4,"last update":"Sun Mar 24 10:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.51500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]}]},{"osd":2,"up_from":19,"seq":81604378653,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297640,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447512,"statfs":{"total":95995035648,"available":95690252288,"internally_reserved":0,"allocated":360448,"data_stored":138532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Mar 24 10:43:10 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.373,"15min":0.379},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.496,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.38800000000000001,"15min":0.39600000000000002},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.32400000000000001}]},{"osd":1,"last update":"Sun Mar 24 10:43:10 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.38800000000000001,"15min":0.39600000000000002},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.53000000000000003,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.40600000000000003,"15min":0.41399999999999998},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.53900000000000003,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.34599999999999997}]},{"osd":3,"last update":"Sun Mar 24 10:42:32 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.52500000000000002}]},{"osd":4,"last update":"Sun Mar 24 10:42:58 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.53900000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]}]},{"osd":0,"up_from":9,"seq":38654705702,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298224,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446928,"statfs":{"total":95995035648,"available":95689654272,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Mar 24 10:42:45 2024","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.31900000000000001,"15min":0.315},"min":{"1min":0.23100000000000001,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.499,"5min":0.499,"15min":0.499},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.33800000000000002,"15min":0.33400000000000002},"min":{"1min":0.254,"5min":0.186,"15min":0.186},"max":{"1min":0.52100000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.39100000000000001}]},{"osd":2,"last update":"Sun Mar 24 10:42:10 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.433}]},{"osd":3,"last update":"Sun Mar 24 10:42:32 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":9.7699999999999996,"5min":9.7699999999999996,"15min":9.7699999999999996},"last":0.63200000000000001}]},{"osd":4,"last update":"Sun Mar 24 10:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":9.7270000000000003,"5min":9.7270000000000003,"15min":9.7270000000000003},"last":0.65800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542178,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298224,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446928,"statfs":{"total":95995035648,"available":95689654272,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Mar 24 10:42:44 2024","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.32500000000000001,"15min":0.32100000000000001},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.33400000000000002,"15min":0.33100000000000002},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.47799999999999998,"5min":0.498,"15min":0.498},"last":0.315}]},{"osd":2,"last update":"Sun Mar 24 10:42:13 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.27900000000000003}]},{"osd":3,"last update":"Sun Mar 24 10:42:35 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.65000000000000002}]},{"osd":4,"last update":"Sun Mar 24 10:42:58 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.57899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-24T10:43:17.067 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-24T10:43:17.067 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-24T10:43:17.067 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-24T10:43:17.067 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph health --format=json 2024-03-24T10:43:17.320 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:43:18.325 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:18 smithi094 ceph-mon[29446]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-24T10:43:18.325 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:18 smithi094 ceph-mon[29446]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:43:18.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:18 smithi096 ceph-mon[38097]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-24T10:43:18.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:18 smithi096 ceph-mon[38097]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:43:18.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:18.805+0000 7f9fac357700 1 -- 172.21.15.94:0/2528284518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fa40fb4d0 msgr2=0x7f9fa40fb940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:18.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:18.805+0000 7f9fac357700 1 --2- 172.21.15.94:0/2528284518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fa40fb4d0 0x7f9fa40fb940 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9f94004940 tx=0x7f9f94009e30 comp rx=0 tx=0).stop 2024-03-24T10:43:18.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:18.806+0000 7f9fac357700 1 -- 172.21.15.94:0/2528284518 shutdown_connections 2024-03-24T10:43:18.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:18.806+0000 7f9fac357700 1 --2- 172.21.15.94:0/2528284518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fa40fb4d0 0x7f9fa40fb940 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:18.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:18.806+0000 7f9fac357700 1 --2- 172.21.15.94:0/2528284518 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9fa4102f50 0x7f9fa4103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:18.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:18.806+0000 7f9fac357700 1 -- 172.21.15.94:0/2528284518 >> 172.21.15.94:0/2528284518 conn(0x7f9fa40714c0 msgr2=0x7f9fa40718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:43:18.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:18.806+0000 7f9fac357700 1 -- 172.21.15.94:0/2528284518 shutdown_connections 2024-03-24T10:43:18.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:18.806+0000 7f9fac357700 1 -- 172.21.15.94:0/2528284518 wait complete. 2024-03-24T10:43:18.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:18.806+0000 7f9fac357700 1 Processor -- start 2024-03-24T10:43:18.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:18.806+0000 7f9fac357700 1 -- start start 2024-03-24T10:43:18.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:18.806+0000 7f9fac357700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9fa40fb4d0 0x7f9fa41139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:18.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:18.806+0000 7f9fac357700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fa4102f50 0x7f9fa4113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:18.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:18.806+0000 7f9fac357700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa4114530 con 0x7f9fa4102f50 2024-03-24T10:43:18.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:18.806+0000 7f9fac357700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa410db50 con 0x7f9fa40fb4d0 2024-03-24T10:43:18.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:18.807+0000 7f9fa98f2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fa4102f50 0x7f9fa4113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:18.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:18.807+0000 7f9fa98f2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fa4102f50 0x7f9fa4113ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47704/0 (socket says 172.21.15.94:47704) 2024-03-24T10:43:18.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:18.807+0000 7f9fa98f2700 1 -- 172.21.15.94:0/3296101852 learned_addr learned my addr 172.21.15.94:0/3296101852 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:43:18.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:18.807+0000 7f9faa0f3700 1 --2- 172.21.15.94:0/3296101852 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9fa40fb4d0 0x7f9fa41139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:18.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:18.807+0000 7f9fa98f2700 1 -- 172.21.15.94:0/3296101852 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9fa40fb4d0 msgr2=0x7f9fa41139a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:18.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:18.807+0000 7f9fa98f2700 1 --2- 172.21.15.94:0/3296101852 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9fa40fb4d0 0x7f9fa41139a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:18.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:18.807+0000 7f9fa98f2700 1 -- 172.21.15.94:0/3296101852 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f94005040 con 0x7f9fa4102f50 2024-03-24T10:43:18.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:18.807+0000 7f9fa98f2700 1 --2- 172.21.15.94:0/3296101852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fa4102f50 0x7f9fa4113ee0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9f9400b3b0 tx=0x7f9f94009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:43:18.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:18.807+0000 7f9f9affd700 1 -- 172.21.15.94:0/3296101852 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f9400e040 con 0x7f9fa4102f50 2024-03-24T10:43:18.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:18.807+0000 7f9f9affd700 1 -- 172.21.15.94:0/3296101852 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9f94013420 con 0x7f9fa4102f50 2024-03-24T10:43:18.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:18.807+0000 7f9fac357700 1 -- 172.21.15.94:0/3296101852 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fa410ddd0 con 0x7f9fa4102f50 2024-03-24T10:43:18.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:18.807+0000 7f9f9affd700 1 -- 172.21.15.94:0/3296101852 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f9401c630 con 0x7f9fa4102f50 2024-03-24T10:43:18.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:18.807+0000 7f9fac357700 1 -- 172.21.15.94:0/3296101852 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fa410e310 con 0x7f9fa4102f50 2024-03-24T10:43:18.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:18.808+0000 7f9fac357700 1 -- 172.21.15.94:0/3296101852 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fa4105330 con 0x7f9fa4102f50 2024-03-24T10:43:18.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:18.809+0000 7f9f9affd700 1 -- 172.21.15.94:0/3296101852 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f9f94013590 con 0x7f9fa4102f50 2024-03-24T10:43:18.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:18.809+0000 7f9f9affd700 1 --2- 172.21.15.94:0/3296101852 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f9f900692a0 0x7f9f9006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:18.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:18.809+0000 7f9f9affd700 1 -- 172.21.15.94:0/3296101852 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f9f94088030 con 0x7f9fa4102f50 2024-03-24T10:43:18.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:18.809+0000 7f9faa0f3700 1 --2- 172.21.15.94:0/3296101852 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f9f900692a0 0x7f9f9006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:18.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:18.809+0000 7f9faa0f3700 1 --2- 172.21.15.94:0/3296101852 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f9f900692a0 0x7f9f9006b750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4065bf0 tx=0x7f9fa000b040 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:43:18.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:18.813+0000 7f9f9affd700 1 -- 172.21.15.94:0/3296101852 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f9f940546a0 con 0x7f9fa4102f50 2024-03-24T10:43:19.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:19.043+0000 7f9fac357700 1 -- 172.21.15.94:0/3296101852 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f9fa410eff0 con 0x7f9fa4102f50 2024-03-24T10:43:19.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:19.043+0000 7f9f9affd700 1 -- 172.21.15.94:0/3296101852 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f9f94057cc0 con 0x7f9fa4102f50 2024-03-24T10:43:19.044 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:43:19.044 INFO:teuthology.orchestra.run.smithi094.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-24T10:43:19.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:19.045+0000 7f9fac357700 1 -- 172.21.15.94:0/3296101852 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f9f900692a0 msgr2=0x7f9f9006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:19.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:19.045+0000 7f9fac357700 1 --2- 172.21.15.94:0/3296101852 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f9f900692a0 0x7f9f9006b750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4065bf0 tx=0x7f9fa000b040 comp rx=0 tx=0).stop 2024-03-24T10:43:19.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:19.045+0000 7f9fac357700 1 -- 172.21.15.94:0/3296101852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fa4102f50 msgr2=0x7f9fa4113ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:19.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:19.045+0000 7f9fac357700 1 --2- 172.21.15.94:0/3296101852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fa4102f50 0x7f9fa4113ee0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9f9400b3b0 tx=0x7f9f94009e30 comp rx=0 tx=0).stop 2024-03-24T10:43:19.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:19.045+0000 7f9fac357700 1 -- 172.21.15.94:0/3296101852 shutdown_connections 2024-03-24T10:43:19.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:19.045+0000 7f9fac357700 1 --2- 172.21.15.94:0/3296101852 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f9f900692a0 0x7f9f9006b750 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:19.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:19.045+0000 7f9fac357700 1 --2- 172.21.15.94:0/3296101852 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9fa40fb4d0 0x7f9fa41139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:19.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:19.045+0000 7f9fac357700 1 --2- 172.21.15.94:0/3296101852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fa4102f50 0x7f9fa4113ee0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:19.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:19.045+0000 7f9fac357700 1 -- 172.21.15.94:0/3296101852 >> 172.21.15.94:0/3296101852 conn(0x7f9fa40714c0 msgr2=0x7f9fa40fa720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:43:19.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:19.045+0000 7f9fac357700 1 -- 172.21.15.94:0/3296101852 shutdown_connections 2024-03-24T10:43:19.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:19.045+0000 7f9fac357700 1 -- 172.21.15.94:0/3296101852 wait complete. 2024-03-24T10:43:19.355 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:19 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/3296101852' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-24T10:43:19.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:19 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/3296101852' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-24T10:43:19.631 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-24T10:43:19.631 INFO:tasks.cephadm:Setup complete, yielding 2024-03-24T10:43:19.631 INFO:teuthology.run_tasks:Running task print... 2024-03-24T10:43:19.639 INFO:teuthology.task.print:**** done end installing quincy cephadm ... 2024-03-24T10:43:19.640 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-24T10:43:19.664 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi094.front.sepia.ceph.com 2024-03-24T10:43:19.665 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-03-24T10:43:19.924 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:43:20.153 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:20 smithi094 ceph-mon[29446]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:43:20.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:20 smithi096 ceph-mon[38097]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:43:21.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:21.408+0000 7f31bdc5a700 1 -- 172.21.15.94:0/1924765239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31b80fdfe0 msgr2=0x7f31b80fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:21.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:21.408+0000 7f31bdc5a700 1 --2- 172.21.15.94:0/1924765239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31b80fdfe0 0x7f31b80fe450 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f31ac004940 tx=0x7f31ac009e30 comp rx=0 tx=0).stop 2024-03-24T10:43:21.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:21.408+0000 7f31bdc5a700 1 -- 172.21.15.94:0/1924765239 shutdown_connections 2024-03-24T10:43:21.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:21.408+0000 7f31bdc5a700 1 --2- 172.21.15.94:0/1924765239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31b80fdfe0 0x7f31b80fe450 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:21.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:21.408+0000 7f31bdc5a700 1 --2- 172.21.15.94:0/1924765239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f31b80fce70 0x7f31b80fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:21.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:21.408+0000 7f31bdc5a700 1 -- 172.21.15.94:0/1924765239 >> 172.21.15.94:0/1924765239 conn(0x7f31b80f8550 msgr2=0x7f31b80fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:43:21.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:21.408+0000 7f31bdc5a700 1 -- 172.21.15.94:0/1924765239 shutdown_connections 2024-03-24T10:43:21.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:21.408+0000 7f31bdc5a700 1 -- 172.21.15.94:0/1924765239 wait complete. 2024-03-24T10:43:21.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:21.408+0000 7f31bdc5a700 1 Processor -- start 2024-03-24T10:43:21.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:21.408+0000 7f31bdc5a700 1 -- start start 2024-03-24T10:43:21.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:21.409+0000 7f31bdc5a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f31b80fce70 0x7f31b8071c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:21.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:21.409+0000 7f31bdc5a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31b80fdfe0 0x7f31b8070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:21.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:21.409+0000 7f31bdc5a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b8072260 con 0x7f31b80fdfe0 2024-03-24T10:43:21.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:21.409+0000 7f31bdc5a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b8070850 con 0x7f31b80fce70 2024-03-24T10:43:21.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:21.409+0000 7f31b6ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31b80fdfe0 0x7f31b8070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:21.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:21.409+0000 7f31b6ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31b80fdfe0 0x7f31b8070310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55816/0 (socket says 172.21.15.94:55816) 2024-03-24T10:43:21.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:21.409+0000 7f31b6ffd700 1 -- 172.21.15.94:0/3115358664 learned_addr learned my addr 172.21.15.94:0/3115358664 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:43:21.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:21.409+0000 7f31b77fe700 1 --2- 172.21.15.94:0/3115358664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f31b80fce70 0x7f31b8071c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:21.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:21.409+0000 7f31b6ffd700 1 -- 172.21.15.94:0/3115358664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f31b80fce70 msgr2=0x7f31b8071c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:21.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:21.409+0000 7f31b6ffd700 1 --2- 172.21.15.94:0/3115358664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f31b80fce70 0x7f31b8071c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:21.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:21.409+0000 7f31b6ffd700 1 -- 172.21.15.94:0/3115358664 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31ac005040 con 0x7f31b80fdfe0 2024-03-24T10:43:21.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:21.409+0000 7f31b6ffd700 1 --2- 172.21.15.94:0/3115358664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31b80fdfe0 0x7f31b8070310 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f31ac00b3b0 tx=0x7f31ac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:43:21.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:21.409+0000 7f319ffff700 1 -- 172.21.15.94:0/3115358664 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31ac00e040 con 0x7f31b80fdfe0 2024-03-24T10:43:21.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:21.409+0000 7f31bdc5a700 1 -- 172.21.15.94:0/3115358664 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31b8070aa0 con 0x7f31b80fdfe0 2024-03-24T10:43:21.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:21.409+0000 7f319ffff700 1 -- 172.21.15.94:0/3115358664 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f31ac00bbf0 con 0x7f31b80fdfe0 2024-03-24T10:43:21.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:21.409+0000 7f319ffff700 1 -- 172.21.15.94:0/3115358664 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31ac01c630 con 0x7f31b80fdfe0 2024-03-24T10:43:21.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:21.409+0000 7f31bdc5a700 1 -- 172.21.15.94:0/3115358664 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31b8071060 con 0x7f31b80fdfe0 2024-03-24T10:43:21.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:21.410+0000 7f31bdc5a700 1 -- 172.21.15.94:0/3115358664 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31b804c710 con 0x7f31b80fdfe0 2024-03-24T10:43:21.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:21.411+0000 7f319ffff700 1 -- 172.21.15.94:0/3115358664 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f31ac00bd60 con 0x7f31b80fdfe0 2024-03-24T10:43:21.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:21.411+0000 7f319ffff700 1 --2- 172.21.15.94:0/3115358664 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f31a00692a0 0x7f31a006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:21.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:21.411+0000 7f319ffff700 1 -- 172.21.15.94:0/3115358664 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f31ac087b00 con 0x7f31b80fdfe0 2024-03-24T10:43:21.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:21.411+0000 7f31b77fe700 1 --2- 172.21.15.94:0/3115358664 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f31a00692a0 0x7f31a006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:21.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:21.412+0000 7f31b77fe700 1 --2- 172.21.15.94:0/3115358664 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f31a00692a0 0x7f31a006b750 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f31b0007a40 tx=0x7f31b0009da0 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:43:21.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:21.416+0000 7f319ffff700 1 -- 172.21.15.94:0/3115358664 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f31ac057c00 con 0x7f31b80fdfe0 2024-03-24T10:43:21.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:21.605+0000 7f31bdc5a700 1 -- 172.21.15.94:0/3115358664 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7f31b81058d0 con 0x7f31b80fdfe0 2024-03-24T10:43:21.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:21.609+0000 7f319ffff700 1 -- 172.21.15.94:0/3115358664 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7f31ac057790 con 0x7f31b80fdfe0 2024-03-24T10:43:21.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:21.611+0000 7f31bdc5a700 1 -- 172.21.15.94:0/3115358664 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f31a00692a0 msgr2=0x7f31a006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:21.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:21.611+0000 7f31bdc5a700 1 --2- 172.21.15.94:0/3115358664 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f31a00692a0 0x7f31a006b750 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f31b0007a40 tx=0x7f31b0009da0 comp rx=0 tx=0).stop 2024-03-24T10:43:21.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:21.611+0000 7f31bdc5a700 1 -- 172.21.15.94:0/3115358664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31b80fdfe0 msgr2=0x7f31b8070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:21.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:21.611+0000 7f31bdc5a700 1 --2- 172.21.15.94:0/3115358664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31b80fdfe0 0x7f31b8070310 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f31ac00b3b0 tx=0x7f31ac009e30 comp rx=0 tx=0).stop 2024-03-24T10:43:21.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:21.611+0000 7f31bdc5a700 1 -- 172.21.15.94:0/3115358664 shutdown_connections 2024-03-24T10:43:21.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:21.611+0000 7f31bdc5a700 1 --2- 172.21.15.94:0/3115358664 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f31a00692a0 0x7f31a006b750 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:21.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:21.611+0000 7f31bdc5a700 1 --2- 172.21.15.94:0/3115358664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f31b80fce70 0x7f31b8071c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:21.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:21.611+0000 7f31bdc5a700 1 --2- 172.21.15.94:0/3115358664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31b80fdfe0 0x7f31b8070310 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:21.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:21.611+0000 7f31bdc5a700 1 -- 172.21.15.94:0/3115358664 >> 172.21.15.94:0/3115358664 conn(0x7f31b80f8550 msgr2=0x7f31b80f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:43:21.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:21.611+0000 7f31bdc5a700 1 -- 172.21.15.94:0/3115358664 shutdown_connections 2024-03-24T10:43:21.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:21.611+0000 7f31bdc5a700 1 -- 172.21.15.94:0/3115358664 wait complete. 2024-03-24T10:43:22.207 INFO:teuthology.run_tasks:Running task print... 2024-03-24T10:43:22.216 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-03-24T10:43:22.216 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-24T10:43:22.224 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi094.front.sepia.ceph.com 2024-03-24T10:43:22.224 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- bash -c 'ceph orch status' 2024-03-24T10:43:22.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:22 smithi094 ceph-mon[29446]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:43:22.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:22 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/3115358664' entity='client.admin' 2024-03-24T10:43:22.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:22 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:43:22.499 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:43:22.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:22 smithi096 ceph-mon[38097]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:43:22.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:22 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/3115358664' entity='client.admin' 2024-03-24T10:43:22.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:22 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:43:23.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:23 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:43:23.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:23 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:43:23.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:23 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:43:23.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:23 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:43:23.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:23 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:43:23.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:23 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:43:24.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:24.035+0000 7f04f66f8700 1 -- 172.21.15.94:0/95240361 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04f0066f10 msgr2=0x7f04f00ffa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:24.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:24.035+0000 7f04f66f8700 1 --2- 172.21.15.94:0/95240361 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04f0066f10 0x7f04f00ffa50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f04e0004940 tx=0x7f04e0009e30 comp rx=0 tx=0).stop 2024-03-24T10:43:24.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:24.035+0000 7f04f66f8700 1 -- 172.21.15.94:0/95240361 shutdown_connections 2024-03-24T10:43:24.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:24.035+0000 7f04f66f8700 1 --2- 172.21.15.94:0/95240361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f04f00fff90 0x7f04f0102370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:24.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:24.035+0000 7f04f66f8700 1 --2- 172.21.15.94:0/95240361 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04f0066f10 0x7f04f00ffa50 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:24.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:24.035+0000 7f04f66f8700 1 -- 172.21.15.94:0/95240361 >> 172.21.15.94:0/95240361 conn(0x7f04f00f7480 msgr2=0x7f04f00f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:43:24.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:24.035+0000 7f04f66f8700 1 -- 172.21.15.94:0/95240361 shutdown_connections 2024-03-24T10:43:24.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:24.035+0000 7f04f66f8700 1 -- 172.21.15.94:0/95240361 wait complete. 2024-03-24T10:43:24.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:24.036+0000 7f04f66f8700 1 Processor -- start 2024-03-24T10:43:24.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:24.036+0000 7f04f66f8700 1 -- start start 2024-03-24T10:43:24.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:24.036+0000 7f04f66f8700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04f0066f10 0x7f04f00fd250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:24.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:24.036+0000 7f04f66f8700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f04f00fff90 0x7f04f00fb950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:24.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:24.036+0000 7f04f66f8700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f00fd8e0 con 0x7f04f0066f10 2024-03-24T10:43:24.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:24.036+0000 7f04f66f8700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f00fbec0 con 0x7f04f00fff90 2024-03-24T10:43:24.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:24.036+0000 7f04effff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04f0066f10 0x7f04f00fd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:24.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:24.036+0000 7f04effff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04f0066f10 0x7f04f00fd250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55836/0 (socket says 172.21.15.94:55836) 2024-03-24T10:43:24.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:24.036+0000 7f04effff700 1 -- 172.21.15.94:0/4088317001 learned_addr learned my addr 172.21.15.94:0/4088317001 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:43:24.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:24.037+0000 7f04ef7fe700 1 --2- 172.21.15.94:0/4088317001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f04f00fff90 0x7f04f00fb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:24.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:24.037+0000 7f04effff700 1 -- 172.21.15.94:0/4088317001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f04f00fff90 msgr2=0x7f04f00fb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:24.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:24.037+0000 7f04effff700 1 --2- 172.21.15.94:0/4088317001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f04f00fff90 0x7f04f00fb950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:24.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:24.037+0000 7f04effff700 1 -- 172.21.15.94:0/4088317001 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04e0005040 con 0x7f04f0066f10 2024-03-24T10:43:24.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:24.037+0000 7f04effff700 1 --2- 172.21.15.94:0/4088317001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04f0066f10 0x7f04f00fd250 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f04e000b3b0 tx=0x7f04e0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:43:24.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:24.037+0000 7f04ecff9700 1 -- 172.21.15.94:0/4088317001 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04e000e040 con 0x7f04f0066f10 2024-03-24T10:43:24.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:24.037+0000 7f04ecff9700 1 -- 172.21.15.94:0/4088317001 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f04e000bcc0 con 0x7f04f0066f10 2024-03-24T10:43:24.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:24.037+0000 7f04f66f8700 1 -- 172.21.15.94:0/4088317001 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04f00fc140 con 0x7f04f0066f10 2024-03-24T10:43:24.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:24.037+0000 7f04ecff9700 1 -- 172.21.15.94:0/4088317001 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04e001c630 con 0x7f04f0066f10 2024-03-24T10:43:24.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:24.037+0000 7f04f66f8700 1 -- 172.21.15.94:0/4088317001 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04f00fc700 con 0x7f04f0066f10 2024-03-24T10:43:24.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:24.038+0000 7f04f66f8700 1 -- 172.21.15.94:0/4088317001 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04f004c710 con 0x7f04f0066f10 2024-03-24T10:43:24.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:24.038+0000 7f04ecff9700 1 -- 172.21.15.94:0/4088317001 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f04e00139f0 con 0x7f04f0066f10 2024-03-24T10:43:24.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:24.039+0000 7f04ecff9700 1 --2- 172.21.15.94:0/4088317001 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f04d80692a0 0x7f04d806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:24.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:24.039+0000 7f04ecff9700 1 -- 172.21.15.94:0/4088317001 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f04e0087c60 con 0x7f04f0066f10 2024-03-24T10:43:24.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:24.039+0000 7f04ef7fe700 1 --2- 172.21.15.94:0/4088317001 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f04d80692a0 0x7f04d806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:24.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:24.039+0000 7f04ef7fe700 1 --2- 172.21.15.94:0/4088317001 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f04d80692a0 0x7f04d806b750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f04e4006fd0 tx=0x7f04e4008040 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:43:24.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:24.043+0000 7f04ecff9700 1 -- 172.21.15.94:0/4088317001 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f04e0057d00 con 0x7f04f0066f10 2024-03-24T10:43:24.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:24.241+0000 7f04f66f8700 1 -- 172.21.15.94:0/4088317001 --> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f04f00f8f70 con 0x7f04d80692a0 2024-03-24T10:43:24.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:24.243+0000 7f04ecff9700 1 -- 172.21.15.94:0/4088317001 <== mgr.14232 v2:172.21.15.94:6800/2184357820 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f04f00f8f70 con 0x7f04d80692a0 2024-03-24T10:43:24.243 INFO:teuthology.orchestra.run.smithi094.stdout:Backend: cephadm 2024-03-24T10:43:24.243 INFO:teuthology.orchestra.run.smithi094.stdout:Available: Yes 2024-03-24T10:43:24.244 INFO:teuthology.orchestra.run.smithi094.stdout:Paused: No 2024-03-24T10:43:24.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:24.246+0000 7f04f66f8700 1 -- 172.21.15.94:0/4088317001 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f04d80692a0 msgr2=0x7f04d806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:24.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:24.246+0000 7f04f66f8700 1 --2- 172.21.15.94:0/4088317001 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f04d80692a0 0x7f04d806b750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f04e4006fd0 tx=0x7f04e4008040 comp rx=0 tx=0).stop 2024-03-24T10:43:24.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:24.246+0000 7f04f66f8700 1 -- 172.21.15.94:0/4088317001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04f0066f10 msgr2=0x7f04f00fd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:24.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:24.246+0000 7f04f66f8700 1 --2- 172.21.15.94:0/4088317001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04f0066f10 0x7f04f00fd250 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f04e000b3b0 tx=0x7f04e0009e30 comp rx=0 tx=0).stop 2024-03-24T10:43:24.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:24.246+0000 7f04f66f8700 1 -- 172.21.15.94:0/4088317001 shutdown_connections 2024-03-24T10:43:24.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:24.246+0000 7f04f66f8700 1 --2- 172.21.15.94:0/4088317001 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f04d80692a0 0x7f04d806b750 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:24.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:24.246+0000 7f04f66f8700 1 --2- 172.21.15.94:0/4088317001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04f0066f10 0x7f04f00fd250 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:24.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:24.246+0000 7f04f66f8700 1 --2- 172.21.15.94:0/4088317001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f04f00fff90 0x7f04f00fb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:24.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:24.246+0000 7f04f66f8700 1 -- 172.21.15.94:0/4088317001 >> 172.21.15.94:0/4088317001 conn(0x7f04f00f7480 msgr2=0x7f04f00f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:43:24.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:24.246+0000 7f04f66f8700 1 -- 172.21.15.94:0/4088317001 shutdown_connections 2024-03-24T10:43:24.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:24.246+0000 7f04f66f8700 1 -- 172.21.15.94:0/4088317001 wait complete. 2024-03-24T10:43:24.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:24 smithi094 ceph-mon[29446]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:43:24.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:24 smithi096 ceph-mon[38097]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:43:24.930 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-24T10:43:25.189 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:43:25.624 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:25 smithi094 ceph-mon[29446]: from='client.14456 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:43:25.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:25 smithi096 ceph-mon[38097]: from='client.14456 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:43:26.554 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:26 smithi094 ceph-mon[29446]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:43:26.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:26.777+0000 7fe5cf965700 1 -- 172.21.15.94:0/2472601245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5c80fbce0 msgr2=0x7fe5c80fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:26.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:26.777+0000 7fe5cf965700 1 --2- 172.21.15.94:0/2472601245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5c80fbce0 0x7fe5c80fc0f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4009ce0 tx=0x7fe5c4004d10 comp rx=0 tx=0).stop 2024-03-24T10:43:26.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:26.777+0000 7fe5cf965700 1 -- 172.21.15.94:0/2472601245 shutdown_connections 2024-03-24T10:43:26.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:26.777+0000 7fe5cf965700 1 --2- 172.21.15.94:0/2472601245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5c80fcf90 0x7fe5c80fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:26.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:26.777+0000 7fe5cf965700 1 --2- 172.21.15.94:0/2472601245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5c80fbce0 0x7fe5c80fc0f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:26.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:26.777+0000 7fe5cf965700 1 -- 172.21.15.94:0/2472601245 >> 172.21.15.94:0/2472601245 conn(0x7fe5c80f7440 msgr2=0x7fe5c80f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:43:26.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:26.777+0000 7fe5cf965700 1 -- 172.21.15.94:0/2472601245 shutdown_connections 2024-03-24T10:43:26.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:26.777+0000 7fe5cf965700 1 -- 172.21.15.94:0/2472601245 wait complete. 2024-03-24T10:43:26.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:26.777+0000 7fe5cf965700 1 Processor -- start 2024-03-24T10:43:26.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:26.777+0000 7fe5cf965700 1 -- start start 2024-03-24T10:43:26.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:26.778+0000 7fe5cf965700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5c80fbce0 0x7fe5c810dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:26.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:26.778+0000 7fe5cf965700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5c80fcf90 0x7fe5c810e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:26.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:26.778+0000 7fe5cf965700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c810e800 con 0x7fe5c80fcf90 2024-03-24T10:43:26.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:26.778+0000 7fe5cf965700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c810ef50 con 0x7fe5c80fbce0 2024-03-24T10:43:26.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:26.778+0000 7fe5ccf00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5c80fcf90 0x7fe5c810e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:26.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:26.778+0000 7fe5ccf00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5c80fcf90 0x7fe5c810e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55868/0 (socket says 172.21.15.94:55868) 2024-03-24T10:43:26.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:26.778+0000 7fe5ccf00700 1 -- 172.21.15.94:0/2263416545 learned_addr learned my addr 172.21.15.94:0/2263416545 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:43:26.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:26.778+0000 7fe5ccf00700 1 -- 172.21.15.94:0/2263416545 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5c80fbce0 msgr2=0x7fe5c810dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:26.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:26.778+0000 7fe5ccf00700 1 --2- 172.21.15.94:0/2263416545 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5c80fbce0 0x7fe5c810dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:26.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:26.778+0000 7fe5ccf00700 1 -- 172.21.15.94:0/2263416545 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c4009940 con 0x7fe5c80fcf90 2024-03-24T10:43:26.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:26.778+0000 7fe5ccf00700 1 --2- 172.21.15.94:0/2263416545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5c80fcf90 0x7fe5c810e170 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe5b800b790 tx=0x7fe5b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:43:26.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:26.778+0000 7fe5bdffb700 1 -- 172.21.15.94:0/2263416545 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5b800f770 con 0x7fe5c80fcf90 2024-03-24T10:43:26.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:26.778+0000 7fe5bdffb700 1 -- 172.21.15.94:0/2263416545 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe5b800fda0 con 0x7fe5c80fcf90 2024-03-24T10:43:26.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:26.778+0000 7fe5bdffb700 1 -- 172.21.15.94:0/2263416545 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5b800d310 con 0x7fe5c80fcf90 2024-03-24T10:43:26.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:26.778+0000 7fe5cf965700 1 -- 172.21.15.94:0/2263416545 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5c810f230 con 0x7fe5c80fcf90 2024-03-24T10:43:26.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:26.778+0000 7fe5cf965700 1 -- 172.21.15.94:0/2263416545 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5c810f7f0 con 0x7fe5c80fcf90 2024-03-24T10:43:26.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:26.780+0000 7fe5cf965700 1 -- 172.21.15.94:0/2263416545 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5c8101b20 con 0x7fe5c80fcf90 2024-03-24T10:43:26.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:26.780+0000 7fe5bdffb700 1 -- 172.21.15.94:0/2263416545 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fe5b8010420 con 0x7fe5c80fcf90 2024-03-24T10:43:26.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:26.780+0000 7fe5bdffb700 1 --2- 172.21.15.94:0/2263416545 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fe5b40692a0 0x7fe5b406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:26.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:26.780+0000 7fe5bdffb700 1 -- 172.21.15.94:0/2263416545 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fe5b8086d60 con 0x7fe5c80fcf90 2024-03-24T10:43:26.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:26.780+0000 7fe5cd701700 1 --2- 172.21.15.94:0/2263416545 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fe5b40692a0 0x7fe5b406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:26.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:26.781+0000 7fe5cd701700 1 --2- 172.21.15.94:0/2263416545 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fe5b40692a0 0x7fe5b406b750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4000f80 tx=0x7fe5c4004d10 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:43:26.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:26.785+0000 7fe5bdffb700 1 -- 172.21.15.94:0/2263416545 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fe5b8056f00 con 0x7fe5c80fcf90 2024-03-24T10:43:26.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:26 smithi096 ceph-mon[38097]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:43:26.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:26.978+0000 7fe5cf965700 1 -- 172.21.15.94:0/2263416545 --> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe5c810fac0 con 0x7fe5b40692a0 2024-03-24T10:43:26.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:26.986+0000 7fe5bdffb700 1 -- 172.21.15.94:0/2263416545 <== mgr.14232 v2:172.21.15.94:6800/2184357820 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2682 (secure 0 0 0) 0x7fe5c810fac0 con 0x7fe5b40692a0 2024-03-24T10:43:26.987 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T10:43:26.987 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.smithi094 smithi094 *:9093,9094 running (3m) 2m ago 5m 26.3M - 0.25.0 c8568f914cd2 8072d4b65986 2024-03-24T10:43:26.987 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi094 smithi094 running (5m) 2m ago 5m 6660k - 17.2.7-726-g9c4e0c34 837124da804f fb5772af46c5 2024-03-24T10:43:26.987 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi096 smithi096 running (4m) 64s ago 4m 6669k - 17.2.7-726-g9c4e0c34 837124da804f 6087ad4e0734 2024-03-24T10:43:26.987 INFO:teuthology.orchestra.run.smithi094.stdout:grafana.smithi094 smithi094 *:3000 running (3m) 2m ago 4m 77.3M - 9.4.7 954c08fa6188 7b534968fb36 2024-03-24T10:43:26.988 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi094.yaethu smithi094 *:9283 running (6m) 2m ago 6m 473M - 17.2.7-726-g9c4e0c34 837124da804f 327d527697e0 2024-03-24T10:43:26.988 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi096.gfonnu smithi096 *:8443,9283 running (4m) 64s ago 4m 432M - 17.2.7-726-g9c4e0c34 837124da804f efd3f3b01cb6 2024-03-24T10:43:26.988 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi094 smithi094 running (7m) 2m ago 7m 44.0M 2048M 17.2.7-726-g9c4e0c34 837124da804f 76584fceb32b 2024-03-24T10:43:26.988 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi096 smithi096 running (4m) 64s ago 4m 38.6M 2048M 17.2.7-726-g9c4e0c34 837124da804f 710030b222cb 2024-03-24T10:43:26.988 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi094 smithi094 *:9100 running (5m) 2m ago 5m 16.3M - 1.5.0 0da6a335fe13 59895363f7e6 2024-03-24T10:43:26.988 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi096 smithi096 *:9100 running (4m) 64s ago 4m 15.2M - 1.5.0 0da6a335fe13 fc629282835f 2024-03-24T10:43:26.988 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (3m) 2m ago 3m 41.2M 4288M 17.2.7-726-g9c4e0c34 837124da804f b664fd3557bd 2024-03-24T10:43:26.988 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi094 running (2m) 2m ago 2m 39.7M 4288M 17.2.7-726-g9c4e0c34 837124da804f 4e737a80d733 2024-03-24T10:43:26.988 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi094 running (2m) 2m ago 2m 43.4M 4288M 17.2.7-726-g9c4e0c34 837124da804f cf0aefd20c76 2024-03-24T10:43:26.988 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi096 running (2m) 64s ago 2m 39.3M 5312M 17.2.7-726-g9c4e0c34 837124da804f f4650d0f6241 2024-03-24T10:43:26.988 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi096 running (95s) 64s ago 95s 38.9M 5312M 17.2.7-726-g9c4e0c34 837124da804f a63d7c9e705f 2024-03-24T10:43:26.988 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi096 running (69s) 64s ago 69s 10.5M 5312M 17.2.7-726-g9c4e0c34 837124da804f f56fad2959c3 2024-03-24T10:43:26.988 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.smithi094 smithi094 *:9095 running (3m) 2m ago 4m 37.0M - 2.43.0 a07b618ecd1d 023eb0a49108 2024-03-24T10:43:26.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:26.988+0000 7fe5cf965700 1 -- 172.21.15.94:0/2263416545 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fe5b40692a0 msgr2=0x7fe5b406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:26.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:26.988+0000 7fe5cf965700 1 --2- 172.21.15.94:0/2263416545 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fe5b40692a0 0x7fe5b406b750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4000f80 tx=0x7fe5c4004d10 comp rx=0 tx=0).stop 2024-03-24T10:43:26.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:26.988+0000 7fe5cf965700 1 -- 172.21.15.94:0/2263416545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5c80fcf90 msgr2=0x7fe5c810e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:26.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:26.988+0000 7fe5cf965700 1 --2- 172.21.15.94:0/2263416545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5c80fcf90 0x7fe5c810e170 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe5b800b790 tx=0x7fe5b8009250 comp rx=0 tx=0).stop 2024-03-24T10:43:26.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:26.988+0000 7fe5cf965700 1 -- 172.21.15.94:0/2263416545 shutdown_connections 2024-03-24T10:43:26.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:26.988+0000 7fe5cf965700 1 --2- 172.21.15.94:0/2263416545 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fe5b40692a0 0x7fe5b406b750 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:26.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:26.988+0000 7fe5cf965700 1 --2- 172.21.15.94:0/2263416545 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5c80fbce0 0x7fe5c810dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:26.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:26.988+0000 7fe5cf965700 1 --2- 172.21.15.94:0/2263416545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5c80fcf90 0x7fe5c810e170 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:26.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:26.988+0000 7fe5cf965700 1 -- 172.21.15.94:0/2263416545 >> 172.21.15.94:0/2263416545 conn(0x7fe5c80f7440 msgr2=0x7fe5c8066370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:43:26.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:26.989+0000 7fe5cf965700 1 -- 172.21.15.94:0/2263416545 shutdown_connections 2024-03-24T10:43:26.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:26.989+0000 7fe5cf965700 1 -- 172.21.15.94:0/2263416545 wait complete. 2024-03-24T10:43:27.601 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-24T10:43:27.856 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:43:28.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:28 smithi094 ceph-mon[29446]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:43:28.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:28 smithi094 ceph-mon[29446]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:43:28.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:28 smithi096 ceph-mon[38097]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:43:28.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:28 smithi096 ceph-mon[38097]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:43:29.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:29.364+0000 7f36a73dd700 1 -- 172.21.15.94:0/3971478252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36a00fbcd0 msgr2=0x7f36a00fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:29.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:29.364+0000 7f36a73dd700 1 --2- 172.21.15.94:0/3971478252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36a00fbcd0 0x7f36a00fc0e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f369c004940 tx=0x7f369c009e30 comp rx=0 tx=0).stop 2024-03-24T10:43:29.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:29.365+0000 7f36a73dd700 1 -- 172.21.15.94:0/3971478252 shutdown_connections 2024-03-24T10:43:29.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:29.365+0000 7f36a73dd700 1 --2- 172.21.15.94:0/3971478252 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36a00fcf80 0x7f36a00fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:29.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:29.365+0000 7f36a73dd700 1 --2- 172.21.15.94:0/3971478252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36a00fbcd0 0x7f36a00fc0e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:29.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:29.365+0000 7f36a73dd700 1 -- 172.21.15.94:0/3971478252 >> 172.21.15.94:0/3971478252 conn(0x7f36a00f7430 msgr2=0x7f36a00f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:43:29.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:29.365+0000 7f36a73dd700 1 -- 172.21.15.94:0/3971478252 shutdown_connections 2024-03-24T10:43:29.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:29.365+0000 7f36a73dd700 1 -- 172.21.15.94:0/3971478252 wait complete. 2024-03-24T10:43:29.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:29.365+0000 7f36a73dd700 1 Processor -- start 2024-03-24T10:43:29.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:29.365+0000 7f36a73dd700 1 -- start start 2024-03-24T10:43:29.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:29.365+0000 7f36a73dd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36a00fbcd0 0x7f36a010c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:29.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:29.365+0000 7f36a73dd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36a00fcf80 0x7f36a010cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:29.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:29.365+0000 7f36a73dd700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a010d400 con 0x7f36a00fcf80 2024-03-24T10:43:29.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:29.365+0000 7f36a73dd700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a0111da0 con 0x7f36a00fbcd0 2024-03-24T10:43:29.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:29.365+0000 7f36a4978700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36a00fcf80 0x7f36a010cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:29.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:29.365+0000 7f36a4978700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36a00fcf80 0x7f36a010cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55888/0 (socket says 172.21.15.94:55888) 2024-03-24T10:43:29.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:29.365+0000 7f36a4978700 1 -- 172.21.15.94:0/270075069 learned_addr learned my addr 172.21.15.94:0/270075069 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:43:29.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:29.365+0000 7f36a5179700 1 --2- 172.21.15.94:0/270075069 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36a00fbcd0 0x7f36a010c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:29.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:29.366+0000 7f36a4978700 1 -- 172.21.15.94:0/270075069 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36a00fbcd0 msgr2=0x7f36a010c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:29.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:29.366+0000 7f36a4978700 1 --2- 172.21.15.94:0/270075069 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36a00fbcd0 0x7f36a010c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:29.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:29.366+0000 7f36a4978700 1 -- 172.21.15.94:0/270075069 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f369c005040 con 0x7f36a00fcf80 2024-03-24T10:43:29.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:29.366+0000 7f36a4978700 1 --2- 172.21.15.94:0/270075069 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36a00fcf80 0x7f36a010cd70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f369000b790 tx=0x7f3690009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:43:29.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:29.366+0000 7f3695ffb700 1 -- 172.21.15.94:0/270075069 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f369000f6f0 con 0x7f36a00fcf80 2024-03-24T10:43:29.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:29.366+0000 7f36a73dd700 1 -- 172.21.15.94:0/270075069 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36a0112000 con 0x7f36a00fcf80 2024-03-24T10:43:29.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:29.366+0000 7f3695ffb700 1 -- 172.21.15.94:0/270075069 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f369000fd20 con 0x7f36a00fcf80 2024-03-24T10:43:29.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:29.366+0000 7f3695ffb700 1 -- 172.21.15.94:0/270075069 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3690011400 con 0x7f36a00fcf80 2024-03-24T10:43:29.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:29.366+0000 7f36a73dd700 1 -- 172.21.15.94:0/270075069 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36a0112540 con 0x7f36a00fcf80 2024-03-24T10:43:29.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:29.367+0000 7f36a73dd700 1 -- 172.21.15.94:0/270075069 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36a004c710 con 0x7f36a00fcf80 2024-03-24T10:43:29.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:29.367+0000 7f3695ffb700 1 -- 172.21.15.94:0/270075069 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f369001a400 con 0x7f36a00fcf80 2024-03-24T10:43:29.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:29.368+0000 7f3695ffb700 1 --2- 172.21.15.94:0/270075069 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f368c0692a0 0x7f368c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:29.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:29.368+0000 7f3695ffb700 1 -- 172.21.15.94:0/270075069 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f3690086b00 con 0x7f36a00fcf80 2024-03-24T10:43:29.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:29.368+0000 7f36a5179700 1 --2- 172.21.15.94:0/270075069 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f368c0692a0 0x7f368c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:29.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:29.368+0000 7f36a5179700 1 --2- 172.21.15.94:0/270075069 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f368c0692a0 0x7f368c06b750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f369c00b970 tx=0x7f369c009e30 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:43:29.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:29.372+0000 7f3695ffb700 1 -- 172.21.15.94:0/270075069 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f3690056ca0 con 0x7f36a00fcf80 2024-03-24T10:43:29.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:29.567+0000 7f36a73dd700 1 -- 172.21.15.94:0/270075069 --> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f36a0101220 con 0x7f368c0692a0 2024-03-24T10:43:29.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:29.571+0000 7f3695ffb700 1 -- 172.21.15.94:0/270075069 <== mgr.14232 v2:172.21.15.94:6800/2184357820 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1161 (secure 0 0 0) 0x7f36a0101220 con 0x7f368c0692a0 2024-03-24T10:43:29.572 INFO:teuthology.orchestra.run.smithi094.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-24T10:43:29.572 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager ?:9093,9094 1/1 2m ago 6m count:1 2024-03-24T10:43:29.572 INFO:teuthology.orchestra.run.smithi094.stdout:crash 2/2 2m ago 6m * 2024-03-24T10:43:29.572 INFO:teuthology.orchestra.run.smithi094.stdout:grafana ?:3000 1/1 2m ago 6m count:1 2024-03-24T10:43:29.573 INFO:teuthology.orchestra.run.smithi094.stdout:mgr 2/2 2m ago 6m count:2 2024-03-24T10:43:29.573 INFO:teuthology.orchestra.run.smithi094.stdout:mon 2/2 2m ago 5m smithi094:172.21.15.94=smithi094;smithi096:172.21.15.96=smithi096;count:2 2024-03-24T10:43:29.573 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter ?:9100 2/2 2m ago 6m * 2024-03-24T10:43:29.573 INFO:teuthology.orchestra.run.smithi094.stdout:osd 6 2m ago - 2024-03-24T10:43:29.573 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus ?:9095 1/1 2m ago 6m count:1 2024-03-24T10:43:29.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:29.573+0000 7f36a73dd700 1 -- 172.21.15.94:0/270075069 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f368c0692a0 msgr2=0x7f368c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:29.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:29.573+0000 7f36a73dd700 1 --2- 172.21.15.94:0/270075069 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f368c0692a0 0x7f368c06b750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f369c00b970 tx=0x7f369c009e30 comp rx=0 tx=0).stop 2024-03-24T10:43:29.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:29.573+0000 7f36a73dd700 1 -- 172.21.15.94:0/270075069 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36a00fcf80 msgr2=0x7f36a010cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:29.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:29.573+0000 7f36a73dd700 1 --2- 172.21.15.94:0/270075069 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36a00fcf80 0x7f36a010cd70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f369000b790 tx=0x7f3690009250 comp rx=0 tx=0).stop 2024-03-24T10:43:29.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:29.573+0000 7f36a73dd700 1 -- 172.21.15.94:0/270075069 shutdown_connections 2024-03-24T10:43:29.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:29.573+0000 7f36a73dd700 1 --2- 172.21.15.94:0/270075069 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f368c0692a0 0x7f368c06b750 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:29.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:29.573+0000 7f36a73dd700 1 --2- 172.21.15.94:0/270075069 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36a00fbcd0 0x7f36a010c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:29.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:29.573+0000 7f36a73dd700 1 --2- 172.21.15.94:0/270075069 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36a00fcf80 0x7f36a010cd70 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:29.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:29.573+0000 7f36a73dd700 1 -- 172.21.15.94:0/270075069 >> 172.21.15.94:0/270075069 conn(0x7f36a00f7430 msgr2=0x7f36a00f8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:43:29.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:29.574+0000 7f36a73dd700 1 -- 172.21.15.94:0/270075069 shutdown_connections 2024-03-24T10:43:29.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:29.574+0000 7f36a73dd700 1 -- 172.21.15.94:0/270075069 wait complete. 2024-03-24T10:43:30.163 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-24T10:43:30.418 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:43:30.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:30 smithi094 ceph-mon[29446]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:43:30.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:30 smithi096 ceph-mon[38097]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:43:31.495 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:31 smithi094 ceph-mon[29446]: from='client.14464 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:43:31.495 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:31 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:43:31.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:31 smithi096 ceph-mon[38097]: from='client.14464 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:43:31.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:31 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:43:31.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:31.995+0000 7f1741264700 1 -- 172.21.15.94:0/2961416036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f173c0fff00 msgr2=0x7f173c102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:31.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:31.995+0000 7f1741264700 1 --2- 172.21.15.94:0/2961416036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f173c0fff00 0x7f173c102320 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1724004940 tx=0x7f1724009e30 comp rx=0 tx=0).stop 2024-03-24T10:43:31.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:31.995+0000 7f1741264700 1 -- 172.21.15.94:0/2961416036 shutdown_connections 2024-03-24T10:43:31.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:31.995+0000 7f1741264700 1 --2- 172.21.15.94:0/2961416036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f173c0fff00 0x7f173c102320 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:31.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:31.995+0000 7f1741264700 1 --2- 172.21.15.94:0/2961416036 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f173c0662e0 0x7f173c0ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:31.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:31.995+0000 7f1741264700 1 -- 172.21.15.94:0/2961416036 >> 172.21.15.94:0/2961416036 conn(0x7f173c0f7430 msgr2=0x7f173c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:43:31.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:31.995+0000 7f1741264700 1 -- 172.21.15.94:0/2961416036 shutdown_connections 2024-03-24T10:43:31.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:31.995+0000 7f1741264700 1 -- 172.21.15.94:0/2961416036 wait complete. 2024-03-24T10:43:31.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:31.995+0000 7f1741264700 1 Processor -- start 2024-03-24T10:43:31.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:31.996+0000 7f1741264700 1 -- start start 2024-03-24T10:43:31.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:31.996+0000 7f1741264700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f173c0662e0 0x7f173c10dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:31.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:31.996+0000 7f1741264700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f173c0fff00 0x7f173c10e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:31.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:31.996+0000 7f1741264700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173c10e690 con 0x7f173c0fff00 2024-03-24T10:43:31.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:31.996+0000 7f1741264700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173c10ee30 con 0x7f173c0662e0 2024-03-24T10:43:31.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:31.996+0000 7f17327fc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f173c0fff00 0x7f173c10e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:31.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:31.996+0000 7f17327fc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f173c0fff00 0x7f173c10e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53306/0 (socket says 172.21.15.94:53306) 2024-03-24T10:43:31.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:31.996+0000 7f17327fc700 1 -- 172.21.15.94:0/4226364338 learned_addr learned my addr 172.21.15.94:0/4226364338 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:43:31.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:31.996+0000 7f173affd700 1 --2- 172.21.15.94:0/4226364338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f173c0662e0 0x7f173c10dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:31.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:31.996+0000 7f17327fc700 1 -- 172.21.15.94:0/4226364338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f173c0662e0 msgr2=0x7f173c10dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:31.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:31.996+0000 7f17327fc700 1 --2- 172.21.15.94:0/4226364338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f173c0662e0 0x7f173c10dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:32.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:31.996+0000 7f17327fc700 1 -- 172.21.15.94:0/4226364338 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1724005040 con 0x7f173c0fff00 2024-03-24T10:43:32.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:31.996+0000 7f173affd700 1 --2- 172.21.15.94:0/4226364338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f173c0662e0 0x7f173c10dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T10:43:32.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:31.996+0000 7f17327fc700 1 --2- 172.21.15.94:0/4226364338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f173c0fff00 0x7f173c10e000 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f172400b970 tx=0x7f1724009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:43:32.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:31.996+0000 7f1733fff700 1 -- 172.21.15.94:0/4226364338 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f172400e040 con 0x7f173c0fff00 2024-03-24T10:43:32.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:31.997+0000 7f1733fff700 1 -- 172.21.15.94:0/4226364338 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f172400bcc0 con 0x7f173c0fff00 2024-03-24T10:43:32.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:31.997+0000 7f1741264700 1 -- 172.21.15.94:0/4226364338 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f173c10f0b0 con 0x7f173c0fff00 2024-03-24T10:43:32.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:31.997+0000 7f1733fff700 1 -- 172.21.15.94:0/4226364338 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1724013660 con 0x7f173c0fff00 2024-03-24T10:43:32.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:31.997+0000 7f1741264700 1 -- 172.21.15.94:0/4226364338 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f173c10f670 con 0x7f173c0fff00 2024-03-24T10:43:32.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:31.997+0000 7f1741264700 1 -- 172.21.15.94:0/4226364338 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f173c05f5f0 con 0x7f173c0fff00 2024-03-24T10:43:32.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:31.998+0000 7f1733fff700 1 -- 172.21.15.94:0/4226364338 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f172401ca00 con 0x7f173c0fff00 2024-03-24T10:43:32.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:31.998+0000 7f1733fff700 1 --2- 172.21.15.94:0/4226364338 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f17280692a0 0x7f172806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:32.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:31.999+0000 7f173affd700 1 --2- 172.21.15.94:0/4226364338 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f17280692a0 0x7f172806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:32.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:31.999+0000 7f1733fff700 1 -- 172.21.15.94:0/4226364338 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f1724087b70 con 0x7f173c0fff00 2024-03-24T10:43:32.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:31.999+0000 7f173affd700 1 --2- 172.21.15.94:0/4226364338 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f17280692a0 0x7f172806b750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f172c006fd0 tx=0x7f172c008040 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:43:32.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:32.003+0000 7f1733fff700 1 -- 172.21.15.94:0/4226364338 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f1724057d20 con 0x7f173c0fff00 2024-03-24T10:43:32.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:32.194+0000 7f1741264700 1 -- 172.21.15.94:0/4226364338 --> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f173c10f900 con 0x7f17280692a0 2024-03-24T10:43:32.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:32.195+0000 7f1733fff700 1 -- 172.21.15.94:0/4226364338 <== mgr.14232 v2:172.21.15.94:6800/2184357820 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+145 (secure 0 0 0) 0x7f173c10f900 con 0x7f17280692a0 2024-03-24T10:43:32.196 INFO:teuthology.orchestra.run.smithi094.stdout:HOST ADDR LABELS STATUS 2024-03-24T10:43:32.197 INFO:teuthology.orchestra.run.smithi094.stdout:smithi094 172.21.15.94 2024-03-24T10:43:32.197 INFO:teuthology.orchestra.run.smithi094.stdout:smithi096 172.21.15.96 2024-03-24T10:43:32.197 INFO:teuthology.orchestra.run.smithi094.stdout:2 hosts in cluster 2024-03-24T10:43:32.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:32.197+0000 7f1741264700 1 -- 172.21.15.94:0/4226364338 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f17280692a0 msgr2=0x7f172806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:32.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:32.197+0000 7f1741264700 1 --2- 172.21.15.94:0/4226364338 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f17280692a0 0x7f172806b750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f172c006fd0 tx=0x7f172c008040 comp rx=0 tx=0).stop 2024-03-24T10:43:32.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:32.197+0000 7f1741264700 1 -- 172.21.15.94:0/4226364338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f173c0fff00 msgr2=0x7f173c10e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:32.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:32.197+0000 7f1741264700 1 --2- 172.21.15.94:0/4226364338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f173c0fff00 0x7f173c10e000 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f172400b970 tx=0x7f1724009e30 comp rx=0 tx=0).stop 2024-03-24T10:43:32.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:32.198+0000 7f1741264700 1 -- 172.21.15.94:0/4226364338 shutdown_connections 2024-03-24T10:43:32.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:32.198+0000 7f1741264700 1 --2- 172.21.15.94:0/4226364338 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f17280692a0 0x7f172806b750 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:32.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:32.198+0000 7f1741264700 1 --2- 172.21.15.94:0/4226364338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f173c0662e0 0x7f173c10dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:32.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:32.198+0000 7f1741264700 1 --2- 172.21.15.94:0/4226364338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f173c0fff00 0x7f173c10e000 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:32.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:32.198+0000 7f1741264700 1 -- 172.21.15.94:0/4226364338 >> 172.21.15.94:0/4226364338 conn(0x7f173c0f7430 msgr2=0x7f173c0f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:43:32.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:32.198+0000 7f1741264700 1 -- 172.21.15.94:0/4226364338 shutdown_connections 2024-03-24T10:43:32.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:32.198+0000 7f1741264700 1 -- 172.21.15.94:0/4226364338 wait complete. 2024-03-24T10:43:32.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:32 smithi094 ceph-mon[29446]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:43:32.767 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-24T10:43:32.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:32 smithi096 ceph-mon[38097]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:43:33.024 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:43:33.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:33 smithi094 ceph-mon[29446]: from='client.14468 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:43:33.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:33 smithi096 ceph-mon[38097]: from='client.14468 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:43:34.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:34.616+0000 7fa3673c0700 1 -- 172.21.15.94:0/1502550691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3600fff80 msgr2=0x7fa360102360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:34.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:34.616+0000 7fa3673c0700 1 --2- 172.21.15.94:0/1502550691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3600fff80 0x7fa360102360 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa350004940 tx=0x7fa350009e30 comp rx=0 tx=0).stop 2024-03-24T10:43:34.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:34.616+0000 7fa3673c0700 1 -- 172.21.15.94:0/1502550691 shutdown_connections 2024-03-24T10:43:34.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:34.616+0000 7fa3673c0700 1 --2- 172.21.15.94:0/1502550691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3600fff80 0x7fa360102360 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:34.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:34.616+0000 7fa3673c0700 1 --2- 172.21.15.94:0/1502550691 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa360066260 0x7fa3600ffa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:34.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:34.616+0000 7fa3673c0700 1 -- 172.21.15.94:0/1502550691 >> 172.21.15.94:0/1502550691 conn(0x7fa3600f7430 msgr2=0x7fa3600f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:43:34.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:34.616+0000 7fa3673c0700 1 -- 172.21.15.94:0/1502550691 shutdown_connections 2024-03-24T10:43:34.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:34.616+0000 7fa3673c0700 1 -- 172.21.15.94:0/1502550691 wait complete. 2024-03-24T10:43:34.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:34.617+0000 7fa3673c0700 1 Processor -- start 2024-03-24T10:43:34.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:34.617+0000 7fa3673c0700 1 -- start start 2024-03-24T10:43:34.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:34.617+0000 7fa3673c0700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa360066260 0x7fa36010da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:34.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:34.617+0000 7fa3673c0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3600fff80 0x7fa36010df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:34.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:34.617+0000 7fa3673c0700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36010e570 con 0x7fa3600fff80 2024-03-24T10:43:34.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:34.617+0000 7fa3673c0700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36010edc0 con 0x7fa360066260 2024-03-24T10:43:34.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:34.617+0000 7fa36495b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3600fff80 0x7fa36010df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:34.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:34.617+0000 7fa36495b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3600fff80 0x7fa36010df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53324/0 (socket says 172.21.15.94:53324) 2024-03-24T10:43:34.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:34.617+0000 7fa36495b700 1 -- 172.21.15.94:0/3073847098 learned_addr learned my addr 172.21.15.94:0/3073847098 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:43:34.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:34.617+0000 7fa36515c700 1 --2- 172.21.15.94:0/3073847098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa360066260 0x7fa36010da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:34.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:34.617+0000 7fa36495b700 1 -- 172.21.15.94:0/3073847098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa360066260 msgr2=0x7fa36010da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:34.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:34.617+0000 7fa36495b700 1 --2- 172.21.15.94:0/3073847098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa360066260 0x7fa36010da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:34.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:34.617+0000 7fa36495b700 1 -- 172.21.15.94:0/3073847098 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa350005040 con 0x7fa3600fff80 2024-03-24T10:43:34.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:34.617+0000 7fa36515c700 1 --2- 172.21.15.94:0/3073847098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa360066260 0x7fa36010da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T10:43:34.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:34.617+0000 7fa36495b700 1 --2- 172.21.15.94:0/3073847098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3600fff80 0x7fa36010df70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa35000b3b0 tx=0x7fa350009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:43:34.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:34.618+0000 7fa355ffb700 1 -- 172.21.15.94:0/3073847098 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa35000e040 con 0x7fa3600fff80 2024-03-24T10:43:34.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:34.618+0000 7fa3673c0700 1 -- 172.21.15.94:0/3073847098 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa36010f040 con 0x7fa3600fff80 2024-03-24T10:43:34.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:34.618+0000 7fa355ffb700 1 -- 172.21.15.94:0/3073847098 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa35000bcc0 con 0x7fa3600fff80 2024-03-24T10:43:34.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:34.618+0000 7fa355ffb700 1 -- 172.21.15.94:0/3073847098 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa35001c630 con 0x7fa3600fff80 2024-03-24T10:43:34.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:34.618+0000 7fa3673c0700 1 -- 172.21.15.94:0/3073847098 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa36010f580 con 0x7fa3600fff80 2024-03-24T10:43:34.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:34.619+0000 7fa3673c0700 1 -- 172.21.15.94:0/3073847098 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa36005f5f0 con 0x7fa3600fff80 2024-03-24T10:43:34.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:34.619+0000 7fa355ffb700 1 -- 172.21.15.94:0/3073847098 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fa350013980 con 0x7fa3600fff80 2024-03-24T10:43:34.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:34.620+0000 7fa355ffb700 1 --2- 172.21.15.94:0/3073847098 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fa34c0692a0 0x7fa34c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:34.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:34.620+0000 7fa36515c700 1 --2- 172.21.15.94:0/3073847098 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fa34c0692a0 0x7fa34c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:34.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:34.620+0000 7fa355ffb700 1 -- 172.21.15.94:0/3073847098 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fa350087c70 con 0x7fa3600fff80 2024-03-24T10:43:34.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:34.620+0000 7fa36515c700 1 --2- 172.21.15.94:0/3073847098 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fa34c0692a0 0x7fa34c06b750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa35c009fd0 tx=0x7fa35c009f00 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:43:34.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:34.624+0000 7fa355ffb700 1 -- 172.21.15.94:0/3073847098 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fa350057cf0 con 0x7fa3600fff80 2024-03-24T10:43:34.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:34 smithi094 ceph-mon[29446]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:43:34.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:34.817+0000 7fa3673c0700 1 -- 172.21.15.94:0/3073847098 --> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fa36010f8b0 con 0x7fa34c0692a0 2024-03-24T10:43:34.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:34.818+0000 7fa355ffb700 1 -- 172.21.15.94:0/3073847098 <== mgr.14232 v2:172.21.15.94:6800/2184357820 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+399 (secure 0 0 0) 0x7fa36010f8b0 con 0x7fa34c0692a0 2024-03-24T10:43:34.819 INFO:teuthology.orchestra.run.smithi094.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-24T10:43:34.820 INFO:teuthology.orchestra.run.smithi094.stdout:smithi094 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513001L480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-03-24T10:43:34.820 INFO:teuthology.orchestra.run.smithi094.stdout:smithi096 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NH400BGN 372G No 69s ago Has a FileSystem, LVM detected 2024-03-24T10:43:34.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:34.820+0000 7fa3673c0700 1 -- 172.21.15.94:0/3073847098 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fa34c0692a0 msgr2=0x7fa34c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:34.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:34.820+0000 7fa3673c0700 1 --2- 172.21.15.94:0/3073847098 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fa34c0692a0 0x7fa34c06b750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa35c009fd0 tx=0x7fa35c009f00 comp rx=0 tx=0).stop 2024-03-24T10:43:34.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:34.820+0000 7fa3673c0700 1 -- 172.21.15.94:0/3073847098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3600fff80 msgr2=0x7fa36010df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:34.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:34.820+0000 7fa3673c0700 1 --2- 172.21.15.94:0/3073847098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3600fff80 0x7fa36010df70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa35000b3b0 tx=0x7fa350009e30 comp rx=0 tx=0).stop 2024-03-24T10:43:34.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:34.820+0000 7fa3673c0700 1 -- 172.21.15.94:0/3073847098 shutdown_connections 2024-03-24T10:43:34.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:34.820+0000 7fa3673c0700 1 --2- 172.21.15.94:0/3073847098 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fa34c0692a0 0x7fa34c06b750 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:34.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:34.820+0000 7fa3673c0700 1 --2- 172.21.15.94:0/3073847098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa360066260 0x7fa36010da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:34.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:34.820+0000 7fa3673c0700 1 --2- 172.21.15.94:0/3073847098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3600fff80 0x7fa36010df70 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:34.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:34.820+0000 7fa3673c0700 1 -- 172.21.15.94:0/3073847098 >> 172.21.15.94:0/3073847098 conn(0x7fa3600f7430 msgr2=0x7fa3600f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:43:34.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:34.821+0000 7fa3673c0700 1 -- 172.21.15.94:0/3073847098 shutdown_connections 2024-03-24T10:43:34.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:34.821+0000 7fa3673c0700 1 -- 172.21.15.94:0/3073847098 wait complete. 2024-03-24T10:43:34.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:34 smithi096 ceph-mon[38097]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:43:35.430 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-24T10:43:35.438 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi094.front.sepia.ceph.com 2024-03-24T10:43:35.438 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-03-24T10:43:35.698 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:43:36.711 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:36 smithi094 ceph-mon[29446]: from='client.14472 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:43:36.712 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:36 smithi094 ceph-mon[29446]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:43:36.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:36 smithi096 ceph-mon[38097]: from='client.14472 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:43:36.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:36 smithi096 ceph-mon[38097]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:43:37.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:37.192+0000 7f088c777700 1 -- 172.21.15.94:0/447944369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08840fdfe0 msgr2=0x7f08840fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:37.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:37.192+0000 7f088c777700 1 --2- 172.21.15.94:0/447944369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08840fdfe0 0x7f08840fe450 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0874004940 tx=0x7f0874009e30 comp rx=0 tx=0).stop 2024-03-24T10:43:37.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:37.192+0000 7f088c777700 1 -- 172.21.15.94:0/447944369 shutdown_connections 2024-03-24T10:43:37.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:37.192+0000 7f088c777700 1 --2- 172.21.15.94:0/447944369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08840fdfe0 0x7f08840fe450 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:37.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:37.192+0000 7f088c777700 1 --2- 172.21.15.94:0/447944369 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08840fce70 0x7f08840fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:37.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:37.192+0000 7f088c777700 1 -- 172.21.15.94:0/447944369 >> 172.21.15.94:0/447944369 conn(0x7f08840f8550 msgr2=0x7f08840fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:43:37.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:37.192+0000 7f088c777700 1 -- 172.21.15.94:0/447944369 shutdown_connections 2024-03-24T10:43:37.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:37.192+0000 7f088c777700 1 -- 172.21.15.94:0/447944369 wait complete. 2024-03-24T10:43:37.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:37.192+0000 7f088c777700 1 Processor -- start 2024-03-24T10:43:37.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:37.192+0000 7f088c777700 1 -- start start 2024-03-24T10:43:37.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:37.192+0000 7f088c777700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08840fce70 0x7f088410d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:37.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:37.192+0000 7f088c777700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08840fdfe0 0x7f088410de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:37.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:37.192+0000 7f088c777700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088410e500 con 0x7f08840fce70 2024-03-24T10:43:37.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:37.192+0000 7f088c777700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0884112ea0 con 0x7f08840fdfe0 2024-03-24T10:43:37.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:37.193+0000 7f088a513700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08840fce70 0x7f088410d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:37.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:37.193+0000 7f088a513700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08840fce70 0x7f088410d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53334/0 (socket says 172.21.15.94:53334) 2024-03-24T10:43:37.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:37.193+0000 7f088a513700 1 -- 172.21.15.94:0/2277391923 learned_addr learned my addr 172.21.15.94:0/2277391923 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:43:37.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:37.193+0000 7f0889d12700 1 --2- 172.21.15.94:0/2277391923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08840fdfe0 0x7f088410de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:37.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:37.193+0000 7f088a513700 1 -- 172.21.15.94:0/2277391923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08840fdfe0 msgr2=0x7f088410de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:37.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:37.193+0000 7f088a513700 1 --2- 172.21.15.94:0/2277391923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08840fdfe0 0x7f088410de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:37.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:37.193+0000 7f088a513700 1 -- 172.21.15.94:0/2277391923 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0874005040 con 0x7f08840fce70 2024-03-24T10:43:37.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:37.193+0000 7f088a513700 1 --2- 172.21.15.94:0/2277391923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08840fce70 0x7f088410d930 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f088000b790 tx=0x7f0880009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:43:37.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:37.193+0000 7f087affd700 1 -- 172.21.15.94:0/2277391923 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f088000f770 con 0x7f08840fce70 2024-03-24T10:43:37.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:37.193+0000 7f087affd700 1 -- 172.21.15.94:0/2277391923 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f088000fda0 con 0x7f08840fce70 2024-03-24T10:43:37.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:37.193+0000 7f088c777700 1 -- 172.21.15.94:0/2277391923 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0884113100 con 0x7f08840fce70 2024-03-24T10:43:37.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:37.193+0000 7f087affd700 1 -- 172.21.15.94:0/2277391923 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f088000d310 con 0x7f08840fce70 2024-03-24T10:43:37.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:37.193+0000 7f088c777700 1 -- 172.21.15.94:0/2277391923 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0884113670 con 0x7f08840fce70 2024-03-24T10:43:37.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:37.194+0000 7f088c777700 1 -- 172.21.15.94:0/2277391923 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f088404c710 con 0x7f08840fce70 2024-03-24T10:43:37.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:37.195+0000 7f087affd700 1 -- 172.21.15.94:0/2277391923 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f0880010420 con 0x7f08840fce70 2024-03-24T10:43:37.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:37.195+0000 7f087affd700 1 --2- 172.21.15.94:0/2277391923 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f08700692a0 0x7f087006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:37.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:37.195+0000 7f087affd700 1 -- 172.21.15.94:0/2277391923 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f0880086e70 con 0x7f08840fce70 2024-03-24T10:43:37.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:37.195+0000 7f0889d12700 1 --2- 172.21.15.94:0/2277391923 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f08700692a0 0x7f087006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:37.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:37.195+0000 7f0889d12700 1 --2- 172.21.15.94:0/2277391923 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f08700692a0 0x7f087006b750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f087400b3b0 tx=0x7f0874009e30 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:43:37.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:37.199+0000 7f087affd700 1 -- 172.21.15.94:0/2277391923 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f0880056f70 con 0x7f08840fce70 2024-03-24T10:43:37.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:37.417+0000 7f088c777700 1 -- 172.21.15.94:0/2277391923 --> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f0884102300 con 0x7f08700692a0 2024-03-24T10:43:38.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:38 smithi094 ceph-mon[29446]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:43:38.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:38 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-03-24T10:43:38.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:38 smithi096 ceph-mon[38097]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:43:38.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:38 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-03-24T10:43:39.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:39.453+0000 7f087affd700 1 -- 172.21.15.94:0/2277391923 <== mgr.14232 v2:172.21.15.94:6800/2184357820 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f0884102300 con 0x7f08700692a0 2024-03-24T10:43:39.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:39.456+0000 7f088c777700 1 -- 172.21.15.94:0/2277391923 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f08700692a0 msgr2=0x7f087006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:39.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:39.456+0000 7f088c777700 1 --2- 172.21.15.94:0/2277391923 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f08700692a0 0x7f087006b750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f087400b3b0 tx=0x7f0874009e30 comp rx=0 tx=0).stop 2024-03-24T10:43:39.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:39.456+0000 7f088c777700 1 -- 172.21.15.94:0/2277391923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08840fce70 msgr2=0x7f088410d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:39.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:39.456+0000 7f088c777700 1 --2- 172.21.15.94:0/2277391923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08840fce70 0x7f088410d930 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f088000b790 tx=0x7f0880009250 comp rx=0 tx=0).stop 2024-03-24T10:43:39.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:39.457+0000 7f088c777700 1 -- 172.21.15.94:0/2277391923 shutdown_connections 2024-03-24T10:43:39.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:39.457+0000 7f088c777700 1 --2- 172.21.15.94:0/2277391923 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f08700692a0 0x7f087006b750 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:39.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:39.457+0000 7f088c777700 1 --2- 172.21.15.94:0/2277391923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08840fce70 0x7f088410d930 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:39.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:39.457+0000 7f088c777700 1 --2- 172.21.15.94:0/2277391923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08840fdfe0 0x7f088410de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:39.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:39.457+0000 7f088c777700 1 -- 172.21.15.94:0/2277391923 >> 172.21.15.94:0/2277391923 conn(0x7f08840f8550 msgr2=0x7f08840f9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:43:39.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:39.457+0000 7f088c777700 1 -- 172.21.15.94:0/2277391923 shutdown_connections 2024-03-24T10:43:39.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:39.457+0000 7f088c777700 1 -- 172.21.15.94:0/2277391923 wait complete. 2024-03-24T10:43:39.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:39 smithi094 ceph-mon[29446]: from='client.14476 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:43:39.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:39 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-03-24T10:43:39.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:39 smithi094 ceph-mon[29446]: osdmap e41: 6 total, 6 up, 6 in 2024-03-24T10:43:39.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:39 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-03-24T10:43:39.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:39 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-mon-smithi094[29442]: 2024-03-24T10:43:39.435+0000 7febab85c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-24T10:43:39.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:39 smithi096 ceph-mon[38097]: from='client.14476 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:43:39.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:39 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-03-24T10:43:39.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:39 smithi096 ceph-mon[38097]: osdmap e41: 6 total, 6 up, 6 in 2024-03-24T10:43:39.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:39 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-03-24T10:43:40.098 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-24T10:43:40.676 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:40 smithi094 ceph-mon[29446]: pgmap v168: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:43:40.676 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:40 smithi094 ceph-mon[29446]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T10:43:40.676 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:40 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-03-24T10:43:40.677 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:40 smithi094 ceph-mon[29446]: osdmap e42: 6 total, 6 up, 6 in 2024-03-24T10:43:40.677 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:40 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-03-24T10:43:40.677 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:40 smithi094 ceph-mon[29446]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-24T10:43:40.677 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:40 smithi094 ceph-mon[29446]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-24T10:43:40.677 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:40 smithi094 ceph-mon[29446]: osdmap e43: 6 total, 6 up, 6 in 2024-03-24T10:43:40.677 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:40 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-03-24T10:43:40.677 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:40 smithi094 ceph-mon[29446]: fsmap cephfs:0 2024-03-24T10:43:40.677 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:40 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:43:40.677 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:40 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:43:40.677 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:40 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:43:40.677 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:40 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:43:40.677 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:40 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:43:40.677 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:40 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi094.efiqte", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T10:43:40.677 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:40 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi094.efiqte", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T10:43:40.677 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:40 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:43:40.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:40 smithi096 ceph-mon[38097]: pgmap v168: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:43:40.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:40 smithi096 ceph-mon[38097]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T10:43:40.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:40 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-03-24T10:43:40.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:40 smithi096 ceph-mon[38097]: osdmap e42: 6 total, 6 up, 6 in 2024-03-24T10:43:40.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:40 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-03-24T10:43:40.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:40 smithi096 ceph-mon[38097]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-24T10:43:40.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:40 smithi096 ceph-mon[38097]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-24T10:43:40.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:40 smithi096 ceph-mon[38097]: osdmap e43: 6 total, 6 up, 6 in 2024-03-24T10:43:40.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:40 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-03-24T10:43:40.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:40 smithi096 ceph-mon[38097]: fsmap cephfs:0 2024-03-24T10:43:40.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:40 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:43:40.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:40 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:43:40.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:40 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:43:40.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:40 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:43:40.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:40 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:43:40.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:40 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi094.efiqte", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T10:43:40.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:40 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi094.efiqte", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T10:43:40.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:40 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:43:41.430 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:43:41.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:41 smithi094 ceph-mon[29446]: Saving service mds.cephfs spec with placement count:4 2024-03-24T10:43:41.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:41 smithi094 ceph-mon[29446]: Deploying daemon mds.cephfs.smithi094.efiqte on smithi094 2024-03-24T10:43:41.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:41 smithi094 ceph-mon[29446]: osdmap e44: 6 total, 6 up, 6 in 2024-03-24T10:43:41.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:41 smithi094 ceph-mon[29446]: pgmap v172: 65 pgs: 19 creating+peering, 9 active+clean, 37 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:43:41.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:41 smithi096 ceph-mon[38097]: Saving service mds.cephfs spec with placement count:4 2024-03-24T10:43:41.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:41 smithi096 ceph-mon[38097]: Deploying daemon mds.cephfs.smithi094.efiqte on smithi094 2024-03-24T10:43:41.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:41 smithi096 ceph-mon[38097]: osdmap e44: 6 total, 6 up, 6 in 2024-03-24T10:43:41.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:41 smithi096 ceph-mon[38097]: pgmap v172: 65 pgs: 19 creating+peering, 9 active+clean, 37 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:43:42.681 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:42 smithi094 ceph-mon[29446]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-24T10:43:42.681 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:42 smithi094 ceph-mon[29446]: osdmap e45: 6 total, 6 up, 6 in 2024-03-24T10:43:42.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:42 smithi096 ceph-mon[38097]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-24T10:43:42.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:42 smithi096 ceph-mon[38097]: osdmap e45: 6 total, 6 up, 6 in 2024-03-24T10:43:43.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:43.167+0000 7f1c18e14700 1 -- 172.21.15.94:0/2230256021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c140fbcd0 msgr2=0x7f1c140fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:43.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:43.167+0000 7f1c18e14700 1 --2- 172.21.15.94:0/2230256021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c140fbcd0 0x7f1c140fc0e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc004940 tx=0x7f1bfc009e30 comp rx=0 tx=0).stop 2024-03-24T10:43:43.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:43.167+0000 7f1c18e14700 1 -- 172.21.15.94:0/2230256021 shutdown_connections 2024-03-24T10:43:43.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:43.167+0000 7f1c18e14700 1 --2- 172.21.15.94:0/2230256021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c140fcf80 0x7f1c140fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:43.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:43.167+0000 7f1c18e14700 1 --2- 172.21.15.94:0/2230256021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c140fbcd0 0x7f1c140fc0e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:43.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:43.167+0000 7f1c18e14700 1 -- 172.21.15.94:0/2230256021 >> 172.21.15.94:0/2230256021 conn(0x7f1c140f7430 msgr2=0x7f1c140f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:43:43.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:43.167+0000 7f1c18e14700 1 -- 172.21.15.94:0/2230256021 shutdown_connections 2024-03-24T10:43:43.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:43.167+0000 7f1c18e14700 1 -- 172.21.15.94:0/2230256021 wait complete. 2024-03-24T10:43:43.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:43.167+0000 7f1c18e14700 1 Processor -- start 2024-03-24T10:43:43.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:43.168+0000 7f1c18e14700 1 -- start start 2024-03-24T10:43:43.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:43.168+0000 7f1c18e14700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c140fbcd0 0x7f1c14110740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:43.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:43.168+0000 7f1c18e14700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c140fcf80 0x7f1c14112c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:43.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:43.168+0000 7f1c18e14700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c14066350 con 0x7f1c140fbcd0 2024-03-24T10:43:43.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:43.168+0000 7f1c18e14700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c14066490 con 0x7f1c140fcf80 2024-03-24T10:43:43.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:43.168+0000 7f1c1259c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c140fbcd0 0x7f1c14110740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:43.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:43.168+0000 7f1c1259c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c140fbcd0 0x7f1c14110740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36384/0 (socket says 172.21.15.94:36384) 2024-03-24T10:43:43.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:43.168+0000 7f1c1259c700 1 -- 172.21.15.94:0/1651553589 learned_addr learned my addr 172.21.15.94:0/1651553589 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:43:43.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:43.168+0000 7f1c11d9b700 1 --2- 172.21.15.94:0/1651553589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c140fcf80 0x7f1c14112c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:43.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:43.168+0000 7f1c1259c700 1 -- 172.21.15.94:0/1651553589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c140fcf80 msgr2=0x7f1c14112c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:43.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:43.168+0000 7f1c1259c700 1 --2- 172.21.15.94:0/1651553589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c140fcf80 0x7f1c14112c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:43.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:43.168+0000 7f1c1259c700 1 -- 172.21.15.94:0/1651553589 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bfc005040 con 0x7f1c140fbcd0 2024-03-24T10:43:43.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:43.168+0000 7f1c1259c700 1 --2- 172.21.15.94:0/1651553589 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c140fbcd0 0x7f1c14110740 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc00b3b0 tx=0x7f1bfc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:43:43.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:43.168+0000 7f1c0affd700 1 -- 172.21.15.94:0/1651553589 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1bfc00e040 con 0x7f1c140fbcd0 2024-03-24T10:43:43.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:43.168+0000 7f1c0affd700 1 -- 172.21.15.94:0/1651553589 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1bfc00bcc0 con 0x7f1c140fbcd0 2024-03-24T10:43:43.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:43.168+0000 7f1c18e14700 1 -- 172.21.15.94:0/1651553589 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c141131d0 con 0x7f1c140fbcd0 2024-03-24T10:43:43.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:43.168+0000 7f1c0affd700 1 -- 172.21.15.94:0/1651553589 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1bfc01c630 con 0x7f1c140fbcd0 2024-03-24T10:43:43.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:43.168+0000 7f1c18e14700 1 -- 172.21.15.94:0/1651553589 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c14113790 con 0x7f1c140fbcd0 2024-03-24T10:43:43.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:43.169+0000 7f1c18e14700 1 -- 172.21.15.94:0/1651553589 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c1405f5f0 con 0x7f1c140fbcd0 2024-03-24T10:43:43.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:43.170+0000 7f1c0affd700 1 -- 172.21.15.94:0/1651553589 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f1bfc013980 con 0x7f1c140fbcd0 2024-03-24T10:43:43.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:43.170+0000 7f1c0affd700 1 --2- 172.21.15.94:0/1651553589 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f1c000692a0 0x7f1c0006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:43.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:43.170+0000 7f1c0affd700 1 -- 172.21.15.94:0/1651553589 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f1bfc087fc0 con 0x7f1c140fbcd0 2024-03-24T10:43:43.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:43.170+0000 7f1c11d9b700 1 --2- 172.21.15.94:0/1651553589 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f1c000692a0 0x7f1c0006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:43.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:43.171+0000 7f1c11d9b700 1 --2- 172.21.15.94:0/1651553589 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f1c000692a0 0x7f1c0006b750 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1c04007fd0 tx=0x7f1c04009040 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:43:43.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:43.175+0000 7f1c0affd700 1 -- 172.21.15.94:0/1651553589 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f1bfc054270 con 0x7f1c140fbcd0 2024-03-24T10:43:43.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:43.413+0000 7f1c18e14700 1 -- 172.21.15.94:0/1651553589 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1c14113a70 con 0x7f1c140fbcd0 2024-03-24T10:43:43.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:43.413+0000 7f1c0affd700 1 -- 172.21.15.94:0/1651553589 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 2 v2) v1 ==== 75+0+1076 (secure 0 0 0) 0x7f1bfc057890 con 0x7f1c140fbcd0 2024-03-24T10:43:43.413 INFO:teuthology.orchestra.run.smithi094.stdout:e2 2024-03-24T10:43:43.413 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T10:43:43.413 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:43:43.413 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-03-24T10:43:43.413 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:43:43.413 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-03-24T10:43:43.414 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-03-24T10:43:43.414 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 2 2024-03-24T10:43:43.414 INFO:teuthology.orchestra.run.smithi094.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-24T10:43:43.414 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-03-24T10:43:39.435738+0000 2024-03-24T10:43:43.414 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-03-24T10:43:39.435811+0000 2024-03-24T10:43:43.414 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-03-24T10:43:43.414 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-03-24T10:43:43.414 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 60 2024-03-24T10:43:43.414 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-03-24T10:43:43.414 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-03-24T10:43:43.414 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-03-24T10:43:43.414 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-03-24T10:43:43.414 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 0 2024-03-24T10:43:43.414 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:43:43.414 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-03-24T10:43:43.414 INFO:teuthology.orchestra.run.smithi094.stdout:in 2024-03-24T10:43:43.415 INFO:teuthology.orchestra.run.smithi094.stdout:up {} 2024-03-24T10:43:43.415 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-03-24T10:43:43.415 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-03-24T10:43:43.415 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-03-24T10:43:43.415 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-03-24T10:43:43.415 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-03-24T10:43:43.415 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-03-24T10:43:43.415 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-03-24T10:43:43.415 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 0 2024-03-24T10:43:43.415 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:43:43.415 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:43:43.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:43.415+0000 7f1c18e14700 1 -- 172.21.15.94:0/1651553589 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f1c000692a0 msgr2=0x7f1c0006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:43.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:43.415+0000 7f1c18e14700 1 --2- 172.21.15.94:0/1651553589 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f1c000692a0 0x7f1c0006b750 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1c04007fd0 tx=0x7f1c04009040 comp rx=0 tx=0).stop 2024-03-24T10:43:43.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:43.415+0000 7f1c18e14700 1 -- 172.21.15.94:0/1651553589 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c140fbcd0 msgr2=0x7f1c14110740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:43.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:43.415+0000 7f1c18e14700 1 --2- 172.21.15.94:0/1651553589 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c140fbcd0 0x7f1c14110740 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc00b3b0 tx=0x7f1bfc009e30 comp rx=0 tx=0).stop 2024-03-24T10:43:43.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:43.415+0000 7f1c18e14700 1 -- 172.21.15.94:0/1651553589 shutdown_connections 2024-03-24T10:43:43.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:43.415+0000 7f1c18e14700 1 --2- 172.21.15.94:0/1651553589 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f1c000692a0 0x7f1c0006b750 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:43.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:43.415+0000 7f1c18e14700 1 --2- 172.21.15.94:0/1651553589 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c140fbcd0 0x7f1c14110740 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:43.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:43.415+0000 7f1c18e14700 1 --2- 172.21.15.94:0/1651553589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c140fcf80 0x7f1c14112c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:43.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:43.415+0000 7f1c18e14700 1 -- 172.21.15.94:0/1651553589 >> 172.21.15.94:0/1651553589 conn(0x7f1c140f7430 msgr2=0x7f1c140f8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:43:43.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:43.416+0000 7f1c18e14700 1 -- 172.21.15.94:0/1651553589 shutdown_connections 2024-03-24T10:43:43.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:43.416+0000 7f1c18e14700 1 -- 172.21.15.94:0/1651553589 wait complete. 2024-03-24T10:43:43.417 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 2 2024-03-24T10:43:43.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:43 smithi094 ceph-mon[29446]: pgmap v174: 65 pgs: 2 creating+activating, 19 creating+peering, 34 active+clean, 10 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:43:43.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:43 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/1651553589' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T10:43:43.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:43 smithi096 ceph-mon[38097]: pgmap v174: 65 pgs: 2 creating+activating, 19 creating+peering, 34 active+clean, 10 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:43:43.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:43 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/1651553589' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T10:43:43.982 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-24T10:43:43.992 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi094.front.sepia.ceph.com 2024-03-24T10:43:43.993 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- bash -c 'ceph fs set cephfs max_mds 1' 2024-03-24T10:43:44.261 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:43:45.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:45 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:43:45.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:45 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:43:45.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:45 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:43:45.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:45 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi096.egbmoq", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T10:43:45.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:45 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi096.egbmoq", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T10:43:45.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:45 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:43:45.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:45 smithi094 ceph-mon[29446]: Deploying daemon mds.cephfs.smithi096.egbmoq on smithi096 2024-03-24T10:43:45.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:45 smithi094 ceph-mon[29446]: mds.? [v2:172.21.15.94:6826/2001471799,v1:172.21.15.94:6827/2001471799] up:boot 2024-03-24T10:43:45.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:45 smithi094 ceph-mon[29446]: daemon mds.cephfs.smithi094.efiqte assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-03-24T10:43:45.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:45 smithi094 ceph-mon[29446]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-24T10:43:45.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:45 smithi094 ceph-mon[29446]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-24T10:43:45.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:45 smithi094 ceph-mon[29446]: Cluster is now healthy 2024-03-24T10:43:45.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:45 smithi094 ceph-mon[29446]: fsmap cephfs:0 1 up:standby 2024-03-24T10:43:45.249 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:45 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.efiqte"}]: dispatch 2024-03-24T10:43:45.249 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:45 smithi094 ceph-mon[29446]: fsmap cephfs:1 {0=cephfs.smithi094.efiqte=up:creating} 2024-03-24T10:43:45.249 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:45 smithi094 ceph-mon[29446]: daemon mds.cephfs.smithi094.efiqte is now active in filesystem cephfs as rank 0 2024-03-24T10:43:45.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:45 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:43:45.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:45 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:43:45.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:45 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:43:45.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:45 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi096.egbmoq", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T10:43:45.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:45 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi096.egbmoq", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T10:43:45.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:45 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:43:45.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:45 smithi096 ceph-mon[38097]: Deploying daemon mds.cephfs.smithi096.egbmoq on smithi096 2024-03-24T10:43:45.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:45 smithi096 ceph-mon[38097]: mds.? [v2:172.21.15.94:6826/2001471799,v1:172.21.15.94:6827/2001471799] up:boot 2024-03-24T10:43:45.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:45 smithi096 ceph-mon[38097]: daemon mds.cephfs.smithi094.efiqte assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-03-24T10:43:45.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:45 smithi096 ceph-mon[38097]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-24T10:43:45.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:45 smithi096 ceph-mon[38097]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-24T10:43:45.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:45 smithi096 ceph-mon[38097]: Cluster is now healthy 2024-03-24T10:43:45.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:45 smithi096 ceph-mon[38097]: fsmap cephfs:0 1 up:standby 2024-03-24T10:43:45.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:45 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.efiqte"}]: dispatch 2024-03-24T10:43:45.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:45 smithi096 ceph-mon[38097]: fsmap cephfs:1 {0=cephfs.smithi094.efiqte=up:creating} 2024-03-24T10:43:45.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:45 smithi096 ceph-mon[38097]: daemon mds.cephfs.smithi094.efiqte is now active in filesystem cephfs as rank 0 2024-03-24T10:43:45.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:45.731+0000 7f1bd1710700 1 -- 172.21.15.94:0/1386724945 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bcc0fcf80 msgr2=0x7f1bcc0fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:45.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:45.731+0000 7f1bd1710700 1 --2- 172.21.15.94:0/1386724945 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bcc0fcf80 0x7f1bcc0fd3f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0004940 tx=0x7f1bc0009e30 comp rx=0 tx=0).stop 2024-03-24T10:43:45.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:45.731+0000 7f1bd1710700 1 -- 172.21.15.94:0/1386724945 shutdown_connections 2024-03-24T10:43:45.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:45.731+0000 7f1bd1710700 1 --2- 172.21.15.94:0/1386724945 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bcc0fcf80 0x7f1bcc0fd3f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:45.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:45.731+0000 7f1bd1710700 1 --2- 172.21.15.94:0/1386724945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1bcc0fbcd0 0x7f1bcc0fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:45.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:45.731+0000 7f1bd1710700 1 -- 172.21.15.94:0/1386724945 >> 172.21.15.94:0/1386724945 conn(0x7f1bcc0f7430 msgr2=0x7f1bcc0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:43:45.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:45.731+0000 7f1bd1710700 1 -- 172.21.15.94:0/1386724945 shutdown_connections 2024-03-24T10:43:45.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:45.731+0000 7f1bd1710700 1 -- 172.21.15.94:0/1386724945 wait complete. 2024-03-24T10:43:45.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:45.732+0000 7f1bd1710700 1 Processor -- start 2024-03-24T10:43:45.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:45.732+0000 7f1bd1710700 1 -- start start 2024-03-24T10:43:45.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:45.732+0000 7f1bd1710700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1bcc0fbcd0 0x7f1bcc066cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:45.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:45.732+0000 7f1bd1710700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bcc0fcf80 0x7f1bcc112760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:45.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:45.732+0000 7f1bd1710700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bcc112ca0 con 0x7f1bcc0fcf80 2024-03-24T10:43:45.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:45.732+0000 7f1bd1710700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bcc112de0 con 0x7f1bcc0fbcd0 2024-03-24T10:43:45.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:45.732+0000 7f1bca7fc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bcc0fcf80 0x7f1bcc112760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:45.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:45.732+0000 7f1bca7fc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bcc0fcf80 0x7f1bcc112760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36422/0 (socket says 172.21.15.94:36422) 2024-03-24T10:43:45.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:45.732+0000 7f1bca7fc700 1 -- 172.21.15.94:0/1135020057 learned_addr learned my addr 172.21.15.94:0/1135020057 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:43:45.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:45.732+0000 7f1bcaffd700 1 --2- 172.21.15.94:0/1135020057 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1bcc0fbcd0 0x7f1bcc066cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:45.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:45.732+0000 7f1bca7fc700 1 -- 172.21.15.94:0/1135020057 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1bcc0fbcd0 msgr2=0x7f1bcc066cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:45.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:45.732+0000 7f1bca7fc700 1 --2- 172.21.15.94:0/1135020057 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1bcc0fbcd0 0x7f1bcc066cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:45.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:45.732+0000 7f1bca7fc700 1 -- 172.21.15.94:0/1135020057 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bc0005040 con 0x7f1bcc0fcf80 2024-03-24T10:43:45.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:45.733+0000 7f1bca7fc700 1 --2- 172.21.15.94:0/1135020057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bcc0fcf80 0x7f1bcc112760 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1bc000b3b0 tx=0x7f1bc0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:43:45.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:45.733+0000 7f1bb37fe700 1 -- 172.21.15.94:0/1135020057 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1bc000e040 con 0x7f1bcc0fcf80 2024-03-24T10:43:45.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:45.733+0000 7f1bb37fe700 1 -- 172.21.15.94:0/1135020057 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1bc000bbf0 con 0x7f1bcc0fcf80 2024-03-24T10:43:45.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:45.733+0000 7f1bb37fe700 1 -- 172.21.15.94:0/1135020057 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1bc001c630 con 0x7f1bcc0fcf80 2024-03-24T10:43:45.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:45.733+0000 7f1bd1710700 1 -- 172.21.15.94:0/1135020057 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bcc112fe0 con 0x7f1bcc0fcf80 2024-03-24T10:43:45.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:45.733+0000 7f1bd1710700 1 -- 172.21.15.94:0/1135020057 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bcc113570 con 0x7f1bcc0fcf80 2024-03-24T10:43:45.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:45.734+0000 7f1bd1710700 1 -- 172.21.15.94:0/1135020057 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bcc04c710 con 0x7f1bcc0fcf80 2024-03-24T10:43:45.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:45.734+0000 7f1bb37fe700 1 -- 172.21.15.94:0/1135020057 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f1bc000bd60 con 0x7f1bcc0fcf80 2024-03-24T10:43:45.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:45.735+0000 7f1bb37fe700 1 --2- 172.21.15.94:0/1135020057 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f1bb40692a0 0x7f1bb406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:45.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:45.735+0000 7f1bb37fe700 1 -- 172.21.15.94:0/1135020057 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f1bc0087d50 con 0x7f1bcc0fcf80 2024-03-24T10:43:45.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:45.735+0000 7f1bcaffd700 1 --2- 172.21.15.94:0/1135020057 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f1bb40692a0 0x7f1bb406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:45.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:45.735+0000 7f1bcaffd700 1 --2- 172.21.15.94:0/1135020057 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f1bb40692a0 0x7f1bb406b750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc004f40 tx=0x7f1bbc00b040 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:43:45.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:45.739+0000 7f1bb37fe700 1 -- 172.21.15.94:0/1135020057 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f1bc0057bb0 con 0x7f1bcc0fcf80 2024-03-24T10:43:45.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:45.972+0000 7f1bd1710700 1 -- 172.21.15.94:0/1135020057 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) v1 -- 0x7f1bcc1137b0 con 0x7f1bcc0fcf80 2024-03-24T10:43:46.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:46 smithi094 ceph-mon[29446]: pgmap v175: 65 pgs: 2 creating+activating, 14 creating+peering, 49 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:43:46.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:46 smithi094 ceph-mon[29446]: mds.? [v2:172.21.15.94:6826/2001471799,v1:172.21.15.94:6827/2001471799] up:active 2024-03-24T10:43:46.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:46 smithi094 ceph-mon[29446]: fsmap cephfs:1 {0=cephfs.smithi094.efiqte=up:active} 2024-03-24T10:43:46.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:46 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:43:46.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:46 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/1135020057' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-03-24T10:43:46.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:46.432+0000 7f1bb37fe700 1 -- 172.21.15.94:0/1135020057 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v6) v1 ==== 105+0+0 (secure 0 0 0) 0x7f1bc0057740 con 0x7f1bcc0fcf80 2024-03-24T10:43:46.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:46.434+0000 7f1bd1710700 1 -- 172.21.15.94:0/1135020057 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f1bb40692a0 msgr2=0x7f1bb406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:46.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:46.434+0000 7f1bd1710700 1 --2- 172.21.15.94:0/1135020057 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f1bb40692a0 0x7f1bb406b750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc004f40 tx=0x7f1bbc00b040 comp rx=0 tx=0).stop 2024-03-24T10:43:46.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:46.434+0000 7f1bd1710700 1 -- 172.21.15.94:0/1135020057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bcc0fcf80 msgr2=0x7f1bcc112760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:46.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:46.434+0000 7f1bd1710700 1 --2- 172.21.15.94:0/1135020057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bcc0fcf80 0x7f1bcc112760 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1bc000b3b0 tx=0x7f1bc0009e30 comp rx=0 tx=0).stop 2024-03-24T10:43:46.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:46.434+0000 7f1bd1710700 1 -- 172.21.15.94:0/1135020057 shutdown_connections 2024-03-24T10:43:46.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:46.434+0000 7f1bd1710700 1 --2- 172.21.15.94:0/1135020057 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f1bb40692a0 0x7f1bb406b750 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:46.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:46.434+0000 7f1bd1710700 1 --2- 172.21.15.94:0/1135020057 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1bcc0fbcd0 0x7f1bcc066cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:46.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:46.434+0000 7f1bd1710700 1 --2- 172.21.15.94:0/1135020057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bcc0fcf80 0x7f1bcc112760 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:46.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:46.434+0000 7f1bd1710700 1 -- 172.21.15.94:0/1135020057 >> 172.21.15.94:0/1135020057 conn(0x7f1bcc0f7430 msgr2=0x7f1bcc0f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:43:46.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:46.435+0000 7f1bd1710700 1 -- 172.21.15.94:0/1135020057 shutdown_connections 2024-03-24T10:43:46.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:46.435+0000 7f1bd1710700 1 -- 172.21.15.94:0/1135020057 wait complete. 2024-03-24T10:43:46.474 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:46 smithi096 ceph-mon[38097]: pgmap v175: 65 pgs: 2 creating+activating, 14 creating+peering, 49 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:43:46.474 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:46 smithi096 ceph-mon[38097]: mds.? [v2:172.21.15.94:6826/2001471799,v1:172.21.15.94:6827/2001471799] up:active 2024-03-24T10:43:46.474 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:46 smithi096 ceph-mon[38097]: fsmap cephfs:1 {0=cephfs.smithi094.efiqte=up:active} 2024-03-24T10:43:46.474 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:46 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:43:46.474 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:46 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/1135020057' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-03-24T10:43:47.109 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-24T10:43:47.119 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi094.front.sepia.ceph.com 2024-03-24T10:43:47.119 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- bash -c 'ceph fs set cephfs allow_standby_replay true' 2024-03-24T10:43:47.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:47 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/1135020057' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-03-24T10:43:47.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:47 smithi094 ceph-mon[29446]: fsmap cephfs:1 {0=cephfs.smithi094.efiqte=up:active} 2024-03-24T10:43:47.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:47 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:43:47.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:47 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:43:47.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:47 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:43:47.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:47 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi094.llefgu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T10:43:47.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:47 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi094.llefgu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T10:43:47.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:47 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:43:47.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:47 smithi094 ceph-mon[29446]: Deploying daemon mds.cephfs.smithi094.llefgu on smithi094 2024-03-24T10:43:47.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:47 smithi094 ceph-mon[29446]: pgmap v176: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s wr, 3 op/s 2024-03-24T10:43:47.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:47 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/1135020057' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-03-24T10:43:47.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:47 smithi096 ceph-mon[38097]: fsmap cephfs:1 {0=cephfs.smithi094.efiqte=up:active} 2024-03-24T10:43:47.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:47 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:43:47.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:47 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:43:47.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:47 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:43:47.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:47 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi094.llefgu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T10:43:47.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:47 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi094.llefgu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T10:43:47.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:47 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:43:47.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:47 smithi096 ceph-mon[38097]: Deploying daemon mds.cephfs.smithi094.llefgu on smithi094 2024-03-24T10:43:47.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:47 smithi096 ceph-mon[38097]: pgmap v176: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s wr, 3 op/s 2024-03-24T10:43:48.244 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:43:48.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:48 smithi094 ceph-mon[29446]: mds.? [v2:172.21.15.96:6824/1539752208,v1:172.21.15.96:6825/1539752208] up:boot 2024-03-24T10:43:48.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:48 smithi094 ceph-mon[29446]: fsmap cephfs:1 {0=cephfs.smithi094.efiqte=up:active} 1 up:standby 2024-03-24T10:43:48.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:48 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi096.egbmoq"}]: dispatch 2024-03-24T10:43:48.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:48 smithi094 ceph-mon[29446]: fsmap cephfs:1 {0=cephfs.smithi094.efiqte=up:active} 1 up:standby 2024-03-24T10:43:48.717 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:48 smithi096 ceph-mon[38097]: mds.? [v2:172.21.15.96:6824/1539752208,v1:172.21.15.96:6825/1539752208] up:boot 2024-03-24T10:43:48.717 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:48 smithi096 ceph-mon[38097]: fsmap cephfs:1 {0=cephfs.smithi094.efiqte=up:active} 1 up:standby 2024-03-24T10:43:48.717 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:48 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi096.egbmoq"}]: dispatch 2024-03-24T10:43:48.718 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:48 smithi096 ceph-mon[38097]: fsmap cephfs:1 {0=cephfs.smithi094.efiqte=up:active} 1 up:standby 2024-03-24T10:43:49.678 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:49 smithi094 ceph-mon[29446]: pgmap v177: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s wr, 3 op/s 2024-03-24T10:43:49.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:49 smithi096 ceph-mon[38097]: pgmap v177: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s wr, 3 op/s 2024-03-24T10:43:49.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:49.901+0000 7fc08e7b3700 1 -- 172.21.15.94:0/1799643571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0880fce40 msgr2=0x7fc0880fd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:49.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:49.901+0000 7fc08e7b3700 1 --2- 172.21.15.94:0/1799643571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0880fce40 0x7fc0880fd250 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc078004940 tx=0x7fc078009e30 comp rx=0 tx=0).stop 2024-03-24T10:43:49.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:49.901+0000 7fc08e7b3700 1 -- 172.21.15.94:0/1799643571 shutdown_connections 2024-03-24T10:43:49.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:49.901+0000 7fc08e7b3700 1 --2- 172.21.15.94:0/1799643571 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0880fe040 0x7fc0880fe4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:49.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:49.901+0000 7fc08e7b3700 1 --2- 172.21.15.94:0/1799643571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0880fce40 0x7fc0880fd250 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:49.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:49.901+0000 7fc08e7b3700 1 -- 172.21.15.94:0/1799643571 >> 172.21.15.94:0/1799643571 conn(0x7fc0880f8570 msgr2=0x7fc0880fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:43:49.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:49.901+0000 7fc08e7b3700 1 -- 172.21.15.94:0/1799643571 shutdown_connections 2024-03-24T10:43:49.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:49.901+0000 7fc08e7b3700 1 -- 172.21.15.94:0/1799643571 wait complete. 2024-03-24T10:43:49.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:49.901+0000 7fc08e7b3700 1 Processor -- start 2024-03-24T10:43:49.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:49.901+0000 7fc08e7b3700 1 -- start start 2024-03-24T10:43:49.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:49.902+0000 7fc08e7b3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0880fce40 0x7fc08810d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:49.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:49.902+0000 7fc08e7b3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0880fe040 0x7fc08810dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:49.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:49.902+0000 7fc08e7b3700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08810e4e0 con 0x7fc0880fce40 2024-03-24T10:43:49.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:49.902+0000 7fc08e7b3700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08810ed30 con 0x7fc0880fe040 2024-03-24T10:43:49.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:49.902+0000 7fc087fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0880fce40 0x7fc08810d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:49.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:49.902+0000 7fc087fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0880fce40 0x7fc08810d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49704/0 (socket says 172.21.15.94:49704) 2024-03-24T10:43:49.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:49.902+0000 7fc087fff700 1 -- 172.21.15.94:0/4273528146 learned_addr learned my addr 172.21.15.94:0/4273528146 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:43:49.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:49.902+0000 7fc0877fe700 1 --2- 172.21.15.94:0/4273528146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0880fe040 0x7fc08810dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:49.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:49.902+0000 7fc087fff700 1 -- 172.21.15.94:0/4273528146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0880fe040 msgr2=0x7fc08810dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:49.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:49.902+0000 7fc087fff700 1 --2- 172.21.15.94:0/4273528146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0880fe040 0x7fc08810dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:49.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:49.902+0000 7fc087fff700 1 -- 172.21.15.94:0/4273528146 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc078005040 con 0x7fc0880fce40 2024-03-24T10:43:49.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:49.902+0000 7fc087fff700 1 --2- 172.21.15.94:0/4273528146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0880fce40 0x7fc08810d9a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc07800b970 tx=0x7fc078009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:43:49.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:49.902+0000 7fc084ff9700 1 -- 172.21.15.94:0/4273528146 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc07800e040 con 0x7fc0880fce40 2024-03-24T10:43:49.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:49.902+0000 7fc084ff9700 1 -- 172.21.15.94:0/4273528146 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc078004290 con 0x7fc0880fce40 2024-03-24T10:43:49.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:49.902+0000 7fc08e7b3700 1 -- 172.21.15.94:0/4273528146 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc08810efb0 con 0x7fc0880fce40 2024-03-24T10:43:49.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:49.902+0000 7fc084ff9700 1 -- 172.21.15.94:0/4273528146 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc07801c3f0 con 0x7fc0880fce40 2024-03-24T10:43:49.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:49.903+0000 7fc08e7b3700 1 -- 172.21.15.94:0/4273528146 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc08810f4f0 con 0x7fc0880fce40 2024-03-24T10:43:49.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:49.904+0000 7fc08e7b3700 1 -- 172.21.15.94:0/4273528146 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc08805f5f0 con 0x7fc0880fce40 2024-03-24T10:43:49.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:49.904+0000 7fc084ff9700 1 -- 172.21.15.94:0/4273528146 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fc078013420 con 0x7fc0880fce40 2024-03-24T10:43:49.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:49.904+0000 7fc084ff9700 1 --2- 172.21.15.94:0/4273528146 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fc0700692b0 0x7fc07006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:49.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:49.905+0000 7fc084ff9700 1 -- 172.21.15.94:0/4273528146 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fc078088460 con 0x7fc0880fce40 2024-03-24T10:43:49.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:49.905+0000 7fc0877fe700 1 --2- 172.21.15.94:0/4273528146 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fc0700692b0 0x7fc07006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:49.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:49.905+0000 7fc0877fe700 1 --2- 172.21.15.94:0/4273528146 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fc0700692b0 0x7fc07006b760 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc07c009fd0 tx=0x7fc07c00b040 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:43:49.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:49.909+0000 7fc084ff9700 1 -- 172.21.15.94:0/4273528146 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fc078058220 con 0x7fc0880fce40 2024-03-24T10:43:50.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:50.154+0000 7fc08e7b3700 1 -- 172.21.15.94:0/4273528146 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fc0881058d0 con 0x7fc0880fce40 2024-03-24T10:43:50.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:50.445+0000 7fc084ff9700 1 -- 172.21.15.94:0/4273528146 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v9) v1 ==== 121+0+0 (secure 0 0 0) 0x7fc078057db0 con 0x7fc0880fce40 2024-03-24T10:43:50.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:50.448+0000 7fc08e7b3700 1 -- 172.21.15.94:0/4273528146 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fc0700692b0 msgr2=0x7fc07006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:50.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:50.448+0000 7fc08e7b3700 1 --2- 172.21.15.94:0/4273528146 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fc0700692b0 0x7fc07006b760 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc07c009fd0 tx=0x7fc07c00b040 comp rx=0 tx=0).stop 2024-03-24T10:43:50.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:50.448+0000 7fc08e7b3700 1 -- 172.21.15.94:0/4273528146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0880fce40 msgr2=0x7fc08810d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:50.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:50.448+0000 7fc08e7b3700 1 --2- 172.21.15.94:0/4273528146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0880fce40 0x7fc08810d9a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc07800b970 tx=0x7fc078009e30 comp rx=0 tx=0).stop 2024-03-24T10:43:50.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:50.448+0000 7fc08e7b3700 1 -- 172.21.15.94:0/4273528146 shutdown_connections 2024-03-24T10:43:50.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:50.448+0000 7fc08e7b3700 1 --2- 172.21.15.94:0/4273528146 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fc0700692b0 0x7fc07006b760 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:50.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:50.448+0000 7fc08e7b3700 1 --2- 172.21.15.94:0/4273528146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0880fce40 0x7fc08810d9a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:50.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:50.448+0000 7fc08e7b3700 1 --2- 172.21.15.94:0/4273528146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0880fe040 0x7fc08810dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:50.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:50.448+0000 7fc08e7b3700 1 -- 172.21.15.94:0/4273528146 >> 172.21.15.94:0/4273528146 conn(0x7fc0880f8570 msgr2=0x7fc0880f9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:43:50.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:50.448+0000 7fc08e7b3700 1 -- 172.21.15.94:0/4273528146 shutdown_connections 2024-03-24T10:43:50.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:50.448+0000 7fc08e7b3700 1 -- 172.21.15.94:0/4273528146 wait complete. 2024-03-24T10:43:50.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:50 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/4273528146' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-24T10:43:50.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:50 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:43:50.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:50 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:43:50.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:50 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/4273528146' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-24T10:43:50.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:50 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:43:50.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:50 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:43:51.072 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-24T10:43:51.082 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi094.front.sepia.ceph.com 2024-03-24T10:43:51.082 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- bash -c 'ceph fs set cephfs inline_data false' 2024-03-24T10:43:51.338 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:43:51.697 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:51 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/4273528146' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-24T10:43:51.697 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:51 smithi094 ceph-mon[29446]: mds.? [v2:172.21.15.94:6828/126762980,v1:172.21.15.94:6829/126762980] up:boot 2024-03-24T10:43:51.697 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:51 smithi094 ceph-mon[29446]: fsmap cephfs:1 {0=cephfs.smithi094.efiqte=up:active} 2 up:standby 2024-03-24T10:43:51.697 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:51 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.llefgu"}]: dispatch 2024-03-24T10:43:51.697 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:51 smithi094 ceph-mon[29446]: fsmap cephfs:1 {0=cephfs.smithi094.efiqte=up:active} 1 up:standby-replay 1 up:standby 2024-03-24T10:43:51.697 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:51 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:43:51.697 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:51 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi096.jtebhh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T10:43:51.697 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:51 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi096.jtebhh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T10:43:51.697 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:51 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:43:51.697 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:51 smithi094 ceph-mon[29446]: Deploying daemon mds.cephfs.smithi096.jtebhh on smithi096 2024-03-24T10:43:51.697 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:51 smithi094 ceph-mon[29446]: pgmap v178: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s wr, 4 op/s 2024-03-24T10:43:51.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:51 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/4273528146' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-24T10:43:51.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:51 smithi096 ceph-mon[38097]: mds.? [v2:172.21.15.94:6828/126762980,v1:172.21.15.94:6829/126762980] up:boot 2024-03-24T10:43:51.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:51 smithi096 ceph-mon[38097]: fsmap cephfs:1 {0=cephfs.smithi094.efiqte=up:active} 2 up:standby 2024-03-24T10:43:51.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:51 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.llefgu"}]: dispatch 2024-03-24T10:43:51.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:51 smithi096 ceph-mon[38097]: fsmap cephfs:1 {0=cephfs.smithi094.efiqte=up:active} 1 up:standby-replay 1 up:standby 2024-03-24T10:43:51.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:51 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:43:51.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:51 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi096.jtebhh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T10:43:51.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:51 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi096.jtebhh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-24T10:43:51.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:51 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:43:51.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:51 smithi096 ceph-mon[38097]: Deploying daemon mds.cephfs.smithi096.jtebhh on smithi096 2024-03-24T10:43:51.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:51 smithi096 ceph-mon[38097]: pgmap v178: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s wr, 4 op/s 2024-03-24T10:43:53.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:53.064+0000 7fdab8b24700 1 -- 172.21.15.94:0/2677386993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdab40fe040 msgr2=0x7fdab40fe4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:53.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:53.064+0000 7fdab8b24700 1 --2- 172.21.15.94:0/2677386993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdab40fe040 0x7fdab40fe4b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8004940 tx=0x7fdaa8009e30 comp rx=0 tx=0).stop 2024-03-24T10:43:53.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:53.065+0000 7fdab8b24700 1 -- 172.21.15.94:0/2677386993 shutdown_connections 2024-03-24T10:43:53.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:53.065+0000 7fdab8b24700 1 --2- 172.21.15.94:0/2677386993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdab40fe040 0x7fdab40fe4b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:53.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:53.065+0000 7fdab8b24700 1 --2- 172.21.15.94:0/2677386993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdab40fce40 0x7fdab40fd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:53.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:53.065+0000 7fdab8b24700 1 -- 172.21.15.94:0/2677386993 >> 172.21.15.94:0/2677386993 conn(0x7fdab40f8570 msgr2=0x7fdab40fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:43:53.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:53.065+0000 7fdab8b24700 1 -- 172.21.15.94:0/2677386993 shutdown_connections 2024-03-24T10:43:53.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:53.065+0000 7fdab8b24700 1 -- 172.21.15.94:0/2677386993 wait complete. 2024-03-24T10:43:53.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:53.065+0000 7fdab8b24700 1 Processor -- start 2024-03-24T10:43:53.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:53.065+0000 7fdab8b24700 1 -- start start 2024-03-24T10:43:53.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:53.065+0000 7fdab8b24700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdab40fce40 0x7fdab4071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:53.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:53.065+0000 7fdab8b24700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdab40fe040 0x7fdab4070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:53.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:53.065+0000 7fdab8b24700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab40708c0 con 0x7fdab40fce40 2024-03-24T10:43:53.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:53.065+0000 7fdab8b24700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab4070a30 con 0x7fdab40fe040 2024-03-24T10:43:53.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:53.065+0000 7fdab259c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdab40fce40 0x7fdab4071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:53.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:53.065+0000 7fdab259c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdab40fce40 0x7fdab4071e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49738/0 (socket says 172.21.15.94:49738) 2024-03-24T10:43:53.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:53.065+0000 7fdab259c700 1 -- 172.21.15.94:0/55515188 learned_addr learned my addr 172.21.15.94:0/55515188 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:43:53.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:53.065+0000 7fdab1d9b700 1 --2- 172.21.15.94:0/55515188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdab40fe040 0x7fdab4070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:53.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:53.066+0000 7fdab259c700 1 -- 172.21.15.94:0/55515188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdab40fe040 msgr2=0x7fdab4070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:53.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:53.066+0000 7fdab259c700 1 --2- 172.21.15.94:0/55515188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdab40fe040 0x7fdab4070310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:53.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:53.066+0000 7fdab259c700 1 -- 172.21.15.94:0/55515188 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa8005040 con 0x7fdab40fce40 2024-03-24T10:43:53.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:53.066+0000 7fdab1d9b700 1 --2- 172.21.15.94:0/55515188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdab40fe040 0x7fdab4070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T10:43:53.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:53.066+0000 7fdab259c700 1 --2- 172.21.15.94:0/55515188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdab40fce40 0x7fdab4071e40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4002a50 tx=0x7fdaa4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:43:53.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:53.066+0000 7fdaa2ffd700 1 -- 172.21.15.94:0/55515188 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdaa4007cc0 con 0x7fdab40fce40 2024-03-24T10:43:53.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:53.066+0000 7fdab8b24700 1 -- 172.21.15.94:0/55515188 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdab4070d10 con 0x7fdab40fce40 2024-03-24T10:43:53.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:53.066+0000 7fdaa2ffd700 1 -- 172.21.15.94:0/55515188 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdaa400f040 con 0x7fdab40fce40 2024-03-24T10:43:53.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:53.066+0000 7fdaa2ffd700 1 -- 172.21.15.94:0/55515188 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdaa400d940 con 0x7fdab40fce40 2024-03-24T10:43:53.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:53.066+0000 7fdab8b24700 1 -- 172.21.15.94:0/55515188 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdab4115a50 con 0x7fdab40fce40 2024-03-24T10:43:53.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:53.067+0000 7fdab8b24700 1 -- 172.21.15.94:0/55515188 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdab405f5f0 con 0x7fdab40fce40 2024-03-24T10:43:53.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:53.067+0000 7fdaa2ffd700 1 -- 172.21.15.94:0/55515188 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fdaa400dc00 con 0x7fdab40fce40 2024-03-24T10:43:53.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:53.068+0000 7fdaa2ffd700 1 --2- 172.21.15.94:0/55515188 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fda9c0692a0 0x7fda9c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:53.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:53.068+0000 7fdaa2ffd700 1 -- 172.21.15.94:0/55515188 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fdaa4086ed0 con 0x7fdab40fce40 2024-03-24T10:43:53.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:53.068+0000 7fdab1d9b700 1 --2- 172.21.15.94:0/55515188 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fda9c0692a0 0x7fda9c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:53.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:53.068+0000 7fdab1d9b700 1 --2- 172.21.15.94:0/55515188 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fda9c0692a0 0x7fda9c06b750 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdaa800b3b0 tx=0x7fdaa8009e30 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:43:53.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:53.075+0000 7fdaa2ffd700 1 -- 172.21.15.94:0/55515188 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fdaa40531f0 con 0x7fdab40fce40 2024-03-24T10:43:53.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:53.321+0000 7fdab8b24700 1 -- 172.21.15.94:0/55515188 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"} v 0) v1 -- 0x7fdab41058f0 con 0x7fdab40fce40 2024-03-24T10:43:54.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:54.080+0000 7fdaa2ffd700 1 -- 172.21.15.94:0/55515188 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]=0 inline data disabled v12) v1 ==== 133+0+0 (secure 0 0 0) 0x7fdaa4056810 con 0x7fdab40fce40 2024-03-24T10:43:54.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:54.083+0000 7fdab8b24700 1 -- 172.21.15.94:0/55515188 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fda9c0692a0 msgr2=0x7fda9c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:54.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:54.083+0000 7fdab8b24700 1 --2- 172.21.15.94:0/55515188 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fda9c0692a0 0x7fda9c06b750 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdaa800b3b0 tx=0x7fdaa8009e30 comp rx=0 tx=0).stop 2024-03-24T10:43:54.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:54.083+0000 7fdab8b24700 1 -- 172.21.15.94:0/55515188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdab40fce40 msgr2=0x7fdab4071e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:54.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:54.083+0000 7fdab8b24700 1 --2- 172.21.15.94:0/55515188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdab40fce40 0x7fdab4071e40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4002a50 tx=0x7fdaa4004600 comp rx=0 tx=0).stop 2024-03-24T10:43:54.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:54.083+0000 7fdab8b24700 1 -- 172.21.15.94:0/55515188 shutdown_connections 2024-03-24T10:43:54.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:54.083+0000 7fdab8b24700 1 --2- 172.21.15.94:0/55515188 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fda9c0692a0 0x7fda9c06b750 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:54.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:54.083+0000 7fdab8b24700 1 --2- 172.21.15.94:0/55515188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdab40fce40 0x7fdab4071e40 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:54.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:54.083+0000 7fdab8b24700 1 --2- 172.21.15.94:0/55515188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdab40fe040 0x7fdab4070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:54.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:54.083+0000 7fdab8b24700 1 -- 172.21.15.94:0/55515188 >> 172.21.15.94:0/55515188 conn(0x7fdab40f8570 msgr2=0x7fdab40f9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:43:54.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:54.083+0000 7fdab8b24700 1 -- 172.21.15.94:0/55515188 shutdown_connections 2024-03-24T10:43:54.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:54.083+0000 7fdab8b24700 1 -- 172.21.15.94:0/55515188 wait complete. 2024-03-24T10:43:54.084 INFO:teuthology.orchestra.run.smithi094.stderr:inline data disabled 2024-03-24T10:43:54.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:53 smithi094 ceph-mon[29446]: pgmap v179: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 625 B/s rd, 1.1 KiB/s wr, 4 op/s 2024-03-24T10:43:54.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:53 smithi094 ceph-mon[29446]: mds.? [v2:172.21.15.96:6826/1321087930,v1:172.21.15.96:6827/1321087930] up:boot 2024-03-24T10:43:54.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:53 smithi094 ceph-mon[29446]: fsmap cephfs:1 {0=cephfs.smithi094.efiqte=up:active} 1 up:standby-replay 2 up:standby 2024-03-24T10:43:54.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:53 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi096.jtebhh"}]: dispatch 2024-03-24T10:43:54.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:53 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:43:54.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:53 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:43:54.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:53 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:43:54.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:53 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:43:54.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:53 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:43:54.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:53 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:43:54.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:53 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/55515188' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-03-24T10:43:54.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:53 smithi096 ceph-mon[38097]: pgmap v179: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 625 B/s rd, 1.1 KiB/s wr, 4 op/s 2024-03-24T10:43:54.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:53 smithi096 ceph-mon[38097]: mds.? [v2:172.21.15.96:6826/1321087930,v1:172.21.15.96:6827/1321087930] up:boot 2024-03-24T10:43:54.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:53 smithi096 ceph-mon[38097]: fsmap cephfs:1 {0=cephfs.smithi094.efiqte=up:active} 1 up:standby-replay 2 up:standby 2024-03-24T10:43:54.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:53 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi096.jtebhh"}]: dispatch 2024-03-24T10:43:54.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:53 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:43:54.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:53 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:43:54.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:53 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:43:54.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:53 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:43:54.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:53 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:43:54.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:53 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:43:54.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:53 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/55515188' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-03-24T10:43:54.717 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-24T10:43:54.728 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi094.front.sepia.ceph.com 2024-03-24T10:43:54.728 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-24T10:43:55.305 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:55 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/55515188' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-03-24T10:43:55.305 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:55 smithi094 ceph-mon[29446]: fsmap cephfs:1 {0=cephfs.smithi094.efiqte=up:active} 1 up:standby-replay 2 up:standby 2024-03-24T10:43:55.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:55 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/55515188' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-03-24T10:43:55.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:55 smithi096 ceph-mon[38097]: fsmap cephfs:1 {0=cephfs.smithi094.efiqte=up:active} 1 up:standby-replay 2 up:standby 2024-03-24T10:43:56.260 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:43:56.437 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:56 smithi094 ceph-mon[29446]: pgmap v180: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 682 B/s rd, 1.1 KiB/s wr, 4 op/s 2024-03-24T10:43:56.437 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:56 smithi094 ceph-mon[29446]: mds.? [v2:172.21.15.94:6828/126762980,v1:172.21.15.94:6829/126762980] up:standby 2024-03-24T10:43:56.437 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:56 smithi094 ceph-mon[29446]: mds.? [v2:172.21.15.96:6824/1539752208,v1:172.21.15.96:6825/1539752208] up:standby-replay 2024-03-24T10:43:56.437 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:56 smithi094 ceph-mon[29446]: Dropping low affinity active daemon mds.cephfs.smithi094.efiqte in favor of higher affinity standby. 2024-03-24T10:43:56.437 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:56 smithi094 ceph-mon[29446]: Replacing daemon mds.cephfs.smithi094.efiqte as rank 0 with standby daemon mds.cephfs.smithi096.egbmoq 2024-03-24T10:43:56.437 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:56 smithi094 ceph-mon[29446]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-24T10:43:56.437 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:56 smithi094 ceph-mon[29446]: fsmap cephfs:1 {0=cephfs.smithi094.efiqte=up:active} 1 up:standby-replay 2 up:standby 2024-03-24T10:43:56.437 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:56 smithi094 ceph-mon[29446]: osdmap e46: 6 total, 6 up, 6 in 2024-03-24T10:43:56.437 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:56 smithi094 ceph-mon[29446]: fsmap cephfs:1/1 {0=cephfs.smithi096.egbmoq=up:replay} 2 up:standby 2024-03-24T10:43:56.437 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:56 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:43:56.437 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:56 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:43:56.437 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:56 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:43:56.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:56 smithi096 ceph-mon[38097]: pgmap v180: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 682 B/s rd, 1.1 KiB/s wr, 4 op/s 2024-03-24T10:43:56.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:56 smithi096 ceph-mon[38097]: mds.? [v2:172.21.15.94:6828/126762980,v1:172.21.15.94:6829/126762980] up:standby 2024-03-24T10:43:56.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:56 smithi096 ceph-mon[38097]: mds.? [v2:172.21.15.96:6824/1539752208,v1:172.21.15.96:6825/1539752208] up:standby-replay 2024-03-24T10:43:56.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:56 smithi096 ceph-mon[38097]: Dropping low affinity active daemon mds.cephfs.smithi094.efiqte in favor of higher affinity standby. 2024-03-24T10:43:56.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:56 smithi096 ceph-mon[38097]: Replacing daemon mds.cephfs.smithi094.efiqte as rank 0 with standby daemon mds.cephfs.smithi096.egbmoq 2024-03-24T10:43:56.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:56 smithi096 ceph-mon[38097]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-24T10:43:56.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:56 smithi096 ceph-mon[38097]: fsmap cephfs:1 {0=cephfs.smithi094.efiqte=up:active} 1 up:standby-replay 2 up:standby 2024-03-24T10:43:56.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:56 smithi096 ceph-mon[38097]: osdmap e46: 6 total, 6 up, 6 in 2024-03-24T10:43:56.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:56 smithi096 ceph-mon[38097]: fsmap cephfs:1/1 {0=cephfs.smithi096.egbmoq=up:replay} 2 up:standby 2024-03-24T10:43:56.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:56 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:43:56.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:56 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:43:56.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:56 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:43:57.302 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:57 smithi094 ceph-mon[29446]: mds.? [v2:172.21.15.94:6826/1636528860,v1:172.21.15.94:6827/1636528860] up:boot 2024-03-24T10:43:57.302 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:57 smithi094 ceph-mon[29446]: mds.? [v2:172.21.15.96:6824/1539752208,v1:172.21.15.96:6825/1539752208] up:reconnect 2024-03-24T10:43:57.302 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:57 smithi094 ceph-mon[29446]: fsmap cephfs:1/1 {0=cephfs.smithi096.egbmoq=up:reconnect} 3 up:standby 2024-03-24T10:43:57.302 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:57 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.efiqte"}]: dispatch 2024-03-24T10:43:57.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:57 smithi096 ceph-mon[38097]: mds.? [v2:172.21.15.94:6826/1636528860,v1:172.21.15.94:6827/1636528860] up:boot 2024-03-24T10:43:57.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:57 smithi096 ceph-mon[38097]: mds.? [v2:172.21.15.96:6824/1539752208,v1:172.21.15.96:6825/1539752208] up:reconnect 2024-03-24T10:43:57.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:57 smithi096 ceph-mon[38097]: fsmap cephfs:1/1 {0=cephfs.smithi096.egbmoq=up:reconnect} 3 up:standby 2024-03-24T10:43:57.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:57 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.efiqte"}]: dispatch 2024-03-24T10:43:58.477 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:58 smithi096 ceph-mon[38097]: pgmap v182: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 102 B/s wr, 4 op/s 2024-03-24T10:43:58.477 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:58 smithi096 ceph-mon[38097]: mds.? [v2:172.21.15.96:6824/1539752208,v1:172.21.15.96:6825/1539752208] up:rejoin 2024-03-24T10:43:58.477 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:58 smithi096 ceph-mon[38097]: mds.? [v2:172.21.15.96:6826/1321087930,v1:172.21.15.96:6827/1321087930] up:standby 2024-03-24T10:43:58.477 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:58 smithi096 ceph-mon[38097]: fsmap cephfs:1/1 {0=cephfs.smithi096.egbmoq=up:rejoin} 3 up:standby 2024-03-24T10:43:58.477 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:58 smithi096 ceph-mon[38097]: daemon mds.cephfs.smithi096.egbmoq is now active in filesystem cephfs as rank 0 2024-03-24T10:43:58.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:58 smithi094 ceph-mon[29446]: pgmap v182: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 102 B/s wr, 4 op/s 2024-03-24T10:43:58.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:58 smithi094 ceph-mon[29446]: mds.? [v2:172.21.15.96:6824/1539752208,v1:172.21.15.96:6825/1539752208] up:rejoin 2024-03-24T10:43:58.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:58 smithi094 ceph-mon[29446]: mds.? [v2:172.21.15.96:6826/1321087930,v1:172.21.15.96:6827/1321087930] up:standby 2024-03-24T10:43:58.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:58 smithi094 ceph-mon[29446]: fsmap cephfs:1/1 {0=cephfs.smithi096.egbmoq=up:rejoin} 3 up:standby 2024-03-24T10:43:58.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:58 smithi094 ceph-mon[29446]: daemon mds.cephfs.smithi096.egbmoq is now active in filesystem cephfs as rank 0 2024-03-24T10:43:59.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:59.044+0000 7f58768d9700 1 -- 172.21.15.94:0/1599012989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58700fbcd0 msgr2=0x7f58700fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:59.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:59.044+0000 7f58768d9700 1 --2- 172.21.15.94:0/1599012989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58700fbcd0 0x7f58700fc0e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5860004940 tx=0x7f5860009e30 comp rx=0 tx=0).stop 2024-03-24T10:43:59.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:59.044+0000 7f58768d9700 1 -- 172.21.15.94:0/1599012989 shutdown_connections 2024-03-24T10:43:59.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:59.044+0000 7f58768d9700 1 --2- 172.21.15.94:0/1599012989 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58700fcf80 0x7f58700fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:59.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:59.044+0000 7f58768d9700 1 --2- 172.21.15.94:0/1599012989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58700fbcd0 0x7f58700fc0e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:59.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:59.044+0000 7f58768d9700 1 -- 172.21.15.94:0/1599012989 >> 172.21.15.94:0/1599012989 conn(0x7f58700f7450 msgr2=0x7f58700f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:43:59.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:59.044+0000 7f58768d9700 1 -- 172.21.15.94:0/1599012989 shutdown_connections 2024-03-24T10:43:59.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:59.044+0000 7f58768d9700 1 -- 172.21.15.94:0/1599012989 wait complete. 2024-03-24T10:43:59.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:59.044+0000 7f58768d9700 1 Processor -- start 2024-03-24T10:43:59.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:59.044+0000 7f58768d9700 1 -- start start 2024-03-24T10:43:59.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:59.045+0000 7f58768d9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58700fbcd0 0x7f587010da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:59.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:59.045+0000 7f58768d9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58700fcf80 0x7f587010df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:59.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:59.045+0000 7f58768d9700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f587010e5e0 con 0x7f58700fcf80 2024-03-24T10:43:59.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:59.045+0000 7f58768d9700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5870112e70 con 0x7f58700fbcd0 2024-03-24T10:43:59.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:59.045+0000 7f586f7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58700fcf80 0x7f587010df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:59.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:59.045+0000 7f586f7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58700fcf80 0x7f587010df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49762/0 (socket says 172.21.15.94:49762) 2024-03-24T10:43:59.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:59.045+0000 7f586f7fe700 1 -- 172.21.15.94:0/2510222871 learned_addr learned my addr 172.21.15.94:0/2510222871 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:43:59.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:59.045+0000 7f586ffff700 1 --2- 172.21.15.94:0/2510222871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58700fbcd0 0x7f587010da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:59.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:59.045+0000 7f586f7fe700 1 -- 172.21.15.94:0/2510222871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58700fbcd0 msgr2=0x7f587010da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:59.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:59.045+0000 7f586f7fe700 1 --2- 172.21.15.94:0/2510222871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58700fbcd0 0x7f587010da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:59.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:59.045+0000 7f586f7fe700 1 -- 172.21.15.94:0/2510222871 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5860005040 con 0x7f58700fcf80 2024-03-24T10:43:59.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:59.045+0000 7f586ffff700 1 --2- 172.21.15.94:0/2510222871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58700fbcd0 0x7f587010da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T10:43:59.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:59.045+0000 7f586f7fe700 1 --2- 172.21.15.94:0/2510222871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58700fcf80 0x7f587010df50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f586400b790 tx=0x7f5864009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:43:59.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:59.045+0000 7f586cff9700 1 -- 172.21.15.94:0/2510222871 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f586400f770 con 0x7f58700fcf80 2024-03-24T10:43:59.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:59.045+0000 7f58768d9700 1 -- 172.21.15.94:0/2510222871 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58701130d0 con 0x7f58700fcf80 2024-03-24T10:43:59.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:59.045+0000 7f586cff9700 1 -- 172.21.15.94:0/2510222871 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f586400fda0 con 0x7f58700fcf80 2024-03-24T10:43:59.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:59.045+0000 7f586cff9700 1 -- 172.21.15.94:0/2510222871 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f586400d310 con 0x7f58700fcf80 2024-03-24T10:43:59.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:59.045+0000 7f58768d9700 1 -- 172.21.15.94:0/2510222871 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5870113640 con 0x7f58700fcf80 2024-03-24T10:43:59.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:59.046+0000 7f58768d9700 1 -- 172.21.15.94:0/2510222871 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5870101b10 con 0x7f58700fcf80 2024-03-24T10:43:59.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:59.047+0000 7f586cff9700 1 -- 172.21.15.94:0/2510222871 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f5864010420 con 0x7f58700fcf80 2024-03-24T10:43:59.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:59.047+0000 7f586cff9700 1 --2- 172.21.15.94:0/2510222871 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f58580692a0 0x7f585806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:43:59.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:59.047+0000 7f586cff9700 1 -- 172.21.15.94:0/2510222871 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f5864087200 con 0x7f58700fcf80 2024-03-24T10:43:59.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:59.047+0000 7f586ffff700 1 --2- 172.21.15.94:0/2510222871 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f58580692a0 0x7f585806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:43:59.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:59.048+0000 7f586ffff700 1 --2- 172.21.15.94:0/2510222871 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f58580692a0 0x7f585806b750 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f586000b3b0 tx=0x7f5860009e30 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:43:59.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:59.052+0000 7f586cff9700 1 -- 172.21.15.94:0/2510222871 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f5864056f70 con 0x7f58700fcf80 2024-03-24T10:43:59.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:59.306+0000 7f58768d9700 1 -- 172.21.15.94:0/2510222871 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f587004c710 con 0x7f58700fcf80 2024-03-24T10:43:59.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:59.307+0000 7f586cff9700 1 -- 172.21.15.94:0/2510222871 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1833 (secure 0 0 0) 0x7f5864056b00 con 0x7f58700fcf80 2024-03-24T10:43:59.307 INFO:teuthology.orchestra.run.smithi094.stdout:e18 2024-03-24T10:43:59.307 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T10:43:59.307 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:43:59.307 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-03-24T10:43:59.307 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:43:59.308 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-03-24T10:43:59.308 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-03-24T10:43:59.308 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 18 2024-03-24T10:43:59.308 INFO:teuthology.orchestra.run.smithi094.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-24T10:43:59.308 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-03-24T10:43:39.435738+0000 2024-03-24T10:43:59.308 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-03-24T10:43:58.133901+0000 2024-03-24T10:43:59.308 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-03-24T10:43:59.308 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-03-24T10:43:59.308 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 60 2024-03-24T10:43:59.308 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-03-24T10:43:59.308 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-03-24T10:43:59.308 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-03-24T10:43:59.308 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-03-24T10:43:59.308 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 46 2024-03-24T10:43:59.308 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:43:59.309 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-03-24T10:43:59.309 INFO:teuthology.orchestra.run.smithi094.stdout:in 0 2024-03-24T10:43:59.309 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=24251} 2024-03-24T10:43:59.309 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-03-24T10:43:59.309 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-03-24T10:43:59.309 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-03-24T10:43:59.309 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-03-24T10:43:59.309 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-03-24T10:43:59.309 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-03-24T10:43:59.309 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-03-24T10:43:59.309 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 1 2024-03-24T10:43:59.309 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi096.egbmoq{0:24251} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.96:6824/1539752208,v1:172.21.15.96:6825/1539752208] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:43:59.309 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.llefgu{0:14500} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.94:6828/126762980,v1:172.21.15.94:6829/126762980] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:43:59.310 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:43:59.310 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:43:59.310 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-03-24T10:43:59.310 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:43:59.310 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.efiqte{-1:14512} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.94:6826/1636528860,v1:172.21.15.94:6827/1636528860] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:43:59.310 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi096.jtebhh{-1:24261} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.96:6826/1321087930,v1:172.21.15.96:6827/1321087930] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:43:59.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:59.309+0000 7f58768d9700 1 -- 172.21.15.94:0/2510222871 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f58580692a0 msgr2=0x7f585806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:59.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:59.309+0000 7f58768d9700 1 --2- 172.21.15.94:0/2510222871 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f58580692a0 0x7f585806b750 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f586000b3b0 tx=0x7f5860009e30 comp rx=0 tx=0).stop 2024-03-24T10:43:59.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:59.309+0000 7f58768d9700 1 -- 172.21.15.94:0/2510222871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58700fcf80 msgr2=0x7f587010df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:43:59.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:59.309+0000 7f58768d9700 1 --2- 172.21.15.94:0/2510222871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58700fcf80 0x7f587010df50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f586400b790 tx=0x7f5864009250 comp rx=0 tx=0).stop 2024-03-24T10:43:59.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:59.309+0000 7f58768d9700 1 -- 172.21.15.94:0/2510222871 shutdown_connections 2024-03-24T10:43:59.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:59.309+0000 7f58768d9700 1 --2- 172.21.15.94:0/2510222871 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f58580692a0 0x7f585806b750 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:59.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:59.309+0000 7f58768d9700 1 --2- 172.21.15.94:0/2510222871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58700fbcd0 0x7f587010da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:59.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:59.309+0000 7f58768d9700 1 --2- 172.21.15.94:0/2510222871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58700fcf80 0x7f587010df50 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:43:59.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:59.309+0000 7f58768d9700 1 -- 172.21.15.94:0/2510222871 >> 172.21.15.94:0/2510222871 conn(0x7f58700f7450 msgr2=0x7f58700667e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:43:59.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:59.309+0000 7f58768d9700 1 -- 172.21.15.94:0/2510222871 shutdown_connections 2024-03-24T10:43:59.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:43:59.309+0000 7f58768d9700 1 -- 172.21.15.94:0/2510222871 wait complete. 2024-03-24T10:43:59.312 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 18 2024-03-24T10:43:59.431 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:59 smithi094 ceph-mon[29446]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-24T10:43:59.431 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:59 smithi094 ceph-mon[29446]: Cluster is now healthy 2024-03-24T10:43:59.431 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:59 smithi094 ceph-mon[29446]: mds.? [v2:172.21.15.96:6824/1539752208,v1:172.21.15.96:6825/1539752208] up:active 2024-03-24T10:43:59.431 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:59 smithi094 ceph-mon[29446]: fsmap cephfs:1 {0=cephfs.smithi096.egbmoq=up:active} 3 up:standby 2024-03-24T10:43:59.431 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:59 smithi094 ceph-mon[29446]: fsmap cephfs:1 {0=cephfs.smithi096.egbmoq=up:active} 1 up:standby-replay 2 up:standby 2024-03-24T10:43:59.431 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:43:59 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:43:59.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:59 smithi096 ceph-mon[38097]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-24T10:43:59.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:59 smithi096 ceph-mon[38097]: Cluster is now healthy 2024-03-24T10:43:59.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:59 smithi096 ceph-mon[38097]: mds.? [v2:172.21.15.96:6824/1539752208,v1:172.21.15.96:6825/1539752208] up:active 2024-03-24T10:43:59.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:59 smithi096 ceph-mon[38097]: fsmap cephfs:1 {0=cephfs.smithi096.egbmoq=up:active} 3 up:standby 2024-03-24T10:43:59.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:59 smithi096 ceph-mon[38097]: fsmap cephfs:1 {0=cephfs.smithi096.egbmoq=up:active} 1 up:standby-replay 2 up:standby 2024-03-24T10:43:59.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:43:59 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:44:00.009 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-03-24T10:44:00.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:00 smithi094 ceph-mon[29446]: pgmap v183: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 4 op/s 2024-03-24T10:44:00.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:00 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:44:00.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:00 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:44:00.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:00 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:44:00.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:00 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:44:00.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:00 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:44:00.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:00 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/2510222871' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T10:44:00.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:00 smithi096 ceph-mon[38097]: pgmap v183: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 4 op/s 2024-03-24T10:44:00.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:00 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:44:00.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:00 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:44:00.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:00 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:44:00.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:00 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:44:00.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:00 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:44:00.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:00 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/2510222871' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T10:44:00.838 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:44:01.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:01 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:44:01.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:01 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:44:02.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:02.416+0000 7f0ac9797700 1 -- 172.21.15.94:0/3120875762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ac40fe050 msgr2=0x7f0ac40fe4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:02.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:02.416+0000 7f0ac9797700 1 --2- 172.21.15.94:0/3120875762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ac40fe050 0x7f0ac40fe4c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0aac004940 tx=0x7f0aac009e30 comp rx=0 tx=0).stop 2024-03-24T10:44:02.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:02.416+0000 7f0ac9797700 1 -- 172.21.15.94:0/3120875762 shutdown_connections 2024-03-24T10:44:02.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:02.416+0000 7f0ac9797700 1 --2- 172.21.15.94:0/3120875762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ac40fe050 0x7f0ac40fe4c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:02.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:02.416+0000 7f0ac9797700 1 --2- 172.21.15.94:0/3120875762 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ac40fce50 0x7f0ac40fd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:02.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:02.416+0000 7f0ac9797700 1 -- 172.21.15.94:0/3120875762 >> 172.21.15.94:0/3120875762 conn(0x7f0ac40f8560 msgr2=0x7f0ac40fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:44:02.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:02.416+0000 7f0ac9797700 1 -- 172.21.15.94:0/3120875762 shutdown_connections 2024-03-24T10:44:02.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:02.416+0000 7f0ac9797700 1 -- 172.21.15.94:0/3120875762 wait complete. 2024-03-24T10:44:02.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:02.416+0000 7f0ac9797700 1 Processor -- start 2024-03-24T10:44:02.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:02.416+0000 7f0ac9797700 1 -- start start 2024-03-24T10:44:02.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:02.417+0000 7f0ac9797700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ac40fce50 0x7f0ac4071c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:02.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:02.417+0000 7f0ac9797700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ac40fe050 0x7f0ac4070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:02.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:02.417+0000 7f0ac9797700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac4072260 con 0x7f0ac40fe050 2024-03-24T10:44:02.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:02.417+0000 7f0ac9797700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac4070850 con 0x7f0ac40fce50 2024-03-24T10:44:02.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:02.417+0000 7f0aba7fc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ac40fe050 0x7f0ac4070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:02.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:02.417+0000 7f0aba7fc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ac40fe050 0x7f0ac4070310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57466/0 (socket says 172.21.15.94:57466) 2024-03-24T10:44:02.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:02.417+0000 7f0aba7fc700 1 -- 172.21.15.94:0/1405009532 learned_addr learned my addr 172.21.15.94:0/1405009532 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:44:02.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:02.417+0000 7f0ac2ffd700 1 --2- 172.21.15.94:0/1405009532 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ac40fce50 0x7f0ac4071c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:02.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:02.417+0000 7f0aba7fc700 1 -- 172.21.15.94:0/1405009532 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ac40fce50 msgr2=0x7f0ac4071c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:02.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:02.417+0000 7f0aba7fc700 1 --2- 172.21.15.94:0/1405009532 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ac40fce50 0x7f0ac4071c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:02.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:02.417+0000 7f0aba7fc700 1 -- 172.21.15.94:0/1405009532 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aac005040 con 0x7f0ac40fe050 2024-03-24T10:44:02.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:02.417+0000 7f0aba7fc700 1 --2- 172.21.15.94:0/1405009532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ac40fe050 0x7f0ac4070310 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0aac009150 tx=0x7f0aac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:44:02.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:02.417+0000 7f0abbfff700 1 -- 172.21.15.94:0/1405009532 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0aac00e040 con 0x7f0ac40fe050 2024-03-24T10:44:02.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:02.417+0000 7f0abbfff700 1 -- 172.21.15.94:0/1405009532 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0aac004290 con 0x7f0ac40fe050 2024-03-24T10:44:02.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:02.417+0000 7f0ac9797700 1 -- 172.21.15.94:0/1405009532 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ac4070aa0 con 0x7f0ac40fe050 2024-03-24T10:44:02.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:02.417+0000 7f0abbfff700 1 -- 172.21.15.94:0/1405009532 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0aac01c630 con 0x7f0ac40fe050 2024-03-24T10:44:02.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:02.417+0000 7f0ac9797700 1 -- 172.21.15.94:0/1405009532 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac4071060 con 0x7f0ac40fe050 2024-03-24T10:44:02.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:02.419+0000 7f0ac9797700 1 -- 172.21.15.94:0/1405009532 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ac404c710 con 0x7f0ac40fe050 2024-03-24T10:44:02.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:02.419+0000 7f0abbfff700 1 -- 172.21.15.94:0/1405009532 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f0aac00bbf0 con 0x7f0ac40fe050 2024-03-24T10:44:02.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:02.419+0000 7f0abbfff700 1 --2- 172.21.15.94:0/1405009532 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f0ab00692a0 0x7f0ab006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:02.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:02.419+0000 7f0abbfff700 1 -- 172.21.15.94:0/1405009532 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f0aac087dc0 con 0x7f0ac40fe050 2024-03-24T10:44:02.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:02.420+0000 7f0ac2ffd700 1 --2- 172.21.15.94:0/1405009532 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f0ab00692a0 0x7f0ab006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:02.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:02.420+0000 7f0ac2ffd700 1 --2- 172.21.15.94:0/1405009532 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f0ab00692a0 0x7f0ab006b750 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0ac40fdbc0 tx=0x7f0ab4009040 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:44:02.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:02.424+0000 7f0abbfff700 1 -- 172.21.15.94:0/1405009532 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f0aac057bd0 con 0x7f0ac40fe050 2024-03-24T10:44:02.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:02 smithi094 ceph-mon[29446]: pgmap v184: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.4 KiB/s rd, 0 B/s wr, 4 op/s 2024-03-24T10:44:02.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:02 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:44:02.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:02 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:44:02.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:02.658+0000 7f0ac9797700 1 -- 172.21.15.94:0/1405009532 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ac4115a30 con 0x7f0ac40fe050 2024-03-24T10:44:02.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:02.659+0000 7f0abbfff700 1 -- 172.21.15.94:0/1405009532 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4693 (secure 0 0 0) 0x7f0aac057760 con 0x7f0ac40fe050 2024-03-24T10:44:02.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:02.661+0000 7f0ac9797700 1 -- 172.21.15.94:0/1405009532 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f0ab00692a0 msgr2=0x7f0ab006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:02.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:02.661+0000 7f0ac9797700 1 --2- 172.21.15.94:0/1405009532 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f0ab00692a0 0x7f0ab006b750 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0ac40fdbc0 tx=0x7f0ab4009040 comp rx=0 tx=0).stop 2024-03-24T10:44:02.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:02.661+0000 7f0ac9797700 1 -- 172.21.15.94:0/1405009532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ac40fe050 msgr2=0x7f0ac4070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:02.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:02.661+0000 7f0ac9797700 1 --2- 172.21.15.94:0/1405009532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ac40fe050 0x7f0ac4070310 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0aac009150 tx=0x7f0aac009e30 comp rx=0 tx=0).stop 2024-03-24T10:44:02.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:02.661+0000 7f0ac9797700 1 -- 172.21.15.94:0/1405009532 shutdown_connections 2024-03-24T10:44:02.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:02.661+0000 7f0ac9797700 1 --2- 172.21.15.94:0/1405009532 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f0ab00692a0 0x7f0ab006b750 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:02.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:02.661+0000 7f0ac9797700 1 --2- 172.21.15.94:0/1405009532 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ac40fce50 0x7f0ac4071c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:02.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:02.661+0000 7f0ac9797700 1 --2- 172.21.15.94:0/1405009532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ac40fe050 0x7f0ac4070310 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:02.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:02.661+0000 7f0ac9797700 1 -- 172.21.15.94:0/1405009532 >> 172.21.15.94:0/1405009532 conn(0x7f0ac40f8560 msgr2=0x7f0ac40f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:44:02.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:02.661+0000 7f0ac9797700 1 -- 172.21.15.94:0/1405009532 shutdown_connections 2024-03-24T10:44:02.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:02.661+0000 7f0ac9797700 1 -- 172.21.15.94:0/1405009532 wait complete. 2024-03-24T10:44:02.662 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 18 2024-03-24T10:44:02.672 INFO:teuthology.orchestra.run.smithi094.stdout:true 2024-03-24T10:44:02.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:02 smithi096 ceph-mon[38097]: pgmap v184: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.4 KiB/s rd, 0 B/s wr, 4 op/s 2024-03-24T10:44:02.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:02 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:44:02.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:02 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:44:03.309 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-03-24T10:44:03.685 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:44:03.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:03 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/1405009532' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T10:44:03.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:03 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/1405009532' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T10:44:04.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:04 smithi094 ceph-mon[29446]: pgmap v185: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.8 KiB/s rd, 6 op/s 2024-03-24T10:44:04.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:04 smithi096 ceph-mon[38097]: pgmap v185: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.8 KiB/s rd, 6 op/s 2024-03-24T10:44:06.728 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:06 smithi094 ceph-mon[29446]: pgmap v186: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.1 KiB/s rd, 6 op/s 2024-03-24T10:44:06.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:06 smithi096 ceph-mon[38097]: pgmap v186: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.1 KiB/s rd, 6 op/s 2024-03-24T10:44:06.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:06.940+0000 7f879e9d8700 1 -- 172.21.15.94:0/248863304 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87980fed90 msgr2=0x7f87980faed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:06.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:06.940+0000 7f879e9d8700 1 --2- 172.21.15.94:0/248863304 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87980fed90 0x7f87980faed0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f878c004940 tx=0x7f878c009e30 comp rx=0 tx=0).stop 2024-03-24T10:44:06.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:06.940+0000 7f879e9d8700 1 -- 172.21.15.94:0/248863304 shutdown_connections 2024-03-24T10:44:06.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:06.940+0000 7f879e9d8700 1 --2- 172.21.15.94:0/248863304 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87980fed90 0x7f87980faed0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:06.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:06.940+0000 7f879e9d8700 1 --2- 172.21.15.94:0/248863304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87980fe440 0x7f87980fe850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:06.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:06.940+0000 7f879e9d8700 1 -- 172.21.15.94:0/248863304 >> 172.21.15.94:0/248863304 conn(0x7f87980f6be0 msgr2=0x7f87980f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:44:06.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:06.940+0000 7f879e9d8700 1 -- 172.21.15.94:0/248863304 shutdown_connections 2024-03-24T10:44:06.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:06.940+0000 7f879e9d8700 1 -- 172.21.15.94:0/248863304 wait complete. 2024-03-24T10:44:06.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:06.941+0000 7f879e9d8700 1 Processor -- start 2024-03-24T10:44:06.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:06.941+0000 7f879e9d8700 1 -- start start 2024-03-24T10:44:06.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:06.941+0000 7f879e9d8700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87980fe440 0x7f879810d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:06.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:06.941+0000 7f879e9d8700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87980fed90 0x7f879810de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:06.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:06.941+0000 7f879e9d8700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879810e460 con 0x7f87980fe440 2024-03-24T10:44:06.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:06.941+0000 7f879e9d8700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879810e5d0 con 0x7f87980fed90 2024-03-24T10:44:06.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:06.941+0000 7f8797fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87980fe440 0x7f879810d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:06.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:06.941+0000 7f8797fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87980fe440 0x7f879810d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57478/0 (socket says 172.21.15.94:57478) 2024-03-24T10:44:06.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:06.941+0000 7f8797fff700 1 -- 172.21.15.94:0/2627261330 learned_addr learned my addr 172.21.15.94:0/2627261330 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:44:06.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:06.941+0000 7f87977fe700 1 --2- 172.21.15.94:0/2627261330 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87980fed90 0x7f879810de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:06.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:06.942+0000 7f8797fff700 1 -- 172.21.15.94:0/2627261330 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87980fed90 msgr2=0x7f879810de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:06.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:06.942+0000 7f8797fff700 1 --2- 172.21.15.94:0/2627261330 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87980fed90 0x7f879810de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:06.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:06.942+0000 7f8797fff700 1 -- 172.21.15.94:0/2627261330 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f878c005040 con 0x7f87980fe440 2024-03-24T10:44:06.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:06.942+0000 7f8797fff700 1 --2- 172.21.15.94:0/2627261330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87980fe440 0x7f879810d920 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8788002a50 tx=0x7f87880046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:44:06.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:06.942+0000 7f8794ff9700 1 -- 172.21.15.94:0/2627261330 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8788007cc0 con 0x7f87980fe440 2024-03-24T10:44:06.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:06.942+0000 7f8794ff9700 1 -- 172.21.15.94:0/2627261330 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f878800f040 con 0x7f87980fe440 2024-03-24T10:44:06.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:06.942+0000 7f879e9d8700 1 -- 172.21.15.94:0/2627261330 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8798112fd0 con 0x7f87980fe440 2024-03-24T10:44:06.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:06.942+0000 7f8794ff9700 1 -- 172.21.15.94:0/2627261330 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f878800da30 con 0x7f87980fe440 2024-03-24T10:44:06.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:06.942+0000 7f879e9d8700 1 -- 172.21.15.94:0/2627261330 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8798113560 con 0x7f87980fe440 2024-03-24T10:44:06.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:06.943+0000 7f879e9d8700 1 -- 172.21.15.94:0/2627261330 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f879804c710 con 0x7f87980fe440 2024-03-24T10:44:06.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:06.943+0000 7f8794ff9700 1 -- 172.21.15.94:0/2627261330 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f878800db90 con 0x7f87980fe440 2024-03-24T10:44:06.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:06.944+0000 7f8794ff9700 1 --2- 172.21.15.94:0/2627261330 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f87800692a0 0x7f878006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:06.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:06.944+0000 7f8794ff9700 1 -- 172.21.15.94:0/2627261330 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f8788086f90 con 0x7f87980fe440 2024-03-24T10:44:06.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:06.944+0000 7f87977fe700 1 --2- 172.21.15.94:0/2627261330 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f87800692a0 0x7f878006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:06.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:06.944+0000 7f87977fe700 1 --2- 172.21.15.94:0/2627261330 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f87800692a0 0x7f878006b750 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f878c00b970 tx=0x7f878c009e30 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:44:06.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:06.948+0000 7f8794ff9700 1 -- 172.21.15.94:0/2627261330 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f8788056c80 con 0x7f87980fe440 2024-03-24T10:44:07.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:07.265+0000 7f879e9d8700 1 -- 172.21.15.94:0/2627261330 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7f8798113750 con 0x7f87980fe440 2024-03-24T10:44:07.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:07.266+0000 7f8794ff9700 1 -- 172.21.15.94:0/2627261330 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v18) v1 ==== 78+0+99 (secure 0 0 0) 0x7f8788056810 con 0x7f87980fe440 2024-03-24T10:44:07.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:07.268+0000 7f879e9d8700 1 -- 172.21.15.94:0/2627261330 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f87800692a0 msgr2=0x7f878006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:07.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:07.268+0000 7f879e9d8700 1 --2- 172.21.15.94:0/2627261330 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f87800692a0 0x7f878006b750 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f878c00b970 tx=0x7f878c009e30 comp rx=0 tx=0).stop 2024-03-24T10:44:07.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:07.268+0000 7f879e9d8700 1 -- 172.21.15.94:0/2627261330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87980fe440 msgr2=0x7f879810d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:07.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:07.268+0000 7f879e9d8700 1 --2- 172.21.15.94:0/2627261330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87980fe440 0x7f879810d920 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8788002a50 tx=0x7f87880046b0 comp rx=0 tx=0).stop 2024-03-24T10:44:07.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:07.268+0000 7f879e9d8700 1 -- 172.21.15.94:0/2627261330 shutdown_connections 2024-03-24T10:44:07.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:07.268+0000 7f879e9d8700 1 --2- 172.21.15.94:0/2627261330 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f87800692a0 0x7f878006b750 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:07.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:07.268+0000 7f879e9d8700 1 --2- 172.21.15.94:0/2627261330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87980fe440 0x7f879810d920 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:07.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:07.268+0000 7f879e9d8700 1 --2- 172.21.15.94:0/2627261330 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87980fed90 0x7f879810de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:07.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:07.268+0000 7f879e9d8700 1 -- 172.21.15.94:0/2627261330 >> 172.21.15.94:0/2627261330 conn(0x7f87980f6be0 msgr2=0x7f87980f82b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:44:07.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:07.268+0000 7f879e9d8700 1 -- 172.21.15.94:0/2627261330 shutdown_connections 2024-03-24T10:44:07.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:07.268+0000 7f879e9d8700 1 -- 172.21.15.94:0/2627261330 wait complete. 2024-03-24T10:44:07.279 INFO:teuthology.orchestra.run.smithi094.stdout:true 2024-03-24T10:44:07.873 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-03-24T10:44:07.884 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-24T10:44:08.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.004+0000 7f958dcae640 1 -- 172.21.15.94:0/3686818027 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95880fc370 msgr2=0x7f95880fc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:08.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.004+0000 7f958dcae640 1 --2- 172.21.15.94:0/3686818027 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95880fc370 0x7f95880fc7e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f957c0048d0 tx=0x7f957c009da0 comp rx=0 tx=0).stop 2024-03-24T10:44:08.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.004+0000 7f958dcae640 1 -- 172.21.15.94:0/3686818027 shutdown_connections 2024-03-24T10:44:08.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.004+0000 7f958dcae640 1 --2- 172.21.15.94:0/3686818027 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95880fc370 0x7f95880fc7e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:08.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.004+0000 7f958dcae640 1 --2- 172.21.15.94:0/3686818027 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95880fb0a0 0x7f95880fb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:08.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.004+0000 7f958dcae640 1 -- 172.21.15.94:0/3686818027 >> 172.21.15.94:0/3686818027 conn(0x7f95880f6880 msgr2=0x7f95880f8c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:44:08.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.004+0000 7f958dcae640 1 -- 172.21.15.94:0/3686818027 shutdown_connections 2024-03-24T10:44:08.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.004+0000 7f958dcae640 1 -- 172.21.15.94:0/3686818027 wait complete. 2024-03-24T10:44:08.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.005+0000 7f958dcae640 1 Processor -- start 2024-03-24T10:44:08.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.005+0000 7f958dcae640 1 -- start start 2024-03-24T10:44:08.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.005+0000 7f958dcae640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95880fb0a0 0x7f9588065f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:08.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.005+0000 7f958dcae640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95880fc370 0x7f9588111b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:08.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.005+0000 7f958dcae640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9588112050 con 0x7f95880fc370 2024-03-24T10:44:08.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.005+0000 7f958dcae640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9588112190 con 0x7f95880fb0a0 2024-03-24T10:44:08.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.005+0000 7f9586ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95880fc370 0x7f9588111b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:08.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.005+0000 7f9586ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95880fc370 0x7f9588111b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57494/0 (socket says 172.21.15.94:57494) 2024-03-24T10:44:08.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.005+0000 7f9586ffd640 1 -- 172.21.15.94:0/4078019367 learned_addr learned my addr 172.21.15.94:0/4078019367 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:44:08.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.005+0000 7f95877fe640 1 --2- 172.21.15.94:0/4078019367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95880fb0a0 0x7f9588065f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:08.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.005+0000 7f9586ffd640 1 -- 172.21.15.94:0/4078019367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95880fb0a0 msgr2=0x7f9588065f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:08.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.005+0000 7f9586ffd640 1 --2- 172.21.15.94:0/4078019367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95880fb0a0 0x7f9588065f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:08.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.005+0000 7f9586ffd640 1 -- 172.21.15.94:0/4078019367 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f957c005040 con 0x7f95880fc370 2024-03-24T10:44:08.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.006+0000 7f9586ffd640 1 --2- 172.21.15.94:0/4078019367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95880fc370 0x7f9588111b10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f957c0048a0 tx=0x7f957c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:44:08.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.006+0000 7f956ffff640 1 -- 172.21.15.94:0/4078019367 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f957c0094c0 con 0x7f95880fc370 2024-03-24T10:44:08.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.006+0000 7f956ffff640 1 -- 172.21.15.94:0/4078019367 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f957c009620 con 0x7f95880fc370 2024-03-24T10:44:08.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.006+0000 7f958dcae640 1 -- 172.21.15.94:0/4078019367 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9588112390 con 0x7f95880fc370 2024-03-24T10:44:08.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.006+0000 7f956ffff640 1 -- 172.21.15.94:0/4078019367 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f957c03b640 con 0x7f95880fc370 2024-03-24T10:44:08.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.006+0000 7f958dcae640 1 -- 172.21.15.94:0/4078019367 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9588112880 con 0x7f95880fc370 2024-03-24T10:44:08.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.007+0000 7f958dcae640 1 -- 172.21.15.94:0/4078019367 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95881055a0 con 0x7f95880fc370 2024-03-24T10:44:08.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.007+0000 7f956ffff640 1 -- 172.21.15.94:0/4078019367 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f957c02f070 con 0x7f95880fc370 2024-03-24T10:44:08.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.007+0000 7f956ffff640 1 --2- 172.21.15.94:0/4078019367 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f9570069210 0x7f957006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:08.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.008+0000 7f956ffff640 1 -- 172.21.15.94:0/4078019367 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f957c0a9740 con 0x7f95880fc370 2024-03-24T10:44:08.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.008+0000 7f95877fe640 1 --2- 172.21.15.94:0/4078019367 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f9570069210 0x7f957006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:08.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.008+0000 7f95877fe640 1 --2- 172.21.15.94:0/4078019367 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f9570069210 0x7f957006b6c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9578009af0 tx=0x7f957800b040 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:44:08.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.012+0000 7f956ffff640 1 -- 172.21.15.94:0/4078019367 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f957c0759a0 con 0x7f95880fc370 2024-03-24T10:44:08.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.179+0000 7f958dcae640 1 -- 172.21.15.94:0/4078019367 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9588103c10 con 0x7f95880fc370 2024-03-24T10:44:08.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.180+0000 7f956ffff640 1 -- 172.21.15.94:0/4078019367 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+14363 (secure 0 0 0) 0x7f957c078fc0 con 0x7f95880fc370 2024-03-24T10:44:08.181 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:44:08.181 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":46,"fsid":"386110d4-e9ca-11ee-95cd-87774f69a715","created":"2024-03-24T10:36:12.011908+0000","modified":"2024-03-24T10:43:55.084226+0000","last_up_change":"2024-03-24T10:42:23.998853+0000","last_in_change":"2024-03-24T10:42:04.898193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-24T10:41:09.161003+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-24T10:43:37.651734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-24T10:43:38.660001+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"58fd53f9-8ad5-4296-8804-96a98647bba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3857839890},{"type":"v1","addr":"172.21.15.94:6803","nonce":3857839890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3857839890},{"type":"v1","addr":"172.21.15.94:6805","nonce":3857839890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":3857839890},{"type":"v1","addr":"172.21.15.94:6809","nonce":3857839890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3857839890},{"type":"v1","addr":"172.21.15.94:6807","nonce":3857839890}]},"public_addr":"172.21.15.94:6803/3857839890","cluster_addr":"172.21.15.94:6805/3857839890","heartbeat_back_addr":"172.21.15.94:6809/3857839890","heartbeat_front_addr":"172.21.15.94:6807/3857839890","state":["exists","up"]},{"osd":1,"uuid":"3087b482-aede-4e88-848c-3e7438eaccef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":2028968832},{"type":"v1","addr":"172.21.15.94:6811","nonce":2028968832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":2028968832},{"type":"v1","addr":"172.21.15.94:6813","nonce":2028968832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":2028968832},{"type":"v1","addr":"172.21.15.94:6817","nonce":2028968832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":2028968832},{"type":"v1","addr":"172.21.15.94:6815","nonce":2028968832}]},"public_addr":"172.21.15.94:6811/2028968832","cluster_addr":"172.21.15.94:6813/2028968832","heartbeat_back_addr":"172.21.15.94:6817/2028968832","heartbeat_front_addr":"172.21.15.94:6815/2028968832","state":["exists","up"]},{"osd":2,"uuid":"48c00627-22a6-480c-9aef-38bc326fc4dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":2122131371},{"type":"v1","addr":"172.21.15.94:6819","nonce":2122131371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":2122131371},{"type":"v1","addr":"172.21.15.94:6821","nonce":2122131371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2122131371},{"type":"v1","addr":"172.21.15.94:6825","nonce":2122131371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":2122131371},{"type":"v1","addr":"172.21.15.94:6823","nonce":2122131371}]},"public_addr":"172.21.15.94:6819/2122131371","cluster_addr":"172.21.15.94:6821/2122131371","heartbeat_back_addr":"172.21.15.94:6825/2122131371","heartbeat_front_addr":"172.21.15.94:6823/2122131371","state":["exists","up"]},{"osd":3,"uuid":"910859cd-d028-45f8-8e28-af4e46be460b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":3684617676},{"type":"v1","addr":"172.21.15.96:6801","nonce":3684617676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3684617676},{"type":"v1","addr":"172.21.15.96:6803","nonce":3684617676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3684617676},{"type":"v1","addr":"172.21.15.96:6807","nonce":3684617676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3684617676},{"type":"v1","addr":"172.21.15.96:6805","nonce":3684617676}]},"public_addr":"172.21.15.96:6801/3684617676","cluster_addr":"172.21.15.96:6803/3684617676","heartbeat_back_addr":"172.21.15.96:6807/3684617676","heartbeat_front_addr":"172.21.15.96:6805/3684617676","state":["exists","up"]},{"osd":4,"uuid":"9d37e21d-38dc-4e5b-b5ef-3828d7b25096","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":135461738},{"type":"v1","addr":"172.21.15.96:6809","nonce":135461738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":135461738},{"type":"v1","addr":"172.21.15.96:6811","nonce":135461738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":135461738},{"type":"v1","addr":"172.21.15.96:6815","nonce":135461738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":135461738},{"type":"v1","addr":"172.21.15.96:6813","nonce":135461738}]},"public_addr":"172.21.15.96:6809/135461738","cluster_addr":"172.21.15.96:6811/135461738","heartbeat_back_addr":"172.21.15.96:6815/135461738","heartbeat_front_addr":"172.21.15.96:6813/135461738","state":["exists","up"]},{"osd":5,"uuid":"838cf934-b936-4257-8510-3d5bbf4ef034","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3738830217},{"type":"v1","addr":"172.21.15.96:6817","nonce":3738830217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3738830217},{"type":"v1","addr":"172.21.15.96:6819","nonce":3738830217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3738830217},{"type":"v1","addr":"172.21.15.96:6823","nonce":3738830217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3738830217},{"type":"v1","addr":"172.21.15.96:6821","nonce":3738830217}]},"public_addr":"172.21.15.96:6817/3738830217","cluster_addr":"172.21.15.96:6819/3738830217","heartbeat_back_addr":"172.21.15.96:6823/3738830217","heartbeat_front_addr":"172.21.15.96:6821/3738830217","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T10:40:12.038732+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T10:40:38.407784+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T10:41:06.693615+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T10:41:30.382956+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T10:41:56.733747+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T10:42:22.324376+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:6827/2001471799":"2024-03-25T10:43:55.084207+0000","172.21.15.94:6800/1996608546":"2024-03-25T10:37:18.183215+0000","172.21.15.94:6801/1013449112":"2024-03-25T10:36:48.432382+0000","172.21.15.94:0/4239612108":"2024-03-25T10:36:48.432382+0000","172.21.15.94:6801/1996608546":"2024-03-25T10:37:18.183215+0000","172.21.15.94:0/2837872991":"2024-03-25T10:36:48.432382+0000","172.21.15.94:0/2130298637":"2024-03-25T10:39:00.830398+0000","172.21.15.94:6800/1013449112":"2024-03-25T10:36:48.432382+0000","172.21.15.94:6801/2406899470":"2024-03-25T10:39:00.830398+0000","172.21.15.94:0/2155575468":"2024-03-25T10:39:00.830398+0000","172.21.15.94:0/2641430900":"2024-03-25T10:36:48.432382+0000","172.21.15.94:0/629634217":"2024-03-25T10:37:18.183215+0000","172.21.15.94:0/3628417758":"2024-03-25T10:37:18.183215+0000","172.21.15.94:0/2661420513":"2024-03-25T10:37:18.183215+0000","172.21.15.94:6800/2406899470":"2024-03-25T10:39:00.830398+0000","172.21.15.94:0/2700857461":"2024-03-25T10:39:00.830398+0000","172.21.15.94:6826/2001471799":"2024-03-25T10:43:55.084207+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-24T10:44:08.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.182+0000 7f958dcae640 1 -- 172.21.15.94:0/4078019367 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f9570069210 msgr2=0x7f957006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:08.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.182+0000 7f958dcae640 1 --2- 172.21.15.94:0/4078019367 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f9570069210 0x7f957006b6c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9578009af0 tx=0x7f957800b040 comp rx=0 tx=0).stop 2024-03-24T10:44:08.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.182+0000 7f958dcae640 1 -- 172.21.15.94:0/4078019367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95880fc370 msgr2=0x7f9588111b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:08.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.182+0000 7f958dcae640 1 --2- 172.21.15.94:0/4078019367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95880fc370 0x7f9588111b10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f957c0048a0 tx=0x7f957c009da0 comp rx=0 tx=0).stop 2024-03-24T10:44:08.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.182+0000 7f958dcae640 1 -- 172.21.15.94:0/4078019367 shutdown_connections 2024-03-24T10:44:08.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.182+0000 7f958dcae640 1 --2- 172.21.15.94:0/4078019367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95880fc370 0x7f9588111b10 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:08.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.182+0000 7f958dcae640 1 --2- 172.21.15.94:0/4078019367 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f9570069210 0x7f957006b6c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:08.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.182+0000 7f958dcae640 1 --2- 172.21.15.94:0/4078019367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95880fb0a0 0x7f9588065f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:08.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.182+0000 7f958dcae640 1 -- 172.21.15.94:0/4078019367 >> 172.21.15.94:0/4078019367 conn(0x7f95880f6880 msgr2=0x7f95880ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:44:08.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.182+0000 7f958dcae640 1 -- 172.21.15.94:0/4078019367 shutdown_connections 2024-03-24T10:44:08.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.182+0000 7f958dcae640 1 -- 172.21.15.94:0/4078019367 wait complete. 2024-03-24T10:44:08.193 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:08 smithi094 ceph-mon[29446]: pgmap v187: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.4 KiB/s rd, 0 B/s wr, 6 op/s 2024-03-24T10:44:08.193 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:08 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:44:08.193 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:08 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:44:08.193 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:08 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:44:08.193 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:08 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:44:08.193 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:08 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:44:08.193 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:08 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/2627261330' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-03-24T10:44:08.194 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-24T10:41:09.161003+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-24T10:43:37.651734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-24T10:43:38.660001+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-24T10:44:08.195 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-24T10:44:08.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.293+0000 7fe7fbeae640 1 -- 172.21.15.94:0/1510350796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7f40ff2e0 msgr2=0x7fe7f41016c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:08.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.293+0000 7fe7fbeae640 1 --2- 172.21.15.94:0/1510350796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7f40ff2e0 0x7fe7f41016c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe7e40048d0 tx=0x7fe7e4009da0 comp rx=0 tx=0).stop 2024-03-24T10:44:08.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.293+0000 7fe7fbeae640 1 -- 172.21.15.94:0/1510350796 shutdown_connections 2024-03-24T10:44:08.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.293+0000 7fe7fbeae640 1 --2- 172.21.15.94:0/1510350796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7f40ff2e0 0x7fe7f41016c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:08.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.293+0000 7fe7fbeae640 1 --2- 172.21.15.94:0/1510350796 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7f4065030 0x7fe7f40feda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:08.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.293+0000 7fe7fbeae640 1 -- 172.21.15.94:0/1510350796 >> 172.21.15.94:0/1510350796 conn(0x7fe7f40f6870 msgr2=0x7fe7f40f8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:44:08.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.293+0000 7fe7fbeae640 1 -- 172.21.15.94:0/1510350796 shutdown_connections 2024-03-24T10:44:08.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.293+0000 7fe7fbeae640 1 -- 172.21.15.94:0/1510350796 wait complete. 2024-03-24T10:44:08.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.294+0000 7fe7fbeae640 1 Processor -- start 2024-03-24T10:44:08.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.294+0000 7fe7fbeae640 1 -- start start 2024-03-24T10:44:08.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.294+0000 7fe7fbeae640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7f4065030 0x7fe7f410cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:08.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.294+0000 7fe7fbeae640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7f40ff2e0 0x7fe7f410d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:08.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.294+0000 7fe7fbeae640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f410d8e0 con 0x7fe7f4065030 2024-03-24T10:44:08.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.294+0000 7fe7fbeae640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f4112250 con 0x7fe7f40ff2e0 2024-03-24T10:44:08.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.294+0000 7fe7f9c4a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7f4065030 0x7fe7f410cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:08.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.294+0000 7fe7f9c4a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7f4065030 0x7fe7f410cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57512/0 (socket says 172.21.15.94:57512) 2024-03-24T10:44:08.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.294+0000 7fe7f9c4a640 1 -- 172.21.15.94:0/2210990867 learned_addr learned my addr 172.21.15.94:0/2210990867 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:44:08.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.294+0000 7fe7f9449640 1 --2- 172.21.15.94:0/2210990867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7f40ff2e0 0x7fe7f410d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:08.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.294+0000 7fe7f9c4a640 1 -- 172.21.15.94:0/2210990867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7f40ff2e0 msgr2=0x7fe7f410d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:08.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.294+0000 7fe7f9c4a640 1 --2- 172.21.15.94:0/2210990867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7f40ff2e0 0x7fe7f410d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:08.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.294+0000 7fe7f9c4a640 1 -- 172.21.15.94:0/2210990867 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e4005040 con 0x7fe7f4065030 2024-03-24T10:44:08.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.295+0000 7fe7f9c4a640 1 --2- 172.21.15.94:0/2210990867 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7f4065030 0x7fe7f410cd20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0002870 tx=0x7fe7f0004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:44:08.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.295+0000 7fe7ea7fc640 1 -- 172.21.15.94:0/2210990867 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7f0007c30 con 0x7fe7f4065030 2024-03-24T10:44:08.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.295+0000 7fe7ea7fc640 1 -- 172.21.15.94:0/2210990867 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe7f000f040 con 0x7fe7f4065030 2024-03-24T10:44:08.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.295+0000 7fe7fbeae640 1 -- 172.21.15.94:0/2210990867 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7f41123f0 con 0x7fe7f4065030 2024-03-24T10:44:08.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.295+0000 7fe7ea7fc640 1 -- 172.21.15.94:0/2210990867 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7f00094d0 con 0x7fe7f4065030 2024-03-24T10:44:08.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.295+0000 7fe7fbeae640 1 -- 172.21.15.94:0/2210990867 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7f41128e0 con 0x7fe7f4065030 2024-03-24T10:44:08.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.296+0000 7fe7fbeae640 1 -- 172.21.15.94:0/2210990867 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7f404bcc0 con 0x7fe7f4065030 2024-03-24T10:44:08.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.296+0000 7fe7ea7fc640 1 -- 172.21.15.94:0/2210990867 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fe7f000dc80 con 0x7fe7f4065030 2024-03-24T10:44:08.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.296+0000 7fe7ea7fc640 1 --2- 172.21.15.94:0/2210990867 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fe7e0069210 0x7fe7e006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:08.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.296+0000 7fe7f9449640 1 --2- 172.21.15.94:0/2210990867 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fe7e0069210 0x7fe7e006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:08.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.296+0000 7fe7ea7fc640 1 -- 172.21.15.94:0/2210990867 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fe7f00870d0 con 0x7fe7f4065030 2024-03-24T10:44:08.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.297+0000 7fe7f9449640 1 --2- 172.21.15.94:0/2210990867 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fe7e0069210 0x7fe7e006b6c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe7f410e1c0 tx=0x7fe7e4009da0 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:44:08.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.301+0000 7fe7ea7fc640 1 -- 172.21.15.94:0/2210990867 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fe7f00533b0 con 0x7fe7f4065030 2024-03-24T10:44:08.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.469+0000 7fe7fbeae640 1 -- 172.21.15.94:0/2210990867 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe7f4112b50 con 0x7fe7f4065030 2024-03-24T10:44:08.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.469+0000 7fe7ea7fc640 1 -- 172.21.15.94:0/2210990867 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe7f00569d0 con 0x7fe7f4065030 2024-03-24T10:44:08.470 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-03-24T10:44:08.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.471+0000 7fe7fbeae640 1 -- 172.21.15.94:0/2210990867 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fe7e0069210 msgr2=0x7fe7e006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:08.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.471+0000 7fe7fbeae640 1 --2- 172.21.15.94:0/2210990867 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fe7e0069210 0x7fe7e006b6c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe7f410e1c0 tx=0x7fe7e4009da0 comp rx=0 tx=0).stop 2024-03-24T10:44:08.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.471+0000 7fe7fbeae640 1 -- 172.21.15.94:0/2210990867 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7f4065030 msgr2=0x7fe7f410cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:08.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.471+0000 7fe7fbeae640 1 --2- 172.21.15.94:0/2210990867 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7f4065030 0x7fe7f410cd20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0002870 tx=0x7fe7f0004570 comp rx=0 tx=0).stop 2024-03-24T10:44:08.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.471+0000 7fe7fbeae640 1 -- 172.21.15.94:0/2210990867 shutdown_connections 2024-03-24T10:44:08.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.471+0000 7fe7fbeae640 1 --2- 172.21.15.94:0/2210990867 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fe7e0069210 0x7fe7e006b6c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:08.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.471+0000 7fe7fbeae640 1 --2- 172.21.15.94:0/2210990867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7f40ff2e0 0x7fe7f410d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:08.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.471+0000 7fe7fbeae640 1 --2- 172.21.15.94:0/2210990867 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7f4065030 0x7fe7f410cd20 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:08.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.471+0000 7fe7fbeae640 1 -- 172.21.15.94:0/2210990867 >> 172.21.15.94:0/2210990867 conn(0x7fe7f40f6870 msgr2=0x7fe7f40f6c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:44:08.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.471+0000 7fe7fbeae640 1 -- 172.21.15.94:0/2210990867 shutdown_connections 2024-03-24T10:44:08.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.471+0000 7fe7fbeae640 1 -- 172.21.15.94:0/2210990867 wait complete. 2024-03-24T10:44:08.483 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-24T10:44:08.529 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:08 smithi096 ceph-mon[38097]: pgmap v187: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.4 KiB/s rd, 0 B/s wr, 6 op/s 2024-03-24T10:44:08.529 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:08 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:44:08.529 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:08 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:44:08.529 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:08 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:44:08.529 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:08 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:44:08.529 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:08 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:44:08.529 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:08 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/2627261330' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-03-24T10:44:08.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.581+0000 7f022f218640 1 -- 172.21.15.94:0/1316373845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02280fc360 msgr2=0x7f02280fc7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:08.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.581+0000 7f022f218640 1 --2- 172.21.15.94:0/1316373845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02280fc360 0x7f02280fc7d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f021c0048d0 tx=0x7f021c009da0 comp rx=0 tx=0).stop 2024-03-24T10:44:08.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.581+0000 7f022f218640 1 -- 172.21.15.94:0/1316373845 shutdown_connections 2024-03-24T10:44:08.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.581+0000 7f022f218640 1 --2- 172.21.15.94:0/1316373845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02280fc360 0x7f02280fc7d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:08.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.581+0000 7f022f218640 1 --2- 172.21.15.94:0/1316373845 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02280fb090 0x7f02280fb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:08.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.581+0000 7f022f218640 1 -- 172.21.15.94:0/1316373845 >> 172.21.15.94:0/1316373845 conn(0x7f02280f6870 msgr2=0x7f02280f8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:44:08.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.581+0000 7f022f218640 1 -- 172.21.15.94:0/1316373845 shutdown_connections 2024-03-24T10:44:08.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.581+0000 7f022f218640 1 -- 172.21.15.94:0/1316373845 wait complete. 2024-03-24T10:44:08.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.582+0000 7f022f218640 1 Processor -- start 2024-03-24T10:44:08.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.582+0000 7f022f218640 1 -- start start 2024-03-24T10:44:08.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.582+0000 7f022f218640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02280fb090 0x7f022810ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:08.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.582+0000 7f022f218640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02280fc360 0x7f022810d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:08.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.582+0000 7f022f218640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022810d860 con 0x7f02280fb090 2024-03-24T10:44:08.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.582+0000 7f022f218640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02281121d0 con 0x7f02280fc360 2024-03-24T10:44:08.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.582+0000 7f022cfb4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02280fb090 0x7f022810ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:08.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.582+0000 7f022cfb4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02280fb090 0x7f022810ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57532/0 (socket says 172.21.15.94:57532) 2024-03-24T10:44:08.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.582+0000 7f022cfb4640 1 -- 172.21.15.94:0/4227710655 learned_addr learned my addr 172.21.15.94:0/4227710655 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:44:08.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.582+0000 7f0227fff640 1 --2- 172.21.15.94:0/4227710655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02280fc360 0x7f022810d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:08.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.582+0000 7f022cfb4640 1 -- 172.21.15.94:0/4227710655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02280fc360 msgr2=0x7f022810d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:08.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.582+0000 7f022cfb4640 1 --2- 172.21.15.94:0/4227710655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02280fc360 0x7f022810d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:08.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.582+0000 7f022cfb4640 1 -- 172.21.15.94:0/4227710655 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f021c005040 con 0x7f02280fb090 2024-03-24T10:44:08.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.583+0000 7f022cfb4640 1 --2- 172.21.15.94:0/4227710655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02280fb090 0x7f022810ccd0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f021800c810 tx=0x7f02180043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:44:08.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.583+0000 7f02257fa640 1 -- 172.21.15.94:0/4227710655 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f021800a820 con 0x7f02280fb090 2024-03-24T10:44:08.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.583+0000 7f02257fa640 1 -- 172.21.15.94:0/4227710655 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f021800ae50 con 0x7f02280fb090 2024-03-24T10:44:08.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.583+0000 7f022f218640 1 -- 172.21.15.94:0/4227710655 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0228112370 con 0x7f02280fb090 2024-03-24T10:44:08.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.583+0000 7f02257fa640 1 -- 172.21.15.94:0/4227710655 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02180115c0 con 0x7f02280fb090 2024-03-24T10:44:08.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.583+0000 7f022f218640 1 -- 172.21.15.94:0/4227710655 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0228112840 con 0x7f02280fb090 2024-03-24T10:44:08.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.584+0000 7f022f218640 1 -- 172.21.15.94:0/4227710655 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f022804bcc0 con 0x7f02280fb090 2024-03-24T10:44:08.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.584+0000 7f02257fa640 1 -- 172.21.15.94:0/4227710655 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f021800a980 con 0x7f02280fb090 2024-03-24T10:44:08.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.584+0000 7f02257fa640 1 --2- 172.21.15.94:0/4227710655 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f0210069210 0x7f021006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:08.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.585+0000 7f02257fa640 1 -- 172.21.15.94:0/4227710655 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f0218087740 con 0x7f02280fb090 2024-03-24T10:44:08.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.585+0000 7f0227fff640 1 --2- 172.21.15.94:0/4227710655 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f0210069210 0x7f021006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:08.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.585+0000 7f0227fff640 1 --2- 172.21.15.94:0/4227710655 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f0210069210 0x7f021006b6c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f021c0317e0 tx=0x7f021c009da0 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:44:08.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.589+0000 7f02257fa640 1 -- 172.21.15.94:0/4227710655 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f02180539a0 con 0x7f02280fb090 2024-03-24T10:44:08.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.756+0000 7f022f218640 1 -- 172.21.15.94:0/4227710655 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f0228112ac0 con 0x7f02280fb090 2024-03-24T10:44:08.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.756+0000 7f02257fa640 1 -- 172.21.15.94:0/4227710655 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v46) v1 ==== 107+0+11 (secure 0 0 0) 0x7f0218016020 con 0x7f02280fb090 2024-03-24T10:44:08.757 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-03-24T10:44:08.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.758+0000 7f022f218640 1 -- 172.21.15.94:0/4227710655 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f0210069210 msgr2=0x7f021006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:08.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.758+0000 7f022f218640 1 --2- 172.21.15.94:0/4227710655 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f0210069210 0x7f021006b6c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f021c0317e0 tx=0x7f021c009da0 comp rx=0 tx=0).stop 2024-03-24T10:44:08.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.758+0000 7f022f218640 1 -- 172.21.15.94:0/4227710655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02280fb090 msgr2=0x7f022810ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:08.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.758+0000 7f022f218640 1 --2- 172.21.15.94:0/4227710655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02280fb090 0x7f022810ccd0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f021800c810 tx=0x7f02180043c0 comp rx=0 tx=0).stop 2024-03-24T10:44:08.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.758+0000 7f022f218640 1 -- 172.21.15.94:0/4227710655 shutdown_connections 2024-03-24T10:44:08.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.758+0000 7f022f218640 1 --2- 172.21.15.94:0/4227710655 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f0210069210 0x7f021006b6c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:08.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.758+0000 7f022f218640 1 --2- 172.21.15.94:0/4227710655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02280fc360 0x7f022810d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:08.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.758+0000 7f022f218640 1 --2- 172.21.15.94:0/4227710655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02280fb090 0x7f022810ccd0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:08.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.759+0000 7f022f218640 1 -- 172.21.15.94:0/4227710655 >> 172.21.15.94:0/4227710655 conn(0x7f02280f6870 msgr2=0x7f0228064370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:44:08.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.759+0000 7f022f218640 1 -- 172.21.15.94:0/4227710655 shutdown_connections 2024-03-24T10:44:08.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.759+0000 7f022f218640 1 -- 172.21.15.94:0/4227710655 wait complete. 2024-03-24T10:44:08.770 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-24T10:44:08.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.869+0000 7ff0f34ae640 1 -- 172.21.15.94:0/1723252419 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0ec0fe1d0 msgr2=0x7ff0ec0fa330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:08.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.869+0000 7ff0f34ae640 1 --2- 172.21.15.94:0/1723252419 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0ec0fe1d0 0x7ff0ec0fa330 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc0048d0 tx=0x7ff0dc009da0 comp rx=0 tx=0).stop 2024-03-24T10:44:08.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.870+0000 7ff0f34ae640 1 -- 172.21.15.94:0/1723252419 shutdown_connections 2024-03-24T10:44:08.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.870+0000 7ff0f34ae640 1 --2- 172.21.15.94:0/1723252419 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0ec0fe1d0 0x7ff0ec0fa330 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:08.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.870+0000 7ff0f34ae640 1 --2- 172.21.15.94:0/1723252419 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0ec0fd880 0x7ff0ec0fdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:08.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.870+0000 7ff0f34ae640 1 -- 172.21.15.94:0/1723252419 >> 172.21.15.94:0/1723252419 conn(0x7ff0ec0f6000 msgr2=0x7ff0ec0f8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:44:08.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.870+0000 7ff0f34ae640 1 -- 172.21.15.94:0/1723252419 shutdown_connections 2024-03-24T10:44:08.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.870+0000 7ff0f34ae640 1 -- 172.21.15.94:0/1723252419 wait complete. 2024-03-24T10:44:08.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.870+0000 7ff0f34ae640 1 Processor -- start 2024-03-24T10:44:08.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.870+0000 7ff0f34ae640 1 -- start start 2024-03-24T10:44:08.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.870+0000 7ff0f34ae640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0ec0fd880 0x7ff0ec10cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:08.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.870+0000 7ff0f34ae640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0ec0fe1d0 0x7ff0ec10d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:08.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.870+0000 7ff0f34ae640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ec10d980 con 0x7ff0ec0fe1d0 2024-03-24T10:44:08.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.870+0000 7ff0f34ae640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ec1122f0 con 0x7ff0ec0fd880 2024-03-24T10:44:08.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.870+0000 7ff0f0a49640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0ec0fe1d0 0x7ff0ec10d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:08.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.870+0000 7ff0f0a49640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0ec0fe1d0 0x7ff0ec10d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57548/0 (socket says 172.21.15.94:57548) 2024-03-24T10:44:08.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.870+0000 7ff0f0a49640 1 -- 172.21.15.94:0/1558669794 learned_addr learned my addr 172.21.15.94:0/1558669794 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:44:08.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.870+0000 7ff0f124a640 1 --2- 172.21.15.94:0/1558669794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0ec0fd880 0x7ff0ec10cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:08.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.871+0000 7ff0f0a49640 1 -- 172.21.15.94:0/1558669794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0ec0fd880 msgr2=0x7ff0ec10cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:08.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.871+0000 7ff0f0a49640 1 --2- 172.21.15.94:0/1558669794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0ec0fd880 0x7ff0ec10cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:08.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.871+0000 7ff0f0a49640 1 -- 172.21.15.94:0/1558669794 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0dc005040 con 0x7ff0ec0fe1d0 2024-03-24T10:44:08.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.871+0000 7ff0f0a49640 1 --2- 172.21.15.94:0/1558669794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0ec0fe1d0 0x7ff0ec10d2a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc02bda0 tx=0x7ff0dc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:44:08.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.871+0000 7ff0e1ffb640 1 -- 172.21.15.94:0/1558669794 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0dc009590 con 0x7ff0ec0fe1d0 2024-03-24T10:44:08.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.871+0000 7ff0e1ffb640 1 -- 172.21.15.94:0/1558669794 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff0dc0096f0 con 0x7ff0ec0fe1d0 2024-03-24T10:44:08.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.871+0000 7ff0f34ae640 1 -- 172.21.15.94:0/1558669794 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0ec1124f0 con 0x7ff0ec0fe1d0 2024-03-24T10:44:08.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.871+0000 7ff0e1ffb640 1 -- 172.21.15.94:0/1558669794 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0dc03b640 con 0x7ff0ec0fe1d0 2024-03-24T10:44:08.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.871+0000 7ff0f34ae640 1 -- 172.21.15.94:0/1558669794 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0ec112a10 con 0x7ff0ec0fe1d0 2024-03-24T10:44:08.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.872+0000 7ff0f34ae640 1 -- 172.21.15.94:0/1558669794 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0ec04bcc0 con 0x7ff0ec0fe1d0 2024-03-24T10:44:08.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.872+0000 7ff0e1ffb640 1 -- 172.21.15.94:0/1558669794 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7ff0dc02f070 con 0x7ff0ec0fe1d0 2024-03-24T10:44:08.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.873+0000 7ff0e1ffb640 1 --2- 172.21.15.94:0/1558669794 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7ff0d8069210 0x7ff0d806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:08.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.873+0000 7ff0e1ffb640 1 -- 172.21.15.94:0/1558669794 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7ff0dc0a9840 con 0x7ff0ec0fe1d0 2024-03-24T10:44:08.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.873+0000 7ff0f124a640 1 --2- 172.21.15.94:0/1558669794 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7ff0d8069210 0x7ff0d806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:08.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.873+0000 7ff0f124a640 1 --2- 172.21.15.94:0/1558669794 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7ff0d8069210 0x7ff0d806b6c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8009b50 tx=0x7ff0e8009120 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:44:08.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:08.877+0000 7ff0e1ffb640 1 -- 172.21.15.94:0/1558669794 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7ff0dc075aa0 con 0x7ff0ec0fe1d0 2024-03-24T10:44:09.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.043+0000 7ff0f34ae640 1 -- 172.21.15.94:0/1558669794 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7ff0ec104db0 con 0x7ff0ec0fe1d0 2024-03-24T10:44:09.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.043+0000 7ff0e1ffb640 1 -- 172.21.15.94:0/1558669794 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v46) v1 ==== 107+0+11 (secure 0 0 0) 0x7ff0dc0790c0 con 0x7ff0ec0fe1d0 2024-03-24T10:44:09.044 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-03-24T10:44:09.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.045+0000 7ff0f34ae640 1 -- 172.21.15.94:0/1558669794 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7ff0d8069210 msgr2=0x7ff0d806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:09.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.045+0000 7ff0f34ae640 1 --2- 172.21.15.94:0/1558669794 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7ff0d8069210 0x7ff0d806b6c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8009b50 tx=0x7ff0e8009120 comp rx=0 tx=0).stop 2024-03-24T10:44:09.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.045+0000 7ff0f34ae640 1 -- 172.21.15.94:0/1558669794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0ec0fe1d0 msgr2=0x7ff0ec10d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:09.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.045+0000 7ff0f34ae640 1 --2- 172.21.15.94:0/1558669794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0ec0fe1d0 0x7ff0ec10d2a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc02bda0 tx=0x7ff0dc009da0 comp rx=0 tx=0).stop 2024-03-24T10:44:09.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.045+0000 7ff0f34ae640 1 -- 172.21.15.94:0/1558669794 shutdown_connections 2024-03-24T10:44:09.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.045+0000 7ff0f34ae640 1 --2- 172.21.15.94:0/1558669794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0ec0fe1d0 0x7ff0ec10d2a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:09.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.045+0000 7ff0f34ae640 1 --2- 172.21.15.94:0/1558669794 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7ff0d8069210 0x7ff0d806b6c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:09.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.045+0000 7ff0f34ae640 1 --2- 172.21.15.94:0/1558669794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0ec0fd880 0x7ff0ec10cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:09.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.045+0000 7ff0f34ae640 1 -- 172.21.15.94:0/1558669794 >> 172.21.15.94:0/1558669794 conn(0x7ff0ec0f6000 msgr2=0x7ff0ec0f76b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:44:09.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.045+0000 7ff0f34ae640 1 -- 172.21.15.94:0/1558669794 shutdown_connections 2024-03-24T10:44:09.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.045+0000 7ff0f34ae640 1 -- 172.21.15.94:0/1558669794 wait complete. 2024-03-24T10:44:09.057 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-24T10:44:09.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.153+0000 7faba275d640 1 -- 172.21.15.94:0/3496181534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab9c066230 msgr2=0x7fab9c0feda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:09.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.153+0000 7faba275d640 1 --2- 172.21.15.94:0/3496181534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab9c066230 0x7fab9c0feda0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fab900048d0 tx=0x7fab90009da0 comp rx=0 tx=0).stop 2024-03-24T10:44:09.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.153+0000 7faba275d640 1 -- 172.21.15.94:0/3496181534 shutdown_connections 2024-03-24T10:44:09.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.153+0000 7faba275d640 1 --2- 172.21.15.94:0/3496181534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab9c0ff2e0 0x7fab9c1016c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:09.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.153+0000 7faba275d640 1 --2- 172.21.15.94:0/3496181534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab9c066230 0x7fab9c0feda0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:09.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.153+0000 7faba275d640 1 -- 172.21.15.94:0/3496181534 >> 172.21.15.94:0/3496181534 conn(0x7fab9c0f6850 msgr2=0x7fab9c0f8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:44:09.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.153+0000 7faba275d640 1 -- 172.21.15.94:0/3496181534 shutdown_connections 2024-03-24T10:44:09.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.153+0000 7faba275d640 1 -- 172.21.15.94:0/3496181534 wait complete. 2024-03-24T10:44:09.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.153+0000 7faba275d640 1 Processor -- start 2024-03-24T10:44:09.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.154+0000 7faba275d640 1 -- start start 2024-03-24T10:44:09.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.154+0000 7faba275d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab9c066230 0x7fab9c10cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:09.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.154+0000 7faba275d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab9c0ff2e0 0x7fab9c10d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:09.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.154+0000 7faba275d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c10d910 con 0x7fab9c066230 2024-03-24T10:44:09.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.154+0000 7faba275d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c112280 con 0x7fab9c0ff2e0 2024-03-24T10:44:09.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.154+0000 7fab9bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab9c066230 0x7fab9c10cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:09.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.154+0000 7fab9bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab9c066230 0x7fab9c10cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57570/0 (socket says 172.21.15.94:57570) 2024-03-24T10:44:09.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.154+0000 7fab9bfff640 1 -- 172.21.15.94:0/1872436114 learned_addr learned my addr 172.21.15.94:0/1872436114 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:44:09.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.154+0000 7fab9b7fe640 1 --2- 172.21.15.94:0/1872436114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab9c0ff2e0 0x7fab9c10d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:09.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.154+0000 7fab9bfff640 1 -- 172.21.15.94:0/1872436114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab9c0ff2e0 msgr2=0x7fab9c10d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:09.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.154+0000 7fab9bfff640 1 --2- 172.21.15.94:0/1872436114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab9c0ff2e0 0x7fab9c10d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:09.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.154+0000 7fab9bfff640 1 -- 172.21.15.94:0/1872436114 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab90005040 con 0x7fab9c066230 2024-03-24T10:44:09.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.154+0000 7fab9bfff640 1 --2- 172.21.15.94:0/1872436114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab9c066230 0x7fab9c10cd80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fab900048a0 tx=0x7fab90009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:44:09.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.154+0000 7fab98ff9640 1 -- 172.21.15.94:0/1872436114 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab900094c0 con 0x7fab9c066230 2024-03-24T10:44:09.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.154+0000 7fab98ff9640 1 -- 172.21.15.94:0/1872436114 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab90007cc0 con 0x7fab9c066230 2024-03-24T10:44:09.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.155+0000 7faba275d640 1 -- 172.21.15.94:0/1872436114 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab9c1123c0 con 0x7fab9c066230 2024-03-24T10:44:09.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.155+0000 7fab98ff9640 1 -- 172.21.15.94:0/1872436114 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab9003b400 con 0x7fab9c066230 2024-03-24T10:44:09.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.155+0000 7faba275d640 1 -- 172.21.15.94:0/1872436114 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab9c112890 con 0x7fab9c066230 2024-03-24T10:44:09.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.155+0000 7faba275d640 1 -- 172.21.15.94:0/1872436114 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab9c04bcc0 con 0x7fab9c066230 2024-03-24T10:44:09.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.156+0000 7fab98ff9640 1 -- 172.21.15.94:0/1872436114 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fab9002f070 con 0x7fab9c066230 2024-03-24T10:44:09.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.156+0000 7fab98ff9640 1 --2- 172.21.15.94:0/1872436114 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fab84069210 0x7fab8406b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:09.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.156+0000 7fab98ff9640 1 -- 172.21.15.94:0/1872436114 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fab900a96f0 con 0x7fab9c066230 2024-03-24T10:44:09.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.156+0000 7fab9b7fe640 1 --2- 172.21.15.94:0/1872436114 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fab84069210 0x7fab8406b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:09.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.157+0000 7fab9b7fe640 1 --2- 172.21.15.94:0/1872436114 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fab84069210 0x7fab8406b6c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fab8c004a70 tx=0x7fab8c00b040 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:44:09.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.160+0000 7fab98ff9640 1 -- 172.21.15.94:0/1872436114 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fab90075950 con 0x7fab9c066230 2024-03-24T10:44:09.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.364+0000 7faba275d640 1 -- 172.21.15.94:0/1872436114 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab9c112a20 con 0x7fab9c066230 2024-03-24T10:44:09.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.364+0000 7fab98ff9640 1 -- 172.21.15.94:0/1872436114 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4693 (secure 0 0 0) 0x7fab90078f70 con 0x7fab9c066230 2024-03-24T10:44:09.366 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:44:09.366 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14512,"name":"cephfs.smithi094.efiqte","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6827/1636528860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1636528860},{"type":"v1","addr":"172.21.15.94:6827","nonce":1636528860}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":24261,"name":"cephfs.smithi096.jtebhh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.96:6827/1321087930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1321087930},{"type":"v1","addr":"172.21.15.96:6827","nonce":1321087930}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16}],"filesystems":[{"mdsmap":{"epoch":18,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-24T10:43:39.435738+0000","modified":"2024-03-24T10:43:58.133901+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14500":{"gid":14500,"name":"cephfs.smithi094.llefgu","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.94:6829/126762980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":126762980},{"type":"v1","addr":"172.21.15.94:6829","nonce":126762980}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24251":{"gid":24251,"name":"cephfs.smithi096.egbmoq","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.96:6825/1539752208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1539752208},{"type":"v1","addr":"172.21.15.96:6825","nonce":1539752208}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-03-24T10:44:09.366 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 18 2024-03-24T10:44:09.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.366+0000 7faba275d640 1 -- 172.21.15.94:0/1872436114 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fab84069210 msgr2=0x7fab8406b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:09.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.366+0000 7faba275d640 1 --2- 172.21.15.94:0/1872436114 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fab84069210 0x7fab8406b6c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fab8c004a70 tx=0x7fab8c00b040 comp rx=0 tx=0).stop 2024-03-24T10:44:09.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.366+0000 7faba275d640 1 -- 172.21.15.94:0/1872436114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab9c066230 msgr2=0x7fab9c10cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:09.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.366+0000 7faba275d640 1 --2- 172.21.15.94:0/1872436114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab9c066230 0x7fab9c10cd80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fab900048a0 tx=0x7fab90009da0 comp rx=0 tx=0).stop 2024-03-24T10:44:09.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.366+0000 7faba275d640 1 -- 172.21.15.94:0/1872436114 shutdown_connections 2024-03-24T10:44:09.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.366+0000 7faba275d640 1 --2- 172.21.15.94:0/1872436114 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fab84069210 0x7fab8406b6c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:09.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.366+0000 7faba275d640 1 --2- 172.21.15.94:0/1872436114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab9c0ff2e0 0x7fab9c10d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:09.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.366+0000 7faba275d640 1 --2- 172.21.15.94:0/1872436114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab9c066230 0x7fab9c10cd80 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:09.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.366+0000 7faba275d640 1 -- 172.21.15.94:0/1872436114 >> 172.21.15.94:0/1872436114 conn(0x7fab9c0f6850 msgr2=0x7fab9c0f6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:44:09.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.367+0000 7faba275d640 1 -- 172.21.15.94:0/1872436114 shutdown_connections 2024-03-24T10:44:09.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:09.367+0000 7faba275d640 1 -- 172.21.15.94:0/1872436114 wait complete. 2024-03-24T10:44:09.377 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:09 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/4078019367' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T10:44:09.377 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:09 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/2210990867' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T10:44:09.377 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:09 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/4227710655' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-24T10:44:09.377 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:09 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/1558669794' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-24T10:44:09.378 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 18, 'max_mds': 1, 'flags': 50} 2024-03-24T10:44:09.378 INFO:teuthology.run_tasks:Running task kclient... 2024-03-24T10:44:09.403 INFO:tasks.kclient:Mounting kernel clients... 2024-03-24T10:44:09.403 INFO:tasks.kclient:config is {'client.0': {}, 'client.1': {}} 2024-03-24T10:44:09.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T10:44:09.404 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-03-24T10:44:09.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T10:44:09.419 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link delete ceph-brx 2024-03-24T10:44:09.488 INFO:teuthology.orchestra.run.smithi094.stderr:Cannot find device "ceph-brx" 2024-03-24T10:44:09.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-24T10:44:09.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T10:44:09.490 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list 2024-03-24T10:44:09.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T10:44:09.509 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link delete ceph-brx 2024-03-24T10:44:09.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:09 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/4078019367' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T10:44:09.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:09 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/2210990867' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T10:44:09.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:09 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/4227710655' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-24T10:44:09.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:09 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/1558669794' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-24T10:44:09.583 INFO:teuthology.orchestra.run.smithi096.stderr:Cannot find device "ceph-brx" 2024-03-24T10:44:09.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-24T10:44:09.584 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-03-24T10:44:09.585 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-24T10:44:09.585 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-24T10:44:09.585 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph fs ls 2024-03-24T10:44:09.839 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:44:10.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:10 smithi094 ceph-mon[29446]: pgmap v188: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.7 KiB/s rd, 0 B/s wr, 5 op/s 2024-03-24T10:44:10.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:10 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/1872436114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T10:44:10.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:10 smithi096 ceph-mon[38097]: pgmap v188: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.7 KiB/s rd, 0 B/s wr, 5 op/s 2024-03-24T10:44:10.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:10 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/1872436114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T10:44:11.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:11.381+0000 7f461a073700 1 -- 172.21.15.94:0/2271775313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46140fb540 msgr2=0x7f46140fb9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:11.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:11.381+0000 7f461a073700 1 --2- 172.21.15.94:0/2271775313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46140fb540 0x7f46140fb9b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4608004940 tx=0x7f4608009e30 comp rx=0 tx=0).stop 2024-03-24T10:44:11.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:11.381+0000 7f461a073700 1 -- 172.21.15.94:0/2271775313 shutdown_connections 2024-03-24T10:44:11.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:11.381+0000 7f461a073700 1 --2- 172.21.15.94:0/2271775313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46140fb540 0x7f46140fb9b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:11.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:11.381+0000 7f461a073700 1 --2- 172.21.15.94:0/2271775313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46140fdd80 0x7f46140fe150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:11.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:11.381+0000 7f461a073700 1 -- 172.21.15.94:0/2271775313 >> 172.21.15.94:0/2271775313 conn(0x7f4614071530 msgr2=0x7f4614071930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:44:11.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:11.381+0000 7f461a073700 1 -- 172.21.15.94:0/2271775313 shutdown_connections 2024-03-24T10:44:11.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:11.381+0000 7f461a073700 1 -- 172.21.15.94:0/2271775313 wait complete. 2024-03-24T10:44:11.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:11.381+0000 7f461a073700 1 Processor -- start 2024-03-24T10:44:11.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:11.381+0000 7f461a073700 1 -- start start 2024-03-24T10:44:11.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:11.381+0000 7f461a073700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46140fb540 0x7f4614115b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:11.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:11.382+0000 7f461a073700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46140fdd80 0x7f46141160b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:11.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:11.382+0000 7f461a073700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4614116700 con 0x7f46140fb540 2024-03-24T10:44:11.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:11.382+0000 7f461a073700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461410fce0 con 0x7f46140fdd80 2024-03-24T10:44:11.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:11.382+0000 7f46137fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46140fb540 0x7f4614115b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:11.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:11.382+0000 7f46137fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46140fb540 0x7f4614115b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42498/0 (socket says 172.21.15.94:42498) 2024-03-24T10:44:11.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:11.382+0000 7f46137fe700 1 -- 172.21.15.94:0/2147500021 learned_addr learned my addr 172.21.15.94:0/2147500021 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:44:11.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:11.382+0000 7f4612ffd700 1 --2- 172.21.15.94:0/2147500021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46140fdd80 0x7f46141160b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:11.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:11.382+0000 7f46137fe700 1 -- 172.21.15.94:0/2147500021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46140fdd80 msgr2=0x7f46141160b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:11.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:11.382+0000 7f46137fe700 1 --2- 172.21.15.94:0/2147500021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46140fdd80 0x7f46141160b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:11.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:11.382+0000 7f46137fe700 1 -- 172.21.15.94:0/2147500021 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4608005040 con 0x7f46140fb540 2024-03-24T10:44:11.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:11.382+0000 7f46137fe700 1 --2- 172.21.15.94:0/2147500021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46140fb540 0x7f4614115b70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f460400c8e0 tx=0x7f46040044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:44:11.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:11.382+0000 7f45fbfff700 1 -- 172.21.15.94:0/2147500021 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f460400a760 con 0x7f46140fb540 2024-03-24T10:44:11.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:11.382+0000 7f45fbfff700 1 -- 172.21.15.94:0/2147500021 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f460400ad90 con 0x7f46140fb540 2024-03-24T10:44:11.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:11.382+0000 7f461a073700 1 -- 172.21.15.94:0/2147500021 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f461410ffc0 con 0x7f46140fb540 2024-03-24T10:44:11.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:11.382+0000 7f45fbfff700 1 -- 172.21.15.94:0/2147500021 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46040113a0 con 0x7f46140fb540 2024-03-24T10:44:11.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:11.382+0000 7f461a073700 1 -- 172.21.15.94:0/2147500021 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4614110580 con 0x7f46140fb540 2024-03-24T10:44:11.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:11.383+0000 7f461a073700 1 -- 172.21.15.94:0/2147500021 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46141073f0 con 0x7f46140fb540 2024-03-24T10:44:11.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:11.384+0000 7f45fbfff700 1 -- 172.21.15.94:0/2147500021 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f460400a8c0 con 0x7f46140fb540 2024-03-24T10:44:11.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:11.384+0000 7f45fbfff700 1 --2- 172.21.15.94:0/2147500021 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f45fc0692a0 0x7f45fc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:11.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:11.384+0000 7f45fbfff700 1 -- 172.21.15.94:0/2147500021 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f4604087260 con 0x7f46140fb540 2024-03-24T10:44:11.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:11.384+0000 7f4612ffd700 1 --2- 172.21.15.94:0/2147500021 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f45fc0692a0 0x7f45fc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:11.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:11.385+0000 7f4612ffd700 1 --2- 172.21.15.94:0/2147500021 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f45fc0692a0 0x7f45fc06b750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4608004f90 tx=0x7f4608009e30 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:44:11.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:11.388+0000 7f45fbfff700 1 -- 172.21.15.94:0/2147500021 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f4604056fd0 con 0x7f46140fb540 2024-03-24T10:44:11.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:11.622+0000 7f461a073700 1 -- 172.21.15.94:0/2147500021 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f461410c230 con 0x7f46140fb540 2024-03-24T10:44:11.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:11.622+0000 7f45fbfff700 1 -- 172.21.15.94:0/2147500021 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) v1 ==== 53+0+83 (secure 0 0 0) 0x7f4604056b60 con 0x7f46140fb540 2024-03-24T10:44:11.623 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-03-24T10:44:11.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:11.624+0000 7f461a073700 1 -- 172.21.15.94:0/2147500021 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f45fc0692a0 msgr2=0x7f45fc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:11.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:11.624+0000 7f461a073700 1 --2- 172.21.15.94:0/2147500021 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f45fc0692a0 0x7f45fc06b750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4608004f90 tx=0x7f4608009e30 comp rx=0 tx=0).stop 2024-03-24T10:44:11.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:11.624+0000 7f461a073700 1 -- 172.21.15.94:0/2147500021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46140fb540 msgr2=0x7f4614115b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:11.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:11.624+0000 7f461a073700 1 --2- 172.21.15.94:0/2147500021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46140fb540 0x7f4614115b70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f460400c8e0 tx=0x7f46040044c0 comp rx=0 tx=0).stop 2024-03-24T10:44:11.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:11.624+0000 7f461a073700 1 -- 172.21.15.94:0/2147500021 shutdown_connections 2024-03-24T10:44:11.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:11.624+0000 7f461a073700 1 --2- 172.21.15.94:0/2147500021 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f45fc0692a0 0x7f45fc06b750 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:11.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:11.624+0000 7f461a073700 1 --2- 172.21.15.94:0/2147500021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46140fb540 0x7f4614115b70 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:11.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:11.624+0000 7f461a073700 1 --2- 172.21.15.94:0/2147500021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46140fdd80 0x7f46141160b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:11.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:11.624+0000 7f461a073700 1 -- 172.21.15.94:0/2147500021 >> 172.21.15.94:0/2147500021 conn(0x7f4614071530 msgr2=0x7f46140f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:44:11.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:11.624+0000 7f461a073700 1 -- 172.21.15.94:0/2147500021 shutdown_connections 2024-03-24T10:44:11.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:11.624+0000 7f461a073700 1 -- 172.21.15.94:0/2147500021 wait complete. 2024-03-24T10:44:12.230 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:12 smithi094 ceph-mon[29446]: pgmap v189: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.7 KiB/s rd, 0 B/s wr, 5 op/s 2024-03-24T10:44:12.230 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:12 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/2147500021' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-24T10:44:12.255 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-24T10:44:12.256 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-24T10:44:12.256 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi094.front.sepia.ceph.com 2024-03-24T10:44:12.256 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-24T10:44:12.256 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-24T10:44:12.256 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-24T10:44:12.256 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-24T10:44:12.256 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-24T10:44:12.256 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-24T10:44:12.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T10:44:12.256 DEBUG:teuthology.orchestra.run.smithi094:> ip addr 2024-03-24T10:44:12.272 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-24T10:44:12.272 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-24T10:44:12.272 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2024-03-24T10:44:12.272 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-03-24T10:44:12.272 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2024-03-24T10:44:12.273 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-03-24T10:44:12.273 INFO:teuthology.orchestra.run.smithi094.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-24T10:44:12.273 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4c brd ff:ff:ff:ff:ff:ff 2024-03-24T10:44:12.273 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f0 2024-03-24T10:44:12.273 INFO:teuthology.orchestra.run.smithi094.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-24T10:44:12.273 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4d brd ff:ff:ff:ff:ff:ff 2024-03-24T10:44:12.273 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f1 2024-03-24T10:44:12.273 INFO:teuthology.orchestra.run.smithi094.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-24T10:44:12.273 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:62 brd ff:ff:ff:ff:ff:ff 2024-03-24T10:44:12.273 INFO:teuthology.orchestra.run.smithi094.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-24T10:44:12.273 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:63 brd ff:ff:ff:ff:ff:ff 2024-03-24T10:44:12.273 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.21.15.94/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-24T10:44:12.273 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft 42256sec preferred_lft 42256sec 2024-03-24T10:44:12.273 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::ec4:7aff:fe8f:d763/64 scope link 2024-03-24T10:44:12.274 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-03-24T10:44:12.274 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-24T10:44:12.274 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-24T10:44:12.274 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-03-24T10:44:12.274 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link add name ceph-brx type bridge 2024-03-24T10:44:12.274 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip addr flush dev ceph-brx 2024-03-24T10:44:12.274 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set ceph-brx up 2024-03-24T10:44:12.274 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-24T10:44:12.275 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-03-24T10:44:12.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-24T10:44:12.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-24T10:44:12.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T10:44:12.480 DEBUG:teuthology.orchestra.run.smithi094:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-24T10:44:12.509 INFO:teuthology.orchestra.run.smithi094.stdout:1 2024-03-24T10:44:12.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T10:44:12.510 DEBUG:teuthology.orchestra.run.smithi094:> route 2024-03-24T10:44:12.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:12 smithi096 ceph-mon[38097]: pgmap v189: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.7 KiB/s rd, 0 B/s wr, 5 op/s 2024-03-24T10:44:12.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:12 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/2147500021' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-24T10:44:12.583 INFO:teuthology.orchestra.run.smithi094.stdout:Kernel IP routing table 2024-03-24T10:44:12.583 INFO:teuthology.orchestra.run.smithi094.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-24T10:44:12.583 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-24T10:44:12.583 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-24T10:44:12.583 INFO:teuthology.orchestra.run.smithi094.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-24T10:44:12.583 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-24T10:44:12.583 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-03-24T10:44:12.584 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-24T10:44:12.584 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-24T10:44:12.584 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-24T10:44:12.584 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-03-24T10:44:12.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-24T10:44:12.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-24T10:44:12.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T10:44:12.906 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-03-24T10:44:12.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T10:44:12.921 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list-id 2024-03-24T10:44:12.975 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-24T10:44:12.975 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-03-24T10:44:12.975 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-24T10:44:12.975 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-24T10:44:12.975 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-03-24T10:44:13.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-24T10:44:13.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-24T10:44:13.097 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-24T10:44:13.098 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-24T10:44:13.098 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-03-24T10:44:13.098 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-24T10:44:13.098 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-24T10:44:13.098 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-24T10:44:13.098 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-24T10:44:13.098 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-24T10:44:13.098 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-03-24T10:44:13.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-24T10:44:13.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-24T10:44:13.231 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-24T10:44:13.231 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-03-24T10:44:13.231 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set brx.0 up 2024-03-24T10:44:13.231 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set dev brx.0 master ceph-brx 2024-03-24T10:44:13.231 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-03-24T10:44:13.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-24T10:44:13.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-24T10:44:13.312 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-24T10:44:13.312 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-24T10:44:13.339 INFO:teuthology.orchestra.run.smithi094.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-24T10:44:13.339 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-24T10:44:13.339 DEBUG:teuthology.orchestra.run.smithi094:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-24T10:44:13.394 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-24T10:44:13.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T10:44:13.394 DEBUG:teuthology.orchestra.run.smithi094:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-24T10:44:13.484 INFO:teuthology.orchestra.run.smithi094.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-24T10:44:13.484 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-24T10:44:13.484 INFO:teuthology.orchestra.run.smithi094.stdout:invalid new device string format 2024-03-24T10:44:13.599 INFO:teuthology.orchestra.run.smithi094.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-24T10:44:13.599 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-24T10:44:13.599 INFO:teuthology.orchestra.run.smithi094.stdout:invalid new device string format 2024-03-24T10:44:13.599 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: resolved to: "172.21.15.94:3300,172.21.15.96:3300" 2024-03-24T10:44:13.599 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-24T10:44:13.605 INFO:teuthology.orchestra.run.smithi094.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-24T10:44:13.605 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-24T10:44:13.605 INFO:teuthology.orchestra.run.smithi094.stdout:invalid new device string format 2024-03-24T10:44:13.605 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: resolved to: "172.21.15.94:3300,172.21.15.96:3300" 2024-03-24T10:44:13.605 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-24T10:44:13.605 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: resolved to: "172.21.15.94:6789,172.21.15.96:6789" 2024-03-24T10:44:13.605 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.94:6789,172.21.15.96:6789:/ 2024-03-24T10:44:13.605 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=386110d4-e9ca-11ee-95cd-87774f69a715" will pass to kernel 2024-03-24T10:44:13.606 INFO:teuthology.orchestra.run.smithi094.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-24T10:44:13.606 INFO:teuthology.orchestra.run.smithi094.stdout: You just mounted a file system that supports labels which does not 2024-03-24T10:44:13.606 INFO:teuthology.orchestra.run.smithi094.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-24T10:44:13.606 INFO:teuthology.orchestra.run.smithi094.stdout: applications will generate AVC messages and not be allowed access to 2024-03-24T10:44:13.606 INFO:teuthology.orchestra.run.smithi094.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-24T10:44:13.608 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-24T10:44:13.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T10:44:13.608 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-24T10:44:13.635 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-24T10:44:13.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:13 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-24T10:44:13.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-24T10:44:13.742 INFO:teuthology.orchestra.run.smithi094.stdout:client14550 2024-03-24T10:44:13.742 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-03-24T10:44:13.742 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-24T10:44:13.758 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-24T10:44:13.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:13 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-24T10:44:13.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-24T10:44:13.864 INFO:teuthology.orchestra.run.smithi094.stdout:client14550 2024-03-24T10:44:13.864 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-24T10:44:13.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:13 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-24T10:44:13.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-24T10:44:13.929 INFO:teuthology.orchestra.run.smithi094.stdout:client14550 2024-03-24T10:44:13.929 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/386110d4-e9ca-11ee-95cd-87774f69a715.client14550/status') 2024-03-24T10:44:13.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-24T10:44:13.998 INFO:teuthology.orchestra.run.smithi094.stdout:instance: client.14550 (3)192.168.0.1:0/449160647 2024-03-24T10:44:13.998 INFO:teuthology.orchestra.run.smithi094.stdout:blocklisted: false 2024-03-24T10:44:13.998 INFO:teuthology.orchestra.run.smithi094.stderr:0+1 records in 2024-03-24T10:44:13.998 INFO:teuthology.orchestra.run.smithi094.stderr:0+1 records out 2024-03-24T10:44:13.998 INFO:teuthology.orchestra.run.smithi094.stderr:69 bytes copied, 3.7346e-05 s, 1.8 MB/s 2024-03-24T10:44:13.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-24T10:44:14.005 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-03-24T10:44:14.005 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-24T10:44:14.005 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-24T10:44:14.005 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- ceph fs ls 2024-03-24T10:44:14.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:14 smithi094 ceph-mon[29446]: pgmap v190: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-03-24T10:44:14.260 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:44:14.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:14 smithi096 ceph-mon[38097]: pgmap v190: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-03-24T10:44:15.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:15.807+0000 7fba0a0db700 1 -- 172.21.15.94:0/2201633652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba04102e00 msgr2=0x7fba041031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:15.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:15.807+0000 7fba0a0db700 1 --2- 172.21.15.94:0/2201633652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba04102e00 0x7fba041031d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4004940 tx=0x7fb9f4009e30 comp rx=0 tx=0).stop 2024-03-24T10:44:15.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:15.807+0000 7fba0a0db700 1 -- 172.21.15.94:0/2201633652 shutdown_connections 2024-03-24T10:44:15.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:15.807+0000 7fba0a0db700 1 --2- 172.21.15.94:0/2201633652 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba041037a0 0x7fba040fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:15.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:15.807+0000 7fba0a0db700 1 --2- 172.21.15.94:0/2201633652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba04102e00 0x7fba041031d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:15.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:15.807+0000 7fba0a0db700 1 -- 172.21.15.94:0/2201633652 >> 172.21.15.94:0/2201633652 conn(0x7fba040f89b0 msgr2=0x7fba040fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:44:15.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:15.807+0000 7fba0a0db700 1 -- 172.21.15.94:0/2201633652 shutdown_connections 2024-03-24T10:44:15.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:15.807+0000 7fba0a0db700 1 -- 172.21.15.94:0/2201633652 wait complete. 2024-03-24T10:44:15.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:15.808+0000 7fba0a0db700 1 Processor -- start 2024-03-24T10:44:15.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:15.808+0000 7fba0a0db700 1 -- start start 2024-03-24T10:44:15.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:15.808+0000 7fba0a0db700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba04102e00 0x7fba040702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:15.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:15.808+0000 7fba0a0db700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba041037a0 0x7fba04070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:15.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:15.808+0000 7fba0a0db700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba04070d40 con 0x7fba041037a0 2024-03-24T10:44:15.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:15.808+0000 7fba0a0db700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba04070e80 con 0x7fba04102e00 2024-03-24T10:44:15.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:15.808+0000 7fb9faffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba041037a0 0x7fba04070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:15.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:15.808+0000 7fb9faffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba041037a0 0x7fba04070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42504/0 (socket says 172.21.15.94:42504) 2024-03-24T10:44:15.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:15.808+0000 7fb9faffd700 1 -- 172.21.15.94:0/3143222470 learned_addr learned my addr 172.21.15.94:0/3143222470 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:44:15.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:15.809+0000 7fba037fe700 1 --2- 172.21.15.94:0/3143222470 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba04102e00 0x7fba040702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:15.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:15.809+0000 7fb9faffd700 1 -- 172.21.15.94:0/3143222470 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba04102e00 msgr2=0x7fba040702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:15.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:15.809+0000 7fb9faffd700 1 --2- 172.21.15.94:0/3143222470 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba04102e00 0x7fba040702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:15.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:15.809+0000 7fb9faffd700 1 -- 172.21.15.94:0/3143222470 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9f4005040 con 0x7fba041037a0 2024-03-24T10:44:15.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:15.809+0000 7fb9faffd700 1 --2- 172.21.15.94:0/3143222470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba041037a0 0x7fba04070800 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec00c800 tx=0x7fb9ec004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:44:15.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:15.809+0000 7fba00ff9700 1 -- 172.21.15.94:0/3143222470 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9ec0106f0 con 0x7fba041037a0 2024-03-24T10:44:15.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:15.809+0000 7fba0a0db700 1 -- 172.21.15.94:0/3143222470 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba04071130 con 0x7fba041037a0 2024-03-24T10:44:15.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:15.809+0000 7fba00ff9700 1 -- 172.21.15.94:0/3143222470 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb9ec010d20 con 0x7fba041037a0 2024-03-24T10:44:15.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:15.809+0000 7fba00ff9700 1 -- 172.21.15.94:0/3143222470 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9ec0113a0 con 0x7fba041037a0 2024-03-24T10:44:15.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:15.809+0000 7fba0a0db700 1 -- 172.21.15.94:0/3143222470 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba0411a320 con 0x7fba041037a0 2024-03-24T10:44:15.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:15.810+0000 7fba0a0db700 1 -- 172.21.15.94:0/3143222470 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba04100110 con 0x7fba041037a0 2024-03-24T10:44:15.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:15.810+0000 7fba00ff9700 1 -- 172.21.15.94:0/3143222470 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fb9ec012400 con 0x7fba041037a0 2024-03-24T10:44:15.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:15.811+0000 7fba00ff9700 1 --2- 172.21.15.94:0/3143222470 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fb9f00692a0 0x7fb9f006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:15.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:15.811+0000 7fba037fe700 1 --2- 172.21.15.94:0/3143222470 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fb9f00692a0 0x7fb9f006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:15.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:15.811+0000 7fba00ff9700 1 -- 172.21.15.94:0/3143222470 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fb9ec086f00 con 0x7fba041037a0 2024-03-24T10:44:15.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:15.811+0000 7fba037fe700 1 --2- 172.21.15.94:0/3143222470 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fb9f00692a0 0x7fb9f006b750 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4004f90 tx=0x7fb9f4009e30 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:44:15.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:15.815+0000 7fba00ff9700 1 -- 172.21.15.94:0/3143222470 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fb9ec053280 con 0x7fba041037a0 2024-03-24T10:44:16.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:16.046+0000 7fba0a0db700 1 -- 172.21.15.94:0/3143222470 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fba04109f00 con 0x7fba041037a0 2024-03-24T10:44:16.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:16.046+0000 7fba00ff9700 1 -- 172.21.15.94:0/3143222470 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) v1 ==== 53+0+83 (secure 0 0 0) 0x7fb9ec0568a0 con 0x7fba041037a0 2024-03-24T10:44:16.047 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-03-24T10:44:16.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:16.048+0000 7fba0a0db700 1 -- 172.21.15.94:0/3143222470 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fb9f00692a0 msgr2=0x7fb9f006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:16.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:16.048+0000 7fba0a0db700 1 --2- 172.21.15.94:0/3143222470 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fb9f00692a0 0x7fb9f006b750 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4004f90 tx=0x7fb9f4009e30 comp rx=0 tx=0).stop 2024-03-24T10:44:16.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:16.048+0000 7fba0a0db700 1 -- 172.21.15.94:0/3143222470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba041037a0 msgr2=0x7fba04070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:16.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:16.048+0000 7fba0a0db700 1 --2- 172.21.15.94:0/3143222470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba041037a0 0x7fba04070800 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec00c800 tx=0x7fb9ec004470 comp rx=0 tx=0).stop 2024-03-24T10:44:16.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:16.048+0000 7fba0a0db700 1 -- 172.21.15.94:0/3143222470 shutdown_connections 2024-03-24T10:44:16.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:16.048+0000 7fba0a0db700 1 --2- 172.21.15.94:0/3143222470 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fb9f00692a0 0x7fb9f006b750 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:16.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:16.048+0000 7fba0a0db700 1 --2- 172.21.15.94:0/3143222470 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba04102e00 0x7fba040702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:16.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:16.048+0000 7fba0a0db700 1 --2- 172.21.15.94:0/3143222470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba041037a0 0x7fba04070800 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:16.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:16.048+0000 7fba0a0db700 1 -- 172.21.15.94:0/3143222470 >> 172.21.15.94:0/3143222470 conn(0x7fba040f89b0 msgr2=0x7fba041073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:44:16.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:16.048+0000 7fba0a0db700 1 -- 172.21.15.94:0/3143222470 shutdown_connections 2024-03-24T10:44:16.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:16.048+0000 7fba0a0db700 1 -- 172.21.15.94:0/3143222470 wait complete. 2024-03-24T10:44:16.378 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:16 smithi094 ceph-mon[29446]: pgmap v191: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-24T10:44:16.378 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:16 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:44:16.379 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:16 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/3143222470' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-24T10:44:16.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:16 smithi096 ceph-mon[38097]: pgmap v191: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-24T10:44:16.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:16 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:44:16.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:16 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/3143222470' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-24T10:44:16.630 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-24T10:44:16.630 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-24T10:44:16.630 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi096.front.sepia.ceph.com 2024-03-24T10:44:16.630 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-24T10:44:16.630 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-24T10:44:16.630 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-24T10:44:16.630 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-24T10:44:16.630 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-24T10:44:16.630 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-24T10:44:16.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T10:44:16.631 DEBUG:teuthology.orchestra.run.smithi096:> ip addr 2024-03-24T10:44:16.649 INFO:teuthology.orchestra.run.smithi096.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-24T10:44:16.649 INFO:teuthology.orchestra.run.smithi096.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-24T10:44:16.649 INFO:teuthology.orchestra.run.smithi096.stdout: inet 127.0.0.1/8 scope host lo 2024-03-24T10:44:16.649 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-03-24T10:44:16.649 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 ::1/128 scope host 2024-03-24T10:44:16.650 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-03-24T10:44:16.650 INFO:teuthology.orchestra.run.smithi096.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-24T10:44:16.650 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:d8:d9:bc brd ff:ff:ff:ff:ff:ff 2024-03-24T10:44:16.650 INFO:teuthology.orchestra.run.smithi096.stdout: altname enp7s0f0 2024-03-24T10:44:16.650 INFO:teuthology.orchestra.run.smithi096.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-24T10:44:16.650 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:d8:d9:bd brd ff:ff:ff:ff:ff:ff 2024-03-24T10:44:16.650 INFO:teuthology.orchestra.run.smithi096.stdout: altname enp7s0f1 2024-03-24T10:44:16.650 INFO:teuthology.orchestra.run.smithi096.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-24T10:44:16.650 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:8f:cc:0c brd ff:ff:ff:ff:ff:ff 2024-03-24T10:44:16.650 INFO:teuthology.orchestra.run.smithi096.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-24T10:44:16.650 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:8f:cc:0d brd ff:ff:ff:ff:ff:ff 2024-03-24T10:44:16.650 INFO:teuthology.orchestra.run.smithi096.stdout: inet 172.21.15.96/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-24T10:44:16.650 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft 42255sec preferred_lft 42255sec 2024-03-24T10:44:16.650 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::ec4:7aff:fe8f:cc0d/64 scope link 2024-03-24T10:44:16.651 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-03-24T10:44:16.651 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-24T10:44:16.651 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-24T10:44:16.651 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-03-24T10:44:16.651 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link add name ceph-brx type bridge 2024-03-24T10:44:16.651 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip addr flush dev ceph-brx 2024-03-24T10:44:16.651 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link set ceph-brx up 2024-03-24T10:44:16.651 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-24T10:44:16.651 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-03-24T10:44:16.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:44:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-24T10:44:16.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:44:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-24T10:44:16.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T10:44:16.858 DEBUG:teuthology.orchestra.run.smithi096:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-24T10:44:16.884 INFO:teuthology.orchestra.run.smithi096.stdout:1 2024-03-24T10:44:16.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T10:44:16.886 DEBUG:teuthology.orchestra.run.smithi096:> route 2024-03-24T10:44:16.958 INFO:teuthology.orchestra.run.smithi096.stdout:Kernel IP routing table 2024-03-24T10:44:16.958 INFO:teuthology.orchestra.run.smithi096.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-24T10:44:16.959 INFO:teuthology.orchestra.run.smithi096.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-24T10:44:16.959 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-24T10:44:16.959 INFO:teuthology.orchestra.run.smithi096.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-24T10:44:16.959 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-24T10:44:16.959 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-03-24T10:44:16.959 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-24T10:44:16.959 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-24T10:44:16.959 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-24T10:44:16.959 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-03-24T10:44:17.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:44:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-24T10:44:17.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:44:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-24T10:44:17.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T10:44:17.293 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list 2024-03-24T10:44:17.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T10:44:17.308 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list-id 2024-03-24T10:44:17.363 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-24T10:44:17.363 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-03-24T10:44:17.363 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-24T10:44:17.363 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-03-24T10:44:17.363 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-03-24T10:44:17.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:44:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-24T10:44:17.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:44:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-24T10:44:17.488 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-03-24T10:44:17.488 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-24T10:44:17.488 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-03-24T10:44:17.488 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-03-24T10:44:17.489 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-24T10:44:17.489 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-03-24T10:44:17.489 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-03-24T10:44:17.489 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-03-24T10:44:17.489 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-03-24T10:44:17.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:44:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-24T10:44:17.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:44:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-24T10:44:17.629 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-24T10:44:17.629 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-03-24T10:44:17.629 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link set brx.0 up 2024-03-24T10:44:17.629 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link set dev brx.0 master ceph-brx 2024-03-24T10:44:17.629 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-03-24T10:44:17.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:44:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-24T10:44:17.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:44:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-24T10:44:17.711 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-24T10:44:17.712 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-24T10:44:17.726 INFO:teuthology.orchestra.run.smithi096.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-24T10:44:17.726 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-24T10:44:17.726 DEBUG:teuthology.orchestra.run.smithi096:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-24T10:44:17.780 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-24T10:44:17.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T10:44:17.781 DEBUG:teuthology.orchestra.run.smithi096:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-24T10:44:17.844 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:17 smithi096 ceph-mon[38097]: pgmap v192: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-24T10:44:17.919 INFO:teuthology.orchestra.run.smithi096.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-24T10:44:17.919 INFO:teuthology.orchestra.run.smithi096.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-03-24T10:44:17.919 INFO:teuthology.orchestra.run.smithi096.stdout:invalid new device string format 2024-03-24T10:44:17.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:17 smithi094 ceph-mon[29446]: pgmap v192: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-24T10:44:18.025 INFO:teuthology.orchestra.run.smithi096.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-24T10:44:18.025 INFO:teuthology.orchestra.run.smithi096.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-03-24T10:44:18.025 INFO:teuthology.orchestra.run.smithi096.stdout:invalid new device string format 2024-03-24T10:44:18.025 INFO:teuthology.orchestra.run.smithi096.stdout:mount.ceph: resolved to: "172.21.15.94:3300,172.21.15.96:3300" 2024-03-24T10:44:18.025 INFO:teuthology.orchestra.run.smithi096.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-24T10:44:18.030 INFO:teuthology.orchestra.run.smithi096.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-24T10:44:18.031 INFO:teuthology.orchestra.run.smithi096.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-03-24T10:44:18.031 INFO:teuthology.orchestra.run.smithi096.stdout:invalid new device string format 2024-03-24T10:44:18.031 INFO:teuthology.orchestra.run.smithi096.stdout:mount.ceph: resolved to: "172.21.15.94:3300,172.21.15.96:3300" 2024-03-24T10:44:18.031 INFO:teuthology.orchestra.run.smithi096.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-24T10:44:18.031 INFO:teuthology.orchestra.run.smithi096.stdout:mount.ceph: resolved to: "172.21.15.94:6789,172.21.15.96:6789" 2024-03-24T10:44:18.031 INFO:teuthology.orchestra.run.smithi096.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.94:6789,172.21.15.96:6789:/ 2024-03-24T10:44:18.031 INFO:teuthology.orchestra.run.smithi096.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=386110d4-e9ca-11ee-95cd-87774f69a715" will pass to kernel 2024-03-24T10:44:18.034 INFO:teuthology.orchestra.run.smithi096.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-03-24T10:44:18.035 INFO:teuthology.orchestra.run.smithi096.stdout: You just mounted a file system that supports labels which does not 2024-03-24T10:44:18.035 INFO:teuthology.orchestra.run.smithi096.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-24T10:44:18.035 INFO:teuthology.orchestra.run.smithi096.stdout: applications will generate AVC messages and not be allowed access to 2024-03-24T10:44:18.035 INFO:teuthology.orchestra.run.smithi096.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-24T10:44:18.035 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-24T10:44:18.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T10:44:18.035 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-24T10:44:18.109 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-24T10:44:18.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:44:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-24T10:44:18.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:44:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-24T10:44:18.204 INFO:teuthology.orchestra.run.smithi096.stdout:client14556 2024-03-24T10:44:18.205 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-24T10:44:18.205 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-24T10:44:18.221 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-24T10:44:18.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:44:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-24T10:44:18.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:44:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-24T10:44:18.317 INFO:teuthology.orchestra.run.smithi096.stdout:client14556 2024-03-24T10:44:18.317 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-24T10:44:18.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:44:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-24T10:44:18.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:44:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-24T10:44:18.372 INFO:teuthology.orchestra.run.smithi096.stdout:client14556 2024-03-24T10:44:18.372 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/386110d4-e9ca-11ee-95cd-87774f69a715.client14556/status') 2024-03-24T10:44:18.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:44:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-24T10:44:18.432 INFO:teuthology.orchestra.run.smithi096.stdout:instance: client.14556 (3)172.21.15.96:0/2944380590 2024-03-24T10:44:18.432 INFO:teuthology.orchestra.run.smithi096.stdout:blocklisted: false 2024-03-24T10:44:18.433 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records in 2024-03-24T10:44:18.433 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records out 2024-03-24T10:44:18.433 INFO:teuthology.orchestra.run.smithi096.stderr:71 bytes copied, 2.0926e-05 s, 3.4 MB/s 2024-03-24T10:44:18.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-24T10:44:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-24T10:44:18.439 INFO:teuthology.run_tasks:Running task print... 2024-03-24T10:44:18.450 INFO:teuthology.task.print:**** done client 2024-03-24T10:44:18.450 INFO:teuthology.run_tasks:Running task parallel... 2024-03-24T10:44:18.477 INFO:teuthology.task.parallel:starting parallel... 2024-03-24T10:44:18.477 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-03-24T10:44:18.478 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-24T10:44:18.478 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi094.front.sepia.ceph.com 2024-03-24T10:44:18.479 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -e sha1=21b3bd672163092b4e542f772797fcba02d3d4de -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs false || true' 2024-03-24T10:44:18.479 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-03-24T10:44:18.479 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-03-24T10:44:18.483 INFO:tasks.workunit:Pulling workunits from ref d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b 2024-03-24T10:44:18.483 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-24T10:44:18.483 INFO:tasks.workunit:timeout=3h 2024-03-24T10:44:18.483 INFO:tasks.workunit:cleanup=True 2024-03-24T10:44:18.483 DEBUG:teuthology.orchestra.run.smithi094:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-24T10:44:18.504 INFO:teuthology.orchestra.run.smithi094.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-24T10:44:18.504 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-03-24T10:44:18.504 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 51h/81d Inode: 1 Links: 2 2024-03-24T10:44:18.504 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-24T10:44:18.504 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-03-24T10:44:18.504 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-03-24T10:44:18.504 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-03-24 10:43:44.425739491 +0000 2024-03-24T10:44:18.504 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-03-24 10:44:13.632844889 +0000 2024-03-24T10:44:18.504 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-03-24 10:43:44.425739491 +0000 2024-03-24T10:44:18.504 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-24T10:44:18.505 DEBUG:teuthology.orchestra.run.smithi094:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-24T10:44:18.576 DEBUG:teuthology.orchestra.run.smithi096:> stat -- /home/ubuntu/cephtest/mnt.1 2024-03-24T10:44:18.592 INFO:teuthology.orchestra.run.smithi096.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-03-24T10:44:18.592 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 1 Blocks: 0 IO Block: 65536 directory 2024-03-24T10:44:18.592 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 73h/115d Inode: 1 Links: 3 2024-03-24T10:44:18.593 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-24T10:44:18.593 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-03-24T10:44:18.593 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-03-24T10:44:18.593 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-03-24 10:44:18.571760998 +0000 2024-03-24T10:44:18.593 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-03-24 10:44:18.571760998 +0000 2024-03-24T10:44:18.593 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-03-24 10:43:44.425739491 +0000 2024-03-24T10:44:18.593 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-03-24T10:44:18.593 DEBUG:teuthology.orchestra.run.smithi096:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-03-24T10:44:18.663 DEBUG:teuthology.orchestra.run.smithi094:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/batrick/ceph /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b 2024-03-24T10:44:18.664 DEBUG:teuthology.orchestra.run.smithi096:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://github.com/batrick/ceph /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b 2024-03-24T10:44:18.679 INFO:tasks.workunit.client.0.smithi094.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-24T10:44:18.718 INFO:tasks.workunit.client.1.smithi096.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-03-24T10:44:18.754 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:44:20.066 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:19 smithi094 ceph-mon[29446]: pgmap v193: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-03-24T10:44:20.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:20.270+0000 7fcaca089700 1 -- 172.21.15.94:0/1250907661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcac40fce70 msgr2=0x7fcac40fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:20.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:20.270+0000 7fcaca089700 1 --2- 172.21.15.94:0/1250907661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcac40fce70 0x7fcac40fd280 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fcab4009ce0 tx=0x7fcab4004d10 comp rx=0 tx=0).stop 2024-03-24T10:44:20.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:20.270+0000 7fcaca089700 1 -- 172.21.15.94:0/1250907661 shutdown_connections 2024-03-24T10:44:20.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:20.270+0000 7fcaca089700 1 --2- 172.21.15.94:0/1250907661 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcac40fdfe0 0x7fcac40fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:20.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:20.270+0000 7fcaca089700 1 --2- 172.21.15.94:0/1250907661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcac40fce70 0x7fcac40fd280 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:20.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:20.270+0000 7fcaca089700 1 -- 172.21.15.94:0/1250907661 >> 172.21.15.94:0/1250907661 conn(0x7fcac40f8550 msgr2=0x7fcac40fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:44:20.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:20.270+0000 7fcaca089700 1 -- 172.21.15.94:0/1250907661 shutdown_connections 2024-03-24T10:44:20.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:20.270+0000 7fcaca089700 1 -- 172.21.15.94:0/1250907661 wait complete. 2024-03-24T10:44:20.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:20.271+0000 7fcaca089700 1 Processor -- start 2024-03-24T10:44:20.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:20.271+0000 7fcaca089700 1 -- start start 2024-03-24T10:44:20.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:20.271+0000 7fcaca089700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcac40fce70 0x7fcac410da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:20.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:20.271+0000 7fcaca089700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcac40fdfe0 0x7fcac410dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:20.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:20.271+0000 7fcaca089700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac410e5d0 con 0x7fcac40fdfe0 2024-03-24T10:44:20.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:20.271+0000 7fcaca089700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac410e740 con 0x7fcac40fce70 2024-03-24T10:44:20.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:20.271+0000 7fcac2ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcac40fdfe0 0x7fcac410dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:20.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:20.271+0000 7fcac2ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcac40fdfe0 0x7fcac410dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45308/0 (socket says 172.21.15.94:45308) 2024-03-24T10:44:20.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:20.271+0000 7fcac2ffd700 1 -- 172.21.15.94:0/2011984184 learned_addr learned my addr 172.21.15.94:0/2011984184 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:44:20.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:20.271+0000 7fcac37fe700 1 --2- 172.21.15.94:0/2011984184 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcac40fce70 0x7fcac410da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:20.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:20.272+0000 7fcac2ffd700 1 -- 172.21.15.94:0/2011984184 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcac40fce70 msgr2=0x7fcac410da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:20.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:20.272+0000 7fcac2ffd700 1 --2- 172.21.15.94:0/2011984184 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcac40fce70 0x7fcac410da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:20.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:20.272+0000 7fcac2ffd700 1 -- 172.21.15.94:0/2011984184 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab4009940 con 0x7fcac40fdfe0 2024-03-24T10:44:20.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:20.272+0000 7fcac2ffd700 1 --2- 172.21.15.94:0/2011984184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcac40fdfe0 0x7fcac410dfd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fcab800c8c0 tx=0x7fcab80044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:44:20.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:20.272+0000 7fcaabfff700 1 -- 172.21.15.94:0/2011984184 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcab800a760 con 0x7fcac40fdfe0 2024-03-24T10:44:20.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:20.272+0000 7fcaabfff700 1 -- 172.21.15.94:0/2011984184 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcab800ad90 con 0x7fcac40fdfe0 2024-03-24T10:44:20.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:20.272+0000 7fcaca089700 1 -- 172.21.15.94:0/2011984184 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcac41131a0 con 0x7fcac40fdfe0 2024-03-24T10:44:20.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:20.272+0000 7fcaabfff700 1 -- 172.21.15.94:0/2011984184 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcab80113a0 con 0x7fcac40fdfe0 2024-03-24T10:44:20.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:20.272+0000 7fcaca089700 1 -- 172.21.15.94:0/2011984184 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcac4113760 con 0x7fcac40fdfe0 2024-03-24T10:44:20.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:20.273+0000 7fcaca089700 1 -- 172.21.15.94:0/2011984184 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcac404c710 con 0x7fcac40fdfe0 2024-03-24T10:44:20.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:20.273+0000 7fcaabfff700 1 -- 172.21.15.94:0/2011984184 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fcab8012400 con 0x7fcac40fdfe0 2024-03-24T10:44:20.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:20.274+0000 7fcaabfff700 1 --2- 172.21.15.94:0/2011984184 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fcaac0692a0 0x7fcaac06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:20.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:20.274+0000 7fcaabfff700 1 -- 172.21.15.94:0/2011984184 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fcab8087200 con 0x7fcac40fdfe0 2024-03-24T10:44:20.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:20.274+0000 7fcac37fe700 1 --2- 172.21.15.94:0/2011984184 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fcaac0692a0 0x7fcaac06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:20.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:20.274+0000 7fcac37fe700 1 --2- 172.21.15.94:0/2011984184 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fcaac0692a0 0x7fcaac06b750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcab4015040 tx=0x7fcab4004d10 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:44:20.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:20.278+0000 7fcaabfff700 1 -- 172.21.15.94:0/2011984184 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fcab8056f70 con 0x7fcac40fdfe0 2024-03-24T10:44:20.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:19 smithi096 ceph-mon[38097]: pgmap v193: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-03-24T10:44:20.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:20.463+0000 7fcaca089700 1 -- 172.21.15.94:0/2011984184 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7fcac41058d0 con 0x7fcac40fdfe0 2024-03-24T10:44:20.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:20.463+0000 7fcaabfff700 1 -- 172.21.15.94:0/2011984184 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=-22 unrecognized config option 'mgr/orchestrator/fail_fs' v21)=-22 unrecognized config option 'mgr/orchestrator/fail_fs' v21) v1 ==== 179+0+0 (secure 0 0 0) 0x7fcab8056b00 con 0x7fcac40fdfe0 2024-03-24T10:44:20.466 INFO:teuthology.orchestra.run.smithi094.stderr:Error EINVAL: unrecognized config option 'mgr/orchestrator/fail_fs' 2024-03-24T10:44:20.982 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-24T10:44:20.982 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi094.front.sepia.ceph.com 2024-03-24T10:44:20.982 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -e sha1=21b3bd672163092b4e542f772797fcba02d3d4de -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-03-24T10:44:21.281 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:44:22.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:21 smithi094 ceph-mon[29446]: pgmap v194: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-03-24T10:44:22.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:21 smithi096 ceph-mon[38097]: pgmap v194: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-03-24T10:44:22.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:22.936+0000 7f32cc929700 1 -- 172.21.15.94:0/2525175924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32c81002e0 msgr2=0x7f32c81026c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:22.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:22.936+0000 7f32cc929700 1 --2- 172.21.15.94:0/2525175924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32c81002e0 0x7f32c81026c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f32bc004940 tx=0x7f32bc009e30 comp rx=0 tx=0).stop 2024-03-24T10:44:22.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:22.936+0000 7f32cc929700 1 -- 172.21.15.94:0/2525175924 shutdown_connections 2024-03-24T10:44:22.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:22.936+0000 7f32cc929700 1 --2- 172.21.15.94:0/2525175924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32c81002e0 0x7f32c81026c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:22.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:22.936+0000 7f32cc929700 1 --2- 172.21.15.94:0/2525175924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32c80fd9c0 0x7f32c80ffda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:22.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:22.936+0000 7f32cc929700 1 -- 172.21.15.94:0/2525175924 >> 172.21.15.94:0/2525175924 conn(0x7f32c80f7480 msgr2=0x7f32c80f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:44:22.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:22.936+0000 7f32cc929700 1 -- 172.21.15.94:0/2525175924 shutdown_connections 2024-03-24T10:44:22.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:22.936+0000 7f32cc929700 1 -- 172.21.15.94:0/2525175924 wait complete. 2024-03-24T10:44:22.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:22.936+0000 7f32cc929700 1 Processor -- start 2024-03-24T10:44:22.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:22.936+0000 7f32cc929700 1 -- start start 2024-03-24T10:44:22.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:22.937+0000 7f32cc929700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32c80fd9c0 0x7f32c80fd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:22.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:22.937+0000 7f32cc929700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32c81002e0 0x7f32c80fb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:22.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:22.937+0000 7f32cc929700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c80fbe60 con 0x7f32c80fd9c0 2024-03-24T10:44:22.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:22.937+0000 7f32cc929700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c80fbfa0 con 0x7f32c81002e0 2024-03-24T10:44:22.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:22.937+0000 7f32c659c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32c80fd9c0 0x7f32c80fd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:22.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:22.937+0000 7f32c659c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32c80fd9c0 0x7f32c80fd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45312/0 (socket says 172.21.15.94:45312) 2024-03-24T10:44:22.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:22.937+0000 7f32c659c700 1 -- 172.21.15.94:0/2832704686 learned_addr learned my addr 172.21.15.94:0/2832704686 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:44:22.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:22.937+0000 7f32c5d9b700 1 --2- 172.21.15.94:0/2832704686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32c81002e0 0x7f32c80fb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:22.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:22.937+0000 7f32c659c700 1 -- 172.21.15.94:0/2832704686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32c81002e0 msgr2=0x7f32c80fb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:22.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:22.937+0000 7f32c659c700 1 --2- 172.21.15.94:0/2832704686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32c81002e0 0x7f32c80fb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:22.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:22.937+0000 7f32c659c700 1 -- 172.21.15.94:0/2832704686 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32bc005040 con 0x7f32c80fd9c0 2024-03-24T10:44:22.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:22.937+0000 7f32c659c700 1 --2- 172.21.15.94:0/2832704686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32c80fd9c0 0x7f32c80fd3b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f32b800eb40 tx=0x7f32b800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:44:22.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:22.937+0000 7f32b6ffd700 1 -- 172.21.15.94:0/2832704686 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32b8007cc0 con 0x7f32c80fd9c0 2024-03-24T10:44:22.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:22.938+0000 7f32b6ffd700 1 -- 172.21.15.94:0/2832704686 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f32b800f040 con 0x7f32c80fd9c0 2024-03-24T10:44:22.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:22.938+0000 7f32cc929700 1 -- 172.21.15.94:0/2832704686 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32c80fc220 con 0x7f32c80fd9c0 2024-03-24T10:44:22.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:22.938+0000 7f32b6ffd700 1 -- 172.21.15.94:0/2832704686 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32b800c9d0 con 0x7f32c80fd9c0 2024-03-24T10:44:22.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:22.938+0000 7f32cc929700 1 -- 172.21.15.94:0/2832704686 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32c80fc6e0 con 0x7f32c80fd9c0 2024-03-24T10:44:22.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:22.939+0000 7f32cc929700 1 -- 172.21.15.94:0/2832704686 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32c805f5f0 con 0x7f32c80fd9c0 2024-03-24T10:44:22.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:22.939+0000 7f32b6ffd700 1 -- 172.21.15.94:0/2832704686 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f32b800cc40 con 0x7f32c80fd9c0 2024-03-24T10:44:22.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:22.939+0000 7f32b6ffd700 1 --2- 172.21.15.94:0/2832704686 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f32b00692b0 0x7f32b006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:22.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:22.940+0000 7f32b6ffd700 1 -- 172.21.15.94:0/2832704686 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f32b8086c10 con 0x7f32c80fd9c0 2024-03-24T10:44:22.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:22.940+0000 7f32c5d9b700 1 --2- 172.21.15.94:0/2832704686 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f32b00692b0 0x7f32b006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:22.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:22.940+0000 7f32c5d9b700 1 --2- 172.21.15.94:0/2832704686 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f32b00692b0 0x7f32b006b760 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f32bc00b0d0 tx=0x7f32bc009e30 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:44:22.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:22.944+0000 7f32b6ffd700 1 -- 172.21.15.94:0/2832704686 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f32b804e190 con 0x7f32c80fd9c0 2024-03-24T10:44:23.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:23.150+0000 7f32cc929700 1 -- 172.21.15.94:0/2832704686 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f32c8115ac0 con 0x7f32c80fd9c0 2024-03-24T10:44:23.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:23.150+0000 7f32b6ffd700 1 -- 172.21.15.94:0/2832704686 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v21)=0 v21) v1 ==== 155+0+0 (secure 0 0 0) 0x7f32b8052f20 con 0x7f32c80fd9c0 2024-03-24T10:44:23.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:23.152+0000 7f32cc929700 1 -- 172.21.15.94:0/2832704686 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f32b00692b0 msgr2=0x7f32b006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:23.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:23.152+0000 7f32cc929700 1 --2- 172.21.15.94:0/2832704686 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f32b00692b0 0x7f32b006b760 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f32bc00b0d0 tx=0x7f32bc009e30 comp rx=0 tx=0).stop 2024-03-24T10:44:23.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:23.152+0000 7f32cc929700 1 -- 172.21.15.94:0/2832704686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32c80fd9c0 msgr2=0x7f32c80fd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:23.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:23.152+0000 7f32cc929700 1 --2- 172.21.15.94:0/2832704686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32c80fd9c0 0x7f32c80fd3b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f32b800eb40 tx=0x7f32b800b040 comp rx=0 tx=0).stop 2024-03-24T10:44:23.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:23.152+0000 7f32cc929700 1 -- 172.21.15.94:0/2832704686 shutdown_connections 2024-03-24T10:44:23.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:23.152+0000 7f32cc929700 1 --2- 172.21.15.94:0/2832704686 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f32b00692b0 0x7f32b006b760 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:23.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:23.152+0000 7f32cc929700 1 --2- 172.21.15.94:0/2832704686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32c80fd9c0 0x7f32c80fd3b0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:23.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:23.152+0000 7f32cc929700 1 --2- 172.21.15.94:0/2832704686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32c81002e0 0x7f32c80fb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:23.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:23.152+0000 7f32cc929700 1 -- 172.21.15.94:0/2832704686 >> 172.21.15.94:0/2832704686 conn(0x7f32c80f7480 msgr2=0x7f32c80f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:44:23.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:23.152+0000 7f32cc929700 1 -- 172.21.15.94:0/2832704686 shutdown_connections 2024-03-24T10:44:23.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:23.152+0000 7f32cc929700 1 -- 172.21.15.94:0/2832704686 wait complete. 2024-03-24T10:44:23.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:23 smithi094 ceph-mon[29446]: pgmap v195: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 682 B/s wr, 3 op/s 2024-03-24T10:44:24.049 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -e sha1=21b3bd672163092b4e542f772797fcba02d3d4de -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-03-24T10:44:24.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:23 smithi096 ceph-mon[38097]: pgmap v195: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 682 B/s wr, 3 op/s 2024-03-24T10:44:24.330 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:44:25.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:25.976+0000 7f81b64cb700 1 -- 172.21.15.94:0/1823680058 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81b00fcfa0 msgr2=0x7f81b00fd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:25.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:25.976+0000 7f81b64cb700 1 --2- 172.21.15.94:0/1823680058 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81b00fcfa0 0x7f81b00fd410 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8198004940 tx=0x7f8198009e30 comp rx=0 tx=0).stop 2024-03-24T10:44:25.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:25.976+0000 7f81b64cb700 1 -- 172.21.15.94:0/1823680058 shutdown_connections 2024-03-24T10:44:25.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:25.976+0000 7f81b64cb700 1 --2- 172.21.15.94:0/1823680058 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81b00fcfa0 0x7f81b00fd410 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:25.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:25.976+0000 7f81b64cb700 1 --2- 172.21.15.94:0/1823680058 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81b00fbe30 0x7f81b00fc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:25.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:25.976+0000 7f81b64cb700 1 -- 172.21.15.94:0/1823680058 >> 172.21.15.94:0/1823680058 conn(0x7f81b00f7480 msgr2=0x7f81b00f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:44:25.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:25.976+0000 7f81b64cb700 1 -- 172.21.15.94:0/1823680058 shutdown_connections 2024-03-24T10:44:25.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:25.976+0000 7f81b64cb700 1 -- 172.21.15.94:0/1823680058 wait complete. 2024-03-24T10:44:25.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:25.976+0000 7f81b64cb700 1 Processor -- start 2024-03-24T10:44:25.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:25.976+0000 7f81b64cb700 1 -- start start 2024-03-24T10:44:25.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:25.977+0000 7f81b64cb700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81b00fbe30 0x7f81b010d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:25.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:25.977+0000 7f81b64cb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81b00fcfa0 0x7f81b010df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:25.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:25.977+0000 7f81b64cb700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b010e530 con 0x7f81b00fbe30 2024-03-24T10:44:25.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:25.977+0000 7f81b64cb700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b010ed80 con 0x7f81b00fcfa0 2024-03-24T10:44:25.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:25.977+0000 7f81a7fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81b00fbe30 0x7f81b010d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:25.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:25.977+0000 7f81a7fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81b00fbe30 0x7f81b010d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45318/0 (socket says 172.21.15.94:45318) 2024-03-24T10:44:25.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:25.977+0000 7f81a7fff700 1 -- 172.21.15.94:0/3208530282 learned_addr learned my addr 172.21.15.94:0/3208530282 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:44:25.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:25.977+0000 7f81affff700 1 --2- 172.21.15.94:0/3208530282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81b00fcfa0 0x7f81b010df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:25.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:25.977+0000 7f81a7fff700 1 -- 172.21.15.94:0/3208530282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81b00fcfa0 msgr2=0x7f81b010df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:25.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:25.977+0000 7f81a7fff700 1 --2- 172.21.15.94:0/3208530282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81b00fcfa0 0x7f81b010df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:25.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:25.977+0000 7f81a7fff700 1 -- 172.21.15.94:0/3208530282 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8198005040 con 0x7f81b00fbe30 2024-03-24T10:44:25.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:25.977+0000 7f81a7fff700 1 --2- 172.21.15.94:0/3208530282 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81b00fbe30 0x7f81b010d9f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f81a0002a50 tx=0x7f81a00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:44:25.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:25.977+0000 7f81ad7fa700 1 -- 172.21.15.94:0/3208530282 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81a0007cc0 con 0x7f81b00fbe30 2024-03-24T10:44:25.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:25.977+0000 7f81b64cb700 1 -- 172.21.15.94:0/3208530282 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81b010f060 con 0x7f81b00fbe30 2024-03-24T10:44:25.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:25.977+0000 7f81ad7fa700 1 -- 172.21.15.94:0/3208530282 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f81a000f040 con 0x7f81b00fbe30 2024-03-24T10:44:25.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:25.978+0000 7f81b64cb700 1 -- 172.21.15.94:0/3208530282 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81b010f5a0 con 0x7f81b00fbe30 2024-03-24T10:44:25.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:25.978+0000 7f81ad7fa700 1 -- 172.21.15.94:0/3208530282 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81a000da30 con 0x7f81b00fbe30 2024-03-24T10:44:25.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:25.979+0000 7f81ad7fa700 1 -- 172.21.15.94:0/3208530282 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f81a000dc70 con 0x7f81b00fbe30 2024-03-24T10:44:25.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:25.979+0000 7f81b64cb700 1 -- 172.21.15.94:0/3208530282 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81900052e0 con 0x7f81b00fbe30 2024-03-24T10:44:25.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:25.979+0000 7f81ad7fa700 1 --2- 172.21.15.94:0/3208530282 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f819c0692a0 0x7f819c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:25.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:25.980+0000 7f81ad7fa700 1 -- 172.21.15.94:0/3208530282 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f81a0086f10 con 0x7f81b00fbe30 2024-03-24T10:44:25.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:25.980+0000 7f81affff700 1 --2- 172.21.15.94:0/3208530282 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f819c0692a0 0x7f819c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:25.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:25.980+0000 7f81affff700 1 --2- 172.21.15.94:0/3208530282 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f819c0692a0 0x7f819c06b750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f819800b3b0 tx=0x7f8198009e30 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:44:25.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:25.988+0000 7f81ad7fa700 1 -- 172.21.15.94:0/3208530282 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f81a0057cc0 con 0x7f81b00fbe30 2024-03-24T10:44:26.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:26.188+0000 7f81b64cb700 1 -- 172.21.15.94:0/3208530282 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f8190005590 con 0x7f81b00fbe30 2024-03-24T10:44:26.189 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:25 smithi094 ceph-mon[29446]: pgmap v196: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 682 B/s wr, 2 op/s 2024-03-24T10:44:26.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:26.188+0000 7f81ad7fa700 1 -- 172.21.15.94:0/3208530282 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v21)=0 v21) v1 ==== 163+0+0 (secure 0 0 0) 0x7f81a0057850 con 0x7f81b00fbe30 2024-03-24T10:44:26.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:26.191+0000 7f81b64cb700 1 -- 172.21.15.94:0/3208530282 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f819c0692a0 msgr2=0x7f819c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:26.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:26.191+0000 7f81b64cb700 1 --2- 172.21.15.94:0/3208530282 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f819c0692a0 0x7f819c06b750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f819800b3b0 tx=0x7f8198009e30 comp rx=0 tx=0).stop 2024-03-24T10:44:26.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:26.191+0000 7f81b64cb700 1 -- 172.21.15.94:0/3208530282 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81b00fbe30 msgr2=0x7f81b010d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:26.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:26.191+0000 7f81b64cb700 1 --2- 172.21.15.94:0/3208530282 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81b00fbe30 0x7f81b010d9f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f81a0002a50 tx=0x7f81a00046b0 comp rx=0 tx=0).stop 2024-03-24T10:44:26.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:26.191+0000 7f81b64cb700 1 -- 172.21.15.94:0/3208530282 shutdown_connections 2024-03-24T10:44:26.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:26.191+0000 7f81b64cb700 1 --2- 172.21.15.94:0/3208530282 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f819c0692a0 0x7f819c06b750 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:26.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:26.191+0000 7f81b64cb700 1 --2- 172.21.15.94:0/3208530282 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81b00fbe30 0x7f81b010d9f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:26.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:26.191+0000 7f81b64cb700 1 --2- 172.21.15.94:0/3208530282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81b00fcfa0 0x7f81b010df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:26.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:26.191+0000 7f81b64cb700 1 -- 172.21.15.94:0/3208530282 >> 172.21.15.94:0/3208530282 conn(0x7f81b00f7480 msgr2=0x7f81b00f8c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:44:26.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:26.191+0000 7f81b64cb700 1 -- 172.21.15.94:0/3208530282 shutdown_connections 2024-03-24T10:44:26.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:26.191+0000 7f81b64cb700 1 -- 172.21.15.94:0/3208530282 wait complete. 2024-03-24T10:44:26.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:25 smithi096 ceph-mon[38097]: pgmap v196: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 682 B/s wr, 2 op/s 2024-03-24T10:44:26.884 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -e sha1=21b3bd672163092b4e542f772797fcba02d3d4de -- bash -c 'ceph config set global log_to_journald false --force' 2024-03-24T10:44:27.152 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:44:28.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:27 smithi096 ceph-mon[38097]: pgmap v197: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 767 B/s wr, 3 op/s 2024-03-24T10:44:28.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:27 smithi094 ceph-mon[29446]: pgmap v197: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 767 B/s wr, 3 op/s 2024-03-24T10:44:28.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:28.792+0000 7f84cd594700 1 -- 172.21.15.94:0/1416693734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84c8066280 msgr2=0x7f84c8101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:28.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:28.792+0000 7f84cd594700 1 --2- 172.21.15.94:0/1416693734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84c8066280 0x7f84c8101e20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f84b8004940 tx=0x7f84b8009e30 comp rx=0 tx=0).stop 2024-03-24T10:44:28.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:28.792+0000 7f84cd594700 1 -- 172.21.15.94:0/1416693734 shutdown_connections 2024-03-24T10:44:28.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:28.792+0000 7f84cd594700 1 --2- 172.21.15.94:0/1416693734 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84c8102360 0x7f84c8104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:28.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:28.792+0000 7f84cd594700 1 --2- 172.21.15.94:0/1416693734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84c8066280 0x7f84c8101e20 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:28.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:28.792+0000 7f84cd594700 1 -- 172.21.15.94:0/1416693734 >> 172.21.15.94:0/1416693734 conn(0x7f84c80f7430 msgr2=0x7f84c80f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:44:28.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:28.792+0000 7f84cd594700 1 -- 172.21.15.94:0/1416693734 shutdown_connections 2024-03-24T10:44:28.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:28.792+0000 7f84cd594700 1 -- 172.21.15.94:0/1416693734 wait complete. 2024-03-24T10:44:28.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:28.793+0000 7f84cd594700 1 Processor -- start 2024-03-24T10:44:28.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:28.793+0000 7f84cd594700 1 -- start start 2024-03-24T10:44:28.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:28.793+0000 7f84cd594700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84c8066280 0x7f84c810fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:28.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:28.793+0000 7f84cd594700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84c8102360 0x7f84c81101d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:28.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:28.793+0000 7f84cd594700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c81107d0 con 0x7f84c8102360 2024-03-24T10:44:28.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:28.793+0000 7f84cd594700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c8110fd0 con 0x7f84c8066280 2024-03-24T10:44:28.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:28.793+0000 7f84c6ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84c8066280 0x7f84c810fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:28.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:28.793+0000 7f84c67fc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84c8102360 0x7f84c81101d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:28.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:28.793+0000 7f84c6ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84c8066280 0x7f84c810fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.94:43974/0 (socket says 172.21.15.94:43974) 2024-03-24T10:44:28.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:28.793+0000 7f84c6ffd700 1 -- 172.21.15.94:0/3142990900 learned_addr learned my addr 172.21.15.94:0/3142990900 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:44:28.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:28.793+0000 7f84c6ffd700 1 -- 172.21.15.94:0/3142990900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84c8102360 msgr2=0x7f84c81101d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:28.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:28.793+0000 7f84c6ffd700 1 --2- 172.21.15.94:0/3142990900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84c8102360 0x7f84c81101d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:28.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:28.793+0000 7f84c6ffd700 1 -- 172.21.15.94:0/3142990900 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b8005040 con 0x7f84c8066280 2024-03-24T10:44:28.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:28.793+0000 7f84c67fc700 1 --2- 172.21.15.94:0/3142990900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84c8102360 0x7f84c81101d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T10:44:28.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:28.794+0000 7f84c6ffd700 1 --2- 172.21.15.94:0/3142990900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84c8066280 0x7f84c810fc90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f84b8000c00 tx=0x7f84b8009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:44:28.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:28.794+0000 7f84af7fe700 1 -- 172.21.15.94:0/3142990900 <== mon.1 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84b800e030 con 0x7f84c8066280 2024-03-24T10:44:28.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:28.794+0000 7f84cd594700 1 -- 172.21.15.94:0/3142990900 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84c8111250 con 0x7f84c8066280 2024-03-24T10:44:28.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:28.794+0000 7f84af7fe700 1 -- 172.21.15.94:0/3142990900 <== mon.1 v2:172.21.15.96:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f84b8013de0 con 0x7f84c8066280 2024-03-24T10:44:28.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:28.794+0000 7f84af7fe700 1 -- 172.21.15.94:0/3142990900 <== mon.1 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84b801ce50 con 0x7f84c8066280 2024-03-24T10:44:28.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:28.794+0000 7f84cd594700 1 -- 172.21.15.94:0/3142990900 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84c8111790 con 0x7f84c8066280 2024-03-24T10:44:28.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:28.795+0000 7f84cd594700 1 -- 172.21.15.94:0/3142990900 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84c805f5f0 con 0x7f84c8066280 2024-03-24T10:44:28.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:28.796+0000 7f84af7fe700 1 -- 172.21.15.94:0/3142990900 <== mon.1 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f84b801c400 con 0x7f84c8066280 2024-03-24T10:44:28.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:28.796+0000 7f84af7fe700 1 --2- 172.21.15.94:0/3142990900 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f84b0069230 0x7f84b006b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:28.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:28.796+0000 7f84af7fe700 1 -- 172.21.15.94:0/3142990900 <== mon.1 v2:172.21.15.96:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f84b80890c0 con 0x7f84c8066280 2024-03-24T10:44:28.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:28.796+0000 7f84c67fc700 1 --2- 172.21.15.94:0/3142990900 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f84b0069230 0x7f84b006b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:28.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:28.797+0000 7f84c67fc700 1 --2- 172.21.15.94:0/3142990900 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f84b0069230 0x7f84b006b6e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f84bc004fd0 tx=0x7f84bc00b040 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:44:28.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:28.800+0000 7f84af7fe700 1 -- 172.21.15.94:0/3142990900 <== mon.1 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f84b8058d90 con 0x7f84c8066280 2024-03-24T10:44:29.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:29.008+0000 7f84cd594700 1 -- 172.21.15.94:0/3142990900 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f84c804bed0 con 0x7f84c8066280 2024-03-24T10:44:29.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:29.009+0000 7f84af7fe700 1 -- 172.21.15.94:0/3142990900 <== mon.1 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v21)=0 v21) v1 ==== 135+0+0 (secure 0 0 0) 0x7f84b8058920 con 0x7f84c8066280 2024-03-24T10:44:29.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:29.011+0000 7f84cd594700 1 -- 172.21.15.94:0/3142990900 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f84b0069230 msgr2=0x7f84b006b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:29.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:29.011+0000 7f84cd594700 1 --2- 172.21.15.94:0/3142990900 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f84b0069230 0x7f84b006b6e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f84bc004fd0 tx=0x7f84bc00b040 comp rx=0 tx=0).stop 2024-03-24T10:44:29.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:29.011+0000 7f84cd594700 1 -- 172.21.15.94:0/3142990900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84c8066280 msgr2=0x7f84c810fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:29.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:29.011+0000 7f84cd594700 1 --2- 172.21.15.94:0/3142990900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84c8066280 0x7f84c810fc90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f84b8000c00 tx=0x7f84b8009e30 comp rx=0 tx=0).stop 2024-03-24T10:44:29.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:29.011+0000 7f84cd594700 1 -- 172.21.15.94:0/3142990900 shutdown_connections 2024-03-24T10:44:29.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:29.011+0000 7f84cd594700 1 --2- 172.21.15.94:0/3142990900 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f84b0069230 0x7f84b006b6e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:29.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:29.011+0000 7f84cd594700 1 --2- 172.21.15.94:0/3142990900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84c8066280 0x7f84c810fc90 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:29.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:29.011+0000 7f84cd594700 1 --2- 172.21.15.94:0/3142990900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84c8102360 0x7f84c81101d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:29.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:29.011+0000 7f84cd594700 1 -- 172.21.15.94:0/3142990900 >> 172.21.15.94:0/3142990900 conn(0x7f84c80f7430 msgr2=0x7f84c80f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:44:29.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:29.011+0000 7f84cd594700 1 -- 172.21.15.94:0/3142990900 shutdown_connections 2024-03-24T10:44:29.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:29.011+0000 7f84cd594700 1 -- 172.21.15.94:0/3142990900 wait complete. 2024-03-24T10:44:29.619 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -e sha1=21b3bd672163092b4e542f772797fcba02d3d4de -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-03-24T10:44:29.895 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:44:30.158 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:29 smithi094 ceph-mon[29446]: pgmap v198: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-24T10:44:30.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:29 smithi096 ceph-mon[38097]: pgmap v198: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-24T10:44:30.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:30 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:44:31.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:30 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:44:31.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:31.552+0000 7fa24da0b700 1 -- 172.21.15.94:0/2675364160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2480fcf90 msgr2=0x7fa2480fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:31.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:31.552+0000 7fa24da0b700 1 --2- 172.21.15.94:0/2675364160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2480fcf90 0x7fa2480fd400 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa23c004940 tx=0x7fa23c009e30 comp rx=0 tx=0).stop 2024-03-24T10:44:31.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:31.552+0000 7fa24da0b700 1 -- 172.21.15.94:0/2675364160 shutdown_connections 2024-03-24T10:44:31.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:31.552+0000 7fa24da0b700 1 --2- 172.21.15.94:0/2675364160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2480fcf90 0x7fa2480fd400 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:31.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:31.552+0000 7fa24da0b700 1 --2- 172.21.15.94:0/2675364160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2480fbce0 0x7fa2480fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:31.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:31.552+0000 7fa24da0b700 1 -- 172.21.15.94:0/2675364160 >> 172.21.15.94:0/2675364160 conn(0x7fa2480f7480 msgr2=0x7fa2480f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:44:31.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:31.552+0000 7fa24da0b700 1 -- 172.21.15.94:0/2675364160 shutdown_connections 2024-03-24T10:44:31.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:31.552+0000 7fa24da0b700 1 -- 172.21.15.94:0/2675364160 wait complete. 2024-03-24T10:44:31.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:31.553+0000 7fa24da0b700 1 Processor -- start 2024-03-24T10:44:31.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:31.553+0000 7fa24da0b700 1 -- start start 2024-03-24T10:44:31.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:31.553+0000 7fa24da0b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2480fbce0 0x7fa24810c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:31.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:31.553+0000 7fa24da0b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2480fcf90 0x7fa24810cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:31.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:31.553+0000 7fa24da0b700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa24810d3e0 con 0x7fa2480fcf90 2024-03-24T10:44:31.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:31.553+0000 7fa24da0b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa24810dc30 con 0x7fa2480fbce0 2024-03-24T10:44:31.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:31.553+0000 7fa2467fc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2480fcf90 0x7fa24810cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:31.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:31.553+0000 7fa2467fc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2480fcf90 0x7fa24810cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38672/0 (socket says 172.21.15.94:38672) 2024-03-24T10:44:31.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:31.553+0000 7fa2467fc700 1 -- 172.21.15.94:0/1753650547 learned_addr learned my addr 172.21.15.94:0/1753650547 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:44:31.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:31.553+0000 7fa246ffd700 1 --2- 172.21.15.94:0/1753650547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2480fbce0 0x7fa24810c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:31.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:31.554+0000 7fa2467fc700 1 -- 172.21.15.94:0/1753650547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2480fbce0 msgr2=0x7fa24810c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:31.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:31.554+0000 7fa2467fc700 1 --2- 172.21.15.94:0/1753650547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2480fbce0 0x7fa24810c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:31.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:31.554+0000 7fa2467fc700 1 -- 172.21.15.94:0/1753650547 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa23c005040 con 0x7fa2480fcf90 2024-03-24T10:44:31.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:31.554+0000 7fa246ffd700 1 --2- 172.21.15.94:0/1753650547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2480fbce0 0x7fa24810c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T10:44:31.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:31.554+0000 7fa2467fc700 1 --2- 172.21.15.94:0/1753650547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2480fcf90 0x7fa24810cde0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa23c013870 tx=0x7fa23c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:44:31.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:31.554+0000 7fa22f7fe700 1 -- 172.21.15.94:0/1753650547 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa23c021070 con 0x7fa2480fcf90 2024-03-24T10:44:31.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:31.554+0000 7fa22f7fe700 1 -- 172.21.15.94:0/1753650547 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa23c009430 con 0x7fa2480fcf90 2024-03-24T10:44:31.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:31.554+0000 7fa24da0b700 1 -- 172.21.15.94:0/1753650547 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa24810de30 con 0x7fa2480fcf90 2024-03-24T10:44:31.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:31.554+0000 7fa22f7fe700 1 -- 172.21.15.94:0/1753650547 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa23c00e070 con 0x7fa2480fcf90 2024-03-24T10:44:31.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:31.554+0000 7fa24da0b700 1 -- 172.21.15.94:0/1753650547 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa24810e370 con 0x7fa2480fcf90 2024-03-24T10:44:31.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:31.555+0000 7fa24da0b700 1 -- 172.21.15.94:0/1753650547 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa24805f5f0 con 0x7fa2480fcf90 2024-03-24T10:44:31.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:31.556+0000 7fa22f7fe700 1 -- 172.21.15.94:0/1753650547 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fa23c013af0 con 0x7fa2480fcf90 2024-03-24T10:44:31.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:31.556+0000 7fa22f7fe700 1 --2- 172.21.15.94:0/1753650547 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fa230069230 0x7fa23006b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:31.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:31.556+0000 7fa22f7fe700 1 -- 172.21.15.94:0/1753650547 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fa23c088490 con 0x7fa2480fcf90 2024-03-24T10:44:31.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:31.556+0000 7fa246ffd700 1 --2- 172.21.15.94:0/1753650547 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fa230069230 0x7fa23006b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:31.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:31.557+0000 7fa246ffd700 1 --2- 172.21.15.94:0/1753650547 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fa230069230 0x7fa23006b6e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa238009b80 tx=0x7fa23800b040 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:44:31.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:31.560+0000 7fa22f7fe700 1 -- 172.21.15.94:0/1753650547 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fa23c058200 con 0x7fa2480fcf90 2024-03-24T10:44:31.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:31.803+0000 7fa24da0b700 1 -- 172.21.15.94:0/1753650547 --> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de", "target": ["mon-mgr", ""]}) v1 -- 0x7fa24810e790 con 0x7fa230069230 2024-03-24T10:44:31.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:31.809+0000 7fa22f7fe700 1 -- 172.21.15.94:0/1753650547 <== mgr.14232 v2:172.21.15.94:6800/2184357820 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7fa24810e790 con 0x7fa230069230 2024-03-24T10:44:31.811 INFO:teuthology.orchestra.run.smithi094.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de 2024-03-24T10:44:31.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:31.811+0000 7fa24da0b700 1 -- 172.21.15.94:0/1753650547 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fa230069230 msgr2=0x7fa23006b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:31.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:31.811+0000 7fa24da0b700 1 --2- 172.21.15.94:0/1753650547 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fa230069230 0x7fa23006b6e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa238009b80 tx=0x7fa23800b040 comp rx=0 tx=0).stop 2024-03-24T10:44:31.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:31.811+0000 7fa24da0b700 1 -- 172.21.15.94:0/1753650547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2480fcf90 msgr2=0x7fa24810cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:31.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:31.811+0000 7fa24da0b700 1 --2- 172.21.15.94:0/1753650547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2480fcf90 0x7fa24810cde0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa23c013870 tx=0x7fa23c009e30 comp rx=0 tx=0).stop 2024-03-24T10:44:31.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:31.812+0000 7fa24da0b700 1 -- 172.21.15.94:0/1753650547 shutdown_connections 2024-03-24T10:44:31.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:31.812+0000 7fa24da0b700 1 --2- 172.21.15.94:0/1753650547 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fa230069230 0x7fa23006b6e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:31.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:31.812+0000 7fa24da0b700 1 --2- 172.21.15.94:0/1753650547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2480fbce0 0x7fa24810c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:31.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:31.812+0000 7fa24da0b700 1 --2- 172.21.15.94:0/1753650547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2480fcf90 0x7fa24810cde0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:31.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:31.812+0000 7fa24da0b700 1 -- 172.21.15.94:0/1753650547 >> 172.21.15.94:0/1753650547 conn(0x7fa2480f7480 msgr2=0x7fa2480f8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:44:31.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:31.812+0000 7fa24da0b700 1 -- 172.21.15.94:0/1753650547 shutdown_connections 2024-03-24T10:44:31.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:31.812+0000 7fa24da0b700 1 -- 172.21.15.94:0/1753650547 wait complete. 2024-03-24T10:44:32.178 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:31 smithi094 ceph-mon[29446]: pgmap v199: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-24T10:44:32.179 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:31 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:44:32.179 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:31 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:44:32.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:31 smithi096 ceph-mon[38097]: pgmap v199: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-24T10:44:32.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:31 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:44:32.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:31 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:44:32.528 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-24T10:44:32.528 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi094.front.sepia.ceph.com 2024-03-24T10:44:32.529 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -e sha1=21b3bd672163092b4e542f772797fcba02d3d4de -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-03-24T10:44:32.877 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:44:33.182 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:32 smithi094 ceph-mon[29446]: from='client.14576 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:44:33.182 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:32 smithi094 ceph-mon[29446]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de 2024-03-24T10:44:33.182 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:32 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:44:33.182 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:32 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:44:33.182 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:32 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:44:33.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:32 smithi096 ceph-mon[38097]: from='client.14576 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:44:33.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:32 smithi096 ceph-mon[38097]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de 2024-03-24T10:44:33.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:32 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:44:33.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:32 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:44:33.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:32 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:44:34.190 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:33 smithi094 ceph-mon[29446]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de 2024-03-24T10:44:34.190 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:33 smithi094 ceph-mon[29446]: pgmap v200: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-24T10:44:34.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:33 smithi096 ceph-mon[38097]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de 2024-03-24T10:44:34.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:33 smithi096 ceph-mon[38097]: pgmap v200: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-24T10:44:34.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:34.713+0000 7f48e9f69700 1 -- 172.21.15.94:0/1592128782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48e40fff00 msgr2=0x7f48e4102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:34.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:34.713+0000 7f48e9f69700 1 --2- 172.21.15.94:0/1592128782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48e40fff00 0x7f48e4102320 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f48d8004940 tx=0x7f48d8009e30 comp rx=0 tx=0).stop 2024-03-24T10:44:34.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:34.713+0000 7f48e9f69700 1 -- 172.21.15.94:0/1592128782 shutdown_connections 2024-03-24T10:44:34.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:34.713+0000 7f48e9f69700 1 --2- 172.21.15.94:0/1592128782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48e40fff00 0x7f48e4102320 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:34.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:34.713+0000 7f48e9f69700 1 --2- 172.21.15.94:0/1592128782 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48e40662e0 0x7f48e40ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:34.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:34.713+0000 7f48e9f69700 1 -- 172.21.15.94:0/1592128782 >> 172.21.15.94:0/1592128782 conn(0x7f48e40f7430 msgr2=0x7f48e40f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:44:34.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:34.713+0000 7f48e9f69700 1 -- 172.21.15.94:0/1592128782 shutdown_connections 2024-03-24T10:44:34.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:34.713+0000 7f48e9f69700 1 -- 172.21.15.94:0/1592128782 wait complete. 2024-03-24T10:44:34.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:34.714+0000 7f48e9f69700 1 Processor -- start 2024-03-24T10:44:34.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:34.714+0000 7f48e9f69700 1 -- start start 2024-03-24T10:44:34.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:34.714+0000 7f48e9f69700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48e40662e0 0x7f48e410da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:34.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:34.714+0000 7f48e9f69700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48e40fff00 0x7f48e410df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:34.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:34.714+0000 7f48e9f69700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e410e580 con 0x7f48e40fff00 2024-03-24T10:44:34.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:34.714+0000 7f48e9f69700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e410ed80 con 0x7f48e40662e0 2024-03-24T10:44:34.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:34.714+0000 7f48e2ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48e40fff00 0x7f48e410df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:34.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:34.714+0000 7f48e2ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48e40fff00 0x7f48e410df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38688/0 (socket says 172.21.15.94:38688) 2024-03-24T10:44:34.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:34.714+0000 7f48e37fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48e40662e0 0x7f48e410da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:34.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:34.714+0000 7f48e2ffd700 1 -- 172.21.15.94:0/3218321909 learned_addr learned my addr 172.21.15.94:0/3218321909 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:44:34.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:34.715+0000 7f48e2ffd700 1 -- 172.21.15.94:0/3218321909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48e40662e0 msgr2=0x7f48e410da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:34.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:34.715+0000 7f48e2ffd700 1 --2- 172.21.15.94:0/3218321909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48e40662e0 0x7f48e410da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:34.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:34.715+0000 7f48e2ffd700 1 -- 172.21.15.94:0/3218321909 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d8005040 con 0x7f48e40fff00 2024-03-24T10:44:34.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:34.715+0000 7f48e37fe700 1 --2- 172.21.15.94:0/3218321909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48e40662e0 0x7f48e410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T10:44:34.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:34.715+0000 7f48e2ffd700 1 --2- 172.21.15.94:0/3218321909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48e40fff00 0x7f48e410df80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f48d8013870 tx=0x7f48d8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:44:34.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:34.715+0000 7f48cbfff700 1 -- 172.21.15.94:0/3218321909 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48d8021070 con 0x7f48e40fff00 2024-03-24T10:44:34.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:34.715+0000 7f48e9f69700 1 -- 172.21.15.94:0/3218321909 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48e410f000 con 0x7f48e40fff00 2024-03-24T10:44:34.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:34.715+0000 7f48cbfff700 1 -- 172.21.15.94:0/3218321909 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f48d8009430 con 0x7f48e40fff00 2024-03-24T10:44:34.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:34.715+0000 7f48e9f69700 1 -- 172.21.15.94:0/3218321909 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48e410f540 con 0x7f48e40fff00 2024-03-24T10:44:34.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:34.716+0000 7f48cbfff700 1 -- 172.21.15.94:0/3218321909 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48d800e070 con 0x7f48e40fff00 2024-03-24T10:44:34.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:34.716+0000 7f48e9f69700 1 -- 172.21.15.94:0/3218321909 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48e405f5f0 con 0x7f48e40fff00 2024-03-24T10:44:34.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:34.718+0000 7f48cbfff700 1 -- 172.21.15.94:0/3218321909 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f48d801c750 con 0x7f48e40fff00 2024-03-24T10:44:34.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:34.718+0000 7f48cbfff700 1 --2- 172.21.15.94:0/3218321909 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f48cc069230 0x7f48cc06b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:34.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:34.718+0000 7f48cbfff700 1 -- 172.21.15.94:0/3218321909 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f48d8088ae0 con 0x7f48e40fff00 2024-03-24T10:44:34.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:34.718+0000 7f48e37fe700 1 --2- 172.21.15.94:0/3218321909 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f48cc069230 0x7f48cc06b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:34.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:34.719+0000 7f48e37fe700 1 --2- 172.21.15.94:0/3218321909 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f48cc069230 0x7f48cc06b6e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f48d4009b80 tx=0x7f48d400b040 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:44:34.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:34.723+0000 7f48cbfff700 1 -- 172.21.15.94:0/3218321909 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f48d80588f0 con 0x7f48e40fff00 2024-03-24T10:44:34.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:34.983+0000 7f48e9f69700 1 -- 172.21.15.94:0/3218321909 --> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f48e410f820 con 0x7f48cc069230 2024-03-24T10:44:34.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:34.984+0000 7f48cbfff700 1 -- 172.21.15.94:0/3218321909 <== mgr.14232 v2:172.21.15.94:6800/2184357820 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f48e410f820 con 0x7f48cc069230 2024-03-24T10:44:34.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:34.987+0000 7f48e9f69700 1 -- 172.21.15.94:0/3218321909 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f48cc069230 msgr2=0x7f48cc06b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:34.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:34.987+0000 7f48e9f69700 1 --2- 172.21.15.94:0/3218321909 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f48cc069230 0x7f48cc06b6e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f48d4009b80 tx=0x7f48d400b040 comp rx=0 tx=0).stop 2024-03-24T10:44:34.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:34.987+0000 7f48e9f69700 1 -- 172.21.15.94:0/3218321909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48e40fff00 msgr2=0x7f48e410df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:34.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:34.987+0000 7f48e9f69700 1 --2- 172.21.15.94:0/3218321909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48e40fff00 0x7f48e410df80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f48d8013870 tx=0x7f48d8009e30 comp rx=0 tx=0).stop 2024-03-24T10:44:34.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:34.987+0000 7f48e9f69700 1 -- 172.21.15.94:0/3218321909 shutdown_connections 2024-03-24T10:44:34.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:34.987+0000 7f48e9f69700 1 --2- 172.21.15.94:0/3218321909 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f48cc069230 0x7f48cc06b6e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:34.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:34.987+0000 7f48e9f69700 1 --2- 172.21.15.94:0/3218321909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48e40662e0 0x7f48e410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:34.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:34.987+0000 7f48e9f69700 1 --2- 172.21.15.94:0/3218321909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48e40fff00 0x7f48e410df80 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:34.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:34.987+0000 7f48e9f69700 1 -- 172.21.15.94:0/3218321909 >> 172.21.15.94:0/3218321909 conn(0x7f48e40f7430 msgr2=0x7f48e40f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:44:34.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:34.987+0000 7f48e9f69700 1 -- 172.21.15.94:0/3218321909 shutdown_connections 2024-03-24T10:44:34.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:34.987+0000 7f48e9f69700 1 -- 172.21.15.94:0/3218321909 wait complete. 2024-03-24T10:44:35.000 INFO:teuthology.orchestra.run.smithi094.stdout:true 2024-03-24T10:44:35.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.103+0000 7fedadb36700 1 -- 172.21.15.94:0/237675752 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feda8066220 msgr2=0x7feda80ffa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:35.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.103+0000 7fedadb36700 1 --2- 172.21.15.94:0/237675752 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feda8066220 0x7feda80ffa80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fed98004940 tx=0x7fed98009e30 comp rx=0 tx=0).stop 2024-03-24T10:44:35.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.104+0000 7fedadb36700 1 -- 172.21.15.94:0/237675752 shutdown_connections 2024-03-24T10:44:35.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.104+0000 7fedadb36700 1 --2- 172.21.15.94:0/237675752 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feda80fffc0 0x7feda81023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:35.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.104+0000 7fedadb36700 1 --2- 172.21.15.94:0/237675752 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feda8066220 0x7feda80ffa80 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:35.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.104+0000 7fedadb36700 1 -- 172.21.15.94:0/237675752 >> 172.21.15.94:0/237675752 conn(0x7feda80f7430 msgr2=0x7feda80f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:44:35.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.104+0000 7fedadb36700 1 -- 172.21.15.94:0/237675752 shutdown_connections 2024-03-24T10:44:35.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.104+0000 7fedadb36700 1 -- 172.21.15.94:0/237675752 wait complete. 2024-03-24T10:44:35.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.104+0000 7fedadb36700 1 Processor -- start 2024-03-24T10:44:35.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.105+0000 7fedadb36700 1 -- start start 2024-03-24T10:44:35.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.105+0000 7fedadb36700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feda8066220 0x7feda80fd460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:35.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.105+0000 7fedadb36700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feda80fffc0 0x7feda80fb980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:35.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.105+0000 7fedadb36700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda80fbf10 con 0x7feda8066220 2024-03-24T10:44:35.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.105+0000 7fedadb36700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda80fc050 con 0x7feda80fffc0 2024-03-24T10:44:35.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.105+0000 7feda77fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feda8066220 0x7feda80fd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:35.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.105+0000 7feda77fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feda8066220 0x7feda80fd460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38706/0 (socket says 172.21.15.94:38706) 2024-03-24T10:44:35.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.105+0000 7feda77fe700 1 -- 172.21.15.94:0/3428020485 learned_addr learned my addr 172.21.15.94:0/3428020485 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:44:35.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.105+0000 7feda77fe700 1 -- 172.21.15.94:0/3428020485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feda80fffc0 msgr2=0x7feda80fb980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-24T10:44:35.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.105+0000 7feda77fe700 1 --2- 172.21.15.94:0/3428020485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feda80fffc0 0x7feda80fb980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:35.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.105+0000 7feda77fe700 1 -- 172.21.15.94:0/3428020485 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed98005040 con 0x7feda8066220 2024-03-24T10:44:35.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.106+0000 7feda77fe700 1 --2- 172.21.15.94:0/3428020485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feda8066220 0x7feda80fd460 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fed98004090 tx=0x7fed98009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:44:35.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.106+0000 7fed8ffff700 1 -- 172.21.15.94:0/3428020485 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed98021070 con 0x7feda8066220 2024-03-24T10:44:35.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.106+0000 7fedadb36700 1 -- 172.21.15.94:0/3428020485 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feda80fc2d0 con 0x7feda8066220 2024-03-24T10:44:35.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.106+0000 7fed8ffff700 1 -- 172.21.15.94:0/3428020485 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fed9801c400 con 0x7feda8066220 2024-03-24T10:44:35.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.106+0000 7fed8ffff700 1 -- 172.21.15.94:0/3428020485 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed9800e070 con 0x7feda8066220 2024-03-24T10:44:35.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.106+0000 7fedadb36700 1 -- 172.21.15.94:0/3428020485 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feda80fc7b0 con 0x7feda8066220 2024-03-24T10:44:35.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.107+0000 7fedadb36700 1 -- 172.21.15.94:0/3428020485 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feda805f5f0 con 0x7feda8066220 2024-03-24T10:44:35.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.108+0000 7fed8ffff700 1 -- 172.21.15.94:0/3428020485 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fed9801c9e0 con 0x7feda8066220 2024-03-24T10:44:35.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.108+0000 7fed8ffff700 1 --2- 172.21.15.94:0/3428020485 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fed900692b0 0x7fed9006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:35.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.108+0000 7fed8ffff700 1 -- 172.21.15.94:0/3428020485 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fed980884e0 con 0x7feda8066220 2024-03-24T10:44:35.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.108+0000 7feda6ffd700 1 --2- 172.21.15.94:0/3428020485 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fed900692b0 0x7fed9006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:35.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.109+0000 7feda6ffd700 1 --2- 172.21.15.94:0/3428020485 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fed900692b0 0x7fed9006b760 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7feda80fda40 tx=0x7fed9c00b040 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:44:35.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.114+0000 7fed8ffff700 1 -- 172.21.15.94:0/3428020485 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fed98054760 con 0x7feda8066220 2024-03-24T10:44:35.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.359+0000 7fedadb36700 1 -- 172.21.15.94:0/3428020485 --> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7feda80f8f70 con 0x7fed900692b0 2024-03-24T10:44:35.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.360+0000 7fed8ffff700 1 -- 172.21.15.94:0/3428020485 <== mgr.14232 v2:172.21.15.94:6800/2184357820 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7feda80f8f70 con 0x7fed900692b0 2024-03-24T10:44:35.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.362+0000 7fedadb36700 1 -- 172.21.15.94:0/3428020485 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fed900692b0 msgr2=0x7fed9006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:35.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.362+0000 7fedadb36700 1 --2- 172.21.15.94:0/3428020485 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fed900692b0 0x7fed9006b760 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7feda80fda40 tx=0x7fed9c00b040 comp rx=0 tx=0).stop 2024-03-24T10:44:35.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.362+0000 7fedadb36700 1 -- 172.21.15.94:0/3428020485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feda8066220 msgr2=0x7feda80fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:35.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.362+0000 7fedadb36700 1 --2- 172.21.15.94:0/3428020485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feda8066220 0x7feda80fd460 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fed98004090 tx=0x7fed98009e30 comp rx=0 tx=0).stop 2024-03-24T10:44:35.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.362+0000 7fedadb36700 1 -- 172.21.15.94:0/3428020485 shutdown_connections 2024-03-24T10:44:35.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.362+0000 7fedadb36700 1 --2- 172.21.15.94:0/3428020485 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fed900692b0 0x7fed9006b760 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:35.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.362+0000 7fedadb36700 1 --2- 172.21.15.94:0/3428020485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feda8066220 0x7feda80fd460 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:35.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.362+0000 7fedadb36700 1 --2- 172.21.15.94:0/3428020485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feda80fffc0 0x7feda80fb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:35.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.362+0000 7fedadb36700 1 -- 172.21.15.94:0/3428020485 >> 172.21.15.94:0/3428020485 conn(0x7feda80f7430 msgr2=0x7feda80f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:44:35.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.363+0000 7fedadb36700 1 -- 172.21.15.94:0/3428020485 shutdown_connections 2024-03-24T10:44:35.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.363+0000 7fedadb36700 1 -- 172.21.15.94:0/3428020485 wait complete. 2024-03-24T10:44:35.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.480+0000 7f8b4f84d700 1 -- 172.21.15.94:0/1104055250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b480fbcd0 msgr2=0x7f8b480fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:35.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.480+0000 7f8b4f84d700 1 --2- 172.21.15.94:0/1104055250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b480fbcd0 0x7f8b480fc0e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8b440049c0 tx=0x7f8b44009e30 comp rx=0 tx=0).stop 2024-03-24T10:44:35.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.480+0000 7f8b4f84d700 1 -- 172.21.15.94:0/1104055250 shutdown_connections 2024-03-24T10:44:35.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.480+0000 7f8b4f84d700 1 --2- 172.21.15.94:0/1104055250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b480fcf80 0x7f8b480fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:35.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.480+0000 7f8b4f84d700 1 --2- 172.21.15.94:0/1104055250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b480fbcd0 0x7f8b480fc0e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:35.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.480+0000 7f8b4f84d700 1 -- 172.21.15.94:0/1104055250 >> 172.21.15.94:0/1104055250 conn(0x7f8b480f7430 msgr2=0x7f8b480f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:44:35.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.480+0000 7f8b4f84d700 1 -- 172.21.15.94:0/1104055250 shutdown_connections 2024-03-24T10:44:35.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.480+0000 7f8b4f84d700 1 -- 172.21.15.94:0/1104055250 wait complete. 2024-03-24T10:44:35.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.481+0000 7f8b4f84d700 1 Processor -- start 2024-03-24T10:44:35.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.481+0000 7f8b4f84d700 1 -- start start 2024-03-24T10:44:35.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.481+0000 7f8b4f84d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b480fbcd0 0x7f8b4810b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:35.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.481+0000 7f8b4f84d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b480fcf80 0x7f8b4810bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:35.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.481+0000 7f8b4f84d700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4810c350 con 0x7f8b480fcf80 2024-03-24T10:44:35.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.481+0000 7f8b4f84d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4810cba0 con 0x7f8b480fbcd0 2024-03-24T10:44:35.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.481+0000 7f8b4d5e9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b480fbcd0 0x7f8b4810b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:35.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.481+0000 7f8b4cde8700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b480fcf80 0x7f8b4810bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:35.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.481+0000 7f8b4d5e9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b480fbcd0 0x7f8b4810b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.94:41388/0 (socket says 172.21.15.94:41388) 2024-03-24T10:44:35.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.481+0000 7f8b4d5e9700 1 -- 172.21.15.94:0/732967817 learned_addr learned my addr 172.21.15.94:0/732967817 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:44:35.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.482+0000 7f8b4d5e9700 1 -- 172.21.15.94:0/732967817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b480fcf80 msgr2=0x7f8b4810bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:35.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.482+0000 7f8b4d5e9700 1 --2- 172.21.15.94:0/732967817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b480fcf80 0x7f8b4810bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:35.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.482+0000 7f8b4d5e9700 1 -- 172.21.15.94:0/732967817 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b44005040 con 0x7f8b480fbcd0 2024-03-24T10:44:35.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.482+0000 7f8b4d5e9700 1 --2- 172.21.15.94:0/732967817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b480fbcd0 0x7f8b4810b810 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8b44009c90 tx=0x7f8b44009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:44:35.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.482+0000 7f8b3dffb700 1 -- 172.21.15.94:0/732967817 <== mon.1 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b44004290 con 0x7f8b480fbcd0 2024-03-24T10:44:35.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.482+0000 7f8b4f84d700 1 -- 172.21.15.94:0/732967817 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b4810ce20 con 0x7f8b480fbcd0 2024-03-24T10:44:35.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.482+0000 7f8b3dffb700 1 -- 172.21.15.94:0/732967817 <== mon.1 v2:172.21.15.96:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8b44009620 con 0x7f8b480fbcd0 2024-03-24T10:44:35.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.482+0000 7f8b3dffb700 1 -- 172.21.15.94:0/732967817 <== mon.1 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b4401b430 con 0x7f8b480fbcd0 2024-03-24T10:44:35.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.482+0000 7f8b4f84d700 1 -- 172.21.15.94:0/732967817 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b4810d360 con 0x7f8b480fbcd0 2024-03-24T10:44:35.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.483+0000 7f8b4f84d700 1 -- 172.21.15.94:0/732967817 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b4805f5f0 con 0x7f8b480fbcd0 2024-03-24T10:44:35.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.484+0000 7f8b3dffb700 1 -- 172.21.15.94:0/732967817 <== mon.1 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f8b44019040 con 0x7f8b480fbcd0 2024-03-24T10:44:35.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.484+0000 7f8b3dffb700 1 --2- 172.21.15.94:0/732967817 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f8b340692a0 0x7f8b3406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:35.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.484+0000 7f8b3dffb700 1 -- 172.21.15.94:0/732967817 <== mon.1 v2:172.21.15.96:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f8b440885b0 con 0x7f8b480fbcd0 2024-03-24T10:44:35.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.484+0000 7f8b4cde8700 1 --2- 172.21.15.94:0/732967817 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f8b340692a0 0x7f8b3406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:35.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.485+0000 7f8b4cde8700 1 --2- 172.21.15.94:0/732967817 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f8b340692a0 0x7f8b3406b750 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8b38009b80 tx=0x7f8b38011040 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:44:35.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.489+0000 7f8b3dffb700 1 -- 172.21.15.94:0/732967817 <== mon.1 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f8b440582a0 con 0x7f8b480fbcd0 2024-03-24T10:44:35.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.811+0000 7f8b4f84d700 1 -- 172.21.15.94:0/732967817 --> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8b4810d640 con 0x7f8b340692a0 2024-03-24T10:44:35.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.822+0000 7f8b3dffb700 1 -- 172.21.15.94:0/732967817 <== mgr.14232 v2:172.21.15.94:6800/2184357820 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3366 (secure 0 0 0) 0x7f8b4810d640 con 0x7f8b340692a0 2024-03-24T10:44:35.823 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T10:44:35.823 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.smithi094 smithi094 *:9093,9094 running (4m) 28s ago 6m 25.2M - 0.25.0 c8568f914cd2 8072d4b65986 2024-03-24T10:44:35.823 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi094 smithi094 running (7m) 28s ago 7m 6660k - 17.2.7-726-g9c4e0c34 837124da804f fb5772af46c5 2024-03-24T10:44:35.823 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi096 smithi096 running (5m) 34s ago 5m 6669k - 17.2.7-726-g9c4e0c34 837124da804f 6087ad4e0734 2024-03-24T10:44:35.823 INFO:teuthology.orchestra.run.smithi094.stdout:grafana.smithi094 smithi094 *:3000 running (4m) 28s ago 5m 83.6M - 9.4.7 954c08fa6188 7b534968fb36 2024-03-24T10:44:35.823 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.efiqte smithi094 running (52s) 28s ago 51s 10.4M - 17.2.7-726-g9c4e0c34 837124da804f f0df0a056c21 2024-03-24T10:44:35.823 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.llefgu smithi094 running (45s) 28s ago 45s 11.9M - 17.2.7-726-g9c4e0c34 837124da804f 52a9771dc8e0 2024-03-24T10:44:35.823 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi096.egbmoq smithi096 running (49s) 34s ago 49s 14.5M - 17.2.7-726-g9c4e0c34 837124da804f 1cfc930dbab0 2024-03-24T10:44:35.823 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi096.jtebhh smithi096 running (43s) 34s ago 42s 13.5M - 17.2.7-726-g9c4e0c34 837124da804f 4ff3451a6f6e 2024-03-24T10:44:35.823 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi094.yaethu smithi094 *:9283 running (8m) 28s ago 8m 481M - 17.2.7-726-g9c4e0c34 837124da804f 327d527697e0 2024-03-24T10:44:35.823 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi096.gfonnu smithi096 *:8443,9283 running (5m) 34s ago 5m 433M - 17.2.7-726-g9c4e0c34 837124da804f efd3f3b01cb6 2024-03-24T10:44:35.823 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi094 smithi094 running (8m) 28s ago 8m 43.3M 2048M 17.2.7-726-g9c4e0c34 837124da804f 76584fceb32b 2024-03-24T10:44:35.823 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi096 smithi096 running (5m) 34s ago 5m 36.3M 2048M 17.2.7-726-g9c4e0c34 837124da804f 710030b222cb 2024-03-24T10:44:35.823 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi094 smithi094 *:9100 running (6m) 28s ago 6m 15.4M - 1.5.0 0da6a335fe13 59895363f7e6 2024-03-24T10:44:35.824 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi096 smithi096 *:9100 running (5m) 34s ago 5m 15.3M - 1.5.0 0da6a335fe13 fc629282835f 2024-03-24T10:44:35.824 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (4m) 28s ago 4m 47.2M 4288M 17.2.7-726-g9c4e0c34 837124da804f b664fd3557bd 2024-03-24T10:44:35.824 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi094 running (4m) 28s ago 4m 46.3M 4288M 17.2.7-726-g9c4e0c34 837124da804f 4e737a80d733 2024-03-24T10:44:35.824 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi094 running (3m) 28s ago 3m 47.2M 4288M 17.2.7-726-g9c4e0c34 837124da804f cf0aefd20c76 2024-03-24T10:44:35.824 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi096 running (3m) 34s ago 3m 44.7M 5312M 17.2.7-726-g9c4e0c34 837124da804f f4650d0f6241 2024-03-24T10:44:35.824 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi096 running (2m) 34s ago 2m 44.6M 5312M 17.2.7-726-g9c4e0c34 837124da804f a63d7c9e705f 2024-03-24T10:44:35.824 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi096 running (2m) 34s ago 2m 41.7M 5312M 17.2.7-726-g9c4e0c34 837124da804f f56fad2959c3 2024-03-24T10:44:35.824 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.smithi094 smithi094 *:9095 running (4m) 28s ago 5m 57.0M - 2.43.0 a07b618ecd1d 023eb0a49108 2024-03-24T10:44:35.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.824+0000 7f8b4f84d700 1 -- 172.21.15.94:0/732967817 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f8b340692a0 msgr2=0x7f8b3406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:35.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.824+0000 7f8b4f84d700 1 --2- 172.21.15.94:0/732967817 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f8b340692a0 0x7f8b3406b750 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8b38009b80 tx=0x7f8b38011040 comp rx=0 tx=0).stop 2024-03-24T10:44:35.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.824+0000 7f8b4f84d700 1 -- 172.21.15.94:0/732967817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b480fbcd0 msgr2=0x7f8b4810b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:35.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.824+0000 7f8b4f84d700 1 --2- 172.21.15.94:0/732967817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b480fbcd0 0x7f8b4810b810 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8b44009c90 tx=0x7f8b44009e30 comp rx=0 tx=0).stop 2024-03-24T10:44:35.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.824+0000 7f8b4f84d700 1 -- 172.21.15.94:0/732967817 shutdown_connections 2024-03-24T10:44:35.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.824+0000 7f8b4f84d700 1 --2- 172.21.15.94:0/732967817 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f8b340692a0 0x7f8b3406b750 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:35.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.824+0000 7f8b4f84d700 1 --2- 172.21.15.94:0/732967817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b480fbcd0 0x7f8b4810b810 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:35.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.824+0000 7f8b4f84d700 1 --2- 172.21.15.94:0/732967817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b480fcf80 0x7f8b4810bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:35.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.824+0000 7f8b4f84d700 1 -- 172.21.15.94:0/732967817 >> 172.21.15.94:0/732967817 conn(0x7f8b480f7430 msgr2=0x7f8b480f8be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:44:35.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.824+0000 7f8b4f84d700 1 -- 172.21.15.94:0/732967817 shutdown_connections 2024-03-24T10:44:35.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.825+0000 7f8b4f84d700 1 -- 172.21.15.94:0/732967817 wait complete. 2024-03-24T10:44:35.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.933+0000 7fcb38f3a700 1 -- 172.21.15.94:0/1084816372 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb340fee10 msgr2=0x7fcb340faf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:35.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.933+0000 7fcb38f3a700 1 --2- 172.21.15.94:0/1084816372 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb340fee10 0x7fcb340faf50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fcb28004940 tx=0x7fcb28009e30 comp rx=0 tx=0).stop 2024-03-24T10:44:35.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.933+0000 7fcb38f3a700 1 -- 172.21.15.94:0/1084816372 shutdown_connections 2024-03-24T10:44:35.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.933+0000 7fcb38f3a700 1 --2- 172.21.15.94:0/1084816372 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb340fee10 0x7fcb340faf50 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:35.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.933+0000 7fcb38f3a700 1 --2- 172.21.15.94:0/1084816372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb340fe4c0 0x7fcb340fe8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:35.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.933+0000 7fcb38f3a700 1 -- 172.21.15.94:0/1084816372 >> 172.21.15.94:0/1084816372 conn(0x7fcb340f6ba0 msgr2=0x7fcb340f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:44:35.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.933+0000 7fcb38f3a700 1 -- 172.21.15.94:0/1084816372 shutdown_connections 2024-03-24T10:44:35.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.933+0000 7fcb38f3a700 1 -- 172.21.15.94:0/1084816372 wait complete. 2024-03-24T10:44:35.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.934+0000 7fcb38f3a700 1 Processor -- start 2024-03-24T10:44:35.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.934+0000 7fcb38f3a700 1 -- start start 2024-03-24T10:44:35.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.934+0000 7fcb38f3a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb340fe4c0 0x7fcb3410d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:35.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.934+0000 7fcb38f3a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb340fee10 0x7fcb3410df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:35.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.934+0000 7fcb38f3a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3410e530 con 0x7fcb340fee10 2024-03-24T10:44:35.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.934+0000 7fcb38f3a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3410ecc0 con 0x7fcb340fe4c0 2024-03-24T10:44:35.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.934+0000 7fcb31d9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb340fee10 0x7fcb3410df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:35.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.934+0000 7fcb31d9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb340fee10 0x7fcb3410df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38730/0 (socket says 172.21.15.94:38730) 2024-03-24T10:44:35.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.934+0000 7fcb31d9b700 1 -- 172.21.15.94:0/4243102307 learned_addr learned my addr 172.21.15.94:0/4243102307 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:44:35.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.934+0000 7fcb3259c700 1 --2- 172.21.15.94:0/4243102307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb340fe4c0 0x7fcb3410d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:35.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.934+0000 7fcb3259c700 1 -- 172.21.15.94:0/4243102307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb340fee10 msgr2=0x7fcb3410df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:35.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.934+0000 7fcb3259c700 1 --2- 172.21.15.94:0/4243102307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb340fee10 0x7fcb3410df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:35.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.934+0000 7fcb3259c700 1 -- 172.21.15.94:0/4243102307 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb28005040 con 0x7fcb340fe4c0 2024-03-24T10:44:35.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.934+0000 7fcb3259c700 1 --2- 172.21.15.94:0/4243102307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb340fe4c0 0x7fcb3410d9f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcb24009fd0 tx=0x7fcb2400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:44:35.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.935+0000 7fcb22ffd700 1 -- 172.21.15.94:0/4243102307 <== mon.1 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb24004e90 con 0x7fcb340fe4c0 2024-03-24T10:44:35.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.935+0000 7fcb38f3a700 1 -- 172.21.15.94:0/4243102307 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb3410ef40 con 0x7fcb340fe4c0 2024-03-24T10:44:35.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.935+0000 7fcb22ffd700 1 -- 172.21.15.94:0/4243102307 <== mon.1 v2:172.21.15.96:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcb2400f040 con 0x7fcb340fe4c0 2024-03-24T10:44:35.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.935+0000 7fcb22ffd700 1 -- 172.21.15.94:0/4243102307 <== mon.1 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb2400ca50 con 0x7fcb340fe4c0 2024-03-24T10:44:35.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.935+0000 7fcb38f3a700 1 -- 172.21.15.94:0/4243102307 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb3410f480 con 0x7fcb340fe4c0 2024-03-24T10:44:35.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.936+0000 7fcb38f3a700 1 -- 172.21.15.94:0/4243102307 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb3405f5f0 con 0x7fcb340fe4c0 2024-03-24T10:44:35.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.937+0000 7fcb22ffd700 1 -- 172.21.15.94:0/4243102307 <== mon.1 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fcb24007cc0 con 0x7fcb340fe4c0 2024-03-24T10:44:35.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.937+0000 7fcb22ffd700 1 --2- 172.21.15.94:0/4243102307 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fcb1c0692a0 0x7fcb1c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:35.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.937+0000 7fcb22ffd700 1 -- 172.21.15.94:0/4243102307 <== mon.1 v2:172.21.15.96:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fcb24086ca0 con 0x7fcb340fe4c0 2024-03-24T10:44:35.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.937+0000 7fcb31d9b700 1 --2- 172.21.15.94:0/4243102307 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fcb1c0692a0 0x7fcb1c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:35.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.937+0000 7fcb31d9b700 1 --2- 172.21.15.94:0/4243102307 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fcb1c0692a0 0x7fcb1c06b750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcb280043d0 tx=0x7fcb28009e30 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:44:35.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:35.941+0000 7fcb22ffd700 1 -- 172.21.15.94:0/4243102307 <== mon.1 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fcb2404e190 con 0x7fcb340fe4c0 2024-03-24T10:44:36.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:35 smithi094 ceph-mon[29446]: pgmap v201: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-03-24T10:44:36.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:35 smithi094 ceph-mon[29446]: from='client.14580 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:44:36.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:35 smithi094 ceph-mon[29446]: from='client.14584 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:44:36.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.278+0000 7fcb38f3a700 1 -- 172.21.15.94:0/4243102307 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fcb3410f7b0 con 0x7fcb340fe4c0 2024-03-24T10:44:36.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.279+0000 7fcb22ffd700 1 -- 172.21.15.94:0/4243102307 <== mon.1 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7fcb240569a0 con 0x7fcb340fe4c0 2024-03-24T10:44:36.279 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-03-24T10:44:36.280 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": { 2024-03-24T10:44:36.280 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-24T10:44:36.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:44:36.280 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": { 2024-03-24T10:44:36.280 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-24T10:44:36.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:44:36.280 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": { 2024-03-24T10:44:36.280 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-24T10:44:36.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:44:36.280 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-03-24T10:44:36.280 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-24T10:44:36.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:44:36.280 INFO:teuthology.orchestra.run.smithi094.stdout: "overall": { 2024-03-24T10:44:36.280 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 14 2024-03-24T10:44:36.281 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-03-24T10:44:36.281 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-03-24T10:44:36.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.281+0000 7fcb38f3a700 1 -- 172.21.15.94:0/4243102307 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fcb1c0692a0 msgr2=0x7fcb1c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:36.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.281+0000 7fcb38f3a700 1 --2- 172.21.15.94:0/4243102307 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fcb1c0692a0 0x7fcb1c06b750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcb280043d0 tx=0x7fcb28009e30 comp rx=0 tx=0).stop 2024-03-24T10:44:36.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.281+0000 7fcb38f3a700 1 -- 172.21.15.94:0/4243102307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb340fe4c0 msgr2=0x7fcb3410d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:36.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.281+0000 7fcb38f3a700 1 --2- 172.21.15.94:0/4243102307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb340fe4c0 0x7fcb3410d9f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcb24009fd0 tx=0x7fcb2400b040 comp rx=0 tx=0).stop 2024-03-24T10:44:36.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.281+0000 7fcb38f3a700 1 -- 172.21.15.94:0/4243102307 shutdown_connections 2024-03-24T10:44:36.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.281+0000 7fcb38f3a700 1 --2- 172.21.15.94:0/4243102307 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fcb1c0692a0 0x7fcb1c06b750 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:36.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.281+0000 7fcb38f3a700 1 --2- 172.21.15.94:0/4243102307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb340fe4c0 0x7fcb3410d9f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:36.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.281+0000 7fcb38f3a700 1 --2- 172.21.15.94:0/4243102307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb340fee10 0x7fcb3410df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:36.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.281+0000 7fcb38f3a700 1 -- 172.21.15.94:0/4243102307 >> 172.21.15.94:0/4243102307 conn(0x7fcb340f6ba0 msgr2=0x7fcb340f8200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:44:36.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.281+0000 7fcb38f3a700 1 -- 172.21.15.94:0/4243102307 shutdown_connections 2024-03-24T10:44:36.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.281+0000 7fcb38f3a700 1 -- 172.21.15.94:0/4243102307 wait complete. 2024-03-24T10:44:36.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:35 smithi096 ceph-mon[38097]: pgmap v201: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-03-24T10:44:36.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:35 smithi096 ceph-mon[38097]: from='client.14580 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:44:36.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:35 smithi096 ceph-mon[38097]: from='client.14584 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:44:36.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.402+0000 7f0d391c4700 1 -- 172.21.15.94:0/2596926778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d340fbcd0 msgr2=0x7f0d340fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:36.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.402+0000 7f0d391c4700 1 --2- 172.21.15.94:0/2596926778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d340fbcd0 0x7f0d340fc0e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0d24004940 tx=0x7f0d24009e30 comp rx=0 tx=0).stop 2024-03-24T10:44:36.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.402+0000 7f0d391c4700 1 -- 172.21.15.94:0/2596926778 shutdown_connections 2024-03-24T10:44:36.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.402+0000 7f0d391c4700 1 --2- 172.21.15.94:0/2596926778 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d340fcf80 0x7f0d340fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:36.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.402+0000 7f0d391c4700 1 --2- 172.21.15.94:0/2596926778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d340fbcd0 0x7f0d340fc0e0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:36.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.402+0000 7f0d391c4700 1 -- 172.21.15.94:0/2596926778 >> 172.21.15.94:0/2596926778 conn(0x7f0d340f7430 msgr2=0x7f0d340f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:44:36.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.402+0000 7f0d391c4700 1 -- 172.21.15.94:0/2596926778 shutdown_connections 2024-03-24T10:44:36.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.402+0000 7f0d391c4700 1 -- 172.21.15.94:0/2596926778 wait complete. 2024-03-24T10:44:36.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.403+0000 7f0d391c4700 1 Processor -- start 2024-03-24T10:44:36.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.403+0000 7f0d391c4700 1 -- start start 2024-03-24T10:44:36.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.403+0000 7f0d391c4700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d340fbcd0 0x7f0d3410d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:36.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.403+0000 7f0d391c4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d340fcf80 0x7f0d3410deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:36.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.403+0000 7f0d391c4700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3410e4b0 con 0x7f0d340fcf80 2024-03-24T10:44:36.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.403+0000 7f0d391c4700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3410ed00 con 0x7f0d340fbcd0 2024-03-24T10:44:36.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.403+0000 7f0d3259c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d340fcf80 0x7f0d3410deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:36.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.403+0000 7f0d32d9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d340fbcd0 0x7f0d3410d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:36.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.403+0000 7f0d3259c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d340fcf80 0x7f0d3410deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38756/0 (socket says 172.21.15.94:38756) 2024-03-24T10:44:36.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.403+0000 7f0d3259c700 1 -- 172.21.15.94:0/1065025155 learned_addr learned my addr 172.21.15.94:0/1065025155 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:44:36.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.403+0000 7f0d3259c700 1 -- 172.21.15.94:0/1065025155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d340fbcd0 msgr2=0x7f0d3410d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:36.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.403+0000 7f0d3259c700 1 --2- 172.21.15.94:0/1065025155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d340fbcd0 0x7f0d3410d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:36.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.403+0000 7f0d3259c700 1 -- 172.21.15.94:0/1065025155 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d24005040 con 0x7f0d340fcf80 2024-03-24T10:44:36.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.403+0000 7f0d3259c700 1 --2- 172.21.15.94:0/1065025155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d340fcf80 0x7f0d3410deb0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0d2800eb40 tx=0x7f0d2800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:44:36.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.404+0000 7f0d1b7fe700 1 -- 172.21.15.94:0/1065025155 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d28007cc0 con 0x7f0d340fcf80 2024-03-24T10:44:36.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.404+0000 7f0d1b7fe700 1 -- 172.21.15.94:0/1065025155 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d2800f040 con 0x7f0d340fcf80 2024-03-24T10:44:36.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.404+0000 7f0d391c4700 1 -- 172.21.15.94:0/1065025155 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d3410ef80 con 0x7f0d340fcf80 2024-03-24T10:44:36.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.404+0000 7f0d1b7fe700 1 -- 172.21.15.94:0/1065025155 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d2800ca10 con 0x7f0d340fcf80 2024-03-24T10:44:36.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.404+0000 7f0d391c4700 1 -- 172.21.15.94:0/1065025155 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d3410f4c0 con 0x7f0d340fcf80 2024-03-24T10:44:36.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.405+0000 7f0d391c4700 1 -- 172.21.15.94:0/1065025155 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d3405f5f0 con 0x7f0d340fcf80 2024-03-24T10:44:36.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.405+0000 7f0d1b7fe700 1 -- 172.21.15.94:0/1065025155 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f0d2800cb70 con 0x7f0d340fcf80 2024-03-24T10:44:36.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.406+0000 7f0d1b7fe700 1 --2- 172.21.15.94:0/1065025155 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f0d1c0692a0 0x7f0d1c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:36.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.406+0000 7f0d1b7fe700 1 -- 172.21.15.94:0/1065025155 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f0d28086cc0 con 0x7f0d340fcf80 2024-03-24T10:44:36.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.406+0000 7f0d32d9d700 1 --2- 172.21.15.94:0/1065025155 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f0d1c0692a0 0x7f0d1c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:36.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.406+0000 7f0d32d9d700 1 --2- 172.21.15.94:0/1065025155 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f0d1c0692a0 0x7f0d1c06b750 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0d24009b30 tx=0x7f0d24009e30 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:44:36.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.411+0000 7f0d1b7fe700 1 -- 172.21.15.94:0/1065025155 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f0d2804e190 con 0x7f0d340fcf80 2024-03-24T10:44:36.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.693+0000 7f0d391c4700 1 -- 172.21.15.94:0/1065025155 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0d341047c0 con 0x7f0d340fcf80 2024-03-24T10:44:36.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.694+0000 7f0d1b7fe700 1 -- 172.21.15.94:0/1065025155 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1833 (secure 0 0 0) 0x7f0d28056850 con 0x7f0d340fcf80 2024-03-24T10:44:36.694 INFO:teuthology.orchestra.run.smithi094.stdout:e18 2024-03-24T10:44:36.694 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T10:44:36.694 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:44:36.694 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-03-24T10:44:36.694 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:44:36.694 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-03-24T10:44:36.694 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-03-24T10:44:36.694 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 18 2024-03-24T10:44:36.695 INFO:teuthology.orchestra.run.smithi094.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-24T10:44:36.695 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-03-24T10:43:39.435738+0000 2024-03-24T10:44:36.695 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-03-24T10:43:58.133901+0000 2024-03-24T10:44:36.695 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-03-24T10:44:36.695 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-03-24T10:44:36.695 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 60 2024-03-24T10:44:36.695 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-03-24T10:44:36.695 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-03-24T10:44:36.695 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-03-24T10:44:36.695 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-03-24T10:44:36.695 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 46 2024-03-24T10:44:36.695 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:44:36.695 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-03-24T10:44:36.695 INFO:teuthology.orchestra.run.smithi094.stdout:in 0 2024-03-24T10:44:36.695 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=24251} 2024-03-24T10:44:36.696 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-03-24T10:44:36.696 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-03-24T10:44:36.696 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-03-24T10:44:36.696 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-03-24T10:44:36.696 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-03-24T10:44:36.696 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-03-24T10:44:36.696 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-03-24T10:44:36.696 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 1 2024-03-24T10:44:36.696 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi096.egbmoq{0:24251} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.96:6824/1539752208,v1:172.21.15.96:6825/1539752208] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:44:36.696 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.llefgu{0:14500} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.94:6828/126762980,v1:172.21.15.94:6829/126762980] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:44:36.696 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:44:36.696 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:44:36.696 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-03-24T10:44:36.696 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:44:36.696 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.efiqte{-1:14512} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.94:6826/1636528860,v1:172.21.15.94:6827/1636528860] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:44:36.696 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi096.jtebhh{-1:24261} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.96:6826/1321087930,v1:172.21.15.96:6827/1321087930] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:44:36.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.697+0000 7f0d391c4700 1 -- 172.21.15.94:0/1065025155 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f0d1c0692a0 msgr2=0x7f0d1c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:36.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.697+0000 7f0d391c4700 1 --2- 172.21.15.94:0/1065025155 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f0d1c0692a0 0x7f0d1c06b750 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0d24009b30 tx=0x7f0d24009e30 comp rx=0 tx=0).stop 2024-03-24T10:44:36.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.697+0000 7f0d391c4700 1 -- 172.21.15.94:0/1065025155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d340fcf80 msgr2=0x7f0d3410deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:36.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.697+0000 7f0d391c4700 1 --2- 172.21.15.94:0/1065025155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d340fcf80 0x7f0d3410deb0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0d2800eb40 tx=0x7f0d2800b040 comp rx=0 tx=0).stop 2024-03-24T10:44:36.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.698+0000 7f0d391c4700 1 -- 172.21.15.94:0/1065025155 shutdown_connections 2024-03-24T10:44:36.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.698+0000 7f0d391c4700 1 --2- 172.21.15.94:0/1065025155 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f0d1c0692a0 0x7f0d1c06b750 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:36.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.698+0000 7f0d391c4700 1 --2- 172.21.15.94:0/1065025155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d340fbcd0 0x7f0d3410d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:36.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.698+0000 7f0d391c4700 1 --2- 172.21.15.94:0/1065025155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d340fcf80 0x7f0d3410deb0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:36.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.698+0000 7f0d391c4700 1 -- 172.21.15.94:0/1065025155 >> 172.21.15.94:0/1065025155 conn(0x7f0d340f7430 msgr2=0x7f0d340f8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:44:36.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.698+0000 7f0d391c4700 1 -- 172.21.15.94:0/1065025155 shutdown_connections 2024-03-24T10:44:36.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.698+0000 7f0d391c4700 1 -- 172.21.15.94:0/1065025155 wait complete. 2024-03-24T10:44:36.698 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 18 2024-03-24T10:44:36.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.827+0000 7fe31bdbf700 1 -- 172.21.15.94:0/907206736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3140feff0 msgr2=0x7fe3140ff400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:36.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.827+0000 7fe31bdbf700 1 --2- 172.21.15.94:0/907206736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3140feff0 0x7fe3140ff400 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe310004940 tx=0x7fe310009e30 comp rx=0 tx=0).stop 2024-03-24T10:44:36.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.828+0000 7fe31bdbf700 1 -- 172.21.15.94:0/907206736 shutdown_connections 2024-03-24T10:44:36.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.828+0000 7fe31bdbf700 1 --2- 172.21.15.94:0/907206736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe3141002a0 0x7fe314100710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:36.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.828+0000 7fe31bdbf700 1 --2- 172.21.15.94:0/907206736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3140feff0 0x7fe3140ff400 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:36.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.828+0000 7fe31bdbf700 1 -- 172.21.15.94:0/907206736 >> 172.21.15.94:0/907206736 conn(0x7fe3140fa950 msgr2=0x7fe3140fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:44:36.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.828+0000 7fe31bdbf700 1 -- 172.21.15.94:0/907206736 shutdown_connections 2024-03-24T10:44:36.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.828+0000 7fe31bdbf700 1 -- 172.21.15.94:0/907206736 wait complete. 2024-03-24T10:44:36.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.828+0000 7fe31bdbf700 1 Processor -- start 2024-03-24T10:44:36.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.828+0000 7fe31bdbf700 1 -- start start 2024-03-24T10:44:36.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.828+0000 7fe31bdbf700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3140feff0 0x7fe31410fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:36.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.828+0000 7fe31bdbf700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe3141002a0 0x7fe314110140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:36.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.828+0000 7fe31bdbf700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe314110740 con 0x7fe3140feff0 2024-03-24T10:44:36.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.828+0000 7fe31bdbf700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe314110f40 con 0x7fe3141002a0 2024-03-24T10:44:36.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.829+0000 7fe319b5b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3140feff0 0x7fe31410fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:36.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.829+0000 7fe31935a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe3141002a0 0x7fe314110140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:36.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.829+0000 7fe31935a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe3141002a0 0x7fe314110140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.94:41440/0 (socket says 172.21.15.94:41440) 2024-03-24T10:44:36.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.829+0000 7fe319b5b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3140feff0 0x7fe31410fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38762/0 (socket says 172.21.15.94:38762) 2024-03-24T10:44:36.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.829+0000 7fe31935a700 1 -- 172.21.15.94:0/35750014 learned_addr learned my addr 172.21.15.94:0/35750014 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:44:36.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.829+0000 7fe31935a700 1 -- 172.21.15.94:0/35750014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3140feff0 msgr2=0x7fe31410fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:36.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.829+0000 7fe31935a700 1 --2- 172.21.15.94:0/35750014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3140feff0 0x7fe31410fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:36.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.829+0000 7fe31935a700 1 -- 172.21.15.94:0/35750014 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe310005040 con 0x7fe3141002a0 2024-03-24T10:44:36.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.829+0000 7fe319b5b700 1 --2- 172.21.15.94:0/35750014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3140feff0 0x7fe31410fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T10:44:36.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.829+0000 7fe31935a700 1 --2- 172.21.15.94:0/35750014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe3141002a0 0x7fe314110140 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe30400eb40 tx=0x7fe30400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:44:36.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.829+0000 7fe30a7fc700 1 -- 172.21.15.94:0/35750014 <== mon.1 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe304018070 con 0x7fe3141002a0 2024-03-24T10:44:36.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.829+0000 7fe31bdbf700 1 -- 172.21.15.94:0/35750014 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3141111c0 con 0x7fe3141002a0 2024-03-24T10:44:36.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.829+0000 7fe30a7fc700 1 -- 172.21.15.94:0/35750014 <== mon.1 v2:172.21.15.96:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe30400f040 con 0x7fe3141002a0 2024-03-24T10:44:36.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.829+0000 7fe30a7fc700 1 -- 172.21.15.94:0/35750014 <== mon.1 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe30400c910 con 0x7fe3141002a0 2024-03-24T10:44:36.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.829+0000 7fe31bdbf700 1 -- 172.21.15.94:0/35750014 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe314111700 con 0x7fe3141002a0 2024-03-24T10:44:36.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.830+0000 7fe31bdbf700 1 -- 172.21.15.94:0/35750014 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe31405f5f0 con 0x7fe3141002a0 2024-03-24T10:44:36.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.831+0000 7fe30a7fc700 1 -- 172.21.15.94:0/35750014 <== mon.1 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fe304007cc0 con 0x7fe3141002a0 2024-03-24T10:44:36.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.831+0000 7fe30a7fc700 1 --2- 172.21.15.94:0/35750014 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fe3000692a0 0x7fe30006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:36.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.832+0000 7fe30a7fc700 1 -- 172.21.15.94:0/35750014 <== mon.1 v2:172.21.15.96:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fe3040872e0 con 0x7fe3141002a0 2024-03-24T10:44:36.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.832+0000 7fe319b5b700 1 --2- 172.21.15.94:0/35750014 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fe3000692a0 0x7fe30006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:36.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.832+0000 7fe319b5b700 1 --2- 172.21.15.94:0/35750014 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fe3000692a0 0x7fe30006b750 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe310009b30 tx=0x7fe310009e30 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:44:36.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:36.839+0000 7fe30a7fc700 1 -- 172.21.15.94:0/35750014 <== mon.1 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fe304056fd0 con 0x7fe3141002a0 2024-03-24T10:44:37.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.097+0000 7fe31bdbf700 1 -- 172.21.15.94:0/35750014 --> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe314111970 con 0x7fe3000692a0 2024-03-24T10:44:37.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.098+0000 7fe30a7fc700 1 -- 172.21.15.94:0/35750014 <== mgr.14232 v2:172.21.15.94:6800/2184357820 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7fe314111970 con 0x7fe3000692a0 2024-03-24T10:44:37.098 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-03-24T10:44:37.098 INFO:teuthology.orchestra.run.smithi094.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de", 2024-03-24T10:44:37.099 INFO:teuthology.orchestra.run.smithi094.stdout: "in_progress": true, 2024-03-24T10:44:37.099 INFO:teuthology.orchestra.run.smithi094.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T10:44:37.099 INFO:teuthology.orchestra.run.smithi094.stdout: "services_complete": [], 2024-03-24T10:44:37.099 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": "", 2024-03-24T10:44:37.099 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "Doing first pull of quay.ceph.io/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de image", 2024-03-24T10:44:37.099 INFO:teuthology.orchestra.run.smithi094.stdout: "is_paused": false 2024-03-24T10:44:37.099 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-03-24T10:44:37.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.100+0000 7fe31bdbf700 1 -- 172.21.15.94:0/35750014 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fe3000692a0 msgr2=0x7fe30006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:37.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.100+0000 7fe31bdbf700 1 --2- 172.21.15.94:0/35750014 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fe3000692a0 0x7fe30006b750 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe310009b30 tx=0x7fe310009e30 comp rx=0 tx=0).stop 2024-03-24T10:44:37.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.100+0000 7fe31bdbf700 1 -- 172.21.15.94:0/35750014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe3141002a0 msgr2=0x7fe314110140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:37.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.100+0000 7fe31bdbf700 1 --2- 172.21.15.94:0/35750014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe3141002a0 0x7fe314110140 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe30400eb40 tx=0x7fe30400b040 comp rx=0 tx=0).stop 2024-03-24T10:44:37.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.101+0000 7fe31bdbf700 1 -- 172.21.15.94:0/35750014 shutdown_connections 2024-03-24T10:44:37.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.101+0000 7fe31bdbf700 1 --2- 172.21.15.94:0/35750014 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fe3000692a0 0x7fe30006b750 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:37.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.101+0000 7fe31bdbf700 1 --2- 172.21.15.94:0/35750014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3140feff0 0x7fe31410fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:37.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.101+0000 7fe31bdbf700 1 --2- 172.21.15.94:0/35750014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe3141002a0 0x7fe314110140 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:37.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.101+0000 7fe31bdbf700 1 -- 172.21.15.94:0/35750014 >> 172.21.15.94:0/35750014 conn(0x7fe3140fa950 msgr2=0x7fe3140fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:44:37.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.101+0000 7fe31bdbf700 1 -- 172.21.15.94:0/35750014 shutdown_connections 2024-03-24T10:44:37.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.101+0000 7fe31bdbf700 1 -- 172.21.15.94:0/35750014 wait complete. 2024-03-24T10:44:37.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.227+0000 7f9d23d1d700 1 -- 172.21.15.94:0/3117116544 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d1c0662e0 msgr2=0x7f9d1c0ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:37.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.227+0000 7f9d23d1d700 1 --2- 172.21.15.94:0/3117116544 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d1c0662e0 0x7f9d1c0ff9c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9d18004940 tx=0x7f9d18009e30 comp rx=0 tx=0).stop 2024-03-24T10:44:37.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.227+0000 7f9d23d1d700 1 -- 172.21.15.94:0/3117116544 shutdown_connections 2024-03-24T10:44:37.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.227+0000 7f9d23d1d700 1 --2- 172.21.15.94:0/3117116544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d1c0fff00 0x7f9d1c102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:37.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.227+0000 7f9d23d1d700 1 --2- 172.21.15.94:0/3117116544 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d1c0662e0 0x7f9d1c0ff9c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:37.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.227+0000 7f9d23d1d700 1 -- 172.21.15.94:0/3117116544 >> 172.21.15.94:0/3117116544 conn(0x7f9d1c0f7430 msgr2=0x7f9d1c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:44:37.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.227+0000 7f9d23d1d700 1 -- 172.21.15.94:0/3117116544 shutdown_connections 2024-03-24T10:44:37.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.228+0000 7f9d23d1d700 1 -- 172.21.15.94:0/3117116544 wait complete. 2024-03-24T10:44:37.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.228+0000 7f9d23d1d700 1 Processor -- start 2024-03-24T10:44:37.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.228+0000 7f9d23d1d700 1 -- start start 2024-03-24T10:44:37.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.228+0000 7f9d23d1d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d1c0662e0 0x7f9d1c10da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:37.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.228+0000 7f9d23d1d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d1c0fff00 0x7f9d1c10df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:37.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.228+0000 7f9d23d1d700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c10e540 con 0x7f9d1c0662e0 2024-03-24T10:44:37.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.228+0000 7f9d23d1d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c10ed40 con 0x7f9d1c0fff00 2024-03-24T10:44:37.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.228+0000 7f9d21ab9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d1c0662e0 0x7f9d1c10da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:37.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.228+0000 7f9d21ab9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d1c0662e0 0x7f9d1c10da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38774/0 (socket says 172.21.15.94:38774) 2024-03-24T10:44:37.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.228+0000 7f9d21ab9700 1 -- 172.21.15.94:0/516140649 learned_addr learned my addr 172.21.15.94:0/516140649 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:44:37.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.228+0000 7f9d212b8700 1 --2- 172.21.15.94:0/516140649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d1c0fff00 0x7f9d1c10df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:37.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.229+0000 7f9d21ab9700 1 -- 172.21.15.94:0/516140649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d1c0fff00 msgr2=0x7f9d1c10df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:37.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.229+0000 7f9d21ab9700 1 --2- 172.21.15.94:0/516140649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d1c0fff00 0x7f9d1c10df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:37.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.229+0000 7f9d21ab9700 1 -- 172.21.15.94:0/516140649 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d18005040 con 0x7f9d1c0662e0 2024-03-24T10:44:37.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.229+0000 7f9d21ab9700 1 --2- 172.21.15.94:0/516140649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d1c0662e0 0x7f9d1c10da00 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9d18000c00 tx=0x7f9d18009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:44:37.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.229+0000 7f9d127fc700 1 -- 172.21.15.94:0/516140649 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d1800e030 con 0x7f9d1c0662e0 2024-03-24T10:44:37.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.229+0000 7f9d127fc700 1 -- 172.21.15.94:0/516140649 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9d18013de0 con 0x7f9d1c0662e0 2024-03-24T10:44:37.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.229+0000 7f9d127fc700 1 -- 172.21.15.94:0/516140649 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d1801ce50 con 0x7f9d1c0662e0 2024-03-24T10:44:37.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.229+0000 7f9d23d1d700 1 -- 172.21.15.94:0/516140649 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d1c10efc0 con 0x7f9d1c0662e0 2024-03-24T10:44:37.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.229+0000 7f9d23d1d700 1 -- 172.21.15.94:0/516140649 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d1c10f4b0 con 0x7f9d1c0662e0 2024-03-24T10:44:37.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.231+0000 7f9d23d1d700 1 -- 172.21.15.94:0/516140649 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d1c05f5f0 con 0x7f9d1c0662e0 2024-03-24T10:44:37.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.231+0000 7f9d127fc700 1 -- 172.21.15.94:0/516140649 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f9d1801c400 con 0x7f9d1c0662e0 2024-03-24T10:44:37.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.231+0000 7f9d127fc700 1 --2- 172.21.15.94:0/516140649 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f9d080692a0 0x7f9d0806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:44:37.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.231+0000 7f9d127fc700 1 -- 172.21.15.94:0/516140649 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f9d18088bd0 con 0x7f9d1c0662e0 2024-03-24T10:44:37.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.231+0000 7f9d212b8700 1 --2- 172.21.15.94:0/516140649 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f9d080692a0 0x7f9d0806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:44:37.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.232+0000 7f9d212b8700 1 --2- 172.21.15.94:0/516140649 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f9d080692a0 0x7f9d0806b750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c009b80 tx=0x7f9d0c00b040 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:44:37.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.236+0000 7f9d127fc700 1 -- 172.21.15.94:0/516140649 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f9d18058940 con 0x7f9d1c0662e0 2024-03-24T10:44:37.236 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:36 smithi094 ceph-mon[29446]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:44:37.236 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:36 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/4243102307' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:44:37.236 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:36 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/1065025155' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T10:44:37.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:36 smithi096 ceph-mon[38097]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:44:37.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:36 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/4243102307' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:44:37.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:36 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/1065025155' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T10:44:37.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.560+0000 7f9d23d1d700 1 -- 172.21.15.94:0/516140649 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f9d1c04bed0 con 0x7f9d1c0662e0 2024-03-24T10:44:37.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.560+0000 7f9d127fc700 1 -- 172.21.15.94:0/516140649 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f9d180584d0 con 0x7f9d1c0662e0 2024-03-24T10:44:37.560 INFO:teuthology.orchestra.run.smithi094.stdout:HEALTH_OK 2024-03-24T10:44:37.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.562+0000 7f9d23d1d700 1 -- 172.21.15.94:0/516140649 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f9d080692a0 msgr2=0x7f9d0806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:37.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.562+0000 7f9d23d1d700 1 --2- 172.21.15.94:0/516140649 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f9d080692a0 0x7f9d0806b750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c009b80 tx=0x7f9d0c00b040 comp rx=0 tx=0).stop 2024-03-24T10:44:37.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.562+0000 7f9d23d1d700 1 -- 172.21.15.94:0/516140649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d1c0662e0 msgr2=0x7f9d1c10da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:44:37.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.562+0000 7f9d23d1d700 1 --2- 172.21.15.94:0/516140649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d1c0662e0 0x7f9d1c10da00 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9d18000c00 tx=0x7f9d18009e30 comp rx=0 tx=0).stop 2024-03-24T10:44:37.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.562+0000 7f9d23d1d700 1 -- 172.21.15.94:0/516140649 shutdown_connections 2024-03-24T10:44:37.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.562+0000 7f9d23d1d700 1 --2- 172.21.15.94:0/516140649 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f9d080692a0 0x7f9d0806b750 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:37.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.562+0000 7f9d23d1d700 1 --2- 172.21.15.94:0/516140649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d1c0662e0 0x7f9d1c10da00 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:37.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.562+0000 7f9d23d1d700 1 --2- 172.21.15.94:0/516140649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d1c0fff00 0x7f9d1c10df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:44:37.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.562+0000 7f9d23d1d700 1 -- 172.21.15.94:0/516140649 >> 172.21.15.94:0/516140649 conn(0x7f9d1c0f7430 msgr2=0x7f9d1c0f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:44:37.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.562+0000 7f9d23d1d700 1 -- 172.21.15.94:0/516140649 shutdown_connections 2024-03-24T10:44:37.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:44:37.562+0000 7f9d23d1d700 1 -- 172.21.15.94:0/516140649 wait complete. 2024-03-24T10:44:38.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:37 smithi094 ceph-mon[29446]: pgmap v202: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-03-24T10:44:38.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:37 smithi094 ceph-mon[29446]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:44:38.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:37 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/516140649' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T10:44:38.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:37 smithi096 ceph-mon[38097]: pgmap v202: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-03-24T10:44:38.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:37 smithi096 ceph-mon[38097]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:44:38.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:37 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/516140649' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T10:44:40.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:39 smithi094 ceph-mon[29446]: pgmap v203: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:44:40.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:39 smithi096 ceph-mon[38097]: pgmap v203: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:44:42.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:41 smithi094 ceph-mon[29446]: pgmap v204: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:44:42.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:41 smithi096 ceph-mon[38097]: pgmap v204: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:44:44.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:43 smithi094 ceph-mon[29446]: pgmap v205: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:44:44.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:43 smithi096 ceph-mon[38097]: pgmap v205: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:44:46.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:45 smithi094 ceph-mon[29446]: pgmap v206: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:44:46.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:45 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:44:46.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:45 smithi096 ceph-mon[38097]: pgmap v206: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:44:46.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:45 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:44:48.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:47 smithi094 ceph-mon[29446]: pgmap v207: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:44:48.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:47 smithi096 ceph-mon[38097]: pgmap v207: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:44:50.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:49 smithi094 ceph-mon[29446]: pgmap v208: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:44:50.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:49 smithi096 ceph-mon[38097]: pgmap v208: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:44:52.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:51 smithi096 ceph-mon[38097]: pgmap v209: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:44:52.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:51 smithi094 ceph-mon[29446]: pgmap v209: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:44:54.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:54 smithi096 ceph-mon[38097]: pgmap v210: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 426 B/s wr, 2 op/s 2024-03-24T10:44:54.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:54 smithi094 ceph-mon[29446]: pgmap v210: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 426 B/s wr, 2 op/s 2024-03-24T10:44:56.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:56 smithi096 ceph-mon[38097]: pgmap v211: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 426 B/s wr, 1 op/s 2024-03-24T10:44:56.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:56 smithi094 ceph-mon[29446]: pgmap v211: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 426 B/s wr, 1 op/s 2024-03-24T10:44:58.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:44:58 smithi096 ceph-mon[38097]: pgmap v212: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-24T10:44:58.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:44:58 smithi094 ceph-mon[29446]: pgmap v212: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-24T10:45:00.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:00 smithi094 ceph-mon[29446]: pgmap v213: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-24T10:45:00.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:00 smithi096 ceph-mon[38097]: pgmap v213: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-24T10:45:01.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:01 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:45:01.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:01 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:45:02.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:02 smithi096 ceph-mon[38097]: pgmap v214: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-24T10:45:02.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:02 smithi094 ceph-mon[29446]: pgmap v214: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-24T10:45:04.267 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:04 smithi094 ceph-mon[29446]: pgmap v215: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-24T10:45:04.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:04 smithi096 ceph-mon[38097]: pgmap v215: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-24T10:45:06.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:06 smithi096 ceph-mon[38097]: pgmap v216: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-03-24T10:45:06.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:06 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:45:06.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:06 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:45:06.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:06 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:45:06.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:06 smithi094 ceph-mon[29446]: pgmap v216: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-03-24T10:45:06.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:06 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:45:06.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:06 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:45:06.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:06 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:45:07.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:07 smithi096 ceph-mon[38097]: Upgrade: Target is version 19.0.0-1855-g21b3bd67 (unknown) 2024-03-24T10:45:07.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:07 smithi096 ceph-mon[38097]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, digests ['quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919'] 2024-03-24T10:45:07.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:07 smithi096 ceph-mon[38097]: Upgrade: Need to upgrade myself (mgr.smithi094.yaethu) 2024-03-24T10:45:07.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:07 smithi096 ceph-mon[38097]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 on smithi096 2024-03-24T10:45:07.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:07 smithi094 ceph-mon[29446]: Upgrade: Target is version 19.0.0-1855-g21b3bd67 (unknown) 2024-03-24T10:45:07.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:07 smithi094 ceph-mon[29446]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, digests ['quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919'] 2024-03-24T10:45:07.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:07 smithi094 ceph-mon[29446]: Upgrade: Need to upgrade myself (mgr.smithi094.yaethu) 2024-03-24T10:45:07.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:07 smithi094 ceph-mon[29446]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 on smithi096 2024-03-24T10:45:07.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:07.684+0000 7f41c6475700 1 -- 172.21.15.94:0/1734296792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41c00fcf80 msgr2=0x7f41c00fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:07.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:07.684+0000 7f41c6475700 1 --2- 172.21.15.94:0/1734296792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41c00fcf80 0x7f41c00fd3f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f41b4004940 tx=0x7f41b4009e30 comp rx=0 tx=0).stop 2024-03-24T10:45:07.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:07.684+0000 7f41c6475700 1 -- 172.21.15.94:0/1734296792 shutdown_connections 2024-03-24T10:45:07.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:07.684+0000 7f41c6475700 1 --2- 172.21.15.94:0/1734296792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41c00fcf80 0x7f41c00fd3f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:07.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:07.684+0000 7f41c6475700 1 --2- 172.21.15.94:0/1734296792 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41c00fbcd0 0x7f41c00fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:07.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:07.684+0000 7f41c6475700 1 -- 172.21.15.94:0/1734296792 >> 172.21.15.94:0/1734296792 conn(0x7f41c00f7430 msgr2=0x7f41c00f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:45:07.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:07.684+0000 7f41c6475700 1 -- 172.21.15.94:0/1734296792 shutdown_connections 2024-03-24T10:45:07.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:07.684+0000 7f41c6475700 1 -- 172.21.15.94:0/1734296792 wait complete. 2024-03-24T10:45:07.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:07.685+0000 7f41c6475700 1 Processor -- start 2024-03-24T10:45:07.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:07.685+0000 7f41c6475700 1 -- start start 2024-03-24T10:45:07.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:07.685+0000 7f41c6475700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41c00fbcd0 0x7f41c0110700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:45:07.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:07.685+0000 7f41c6475700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41c00fcf80 0x7f41c0112c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:45:07.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:07.686+0000 7f41c6475700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c0066350 con 0x7f41c00fbcd0 2024-03-24T10:45:07.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:07.686+0000 7f41c6475700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c0066490 con 0x7f41c00fcf80 2024-03-24T10:45:07.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:07.686+0000 7f41bffff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41c00fbcd0 0x7f41c0110700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:45:07.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:07.686+0000 7f41bffff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41c00fbcd0 0x7f41c0110700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46792/0 (socket says 172.21.15.94:46792) 2024-03-24T10:45:07.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:07.686+0000 7f41bffff700 1 -- 172.21.15.94:0/73350654 learned_addr learned my addr 172.21.15.94:0/73350654 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:45:07.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:07.686+0000 7f41bf7fe700 1 --2- 172.21.15.94:0/73350654 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41c00fcf80 0x7f41c0112c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:45:07.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:07.686+0000 7f41bffff700 1 -- 172.21.15.94:0/73350654 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41c00fcf80 msgr2=0x7f41c0112c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:07.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:07.686+0000 7f41bffff700 1 --2- 172.21.15.94:0/73350654 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41c00fcf80 0x7f41c0112c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:07.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:07.686+0000 7f41bffff700 1 -- 172.21.15.94:0/73350654 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b4005040 con 0x7f41c00fbcd0 2024-03-24T10:45:07.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:07.686+0000 7f41bffff700 1 --2- 172.21.15.94:0/73350654 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41c00fbcd0 0x7f41c0110700 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f41b000eb70 tx=0x7f41b000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:45:07.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:07.686+0000 7f41bcff9700 1 -- 172.21.15.94:0/73350654 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41b0007cc0 con 0x7f41c00fbcd0 2024-03-24T10:45:07.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:07.686+0000 7f41bcff9700 1 -- 172.21.15.94:0/73350654 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f41b000f040 con 0x7f41c00fbcd0 2024-03-24T10:45:07.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:07.686+0000 7f41c6475700 1 -- 172.21.15.94:0/73350654 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41c0113190 con 0x7f41c00fbcd0 2024-03-24T10:45:07.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:07.686+0000 7f41bcff9700 1 -- 172.21.15.94:0/73350654 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41b000ca90 con 0x7f41c00fbcd0 2024-03-24T10:45:07.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:07.686+0000 7f41c6475700 1 -- 172.21.15.94:0/73350654 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41c01136d0 con 0x7f41c00fbcd0 2024-03-24T10:45:07.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:07.688+0000 7f41c6475700 1 -- 172.21.15.94:0/73350654 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41c005f5f0 con 0x7f41c00fbcd0 2024-03-24T10:45:07.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:07.688+0000 7f41bcff9700 1 -- 172.21.15.94:0/73350654 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f41b000cbf0 con 0x7f41c00fbcd0 2024-03-24T10:45:07.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:07.689+0000 7f41bcff9700 1 --2- 172.21.15.94:0/73350654 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f41a80692f0 0x7f41a806b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:45:07.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:07.689+0000 7f41bcff9700 1 -- 172.21.15.94:0/73350654 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f41b0086e10 con 0x7f41c00fbcd0 2024-03-24T10:45:07.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:07.689+0000 7f41bf7fe700 1 --2- 172.21.15.94:0/73350654 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f41a80692f0 0x7f41a806b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:45:07.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:07.690+0000 7f41bf7fe700 1 --2- 172.21.15.94:0/73350654 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f41a80692f0 0x7f41a806b7a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f41b4009b30 tx=0x7f41b4009e30 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:45:07.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:07.693+0000 7f41bcff9700 1 -- 172.21.15.94:0/73350654 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f41b0052fd0 con 0x7f41c00fbcd0 2024-03-24T10:45:07.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:07.907+0000 7f41c6475700 1 -- 172.21.15.94:0/73350654 --> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f41c0101250 con 0x7f41a80692f0 2024-03-24T10:45:07.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:07.908+0000 7f41bcff9700 1 -- 172.21.15.94:0/73350654 <== mgr.14232 v2:172.21.15.94:6800/2184357820 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7f41c0101250 con 0x7f41a80692f0 2024-03-24T10:45:07.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:07.910+0000 7f41c6475700 1 -- 172.21.15.94:0/73350654 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f41a80692f0 msgr2=0x7f41a806b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:07.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:07.910+0000 7f41c6475700 1 --2- 172.21.15.94:0/73350654 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f41a80692f0 0x7f41a806b7a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f41b4009b30 tx=0x7f41b4009e30 comp rx=0 tx=0).stop 2024-03-24T10:45:07.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:07.910+0000 7f41c6475700 1 -- 172.21.15.94:0/73350654 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41c00fbcd0 msgr2=0x7f41c0110700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:07.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:07.910+0000 7f41c6475700 1 --2- 172.21.15.94:0/73350654 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41c00fbcd0 0x7f41c0110700 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f41b000eb70 tx=0x7f41b000b040 comp rx=0 tx=0).stop 2024-03-24T10:45:07.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:07.910+0000 7f41c6475700 1 -- 172.21.15.94:0/73350654 shutdown_connections 2024-03-24T10:45:07.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:07.910+0000 7f41c6475700 1 --2- 172.21.15.94:0/73350654 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f41a80692f0 0x7f41a806b7a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:07.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:07.910+0000 7f41c6475700 1 --2- 172.21.15.94:0/73350654 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41c00fbcd0 0x7f41c0110700 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:07.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:07.910+0000 7f41c6475700 1 --2- 172.21.15.94:0/73350654 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41c00fcf80 0x7f41c0112c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:07.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:07.910+0000 7f41c6475700 1 -- 172.21.15.94:0/73350654 >> 172.21.15.94:0/73350654 conn(0x7f41c00f7430 msgr2=0x7f41c00f8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:45:07.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:07.911+0000 7f41c6475700 1 -- 172.21.15.94:0/73350654 shutdown_connections 2024-03-24T10:45:07.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:07.911+0000 7f41c6475700 1 -- 172.21.15.94:0/73350654 wait complete. 2024-03-24T10:45:07.925 INFO:teuthology.orchestra.run.smithi094.stdout:true 2024-03-24T10:45:08.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.062+0000 7f7d5dcf5700 1 -- 172.21.15.94:0/3400305274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d580662e0 msgr2=0x7f7d580ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:08.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.062+0000 7f7d5dcf5700 1 --2- 172.21.15.94:0/3400305274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d580662e0 0x7f7d580ff9c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7d48004940 tx=0x7f7d48009e30 comp rx=0 tx=0).stop 2024-03-24T10:45:08.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.062+0000 7f7d5dcf5700 1 -- 172.21.15.94:0/3400305274 shutdown_connections 2024-03-24T10:45:08.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.062+0000 7f7d5dcf5700 1 --2- 172.21.15.94:0/3400305274 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d580fff00 0x7f7d58102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:08.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.062+0000 7f7d5dcf5700 1 --2- 172.21.15.94:0/3400305274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d580662e0 0x7f7d580ff9c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:08.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.062+0000 7f7d5dcf5700 1 -- 172.21.15.94:0/3400305274 >> 172.21.15.94:0/3400305274 conn(0x7f7d580f7430 msgr2=0x7f7d580f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:45:08.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.062+0000 7f7d5dcf5700 1 -- 172.21.15.94:0/3400305274 shutdown_connections 2024-03-24T10:45:08.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.062+0000 7f7d5dcf5700 1 -- 172.21.15.94:0/3400305274 wait complete. 2024-03-24T10:45:08.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.063+0000 7f7d5dcf5700 1 Processor -- start 2024-03-24T10:45:08.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.063+0000 7f7d5dcf5700 1 -- start start 2024-03-24T10:45:08.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.063+0000 7f7d5dcf5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d580662e0 0x7f7d5810da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:45:08.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.063+0000 7f7d5dcf5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d580fff00 0x7f7d5810df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:45:08.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.063+0000 7f7d5dcf5700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5810e560 con 0x7f7d580662e0 2024-03-24T10:45:08.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.063+0000 7f7d5dcf5700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5810ed60 con 0x7f7d580fff00 2024-03-24T10:45:08.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.064+0000 7f7d577fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d580662e0 0x7f7d5810da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:45:08.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.064+0000 7f7d577fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d580662e0 0x7f7d5810da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46806/0 (socket says 172.21.15.94:46806) 2024-03-24T10:45:08.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.064+0000 7f7d577fe700 1 -- 172.21.15.94:0/1704677387 learned_addr learned my addr 172.21.15.94:0/1704677387 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:45:08.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.064+0000 7f7d56ffd700 1 --2- 172.21.15.94:0/1704677387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d580fff00 0x7f7d5810df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:45:08.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.064+0000 7f7d577fe700 1 -- 172.21.15.94:0/1704677387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d580fff00 msgr2=0x7f7d5810df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:08.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.064+0000 7f7d577fe700 1 --2- 172.21.15.94:0/1704677387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d580fff00 0x7f7d5810df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:08.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.064+0000 7f7d577fe700 1 -- 172.21.15.94:0/1704677387 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d48005040 con 0x7f7d580662e0 2024-03-24T10:45:08.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.064+0000 7f7d577fe700 1 --2- 172.21.15.94:0/1704677387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d580662e0 0x7f7d5810da20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7d48000c00 tx=0x7f7d48009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:45:08.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.064+0000 7f7d3ffff700 1 -- 172.21.15.94:0/1704677387 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d4800e030 con 0x7f7d580662e0 2024-03-24T10:45:08.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.064+0000 7f7d3ffff700 1 -- 172.21.15.94:0/1704677387 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7d48013de0 con 0x7f7d580662e0 2024-03-24T10:45:08.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.064+0000 7f7d5dcf5700 1 -- 172.21.15.94:0/1704677387 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d5810efe0 con 0x7f7d580662e0 2024-03-24T10:45:08.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.065+0000 7f7d3ffff700 1 -- 172.21.15.94:0/1704677387 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d4801ce50 con 0x7f7d580662e0 2024-03-24T10:45:08.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.065+0000 7f7d5dcf5700 1 -- 172.21.15.94:0/1704677387 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d5810f460 con 0x7f7d580662e0 2024-03-24T10:45:08.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.066+0000 7f7d5dcf5700 1 -- 172.21.15.94:0/1704677387 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d5805f5f0 con 0x7f7d580662e0 2024-03-24T10:45:08.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.066+0000 7f7d3ffff700 1 -- 172.21.15.94:0/1704677387 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f7d4801c400 con 0x7f7d580662e0 2024-03-24T10:45:08.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.067+0000 7f7d3ffff700 1 --2- 172.21.15.94:0/1704677387 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f7d400692a0 0x7f7d4006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:45:08.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.067+0000 7f7d56ffd700 1 --2- 172.21.15.94:0/1704677387 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f7d400692a0 0x7f7d4006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:45:08.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.067+0000 7f7d3ffff700 1 -- 172.21.15.94:0/1704677387 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f7d48088ba0 con 0x7f7d580662e0 2024-03-24T10:45:08.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.067+0000 7f7d56ffd700 1 --2- 172.21.15.94:0/1704677387 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f7d400692a0 0x7f7d4006b750 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c009b80 tx=0x7f7d4c00b040 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:45:08.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.071+0000 7f7d3ffff700 1 -- 172.21.15.94:0/1704677387 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f7d48058910 con 0x7f7d580662e0 2024-03-24T10:45:08.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:08 smithi096 ceph-mon[38097]: pgmap v217: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 170 B/s wr, 3 op/s 2024-03-24T10:45:08.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.410+0000 7f7d5dcf5700 1 -- 172.21.15.94:0/1704677387 --> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d5805f5f0 con 0x7f7d400692a0 2024-03-24T10:45:08.411 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:08 smithi094 ceph-mon[29446]: pgmap v217: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 170 B/s wr, 3 op/s 2024-03-24T10:45:08.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.411+0000 7f7d3ffff700 1 -- 172.21.15.94:0/1704677387 <== mgr.14232 v2:172.21.15.94:6800/2184357820 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7f7d5805f5f0 con 0x7f7d400692a0 2024-03-24T10:45:08.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.414+0000 7f7d5dcf5700 1 -- 172.21.15.94:0/1704677387 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f7d400692a0 msgr2=0x7f7d4006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:08.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.414+0000 7f7d5dcf5700 1 --2- 172.21.15.94:0/1704677387 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f7d400692a0 0x7f7d4006b750 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c009b80 tx=0x7f7d4c00b040 comp rx=0 tx=0).stop 2024-03-24T10:45:08.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.414+0000 7f7d5dcf5700 1 -- 172.21.15.94:0/1704677387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d580662e0 msgr2=0x7f7d5810da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:08.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.414+0000 7f7d5dcf5700 1 --2- 172.21.15.94:0/1704677387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d580662e0 0x7f7d5810da20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7d48000c00 tx=0x7f7d48009e30 comp rx=0 tx=0).stop 2024-03-24T10:45:08.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.414+0000 7f7d5dcf5700 1 -- 172.21.15.94:0/1704677387 shutdown_connections 2024-03-24T10:45:08.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.414+0000 7f7d5dcf5700 1 --2- 172.21.15.94:0/1704677387 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f7d400692a0 0x7f7d4006b750 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:08.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.414+0000 7f7d5dcf5700 1 --2- 172.21.15.94:0/1704677387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d580662e0 0x7f7d5810da20 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:08.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.414+0000 7f7d5dcf5700 1 --2- 172.21.15.94:0/1704677387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d580fff00 0x7f7d5810df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:08.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.414+0000 7f7d5dcf5700 1 -- 172.21.15.94:0/1704677387 >> 172.21.15.94:0/1704677387 conn(0x7f7d580f7430 msgr2=0x7f7d580f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:45:08.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.414+0000 7f7d5dcf5700 1 -- 172.21.15.94:0/1704677387 shutdown_connections 2024-03-24T10:45:08.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.415+0000 7f7d5dcf5700 1 -- 172.21.15.94:0/1704677387 wait complete. 2024-03-24T10:45:08.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.554+0000 7f9c09ffc700 1 -- 172.21.15.94:0/258167184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c040662e0 msgr2=0x7f9c040ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:08.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.554+0000 7f9c09ffc700 1 --2- 172.21.15.94:0/258167184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c040662e0 0x7f9c040ff9c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4004940 tx=0x7f9bf4009e30 comp rx=0 tx=0).stop 2024-03-24T10:45:08.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.555+0000 7f9c09ffc700 1 -- 172.21.15.94:0/258167184 shutdown_connections 2024-03-24T10:45:08.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.555+0000 7f9c09ffc700 1 --2- 172.21.15.94:0/258167184 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c040fff00 0x7f9c04102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:08.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.555+0000 7f9c09ffc700 1 --2- 172.21.15.94:0/258167184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c040662e0 0x7f9c040ff9c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:08.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.555+0000 7f9c09ffc700 1 -- 172.21.15.94:0/258167184 >> 172.21.15.94:0/258167184 conn(0x7f9c040f7430 msgr2=0x7f9c040f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:45:08.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.555+0000 7f9c09ffc700 1 -- 172.21.15.94:0/258167184 shutdown_connections 2024-03-24T10:45:08.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.555+0000 7f9c09ffc700 1 -- 172.21.15.94:0/258167184 wait complete. 2024-03-24T10:45:08.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.555+0000 7f9c09ffc700 1 Processor -- start 2024-03-24T10:45:08.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.556+0000 7f9c09ffc700 1 -- start start 2024-03-24T10:45:08.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.556+0000 7f9c09ffc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c040662e0 0x7f9c040fd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:45:08.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.556+0000 7f9c09ffc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c040fff00 0x7f9c040fb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:45:08.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.556+0000 7f9c09ffc700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c040fbe50 con 0x7f9c040fff00 2024-03-24T10:45:08.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.556+0000 7f9c09ffc700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c040fbf90 con 0x7f9c040662e0 2024-03-24T10:45:08.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.556+0000 7f9c02ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c040fff00 0x7f9c040fb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:45:08.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.556+0000 7f9c02ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c040fff00 0x7f9c040fb8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46820/0 (socket says 172.21.15.94:46820) 2024-03-24T10:45:08.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.556+0000 7f9c02ffd700 1 -- 172.21.15.94:0/2606091137 learned_addr learned my addr 172.21.15.94:0/2606091137 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:45:08.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.556+0000 7f9c02ffd700 1 -- 172.21.15.94:0/2606091137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c040662e0 msgr2=0x7f9c040fd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:08.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.556+0000 7f9c037fe700 1 --2- 172.21.15.94:0/2606091137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c040662e0 0x7f9c040fd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:45:08.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.556+0000 7f9c02ffd700 1 --2- 172.21.15.94:0/2606091137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c040662e0 0x7f9c040fd3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:08.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.556+0000 7f9c02ffd700 1 -- 172.21.15.94:0/2606091137 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf4005040 con 0x7f9c040fff00 2024-03-24T10:45:08.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.557+0000 7f9c037fe700 1 --2- 172.21.15.94:0/2606091137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c040662e0 0x7f9c040fd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T10:45:08.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.557+0000 7f9c02ffd700 1 --2- 172.21.15.94:0/2606091137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c040fff00 0x7f9c040fb8c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9bf800eb40 tx=0x7f9bf800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:45:08.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.557+0000 7f9bebfff700 1 -- 172.21.15.94:0/2606091137 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bf8007cc0 con 0x7f9c040fff00 2024-03-24T10:45:08.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.557+0000 7f9bebfff700 1 -- 172.21.15.94:0/2606091137 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9bf800f040 con 0x7f9c040fff00 2024-03-24T10:45:08.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.557+0000 7f9bebfff700 1 -- 172.21.15.94:0/2606091137 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bf8009620 con 0x7f9c040fff00 2024-03-24T10:45:08.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.557+0000 7f9c09ffc700 1 -- 172.21.15.94:0/2606091137 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c040fc210 con 0x7f9c040fff00 2024-03-24T10:45:08.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.557+0000 7f9c09ffc700 1 -- 172.21.15.94:0/2606091137 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c04115a30 con 0x7f9c040fff00 2024-03-24T10:45:08.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.558+0000 7f9be9ffb700 1 -- 172.21.15.94:0/2606091137 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9be40052b0 con 0x7f9c040fff00 2024-03-24T10:45:08.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.559+0000 7f9bebfff700 1 -- 172.21.15.94:0/2606091137 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f9bf8009780 con 0x7f9c040fff00 2024-03-24T10:45:08.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.560+0000 7f9bebfff700 1 --2- 172.21.15.94:0/2606091137 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f9bec069370 0x7f9bec06b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:45:08.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.560+0000 7f9bebfff700 1 -- 172.21.15.94:0/2606091137 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f9bf8086c80 con 0x7f9c040fff00 2024-03-24T10:45:08.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.560+0000 7f9c037fe700 1 --2- 172.21.15.94:0/2606091137 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f9bec069370 0x7f9bec06b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:45:08.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.561+0000 7f9c037fe700 1 --2- 172.21.15.94:0/2606091137 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f9bec069370 0x7f9bec06b820 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4004890 tx=0x7f9bf4009e30 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:45:08.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.564+0000 7f9bebfff700 1 -- 172.21.15.94:0/2606091137 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f9bf804e190 con 0x7f9c040fff00 2024-03-24T10:45:08.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.799+0000 7f9be9ffb700 1 -- 172.21.15.94:0/2606091137 --> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9be4000bc0 con 0x7f9bec069370 2024-03-24T10:45:08.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.809+0000 7f9bebfff700 1 -- 172.21.15.94:0/2606091137 <== mgr.14232 v2:172.21.15.94:6800/2184357820 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3366 (secure 0 0 0) 0x7f9be4000bc0 con 0x7f9bec069370 2024-03-24T10:45:08.809 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T10:45:08.809 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.smithi094 smithi094 *:9093,9094 running (5m) 61s ago 7m 25.2M - 0.25.0 c8568f914cd2 8072d4b65986 2024-03-24T10:45:08.810 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi094 smithi094 running (7m) 61s ago 7m 6660k - 17.2.7-726-g9c4e0c34 837124da804f fb5772af46c5 2024-03-24T10:45:08.810 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi096 smithi096 running (5m) 67s ago 5m 6669k - 17.2.7-726-g9c4e0c34 837124da804f 6087ad4e0734 2024-03-24T10:45:08.810 INFO:teuthology.orchestra.run.smithi094.stdout:grafana.smithi094 smithi094 *:3000 running (5m) 61s ago 6m 83.6M - 9.4.7 954c08fa6188 7b534968fb36 2024-03-24T10:45:08.810 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.efiqte smithi094 running (85s) 61s ago 84s 10.4M - 17.2.7-726-g9c4e0c34 837124da804f f0df0a056c21 2024-03-24T10:45:08.810 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.llefgu smithi094 running (78s) 61s ago 78s 11.9M - 17.2.7-726-g9c4e0c34 837124da804f 52a9771dc8e0 2024-03-24T10:45:08.810 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi096.egbmoq smithi096 running (82s) 67s ago 82s 14.5M - 17.2.7-726-g9c4e0c34 837124da804f 1cfc930dbab0 2024-03-24T10:45:08.810 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi096.jtebhh smithi096 running (76s) 67s ago 75s 13.5M - 17.2.7-726-g9c4e0c34 837124da804f 4ff3451a6f6e 2024-03-24T10:45:08.810 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi094.yaethu smithi094 *:9283 running (8m) 61s ago 8m 481M - 17.2.7-726-g9c4e0c34 837124da804f 327d527697e0 2024-03-24T10:45:08.810 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi096.gfonnu smithi096 *:8443,9283 running (5m) 67s ago 5m 433M - 17.2.7-726-g9c4e0c34 837124da804f efd3f3b01cb6 2024-03-24T10:45:08.810 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi094 smithi094 running (8m) 61s ago 8m 43.3M 2048M 17.2.7-726-g9c4e0c34 837124da804f 76584fceb32b 2024-03-24T10:45:08.810 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi096 smithi096 running (5m) 67s ago 5m 36.3M 2048M 17.2.7-726-g9c4e0c34 837124da804f 710030b222cb 2024-03-24T10:45:08.810 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi094 smithi094 *:9100 running (7m) 61s ago 7m 15.4M - 1.5.0 0da6a335fe13 59895363f7e6 2024-03-24T10:45:08.810 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi096 smithi096 *:9100 running (5m) 67s ago 5m 15.3M - 1.5.0 0da6a335fe13 fc629282835f 2024-03-24T10:45:08.810 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (5m) 61s ago 5m 47.2M 4288M 17.2.7-726-g9c4e0c34 837124da804f b664fd3557bd 2024-03-24T10:45:08.810 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi094 running (4m) 61s ago 4m 46.3M 4288M 17.2.7-726-g9c4e0c34 837124da804f 4e737a80d733 2024-03-24T10:45:08.810 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi094 running (4m) 61s ago 4m 47.2M 4288M 17.2.7-726-g9c4e0c34 837124da804f cf0aefd20c76 2024-03-24T10:45:08.811 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi096 running (3m) 67s ago 3m 44.7M 5312M 17.2.7-726-g9c4e0c34 837124da804f f4650d0f6241 2024-03-24T10:45:08.811 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi096 running (3m) 67s ago 3m 44.6M 5312M 17.2.7-726-g9c4e0c34 837124da804f a63d7c9e705f 2024-03-24T10:45:08.811 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi096 running (2m) 67s ago 2m 41.7M 5312M 17.2.7-726-g9c4e0c34 837124da804f f56fad2959c3 2024-03-24T10:45:08.811 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.smithi094 smithi094 *:9095 running (5m) 61s ago 6m 57.0M - 2.43.0 a07b618ecd1d 023eb0a49108 2024-03-24T10:45:08.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.811+0000 7f9be9ffb700 1 -- 172.21.15.94:0/2606091137 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f9bec069370 msgr2=0x7f9bec06b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:08.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.811+0000 7f9be9ffb700 1 --2- 172.21.15.94:0/2606091137 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f9bec069370 0x7f9bec06b820 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4004890 tx=0x7f9bf4009e30 comp rx=0 tx=0).stop 2024-03-24T10:45:08.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.811+0000 7f9be9ffb700 1 -- 172.21.15.94:0/2606091137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c040fff00 msgr2=0x7f9c040fb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:08.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.811+0000 7f9be9ffb700 1 --2- 172.21.15.94:0/2606091137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c040fff00 0x7f9c040fb8c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9bf800eb40 tx=0x7f9bf800b040 comp rx=0 tx=0).stop 2024-03-24T10:45:08.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.811+0000 7f9be9ffb700 1 -- 172.21.15.94:0/2606091137 shutdown_connections 2024-03-24T10:45:08.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.811+0000 7f9be9ffb700 1 --2- 172.21.15.94:0/2606091137 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f9bec069370 0x7f9bec06b820 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:08.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.811+0000 7f9be9ffb700 1 --2- 172.21.15.94:0/2606091137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c040662e0 0x7f9c040fd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:08.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.811+0000 7f9be9ffb700 1 --2- 172.21.15.94:0/2606091137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c040fff00 0x7f9c040fb8c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:08.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.811+0000 7f9be9ffb700 1 -- 172.21.15.94:0/2606091137 >> 172.21.15.94:0/2606091137 conn(0x7f9c040f7430 msgr2=0x7f9c040f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:45:08.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.811+0000 7f9be9ffb700 1 -- 172.21.15.94:0/2606091137 shutdown_connections 2024-03-24T10:45:08.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.811+0000 7f9be9ffb700 1 -- 172.21.15.94:0/2606091137 wait complete. 2024-03-24T10:45:08.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.913+0000 7f456674c700 1 -- 172.21.15.94:0/3161240282 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4560102360 msgr2=0x7f4560104740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:08.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.913+0000 7f456674c700 1 --2- 172.21.15.94:0/3161240282 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4560102360 0x7f4560104740 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4554004940 tx=0x7f4554009e30 comp rx=0 tx=0).stop 2024-03-24T10:45:08.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.913+0000 7f456674c700 1 -- 172.21.15.94:0/3161240282 shutdown_connections 2024-03-24T10:45:08.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.913+0000 7f456674c700 1 --2- 172.21.15.94:0/3161240282 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4560102360 0x7f4560104740 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:08.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.913+0000 7f456674c700 1 --2- 172.21.15.94:0/3161240282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4560066280 0x7f4560101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:08.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.913+0000 7f456674c700 1 -- 172.21.15.94:0/3161240282 >> 172.21.15.94:0/3161240282 conn(0x7f45600f7430 msgr2=0x7f45600f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:45:08.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.913+0000 7f456674c700 1 -- 172.21.15.94:0/3161240282 shutdown_connections 2024-03-24T10:45:08.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.913+0000 7f456674c700 1 -- 172.21.15.94:0/3161240282 wait complete. 2024-03-24T10:45:08.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.913+0000 7f456674c700 1 Processor -- start 2024-03-24T10:45:08.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.914+0000 7f456674c700 1 -- start start 2024-03-24T10:45:08.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.914+0000 7f456674c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4560066280 0x7f456010fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:45:08.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.914+0000 7f456674c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4560102360 0x7f45601100f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:45:08.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.914+0000 7f456674c700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45601106f0 con 0x7f4560066280 2024-03-24T10:45:08.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.914+0000 7f456674c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4560110f40 con 0x7f4560102360 2024-03-24T10:45:08.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.914+0000 7f455f7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4560102360 0x7f45601100f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:45:08.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.914+0000 7f455f7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4560102360 0x7f45601100f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.94:47340/0 (socket says 172.21.15.94:47340) 2024-03-24T10:45:08.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.914+0000 7f455f7fe700 1 -- 172.21.15.94:0/3231681911 learned_addr learned my addr 172.21.15.94:0/3231681911 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:45:08.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.914+0000 7f455ffff700 1 --2- 172.21.15.94:0/3231681911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4560066280 0x7f456010fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:45:08.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.914+0000 7f455f7fe700 1 -- 172.21.15.94:0/3231681911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4560066280 msgr2=0x7f456010fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:08.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.914+0000 7f455f7fe700 1 --2- 172.21.15.94:0/3231681911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4560066280 0x7f456010fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:08.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.914+0000 7f455f7fe700 1 -- 172.21.15.94:0/3231681911 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4554005040 con 0x7f4560102360 2024-03-24T10:45:08.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.915+0000 7f455f7fe700 1 --2- 172.21.15.94:0/3231681911 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4560102360 0x7f45601100f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4554013870 tx=0x7f4554009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:45:08.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.915+0000 7f455cff9700 1 -- 172.21.15.94:0/3231681911 <== mon.1 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4554021070 con 0x7f4560102360 2024-03-24T10:45:08.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.915+0000 7f456674c700 1 -- 172.21.15.94:0/3231681911 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45601111c0 con 0x7f4560102360 2024-03-24T10:45:08.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.915+0000 7f456674c700 1 -- 172.21.15.94:0/3231681911 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4560111700 con 0x7f4560102360 2024-03-24T10:45:08.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.915+0000 7f455cff9700 1 -- 172.21.15.94:0/3231681911 <== mon.1 v2:172.21.15.96:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4554009430 con 0x7f4560102360 2024-03-24T10:45:08.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.915+0000 7f455cff9700 1 -- 172.21.15.94:0/3231681911 <== mon.1 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f455400e070 con 0x7f4560102360 2024-03-24T10:45:08.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.916+0000 7f456674c700 1 -- 172.21.15.94:0/3231681911 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f454c0052e0 con 0x7f4560102360 2024-03-24T10:45:08.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.917+0000 7f455cff9700 1 -- 172.21.15.94:0/3231681911 <== mon.1 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f4554013af0 con 0x7f4560102360 2024-03-24T10:45:08.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.917+0000 7f455cff9700 1 --2- 172.21.15.94:0/3231681911 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f45480692a0 0x7f454806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:45:08.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.917+0000 7f455cff9700 1 -- 172.21.15.94:0/3231681911 <== mon.1 v2:172.21.15.96:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f45540883b0 con 0x7f4560102360 2024-03-24T10:45:08.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.917+0000 7f455ffff700 1 --2- 172.21.15.94:0/3231681911 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f45480692a0 0x7f454806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:45:08.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.918+0000 7f455ffff700 1 --2- 172.21.15.94:0/3231681911 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f45480692a0 0x7f454806b750 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4550009b80 tx=0x7f455000b040 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:45:08.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:08.921+0000 7f455cff9700 1 -- 172.21.15.94:0/3231681911 <== mon.1 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f45540581c0 con 0x7f4560102360 2024-03-24T10:45:09.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.199+0000 7f456674c700 1 -- 172.21.15.94:0/3231681911 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f454c005860 con 0x7f4560102360 2024-03-24T10:45:09.200 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:09 smithi094 ceph-mon[29446]: from='client.14608 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:45:09.200 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:09 smithi094 ceph-mon[29446]: from='client.14612 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:45:09.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.199+0000 7f455cff9700 1 -- 172.21.15.94:0/3231681911 <== mon.1 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7f4554004490 con 0x7f4560102360 2024-03-24T10:45:09.201 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-03-24T10:45:09.201 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": { 2024-03-24T10:45:09.201 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-24T10:45:09.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:45:09.201 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": { 2024-03-24T10:45:09.201 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-24T10:45:09.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:45:09.201 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": { 2024-03-24T10:45:09.201 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-24T10:45:09.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:45:09.201 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-03-24T10:45:09.201 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-24T10:45:09.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:45:09.201 INFO:teuthology.orchestra.run.smithi094.stdout: "overall": { 2024-03-24T10:45:09.202 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 14 2024-03-24T10:45:09.202 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-03-24T10:45:09.202 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-03-24T10:45:09.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.202+0000 7f456674c700 1 -- 172.21.15.94:0/3231681911 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f45480692a0 msgr2=0x7f454806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:09.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.202+0000 7f456674c700 1 --2- 172.21.15.94:0/3231681911 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f45480692a0 0x7f454806b750 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4550009b80 tx=0x7f455000b040 comp rx=0 tx=0).stop 2024-03-24T10:45:09.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.202+0000 7f456674c700 1 -- 172.21.15.94:0/3231681911 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4560102360 msgr2=0x7f45601100f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:09.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.202+0000 7f456674c700 1 --2- 172.21.15.94:0/3231681911 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4560102360 0x7f45601100f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4554013870 tx=0x7f4554009e30 comp rx=0 tx=0).stop 2024-03-24T10:45:09.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.202+0000 7f456674c700 1 -- 172.21.15.94:0/3231681911 shutdown_connections 2024-03-24T10:45:09.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.202+0000 7f456674c700 1 --2- 172.21.15.94:0/3231681911 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f45480692a0 0x7f454806b750 secure :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4550009b80 tx=0x7f455000b040 comp rx=0 tx=0).stop 2024-03-24T10:45:09.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.202+0000 7f456674c700 1 --2- 172.21.15.94:0/3231681911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4560066280 0x7f456010fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:09.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.202+0000 7f456674c700 1 --2- 172.21.15.94:0/3231681911 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4560102360 0x7f45601100f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:09.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.202+0000 7f456674c700 1 -- 172.21.15.94:0/3231681911 >> 172.21.15.94:0/3231681911 conn(0x7f45600f7430 msgr2=0x7f45600f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:45:09.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.203+0000 7f456674c700 1 -- 172.21.15.94:0/3231681911 shutdown_connections 2024-03-24T10:45:09.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.203+0000 7f456674c700 1 -- 172.21.15.94:0/3231681911 wait complete. 2024-03-24T10:45:09.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.315+0000 7ff7e501d700 1 -- 172.21.15.94:0/372649833 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7e00662e0 msgr2=0x7ff7e00ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:09.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.315+0000 7ff7e501d700 1 --2- 172.21.15.94:0/372649833 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7e00662e0 0x7ff7e00ff9c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0004940 tx=0x7ff7d0009e30 comp rx=0 tx=0).stop 2024-03-24T10:45:09.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.315+0000 7ff7e501d700 1 -- 172.21.15.94:0/372649833 shutdown_connections 2024-03-24T10:45:09.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.315+0000 7ff7e501d700 1 --2- 172.21.15.94:0/372649833 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7e00fff00 0x7ff7e0102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:09.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.315+0000 7ff7e501d700 1 --2- 172.21.15.94:0/372649833 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7e00662e0 0x7ff7e00ff9c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:09.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.315+0000 7ff7e501d700 1 -- 172.21.15.94:0/372649833 >> 172.21.15.94:0/372649833 conn(0x7ff7e00f7430 msgr2=0x7ff7e00f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:45:09.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.315+0000 7ff7e501d700 1 -- 172.21.15.94:0/372649833 shutdown_connections 2024-03-24T10:45:09.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.316+0000 7ff7e501d700 1 -- 172.21.15.94:0/372649833 wait complete. 2024-03-24T10:45:09.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.316+0000 7ff7e501d700 1 Processor -- start 2024-03-24T10:45:09.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.316+0000 7ff7e501d700 1 -- start start 2024-03-24T10:45:09.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.317+0000 7ff7e501d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7e00662e0 0x7ff7e010da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:45:09.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.317+0000 7ff7e501d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7e00fff00 0x7ff7e010df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:45:09.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.317+0000 7ff7e501d700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e010e540 con 0x7ff7e00662e0 2024-03-24T10:45:09.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.317+0000 7ff7de59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7e00fff00 0x7ff7e010df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:45:09.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.317+0000 7ff7ded9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7e00662e0 0x7ff7e010da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:45:09.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.317+0000 7ff7de59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7e00fff00 0x7ff7e010df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.94:47362/0 (socket says 172.21.15.94:47362) 2024-03-24T10:45:09.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.317+0000 7ff7de59c700 1 -- 172.21.15.94:0/1232305801 learned_addr learned my addr 172.21.15.94:0/1232305801 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:45:09.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.317+0000 7ff7e501d700 1 -- 172.21.15.94:0/1232305801 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e010ed40 con 0x7ff7e00fff00 2024-03-24T10:45:09.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.317+0000 7ff7de59c700 1 -- 172.21.15.94:0/1232305801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7e00662e0 msgr2=0x7ff7e010da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:09.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.317+0000 7ff7de59c700 1 --2- 172.21.15.94:0/1232305801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7e00662e0 0x7ff7e010da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:09.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.317+0000 7ff7de59c700 1 -- 172.21.15.94:0/1232305801 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d4005040 con 0x7ff7e00fff00 2024-03-24T10:45:09.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.318+0000 7ff7de59c700 1 --2- 172.21.15.94:0/1232305801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7e00fff00 0x7ff7e010df40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff7d40096d0 tx=0x7ff7d4009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:45:09.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.318+0000 7ff7c77fe700 1 -- 172.21.15.94:0/1232305801 <== mon.1 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7d400e840 con 0x7ff7e00fff00 2024-03-24T10:45:09.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.318+0000 7ff7c77fe700 1 -- 172.21.15.94:0/1232305801 <== mon.1 v2:172.21.15.96:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff7d400f040 con 0x7ff7e00fff00 2024-03-24T10:45:09.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.318+0000 7ff7e501d700 1 -- 172.21.15.94:0/1232305801 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7d0005040 con 0x7ff7e00fff00 2024-03-24T10:45:09.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.318+0000 7ff7c77fe700 1 -- 172.21.15.94:0/1232305801 <== mon.1 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7d4013dc0 con 0x7ff7e00fff00 2024-03-24T10:45:09.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.318+0000 7ff7e501d700 1 -- 172.21.15.94:0/1232305801 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7e010f370 con 0x7ff7e00fff00 2024-03-24T10:45:09.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.319+0000 7ff7e501d700 1 -- 172.21.15.94:0/1232305801 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7e005f5f0 con 0x7ff7e00fff00 2024-03-24T10:45:09.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.319+0000 7ff7c77fe700 1 -- 172.21.15.94:0/1232305801 <== mon.1 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7ff7d400e9a0 con 0x7ff7e00fff00 2024-03-24T10:45:09.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.320+0000 7ff7c77fe700 1 --2- 172.21.15.94:0/1232305801 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7ff7c80692f0 0x7ff7c806b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:45:09.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.320+0000 7ff7c77fe700 1 -- 172.21.15.94:0/1232305801 <== mon.1 v2:172.21.15.96:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7ff7d4087030 con 0x7ff7e00fff00 2024-03-24T10:45:09.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.320+0000 7ff7ded9d700 1 --2- 172.21.15.94:0/1232305801 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7ff7c80692f0 0x7ff7c806b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:45:09.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.320+0000 7ff7ded9d700 1 --2- 172.21.15.94:0/1232305801 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7ff7c80692f0 0x7ff7c806b7a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0013870 tx=0x7ff7d0009e30 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:45:09.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:09 smithi096 ceph-mon[38097]: from='client.14608 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:45:09.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:09 smithi096 ceph-mon[38097]: from='client.14612 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:45:09.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.324+0000 7ff7c77fe700 1 -- 172.21.15.94:0/1232305801 <== mon.1 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7ff7d4056c80 con 0x7ff7e00fff00 2024-03-24T10:45:09.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.585+0000 7ff7e501d700 1 -- 172.21.15.94:0/1232305801 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff7e010f630 con 0x7ff7e00fff00 2024-03-24T10:45:09.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.586+0000 7ff7c77fe700 1 -- 172.21.15.94:0/1232305801 <== mon.1 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1833 (secure 0 0 0) 0x7ff7d4056810 con 0x7ff7e00fff00 2024-03-24T10:45:09.587 INFO:teuthology.orchestra.run.smithi094.stdout:e18 2024-03-24T10:45:09.587 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T10:45:09.587 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:45:09.587 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-03-24T10:45:09.587 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:45:09.587 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-03-24T10:45:09.587 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-03-24T10:45:09.587 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 18 2024-03-24T10:45:09.587 INFO:teuthology.orchestra.run.smithi094.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-24T10:45:09.587 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-03-24T10:43:39.435738+0000 2024-03-24T10:45:09.587 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-03-24T10:43:58.133901+0000 2024-03-24T10:45:09.587 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-03-24T10:45:09.587 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-03-24T10:45:09.587 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 60 2024-03-24T10:45:09.587 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-03-24T10:45:09.588 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-03-24T10:45:09.588 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-03-24T10:45:09.588 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-03-24T10:45:09.588 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 46 2024-03-24T10:45:09.588 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:45:09.588 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-03-24T10:45:09.588 INFO:teuthology.orchestra.run.smithi094.stdout:in 0 2024-03-24T10:45:09.588 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=24251} 2024-03-24T10:45:09.588 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-03-24T10:45:09.588 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-03-24T10:45:09.588 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-03-24T10:45:09.588 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-03-24T10:45:09.588 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-03-24T10:45:09.588 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-03-24T10:45:09.588 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-03-24T10:45:09.588 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 1 2024-03-24T10:45:09.589 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi096.egbmoq{0:24251} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.96:6824/1539752208,v1:172.21.15.96:6825/1539752208] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:45:09.589 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.llefgu{0:14500} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.94:6828/126762980,v1:172.21.15.94:6829/126762980] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:45:09.589 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:45:09.589 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:45:09.589 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-03-24T10:45:09.589 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:45:09.589 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.efiqte{-1:14512} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.94:6826/1636528860,v1:172.21.15.94:6827/1636528860] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:45:09.589 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi096.jtebhh{-1:24261} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.96:6826/1321087930,v1:172.21.15.96:6827/1321087930] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:45:09.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.588+0000 7ff7e501d700 1 -- 172.21.15.94:0/1232305801 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7ff7c80692f0 msgr2=0x7ff7c806b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:09.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.589+0000 7ff7e501d700 1 --2- 172.21.15.94:0/1232305801 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7ff7c80692f0 0x7ff7c806b7a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0013870 tx=0x7ff7d0009e30 comp rx=0 tx=0).stop 2024-03-24T10:45:09.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.589+0000 7ff7e501d700 1 -- 172.21.15.94:0/1232305801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7e00fff00 msgr2=0x7ff7e010df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:09.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.589+0000 7ff7e501d700 1 --2- 172.21.15.94:0/1232305801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7e00fff00 0x7ff7e010df40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff7d40096d0 tx=0x7ff7d4009e30 comp rx=0 tx=0).stop 2024-03-24T10:45:09.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.589+0000 7ff7e501d700 1 -- 172.21.15.94:0/1232305801 shutdown_connections 2024-03-24T10:45:09.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.589+0000 7ff7e501d700 1 --2- 172.21.15.94:0/1232305801 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7ff7c80692f0 0x7ff7c806b7a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:09.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.589+0000 7ff7e501d700 1 --2- 172.21.15.94:0/1232305801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7e00662e0 0x7ff7e010da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:09.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.589+0000 7ff7e501d700 1 --2- 172.21.15.94:0/1232305801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7e00fff00 0x7ff7e010df40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:09.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.589+0000 7ff7e501d700 1 -- 172.21.15.94:0/1232305801 >> 172.21.15.94:0/1232305801 conn(0x7ff7e00f7430 msgr2=0x7ff7e00f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:45:09.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.589+0000 7ff7e501d700 1 -- 172.21.15.94:0/1232305801 shutdown_connections 2024-03-24T10:45:09.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.589+0000 7ff7e501d700 1 -- 172.21.15.94:0/1232305801 wait complete. 2024-03-24T10:45:09.591 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 18 2024-03-24T10:45:09.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.697+0000 7ff7c586c700 1 -- 172.21.15.94:0/840855480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7c00fbce0 msgr2=0x7ff7c00fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:09.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.697+0000 7ff7c586c700 1 --2- 172.21.15.94:0/840855480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7c00fbce0 0x7ff7c00fc0f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0004940 tx=0x7ff7b0009e30 comp rx=0 tx=0).stop 2024-03-24T10:45:09.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.697+0000 7ff7c586c700 1 -- 172.21.15.94:0/840855480 shutdown_connections 2024-03-24T10:45:09.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.697+0000 7ff7c586c700 1 --2- 172.21.15.94:0/840855480 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7c00fcf90 0x7ff7c00fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:09.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.697+0000 7ff7c586c700 1 --2- 172.21.15.94:0/840855480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7c00fbce0 0x7ff7c00fc0f0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:09.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.697+0000 7ff7c586c700 1 -- 172.21.15.94:0/840855480 >> 172.21.15.94:0/840855480 conn(0x7ff7c00f7440 msgr2=0x7ff7c00f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:45:09.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.697+0000 7ff7c586c700 1 -- 172.21.15.94:0/840855480 shutdown_connections 2024-03-24T10:45:09.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.697+0000 7ff7c586c700 1 -- 172.21.15.94:0/840855480 wait complete. 2024-03-24T10:45:09.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.698+0000 7ff7c586c700 1 Processor -- start 2024-03-24T10:45:09.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.698+0000 7ff7c586c700 1 -- start start 2024-03-24T10:45:09.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.698+0000 7ff7c586c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7c00fbce0 0x7ff7c0109490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:45:09.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.698+0000 7ff7c586c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7c00fcf90 0x7ff7c01099d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:45:09.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.698+0000 7ff7c586c700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c0105f30 con 0x7ff7c00fbce0 2024-03-24T10:45:09.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.698+0000 7ff7c586c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c01060a0 con 0x7ff7c00fcf90 2024-03-24T10:45:09.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.698+0000 7ff7beffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7c00fbce0 0x7ff7c0109490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:45:09.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.698+0000 7ff7be7fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7c00fcf90 0x7ff7c01099d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:45:09.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.698+0000 7ff7beffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7c00fbce0 0x7ff7c0109490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50474/0 (socket says 172.21.15.94:50474) 2024-03-24T10:45:09.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.698+0000 7ff7beffd700 1 -- 172.21.15.94:0/2280606442 learned_addr learned my addr 172.21.15.94:0/2280606442 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:45:09.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.698+0000 7ff7be7fc700 1 -- 172.21.15.94:0/2280606442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7c00fbce0 msgr2=0x7ff7c0109490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:09.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.698+0000 7ff7be7fc700 1 --2- 172.21.15.94:0/2280606442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7c00fbce0 0x7ff7c0109490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:09.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.698+0000 7ff7be7fc700 1 -- 172.21.15.94:0/2280606442 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b0005040 con 0x7ff7c00fcf90 2024-03-24T10:45:09.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.698+0000 7ff7beffd700 1 --2- 172.21.15.94:0/2280606442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7c00fbce0 0x7ff7c0109490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T10:45:09.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.699+0000 7ff7be7fc700 1 --2- 172.21.15.94:0/2280606442 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7c00fcf90 0x7ff7c01099d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4009fd0 tx=0x7ff7b400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:45:09.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.699+0000 7ff7a77fe700 1 -- 172.21.15.94:0/2280606442 <== mon.1 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7b4004e90 con 0x7ff7c00fcf90 2024-03-24T10:45:09.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.699+0000 7ff7c586c700 1 -- 172.21.15.94:0/2280606442 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7c01062d0 con 0x7ff7c00fcf90 2024-03-24T10:45:09.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.699+0000 7ff7a77fe700 1 -- 172.21.15.94:0/2280606442 <== mon.1 v2:172.21.15.96:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff7b400f040 con 0x7ff7c00fcf90 2024-03-24T10:45:09.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.699+0000 7ff7a77fe700 1 -- 172.21.15.94:0/2280606442 <== mon.1 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7b400ca50 con 0x7ff7c00fcf90 2024-03-24T10:45:09.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.699+0000 7ff7c586c700 1 -- 172.21.15.94:0/2280606442 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7c0106890 con 0x7ff7c00fcf90 2024-03-24T10:45:09.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.700+0000 7ff7c586c700 1 -- 172.21.15.94:0/2280606442 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7c0101920 con 0x7ff7c00fcf90 2024-03-24T10:45:09.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.700+0000 7ff7a77fe700 1 -- 172.21.15.94:0/2280606442 <== mon.1 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7ff7b4007cc0 con 0x7ff7c00fcf90 2024-03-24T10:45:09.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.701+0000 7ff7a77fe700 1 --2- 172.21.15.94:0/2280606442 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7ff7a80692a0 0x7ff7a806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:45:09.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.701+0000 7ff7a77fe700 1 -- 172.21.15.94:0/2280606442 <== mon.1 v2:172.21.15.96:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7ff7b4086ca0 con 0x7ff7c00fcf90 2024-03-24T10:45:09.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.701+0000 7ff7beffd700 1 --2- 172.21.15.94:0/2280606442 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7ff7a80692a0 0x7ff7a806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:45:09.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.701+0000 7ff7beffd700 1 --2- 172.21.15.94:0/2280606442 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7ff7a80692a0 0x7ff7a806b750 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff7b000b0d0 tx=0x7ff7b0009e30 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:45:09.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.705+0000 7ff7a77fe700 1 -- 172.21.15.94:0/2280606442 <== mon.1 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7ff7b404e190 con 0x7ff7c00fcf90 2024-03-24T10:45:09.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.951+0000 7ff7c586c700 1 -- 172.21.15.94:0/2280606442 --> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff7c0104940 con 0x7ff7a80692a0 2024-03-24T10:45:09.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.952+0000 7ff7a77fe700 1 -- 172.21.15.94:0/2280606442 <== mgr.14232 v2:172.21.15.94:6800/2184357820 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7ff7c0104940 con 0x7ff7a80692a0 2024-03-24T10:45:09.953 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-03-24T10:45:09.953 INFO:teuthology.orchestra.run.smithi094.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919", 2024-03-24T10:45:09.953 INFO:teuthology.orchestra.run.smithi094.stdout: "in_progress": true, 2024-03-24T10:45:09.953 INFO:teuthology.orchestra.run.smithi094.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T10:45:09.953 INFO:teuthology.orchestra.run.smithi094.stdout: "services_complete": [], 2024-03-24T10:45:09.954 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": "0/21 daemons upgraded", 2024-03-24T10:45:09.954 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 image on host smithi096", 2024-03-24T10:45:09.954 INFO:teuthology.orchestra.run.smithi094.stdout: "is_paused": false 2024-03-24T10:45:09.954 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-03-24T10:45:09.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.955+0000 7ff7c586c700 1 -- 172.21.15.94:0/2280606442 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7ff7a80692a0 msgr2=0x7ff7a806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:09.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.955+0000 7ff7c586c700 1 --2- 172.21.15.94:0/2280606442 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7ff7a80692a0 0x7ff7a806b750 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff7b000b0d0 tx=0x7ff7b0009e30 comp rx=0 tx=0).stop 2024-03-24T10:45:09.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.955+0000 7ff7c586c700 1 -- 172.21.15.94:0/2280606442 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7c00fcf90 msgr2=0x7ff7c01099d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:09.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.955+0000 7ff7c586c700 1 --2- 172.21.15.94:0/2280606442 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7c00fcf90 0x7ff7c01099d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4009fd0 tx=0x7ff7b400b040 comp rx=0 tx=0).stop 2024-03-24T10:45:09.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.955+0000 7ff7c586c700 1 -- 172.21.15.94:0/2280606442 shutdown_connections 2024-03-24T10:45:09.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.955+0000 7ff7c586c700 1 --2- 172.21.15.94:0/2280606442 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7ff7a80692a0 0x7ff7a806b750 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:09.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.955+0000 7ff7c586c700 1 --2- 172.21.15.94:0/2280606442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7c00fbce0 0x7ff7c0109490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:09.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.955+0000 7ff7c586c700 1 --2- 172.21.15.94:0/2280606442 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7c00fcf90 0x7ff7c01099d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:09.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.955+0000 7ff7c586c700 1 -- 172.21.15.94:0/2280606442 >> 172.21.15.94:0/2280606442 conn(0x7ff7c00f7440 msgr2=0x7ff7c00f8a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:45:09.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.955+0000 7ff7c586c700 1 -- 172.21.15.94:0/2280606442 shutdown_connections 2024-03-24T10:45:09.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:09.955+0000 7ff7c586c700 1 -- 172.21.15.94:0/2280606442 wait complete. 2024-03-24T10:45:10.060 INFO:tasks.workunit.client.1.smithi096.stderr:Note: switching to 'd39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b'. 2024-03-24T10:45:10.060 INFO:tasks.workunit.client.1.smithi096.stderr: 2024-03-24T10:45:10.061 INFO:tasks.workunit.client.1.smithi096.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-24T10:45:10.061 INFO:tasks.workunit.client.1.smithi096.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-24T10:45:10.061 INFO:tasks.workunit.client.1.smithi096.stderr:state without impacting any branches by switching back to a branch. 2024-03-24T10:45:10.061 INFO:tasks.workunit.client.1.smithi096.stderr: 2024-03-24T10:45:10.061 INFO:tasks.workunit.client.1.smithi096.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-24T10:45:10.061 INFO:tasks.workunit.client.1.smithi096.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-24T10:45:10.061 INFO:tasks.workunit.client.1.smithi096.stderr: 2024-03-24T10:45:10.061 INFO:tasks.workunit.client.1.smithi096.stderr: git switch -c 2024-03-24T10:45:10.061 INFO:tasks.workunit.client.1.smithi096.stderr: 2024-03-24T10:45:10.061 INFO:tasks.workunit.client.1.smithi096.stderr:Or undo this operation with: 2024-03-24T10:45:10.061 INFO:tasks.workunit.client.1.smithi096.stderr: 2024-03-24T10:45:10.061 INFO:tasks.workunit.client.1.smithi096.stderr: git switch - 2024-03-24T10:45:10.061 INFO:tasks.workunit.client.1.smithi096.stderr: 2024-03-24T10:45:10.062 INFO:tasks.workunit.client.1.smithi096.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-24T10:45:10.062 INFO:tasks.workunit.client.1.smithi096.stderr: 2024-03-24T10:45:10.062 INFO:tasks.workunit.client.1.smithi096.stderr:HEAD is now at d39ab7bb1f1 qa/suites/fs/workload: enable snap_schedule early 2024-03-24T10:45:10.065 DEBUG:teuthology.orchestra.run.smithi096:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-03-24T10:45:10.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:10.071+0000 7ff6e31fb700 1 -- 172.21.15.94:0/2770066954 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6dc070890 msgr2=0x7ff6dc070d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:10.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:10.071+0000 7ff6e31fb700 1 --2- 172.21.15.94:0/2770066954 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6dc070890 0x7ff6dc070d00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8004940 tx=0x7ff6c8009e30 comp rx=0 tx=0).stop 2024-03-24T10:45:10.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:10.071+0000 7ff6e31fb700 1 -- 172.21.15.94:0/2770066954 shutdown_connections 2024-03-24T10:45:10.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:10.071+0000 7ff6e31fb700 1 --2- 172.21.15.94:0/2770066954 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6dc070890 0x7ff6dc070d00 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:10.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:10.071+0000 7ff6e31fb700 1 --2- 172.21.15.94:0/2770066954 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6dc071e40 0x7ff6dc070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:10.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:10.071+0000 7ff6e31fb700 1 -- 172.21.15.94:0/2770066954 >> 172.21.15.94:0/2770066954 conn(0x7ff6dc0f8550 msgr2=0x7ff6dc0fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:45:10.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:10.071+0000 7ff6e31fb700 1 -- 172.21.15.94:0/2770066954 shutdown_connections 2024-03-24T10:45:10.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:10.071+0000 7ff6e31fb700 1 -- 172.21.15.94:0/2770066954 wait complete. 2024-03-24T10:45:10.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:10.072+0000 7ff6e31fb700 1 Processor -- start 2024-03-24T10:45:10.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:10.072+0000 7ff6e31fb700 1 -- start start 2024-03-24T10:45:10.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:10.072+0000 7ff6e31fb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6dc070890 0x7ff6dc10fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:45:10.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:10.072+0000 7ff6e31fb700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6dc071e40 0x7ff6dc110100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:45:10.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:10.072+0000 7ff6e31fb700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6dc110700 con 0x7ff6dc071e40 2024-03-24T10:45:10.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:10.072+0000 7ff6e31fb700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6dc110f50 con 0x7ff6dc070890 2024-03-24T10:45:10.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:10.072+0000 7ff6dbfff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6dc071e40 0x7ff6dc110100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:45:10.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:10.072+0000 7ff6e0f97700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6dc070890 0x7ff6dc10fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:45:10.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:10.072+0000 7ff6dbfff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6dc071e40 0x7ff6dc110100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50498/0 (socket says 172.21.15.94:50498) 2024-03-24T10:45:10.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:10.072+0000 7ff6e0f97700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6dc070890 0x7ff6dc10fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.94:40816/0 (socket says 172.21.15.94:40816) 2024-03-24T10:45:10.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:10.072+0000 7ff6dbfff700 1 -- 172.21.15.94:0/4272949217 learned_addr learned my addr 172.21.15.94:0/4272949217 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:45:10.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:10.073+0000 7ff6dbfff700 1 -- 172.21.15.94:0/4272949217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6dc070890 msgr2=0x7ff6dc10fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:10.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:10.073+0000 7ff6dbfff700 1 --2- 172.21.15.94:0/4272949217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6dc070890 0x7ff6dc10fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:10.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:10.073+0000 7ff6dbfff700 1 -- 172.21.15.94:0/4272949217 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6c8005040 con 0x7ff6dc071e40 2024-03-24T10:45:10.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:10.073+0000 7ff6e0f97700 1 --2- 172.21.15.94:0/4272949217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6dc070890 0x7ff6dc10fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T10:45:10.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:10.073+0000 7ff6dbfff700 1 --2- 172.21.15.94:0/4272949217 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6dc071e40 0x7ff6dc110100 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff6c800b0d0 tx=0x7ff6c8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:45:10.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:10.073+0000 7ff6d97fa700 1 -- 172.21.15.94:0/4272949217 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6c800e030 con 0x7ff6dc071e40 2024-03-24T10:45:10.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:10.073+0000 7ff6e31fb700 1 -- 172.21.15.94:0/4272949217 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6dc1111d0 con 0x7ff6dc071e40 2024-03-24T10:45:10.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:10.073+0000 7ff6d97fa700 1 -- 172.21.15.94:0/4272949217 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff6c801c400 con 0x7ff6dc071e40 2024-03-24T10:45:10.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:10.073+0000 7ff6e31fb700 1 -- 172.21.15.94:0/4272949217 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6dc111710 con 0x7ff6dc071e40 2024-03-24T10:45:10.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:10.073+0000 7ff6d97fa700 1 -- 172.21.15.94:0/4272949217 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6c800b630 con 0x7ff6dc071e40 2024-03-24T10:45:10.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:10.074+0000 7ff6e31fb700 1 -- 172.21.15.94:0/4272949217 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6dc05f5f0 con 0x7ff6dc071e40 2024-03-24T10:45:10.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:10.075+0000 7ff6d97fa700 1 -- 172.21.15.94:0/4272949217 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7ff6c8013af0 con 0x7ff6dc071e40 2024-03-24T10:45:10.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:10.076+0000 7ff6d97fa700 1 --2- 172.21.15.94:0/4272949217 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7ff6cc0692a0 0x7ff6cc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:45:10.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:10.076+0000 7ff6d97fa700 1 -- 172.21.15.94:0/4272949217 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7ff6c8088d10 con 0x7ff6dc071e40 2024-03-24T10:45:10.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:10.076+0000 7ff6e0f97700 1 --2- 172.21.15.94:0/4272949217 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7ff6cc0692a0 0x7ff6cc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:45:10.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:10.076+0000 7ff6e0f97700 1 --2- 172.21.15.94:0/4272949217 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7ff6cc0692a0 0x7ff6cc06b750 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0009b80 tx=0x7ff6d000b040 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:45:10.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:10.083+0000 7ff6d97fa700 1 -- 172.21.15.94:0/4272949217 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7ff6c8058b20 con 0x7ff6dc071e40 2024-03-24T10:45:10.127 INFO:tasks.workunit.client.1.smithi096.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-24T10:45:10.129 INFO:tasks.workunit.client.1.smithi096.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-03-24T10:45:10.129 INFO:tasks.workunit.client.1.smithi096.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-24T10:45:10.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:10 smithi094 ceph-mon[29446]: from='client.14616 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:45:10.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:10 smithi094 ceph-mon[29446]: pgmap v218: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 85 B/s wr, 2 op/s 2024-03-24T10:45:10.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:10 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/3231681911' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:45:10.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:10 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/1232305801' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T10:45:10.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:10 smithi096 ceph-mon[38097]: from='client.14616 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:45:10.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:10 smithi096 ceph-mon[38097]: pgmap v218: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 85 B/s wr, 2 op/s 2024-03-24T10:45:10.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:10 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/3231681911' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:45:10.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:10 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/1232305801' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T10:45:10.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:10.426+0000 7ff6e31fb700 1 -- 172.21.15.94:0/4272949217 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff6dc04bed0 con 0x7ff6dc071e40 2024-03-24T10:45:10.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:10.426+0000 7ff6d97fa700 1 -- 172.21.15.94:0/4272949217 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7ff6c80586b0 con 0x7ff6dc071e40 2024-03-24T10:45:10.427 INFO:teuthology.orchestra.run.smithi094.stdout:HEALTH_OK 2024-03-24T10:45:10.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:10.429+0000 7ff6e31fb700 1 -- 172.21.15.94:0/4272949217 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7ff6cc0692a0 msgr2=0x7ff6cc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:10.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:10.429+0000 7ff6e31fb700 1 --2- 172.21.15.94:0/4272949217 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7ff6cc0692a0 0x7ff6cc06b750 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0009b80 tx=0x7ff6d000b040 comp rx=0 tx=0).stop 2024-03-24T10:45:10.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:10.429+0000 7ff6e31fb700 1 -- 172.21.15.94:0/4272949217 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6dc071e40 msgr2=0x7ff6dc110100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:10.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:10.429+0000 7ff6e31fb700 1 --2- 172.21.15.94:0/4272949217 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6dc071e40 0x7ff6dc110100 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff6c800b0d0 tx=0x7ff6c8009e30 comp rx=0 tx=0).stop 2024-03-24T10:45:10.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:10.429+0000 7ff6e31fb700 1 -- 172.21.15.94:0/4272949217 shutdown_connections 2024-03-24T10:45:10.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:10.429+0000 7ff6e31fb700 1 --2- 172.21.15.94:0/4272949217 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7ff6cc0692a0 0x7ff6cc06b750 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:10.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:10.429+0000 7ff6e31fb700 1 --2- 172.21.15.94:0/4272949217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6dc070890 0x7ff6dc10fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:10.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:10.429+0000 7ff6e31fb700 1 --2- 172.21.15.94:0/4272949217 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6dc071e40 0x7ff6dc110100 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:10.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:10.429+0000 7ff6e31fb700 1 -- 172.21.15.94:0/4272949217 >> 172.21.15.94:0/4272949217 conn(0x7ff6dc0f8550 msgr2=0x7ff6dc0f9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:45:10.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:10.429+0000 7ff6e31fb700 1 -- 172.21.15.94:0/4272949217 shutdown_connections 2024-03-24T10:45:10.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:10.429+0000 7ff6e31fb700 1 -- 172.21.15.94:0/4272949217 wait complete. 2024-03-24T10:45:10.955 INFO:tasks.workunit.client.1.smithi096.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-24T10:45:11.020 INFO:tasks.workunit.client.1.smithi096.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-24T10:45:11.060 INFO:tasks.workunit.client.1.smithi096.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-03-24T10:45:11.062 INFO:tasks.workunit.client.1.smithi096.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-03-24T10:45:11.062 INFO:tasks.workunit.client.1.smithi096.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-24T10:45:11.101 INFO:tasks.workunit.client.1.smithi096.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-03-24T10:45:11.105 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-24T10:45:11.105 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-03-24T10:45:11.161 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-03-24T10:45:11.162 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-03-24T10:45:11.162 DEBUG:teuthology.orchestra.run.smithi096:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-03-24T10:45:11.221 INFO:tasks.workunit.client.1.smithi096.stderr:+ mkdir -p fsstress 2024-03-24T10:45:11.222 INFO:tasks.workunit.client.1.smithi096.stderr:+ pushd fsstress 2024-03-24T10:45:11.222 INFO:tasks.workunit.client.1.smithi096.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-03-24T10:45:11.223 INFO:tasks.workunit.client.1.smithi096.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-03-24T10:45:11.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:11 smithi096 ceph-mon[38097]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:45:11.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:11 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/4272949217' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T10:45:11.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:11 smithi094 ceph-mon[29446]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:45:11.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:11 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/4272949217' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T10:45:11.942 INFO:tasks.workunit.client.1.smithi096.stderr:+ tar xzf ltp-full.tgz 2024-03-24T10:45:12.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:12 smithi096 ceph-mon[38097]: pgmap v219: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 85 B/s wr, 2 op/s 2024-03-24T10:45:12.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:12 smithi094 ceph-mon[29446]: pgmap v219: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 85 B/s wr, 2 op/s 2024-03-24T10:45:13.223 INFO:tasks.workunit.client.0.smithi094.stderr:Note: switching to 'd39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b'. 2024-03-24T10:45:13.223 INFO:tasks.workunit.client.0.smithi094.stderr: 2024-03-24T10:45:13.223 INFO:tasks.workunit.client.0.smithi094.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-24T10:45:13.223 INFO:tasks.workunit.client.0.smithi094.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-24T10:45:13.223 INFO:tasks.workunit.client.0.smithi094.stderr:state without impacting any branches by switching back to a branch. 2024-03-24T10:45:13.224 INFO:tasks.workunit.client.0.smithi094.stderr: 2024-03-24T10:45:13.224 INFO:tasks.workunit.client.0.smithi094.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-24T10:45:13.224 INFO:tasks.workunit.client.0.smithi094.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-24T10:45:13.224 INFO:tasks.workunit.client.0.smithi094.stderr: 2024-03-24T10:45:13.224 INFO:tasks.workunit.client.0.smithi094.stderr: git switch -c 2024-03-24T10:45:13.224 INFO:tasks.workunit.client.0.smithi094.stderr: 2024-03-24T10:45:13.224 INFO:tasks.workunit.client.0.smithi094.stderr:Or undo this operation with: 2024-03-24T10:45:13.224 INFO:tasks.workunit.client.0.smithi094.stderr: 2024-03-24T10:45:13.224 INFO:tasks.workunit.client.0.smithi094.stderr: git switch - 2024-03-24T10:45:13.224 INFO:tasks.workunit.client.0.smithi094.stderr: 2024-03-24T10:45:13.224 INFO:tasks.workunit.client.0.smithi094.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-24T10:45:13.224 INFO:tasks.workunit.client.0.smithi094.stderr: 2024-03-24T10:45:13.224 INFO:tasks.workunit.client.0.smithi094.stderr:HEAD is now at d39ab7bb1f1 qa/suites/fs/workload: enable snap_schedule early 2024-03-24T10:45:13.228 DEBUG:teuthology.orchestra.run.smithi094:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-24T10:45:13.253 INFO:tasks.workunit.client.0.smithi094.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-24T10:45:13.255 INFO:tasks.workunit.client.0.smithi094.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-24T10:45:13.255 INFO:tasks.workunit.client.0.smithi094.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-24T10:45:14.146 INFO:tasks.workunit.client.0.smithi094.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-24T10:45:14.206 INFO:tasks.workunit.client.0.smithi094.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-24T10:45:14.248 INFO:tasks.workunit.client.0.smithi094.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-24T10:45:14.250 INFO:tasks.workunit.client.0.smithi094.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-24T10:45:14.250 INFO:tasks.workunit.client.0.smithi094.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-24T10:45:14.289 INFO:tasks.workunit.client.0.smithi094.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-24T10:45:14.292 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-03-24T10:45:14.292 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-24T10:45:14.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:14 smithi096 ceph-mon[38097]: pgmap v220: 65 pgs: 65 active+clean; 599 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-03-24T10:45:14.348 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-03-24T10:45:14.349 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-03-24T10:45:14.349 DEBUG:teuthology.orchestra.run.smithi094:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-03-24T10:45:14.407 INFO:tasks.workunit.client.0.smithi094.stderr:+ mkdir -p fsstress 2024-03-24T10:45:14.409 INFO:tasks.workunit.client.0.smithi094.stderr:+ pushd fsstress 2024-03-24T10:45:14.409 INFO:tasks.workunit.client.0.smithi094.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-24T10:45:14.409 INFO:tasks.workunit.client.0.smithi094.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-03-24T10:45:14.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:14 smithi094 ceph-mon[29446]: pgmap v220: 65 pgs: 65 active+clean; 599 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-03-24T10:45:15.265 INFO:tasks.workunit.client.0.smithi094.stderr:+ tar xzf ltp-full.tgz 2024-03-24T10:45:16.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:16 smithi096 ceph-mon[38097]: pgmap v221: 65 pgs: 65 active+clean; 599 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-24T10:45:16.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:16 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:45:16.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:16 smithi094 ceph-mon[29446]: pgmap v221: 65 pgs: 65 active+clean; 599 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-24T10:45:16.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:16 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:45:18.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:18 smithi096 ceph-mon[38097]: pgmap v222: 65 pgs: 65 active+clean; 12 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 683 KiB/s rd, 1.0 MiB/s wr, 28 op/s 2024-03-24T10:45:18.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:18 smithi094 ceph-mon[29446]: pgmap v222: 65 pgs: 65 active+clean; 12 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 683 KiB/s rd, 1.0 MiB/s wr, 28 op/s 2024-03-24T10:45:20.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:20 smithi094 ceph-mon[29446]: pgmap v223: 65 pgs: 65 active+clean; 15 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 683 KiB/s rd, 1.2 MiB/s wr, 60 op/s 2024-03-24T10:45:20.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:20 smithi096 ceph-mon[38097]: pgmap v223: 65 pgs: 65 active+clean; 15 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 683 KiB/s rd, 1.2 MiB/s wr, 60 op/s 2024-03-24T10:45:22.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:22 smithi096 ceph-mon[38097]: pgmap v224: 65 pgs: 65 active+clean; 46 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 4.0 MiB/s wr, 312 op/s 2024-03-24T10:45:22.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:22 smithi094 ceph-mon[29446]: pgmap v224: 65 pgs: 65 active+clean; 46 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 4.0 MiB/s wr, 312 op/s 2024-03-24T10:45:24.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:24 smithi096 ceph-mon[38097]: pgmap v225: 65 pgs: 65 active+clean; 69 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 5.9 MiB/s wr, 456 op/s 2024-03-24T10:45:24.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:24 smithi094 ceph-mon[29446]: pgmap v225: 65 pgs: 65 active+clean; 69 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 5.9 MiB/s wr, 456 op/s 2024-03-24T10:45:26.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:26 smithi096 ceph-mon[38097]: pgmap v226: 65 pgs: 65 active+clean; 74 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 6.4 MiB/s wr, 485 op/s 2024-03-24T10:45:26.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:26 smithi094 ceph-mon[29446]: pgmap v226: 65 pgs: 65 active+clean; 74 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 6.4 MiB/s wr, 485 op/s 2024-03-24T10:45:28.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:28 smithi096 ceph-mon[38097]: pgmap v227: 65 pgs: 65 active+clean; 85 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.9 MiB/s rd, 7.3 MiB/s wr, 499 op/s 2024-03-24T10:45:28.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:28 smithi094 ceph-mon[29446]: pgmap v227: 65 pgs: 65 active+clean; 85 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.9 MiB/s rd, 7.3 MiB/s wr, 499 op/s 2024-03-24T10:45:30.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:30 smithi094 ceph-mon[29446]: pgmap v228: 65 pgs: 65 active+clean; 85 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 6.3 MiB/s wr, 472 op/s 2024-03-24T10:45:30.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:30 smithi096 ceph-mon[38097]: pgmap v228: 65 pgs: 65 active+clean; 85 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 6.3 MiB/s wr, 472 op/s 2024-03-24T10:45:31.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:31 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:45:31.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:31 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:45:32.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:32 smithi096 ceph-mon[38097]: pgmap v229: 65 pgs: 65 active+clean; 93 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.8 MiB/s rd, 6.8 MiB/s wr, 466 op/s 2024-03-24T10:45:32.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:32 smithi094 ceph-mon[29446]: pgmap v229: 65 pgs: 65 active+clean; 93 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.8 MiB/s rd, 6.8 MiB/s wr, 466 op/s 2024-03-24T10:45:34.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:34 smithi096 ceph-mon[38097]: pgmap v230: 65 pgs: 65 active+clean; 104 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.6 MiB/s rd, 5.1 MiB/s wr, 336 op/s 2024-03-24T10:45:34.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:34 smithi094 ceph-mon[29446]: pgmap v230: 65 pgs: 65 active+clean; 104 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.6 MiB/s rd, 5.1 MiB/s wr, 336 op/s 2024-03-24T10:45:36.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:36 smithi096 ceph-mon[38097]: pgmap v231: 65 pgs: 65 active+clean; 113 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 357 op/s 2024-03-24T10:45:36.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:36 smithi094 ceph-mon[29446]: pgmap v231: 65 pgs: 65 active+clean; 113 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 357 op/s 2024-03-24T10:45:38.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:38 smithi096 ceph-mon[38097]: pgmap v232: 65 pgs: 65 active+clean; 145 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.6 MiB/s rd, 6.3 MiB/s wr, 787 op/s 2024-03-24T10:45:38.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:38 smithi094 ceph-mon[29446]: pgmap v232: 65 pgs: 65 active+clean; 145 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.6 MiB/s rd, 6.3 MiB/s wr, 787 op/s 2024-03-24T10:45:40.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:40 smithi094 ceph-mon[29446]: pgmap v233: 65 pgs: 65 active+clean; 145 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 5.4 MiB/s wr, 773 op/s 2024-03-24T10:45:40.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:40 smithi096 ceph-mon[38097]: pgmap v233: 65 pgs: 65 active+clean; 145 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 5.4 MiB/s wr, 773 op/s 2024-03-24T10:45:40.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.546+0000 7feb06055700 1 -- 172.21.15.94:0/2824923793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb000fbcd0 msgr2=0x7feb000fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:40.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.546+0000 7feb06055700 1 --2- 172.21.15.94:0/2824923793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb000fbcd0 0x7feb000fc0e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7feaf8004940 tx=0x7feaf8009e30 comp rx=0 tx=0).stop 2024-03-24T10:45:40.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.546+0000 7feb06055700 1 -- 172.21.15.94:0/2824923793 shutdown_connections 2024-03-24T10:45:40.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.546+0000 7feb06055700 1 --2- 172.21.15.94:0/2824923793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb000fcf80 0x7feb000fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:40.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.546+0000 7feb06055700 1 --2- 172.21.15.94:0/2824923793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb000fbcd0 0x7feb000fc0e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:40.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.546+0000 7feb06055700 1 -- 172.21.15.94:0/2824923793 >> 172.21.15.94:0/2824923793 conn(0x7feb000f7430 msgr2=0x7feb000f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:45:40.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.546+0000 7feb06055700 1 -- 172.21.15.94:0/2824923793 shutdown_connections 2024-03-24T10:45:40.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.547+0000 7feb06055700 1 -- 172.21.15.94:0/2824923793 wait complete. 2024-03-24T10:45:40.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.547+0000 7feb06055700 1 Processor -- start 2024-03-24T10:45:40.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.547+0000 7feb06055700 1 -- start start 2024-03-24T10:45:40.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.548+0000 7feb06055700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb000fbcd0 0x7feb0010c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:45:40.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.548+0000 7feb06055700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb000fcf80 0x7feb0010cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:45:40.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.548+0000 7feb06055700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0010d3d0 con 0x7feb000fcf80 2024-03-24T10:45:40.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.548+0000 7feb06055700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0010dc20 con 0x7feb000fbcd0 2024-03-24T10:45:40.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.548+0000 7feafeffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb000fcf80 0x7feb0010cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:45:40.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.548+0000 7feafeffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb000fcf80 0x7feb0010cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33304/0 (socket says 172.21.15.94:33304) 2024-03-24T10:45:40.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.548+0000 7feafeffd700 1 -- 172.21.15.94:0/2737976595 learned_addr learned my addr 172.21.15.94:0/2737976595 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:45:40.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.548+0000 7feaff7fe700 1 --2- 172.21.15.94:0/2737976595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb000fbcd0 0x7feb0010c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:45:40.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.548+0000 7feafeffd700 1 -- 172.21.15.94:0/2737976595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb000fbcd0 msgr2=0x7feb0010c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:40.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.548+0000 7feafeffd700 1 --2- 172.21.15.94:0/2737976595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb000fbcd0 0x7feb0010c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:40.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.548+0000 7feafeffd700 1 -- 172.21.15.94:0/2737976595 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf8005040 con 0x7feb000fcf80 2024-03-24T10:45:40.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.548+0000 7feafeffd700 1 --2- 172.21.15.94:0/2737976595 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb000fcf80 0x7feb0010cdd0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7feae800eb70 tx=0x7feae800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:45:40.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.548+0000 7feaf7fff700 1 -- 172.21.15.94:0/2737976595 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feae8007cc0 con 0x7feb000fcf80 2024-03-24T10:45:40.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.548+0000 7feaf7fff700 1 -- 172.21.15.94:0/2737976595 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feae800f040 con 0x7feb000fcf80 2024-03-24T10:45:40.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.548+0000 7feb06055700 1 -- 172.21.15.94:0/2737976595 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb0010dea0 con 0x7feb000fcf80 2024-03-24T10:45:40.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.548+0000 7feaf7fff700 1 -- 172.21.15.94:0/2737976595 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feae800ca90 con 0x7feb000fcf80 2024-03-24T10:45:40.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.548+0000 7feb06055700 1 -- 172.21.15.94:0/2737976595 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb0010e3e0 con 0x7feb000fcf80 2024-03-24T10:45:40.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.549+0000 7feb06055700 1 -- 172.21.15.94:0/2737976595 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb0005f5f0 con 0x7feb000fcf80 2024-03-24T10:45:40.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.550+0000 7feaf7fff700 1 -- 172.21.15.94:0/2737976595 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7feae800cbf0 con 0x7feb000fcf80 2024-03-24T10:45:40.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.550+0000 7feaf7fff700 1 --2- 172.21.15.94:0/2737976595 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7feaec0692a0 0x7feaec06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:45:40.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.550+0000 7feaf7fff700 1 -- 172.21.15.94:0/2737976595 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7feae8086cd0 con 0x7feb000fcf80 2024-03-24T10:45:40.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.550+0000 7feaff7fe700 1 --2- 172.21.15.94:0/2737976595 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7feaec0692a0 0x7feaec06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:45:40.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.551+0000 7feaff7fe700 1 --2- 172.21.15.94:0/2737976595 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7feaec0692a0 0x7feaec06b750 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7feaf8009b30 tx=0x7feaf8009e30 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:45:40.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.555+0000 7feaf7fff700 1 -- 172.21.15.94:0/2737976595 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7feae8056a40 con 0x7feb000fcf80 2024-03-24T10:45:40.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.764+0000 7feb06055700 1 -- 172.21.15.94:0/2737976595 --> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7feb0010e650 con 0x7feaec0692a0 2024-03-24T10:45:40.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.765+0000 7feaf7fff700 1 -- 172.21.15.94:0/2737976595 <== mgr.14232 v2:172.21.15.94:6800/2184357820 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7feb0010e650 con 0x7feaec0692a0 2024-03-24T10:45:40.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.767+0000 7feb06055700 1 -- 172.21.15.94:0/2737976595 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7feaec0692a0 msgr2=0x7feaec06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:40.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.767+0000 7feb06055700 1 --2- 172.21.15.94:0/2737976595 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7feaec0692a0 0x7feaec06b750 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7feaf8009b30 tx=0x7feaf8009e30 comp rx=0 tx=0).stop 2024-03-24T10:45:40.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.767+0000 7feb06055700 1 -- 172.21.15.94:0/2737976595 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb000fcf80 msgr2=0x7feb0010cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:40.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.767+0000 7feb06055700 1 --2- 172.21.15.94:0/2737976595 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb000fcf80 0x7feb0010cdd0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7feae800eb70 tx=0x7feae800b040 comp rx=0 tx=0).stop 2024-03-24T10:45:40.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.767+0000 7feb06055700 1 -- 172.21.15.94:0/2737976595 shutdown_connections 2024-03-24T10:45:40.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.767+0000 7feb06055700 1 --2- 172.21.15.94:0/2737976595 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7feaec0692a0 0x7feaec06b750 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:40.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.767+0000 7feb06055700 1 --2- 172.21.15.94:0/2737976595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb000fbcd0 0x7feb0010c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:40.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.767+0000 7feb06055700 1 --2- 172.21.15.94:0/2737976595 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb000fcf80 0x7feb0010cdd0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:40.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.767+0000 7feb06055700 1 -- 172.21.15.94:0/2737976595 >> 172.21.15.94:0/2737976595 conn(0x7feb000f7430 msgr2=0x7feb000f8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:45:40.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.767+0000 7feb06055700 1 -- 172.21.15.94:0/2737976595 shutdown_connections 2024-03-24T10:45:40.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.767+0000 7feb06055700 1 -- 172.21.15.94:0/2737976595 wait complete. 2024-03-24T10:45:40.776 INFO:teuthology.orchestra.run.smithi094.stdout:true 2024-03-24T10:45:40.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.869+0000 7efe3d466700 1 -- 172.21.15.94:0/1020825784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe380662e0 msgr2=0x7efe380ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:40.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.869+0000 7efe3d466700 1 --2- 172.21.15.94:0/1020825784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe380662e0 0x7efe380ff9c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7efe28004940 tx=0x7efe28009e30 comp rx=0 tx=0).stop 2024-03-24T10:45:40.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.869+0000 7efe3d466700 1 -- 172.21.15.94:0/1020825784 shutdown_connections 2024-03-24T10:45:40.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.869+0000 7efe3d466700 1 --2- 172.21.15.94:0/1020825784 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe380fff00 0x7efe38102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:40.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.869+0000 7efe3d466700 1 --2- 172.21.15.94:0/1020825784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe380662e0 0x7efe380ff9c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:40.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.869+0000 7efe3d466700 1 -- 172.21.15.94:0/1020825784 >> 172.21.15.94:0/1020825784 conn(0x7efe380f7430 msgr2=0x7efe380f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:45:40.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.869+0000 7efe3d466700 1 -- 172.21.15.94:0/1020825784 shutdown_connections 2024-03-24T10:45:40.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.869+0000 7efe3d466700 1 -- 172.21.15.94:0/1020825784 wait complete. 2024-03-24T10:45:40.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.869+0000 7efe3d466700 1 Processor -- start 2024-03-24T10:45:40.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.870+0000 7efe3d466700 1 -- start start 2024-03-24T10:45:40.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.870+0000 7efe3d466700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe380662e0 0x7efe380fd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:45:40.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.870+0000 7efe3d466700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe380fff00 0x7efe380fb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:45:40.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.870+0000 7efe3d466700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe380fbe50 con 0x7efe380fff00 2024-03-24T10:45:40.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.870+0000 7efe3d466700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe380fbf90 con 0x7efe380662e0 2024-03-24T10:45:40.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.870+0000 7efe2e7fc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe380fff00 0x7efe380fb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:45:40.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.870+0000 7efe2e7fc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe380fff00 0x7efe380fb8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33326/0 (socket says 172.21.15.94:33326) 2024-03-24T10:45:40.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.870+0000 7efe36ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe380662e0 0x7efe380fd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:45:40.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.870+0000 7efe2e7fc700 1 -- 172.21.15.94:0/1912732471 learned_addr learned my addr 172.21.15.94:0/1912732471 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:45:40.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.870+0000 7efe2e7fc700 1 -- 172.21.15.94:0/1912732471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe380662e0 msgr2=0x7efe380fd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:40.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.870+0000 7efe2e7fc700 1 --2- 172.21.15.94:0/1912732471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe380662e0 0x7efe380fd3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:40.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.870+0000 7efe2e7fc700 1 -- 172.21.15.94:0/1912732471 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe28005040 con 0x7efe380fff00 2024-03-24T10:45:40.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.870+0000 7efe2e7fc700 1 --2- 172.21.15.94:0/1912732471 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe380fff00 0x7efe380fb8c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7efe2000eb70 tx=0x7efe2000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:45:40.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.870+0000 7efe2ffff700 1 -- 172.21.15.94:0/1912732471 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe20007cc0 con 0x7efe380fff00 2024-03-24T10:45:40.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.871+0000 7efe2ffff700 1 -- 172.21.15.94:0/1912732471 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efe2000f040 con 0x7efe380fff00 2024-03-24T10:45:40.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.871+0000 7efe3d466700 1 -- 172.21.15.94:0/1912732471 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe380fc210 con 0x7efe380fff00 2024-03-24T10:45:40.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.871+0000 7efe2ffff700 1 -- 172.21.15.94:0/1912732471 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe2000ca90 con 0x7efe380fff00 2024-03-24T10:45:40.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.871+0000 7efe3d466700 1 -- 172.21.15.94:0/1912732471 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe38115a50 con 0x7efe380fff00 2024-03-24T10:45:40.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.872+0000 7efe3d466700 1 -- 172.21.15.94:0/1912732471 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe3805f5f0 con 0x7efe380fff00 2024-03-24T10:45:40.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.872+0000 7efe2ffff700 1 -- 172.21.15.94:0/1912732471 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7efe2000cbf0 con 0x7efe380fff00 2024-03-24T10:45:40.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.872+0000 7efe2ffff700 1 --2- 172.21.15.94:0/1912732471 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7efe240692a0 0x7efe2406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:45:40.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.873+0000 7efe2ffff700 1 -- 172.21.15.94:0/1912732471 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7efe20086d70 con 0x7efe380fff00 2024-03-24T10:45:40.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.873+0000 7efe36ffd700 1 --2- 172.21.15.94:0/1912732471 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7efe240692a0 0x7efe2406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:45:40.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.873+0000 7efe36ffd700 1 --2- 172.21.15.94:0/1912732471 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7efe240692a0 0x7efe2406b750 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7efe28013740 tx=0x7efe28009e30 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:45:40.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:40.877+0000 7efe2ffff700 1 -- 172.21.15.94:0/1912732471 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7efe20052fd0 con 0x7efe380fff00 2024-03-24T10:45:41.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.094+0000 7efe3d466700 1 -- 172.21.15.94:0/1912732471 --> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efe380f8f70 con 0x7efe240692a0 2024-03-24T10:45:41.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.095+0000 7efe2ffff700 1 -- 172.21.15.94:0/1912732471 <== mgr.14232 v2:172.21.15.94:6800/2184357820 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7efe380f8f70 con 0x7efe240692a0 2024-03-24T10:45:41.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.097+0000 7efe3d466700 1 -- 172.21.15.94:0/1912732471 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7efe240692a0 msgr2=0x7efe2406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:41.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.097+0000 7efe3d466700 1 --2- 172.21.15.94:0/1912732471 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7efe240692a0 0x7efe2406b750 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7efe28013740 tx=0x7efe28009e30 comp rx=0 tx=0).stop 2024-03-24T10:45:41.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.097+0000 7efe3d466700 1 -- 172.21.15.94:0/1912732471 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe380fff00 msgr2=0x7efe380fb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:41.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.097+0000 7efe3d466700 1 --2- 172.21.15.94:0/1912732471 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe380fff00 0x7efe380fb8c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7efe2000eb70 tx=0x7efe2000b040 comp rx=0 tx=0).stop 2024-03-24T10:45:41.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.097+0000 7efe3d466700 1 -- 172.21.15.94:0/1912732471 shutdown_connections 2024-03-24T10:45:41.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.097+0000 7efe3d466700 1 --2- 172.21.15.94:0/1912732471 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7efe240692a0 0x7efe2406b750 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:41.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.097+0000 7efe3d466700 1 --2- 172.21.15.94:0/1912732471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe380662e0 0x7efe380fd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:41.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.097+0000 7efe3d466700 1 --2- 172.21.15.94:0/1912732471 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe380fff00 0x7efe380fb8c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:41.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.097+0000 7efe3d466700 1 -- 172.21.15.94:0/1912732471 >> 172.21.15.94:0/1912732471 conn(0x7efe380f7430 msgr2=0x7efe380f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:45:41.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.098+0000 7efe3d466700 1 -- 172.21.15.94:0/1912732471 shutdown_connections 2024-03-24T10:45:41.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.098+0000 7efe3d466700 1 -- 172.21.15.94:0/1912732471 wait complete. 2024-03-24T10:45:41.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.203+0000 7fb82bace700 1 -- 172.21.15.94:0/85874110 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb824066240 msgr2=0x7fb8240ffa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:41.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.203+0000 7fb82bace700 1 --2- 172.21.15.94:0/85874110 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb824066240 0x7fb8240ffa60 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb820004940 tx=0x7fb820009e30 comp rx=0 tx=0).stop 2024-03-24T10:45:41.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.203+0000 7fb82bace700 1 -- 172.21.15.94:0/85874110 shutdown_connections 2024-03-24T10:45:41.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.203+0000 7fb82bace700 1 --2- 172.21.15.94:0/85874110 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8240fffa0 0x7fb824102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:41.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.203+0000 7fb82bace700 1 --2- 172.21.15.94:0/85874110 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb824066240 0x7fb8240ffa60 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:41.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.203+0000 7fb82bace700 1 -- 172.21.15.94:0/85874110 >> 172.21.15.94:0/85874110 conn(0x7fb8240f7430 msgr2=0x7fb8240f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:45:41.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.203+0000 7fb82bace700 1 -- 172.21.15.94:0/85874110 shutdown_connections 2024-03-24T10:45:41.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.203+0000 7fb82bace700 1 -- 172.21.15.94:0/85874110 wait complete. 2024-03-24T10:45:41.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.203+0000 7fb82bace700 1 Processor -- start 2024-03-24T10:45:41.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.204+0000 7fb82bace700 1 -- start start 2024-03-24T10:45:41.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.204+0000 7fb82bace700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb824066240 0x7fb8240fd440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:45:41.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.204+0000 7fb82bace700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8240fffa0 0x7fb8240fb960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:45:41.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.204+0000 7fb82bace700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8240fbef0 con 0x7fb824066240 2024-03-24T10:45:41.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.204+0000 7fb82bace700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8240fc030 con 0x7fb8240fffa0 2024-03-24T10:45:41.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.204+0000 7fb82986a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb824066240 0x7fb8240fd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:45:41.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.204+0000 7fb82986a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb824066240 0x7fb8240fd440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33344/0 (socket says 172.21.15.94:33344) 2024-03-24T10:45:41.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.204+0000 7fb82986a700 1 -- 172.21.15.94:0/3767360262 learned_addr learned my addr 172.21.15.94:0/3767360262 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:45:41.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.204+0000 7fb829069700 1 --2- 172.21.15.94:0/3767360262 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8240fffa0 0x7fb8240fb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:45:41.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.204+0000 7fb82986a700 1 -- 172.21.15.94:0/3767360262 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8240fffa0 msgr2=0x7fb8240fb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:41.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.204+0000 7fb82986a700 1 --2- 172.21.15.94:0/3767360262 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8240fffa0 0x7fb8240fb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:41.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.204+0000 7fb82986a700 1 -- 172.21.15.94:0/3767360262 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb820005040 con 0x7fb824066240 2024-03-24T10:45:41.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.204+0000 7fb82986a700 1 --2- 172.21.15.94:0/3767360262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb824066240 0x7fb8240fd440 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb820000c00 tx=0x7fb820009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:45:41.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.204+0000 7fb81a7fc700 1 -- 172.21.15.94:0/3767360262 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb82000e070 con 0x7fb824066240 2024-03-24T10:45:41.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.204+0000 7fb82bace700 1 -- 172.21.15.94:0/3767360262 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8240fc2b0 con 0x7fb824066240 2024-03-24T10:45:41.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.204+0000 7fb81a7fc700 1 -- 172.21.15.94:0/3767360262 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb82001c400 con 0x7fb824066240 2024-03-24T10:45:41.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.204+0000 7fb81a7fc700 1 -- 172.21.15.94:0/3767360262 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb82000b650 con 0x7fb824066240 2024-03-24T10:45:41.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.205+0000 7fb82bace700 1 -- 172.21.15.94:0/3767360262 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb824115a80 con 0x7fb824066240 2024-03-24T10:45:41.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.205+0000 7fb82bace700 1 -- 172.21.15.94:0/3767360262 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb82405f5f0 con 0x7fb824066240 2024-03-24T10:45:41.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.206+0000 7fb81a7fc700 1 -- 172.21.15.94:0/3767360262 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fb82001c570 con 0x7fb824066240 2024-03-24T10:45:41.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.206+0000 7fb81a7fc700 1 --2- 172.21.15.94:0/3767360262 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fb8100692a0 0x7fb81006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:45:41.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.207+0000 7fb829069700 1 --2- 172.21.15.94:0/3767360262 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fb8100692a0 0x7fb81006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:45:41.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.207+0000 7fb81a7fc700 1 -- 172.21.15.94:0/3767360262 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fb820088870 con 0x7fb824066240 2024-03-24T10:45:41.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.207+0000 7fb829069700 1 --2- 172.21.15.94:0/3767360262 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fb8100692a0 0x7fb81006b750 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb8240fda20 tx=0x7fb81400b040 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:45:41.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.211+0000 7fb81a7fc700 1 -- 172.21.15.94:0/3767360262 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fb820054b50 con 0x7fb824066240 2024-03-24T10:45:41.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.415+0000 7fb82bace700 1 -- 172.21.15.94:0/3767360262 --> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb8240f8f70 con 0x7fb8100692a0 2024-03-24T10:45:41.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.424+0000 7fb81a7fc700 1 -- 172.21.15.94:0/3767360262 <== mgr.14232 v2:172.21.15.94:6800/2184357820 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3410 (secure 0 0 0) 0x7fb8240f8f70 con 0x7fb8100692a0 2024-03-24T10:45:41.424 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T10:45:41.424 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.smithi094 smithi094 *:9093,9094 running (5m) 94s ago 7m 25.2M - 0.25.0 c8568f914cd2 8072d4b65986 2024-03-24T10:45:41.424 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi094 smithi094 running (8m) 94s ago 8m 6660k - 17.2.7-726-g9c4e0c34 837124da804f fb5772af46c5 2024-03-24T10:45:41.424 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi096 smithi096 running (6m) 100s ago 6m 6669k - 17.2.7-726-g9c4e0c34 837124da804f 6087ad4e0734 2024-03-24T10:45:41.424 INFO:teuthology.orchestra.run.smithi094.stdout:grafana.smithi094 smithi094 *:3000 running (5m) 94s ago 6m 83.6M - 9.4.7 954c08fa6188 7b534968fb36 2024-03-24T10:45:41.424 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.efiqte smithi094 running (118s) 94s ago 117s 10.4M - 17.2.7-726-g9c4e0c34 837124da804f f0df0a056c21 2024-03-24T10:45:41.425 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.llefgu smithi094 running (111s) 94s ago 111s 11.9M - 17.2.7-726-g9c4e0c34 837124da804f 52a9771dc8e0 2024-03-24T10:45:41.425 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi096.egbmoq smithi096 running (115s) 100s ago 114s 14.5M - 17.2.7-726-g9c4e0c34 837124da804f 1cfc930dbab0 2024-03-24T10:45:41.425 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi096.jtebhh smithi096 running (108s) 100s ago 108s 13.5M - 17.2.7-726-g9c4e0c34 837124da804f 4ff3451a6f6e 2024-03-24T10:45:41.425 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi094.yaethu smithi094 *:9283 running (9m) 94s ago 9m 481M - 17.2.7-726-g9c4e0c34 837124da804f 327d527697e0 2024-03-24T10:45:41.425 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi096.gfonnu smithi096 *:8443,9283 running (6m) 100s ago 6m 433M - 17.2.7-726-g9c4e0c34 837124da804f efd3f3b01cb6 2024-03-24T10:45:41.425 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi094 smithi094 running (9m) 94s ago 9m 43.3M 2048M 17.2.7-726-g9c4e0c34 837124da804f 76584fceb32b 2024-03-24T10:45:41.425 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi096 smithi096 running (6m) 100s ago 6m 36.3M 2048M 17.2.7-726-g9c4e0c34 837124da804f 710030b222cb 2024-03-24T10:45:41.425 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi094 smithi094 *:9100 running (8m) 94s ago 8m 15.4M - 1.5.0 0da6a335fe13 59895363f7e6 2024-03-24T10:45:41.425 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi096 smithi096 *:9100 running (6m) 100s ago 6m 15.3M - 1.5.0 0da6a335fe13 fc629282835f 2024-03-24T10:45:41.425 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (5m) 94s ago 5m 47.2M 4288M 17.2.7-726-g9c4e0c34 837124da804f b664fd3557bd 2024-03-24T10:45:41.425 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi094 running (5m) 94s ago 5m 46.3M 4288M 17.2.7-726-g9c4e0c34 837124da804f 4e737a80d733 2024-03-24T10:45:41.425 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi094 running (4m) 94s ago 4m 47.2M 4288M 17.2.7-726-g9c4e0c34 837124da804f cf0aefd20c76 2024-03-24T10:45:41.425 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi096 running (4m) 100s ago 4m 44.7M 5312M 17.2.7-726-g9c4e0c34 837124da804f f4650d0f6241 2024-03-24T10:45:41.426 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi096 running (3m) 100s ago 3m 44.6M 5312M 17.2.7-726-g9c4e0c34 837124da804f a63d7c9e705f 2024-03-24T10:45:41.426 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi096 running (3m) 100s ago 3m 41.7M 5312M 17.2.7-726-g9c4e0c34 837124da804f f56fad2959c3 2024-03-24T10:45:41.426 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.smithi094 smithi094 *:9095 running (5m) 94s ago 6m 57.0M - 2.43.0 a07b618ecd1d 023eb0a49108 2024-03-24T10:45:41.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.427+0000 7fb82bace700 1 -- 172.21.15.94:0/3767360262 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fb8100692a0 msgr2=0x7fb81006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:41.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.427+0000 7fb82bace700 1 --2- 172.21.15.94:0/3767360262 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fb8100692a0 0x7fb81006b750 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb8240fda20 tx=0x7fb81400b040 comp rx=0 tx=0).stop 2024-03-24T10:45:41.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.427+0000 7fb82bace700 1 -- 172.21.15.94:0/3767360262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb824066240 msgr2=0x7fb8240fd440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:41.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.427+0000 7fb82bace700 1 --2- 172.21.15.94:0/3767360262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb824066240 0x7fb8240fd440 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb820000c00 tx=0x7fb820009e30 comp rx=0 tx=0).stop 2024-03-24T10:45:41.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.427+0000 7fb82bace700 1 -- 172.21.15.94:0/3767360262 shutdown_connections 2024-03-24T10:45:41.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.427+0000 7fb82bace700 1 --2- 172.21.15.94:0/3767360262 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fb8100692a0 0x7fb81006b750 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:41.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.427+0000 7fb82bace700 1 --2- 172.21.15.94:0/3767360262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb824066240 0x7fb8240fd440 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:41.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.427+0000 7fb82bace700 1 --2- 172.21.15.94:0/3767360262 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8240fffa0 0x7fb8240fb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:41.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.427+0000 7fb82bace700 1 -- 172.21.15.94:0/3767360262 >> 172.21.15.94:0/3767360262 conn(0x7fb8240f7430 msgr2=0x7fb8240f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:45:41.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.427+0000 7fb82bace700 1 -- 172.21.15.94:0/3767360262 shutdown_connections 2024-03-24T10:45:41.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.427+0000 7fb82bace700 1 -- 172.21.15.94:0/3767360262 wait complete. 2024-03-24T10:45:41.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.527+0000 7fbaba763700 1 -- 172.21.15.94:0/610255348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbab40fcf00 msgr2=0x7fbab40fd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:41.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.527+0000 7fbaba763700 1 --2- 172.21.15.94:0/610255348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbab40fcf00 0x7fbab40fd310 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4004940 tx=0x7fbaa4009e30 comp rx=0 tx=0).stop 2024-03-24T10:45:41.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.527+0000 7fbaba763700 1 -- 172.21.15.94:0/610255348 shutdown_connections 2024-03-24T10:45:41.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.527+0000 7fbaba763700 1 --2- 172.21.15.94:0/610255348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbab40fe070 0x7fbab40fe4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:41.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.527+0000 7fbaba763700 1 --2- 172.21.15.94:0/610255348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbab40fcf00 0x7fbab40fd310 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:41.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.527+0000 7fbaba763700 1 -- 172.21.15.94:0/610255348 >> 172.21.15.94:0/610255348 conn(0x7fbab40f85a0 msgr2=0x7fbab40fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:45:41.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.527+0000 7fbaba763700 1 -- 172.21.15.94:0/610255348 shutdown_connections 2024-03-24T10:45:41.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.527+0000 7fbaba763700 1 -- 172.21.15.94:0/610255348 wait complete. 2024-03-24T10:45:41.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.527+0000 7fbaba763700 1 Processor -- start 2024-03-24T10:45:41.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.528+0000 7fbaba763700 1 -- start start 2024-03-24T10:45:41.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.528+0000 7fbaba763700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbab40fcf00 0x7fbab410fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:45:41.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.528+0000 7fbaba763700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbab40fe070 0x7fbab4110110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:45:41.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.528+0000 7fbaba763700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab4110710 con 0x7fbab40fe070 2024-03-24T10:45:41.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.528+0000 7fbaba763700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab4110f60 con 0x7fbab40fcf00 2024-03-24T10:45:41.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.528+0000 7fbab37fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbab40fe070 0x7fbab4110110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:45:41.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.528+0000 7fbab3fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbab40fcf00 0x7fbab410fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:45:41.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.528+0000 7fbab37fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbab40fe070 0x7fbab4110110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33360/0 (socket says 172.21.15.94:33360) 2024-03-24T10:45:41.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.528+0000 7fbab37fe700 1 -- 172.21.15.94:0/2246985655 learned_addr learned my addr 172.21.15.94:0/2246985655 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:45:41.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.528+0000 7fbab37fe700 1 -- 172.21.15.94:0/2246985655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbab40fcf00 msgr2=0x7fbab410fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:41.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.528+0000 7fbab37fe700 1 --2- 172.21.15.94:0/2246985655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbab40fcf00 0x7fbab410fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:41.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.528+0000 7fbab37fe700 1 -- 172.21.15.94:0/2246985655 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaa4005040 con 0x7fbab40fe070 2024-03-24T10:45:41.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.528+0000 7fbab3fff700 1 --2- 172.21.15.94:0/2246985655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbab40fcf00 0x7fbab410fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T10:45:41.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.528+0000 7fbab37fe700 1 --2- 172.21.15.94:0/2246985655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbab40fe070 0x7fbab4110110 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fbaa8002a80 tx=0x7fbaa80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:45:41.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.528+0000 7fbab0ff9700 1 -- 172.21.15.94:0/2246985655 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbaa8007cc0 con 0x7fbab40fe070 2024-03-24T10:45:41.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.529+0000 7fbab0ff9700 1 -- 172.21.15.94:0/2246985655 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbaa800f040 con 0x7fbab40fe070 2024-03-24T10:45:41.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.529+0000 7fbaba763700 1 -- 172.21.15.94:0/2246985655 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbab4111240 con 0x7fbab40fe070 2024-03-24T10:45:41.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.529+0000 7fbab0ff9700 1 -- 172.21.15.94:0/2246985655 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbaa800dac0 con 0x7fbab40fe070 2024-03-24T10:45:41.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.529+0000 7fbaba763700 1 -- 172.21.15.94:0/2246985655 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbab4111780 con 0x7fbab40fe070 2024-03-24T10:45:41.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.529+0000 7fbaba763700 1 -- 172.21.15.94:0/2246985655 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbab4101a00 con 0x7fbab40fe070 2024-03-24T10:45:41.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.530+0000 7fbab0ff9700 1 -- 172.21.15.94:0/2246985655 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fbaa800dcc0 con 0x7fbab40fe070 2024-03-24T10:45:41.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.530+0000 7fbab0ff9700 1 --2- 172.21.15.94:0/2246985655 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fba9c0692a0 0x7fba9c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:45:41.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.531+0000 7fbab3fff700 1 --2- 172.21.15.94:0/2246985655 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fba9c0692a0 0x7fba9c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:45:41.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.531+0000 7fbab0ff9700 1 -- 172.21.15.94:0/2246985655 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fbaa8086e40 con 0x7fbab40fe070 2024-03-24T10:45:41.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.531+0000 7fbab3fff700 1 --2- 172.21.15.94:0/2246985655 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fba9c0692a0 0x7fba9c06b750 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbaa400b3b0 tx=0x7fbaa4009e30 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:45:41.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.535+0000 7fbab0ff9700 1 -- 172.21.15.94:0/2246985655 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fbaa8056b30 con 0x7fbab40fe070 2024-03-24T10:45:41.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.827+0000 7fbaba763700 1 -- 172.21.15.94:0/2246985655 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbab405f5f0 con 0x7fbab40fe070 2024-03-24T10:45:41.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.827+0000 7fbab0ff9700 1 -- 172.21.15.94:0/2246985655 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7fbaa80566c0 con 0x7fbab40fe070 2024-03-24T10:45:41.827 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-03-24T10:45:41.827 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": { 2024-03-24T10:45:41.827 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-24T10:45:41.827 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:45:41.827 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": { 2024-03-24T10:45:41.827 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-24T10:45:41.828 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:45:41.828 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": { 2024-03-24T10:45:41.828 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-24T10:45:41.828 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:45:41.828 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-03-24T10:45:41.828 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-24T10:45:41.828 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:45:41.828 INFO:teuthology.orchestra.run.smithi094.stdout: "overall": { 2024-03-24T10:45:41.828 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 14 2024-03-24T10:45:41.828 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-03-24T10:45:41.828 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-03-24T10:45:41.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.830+0000 7fbaba763700 1 -- 172.21.15.94:0/2246985655 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fba9c0692a0 msgr2=0x7fba9c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:41.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.830+0000 7fbaba763700 1 --2- 172.21.15.94:0/2246985655 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fba9c0692a0 0x7fba9c06b750 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbaa400b3b0 tx=0x7fbaa4009e30 comp rx=0 tx=0).stop 2024-03-24T10:45:41.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.830+0000 7fbaba763700 1 -- 172.21.15.94:0/2246985655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbab40fe070 msgr2=0x7fbab4110110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:41.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.830+0000 7fbaba763700 1 --2- 172.21.15.94:0/2246985655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbab40fe070 0x7fbab4110110 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fbaa8002a80 tx=0x7fbaa80046b0 comp rx=0 tx=0).stop 2024-03-24T10:45:41.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.830+0000 7fbaba763700 1 -- 172.21.15.94:0/2246985655 shutdown_connections 2024-03-24T10:45:41.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.830+0000 7fbaba763700 1 --2- 172.21.15.94:0/2246985655 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fba9c0692a0 0x7fba9c06b750 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:41.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.830+0000 7fbaba763700 1 --2- 172.21.15.94:0/2246985655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbab40fcf00 0x7fbab410fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:41.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.830+0000 7fbaba763700 1 --2- 172.21.15.94:0/2246985655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbab40fe070 0x7fbab4110110 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:41.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.830+0000 7fbaba763700 1 -- 172.21.15.94:0/2246985655 >> 172.21.15.94:0/2246985655 conn(0x7fbab40f85a0 msgr2=0x7fbab40f9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:45:41.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.830+0000 7fbaba763700 1 -- 172.21.15.94:0/2246985655 shutdown_connections 2024-03-24T10:45:41.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.830+0000 7fbaba763700 1 -- 172.21.15.94:0/2246985655 wait complete. 2024-03-24T10:45:41.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.931+0000 7f82eff9a700 1 -- 172.21.15.94:0/1849276218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82e80fbcd0 msgr2=0x7f82e80fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:41.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.931+0000 7f82eff9a700 1 --2- 172.21.15.94:0/1849276218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82e80fbcd0 0x7f82e80fc0e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f82d8004940 tx=0x7f82d8009e30 comp rx=0 tx=0).stop 2024-03-24T10:45:41.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.931+0000 7f82eff9a700 1 -- 172.21.15.94:0/1849276218 shutdown_connections 2024-03-24T10:45:41.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.931+0000 7f82eff9a700 1 --2- 172.21.15.94:0/1849276218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82e80fcf80 0x7f82e80fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:41.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.931+0000 7f82eff9a700 1 --2- 172.21.15.94:0/1849276218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82e80fbcd0 0x7f82e80fc0e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:41.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.931+0000 7f82eff9a700 1 -- 172.21.15.94:0/1849276218 >> 172.21.15.94:0/1849276218 conn(0x7f82e80f7450 msgr2=0x7f82e80f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:45:41.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.931+0000 7f82eff9a700 1 -- 172.21.15.94:0/1849276218 shutdown_connections 2024-03-24T10:45:41.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.931+0000 7f82eff9a700 1 -- 172.21.15.94:0/1849276218 wait complete. 2024-03-24T10:45:41.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.932+0000 7f82eff9a700 1 Processor -- start 2024-03-24T10:45:41.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.932+0000 7f82eff9a700 1 -- start start 2024-03-24T10:45:41.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.932+0000 7f82eff9a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82e80fbcd0 0x7f82e810d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:45:41.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.932+0000 7f82eff9a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82e80fcf80 0x7f82e810dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:45:41.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.932+0000 7f82eff9a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e810e3e0 con 0x7f82e80fcf80 2024-03-24T10:45:41.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.932+0000 7f82eff9a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e810e550 con 0x7f82e80fbcd0 2024-03-24T10:45:41.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.932+0000 7f82edd36700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82e80fbcd0 0x7f82e810d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:45:41.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.932+0000 7f82ed535700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82e80fcf80 0x7f82e810dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:45:41.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.932+0000 7f82edd36700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82e80fbcd0 0x7f82e810d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.94:36210/0 (socket says 172.21.15.94:36210) 2024-03-24T10:45:41.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.932+0000 7f82edd36700 1 -- 172.21.15.94:0/2090466418 learned_addr learned my addr 172.21.15.94:0/2090466418 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:45:41.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.932+0000 7f82edd36700 1 -- 172.21.15.94:0/2090466418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82e80fcf80 msgr2=0x7f82e810dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:41.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.932+0000 7f82edd36700 1 --2- 172.21.15.94:0/2090466418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82e80fcf80 0x7f82e810dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:41.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.932+0000 7f82edd36700 1 -- 172.21.15.94:0/2090466418 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d8005040 con 0x7f82e80fbcd0 2024-03-24T10:45:41.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.932+0000 7f82ed535700 1 --2- 172.21.15.94:0/2090466418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82e80fcf80 0x7f82e810dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T10:45:41.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.933+0000 7f82edd36700 1 --2- 172.21.15.94:0/2090466418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82e80fbcd0 0x7f82e810d8a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f82d8013870 tx=0x7f82d8009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:45:41.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.933+0000 7f82de7fc700 1 -- 172.21.15.94:0/2090466418 <== mon.1 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82d8021070 con 0x7f82e80fbcd0 2024-03-24T10:45:41.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.933+0000 7f82de7fc700 1 -- 172.21.15.94:0/2090466418 <== mon.1 v2:172.21.15.96:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f82d8009430 con 0x7f82e80fbcd0 2024-03-24T10:45:41.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.933+0000 7f82de7fc700 1 -- 172.21.15.94:0/2090466418 <== mon.1 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82d800e070 con 0x7f82e80fbcd0 2024-03-24T10:45:41.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.933+0000 7f82eff9a700 1 -- 172.21.15.94:0/2090466418 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82e8112f50 con 0x7f82e80fbcd0 2024-03-24T10:45:41.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.933+0000 7f82eff9a700 1 -- 172.21.15.94:0/2090466418 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82e8113400 con 0x7f82e80fbcd0 2024-03-24T10:45:41.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.934+0000 7f82eff9a700 1 -- 172.21.15.94:0/2090466418 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82e8101b10 con 0x7f82e80fbcd0 2024-03-24T10:45:41.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.934+0000 7f82de7fc700 1 -- 172.21.15.94:0/2090466418 <== mon.1 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f82d801c750 con 0x7f82e80fbcd0 2024-03-24T10:45:41.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.935+0000 7f82de7fc700 1 --2- 172.21.15.94:0/2090466418 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f82d40692a0 0x7f82d406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:45:41.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.935+0000 7f82de7fc700 1 -- 172.21.15.94:0/2090466418 <== mon.1 v2:172.21.15.96:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f82d80883c0 con 0x7f82e80fbcd0 2024-03-24T10:45:41.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.935+0000 7f82ed535700 1 --2- 172.21.15.94:0/2090466418 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f82d40692a0 0x7f82d406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:45:41.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.935+0000 7f82ed535700 1 --2- 172.21.15.94:0/2090466418 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f82d40692a0 0x7f82d406b750 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f82e4009b80 tx=0x7f82e400b040 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:45:41.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:41.939+0000 7f82de7fc700 1 -- 172.21.15.94:0/2090466418 <== mon.1 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f82d80581d0 con 0x7f82e80fbcd0 2024-03-24T10:45:42.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.182+0000 7f82eff9a700 1 -- 172.21.15.94:0/2090466418 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f82e804c710 con 0x7f82e80fbcd0 2024-03-24T10:45:42.182 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:42 smithi094 ceph-mon[29446]: from='client.14636 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:45:42.182 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:42 smithi094 ceph-mon[29446]: pgmap v234: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.5 MiB/s rd, 6.1 MiB/s wr, 783 op/s 2024-03-24T10:45:42.182 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:42 smithi094 ceph-mon[29446]: from='client.14640 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:45:42.182 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:42 smithi094 ceph-mon[29446]: from='client.14644 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:45:42.182 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:42 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/2246985655' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:45:42.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.182+0000 7f82de7fc700 1 -- 172.21.15.94:0/2090466418 <== mon.1 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1833 (secure 0 0 0) 0x7f82d8004490 con 0x7f82e80fbcd0 2024-03-24T10:45:42.183 INFO:teuthology.orchestra.run.smithi094.stdout:e18 2024-03-24T10:45:42.183 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T10:45:42.183 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:45:42.183 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-03-24T10:45:42.183 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:45:42.183 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-03-24T10:45:42.183 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-03-24T10:45:42.184 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 18 2024-03-24T10:45:42.184 INFO:teuthology.orchestra.run.smithi094.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-24T10:45:42.184 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-03-24T10:43:39.435738+0000 2024-03-24T10:45:42.184 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-03-24T10:43:58.133901+0000 2024-03-24T10:45:42.184 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-03-24T10:45:42.184 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-03-24T10:45:42.184 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 60 2024-03-24T10:45:42.184 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-03-24T10:45:42.184 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-03-24T10:45:42.184 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-03-24T10:45:42.184 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-03-24T10:45:42.184 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 46 2024-03-24T10:45:42.184 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:45:42.184 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-03-24T10:45:42.184 INFO:teuthology.orchestra.run.smithi094.stdout:in 0 2024-03-24T10:45:42.185 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=24251} 2024-03-24T10:45:42.185 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-03-24T10:45:42.185 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-03-24T10:45:42.185 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-03-24T10:45:42.185 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-03-24T10:45:42.185 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-03-24T10:45:42.185 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-03-24T10:45:42.185 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-03-24T10:45:42.185 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 1 2024-03-24T10:45:42.185 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi096.egbmoq{0:24251} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.96:6824/1539752208,v1:172.21.15.96:6825/1539752208] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:45:42.185 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.llefgu{0:14500} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.94:6828/126762980,v1:172.21.15.94:6829/126762980] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:45:42.185 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:45:42.185 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:45:42.186 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-03-24T10:45:42.186 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:45:42.186 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.efiqte{-1:14512} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.94:6826/1636528860,v1:172.21.15.94:6827/1636528860] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:45:42.186 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi096.jtebhh{-1:24261} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.96:6826/1321087930,v1:172.21.15.96:6827/1321087930] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:45:42.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.184+0000 7f82eff9a700 1 -- 172.21.15.94:0/2090466418 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f82d40692a0 msgr2=0x7f82d406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:42.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.184+0000 7f82eff9a700 1 --2- 172.21.15.94:0/2090466418 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f82d40692a0 0x7f82d406b750 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f82e4009b80 tx=0x7f82e400b040 comp rx=0 tx=0).stop 2024-03-24T10:45:42.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.184+0000 7f82eff9a700 1 -- 172.21.15.94:0/2090466418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82e80fbcd0 msgr2=0x7f82e810d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:42.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.184+0000 7f82eff9a700 1 --2- 172.21.15.94:0/2090466418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82e80fbcd0 0x7f82e810d8a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f82d8013870 tx=0x7f82d8009e30 comp rx=0 tx=0).stop 2024-03-24T10:45:42.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.184+0000 7f82eff9a700 1 -- 172.21.15.94:0/2090466418 shutdown_connections 2024-03-24T10:45:42.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.184+0000 7f82eff9a700 1 --2- 172.21.15.94:0/2090466418 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f82d40692a0 0x7f82d406b750 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:42.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.184+0000 7f82eff9a700 1 --2- 172.21.15.94:0/2090466418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82e80fbcd0 0x7f82e810d8a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:42.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.184+0000 7f82eff9a700 1 --2- 172.21.15.94:0/2090466418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82e80fcf80 0x7f82e810dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:42.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.185+0000 7f82eff9a700 1 -- 172.21.15.94:0/2090466418 >> 172.21.15.94:0/2090466418 conn(0x7f82e80f7450 msgr2=0x7f82e8066760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:45:42.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.185+0000 7f82eff9a700 1 -- 172.21.15.94:0/2090466418 shutdown_connections 2024-03-24T10:45:42.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.185+0000 7f82eff9a700 1 -- 172.21.15.94:0/2090466418 wait complete. 2024-03-24T10:45:42.187 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 18 2024-03-24T10:45:42.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.304+0000 7f5516c1c700 1 -- 172.21.15.94:0/2833515526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55100fcee0 msgr2=0x7f55100fd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:42.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.304+0000 7f5516c1c700 1 --2- 172.21.15.94:0/2833515526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55100fcee0 0x7f55100fd310 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5504004940 tx=0x7f5504009e30 comp rx=0 tx=0).stop 2024-03-24T10:45:42.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.304+0000 7f5516c1c700 1 -- 172.21.15.94:0/2833515526 shutdown_connections 2024-03-24T10:45:42.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.304+0000 7f5516c1c700 1 --2- 172.21.15.94:0/2833515526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55100fe0d0 0x7f55100fe540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:42.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.304+0000 7f5516c1c700 1 --2- 172.21.15.94:0/2833515526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55100fcee0 0x7f55100fd310 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:42.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.304+0000 7f5516c1c700 1 -- 172.21.15.94:0/2833515526 >> 172.21.15.94:0/2833515526 conn(0x7f55100f8560 msgr2=0x7f55100fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:45:42.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.304+0000 7f5516c1c700 1 -- 172.21.15.94:0/2833515526 shutdown_connections 2024-03-24T10:45:42.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.304+0000 7f5516c1c700 1 -- 172.21.15.94:0/2833515526 wait complete. 2024-03-24T10:45:42.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.305+0000 7f5516c1c700 1 Processor -- start 2024-03-24T10:45:42.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.305+0000 7f5516c1c700 1 -- start start 2024-03-24T10:45:42.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.305+0000 7f5516c1c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55100fcee0 0x7f5510071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:45:42.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.305+0000 7f5516c1c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55100fe0d0 0x7f5510070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:45:42.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.305+0000 7f5516c1c700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55100708c0 con 0x7f55100fcee0 2024-03-24T10:45:42.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.305+0000 7f5516c1c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5510070a30 con 0x7f55100fe0d0 2024-03-24T10:45:42.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.305+0000 7f55149b8700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55100fcee0 0x7f5510071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:45:42.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.305+0000 7f55149b8700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55100fcee0 0x7f5510071e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33386/0 (socket says 172.21.15.94:33386) 2024-03-24T10:45:42.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.305+0000 7f55149b8700 1 -- 172.21.15.94:0/3508514165 learned_addr learned my addr 172.21.15.94:0/3508514165 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:45:42.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.305+0000 7f550ffff700 1 --2- 172.21.15.94:0/3508514165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55100fe0d0 0x7f5510070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:45:42.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.306+0000 7f55149b8700 1 -- 172.21.15.94:0/3508514165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55100fe0d0 msgr2=0x7f5510070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:42.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.306+0000 7f55149b8700 1 --2- 172.21.15.94:0/3508514165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55100fe0d0 0x7f5510070310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:42.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.306+0000 7f55149b8700 1 -- 172.21.15.94:0/3508514165 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5504005040 con 0x7f55100fcee0 2024-03-24T10:45:42.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.306+0000 7f55149b8700 1 --2- 172.21.15.94:0/3508514165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55100fcee0 0x7f5510071e40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5504000c00 tx=0x7f5504009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:45:42.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.306+0000 7f550d7fa700 1 -- 172.21.15.94:0/3508514165 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f550400e030 con 0x7f55100fcee0 2024-03-24T10:45:42.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.306+0000 7f550d7fa700 1 -- 172.21.15.94:0/3508514165 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5504009430 con 0x7f55100fcee0 2024-03-24T10:45:42.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.306+0000 7f5516c1c700 1 -- 172.21.15.94:0/3508514165 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5510070cb0 con 0x7f55100fcee0 2024-03-24T10:45:42.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.306+0000 7f550d7fa700 1 -- 172.21.15.94:0/3508514165 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f550400b410 con 0x7f55100fcee0 2024-03-24T10:45:42.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.306+0000 7f5516c1c700 1 -- 172.21.15.94:0/3508514165 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5510115a00 con 0x7f55100fcee0 2024-03-24T10:45:42.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.307+0000 7f5516c1c700 1 -- 172.21.15.94:0/3508514165 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f551005f5f0 con 0x7f55100fcee0 2024-03-24T10:45:42.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.308+0000 7f550d7fa700 1 -- 172.21.15.94:0/3508514165 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f550401c770 con 0x7f55100fcee0 2024-03-24T10:45:42.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.308+0000 7f550d7fa700 1 --2- 172.21.15.94:0/3508514165 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f55000692a0 0x7f550006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:45:42.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.308+0000 7f550d7fa700 1 -- 172.21.15.94:0/3508514165 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f5504088640 con 0x7f55100fcee0 2024-03-24T10:45:42.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.308+0000 7f550ffff700 1 --2- 172.21.15.94:0/3508514165 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f55000692a0 0x7f550006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:45:42.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.309+0000 7f550ffff700 1 --2- 172.21.15.94:0/3508514165 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f55000692a0 0x7f550006b750 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f54fc004fd0 tx=0x7f54fc00b040 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:45:42.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.317+0000 7f550d7fa700 1 -- 172.21.15.94:0/3508514165 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f55040549c0 con 0x7f55100fcee0 2024-03-24T10:45:42.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:42 smithi096 ceph-mon[38097]: from='client.14636 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:45:42.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:42 smithi096 ceph-mon[38097]: pgmap v234: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.5 MiB/s rd, 6.1 MiB/s wr, 783 op/s 2024-03-24T10:45:42.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:42 smithi096 ceph-mon[38097]: from='client.14640 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:45:42.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:42 smithi096 ceph-mon[38097]: from='client.14644 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:45:42.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:42 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/2246985655' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:45:42.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.526+0000 7f5516c1c700 1 -- 172.21.15.94:0/3508514165 --> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f55101023e0 con 0x7f55000692a0 2024-03-24T10:45:42.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.528+0000 7f550d7fa700 1 -- 172.21.15.94:0/3508514165 <== mgr.14232 v2:172.21.15.94:6800/2184357820 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7f55101023e0 con 0x7f55000692a0 2024-03-24T10:45:42.528 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-03-24T10:45:42.528 INFO:teuthology.orchestra.run.smithi094.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919", 2024-03-24T10:45:42.528 INFO:teuthology.orchestra.run.smithi094.stdout: "in_progress": true, 2024-03-24T10:45:42.528 INFO:teuthology.orchestra.run.smithi094.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T10:45:42.528 INFO:teuthology.orchestra.run.smithi094.stdout: "services_complete": [], 2024-03-24T10:45:42.528 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": "0/21 daemons upgraded", 2024-03-24T10:45:42.528 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 image on host smithi096", 2024-03-24T10:45:42.528 INFO:teuthology.orchestra.run.smithi094.stdout: "is_paused": false 2024-03-24T10:45:42.528 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-03-24T10:45:42.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.530+0000 7f5516c1c700 1 -- 172.21.15.94:0/3508514165 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f55000692a0 msgr2=0x7f550006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:42.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.530+0000 7f5516c1c700 1 --2- 172.21.15.94:0/3508514165 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f55000692a0 0x7f550006b750 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f54fc004fd0 tx=0x7f54fc00b040 comp rx=0 tx=0).stop 2024-03-24T10:45:42.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.530+0000 7f5516c1c700 1 -- 172.21.15.94:0/3508514165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55100fcee0 msgr2=0x7f5510071e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:42.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.530+0000 7f5516c1c700 1 --2- 172.21.15.94:0/3508514165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55100fcee0 0x7f5510071e40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5504000c00 tx=0x7f5504009e30 comp rx=0 tx=0).stop 2024-03-24T10:45:42.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.530+0000 7f5516c1c700 1 -- 172.21.15.94:0/3508514165 shutdown_connections 2024-03-24T10:45:42.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.530+0000 7f5516c1c700 1 --2- 172.21.15.94:0/3508514165 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f55000692a0 0x7f550006b750 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:42.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.530+0000 7f5516c1c700 1 --2- 172.21.15.94:0/3508514165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55100fcee0 0x7f5510071e40 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:42.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.530+0000 7f5516c1c700 1 --2- 172.21.15.94:0/3508514165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55100fe0d0 0x7f5510070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:42.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.530+0000 7f5516c1c700 1 -- 172.21.15.94:0/3508514165 >> 172.21.15.94:0/3508514165 conn(0x7f55100f8560 msgr2=0x7f55100f9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:45:42.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.530+0000 7f5516c1c700 1 -- 172.21.15.94:0/3508514165 shutdown_connections 2024-03-24T10:45:42.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.530+0000 7f5516c1c700 1 -- 172.21.15.94:0/3508514165 wait complete. 2024-03-24T10:45:42.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.630+0000 7fce21b9e700 1 -- 172.21.15.94:0/3269699613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce1c0fcf80 msgr2=0x7fce1c0fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:42.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.630+0000 7fce21b9e700 1 --2- 172.21.15.94:0/3269699613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce1c0fcf80 0x7fce1c0fd3f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fce04004940 tx=0x7fce04009e30 comp rx=0 tx=0).stop 2024-03-24T10:45:42.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.630+0000 7fce21b9e700 1 -- 172.21.15.94:0/3269699613 shutdown_connections 2024-03-24T10:45:42.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.630+0000 7fce21b9e700 1 --2- 172.21.15.94:0/3269699613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce1c0fcf80 0x7fce1c0fd3f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:42.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.630+0000 7fce21b9e700 1 --2- 172.21.15.94:0/3269699613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce1c0fbcd0 0x7fce1c0fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:42.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.630+0000 7fce21b9e700 1 -- 172.21.15.94:0/3269699613 >> 172.21.15.94:0/3269699613 conn(0x7fce1c0f7430 msgr2=0x7fce1c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:45:42.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.630+0000 7fce21b9e700 1 -- 172.21.15.94:0/3269699613 shutdown_connections 2024-03-24T10:45:42.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.630+0000 7fce21b9e700 1 -- 172.21.15.94:0/3269699613 wait complete. 2024-03-24T10:45:42.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.630+0000 7fce21b9e700 1 Processor -- start 2024-03-24T10:45:42.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.631+0000 7fce21b9e700 1 -- start start 2024-03-24T10:45:42.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.631+0000 7fce21b9e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce1c0fbcd0 0x7fce1c0ffdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:45:42.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.631+0000 7fce21b9e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce1c0fcf80 0x7fce1c0fe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:45:42.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.631+0000 7fce21b9e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1c0fe870 con 0x7fce1c0fbcd0 2024-03-24T10:45:42.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.631+0000 7fce21b9e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1c0fe9e0 con 0x7fce1c0fcf80 2024-03-24T10:45:42.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.631+0000 7fce1b7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce1c0fbcd0 0x7fce1c0ffdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:45:42.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.631+0000 7fce1b7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce1c0fbcd0 0x7fce1c0ffdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33404/0 (socket says 172.21.15.94:33404) 2024-03-24T10:45:42.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.631+0000 7fce1b7fe700 1 -- 172.21.15.94:0/2159911646 learned_addr learned my addr 172.21.15.94:0/2159911646 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:45:42.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.631+0000 7fce1affd700 1 --2- 172.21.15.94:0/2159911646 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce1c0fcf80 0x7fce1c0fe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:45:42.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.631+0000 7fce1b7fe700 1 -- 172.21.15.94:0/2159911646 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce1c0fcf80 msgr2=0x7fce1c0fe2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:42.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.631+0000 7fce1b7fe700 1 --2- 172.21.15.94:0/2159911646 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce1c0fcf80 0x7fce1c0fe2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:42.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.631+0000 7fce1b7fe700 1 -- 172.21.15.94:0/2159911646 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce04005040 con 0x7fce1c0fbcd0 2024-03-24T10:45:42.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.631+0000 7fce1b7fe700 1 --2- 172.21.15.94:0/2159911646 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce1c0fbcd0 0x7fce1c0ffdf0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fce14001d80 tx=0x7fce14009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:45:42.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.632+0000 7fce13fff700 1 -- 172.21.15.94:0/2159911646 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce14007d10 con 0x7fce1c0fbcd0 2024-03-24T10:45:42.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.632+0000 7fce13fff700 1 -- 172.21.15.94:0/2159911646 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fce14001e80 con 0x7fce1c0fbcd0 2024-03-24T10:45:42.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.632+0000 7fce21b9e700 1 -- 172.21.15.94:0/2159911646 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce1c0fecc0 con 0x7fce1c0fbcd0 2024-03-24T10:45:42.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.632+0000 7fce13fff700 1 -- 172.21.15.94:0/2159911646 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce14014640 con 0x7fce1c0fbcd0 2024-03-24T10:45:42.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.632+0000 7fce21b9e700 1 -- 172.21.15.94:0/2159911646 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce1c115a40 con 0x7fce1c0fbcd0 2024-03-24T10:45:42.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.633+0000 7fce21b9e700 1 -- 172.21.15.94:0/2159911646 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce1c05f5f0 con 0x7fce1c0fbcd0 2024-03-24T10:45:42.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.633+0000 7fce13fff700 1 -- 172.21.15.94:0/2159911646 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fce1400e960 con 0x7fce1c0fbcd0 2024-03-24T10:45:42.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.633+0000 7fce13fff700 1 --2- 172.21.15.94:0/2159911646 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fce080692a0 0x7fce0806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:45:42.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.634+0000 7fce13fff700 1 -- 172.21.15.94:0/2159911646 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fce14086ea0 con 0x7fce1c0fbcd0 2024-03-24T10:45:42.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.634+0000 7fce1affd700 1 --2- 172.21.15.94:0/2159911646 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fce080692a0 0x7fce0806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:45:42.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.634+0000 7fce1affd700 1 --2- 172.21.15.94:0/2159911646 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fce080692a0 0x7fce0806b750 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fce0400b970 tx=0x7fce04009e30 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:45:42.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.638+0000 7fce13fff700 1 -- 172.21.15.94:0/2159911646 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fce1404e190 con 0x7fce1c0fbcd0 2024-03-24T10:45:42.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.935+0000 7fce21b9e700 1 -- 172.21.15.94:0/2159911646 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fce1c02b2a0 con 0x7fce1c0fbcd0 2024-03-24T10:45:42.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.935+0000 7fce13fff700 1 -- 172.21.15.94:0/2159911646 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fce14053230 con 0x7fce1c0fbcd0 2024-03-24T10:45:42.935 INFO:teuthology.orchestra.run.smithi094.stdout:HEALTH_OK 2024-03-24T10:45:42.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.937+0000 7fce21b9e700 1 -- 172.21.15.94:0/2159911646 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fce080692a0 msgr2=0x7fce0806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:42.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.937+0000 7fce21b9e700 1 --2- 172.21.15.94:0/2159911646 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fce080692a0 0x7fce0806b750 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fce0400b970 tx=0x7fce04009e30 comp rx=0 tx=0).stop 2024-03-24T10:45:42.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.937+0000 7fce21b9e700 1 -- 172.21.15.94:0/2159911646 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce1c0fbcd0 msgr2=0x7fce1c0ffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:45:42.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.937+0000 7fce21b9e700 1 --2- 172.21.15.94:0/2159911646 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce1c0fbcd0 0x7fce1c0ffdf0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fce14001d80 tx=0x7fce14009ec0 comp rx=0 tx=0).stop 2024-03-24T10:45:42.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.937+0000 7fce21b9e700 1 -- 172.21.15.94:0/2159911646 shutdown_connections 2024-03-24T10:45:42.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.937+0000 7fce21b9e700 1 --2- 172.21.15.94:0/2159911646 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7fce080692a0 0x7fce0806b750 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:42.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.937+0000 7fce21b9e700 1 --2- 172.21.15.94:0/2159911646 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce1c0fbcd0 0x7fce1c0ffdf0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:42.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.937+0000 7fce21b9e700 1 --2- 172.21.15.94:0/2159911646 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce1c0fcf80 0x7fce1c0fe2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:45:42.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.937+0000 7fce21b9e700 1 -- 172.21.15.94:0/2159911646 >> 172.21.15.94:0/2159911646 conn(0x7fce1c0f7430 msgr2=0x7fce1c0f8c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:45:42.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.938+0000 7fce21b9e700 1 -- 172.21.15.94:0/2159911646 shutdown_connections 2024-03-24T10:45:42.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:45:42.938+0000 7fce21b9e700 1 -- 172.21.15.94:0/2159911646 wait complete. 2024-03-24T10:45:42.972 INFO:tasks.workunit.client.1.smithi096.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-03-24T10:45:42.991 INFO:tasks.workunit.client.1.smithi096.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-03-24T10:45:42.991 INFO:tasks.workunit.client.1.smithi096.stderr:+ make 2024-03-24T10:45:43.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:43 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/2090466418' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T10:45:43.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:43 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/2159911646' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T10:45:43.351 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:43 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/2090466418' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T10:45:43.351 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:43 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/2159911646' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T10:45:43.784 INFO:tasks.workunit.client.1.smithi096.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-03-24T10:45:44.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:44 smithi094 ceph-mon[29446]: from='client.14656 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:45:44.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:44 smithi094 ceph-mon[29446]: pgmap v235: 65 pgs: 65 active+clean; 157 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 5.7 MiB/s wr, 762 op/s 2024-03-24T10:45:44.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:44 smithi096 ceph-mon[38097]: from='client.14656 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:45:44.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:44 smithi096 ceph-mon[38097]: pgmap v235: 65 pgs: 65 active+clean; 157 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 5.7 MiB/s wr, 762 op/s 2024-03-24T10:45:44.689 INFO:tasks.workunit.client.0.smithi094.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-03-24T10:45:44.690 INFO:tasks.workunit.client.0.smithi094.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-24T10:45:44.690 INFO:tasks.workunit.client.0.smithi094.stderr:+ make 2024-03-24T10:45:44.772 INFO:tasks.workunit.client.0.smithi094.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-03-24T10:45:45.032 INFO:tasks.workunit.client.0.smithi094.stderr:++ readlink -f fsstress 2024-03-24T10:45:45.033 INFO:tasks.workunit.client.0.smithi094.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-03-24T10:45:45.033 INFO:tasks.workunit.client.0.smithi094.stderr:+ popd 2024-03-24T10:45:45.033 INFO:tasks.workunit.client.0.smithi094.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-24T10:45:45.033 INFO:tasks.workunit.client.0.smithi094.stderr:+ popd 2024-03-24T10:45:45.034 INFO:tasks.workunit.client.0.smithi094.stdout:~/cephtest/mnt.0/client.0/tmp 2024-03-24T10:45:45.034 INFO:tasks.workunit.client.0.smithi094.stderr:++ mktemp -d -p . 2024-03-24T10:45:45.034 INFO:tasks.workunit.client.0.smithi094.stderr:+ T=./tmp.ndA6na18Wr 2024-03-24T10:45:45.035 INFO:tasks.workunit.client.0.smithi094.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.ndA6na18Wr -l 1 -n 1000 -p 10 -v 2024-03-24T10:45:45.036 INFO:tasks.workunit.client.0.smithi094.stdout:seed = 1711314073 2024-03-24T10:45:45.039 INFO:tasks.workunit.client.0.smithi094.stdout:0/0: creat f0 x:0 0 0 2024-03-24T10:45:45.039 INFO:tasks.workunit.client.0.smithi094.stdout:0/1: write f0 [521748,21379] 0 2024-03-24T10:45:45.039 INFO:tasks.workunit.client.0.smithi094.stdout:2/0: dread - no filename 2024-03-24T10:45:45.039 INFO:tasks.workunit.client.0.smithi094.stdout:2/1: rmdir - no directory 2024-03-24T10:45:45.039 INFO:tasks.workunit.client.0.smithi094.stdout:2/2: dwrite - no filename 2024-03-24T10:45:45.040 INFO:tasks.workunit.client.0.smithi094.stdout:1/0: symlink l0 0 2024-03-24T10:45:45.041 INFO:tasks.workunit.client.0.smithi094.stdout:3/0: dread - no filename 2024-03-24T10:45:45.041 INFO:tasks.workunit.client.0.smithi094.stdout:3/1: dread - no filename 2024-03-24T10:45:45.041 INFO:tasks.workunit.client.0.smithi094.stdout:3/2: dread - no filename 2024-03-24T10:45:45.041 INFO:tasks.workunit.client.0.smithi094.stdout:3/3: chown . 19308 1 2024-03-24T10:45:45.041 INFO:tasks.workunit.client.0.smithi094.stdout:3/4: unlink - no file 2024-03-24T10:45:45.041 INFO:tasks.workunit.client.0.smithi094.stdout:3/5: dread - no filename 2024-03-24T10:45:45.042 INFO:tasks.workunit.client.0.smithi094.stdout:2/3: symlink l0 0 2024-03-24T10:45:45.042 INFO:tasks.workunit.client.0.smithi094.stdout:2/4: dread - no filename 2024-03-24T10:45:45.042 INFO:tasks.workunit.client.0.smithi094.stdout:2/5: chown l0 26880310 1 2024-03-24T10:45:45.042 INFO:tasks.workunit.client.0.smithi094.stdout:1/1: creat f1 x:0 0 0 2024-03-24T10:45:45.043 INFO:tasks.workunit.client.0.smithi094.stdout:1/2: creat f2 x:0 0 0 2024-03-24T10:45:45.043 INFO:tasks.workunit.client.0.smithi094.stdout:1/3: read - f2 zero size 2024-03-24T10:45:45.043 INFO:tasks.workunit.client.0.smithi094.stdout:1/4: truncate f1 640948 0 2024-03-24T10:45:45.043 INFO:tasks.workunit.client.0.smithi094.stdout:4/0: truncate - no filename 2024-03-24T10:45:45.043 INFO:tasks.workunit.client.0.smithi094.stdout:3/6: mkdir d0 0 2024-03-24T10:45:45.043 INFO:tasks.workunit.client.0.smithi094.stdout:3/7: link - no file 2024-03-24T10:45:45.043 INFO:tasks.workunit.client.0.smithi094.stdout:3/8: link - no file 2024-03-24T10:45:45.043 INFO:tasks.workunit.client.0.smithi094.stdout:3/9: rename d0 to d0/d1 22 2024-03-24T10:45:45.044 INFO:tasks.workunit.client.0.smithi094.stdout:1/5: mknod c3 0 2024-03-24T10:45:45.044 INFO:tasks.workunit.client.0.smithi094.stdout:1/6: chown l0 20 1 2024-03-24T10:45:45.044 INFO:tasks.workunit.client.0.smithi094.stdout:1/7: write f1 [978678,25317] 0 2024-03-24T10:45:45.045 INFO:tasks.workunit.client.0.smithi094.stdout:3/10: creat d0/f2 x:0 0 0 2024-03-24T10:45:45.045 INFO:tasks.workunit.client.0.smithi094.stdout:6/0: write - no filename 2024-03-24T10:45:45.045 INFO:tasks.workunit.client.0.smithi094.stdout:6/1: read - no filename 2024-03-24T10:45:45.045 INFO:tasks.workunit.client.0.smithi094.stdout:6/2: fdatasync - no filename 2024-03-24T10:45:45.046 INFO:tasks.workunit.client.0.smithi094.stdout:6/3: dread - no filename 2024-03-24T10:45:45.046 INFO:tasks.workunit.client.0.smithi094.stdout:6/4: dwrite - no filename 2024-03-24T10:45:45.046 INFO:tasks.workunit.client.0.smithi094.stdout:6/5: unlink - no file 2024-03-24T10:45:45.046 INFO:tasks.workunit.client.0.smithi094.stdout:6/6: chown . 435 1 2024-03-24T10:45:45.046 INFO:tasks.workunit.client.0.smithi094.stdout:6/7: dread - no filename 2024-03-24T10:45:45.046 INFO:tasks.workunit.client.0.smithi094.stdout:6/8: write - no filename 2024-03-24T10:45:45.047 INFO:tasks.workunit.client.1.smithi096.stderr:++ readlink -f fsstress 2024-03-24T10:45:45.047 INFO:tasks.workunit.client.0.smithi094.stdout:5/0: creat f0 x:0 0 0 2024-03-24T10:45:45.047 INFO:tasks.workunit.client.0.smithi094.stdout:0/2: dread f0 [0,4194304] 0 2024-03-24T10:45:45.047 INFO:tasks.workunit.client.0.smithi094.stdout:0/3: chown f0 1415 1 2024-03-24T10:45:45.048 INFO:tasks.workunit.client.1.smithi096.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-03-24T10:45:45.048 INFO:tasks.workunit.client.1.smithi096.stdout:~/cephtest/mnt.1/client.1/tmp 2024-03-24T10:45:45.048 INFO:tasks.workunit.client.1.smithi096.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-03-24T10:45:45.048 INFO:tasks.workunit.client.1.smithi096.stderr:+ popd 2024-03-24T10:45:45.048 INFO:tasks.workunit.client.1.smithi096.stderr:+ popd 2024-03-24T10:45:45.048 INFO:tasks.workunit.client.1.smithi096.stderr:++ mktemp -d -p . 2024-03-24T10:45:45.050 INFO:tasks.workunit.client.0.smithi094.stdout:7/0: creat f0 x:0 0 0 2024-03-24T10:45:45.050 INFO:tasks.workunit.client.0.smithi094.stdout:0/4: symlink l1 0 2024-03-24T10:45:45.050 INFO:tasks.workunit.client.0.smithi094.stdout:7/1: dread - f0 zero size 2024-03-24T10:45:45.050 INFO:tasks.workunit.client.0.smithi094.stdout:8/0: truncate - no filename 2024-03-24T10:45:45.050 INFO:tasks.workunit.client.0.smithi094.stdout:8/1: rmdir - no directory 2024-03-24T10:45:45.050 INFO:tasks.workunit.client.0.smithi094.stdout:8/2: write - no filename 2024-03-24T10:45:45.050 INFO:tasks.workunit.client.0.smithi094.stdout:8/3: truncate - no filename 2024-03-24T10:45:45.051 INFO:tasks.workunit.client.0.smithi094.stdout:8/4: link - no file 2024-03-24T10:45:45.051 INFO:tasks.workunit.client.0.smithi094.stdout:8/5: rename - no filename 2024-03-24T10:45:45.051 INFO:tasks.workunit.client.0.smithi094.stdout:8/6: chown . 4 1 2024-03-24T10:45:45.051 INFO:tasks.workunit.client.0.smithi094.stdout:8/7: write - no filename 2024-03-24T10:45:45.051 INFO:tasks.workunit.client.0.smithi094.stdout:8/8: dwrite - no filename 2024-03-24T10:45:45.051 INFO:tasks.workunit.client.0.smithi094.stdout:8/9: rmdir - no directory 2024-03-24T10:45:45.051 INFO:tasks.workunit.client.0.smithi094.stdout:8/10: fsync - no filename 2024-03-24T10:45:45.051 INFO:tasks.workunit.client.0.smithi094.stdout:8/11: rmdir - no directory 2024-03-24T10:45:45.051 INFO:tasks.workunit.client.0.smithi094.stdout:8/12: write - no filename 2024-03-24T10:45:45.052 INFO:tasks.workunit.client.1.smithi096.stderr:+ T=./tmp.NH14HyWfuo 2024-03-24T10:45:45.052 INFO:tasks.workunit.client.1.smithi096.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.NH14HyWfuo -l 1 -n 1000 -p 10 -v 2024-03-24T10:45:45.055 INFO:tasks.workunit.client.1.smithi096.stdout:seed = 1711331351 2024-03-24T10:45:45.060 INFO:tasks.workunit.client.0.smithi094.stdout:0/5: rename l1 to l2 0 2024-03-24T10:45:45.060 INFO:tasks.workunit.client.0.smithi094.stdout:8/13: mknod c0 0 2024-03-24T10:45:45.060 INFO:tasks.workunit.client.0.smithi094.stdout:8/14: truncate - no filename 2024-03-24T10:45:45.060 INFO:tasks.workunit.client.0.smithi094.stdout:0/6: creat f3 x:0 0 0 2024-03-24T10:45:45.060 INFO:tasks.workunit.client.0.smithi094.stdout:9/0: readlink - no filename 2024-03-24T10:45:45.064 INFO:tasks.workunit.client.1.smithi096.stdout:2/0: creat f0 x:0 0 0 2024-03-24T10:45:45.069 INFO:tasks.workunit.client.1.smithi096.stdout:5/0: rename - no filename 2024-03-24T10:45:45.069 INFO:tasks.workunit.client.1.smithi096.stdout:5/1: fdatasync - no filename 2024-03-24T10:45:45.069 INFO:tasks.workunit.client.1.smithi096.stdout:5/2: getdents . 0 2024-03-24T10:45:45.070 INFO:tasks.workunit.client.1.smithi096.stdout:5/3: rename - no filename 2024-03-24T10:45:45.070 INFO:tasks.workunit.client.1.smithi096.stdout:5/4: readlink - no filename 2024-03-24T10:45:45.070 INFO:tasks.workunit.client.1.smithi096.stdout:2/1: mknod c1 0 2024-03-24T10:45:45.070 INFO:tasks.workunit.client.0.smithi094.stdout:0/7: symlink l4 0 2024-03-24T10:45:45.070 INFO:tasks.workunit.client.0.smithi094.stdout:8/15: rename c0 to c1 0 2024-03-24T10:45:45.071 INFO:tasks.workunit.client.0.smithi094.stdout:9/1: creat f0 x:0 0 0 2024-03-24T10:45:45.071 INFO:tasks.workunit.client.0.smithi094.stdout:9/2: chown f0 4 1 2024-03-24T10:45:45.073 INFO:tasks.workunit.client.1.smithi096.stdout:5/5: mknod c0 0 2024-03-24T10:45:45.075 INFO:tasks.workunit.client.0.smithi094.stdout:0/8: unlink l4 0 2024-03-24T10:45:45.075 INFO:tasks.workunit.client.0.smithi094.stdout:0/9: creat f5 x:0 0 0 2024-03-24T10:45:45.075 INFO:tasks.workunit.client.0.smithi094.stdout:0/10: truncate f0 1061629 0 2024-03-24T10:45:45.075 INFO:tasks.workunit.client.0.smithi094.stdout:0/11: chown f3 1261930 1 2024-03-24T10:45:45.078 INFO:tasks.workunit.client.1.smithi096.stdout:1/0: mknod c0 0 2024-03-24T10:45:45.078 INFO:tasks.workunit.client.1.smithi096.stdout:1/1: write - no filename 2024-03-24T10:45:45.078 INFO:tasks.workunit.client.1.smithi096.stdout:1/2: dwrite - no filename 2024-03-24T10:45:45.078 INFO:tasks.workunit.client.1.smithi096.stdout:0/0: dread - no filename 2024-03-24T10:45:45.078 INFO:tasks.workunit.client.1.smithi096.stdout:0/1: rmdir - no directory 2024-03-24T10:45:45.081 INFO:tasks.workunit.client.0.smithi094.stdout:0/12: link f5 f6 0 2024-03-24T10:45:45.083 INFO:tasks.workunit.client.1.smithi096.stdout:1/3: rename c0 to c1 0 2024-03-24T10:45:45.084 INFO:tasks.workunit.client.1.smithi096.stdout:1/4: dwrite - no filename 2024-03-24T10:45:45.084 INFO:tasks.workunit.client.1.smithi096.stdout:4/0: link - no file 2024-03-24T10:45:45.084 INFO:tasks.workunit.client.1.smithi096.stdout:4/1: link - no file 2024-03-24T10:45:45.084 INFO:tasks.workunit.client.1.smithi096.stdout:4/2: dwrite - no filename 2024-03-24T10:45:45.084 INFO:tasks.workunit.client.1.smithi096.stdout:4/3: dread - no filename 2024-03-24T10:45:45.084 INFO:tasks.workunit.client.1.smithi096.stdout:0/2: mkdir d0 0 2024-03-24T10:45:45.087 INFO:tasks.workunit.client.1.smithi096.stdout:1/5: mknod c2 0 2024-03-24T10:45:45.087 INFO:tasks.workunit.client.1.smithi096.stdout:4/4: symlink l0 0 2024-03-24T10:45:45.087 INFO:tasks.workunit.client.1.smithi096.stdout:4/5: chown l0 321320 1 2024-03-24T10:45:45.087 INFO:tasks.workunit.client.1.smithi096.stdout:4/6: rmdir - no directory 2024-03-24T10:45:45.087 INFO:tasks.workunit.client.1.smithi096.stdout:4/7: readlink l0 0 2024-03-24T10:45:45.087 INFO:tasks.workunit.client.1.smithi096.stdout:4/8: chown l0 25316602 1 2024-03-24T10:45:45.087 INFO:tasks.workunit.client.1.smithi096.stdout:4/9: dwrite - no filename 2024-03-24T10:45:45.090 INFO:tasks.workunit.client.1.smithi096.stdout:1/6: mkdir d3 0 2024-03-24T10:45:45.090 INFO:tasks.workunit.client.1.smithi096.stdout:1/7: truncate - no filename 2024-03-24T10:45:45.090 INFO:tasks.workunit.client.1.smithi096.stdout:1/8: truncate - no filename 2024-03-24T10:45:45.090 INFO:tasks.workunit.client.1.smithi096.stdout:0/3: mknod d0/c1 0 2024-03-24T10:45:45.090 INFO:tasks.workunit.client.1.smithi096.stdout:0/4: fdatasync - no filename 2024-03-24T10:45:45.090 INFO:tasks.workunit.client.1.smithi096.stdout:0/5: readlink - no filename 2024-03-24T10:45:45.090 INFO:tasks.workunit.client.1.smithi096.stdout:0/6: write - no filename 2024-03-24T10:45:45.091 INFO:tasks.workunit.client.1.smithi096.stdout:0/7: dread - no filename 2024-03-24T10:45:45.091 INFO:tasks.workunit.client.1.smithi096.stdout:4/10: mkdir d1 0 2024-03-24T10:45:45.093 INFO:tasks.workunit.client.1.smithi096.stdout:9/0: mknod c0 0 2024-03-24T10:45:45.093 INFO:tasks.workunit.client.1.smithi096.stdout:9/1: fsync - no filename 2024-03-24T10:45:45.093 INFO:tasks.workunit.client.1.smithi096.stdout:9/2: fsync - no filename 2024-03-24T10:45:45.093 INFO:tasks.workunit.client.1.smithi096.stdout:9/3: dwrite - no filename 2024-03-24T10:45:45.093 INFO:tasks.workunit.client.1.smithi096.stdout:9/4: dread - no filename 2024-03-24T10:45:45.093 INFO:tasks.workunit.client.1.smithi096.stdout:9/5: dwrite - no filename 2024-03-24T10:45:45.095 INFO:tasks.workunit.client.1.smithi096.stdout:6/0: chown . 7 1 2024-03-24T10:45:45.095 INFO:tasks.workunit.client.1.smithi096.stdout:6/1: unlink - no file 2024-03-24T10:45:45.095 INFO:tasks.workunit.client.1.smithi096.stdout:1/9: mknod d3/c4 0 2024-03-24T10:45:45.095 INFO:tasks.workunit.client.1.smithi096.stdout:1/10: dread - no filename 2024-03-24T10:45:45.095 INFO:tasks.workunit.client.1.smithi096.stdout:1/11: write - no filename 2024-03-24T10:45:45.095 INFO:tasks.workunit.client.1.smithi096.stdout:1/12: stat c2 0 2024-03-24T10:45:45.098 INFO:tasks.workunit.client.1.smithi096.stdout:0/8: mkdir d0/d2 0 2024-03-24T10:45:45.098 INFO:tasks.workunit.client.1.smithi096.stdout:4/11: creat d1/f2 x:0 0 0 2024-03-24T10:45:45.098 INFO:tasks.workunit.client.1.smithi096.stdout:4/12: truncate d1/f2 674210 0 2024-03-24T10:45:45.098 INFO:tasks.workunit.client.1.smithi096.stdout:4/13: stat d1 0 2024-03-24T10:45:45.098 INFO:tasks.workunit.client.1.smithi096.stdout:4/14: fdatasync d1/f2 0 2024-03-24T10:45:45.098 INFO:tasks.workunit.client.1.smithi096.stdout:4/15: truncate d1/f2 821644 0 2024-03-24T10:45:45.098 INFO:tasks.workunit.client.1.smithi096.stdout:9/6: creat f1 x:0 0 0 2024-03-24T10:45:45.098 INFO:tasks.workunit.client.1.smithi096.stdout:9/7: dread - f1 zero size 2024-03-24T10:45:45.099 INFO:tasks.workunit.client.1.smithi096.stdout:1/13: rmdir d3 39 2024-03-24T10:45:45.099 INFO:tasks.workunit.client.1.smithi096.stdout:1/14: dread - no filename 2024-03-24T10:45:45.099 INFO:tasks.workunit.client.1.smithi096.stdout:1/15: dread - no filename 2024-03-24T10:45:45.099 INFO:tasks.workunit.client.1.smithi096.stdout:1/16: write - no filename 2024-03-24T10:45:45.099 INFO:tasks.workunit.client.1.smithi096.stdout:1/17: chown c1 15721 1 2024-03-24T10:45:45.099 INFO:tasks.workunit.client.1.smithi096.stdout:1/18: fsync - no filename 2024-03-24T10:45:45.100 INFO:tasks.workunit.client.1.smithi096.stdout:6/2: symlink l0 0 2024-03-24T10:45:45.100 INFO:tasks.workunit.client.1.smithi096.stdout:4/16: link d1/f2 d1/f3 0 2024-03-24T10:45:45.102 INFO:tasks.workunit.client.1.smithi096.stdout:1/19: creat d3/f5 x:0 0 0 2024-03-24T10:45:45.102 INFO:tasks.workunit.client.1.smithi096.stdout:1/20: truncate d3/f5 473701 0 2024-03-24T10:45:45.104 INFO:tasks.workunit.client.1.smithi096.stdout:7/0: write - no filename 2024-03-24T10:45:45.105 INFO:tasks.workunit.client.1.smithi096.stdout:6/3: creat f1 x:0 0 0 2024-03-24T10:45:45.112 INFO:tasks.workunit.client.1.smithi096.stdout:4/17: unlink d1/f2 0 2024-03-24T10:45:45.114 INFO:tasks.workunit.client.1.smithi096.stdout:1/21: link d3/f5 d3/f6 0 2024-03-24T10:45:45.117 INFO:tasks.workunit.client.1.smithi096.stdout:7/1: mkdir d0 0 2024-03-24T10:45:45.121 INFO:tasks.workunit.client.1.smithi096.stdout:4/18: truncate d1/f3 1106654 0 2024-03-24T10:45:45.121 INFO:tasks.workunit.client.1.smithi096.stdout:4/19: write d1/f3 [1568304,17900] 0 2024-03-24T10:45:45.123 INFO:tasks.workunit.client.1.smithi096.stdout:1/22: symlink d3/l7 0 2024-03-24T10:45:45.124 INFO:tasks.workunit.client.1.smithi096.stdout:7/2: mkdir d0/d1 0 2024-03-24T10:45:45.124 INFO:tasks.workunit.client.1.smithi096.stdout:7/3: dwrite - no filename 2024-03-24T10:45:45.128 INFO:tasks.workunit.client.1.smithi096.stdout:4/20: link d1/f3 d1/f4 0 2024-03-24T10:45:45.270 INFO:tasks.workunit.client.1.smithi096.stdout:2/2: dwrite f0 [0,4194304] 0 2024-03-24T10:45:45.270 INFO:tasks.workunit.client.1.smithi096.stdout:2/3: creat f2 x:0 0 0 2024-03-24T10:45:45.300 INFO:tasks.workunit.client.0.smithi094.stdout:8/16: rename c1 to c2 0 2024-03-24T10:45:45.310 INFO:tasks.workunit.client.1.smithi096.stdout:5/6: getdents . 0 2024-03-24T10:45:45.312 INFO:tasks.workunit.client.0.smithi094.stdout:3/11: dwrite d0/f2 [0,4194304] 0 2024-03-24T10:45:45.312 INFO:tasks.workunit.client.0.smithi094.stdout:3/12: stat d0/f2 0 2024-03-24T10:45:45.338 INFO:tasks.workunit.client.1.smithi096.stdout:9/8: fsync f1 0 2024-03-24T10:45:45.345 INFO:tasks.workunit.client.1.smithi096.stdout:6/4: getdents . 0 2024-03-24T10:45:45.369 INFO:tasks.workunit.client.1.smithi096.stdout:2/4: link f0 f3 0 2024-03-24T10:45:45.369 INFO:tasks.workunit.client.1.smithi096.stdout:2/5: creat f4 x:0 0 0 2024-03-24T10:45:45.426 INFO:tasks.workunit.client.1.smithi096.stdout:6/5: creat f2 x:0 0 0 2024-03-24T10:45:45.441 INFO:tasks.workunit.client.0.smithi094.stdout:1/8: dwrite f2 [0,4194304] 0 2024-03-24T10:45:45.446 INFO:tasks.workunit.client.1.smithi096.stdout:2/6: mkdir d5 0 2024-03-24T10:45:45.536 INFO:tasks.workunit.client.1.smithi096.stdout:6/6: read - f1 zero size 2024-03-24T10:45:45.536 INFO:tasks.workunit.client.1.smithi096.stdout:6/7: dread - f2 zero size 2024-03-24T10:45:45.554 INFO:tasks.workunit.client.1.smithi096.stdout:2/7: creat d5/f6 x:0 0 0 2024-03-24T10:45:45.739 INFO:tasks.workunit.client.1.smithi096.stdout:6/8: link f1 f3 0 2024-03-24T10:45:45.803 INFO:tasks.workunit.client.0.smithi094.stdout:5/1: dwrite f0 [0,4194304] 0 2024-03-24T10:45:45.952 INFO:tasks.workunit.client.0.smithi094.stdout:7/2: dwrite f0 [0,4194304] 0 2024-03-24T10:45:46.016 INFO:tasks.workunit.client.1.smithi096.stdout:7/4: dread - no filename 2024-03-24T10:45:46.016 INFO:tasks.workunit.client.1.smithi096.stdout:7/5: stat d0/d1 0 2024-03-24T10:45:46.016 INFO:tasks.workunit.client.1.smithi096.stdout:3/0: stat - no entries 2024-03-24T10:45:46.016 INFO:tasks.workunit.client.1.smithi096.stdout:8/0: readlink - no filename 2024-03-24T10:45:46.016 INFO:tasks.workunit.client.1.smithi096.stdout:8/1: rename - no filename 2024-03-24T10:45:46.016 INFO:tasks.workunit.client.1.smithi096.stdout:8/2: getdents . 0 2024-03-24T10:45:46.016 INFO:tasks.workunit.client.1.smithi096.stdout:8/3: dread - no filename 2024-03-24T10:45:46.061 INFO:tasks.workunit.client.1.smithi096.stdout:9/9: dwrite f1 [0,4194304] 0 2024-03-24T10:45:46.398 INFO:tasks.workunit.client.0.smithi094.stdout:1/9: dwrite f1 [0,4194304] 0 2024-03-24T10:45:46.449 INFO:tasks.workunit.client.0.smithi094.stdout:8/17: creat f3 x:0 0 0 2024-03-24T10:45:46.449 INFO:tasks.workunit.client.0.smithi094.stdout:8/18: creat f4 x:0 0 0 2024-03-24T10:45:46.451 INFO:tasks.workunit.client.0.smithi094.stdout:0/13: dwrite f6 [0,4194304] 0 2024-03-24T10:45:46.451 INFO:tasks.workunit.client.0.smithi094.stdout:0/14: chown f5 96506100 1 2024-03-24T10:45:46.573 INFO:tasks.workunit.client.1.smithi096.stdout:2/8: write f0 [595751,22780] 0 2024-03-24T10:45:46.573 INFO:tasks.workunit.client.1.smithi096.stdout:2/9: dread - f2 zero size 2024-03-24T10:45:46.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:46 smithi094 ceph-mon[29446]: pgmap v236: 65 pgs: 65 active+clean; 161 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.5 MiB/s rd, 5.1 MiB/s wr, 646 op/s 2024-03-24T10:45:46.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:46 smithi094 ceph-mon[29446]: Upgrade: Updating mgr.smithi096.gfonnu 2024-03-24T10:45:46.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:46 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:45:46.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:46 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi096.gfonnu", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T10:45:46.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:46 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T10:45:46.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:46 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:45:46.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:46 smithi094 ceph-mon[29446]: Deploying daemon mgr.smithi096.gfonnu on smithi096 2024-03-24T10:45:46.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:46 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:45:46.806 INFO:tasks.workunit.client.1.smithi096.stdout:4/21: dwrite d1/f3 [0,4194304] 0 2024-03-24T10:45:46.810 INFO:tasks.workunit.client.0.smithi094.stdout:3/13: write d0/f2 [3426956,30872] 0 2024-03-24T10:45:46.813 INFO:tasks.workunit.client.0.smithi094.stdout:9/3: dwrite f0 [0,4194304] 0 2024-03-24T10:45:46.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:46 smithi096 ceph-mon[38097]: pgmap v236: 65 pgs: 65 active+clean; 161 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.5 MiB/s rd, 5.1 MiB/s wr, 646 op/s 2024-03-24T10:45:46.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:46 smithi096 ceph-mon[38097]: Upgrade: Updating mgr.smithi096.gfonnu 2024-03-24T10:45:46.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:46 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:45:46.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:46 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi096.gfonnu", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T10:45:46.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:46 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T10:45:46.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:46 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:45:46.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:46 smithi096 ceph-mon[38097]: Deploying daemon mgr.smithi096.gfonnu on smithi096 2024-03-24T10:45:46.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:46 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:45:46.839 INFO:tasks.workunit.client.0.smithi094.stdout:3/14: dwrite d0/f2 [0,4194304] 0 2024-03-24T10:45:46.867 INFO:tasks.workunit.client.0.smithi094.stdout:3/15: dwrite d0/f2 [0,4194304] 0 2024-03-24T10:45:47.028 INFO:tasks.workunit.client.1.smithi096.stdout:6/9: link f3 f4 0 2024-03-24T10:45:47.028 INFO:tasks.workunit.client.1.smithi096.stdout:6/10: chown f2 299240 1 2024-03-24T10:45:47.046 INFO:tasks.workunit.client.0.smithi094.stdout:8/19: dwrite f4 [0,4194304] 0 2024-03-24T10:45:47.047 INFO:tasks.workunit.client.0.smithi094.stdout:8/20: chown f3 1263949 1 2024-03-24T10:45:47.047 INFO:tasks.workunit.client.0.smithi094.stdout:8/21: chown c2 2849222 1 2024-03-24T10:45:47.047 INFO:tasks.workunit.client.0.smithi094.stdout:8/22: dread - f3 zero size 2024-03-24T10:45:47.055 INFO:tasks.workunit.client.0.smithi094.stdout:0/15: dwrite f0 [0,4194304] 0 2024-03-24T10:45:47.065 INFO:tasks.workunit.client.1.smithi096.stdout:5/7: creat f1 x:0 0 0 2024-03-24T10:45:47.067 INFO:tasks.workunit.client.1.smithi096.stdout:7/6: symlink d0/l2 0 2024-03-24T10:45:47.069 INFO:tasks.workunit.client.1.smithi096.stdout:3/1: mkdir d0 0 2024-03-24T10:45:47.069 INFO:tasks.workunit.client.0.smithi094.stdout:8/23: dwrite f4 [0,4194304] 0 2024-03-24T10:45:47.069 INFO:tasks.workunit.client.0.smithi094.stdout:8/24: read - f3 zero size 2024-03-24T10:45:47.069 INFO:tasks.workunit.client.0.smithi094.stdout:8/25: truncate f3 772378 0 2024-03-24T10:45:47.069 INFO:tasks.workunit.client.0.smithi094.stdout:8/26: fdatasync f3 0 2024-03-24T10:45:47.070 INFO:tasks.workunit.client.1.smithi096.stdout:8/4: creat f0 x:0 0 0 2024-03-24T10:45:47.070 INFO:tasks.workunit.client.1.smithi096.stdout:8/5: dread - f0 zero size 2024-03-24T10:45:47.070 INFO:tasks.workunit.client.1.smithi096.stdout:8/6: read - f0 zero size 2024-03-24T10:45:47.070 INFO:tasks.workunit.client.1.smithi096.stdout:8/7: read - f0 zero size 2024-03-24T10:45:47.071 INFO:tasks.workunit.client.1.smithi096.stdout:8/8: creat f1 x:0 0 0 2024-03-24T10:45:47.179 INFO:tasks.workunit.client.1.smithi096.stdout:9/10: unlink f1 0 2024-03-24T10:45:47.179 INFO:tasks.workunit.client.1.smithi096.stdout:9/11: dwrite - no filename 2024-03-24T10:45:47.179 INFO:tasks.workunit.client.1.smithi096.stdout:9/12: stat c0 0 2024-03-24T10:45:47.179 INFO:tasks.workunit.client.1.smithi096.stdout:9/13: creat f2 x:0 0 0 2024-03-24T10:45:47.179 INFO:tasks.workunit.client.1.smithi096.stdout:9/14: chown c0 821 1 2024-03-24T10:45:47.179 INFO:tasks.workunit.client.1.smithi096.stdout:9/15: chown f2 285105762 1 2024-03-24T10:45:47.212 INFO:tasks.workunit.client.1.smithi096.stdout:2/10: write f0 [4189855,25453] 0 2024-03-24T10:45:47.212 INFO:tasks.workunit.client.1.smithi096.stdout:2/11: creat d5/f7 x:0 0 0 2024-03-24T10:45:47.928 INFO:tasks.workunit.client.1.smithi096.stdout:6/11: creat f5 x:0 0 0 2024-03-24T10:45:47.929 INFO:tasks.workunit.client.1.smithi096.stdout:5/8: mkdir d2 0 2024-03-24T10:45:47.932 INFO:tasks.workunit.client.1.smithi096.stdout:7/7: symlink d0/l3 0 2024-03-24T10:45:47.932 INFO:tasks.workunit.client.1.smithi096.stdout:7/8: truncate - no filename 2024-03-24T10:45:47.932 INFO:tasks.workunit.client.1.smithi096.stdout:7/9: truncate - no filename 2024-03-24T10:45:47.932 INFO:tasks.workunit.client.1.smithi096.stdout:7/10: write - no filename 2024-03-24T10:45:47.932 INFO:tasks.workunit.client.1.smithi096.stdout:7/11: truncate - no filename 2024-03-24T10:45:47.934 INFO:tasks.workunit.client.1.smithi096.stdout:3/2: creat d0/f1 x:0 0 0 2024-03-24T10:45:47.935 INFO:tasks.workunit.client.1.smithi096.stdout:3/3: chown d0/f1 5973305 1 2024-03-24T10:45:47.937 INFO:tasks.workunit.client.1.smithi096.stdout:8/9: symlink l2 0 2024-03-24T10:45:47.948 INFO:tasks.workunit.client.1.smithi096.stdout:2/12: mknod d5/c8 0 2024-03-24T10:45:47.950 INFO:tasks.workunit.client.1.smithi096.stdout:6/12: creat f6 x:0 0 0 2024-03-24T10:45:47.950 INFO:tasks.workunit.client.1.smithi096.stdout:6/13: chown f5 375312 1 2024-03-24T10:45:47.953 INFO:tasks.workunit.client.1.smithi096.stdout:8/10: dwrite f1 [0,4194304] 0 2024-03-24T10:45:47.953 INFO:tasks.workunit.client.1.smithi096.stdout:5/9: creat d2/f3 x:0 0 0 2024-03-24T10:45:47.954 INFO:tasks.workunit.client.1.smithi096.stdout:7/12: symlink d0/d1/l4 0 2024-03-24T10:45:47.954 INFO:tasks.workunit.client.1.smithi096.stdout:7/13: chown d0/l2 403 1 2024-03-24T10:45:47.954 INFO:tasks.workunit.client.1.smithi096.stdout:7/14: fsync - no filename 2024-03-24T10:45:47.958 INFO:tasks.workunit.client.1.smithi096.stdout:2/13: dread f0 [0,4194304] 0 2024-03-24T10:45:47.959 INFO:tasks.workunit.client.1.smithi096.stdout:3/4: mkdir d0/d2 0 2024-03-24T10:45:47.962 INFO:tasks.workunit.client.1.smithi096.stdout:9/16: rename f2 to f3 0 2024-03-24T10:45:47.964 INFO:tasks.workunit.client.1.smithi096.stdout:6/14: creat f7 x:0 0 0 2024-03-24T10:45:47.964 INFO:tasks.workunit.client.1.smithi096.stdout:6/15: read - f1 zero size 2024-03-24T10:45:47.971 INFO:tasks.workunit.client.1.smithi096.stdout:8/11: mknod c3 0 2024-03-24T10:45:47.973 INFO:tasks.workunit.client.1.smithi096.stdout:5/10: dwrite f1 [0,4194304] 0 2024-03-24T10:45:47.973 INFO:tasks.workunit.client.1.smithi096.stdout:5/11: chown c0 332544 1 2024-03-24T10:45:47.974 INFO:tasks.workunit.client.1.smithi096.stdout:7/15: creat d0/d1/f5 x:0 0 0 2024-03-24T10:45:47.974 INFO:tasks.workunit.client.1.smithi096.stdout:7/16: write d0/d1/f5 [669369,51170] 0 2024-03-24T10:45:47.975 INFO:tasks.workunit.client.1.smithi096.stdout:2/14: dwrite d5/f6 [0,4194304] 0 2024-03-24T10:45:47.975 INFO:tasks.workunit.client.1.smithi096.stdout:2/15: creat d5/f9 x:0 0 0 2024-03-24T10:45:47.975 INFO:tasks.workunit.client.1.smithi096.stdout:2/16: write f3 [1694558,111387] 0 2024-03-24T10:45:47.975 INFO:tasks.workunit.client.1.smithi096.stdout:2/17: chown f0 2 1 2024-03-24T10:45:47.979 INFO:tasks.workunit.client.1.smithi096.stdout:3/5: creat d0/d2/f3 x:0 0 0 2024-03-24T10:45:47.979 INFO:tasks.workunit.client.1.smithi096.stdout:3/6: truncate d0/d2/f3 288981 0 2024-03-24T10:45:47.986 INFO:tasks.workunit.client.1.smithi096.stdout:9/17: rename c0 to c4 0 2024-03-24T10:45:47.986 INFO:tasks.workunit.client.1.smithi096.stdout:9/18: write f3 [902637,84331] 0 2024-03-24T10:45:47.986 INFO:tasks.workunit.client.1.smithi096.stdout:9/19: truncate f3 1496458 0 2024-03-24T10:45:47.986 INFO:tasks.workunit.client.1.smithi096.stdout:9/20: creat f5 x:0 0 0 2024-03-24T10:45:47.990 INFO:tasks.workunit.client.1.smithi096.stdout:6/16: unlink f4 0 2024-03-24T10:45:47.993 INFO:tasks.workunit.client.1.smithi096.stdout:8/12: mknod c4 0 2024-03-24T10:45:47.993 INFO:tasks.workunit.client.1.smithi096.stdout:8/13: fdatasync f0 0 2024-03-24T10:45:47.994 INFO:tasks.workunit.client.1.smithi096.stdout:2/18: dwrite f3 [0,4194304] 0 2024-03-24T10:45:48.015 INFO:tasks.workunit.client.1.smithi096.stdout:2/19: unlink f0 0 2024-03-24T10:45:48.015 INFO:tasks.workunit.client.1.smithi096.stdout:2/20: dread - f4 zero size 2024-03-24T10:45:48.015 INFO:tasks.workunit.client.1.smithi096.stdout:2/21: chown f4 3294967 1 2024-03-24T10:45:48.015 INFO:tasks.workunit.client.1.smithi096.stdout:2/22: read - d5/f7 zero size 2024-03-24T10:45:48.015 INFO:tasks.workunit.client.1.smithi096.stdout:2/23: write f2 [813845,92536] 0 2024-03-24T10:45:48.015 INFO:tasks.workunit.client.1.smithi096.stdout:2/24: dread - d5/f7 zero size 2024-03-24T10:45:48.015 INFO:tasks.workunit.client.1.smithi096.stdout:2/25: creat d5/fa x:0 0 0 2024-03-24T10:45:48.015 INFO:tasks.workunit.client.1.smithi096.stdout:2/26: truncate d5/f9 477521 0 2024-03-24T10:45:48.016 INFO:tasks.workunit.client.1.smithi096.stdout:6/17: mkdir d8 0 2024-03-24T10:45:48.016 INFO:tasks.workunit.client.1.smithi096.stdout:6/18: write f7 [773029,10723] 0 2024-03-24T10:45:48.021 INFO:tasks.workunit.client.1.smithi096.stdout:6/19: link f3 d8/f9 0 2024-03-24T10:45:48.025 INFO:tasks.workunit.client.1.smithi096.stdout:6/20: rename f5 to d8/fa 0 2024-03-24T10:45:48.027 INFO:tasks.workunit.client.1.smithi096.stdout:6/21: rmdir d8 39 2024-03-24T10:45:48.029 INFO:tasks.workunit.client.1.smithi096.stdout:2/27: dwrite d5/f7 [0,4194304] 0 2024-03-24T10:45:48.029 INFO:tasks.workunit.client.1.smithi096.stdout:6/22: creat d8/fb x:0 0 0 2024-03-24T10:45:48.125 INFO:tasks.workunit.client.0.smithi094.stdout:5/2: mkdir d1 0 2024-03-24T10:45:48.400 INFO:tasks.workunit.client.1.smithi096.stdout:8/14: fsync f0 0 2024-03-24T10:45:48.408 INFO:tasks.workunit.client.1.smithi096.stdout:8/15: dread f1 [0,4194304] 0 2024-03-24T10:45:48.408 INFO:tasks.workunit.client.1.smithi096.stdout:8/16: readlink l2 0 2024-03-24T10:45:48.408 INFO:tasks.workunit.client.1.smithi096.stdout:8/17: chown f0 40 1 2024-03-24T10:45:48.408 INFO:tasks.workunit.client.1.smithi096.stdout:8/18: creat f5 x:0 0 0 2024-03-24T10:45:48.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:48 smithi094 ceph-mon[29446]: pgmap v237: 65 pgs: 65 active+clean; 184 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 2.8 MiB/s rd, 6.3 MiB/s wr, 637 op/s 2024-03-24T10:45:48.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:48 smithi096 ceph-mon[38097]: pgmap v237: 65 pgs: 65 active+clean; 184 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 2.8 MiB/s rd, 6.3 MiB/s wr, 637 op/s 2024-03-24T10:45:48.879 INFO:tasks.workunit.client.0.smithi094.stdout:1/10: mkdir d4 0 2024-03-24T10:45:48.879 INFO:tasks.workunit.client.0.smithi094.stdout:1/11: chown f1 30427 1 2024-03-24T10:45:48.879 INFO:tasks.workunit.client.0.smithi094.stdout:1/12: fsync f2 0 2024-03-24T10:45:49.282 INFO:tasks.workunit.client.1.smithi096.stdout:3/7: fsync d0/f1 0 2024-03-24T10:45:49.282 INFO:tasks.workunit.client.1.smithi096.stdout:3/8: write d0/f1 [853047,78612] 0 2024-03-24T10:45:49.282 INFO:tasks.workunit.client.1.smithi096.stdout:3/9: chown d0/f1 0 1 2024-03-24T10:45:49.315 INFO:tasks.workunit.client.1.smithi096.stdout:8/19: truncate f1 181308 0 2024-03-24T10:45:49.515 INFO:tasks.workunit.client.1.smithi096.stdout:3/10: dwrite d0/f1 [0,4194304] 0 2024-03-24T10:45:49.517 INFO:tasks.workunit.client.1.smithi096.stdout:4/22: write d1/f3 [913498,6987] 0 2024-03-24T10:45:49.525 INFO:tasks.workunit.client.1.smithi096.stdout:4/23: creat d1/f5 x:0 0 0 2024-03-24T10:45:49.526 INFO:tasks.workunit.client.1.smithi096.stdout:4/24: chown d1/f4 20 1 2024-03-24T10:45:49.526 INFO:tasks.workunit.client.1.smithi096.stdout:4/25: write d1/f5 [500078,108495] 0 2024-03-24T10:45:49.532 INFO:tasks.workunit.client.1.smithi096.stdout:3/11: dwrite d0/f1 [0,4194304] 0 2024-03-24T10:45:49.554 INFO:tasks.workunit.client.1.smithi096.stdout:3/12: dwrite d0/f1 [0,4194304] 0 2024-03-24T10:45:49.554 INFO:tasks.workunit.client.1.smithi096.stdout:3/13: readlink - no filename 2024-03-24T10:45:49.554 INFO:tasks.workunit.client.1.smithi096.stdout:4/26: dwrite d1/f4 [0,4194304] 0 2024-03-24T10:45:49.559 INFO:tasks.workunit.client.1.smithi096.stdout:4/27: symlink d1/l6 0 2024-03-24T10:45:49.564 INFO:tasks.workunit.client.1.smithi096.stdout:3/14: write d0/f1 [577092,125829] 0 2024-03-24T10:45:49.566 INFO:tasks.workunit.client.1.smithi096.stdout:3/15: rename d0/d2/f3 to d0/f4 0 2024-03-24T10:45:49.569 INFO:tasks.workunit.client.1.smithi096.stdout:4/28: write d1/f3 [143503,8530] 0 2024-03-24T10:45:49.573 INFO:tasks.workunit.client.1.smithi096.stdout:4/29: unlink d1/f4 0 2024-03-24T10:45:49.576 INFO:tasks.workunit.client.1.smithi096.stdout:4/30: symlink d1/l7 0 2024-03-24T10:45:49.578 INFO:tasks.workunit.client.1.smithi096.stdout:4/31: creat d1/f8 x:0 0 0 2024-03-24T10:45:49.660 INFO:tasks.workunit.client.0.smithi094.stdout:9/4: unlink f0 0 2024-03-24T10:45:49.660 INFO:tasks.workunit.client.0.smithi094.stdout:9/5: fdatasync - no filename 2024-03-24T10:45:49.660 INFO:tasks.workunit.client.0.smithi094.stdout:9/6: write - no filename 2024-03-24T10:45:49.660 INFO:tasks.workunit.client.0.smithi094.stdout:9/7: write - no filename 2024-03-24T10:45:49.660 INFO:tasks.workunit.client.0.smithi094.stdout:9/8: stat - no entries 2024-03-24T10:45:49.660 INFO:tasks.workunit.client.0.smithi094.stdout:9/9: rmdir - no directory 2024-03-24T10:45:50.247 INFO:tasks.workunit.client.0.smithi094.stdout:0/16: rename l2 to l7 0 2024-03-24T10:45:50.271 INFO:tasks.workunit.client.0.smithi094.stdout:0/17: dwrite f0 [0,4194304] 0 2024-03-24T10:45:50.311 INFO:tasks.workunit.client.0.smithi094.stdout:7/3: getdents . 0 2024-03-24T10:45:50.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:50 smithi094 ceph-mon[29446]: pgmap v238: 65 pgs: 65 active+clean; 202 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 4.9 MiB/s wr, 231 op/s 2024-03-24T10:45:50.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:50 smithi096 ceph-mon[38097]: pgmap v238: 65 pgs: 65 active+clean; 202 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 4.9 MiB/s wr, 231 op/s 2024-03-24T10:45:50.882 INFO:tasks.workunit.client.1.smithi096.stdout:8/20: dwrite f1 [0,4194304] 0 2024-03-24T10:45:50.882 INFO:tasks.workunit.client.1.smithi096.stdout:8/21: dread - f5 zero size 2024-03-24T10:45:50.897 INFO:tasks.workunit.client.1.smithi096.stdout:8/22: dwrite f0 [0,4194304] 0 2024-03-24T10:45:50.903 INFO:tasks.workunit.client.1.smithi096.stdout:8/23: link l2 l6 0 2024-03-24T10:45:50.904 INFO:tasks.workunit.client.1.smithi096.stdout:8/24: chown c3 112371442 1 2024-03-24T10:45:50.908 INFO:tasks.workunit.client.1.smithi096.stdout:8/25: symlink l7 0 2024-03-24T10:45:50.917 INFO:tasks.workunit.client.1.smithi096.stdout:8/26: dread f1 [0,4194304] 0 2024-03-24T10:45:50.917 INFO:tasks.workunit.client.1.smithi096.stdout:8/27: truncate f5 957665 0 2024-03-24T10:45:51.172 INFO:tasks.workunit.client.1.smithi096.stdout:3/16: rmdir d0 39 2024-03-24T10:45:51.172 INFO:tasks.workunit.client.1.smithi096.stdout:3/17: write d0/f4 [943880,39085] 0 2024-03-24T10:45:51.175 INFO:tasks.workunit.client.1.smithi096.stdout:3/18: symlink d0/l5 0 2024-03-24T10:45:51.175 INFO:tasks.workunit.client.1.smithi096.stdout:3/19: truncate d0/f4 1632303 0 2024-03-24T10:45:51.177 INFO:tasks.workunit.client.1.smithi096.stdout:3/20: rename d0/d2 to d0/d6 0 2024-03-24T10:45:51.178 INFO:tasks.workunit.client.1.smithi096.stdout:3/21: stat d0/f4 0 2024-03-24T10:45:51.183 INFO:tasks.workunit.client.1.smithi096.stdout:4/32: getdents d1 0 2024-03-24T10:45:51.183 INFO:tasks.workunit.client.1.smithi096.stdout:4/33: fsync d1/f8 0 2024-03-24T10:45:51.184 INFO:tasks.workunit.client.1.smithi096.stdout:4/34: symlink d1/l9 0 2024-03-24T10:45:51.187 INFO:tasks.workunit.client.1.smithi096.stdout:4/35: symlink d1/la 0 2024-03-24T10:45:51.189 INFO:tasks.workunit.client.1.smithi096.stdout:4/36: write d1/f5 [1310005,102283] 0 2024-03-24T10:45:51.191 INFO:tasks.workunit.client.1.smithi096.stdout:4/37: rmdir d1 39 2024-03-24T10:45:51.191 INFO:tasks.workunit.client.1.smithi096.stdout:4/38: write d1/f5 [1939453,83910] 0 2024-03-24T10:45:51.191 INFO:tasks.workunit.client.1.smithi096.stdout:4/39: stat d1/f5 0 2024-03-24T10:45:51.194 INFO:tasks.workunit.client.1.smithi096.stdout:4/40: mknod d1/cb 0 2024-03-24T10:45:51.209 INFO:tasks.workunit.client.1.smithi096.stdout:4/41: dwrite d1/f8 [0,4194304] 0 2024-03-24T10:45:51.220 INFO:tasks.workunit.client.0.smithi094.stdout:3/16: link d0/f2 d0/f3 0 2024-03-24T10:45:51.222 INFO:tasks.workunit.client.0.smithi094.stdout:5/3: unlink f0 0 2024-03-24T10:45:51.222 INFO:tasks.workunit.client.0.smithi094.stdout:5/4: rename d1 to d1/d2 22 2024-03-24T10:45:51.222 INFO:tasks.workunit.client.0.smithi094.stdout:5/5: readlink - no filename 2024-03-24T10:45:51.222 INFO:tasks.workunit.client.0.smithi094.stdout:7/4: mkdir d1 0 2024-03-24T10:45:51.223 INFO:tasks.workunit.client.0.smithi094.stdout:3/17: mknod d0/c4 0 2024-03-24T10:45:51.223 INFO:tasks.workunit.client.0.smithi094.stdout:3/18: creat d0/f5 x:0 0 0 2024-03-24T10:45:51.223 INFO:tasks.workunit.client.0.smithi094.stdout:3/19: truncate d0/f5 925736 0 2024-03-24T10:45:51.224 INFO:tasks.workunit.client.0.smithi094.stdout:3/20: rename d0 to d0/d6 22 2024-03-24T10:45:51.226 INFO:tasks.workunit.client.0.smithi094.stdout:5/6: rmdir d1 0 2024-03-24T10:45:51.227 INFO:tasks.workunit.client.0.smithi094.stdout:7/5: creat d1/f2 x:0 0 0 2024-03-24T10:45:51.228 INFO:tasks.workunit.client.0.smithi094.stdout:5/7: mknod c3 0 2024-03-24T10:45:51.228 INFO:tasks.workunit.client.0.smithi094.stdout:5/8: dread - no filename 2024-03-24T10:45:51.229 INFO:tasks.workunit.client.0.smithi094.stdout:7/6: chown f0 785539722 1 2024-03-24T10:45:51.230 INFO:tasks.workunit.client.0.smithi094.stdout:5/9: link c3 c4 0 2024-03-24T10:45:51.231 INFO:tasks.workunit.client.1.smithi096.stdout:4/42: dwrite d1/f5 [0,4194304] 0 2024-03-24T10:45:51.231 INFO:tasks.workunit.client.1.smithi096.stdout:4/43: fdatasync d1/f5 0 2024-03-24T10:45:51.233 INFO:tasks.workunit.client.0.smithi094.stdout:5/10: unlink c4 0 2024-03-24T10:45:51.233 INFO:tasks.workunit.client.0.smithi094.stdout:5/11: fsync - no filename 2024-03-24T10:45:51.233 INFO:tasks.workunit.client.0.smithi094.stdout:5/12: write - no filename 2024-03-24T10:45:51.233 INFO:tasks.workunit.client.0.smithi094.stdout:5/13: dwrite - no filename 2024-03-24T10:45:51.233 INFO:tasks.workunit.client.0.smithi094.stdout:5/14: dread - no filename 2024-03-24T10:45:51.233 INFO:tasks.workunit.client.0.smithi094.stdout:5/15: write - no filename 2024-03-24T10:45:51.233 INFO:tasks.workunit.client.0.smithi094.stdout:5/16: dwrite - no filename 2024-03-24T10:45:51.233 INFO:tasks.workunit.client.0.smithi094.stdout:5/17: dread - no filename 2024-03-24T10:45:51.233 INFO:tasks.workunit.client.0.smithi094.stdout:5/18: chown c3 526302 1 2024-03-24T10:45:51.234 INFO:tasks.workunit.client.0.smithi094.stdout:5/19: dread - no filename 2024-03-24T10:45:51.234 INFO:tasks.workunit.client.0.smithi094.stdout:5/20: write - no filename 2024-03-24T10:45:51.234 INFO:tasks.workunit.client.1.smithi096.stdout:4/44: mknod d1/cc 0 2024-03-24T10:45:51.234 INFO:tasks.workunit.client.1.smithi096.stdout:4/45: fdatasync d1/f8 0 2024-03-24T10:45:51.235 INFO:tasks.workunit.client.0.smithi094.stdout:5/21: mknod c5 0 2024-03-24T10:45:51.237 INFO:tasks.workunit.client.0.smithi094.stdout:5/22: link c5 c6 0 2024-03-24T10:45:51.239 INFO:tasks.workunit.client.0.smithi094.stdout:5/23: mkdir d7 0 2024-03-24T10:45:51.239 INFO:tasks.workunit.client.0.smithi094.stdout:5/24: write - no filename 2024-03-24T10:45:51.239 INFO:tasks.workunit.client.0.smithi094.stdout:5/25: fsync - no filename 2024-03-24T10:45:51.239 INFO:tasks.workunit.client.0.smithi094.stdout:5/26: truncate - no filename 2024-03-24T10:45:51.239 INFO:tasks.workunit.client.0.smithi094.stdout:5/27: dread - no filename 2024-03-24T10:45:51.239 INFO:tasks.workunit.client.0.smithi094.stdout:5/28: write - no filename 2024-03-24T10:45:51.239 INFO:tasks.workunit.client.0.smithi094.stdout:5/29: write - no filename 2024-03-24T10:45:51.239 INFO:tasks.workunit.client.0.smithi094.stdout:3/21: dwrite d0/f5 [0,4194304] 0 2024-03-24T10:45:51.240 INFO:tasks.workunit.client.0.smithi094.stdout:5/30: mkdir d7/d8 0 2024-03-24T10:45:51.244 INFO:tasks.workunit.client.0.smithi094.stdout:5/31: mkdir d7/d8/d9 0 2024-03-24T10:45:51.246 INFO:tasks.workunit.client.0.smithi094.stdout:5/32: creat d7/d8/fa x:0 0 0 2024-03-24T10:45:51.256 INFO:tasks.workunit.client.1.smithi096.stdout:4/46: dwrite d1/f3 [0,4194304] 0 2024-03-24T10:45:51.263 INFO:tasks.workunit.client.0.smithi094.stdout:3/22: dwrite d0/f2 [0,4194304] 0 2024-03-24T10:45:51.270 INFO:tasks.workunit.client.0.smithi094.stdout:3/23: dread d0/f5 [0,4194304] 0 2024-03-24T10:45:51.542 INFO:tasks.workunit.client.1.smithi096.stdout:8/28: truncate f5 837017 0 2024-03-24T10:45:51.543 INFO:tasks.workunit.client.1.smithi096.stdout:8/29: mkdir d8 0 2024-03-24T10:45:51.544 INFO:tasks.workunit.client.1.smithi096.stdout:8/30: symlink d8/l9 0 2024-03-24T10:45:51.544 INFO:tasks.workunit.client.1.smithi096.stdout:8/31: symlink d8/la 0 2024-03-24T10:45:51.598 INFO:tasks.workunit.client.1.smithi096.stdout:3/22: rmdir d0/d6 0 2024-03-24T10:45:51.599 INFO:tasks.workunit.client.1.smithi096.stdout:3/23: creat d0/f7 x:0 0 0 2024-03-24T10:45:51.599 INFO:tasks.workunit.client.1.smithi096.stdout:3/24: mknod d0/c8 0 2024-03-24T10:45:51.640 INFO:tasks.workunit.client.0.smithi094.stdout:4/1: sync 2024-03-24T10:45:51.640 INFO:tasks.workunit.client.0.smithi094.stdout:4/2: rename - no filename 2024-03-24T10:45:51.640 INFO:tasks.workunit.client.0.smithi094.stdout:2/6: sync 2024-03-24T10:45:51.640 INFO:tasks.workunit.client.0.smithi094.stdout:6/9: sync 2024-03-24T10:45:51.640 INFO:tasks.workunit.client.0.smithi094.stdout:6/10: dread - no filename 2024-03-24T10:45:51.640 INFO:tasks.workunit.client.0.smithi094.stdout:6/11: rename - no filename 2024-03-24T10:45:51.640 INFO:tasks.workunit.client.0.smithi094.stdout:6/12: unlink - no file 2024-03-24T10:45:51.641 INFO:tasks.workunit.client.0.smithi094.stdout:4/3: fdatasync - no filename 2024-03-24T10:45:51.641 INFO:tasks.workunit.client.0.smithi094.stdout:6/13: stat - no entries 2024-03-24T10:45:51.641 INFO:tasks.workunit.client.0.smithi094.stdout:6/14: truncate - no filename 2024-03-24T10:45:51.641 INFO:tasks.workunit.client.0.smithi094.stdout:6/15: unlink - no file 2024-03-24T10:45:51.641 INFO:tasks.workunit.client.0.smithi094.stdout:9/10: sync 2024-03-24T10:45:51.641 INFO:tasks.workunit.client.0.smithi094.stdout:9/11: dread - no filename 2024-03-24T10:45:51.641 INFO:tasks.workunit.client.0.smithi094.stdout:8/27: sync 2024-03-24T10:45:51.641 INFO:tasks.workunit.client.0.smithi094.stdout:1/13: sync 2024-03-24T10:45:51.641 INFO:tasks.workunit.client.0.smithi094.stdout:4/4: getdents . 0 2024-03-24T10:45:51.641 INFO:tasks.workunit.client.0.smithi094.stdout:1/14: chown l0 3 1 2024-03-24T10:45:51.641 INFO:tasks.workunit.client.0.smithi094.stdout:4/5: truncate - no filename 2024-03-24T10:45:51.641 INFO:tasks.workunit.client.0.smithi094.stdout:4/6: dread - no filename 2024-03-24T10:45:51.641 INFO:tasks.workunit.client.0.smithi094.stdout:4/7: unlink - no file 2024-03-24T10:45:51.642 INFO:tasks.workunit.client.0.smithi094.stdout:0/18: truncate f5 377558 0 2024-03-24T10:45:51.645 INFO:tasks.workunit.client.0.smithi094.stdout:1/15: write f1 [1367996,20499] 0 2024-03-24T10:45:51.645 INFO:tasks.workunit.client.0.smithi094.stdout:1/16: chown d4 109 1 2024-03-24T10:45:51.667 INFO:tasks.workunit.client.1.smithi096.stdout:3/25: truncate d0/f4 551824 0 2024-03-24T10:45:51.669 INFO:tasks.workunit.client.0.smithi094.stdout:3/24: truncate d0/f2 776997 0 2024-03-24T10:45:51.671 INFO:tasks.workunit.client.1.smithi096.stdout:8/32: write f5 [1841265,101765] 0 2024-03-24T10:45:51.671 INFO:tasks.workunit.client.1.smithi096.stdout:8/33: truncate f1 4625548 0 2024-03-24T10:45:51.678 INFO:tasks.workunit.client.1.smithi096.stdout:8/34: truncate f0 3469577 0 2024-03-24T10:45:51.705 INFO:tasks.workunit.client.0.smithi094.stdout:2/7: rename l0 to l1 0 2024-03-24T10:45:51.706 INFO:tasks.workunit.client.0.smithi094.stdout:6/16: mkdir d0 0 2024-03-24T10:45:51.706 INFO:tasks.workunit.client.0.smithi094.stdout:6/17: rename d0 to d0/d1 22 2024-03-24T10:45:51.706 INFO:tasks.workunit.client.0.smithi094.stdout:6/18: write - no filename 2024-03-24T10:45:51.708 INFO:tasks.workunit.client.0.smithi094.stdout:8/28: link f3 f5 0 2024-03-24T10:45:51.708 INFO:tasks.workunit.client.0.smithi094.stdout:8/29: read f5 [383157,66567] 0 2024-03-24T10:45:51.708 INFO:tasks.workunit.client.0.smithi094.stdout:8/30: read f3 [330619,84652] 0 2024-03-24T10:45:51.708 INFO:tasks.workunit.client.0.smithi094.stdout:9/12: symlink l1 0 2024-03-24T10:45:51.709 INFO:tasks.workunit.client.0.smithi094.stdout:9/13: write - no filename 2024-03-24T10:45:51.709 INFO:tasks.workunit.client.0.smithi094.stdout:9/14: dread - no filename 2024-03-24T10:45:51.709 INFO:tasks.workunit.client.0.smithi094.stdout:9/15: dread - no filename 2024-03-24T10:45:51.709 INFO:tasks.workunit.client.0.smithi094.stdout:8/31: dread f5 [0,4194304] 0 2024-03-24T10:45:51.709 INFO:tasks.workunit.client.0.smithi094.stdout:8/32: truncate f3 1220873 0 2024-03-24T10:45:51.710 INFO:tasks.workunit.client.0.smithi094.stdout:1/17: truncate f2 4115993 0 2024-03-24T10:45:51.713 INFO:tasks.workunit.client.0.smithi094.stdout:2/8: symlink l2 0 2024-03-24T10:45:51.715 INFO:tasks.workunit.client.0.smithi094.stdout:6/19: creat d0/f2 x:0 0 0 2024-03-24T10:45:51.715 INFO:tasks.workunit.client.0.smithi094.stdout:6/20: chown d0 44 1 2024-03-24T10:45:51.715 INFO:tasks.workunit.client.0.smithi094.stdout:6/21: dread - d0/f2 zero size 2024-03-24T10:45:51.715 INFO:tasks.workunit.client.0.smithi094.stdout:6/22: truncate d0/f2 72105 0 2024-03-24T10:45:51.716 INFO:tasks.workunit.client.0.smithi094.stdout:9/16: mknod c2 0 2024-03-24T10:45:51.716 INFO:tasks.workunit.client.0.smithi094.stdout:9/17: chown l1 33 1 2024-03-24T10:45:51.716 INFO:tasks.workunit.client.0.smithi094.stdout:9/18: readlink l1 0 2024-03-24T10:45:51.716 INFO:tasks.workunit.client.0.smithi094.stdout:9/19: write - no filename 2024-03-24T10:45:51.716 INFO:tasks.workunit.client.0.smithi094.stdout:9/20: rmdir - no directory 2024-03-24T10:45:51.716 INFO:tasks.workunit.client.0.smithi094.stdout:9/21: stat l1 0 2024-03-24T10:45:51.716 INFO:tasks.workunit.client.0.smithi094.stdout:9/22: dread - no filename 2024-03-24T10:45:51.721 INFO:tasks.workunit.client.0.smithi094.stdout:2/9: symlink l3 0 2024-03-24T10:45:51.723 INFO:tasks.workunit.client.0.smithi094.stdout:9/23: mkdir d3 0 2024-03-24T10:45:51.725 INFO:tasks.workunit.client.0.smithi094.stdout:9/24: rmdir d3 0 2024-03-24T10:45:51.727 INFO:tasks.workunit.client.0.smithi094.stdout:9/25: rename l1 to l4 0 2024-03-24T10:45:51.732 INFO:tasks.workunit.client.0.smithi094.stdout:8/33: dwrite f4 [0,4194304] 0 2024-03-24T10:45:51.732 INFO:tasks.workunit.client.0.smithi094.stdout:8/34: creat f6 x:0 0 0 2024-03-24T10:45:51.732 INFO:tasks.workunit.client.0.smithi094.stdout:6/23: dwrite d0/f2 [0,4194304] 0 2024-03-24T10:45:51.750 INFO:tasks.workunit.client.1.smithi096.stdout:3/26: write d0/f4 [352127,38938] 0 2024-03-24T10:45:51.751 INFO:tasks.workunit.client.0.smithi094.stdout:8/35: symlink l7 0 2024-03-24T10:45:51.752 INFO:tasks.workunit.client.0.smithi094.stdout:8/36: read f3 [1067060,35640] 0 2024-03-24T10:45:51.752 INFO:tasks.workunit.client.0.smithi094.stdout:8/37: write f3 [340332,75211] 0 2024-03-24T10:45:51.752 INFO:tasks.workunit.client.0.smithi094.stdout:8/38: chown c2 1 1 2024-03-24T10:45:51.752 INFO:tasks.workunit.client.0.smithi094.stdout:7/7: sync 2024-03-24T10:45:51.752 INFO:tasks.workunit.client.0.smithi094.stdout:5/33: sync 2024-03-24T10:45:51.752 INFO:tasks.workunit.client.0.smithi094.stdout:7/8: chown f0 765449277 1 2024-03-24T10:45:51.752 INFO:tasks.workunit.client.0.smithi094.stdout:8/39: creat f8 x:0 0 0 2024-03-24T10:45:51.753 INFO:tasks.workunit.client.0.smithi094.stdout:7/9: fsync d1/f2 0 2024-03-24T10:45:51.753 INFO:tasks.workunit.client.0.smithi094.stdout:7/10: creat d1/f3 x:0 0 0 2024-03-24T10:45:51.753 INFO:tasks.workunit.client.0.smithi094.stdout:8/40: write f3 [1576147,55503] 0 2024-03-24T10:45:51.754 INFO:tasks.workunit.client.1.smithi096.stdout:3/27: mkdir d0/d9 0 2024-03-24T10:45:51.755 INFO:tasks.workunit.client.1.smithi096.stdout:3/28: rename d0/d9 to d0/d9/da 22 2024-03-24T10:45:51.755 INFO:tasks.workunit.client.1.smithi096.stdout:3/29: write d0/f7 [990319,83811] 0 2024-03-24T10:45:51.755 INFO:tasks.workunit.client.1.smithi096.stdout:3/30: write d0/f7 [1262318,18522] 0 2024-03-24T10:45:51.755 INFO:tasks.workunit.client.1.smithi096.stdout:3/31: chown d0/c8 384 1 2024-03-24T10:45:51.756 INFO:tasks.workunit.client.0.smithi094.stdout:8/41: dread f5 [0,4194304] 0 2024-03-24T10:45:51.758 INFO:tasks.workunit.client.0.smithi094.stdout:3/25: dread d0/f3 [0,4194304] 0 2024-03-24T10:45:51.760 INFO:tasks.workunit.client.0.smithi094.stdout:3/26: dread d0/f2 [0,4194304] 0 2024-03-24T10:45:51.762 INFO:tasks.workunit.client.0.smithi094.stdout:7/11: symlink d1/l4 0 2024-03-24T10:45:51.763 INFO:tasks.workunit.client.0.smithi094.stdout:3/27: dread d0/f3 [0,4194304] 0 2024-03-24T10:45:51.763 INFO:tasks.workunit.client.0.smithi094.stdout:3/28: fsync d0/f2 0 2024-03-24T10:45:51.763 INFO:tasks.workunit.client.0.smithi094.stdout:3/29: chown d0/f2 3 1 2024-03-24T10:45:51.765 INFO:tasks.workunit.client.0.smithi094.stdout:5/34: rmdir d7/d8 39 2024-03-24T10:45:51.765 INFO:tasks.workunit.client.0.smithi094.stdout:5/35: chown c3 109969250 1 2024-03-24T10:45:51.767 INFO:tasks.workunit.client.0.smithi094.stdout:3/30: write d0/f5 [1016034,19008] 0 2024-03-24T10:45:51.767 INFO:tasks.workunit.client.0.smithi094.stdout:7/12: symlink d1/l5 0 2024-03-24T10:45:51.767 INFO:tasks.workunit.client.0.smithi094.stdout:7/13: creat d1/f6 x:0 0 0 2024-03-24T10:45:51.768 INFO:tasks.workunit.client.0.smithi094.stdout:3/31: mkdir d0/d7 0 2024-03-24T10:45:51.771 INFO:tasks.workunit.client.0.smithi094.stdout:3/32: symlink d0/d7/l8 0 2024-03-24T10:45:51.775 INFO:tasks.workunit.client.0.smithi094.stdout:8/42: dwrite f5 [0,4194304] 0 2024-03-24T10:45:51.779 INFO:tasks.workunit.client.1.smithi096.stdout:3/32: dwrite d0/f1 [0,4194304] 0 2024-03-24T10:45:51.783 INFO:tasks.workunit.client.0.smithi094.stdout:5/36: dwrite d7/d8/fa [0,4194304] 0 2024-03-24T10:45:51.786 INFO:tasks.workunit.client.0.smithi094.stdout:7/14: dwrite d1/f3 [0,4194304] 0 2024-03-24T10:45:51.792 INFO:tasks.workunit.client.0.smithi094.stdout:8/43: fsync f5 0 2024-03-24T10:45:51.795 INFO:tasks.workunit.client.0.smithi094.stdout:7/15: rename d1/f6 to d1/f7 0 2024-03-24T10:45:51.795 INFO:tasks.workunit.client.0.smithi094.stdout:0/19: write f6 [621369,56365] 0 2024-03-24T10:45:51.796 INFO:tasks.workunit.client.0.smithi094.stdout:8/44: mknod c9 0 2024-03-24T10:45:51.796 INFO:tasks.workunit.client.0.smithi094.stdout:8/45: creat fa x:0 0 0 2024-03-24T10:45:51.797 INFO:tasks.workunit.client.0.smithi094.stdout:1/18: rename f2 to d4/f5 0 2024-03-24T10:45:51.798 INFO:tasks.workunit.client.0.smithi094.stdout:7/16: symlink d1/l8 0 2024-03-24T10:45:51.800 INFO:tasks.workunit.client.0.smithi094.stdout:8/46: symlink lb 0 2024-03-24T10:45:51.800 INFO:tasks.workunit.client.0.smithi094.stdout:8/47: truncate f8 988066 0 2024-03-24T10:45:51.801 INFO:tasks.workunit.client.0.smithi094.stdout:8/48: dread f8 [0,4194304] 0 2024-03-24T10:45:51.801 INFO:tasks.workunit.client.0.smithi094.stdout:8/49: creat fc x:0 0 0 2024-03-24T10:45:51.801 INFO:tasks.workunit.client.0.smithi094.stdout:8/50: creat fd x:0 0 0 2024-03-24T10:45:51.801 INFO:tasks.workunit.client.0.smithi094.stdout:3/33: rename d0/f5 to d0/d7/f9 0 2024-03-24T10:45:51.801 INFO:tasks.workunit.client.0.smithi094.stdout:5/37: rename d7/d8/d9 to d7/d8/d9/db 22 2024-03-24T10:45:51.802 INFO:tasks.workunit.client.1.smithi096.stdout:3/33: dwrite d0/f4 [0,4194304] 0 2024-03-24T10:45:51.805 INFO:tasks.workunit.client.0.smithi094.stdout:7/17: write d1/f3 [3852202,97542] 0 2024-03-24T10:45:51.805 INFO:tasks.workunit.client.0.smithi094.stdout:7/18: stat d1/l8 0 2024-03-24T10:45:51.806 INFO:tasks.workunit.client.0.smithi094.stdout:1/19: rename l0 to d4/l6 0 2024-03-24T10:45:51.806 INFO:tasks.workunit.client.0.smithi094.stdout:1/20: stat d4/f5 0 2024-03-24T10:45:51.806 INFO:tasks.workunit.client.0.smithi094.stdout:0/20: dread f5 [0,4194304] 0 2024-03-24T10:45:51.807 INFO:tasks.workunit.client.1.smithi096.stdout:3/34: mkdir d0/d9/db 0 2024-03-24T10:45:51.808 INFO:tasks.workunit.client.0.smithi094.stdout:3/34: rmdir d0/d7 39 2024-03-24T10:45:51.809 INFO:tasks.workunit.client.0.smithi094.stdout:5/38: mkdir d7/dc 0 2024-03-24T10:45:51.810 INFO:tasks.workunit.client.1.smithi096.stdout:3/35: mknod d0/cc 0 2024-03-24T10:45:51.810 INFO:tasks.workunit.client.1.smithi096.stdout:3/36: rename d0/d9 to d0/d9/dd 22 2024-03-24T10:45:51.813 INFO:tasks.workunit.client.0.smithi094.stdout:7/19: mkdir d1/d9 0 2024-03-24T10:45:51.813 INFO:tasks.workunit.client.0.smithi094.stdout:7/20: dread - d1/f2 zero size 2024-03-24T10:45:51.813 INFO:tasks.workunit.client.0.smithi094.stdout:7/21: stat d1 0 2024-03-24T10:45:51.813 INFO:tasks.workunit.client.0.smithi094.stdout:7/22: stat d1/f3 0 2024-03-24T10:45:51.815 INFO:tasks.workunit.client.0.smithi094.stdout:5/39: dread d7/d8/fa [0,4194304] 0 2024-03-24T10:45:51.817 INFO:tasks.workunit.client.0.smithi094.stdout:0/21: rename f6 to f8 0 2024-03-24T10:45:51.817 INFO:tasks.workunit.client.0.smithi094.stdout:0/22: creat f9 x:0 0 0 2024-03-24T10:45:51.817 INFO:tasks.workunit.client.0.smithi094.stdout:0/23: chown f8 300002318 1 2024-03-24T10:45:51.817 INFO:tasks.workunit.client.0.smithi094.stdout:0/24: rmdir - no directory 2024-03-24T10:45:51.817 INFO:tasks.workunit.client.0.smithi094.stdout:0/25: chown f5 1246 1 2024-03-24T10:45:51.817 INFO:tasks.workunit.client.0.smithi094.stdout:0/26: write f0 [5142853,127832] 0 2024-03-24T10:45:51.817 INFO:tasks.workunit.client.0.smithi094.stdout:0/27: readlink l7 0 2024-03-24T10:45:51.820 INFO:tasks.workunit.client.0.smithi094.stdout:5/40: rename c6 to d7/d8/d9/cd 0 2024-03-24T10:45:51.821 INFO:tasks.workunit.client.0.smithi094.stdout:6/24: write d0/f2 [4932116,22583] 0 2024-03-24T10:45:51.824 INFO:tasks.workunit.client.0.smithi094.stdout:6/25: rmdir d0 39 2024-03-24T10:45:51.825 INFO:tasks.workunit.client.1.smithi096.stdout:3/37: dwrite d0/f1 [0,4194304] 0 2024-03-24T10:45:51.825 INFO:tasks.workunit.client.1.smithi096.stdout:3/38: chown d0/d9 644962993 1 2024-03-24T10:45:51.826 INFO:tasks.workunit.client.0.smithi094.stdout:6/26: creat d0/f3 x:0 0 0 2024-03-24T10:45:51.826 INFO:tasks.workunit.client.1.smithi096.stdout:3/39: symlink d0/d9/le 0 2024-03-24T10:45:51.826 INFO:tasks.workunit.client.1.smithi096.stdout:3/40: stat d0/f4 0 2024-03-24T10:45:51.827 INFO:tasks.workunit.client.0.smithi094.stdout:6/27: mknod d0/c4 0 2024-03-24T10:45:51.828 INFO:tasks.workunit.client.1.smithi096.stdout:3/41: rmdir d0/d9/db 0 2024-03-24T10:45:51.829 INFO:tasks.workunit.client.0.smithi094.stdout:6/28: mkdir d0/d5 0 2024-03-24T10:45:51.835 INFO:tasks.workunit.client.0.smithi094.stdout:8/51: write f3 [4606031,19669] 0 2024-03-24T10:45:51.837 INFO:tasks.workunit.client.0.smithi094.stdout:8/52: mknod ce 0 2024-03-24T10:45:51.837 INFO:tasks.workunit.client.0.smithi094.stdout:8/53: dread - fd zero size 2024-03-24T10:45:51.838 INFO:tasks.workunit.client.0.smithi094.stdout:8/54: mkdir df 0 2024-03-24T10:45:51.839 INFO:tasks.workunit.client.0.smithi094.stdout:8/55: symlink df/l10 0 2024-03-24T10:45:51.840 INFO:tasks.workunit.client.0.smithi094.stdout:8/56: write fc [362105,98014] 0 2024-03-24T10:45:51.842 INFO:tasks.workunit.client.1.smithi096.stdout:3/42: dwrite d0/f1 [0,4194304] 0 2024-03-24T10:45:51.844 INFO:tasks.workunit.client.1.smithi096.stdout:3/43: rmdir d0/d9 39 2024-03-24T10:45:51.844 INFO:tasks.workunit.client.1.smithi096.stdout:3/44: creat d0/ff x:0 0 0 2024-03-24T10:45:51.845 INFO:tasks.workunit.client.1.smithi096.stdout:3/45: stat d0/d9/le 0 2024-03-24T10:45:51.845 INFO:tasks.workunit.client.1.smithi096.stdout:3/46: fsync d0/f1 0 2024-03-24T10:45:51.847 INFO:tasks.workunit.client.0.smithi094.stdout:1/21: rename d4/f5 to d4/f7 0 2024-03-24T10:45:51.847 INFO:tasks.workunit.client.0.smithi094.stdout:4/8: sync 2024-03-24T10:45:51.847 INFO:tasks.workunit.client.0.smithi094.stdout:4/9: write - no filename 2024-03-24T10:45:51.847 INFO:tasks.workunit.client.0.smithi094.stdout:4/10: dwrite - no filename 2024-03-24T10:45:51.847 INFO:tasks.workunit.client.0.smithi094.stdout:4/11: rmdir - no directory 2024-03-24T10:45:51.847 INFO:tasks.workunit.client.0.smithi094.stdout:4/12: read - no filename 2024-03-24T10:45:51.847 INFO:tasks.workunit.client.0.smithi094.stdout:2/10: sync 2024-03-24T10:45:51.847 INFO:tasks.workunit.client.0.smithi094.stdout:4/13: fdatasync - no filename 2024-03-24T10:45:51.848 INFO:tasks.workunit.client.0.smithi094.stdout:4/14: dwrite - no filename 2024-03-24T10:45:51.848 INFO:tasks.workunit.client.0.smithi094.stdout:2/11: dread - no filename 2024-03-24T10:45:51.848 INFO:tasks.workunit.client.0.smithi094.stdout:4/15: truncate - no filename 2024-03-24T10:45:51.848 INFO:tasks.workunit.client.0.smithi094.stdout:4/16: link - no file 2024-03-24T10:45:51.848 INFO:tasks.workunit.client.0.smithi094.stdout:4/17: dread - no filename 2024-03-24T10:45:51.848 INFO:tasks.workunit.client.0.smithi094.stdout:4/18: chown . 226 1 2024-03-24T10:45:51.848 INFO:tasks.workunit.client.0.smithi094.stdout:4/19: rename - no filename 2024-03-24T10:45:51.848 INFO:tasks.workunit.client.0.smithi094.stdout:4/20: dwrite - no filename 2024-03-24T10:45:51.848 INFO:tasks.workunit.client.0.smithi094.stdout:4/21: truncate - no filename 2024-03-24T10:45:51.848 INFO:tasks.workunit.client.0.smithi094.stdout:4/22: rename - no filename 2024-03-24T10:45:51.848 INFO:tasks.workunit.client.0.smithi094.stdout:4/23: rename - no filename 2024-03-24T10:45:51.848 INFO:tasks.workunit.client.0.smithi094.stdout:4/24: stat - no entries 2024-03-24T10:45:51.848 INFO:tasks.workunit.client.0.smithi094.stdout:4/25: write - no filename 2024-03-24T10:45:51.848 INFO:tasks.workunit.client.0.smithi094.stdout:4/26: dwrite - no filename 2024-03-24T10:45:51.849 INFO:tasks.workunit.client.0.smithi094.stdout:4/27: rename - no filename 2024-03-24T10:45:51.849 INFO:tasks.workunit.client.0.smithi094.stdout:4/28: read - no filename 2024-03-24T10:45:51.849 INFO:tasks.workunit.client.0.smithi094.stdout:4/29: dread - no filename 2024-03-24T10:45:51.849 INFO:tasks.workunit.client.0.smithi094.stdout:4/30: write - no filename 2024-03-24T10:45:51.849 INFO:tasks.workunit.client.0.smithi094.stdout:1/22: creat d4/f8 x:0 0 0 2024-03-24T10:45:51.850 INFO:tasks.workunit.client.0.smithi094.stdout:1/23: write d4/f8 [1015113,92380] 0 2024-03-24T10:45:51.850 INFO:tasks.workunit.client.0.smithi094.stdout:2/12: stat l1 0 2024-03-24T10:45:51.850 INFO:tasks.workunit.client.0.smithi094.stdout:2/13: chown l2 517942 1 2024-03-24T10:45:51.850 INFO:tasks.workunit.client.0.smithi094.stdout:2/14: stat l1 0 2024-03-24T10:45:51.850 INFO:tasks.workunit.client.0.smithi094.stdout:2/15: write - no filename 2024-03-24T10:45:51.851 INFO:tasks.workunit.client.0.smithi094.stdout:3/35: fsync d0/d7/f9 0 2024-03-24T10:45:51.851 INFO:tasks.workunit.client.0.smithi094.stdout:4/31: mkdir d0 0 2024-03-24T10:45:51.851 INFO:tasks.workunit.client.0.smithi094.stdout:4/32: read - no filename 2024-03-24T10:45:51.851 INFO:tasks.workunit.client.0.smithi094.stdout:4/33: dread - no filename 2024-03-24T10:45:51.852 INFO:tasks.workunit.client.0.smithi094.stdout:4/34: rename d0 to d0/d1 22 2024-03-24T10:45:51.852 INFO:tasks.workunit.client.0.smithi094.stdout:4/35: write - no filename 2024-03-24T10:45:51.852 INFO:tasks.workunit.client.0.smithi094.stdout:4/36: write - no filename 2024-03-24T10:45:51.852 INFO:tasks.workunit.client.0.smithi094.stdout:4/37: readlink - no filename 2024-03-24T10:45:51.852 INFO:tasks.workunit.client.0.smithi094.stdout:4/38: dwrite - no filename 2024-03-24T10:45:51.852 INFO:tasks.workunit.client.0.smithi094.stdout:4/39: chown d0 1260890644 1 2024-03-24T10:45:51.852 INFO:tasks.workunit.client.0.smithi094.stdout:4/40: dwrite - no filename 2024-03-24T10:45:51.852 INFO:tasks.workunit.client.0.smithi094.stdout:4/41: chown d0 61 1 2024-03-24T10:45:51.852 INFO:tasks.workunit.client.0.smithi094.stdout:4/42: chown d0 75252770 1 2024-03-24T10:45:51.852 INFO:tasks.workunit.client.0.smithi094.stdout:1/24: mkdir d4/d9 0 2024-03-24T10:45:51.854 INFO:tasks.workunit.client.0.smithi094.stdout:2/16: symlink l4 0 2024-03-24T10:45:51.855 INFO:tasks.workunit.client.0.smithi094.stdout:2/17: chown l4 6 1 2024-03-24T10:45:51.855 INFO:tasks.workunit.client.0.smithi094.stdout:2/18: write - no filename 2024-03-24T10:45:51.855 INFO:tasks.workunit.client.0.smithi094.stdout:2/19: readlink l2 0 2024-03-24T10:45:51.855 INFO:tasks.workunit.client.0.smithi094.stdout:2/20: dread - no filename 2024-03-24T10:45:51.855 INFO:tasks.workunit.client.0.smithi094.stdout:2/21: dread - no filename 2024-03-24T10:45:51.855 INFO:tasks.workunit.client.0.smithi094.stdout:3/36: mknod d0/d7/ca 0 2024-03-24T10:45:51.856 INFO:tasks.workunit.client.0.smithi094.stdout:4/43: mknod d0/c2 0 2024-03-24T10:45:51.859 INFO:tasks.workunit.client.0.smithi094.stdout:2/22: link l1 l5 0 2024-03-24T10:45:51.859 INFO:tasks.workunit.client.0.smithi094.stdout:2/23: chown l5 1439 1 2024-03-24T10:45:51.860 INFO:tasks.workunit.client.0.smithi094.stdout:9/26: sync 2024-03-24T10:45:51.860 INFO:tasks.workunit.client.0.smithi094.stdout:9/27: stat c2 0 2024-03-24T10:45:51.860 INFO:tasks.workunit.client.0.smithi094.stdout:9/28: rmdir - no directory 2024-03-24T10:45:51.860 INFO:tasks.workunit.client.0.smithi094.stdout:9/29: chown c2 124 1 2024-03-24T10:45:51.861 INFO:tasks.workunit.client.0.smithi094.stdout:4/44: creat d0/f3 x:0 0 0 2024-03-24T10:45:51.861 INFO:tasks.workunit.client.0.smithi094.stdout:7/23: rmdir d1 39 2024-03-24T10:45:51.862 INFO:tasks.workunit.client.1.smithi096.stdout:3/47: dwrite d0/ff [0,4194304] 0 2024-03-24T10:45:51.864 INFO:tasks.workunit.client.0.smithi094.stdout:2/24: rename l4 to l6 0 2024-03-24T10:45:51.864 INFO:tasks.workunit.client.0.smithi094.stdout:2/25: rmdir - no directory 2024-03-24T10:45:51.866 INFO:tasks.workunit.client.0.smithi094.stdout:9/30: mknod c5 0 2024-03-24T10:45:51.866 INFO:tasks.workunit.client.0.smithi094.stdout:9/31: write - no filename 2024-03-24T10:45:51.866 INFO:tasks.workunit.client.0.smithi094.stdout:9/32: dwrite - no filename 2024-03-24T10:45:51.866 INFO:tasks.workunit.client.0.smithi094.stdout:9/33: truncate - no filename 2024-03-24T10:45:51.867 INFO:tasks.workunit.client.0.smithi094.stdout:4/45: mknod d0/c4 0 2024-03-24T10:45:51.869 INFO:tasks.workunit.client.0.smithi094.stdout:7/24: symlink d1/d9/la 0 2024-03-24T10:45:51.872 INFO:tasks.workunit.client.0.smithi094.stdout:2/26: unlink l2 0 2024-03-24T10:45:51.872 INFO:tasks.workunit.client.0.smithi094.stdout:2/27: write - no filename 2024-03-24T10:45:51.873 INFO:tasks.workunit.client.0.smithi094.stdout:9/34: rename l4 to l6 0 2024-03-24T10:45:51.874 INFO:tasks.workunit.client.0.smithi094.stdout:7/25: symlink d1/d9/lb 0 2024-03-24T10:45:51.875 INFO:tasks.workunit.client.0.smithi094.stdout:2/28: stat l5 0 2024-03-24T10:45:51.876 INFO:tasks.workunit.client.0.smithi094.stdout:9/35: link c5 c7 0 2024-03-24T10:45:51.876 INFO:tasks.workunit.client.0.smithi094.stdout:9/36: write - no filename 2024-03-24T10:45:51.877 INFO:tasks.workunit.client.0.smithi094.stdout:8/57: getdents df 0 2024-03-24T10:45:51.878 INFO:tasks.workunit.client.0.smithi094.stdout:7/26: dread d1/f3 [0,4194304] 0 2024-03-24T10:45:51.878 INFO:tasks.workunit.client.0.smithi094.stdout:7/27: dread - d1/f7 zero size 2024-03-24T10:45:51.878 INFO:tasks.workunit.client.0.smithi094.stdout:9/37: mknod c8 0 2024-03-24T10:45:51.878 INFO:tasks.workunit.client.0.smithi094.stdout:9/38: truncate - no filename 2024-03-24T10:45:51.879 INFO:tasks.workunit.client.0.smithi094.stdout:9/39: fdatasync - no filename 2024-03-24T10:45:51.879 INFO:tasks.workunit.client.0.smithi094.stdout:9/40: dwrite - no filename 2024-03-24T10:45:51.879 INFO:tasks.workunit.client.0.smithi094.stdout:9/41: write - no filename 2024-03-24T10:45:51.881 INFO:tasks.workunit.client.0.smithi094.stdout:8/58: link fc df/f11 0 2024-03-24T10:45:51.881 INFO:tasks.workunit.client.0.smithi094.stdout:8/59: write f6 [203825,123516] 0 2024-03-24T10:45:51.882 INFO:tasks.workunit.client.1.smithi096.stdout:3/48: truncate d0/f1 763292 0 2024-03-24T10:45:51.885 INFO:tasks.workunit.client.0.smithi094.stdout:9/42: mknod c9 0 2024-03-24T10:45:51.887 INFO:tasks.workunit.client.1.smithi096.stdout:3/49: mknod d0/c10 0 2024-03-24T10:45:51.888 INFO:tasks.workunit.client.0.smithi094.stdout:9/43: mknod ca 0 2024-03-24T10:45:51.889 INFO:tasks.workunit.client.0.smithi094.stdout:1/25: truncate d4/f8 1022704 0 2024-03-24T10:45:51.889 INFO:tasks.workunit.client.0.smithi094.stdout:1/26: chown d4/l6 9 1 2024-03-24T10:45:51.890 INFO:tasks.workunit.client.0.smithi094.stdout:8/60: mkdir df/d12 0 2024-03-24T10:45:51.891 INFO:tasks.workunit.client.0.smithi094.stdout:9/44: symlink lb 0 2024-03-24T10:45:51.892 INFO:tasks.workunit.client.0.smithi094.stdout:7/28: dwrite f0 [0,4194304] 0 2024-03-24T10:45:51.893 INFO:tasks.workunit.client.0.smithi094.stdout:3/37: rename d0/d7 to d0/db 0 2024-03-24T10:45:51.893 INFO:tasks.workunit.client.0.smithi094.stdout:3/38: chown d0/f3 36491 1 2024-03-24T10:45:51.895 INFO:tasks.workunit.client.0.smithi094.stdout:9/45: rename ca to cc 0 2024-03-24T10:45:51.896 INFO:tasks.workunit.client.0.smithi094.stdout:9/46: dwrite - no filename 2024-03-24T10:45:51.897 INFO:tasks.workunit.client.0.smithi094.stdout:7/29: symlink d1/d9/lc 0 2024-03-24T10:45:51.897 INFO:tasks.workunit.client.0.smithi094.stdout:7/30: chown f0 2499 1 2024-03-24T10:45:51.897 INFO:tasks.workunit.client.0.smithi094.stdout:7/31: stat d1/l8 0 2024-03-24T10:45:51.898 INFO:tasks.workunit.client.0.smithi094.stdout:3/39: mkdir d0/db/dc 0 2024-03-24T10:45:51.898 INFO:tasks.workunit.client.0.smithi094.stdout:3/40: fsync d0/db/f9 0 2024-03-24T10:45:51.899 INFO:tasks.workunit.client.0.smithi094.stdout:8/61: rename c9 to df/c13 0 2024-03-24T10:45:51.899 INFO:tasks.workunit.client.0.smithi094.stdout:8/62: rename df to df/d12/d14 22 2024-03-24T10:45:51.900 INFO:tasks.workunit.client.0.smithi094.stdout:9/47: mknod cd 0 2024-03-24T10:45:51.900 INFO:tasks.workunit.client.0.smithi094.stdout:9/48: dread - no filename 2024-03-24T10:45:51.900 INFO:tasks.workunit.client.0.smithi094.stdout:9/49: creat fe x:0 0 0 2024-03-24T10:45:51.900 INFO:tasks.workunit.client.0.smithi094.stdout:9/50: write fe [554449,32350] 0 2024-03-24T10:45:51.901 INFO:tasks.workunit.client.0.smithi094.stdout:4/46: getdents d0 0 2024-03-24T10:45:51.901 INFO:tasks.workunit.client.0.smithi094.stdout:7/32: creat d1/d9/fd x:0 0 0 2024-03-24T10:45:51.901 INFO:tasks.workunit.client.0.smithi094.stdout:3/41: dread d0/db/f9 [0,4194304] 0 2024-03-24T10:45:51.902 INFO:tasks.workunit.client.0.smithi094.stdout:8/63: stat df/c13 0 2024-03-24T10:45:51.905 INFO:tasks.workunit.client.0.smithi094.stdout:2/29: getdents . 0 2024-03-24T10:45:51.905 INFO:tasks.workunit.client.0.smithi094.stdout:2/30: fdatasync - no filename 2024-03-24T10:45:51.905 INFO:tasks.workunit.client.0.smithi094.stdout:2/31: dwrite - no filename 2024-03-24T10:45:51.906 INFO:tasks.workunit.client.0.smithi094.stdout:4/47: mkdir d0/d5 0 2024-03-24T10:45:51.907 INFO:tasks.workunit.client.0.smithi094.stdout:3/42: symlink d0/db/ld 0 2024-03-24T10:45:51.910 INFO:tasks.workunit.client.0.smithi094.stdout:3/43: dread d0/f3 [0,4194304] 0 2024-03-24T10:45:51.912 INFO:tasks.workunit.client.0.smithi094.stdout:2/32: creat f7 x:0 0 0 2024-03-24T10:45:51.912 INFO:tasks.workunit.client.0.smithi094.stdout:2/33: creat f8 x:0 0 0 2024-03-24T10:45:51.912 INFO:tasks.workunit.client.0.smithi094.stdout:2/34: dread - f7 zero size 2024-03-24T10:45:51.913 INFO:tasks.workunit.client.0.smithi094.stdout:4/48: creat d0/f6 x:0 0 0 2024-03-24T10:45:51.913 INFO:tasks.workunit.client.0.smithi094.stdout:4/49: chown d0/f3 103860 1 2024-03-24T10:45:51.913 INFO:tasks.workunit.client.0.smithi094.stdout:4/50: write d0/f3 [457644,52276] 0 2024-03-24T10:45:51.914 INFO:tasks.workunit.client.0.smithi094.stdout:3/44: mkdir d0/db/de 0 2024-03-24T10:45:51.916 INFO:tasks.workunit.client.0.smithi094.stdout:3/45: write d0/f3 [576929,41512] 0 2024-03-24T10:45:51.916 INFO:tasks.workunit.client.0.smithi094.stdout:4/51: link d0/f6 d0/f7 0 2024-03-24T10:45:51.917 INFO:tasks.workunit.client.0.smithi094.stdout:4/52: write d0/f3 [291616,41418] 0 2024-03-24T10:45:51.919 INFO:tasks.workunit.client.0.smithi094.stdout:4/53: link d0/c4 d0/c8 0 2024-03-24T10:45:51.919 INFO:tasks.workunit.client.0.smithi094.stdout:4/54: chown d0/f7 0 1 2024-03-24T10:45:51.919 INFO:tasks.workunit.client.0.smithi094.stdout:4/55: dread - d0/f7 zero size 2024-03-24T10:45:51.920 INFO:tasks.workunit.client.0.smithi094.stdout:1/27: dread d4/f8 [0,4194304] 0 2024-03-24T10:45:51.920 INFO:tasks.workunit.client.0.smithi094.stdout:4/56: mkdir d0/d5/d9 0 2024-03-24T10:45:51.920 INFO:tasks.workunit.client.0.smithi094.stdout:4/57: truncate d0/f3 1193144 0 2024-03-24T10:45:51.920 INFO:tasks.workunit.client.0.smithi094.stdout:4/58: write d0/f6 [196805,52546] 0 2024-03-24T10:45:51.920 INFO:tasks.workunit.client.0.smithi094.stdout:4/59: chown d0 0 1 2024-03-24T10:45:51.920 INFO:tasks.workunit.client.0.smithi094.stdout:4/60: chown d0/f3 23207225 1 2024-03-24T10:45:51.923 INFO:tasks.workunit.client.0.smithi094.stdout:7/33: dwrite d1/d9/fd [0,4194304] 0 2024-03-24T10:45:51.923 INFO:tasks.workunit.client.0.smithi094.stdout:9/51: dwrite fe [0,4194304] 0 2024-03-24T10:45:51.923 INFO:tasks.workunit.client.0.smithi094.stdout:7/34: creat d1/d9/fe x:0 0 0 2024-03-24T10:45:51.923 INFO:tasks.workunit.client.1.smithi096.stdout:3/50: write d0/f1 [89080,4801] 0 2024-03-24T10:45:51.925 INFO:tasks.workunit.client.1.smithi096.stdout:3/51: rename d0/c8 to d0/c11 0 2024-03-24T10:45:51.933 INFO:tasks.workunit.client.0.smithi094.stdout:9/52: fsync fe 0 2024-03-24T10:45:51.935 INFO:tasks.workunit.client.0.smithi094.stdout:7/35: symlink d1/d9/lf 0 2024-03-24T10:45:51.935 INFO:tasks.workunit.client.0.smithi094.stdout:7/36: dread - d1/f7 zero size 2024-03-24T10:45:51.937 INFO:tasks.workunit.client.0.smithi094.stdout:7/37: unlink d1/f3 0 2024-03-24T10:45:51.942 INFO:tasks.workunit.client.0.smithi094.stdout:3/46: dwrite d0/db/f9 [0,4194304] 0 2024-03-24T10:45:51.943 INFO:tasks.workunit.client.0.smithi094.stdout:4/61: rmdir d0 39 2024-03-24T10:45:51.948 INFO:tasks.workunit.client.0.smithi094.stdout:4/62: fdatasync d0/f6 0 2024-03-24T10:45:51.948 INFO:tasks.workunit.client.0.smithi094.stdout:1/28: dwrite d4/f8 [0,4194304] 0 2024-03-24T10:45:51.950 INFO:tasks.workunit.client.1.smithi096.stdout:3/52: dwrite d0/ff [0,4194304] 0 2024-03-24T10:45:51.950 INFO:tasks.workunit.client.1.smithi096.stdout:3/53: chown d0/c10 550034 1 2024-03-24T10:45:51.951 INFO:tasks.workunit.client.0.smithi094.stdout:4/63: unlink d0/f3 0 2024-03-24T10:45:51.951 INFO:tasks.workunit.client.0.smithi094.stdout:4/64: write d0/f7 [663159,27449] 0 2024-03-24T10:45:51.952 INFO:tasks.workunit.client.1.smithi096.stdout:3/54: mknod d0/c12 0 2024-03-24T10:45:51.952 INFO:tasks.workunit.client.1.smithi096.stdout:3/55: read d0/f7 [714072,124370] 0 2024-03-24T10:45:51.954 INFO:tasks.workunit.client.0.smithi094.stdout:3/47: dread d0/db/f9 [0,4194304] 0 2024-03-24T10:45:51.954 INFO:tasks.workunit.client.0.smithi094.stdout:4/65: symlink d0/la 0 2024-03-24T10:45:51.955 INFO:tasks.workunit.client.0.smithi094.stdout:4/66: read d0/f7 [426494,65768] 0 2024-03-24T10:45:51.955 INFO:tasks.workunit.client.0.smithi094.stdout:4/67: write d0/f6 [1471840,35224] 0 2024-03-24T10:45:51.955 INFO:tasks.workunit.client.0.smithi094.stdout:1/29: dread d4/f7 [0,4194304] 0 2024-03-24T10:45:51.956 INFO:tasks.workunit.client.0.smithi094.stdout:4/68: symlink d0/d5/lb 0 2024-03-24T10:45:51.957 INFO:tasks.workunit.client.0.smithi094.stdout:3/48: write d0/db/f9 [1444037,24684] 0 2024-03-24T10:45:51.957 INFO:tasks.workunit.client.0.smithi094.stdout:4/69: symlink d0/d5/d9/lc 0 2024-03-24T10:45:51.958 INFO:tasks.workunit.client.0.smithi094.stdout:4/70: rename d0/d5/lb to d0/d5/ld 0 2024-03-24T10:45:51.958 INFO:tasks.workunit.client.0.smithi094.stdout:4/71: chown d0/c4 106663970 1 2024-03-24T10:45:51.959 INFO:tasks.workunit.client.0.smithi094.stdout:1/30: dread d4/f7 [0,4194304] 0 2024-03-24T10:45:51.959 INFO:tasks.workunit.client.0.smithi094.stdout:1/31: stat d4/f8 0 2024-03-24T10:45:51.959 INFO:tasks.workunit.client.0.smithi094.stdout:1/32: chown f1 1 1 2024-03-24T10:45:51.959 INFO:tasks.workunit.client.0.smithi094.stdout:1/33: fsync f1 0 2024-03-24T10:45:51.960 INFO:tasks.workunit.client.0.smithi094.stdout:1/34: creat d4/d9/fa x:0 0 0 2024-03-24T10:45:51.966 INFO:tasks.workunit.client.1.smithi096.stdout:3/56: write d0/f4 [607755,38218] 0 2024-03-24T10:45:51.967 INFO:tasks.workunit.client.1.smithi096.stdout:3/57: mkdir d0/d13 0 2024-03-24T10:45:51.970 INFO:tasks.workunit.client.0.smithi094.stdout:9/53: dwrite fe [0,4194304] 0 2024-03-24T10:45:51.972 INFO:tasks.workunit.client.0.smithi094.stdout:9/54: mkdir df 0 2024-03-24T10:45:51.972 INFO:tasks.workunit.client.0.smithi094.stdout:9/55: chown df 12703 1 2024-03-24T10:45:51.972 INFO:tasks.workunit.client.0.smithi094.stdout:9/56: mkdir df/d10 0 2024-03-24T10:45:52.031 INFO:tasks.workunit.client.0.smithi094.stdout:7/38: dwrite d1/f7 [0,4194304] 0 2024-03-24T10:45:52.037 INFO:tasks.workunit.client.0.smithi094.stdout:4/72: dread d0/f7 [0,4194304] 0 2024-03-24T10:45:52.038 INFO:tasks.workunit.client.0.smithi094.stdout:7/39: dread d1/d9/fd [0,4194304] 0 2024-03-24T10:45:52.038 INFO:tasks.workunit.client.0.smithi094.stdout:4/73: mkdir d0/d5/d9/de 0 2024-03-24T10:45:52.038 INFO:tasks.workunit.client.0.smithi094.stdout:4/74: write d0/f6 [1769983,1992] 0 2024-03-24T10:45:52.040 INFO:tasks.workunit.client.0.smithi094.stdout:3/49: dwrite d0/f3 [0,4194304] 0 2024-03-24T10:45:52.042 INFO:tasks.workunit.client.0.smithi094.stdout:1/35: dwrite d4/f7 [4194304,4194304] 0 2024-03-24T10:45:52.042 INFO:tasks.workunit.client.0.smithi094.stdout:9/57: dwrite fe [0,4194304] 0 2024-03-24T10:45:52.042 INFO:tasks.workunit.client.0.smithi094.stdout:4/75: fdatasync d0/f7 0 2024-03-24T10:45:52.045 INFO:tasks.workunit.client.0.smithi094.stdout:1/36: mkdir d4/d9/db 0 2024-03-24T10:45:52.045 INFO:tasks.workunit.client.0.smithi094.stdout:1/37: creat d4/fc x:0 0 0 2024-03-24T10:45:52.045 INFO:tasks.workunit.client.0.smithi094.stdout:1/38: chown f1 0 1 2024-03-24T10:45:52.048 INFO:tasks.workunit.client.0.smithi094.stdout:9/58: mkdir df/d10/d11 0 2024-03-24T10:45:52.050 INFO:tasks.workunit.client.0.smithi094.stdout:4/76: creat d0/d5/d9/ff x:0 0 0 2024-03-24T10:45:52.052 INFO:tasks.workunit.client.0.smithi094.stdout:4/77: mknod d0/d5/d9/c10 0 2024-03-24T10:45:52.052 INFO:tasks.workunit.client.0.smithi094.stdout:4/78: truncate d0/f7 2034265 0 2024-03-24T10:45:52.052 INFO:tasks.workunit.client.0.smithi094.stdout:4/79: write d0/d5/d9/ff [1020000,97248] 0 2024-03-24T10:45:52.052 INFO:tasks.workunit.client.0.smithi094.stdout:4/80: creat d0/d5/d9/f11 x:0 0 0 2024-03-24T10:45:52.052 INFO:tasks.workunit.client.0.smithi094.stdout:4/81: write d0/f7 [218632,39008] 0 2024-03-24T10:45:52.055 INFO:tasks.workunit.client.0.smithi094.stdout:9/59: dread fe [0,4194304] 0 2024-03-24T10:45:52.056 INFO:tasks.workunit.client.1.smithi096.stdout:3/58: dwrite d0/f7 [0,4194304] 0 2024-03-24T10:45:52.058 INFO:tasks.workunit.client.0.smithi094.stdout:9/60: dread fe [0,4194304] 0 2024-03-24T10:45:52.058 INFO:tasks.workunit.client.0.smithi094.stdout:4/82: link d0/d5/d9/ff d0/d5/d9/f12 0 2024-03-24T10:45:52.059 INFO:tasks.workunit.client.0.smithi094.stdout:4/83: truncate d0/f7 3056097 0 2024-03-24T10:45:52.059 INFO:tasks.workunit.client.0.smithi094.stdout:4/84: rename d0/d5/d9 to d0/d5/d9/d13 22 2024-03-24T10:45:52.061 INFO:tasks.workunit.client.0.smithi094.stdout:9/61: unlink cd 0 2024-03-24T10:45:52.062 INFO:tasks.workunit.client.0.smithi094.stdout:4/85: unlink d0/c4 0 2024-03-24T10:45:52.063 INFO:tasks.workunit.client.0.smithi094.stdout:9/62: mkdir df/d12 0 2024-03-24T10:45:52.075 INFO:tasks.workunit.client.0.smithi094.stdout:1/39: truncate d4/f7 8210790 0 2024-03-24T10:45:52.075 INFO:tasks.workunit.client.0.smithi094.stdout:1/40: creat d4/d9/fd x:0 0 0 2024-03-24T10:45:52.075 INFO:tasks.workunit.client.0.smithi094.stdout:1/41: readlink d4/l6 0 2024-03-24T10:45:52.075 INFO:tasks.workunit.client.0.smithi094.stdout:1/42: truncate d4/d9/fd 340565 0 2024-03-24T10:45:52.075 INFO:tasks.workunit.client.0.smithi094.stdout:6/29: sync 2024-03-24T10:45:52.075 INFO:tasks.workunit.client.0.smithi094.stdout:5/41: sync 2024-03-24T10:45:52.075 INFO:tasks.workunit.client.0.smithi094.stdout:6/30: chown d0/c4 165 1 2024-03-24T10:45:52.075 INFO:tasks.workunit.client.0.smithi094.stdout:6/31: fdatasync d0/f3 0 2024-03-24T10:45:52.075 INFO:tasks.workunit.client.0.smithi094.stdout:6/32: truncate d0/f3 216684 0 2024-03-24T10:45:52.076 INFO:tasks.workunit.client.0.smithi094.stdout:0/28: sync 2024-03-24T10:45:52.076 INFO:tasks.workunit.client.0.smithi094.stdout:0/29: creat fa x:0 0 0 2024-03-24T10:45:52.076 INFO:tasks.workunit.client.0.smithi094.stdout:0/30: stat f5 0 2024-03-24T10:45:52.077 INFO:tasks.workunit.client.0.smithi094.stdout:5/42: mkdir d7/de 0 2024-03-24T10:45:52.079 INFO:tasks.workunit.client.0.smithi094.stdout:0/31: symlink lb 0 2024-03-24T10:45:52.079 INFO:tasks.workunit.client.0.smithi094.stdout:0/32: creat fc x:0 0 0 2024-03-24T10:45:52.080 INFO:tasks.workunit.client.0.smithi094.stdout:5/43: mkdir d7/de/df 0 2024-03-24T10:45:52.080 INFO:tasks.workunit.client.0.smithi094.stdout:5/44: chown d7/d8 0 1 2024-03-24T10:45:52.082 INFO:tasks.workunit.client.0.smithi094.stdout:5/45: creat d7/f10 x:0 0 0 2024-03-24T10:45:52.082 INFO:tasks.workunit.client.0.smithi094.stdout:5/46: write d7/f10 [882873,71699] 0 2024-03-24T10:45:52.083 INFO:tasks.workunit.client.0.smithi094.stdout:6/33: write d0/f2 [1474030,77210] 0 2024-03-24T10:45:52.084 INFO:tasks.workunit.client.0.smithi094.stdout:6/34: symlink d0/l6 0 2024-03-24T10:45:52.084 INFO:tasks.workunit.client.0.smithi094.stdout:6/35: readlink d0/l6 0 2024-03-24T10:45:52.085 INFO:tasks.workunit.client.0.smithi094.stdout:7/40: dwrite d1/f7 [0,4194304] 0 2024-03-24T10:45:52.085 INFO:tasks.workunit.client.0.smithi094.stdout:3/50: dwrite d0/f2 [0,4194304] 0 2024-03-24T10:45:52.085 INFO:tasks.workunit.client.0.smithi094.stdout:7/41: chown d1/l4 1971 1 2024-03-24T10:45:52.085 INFO:tasks.workunit.client.0.smithi094.stdout:7/42: chown f0 52049 1 2024-03-24T10:45:52.087 INFO:tasks.workunit.client.0.smithi094.stdout:5/47: write d7/d8/fa [1634053,2719] 0 2024-03-24T10:45:52.089 INFO:tasks.workunit.client.1.smithi096.stdout:3/59: dread d0/f4 [0,4194304] 0 2024-03-24T10:45:52.090 INFO:tasks.workunit.client.0.smithi094.stdout:4/86: dread d0/d5/d9/ff [0,4194304] 0 2024-03-24T10:45:52.090 INFO:tasks.workunit.client.0.smithi094.stdout:9/63: dwrite fe [4194304,4194304] 0 2024-03-24T10:45:52.091 INFO:tasks.workunit.client.0.smithi094.stdout:4/87: write d0/f7 [1315445,105780] 0 2024-03-24T10:45:52.093 INFO:tasks.workunit.client.1.smithi096.stdout:3/60: dread d0/f4 [0,4194304] 0 2024-03-24T10:45:52.096 INFO:tasks.workunit.client.0.smithi094.stdout:1/43: dwrite d4/d9/fd [0,4194304] 0 2024-03-24T10:45:52.096 INFO:tasks.workunit.client.0.smithi094.stdout:1/44: dread - d4/d9/fa zero size 2024-03-24T10:45:52.096 INFO:tasks.workunit.client.1.smithi096.stdout:3/61: mknod d0/d13/c14 0 2024-03-24T10:45:52.099 INFO:tasks.workunit.client.0.smithi094.stdout:3/51: creat d0/db/de/ff x:0 0 0 2024-03-24T10:45:52.099 INFO:tasks.workunit.client.0.smithi094.stdout:3/52: creat d0/f10 x:0 0 0 2024-03-24T10:45:52.099 INFO:tasks.workunit.client.1.smithi096.stdout:3/62: symlink d0/l15 0 2024-03-24T10:45:52.101 INFO:tasks.workunit.client.0.smithi094.stdout:7/43: unlink d1/l4 0 2024-03-24T10:45:52.104 INFO:tasks.workunit.client.0.smithi094.stdout:5/48: link d7/d8/d9/cd d7/c11 0 2024-03-24T10:45:52.104 INFO:tasks.workunit.client.0.smithi094.stdout:5/49: write d7/d8/fa [2353986,92885] 0 2024-03-24T10:45:52.105 INFO:tasks.workunit.client.0.smithi094.stdout:0/33: dwrite f0 [0,4194304] 0 2024-03-24T10:45:52.109 INFO:tasks.workunit.client.0.smithi094.stdout:9/64: write fe [4067229,130609] 0 2024-03-24T10:45:52.113 INFO:tasks.workunit.client.0.smithi094.stdout:4/88: symlink d0/l14 0 2024-03-24T10:45:52.115 INFO:tasks.workunit.client.0.smithi094.stdout:3/53: rename d0/db/f9 to d0/f11 0 2024-03-24T10:45:52.115 INFO:tasks.workunit.client.0.smithi094.stdout:3/54: fdatasync d0/f2 0 2024-03-24T10:45:52.115 INFO:tasks.workunit.client.0.smithi094.stdout:7/44: creat d1/f10 x:0 0 0 2024-03-24T10:45:52.117 INFO:tasks.workunit.client.0.smithi094.stdout:9/65: creat df/d10/d11/f13 x:0 0 0 2024-03-24T10:45:52.120 INFO:tasks.workunit.client.0.smithi094.stdout:7/45: mknod d1/c11 0 2024-03-24T10:45:52.120 INFO:tasks.workunit.client.0.smithi094.stdout:7/46: creat d1/d9/f12 x:0 0 0 2024-03-24T10:45:52.123 INFO:tasks.workunit.client.0.smithi094.stdout:8/64: sync 2024-03-24T10:45:52.124 INFO:tasks.workunit.client.0.smithi094.stdout:8/65: link fa df/d12/f15 0 2024-03-24T10:45:52.126 INFO:tasks.workunit.client.0.smithi094.stdout:8/66: creat df/f16 x:0 0 0 2024-03-24T10:45:52.126 INFO:tasks.workunit.client.0.smithi094.stdout:8/67: truncate fc 1099292 0 2024-03-24T10:45:52.128 INFO:tasks.workunit.client.0.smithi094.stdout:3/55: dread d0/f2 [0,4194304] 0 2024-03-24T10:45:52.134 INFO:tasks.workunit.client.0.smithi094.stdout:8/68: dread f3 [0,4194304] 0 2024-03-24T10:45:52.136 INFO:tasks.workunit.client.0.smithi094.stdout:8/69: mknod df/d12/c17 0 2024-03-24T10:45:52.136 INFO:tasks.workunit.client.0.smithi094.stdout:8/70: chown f4 0 1 2024-03-24T10:45:52.136 INFO:tasks.workunit.client.0.smithi094.stdout:8/71: write fa [281220,60795] 0 2024-03-24T10:45:52.136 INFO:tasks.workunit.client.0.smithi094.stdout:1/45: dwrite d4/d9/fa [0,4194304] 0 2024-03-24T10:45:52.136 INFO:tasks.workunit.client.0.smithi094.stdout:1/46: read - d4/fc zero size 2024-03-24T10:45:52.136 INFO:tasks.workunit.client.0.smithi094.stdout:6/36: dwrite d0/f2 [0,4194304] 0 2024-03-24T10:45:52.137 INFO:tasks.workunit.client.1.smithi096.stdout:3/63: dwrite d0/f4 [0,4194304] 0 2024-03-24T10:45:52.137 INFO:tasks.workunit.client.1.smithi096.stdout:3/64: readlink d0/d9/le 0 2024-03-24T10:45:52.137 INFO:tasks.workunit.client.1.smithi096.stdout:3/65: read d0/f1 [653030,91946] 0 2024-03-24T10:45:52.138 INFO:tasks.workunit.client.0.smithi094.stdout:0/34: dwrite f0 [4194304,4194304] 0 2024-03-24T10:45:52.138 INFO:tasks.workunit.client.0.smithi094.stdout:6/37: dread d0/f3 [0,4194304] 0 2024-03-24T10:45:52.138 INFO:tasks.workunit.client.0.smithi094.stdout:0/35: stat f8 0 2024-03-24T10:45:52.138 INFO:tasks.workunit.client.0.smithi094.stdout:6/38: truncate d0/f3 1014083 0 2024-03-24T10:45:52.138 INFO:tasks.workunit.client.0.smithi094.stdout:0/36: creat fd x:0 0 0 2024-03-24T10:45:52.138 INFO:tasks.workunit.client.0.smithi094.stdout:7/47: dwrite d1/d9/fe [0,4194304] 0 2024-03-24T10:45:52.139 INFO:tasks.workunit.client.0.smithi094.stdout:7/48: creat d1/d9/f13 x:0 0 0 2024-03-24T10:45:52.139 INFO:tasks.workunit.client.0.smithi094.stdout:7/49: stat d1/l8 0 2024-03-24T10:45:52.142 INFO:tasks.workunit.client.0.smithi094.stdout:8/72: mkdir df/d12/d18 0 2024-03-24T10:45:52.142 INFO:tasks.workunit.client.0.smithi094.stdout:2/35: sync 2024-03-24T10:45:52.146 INFO:tasks.workunit.client.0.smithi094.stdout:5/50: dread d7/f10 [0,4194304] 0 2024-03-24T10:45:52.148 INFO:tasks.workunit.client.0.smithi094.stdout:6/39: mknod d0/c7 0 2024-03-24T10:45:52.149 INFO:tasks.workunit.client.0.smithi094.stdout:6/40: fsync d0/f3 0 2024-03-24T10:45:52.149 INFO:tasks.workunit.client.0.smithi094.stdout:6/41: creat d0/f8 x:0 0 0 2024-03-24T10:45:52.149 INFO:tasks.workunit.client.0.smithi094.stdout:3/56: dwrite d0/f2 [0,4194304] 0 2024-03-24T10:45:52.149 INFO:tasks.workunit.client.0.smithi094.stdout:3/57: truncate d0/f10 156018 0 2024-03-24T10:45:52.149 INFO:tasks.workunit.client.0.smithi094.stdout:6/42: dread d0/f3 [0,4194304] 0 2024-03-24T10:45:52.150 INFO:tasks.workunit.client.0.smithi094.stdout:0/37: symlink le 0 2024-03-24T10:45:52.156 INFO:tasks.workunit.client.0.smithi094.stdout:7/50: mkdir d1/d9/d14 0 2024-03-24T10:45:52.156 INFO:tasks.workunit.client.0.smithi094.stdout:7/51: chown d1/f10 30102 1 2024-03-24T10:45:52.157 INFO:tasks.workunit.client.0.smithi094.stdout:8/73: mknod df/d12/c19 0 2024-03-24T10:45:52.157 INFO:tasks.workunit.client.0.smithi094.stdout:8/74: fdatasync f4 0 2024-03-24T10:45:52.157 INFO:tasks.workunit.client.0.smithi094.stdout:8/75: chown fd 16974181 1 2024-03-24T10:45:52.158 INFO:tasks.workunit.client.0.smithi094.stdout:2/36: stat l5 0 2024-03-24T10:45:52.158 INFO:tasks.workunit.client.0.smithi094.stdout:3/58: write d0/f2 [3371111,21944] 0 2024-03-24T10:45:52.159 INFO:tasks.workunit.client.0.smithi094.stdout:1/47: rmdir d4/d9 39 2024-03-24T10:45:52.159 INFO:tasks.workunit.client.0.smithi094.stdout:1/48: read - d4/fc zero size 2024-03-24T10:45:52.161 INFO:tasks.workunit.client.0.smithi094.stdout:5/51: rename d7/de/df to d7/dc/d12 0 2024-03-24T10:45:52.165 INFO:tasks.workunit.client.0.smithi094.stdout:6/43: mkdir d0/d5/d9 0 2024-03-24T10:45:52.167 INFO:tasks.workunit.client.0.smithi094.stdout:9/66: dwrite fe [4194304,4194304] 0 2024-03-24T10:45:52.167 INFO:tasks.workunit.client.0.smithi094.stdout:0/38: rename lb to lf 0 2024-03-24T10:45:52.167 INFO:tasks.workunit.client.0.smithi094.stdout:4/89: dwrite d0/f7 [0,4194304] 0 2024-03-24T10:45:52.168 INFO:tasks.workunit.client.1.smithi096.stdout:3/66: truncate d0/f4 2897628 0 2024-03-24T10:45:52.171 INFO:tasks.workunit.client.0.smithi094.stdout:7/52: dread d1/f7 [0,4194304] 0 2024-03-24T10:45:52.171 INFO:tasks.workunit.client.0.smithi094.stdout:7/53: chown d1/f10 2107402114 1 2024-03-24T10:45:52.171 INFO:tasks.workunit.client.0.smithi094.stdout:7/54: chown d1/f10 1 1 2024-03-24T10:45:52.171 INFO:tasks.workunit.client.0.smithi094.stdout:6/44: dread d0/f3 [0,4194304] 0 2024-03-24T10:45:52.173 INFO:tasks.workunit.client.0.smithi094.stdout:5/52: unlink d7/c11 0 2024-03-24T10:45:52.174 INFO:tasks.workunit.client.0.smithi094.stdout:9/67: creat df/f14 x:0 0 0 2024-03-24T10:45:52.174 INFO:tasks.workunit.client.0.smithi094.stdout:1/49: write d4/f7 [4889149,129289] 0 2024-03-24T10:45:52.174 INFO:tasks.workunit.client.0.smithi094.stdout:4/90: stat d0/c8 0 2024-03-24T10:45:52.175 INFO:tasks.workunit.client.0.smithi094.stdout:3/59: dwrite d0/f10 [0,4194304] 0 2024-03-24T10:45:52.176 INFO:tasks.workunit.client.0.smithi094.stdout:2/37: getdents . 0 2024-03-24T10:45:52.176 INFO:tasks.workunit.client.0.smithi094.stdout:2/38: dread - f8 zero size 2024-03-24T10:45:52.176 INFO:tasks.workunit.client.0.smithi094.stdout:2/39: readlink l6 0 2024-03-24T10:45:52.179 INFO:tasks.workunit.client.0.smithi094.stdout:7/55: link d1/c11 d1/c15 0 2024-03-24T10:45:52.181 INFO:tasks.workunit.client.0.smithi094.stdout:6/45: rename d0/d5/d9 to d0/d5/da 0 2024-03-24T10:45:52.184 INFO:tasks.workunit.client.0.smithi094.stdout:8/76: rename df/l10 to df/d12/l1a 0 2024-03-24T10:45:52.184 INFO:tasks.workunit.client.0.smithi094.stdout:8/77: getdents df/d12/d18 0 2024-03-24T10:45:52.185 INFO:tasks.workunit.client.0.smithi094.stdout:9/68: creat df/d12/f15 x:0 0 0 2024-03-24T10:45:52.186 INFO:tasks.workunit.client.0.smithi094.stdout:9/69: write df/d12/f15 [764330,83372] 0 2024-03-24T10:45:52.186 INFO:tasks.workunit.client.0.smithi094.stdout:9/70: fsync df/d10/d11/f13 0 2024-03-24T10:45:52.186 INFO:tasks.workunit.client.0.smithi094.stdout:9/71: write df/f14 [209834,102101] 0 2024-03-24T10:45:52.186 INFO:tasks.workunit.client.0.smithi094.stdout:1/50: mkdir d4/d9/de 0 2024-03-24T10:45:52.186 INFO:tasks.workunit.client.0.smithi094.stdout:1/51: truncate d4/fc 486435 0 2024-03-24T10:45:52.187 INFO:tasks.workunit.client.0.smithi094.stdout:3/60: read d0/f10 [2593641,31295] 0 2024-03-24T10:45:52.188 INFO:tasks.workunit.client.0.smithi094.stdout:8/78: dread f5 [0,4194304] 0 2024-03-24T10:45:52.189 INFO:tasks.workunit.client.0.smithi094.stdout:0/39: link lf l10 0 2024-03-24T10:45:52.190 INFO:tasks.workunit.client.1.smithi096.stdout:3/67: dwrite d0/f1 [0,4194304] 0 2024-03-24T10:45:52.191 INFO:tasks.workunit.client.0.smithi094.stdout:8/79: dread f3 [0,4194304] 0 2024-03-24T10:45:52.194 INFO:tasks.workunit.client.0.smithi094.stdout:7/56: mkdir d1/d9/d16 0 2024-03-24T10:45:52.194 INFO:tasks.workunit.client.0.smithi094.stdout:7/57: readlink d1/d9/la 0 2024-03-24T10:45:52.194 INFO:tasks.workunit.client.0.smithi094.stdout:7/58: dread - d1/d9/f12 zero size 2024-03-24T10:45:52.195 INFO:tasks.workunit.client.0.smithi094.stdout:6/46: rename d0/c7 to d0/cb 0 2024-03-24T10:45:52.195 INFO:tasks.workunit.client.0.smithi094.stdout:6/47: creat d0/fc x:0 0 0 2024-03-24T10:45:52.196 INFO:tasks.workunit.client.0.smithi094.stdout:0/40: read f0 [6912410,10357] 0 2024-03-24T10:45:52.197 INFO:tasks.workunit.client.0.smithi094.stdout:2/40: dwrite f8 [0,4194304] 0 2024-03-24T10:45:52.197 INFO:tasks.workunit.client.0.smithi094.stdout:2/41: chown f8 0 1 2024-03-24T10:45:52.200 INFO:tasks.workunit.client.0.smithi094.stdout:4/91: symlink d0/d5/d9/l15 0 2024-03-24T10:45:52.202 INFO:tasks.workunit.client.0.smithi094.stdout:1/52: dread d4/d9/fd [0,4194304] 0 2024-03-24T10:45:52.202 INFO:tasks.workunit.client.0.smithi094.stdout:3/61: creat d0/db/dc/f12 x:0 0 0 2024-03-24T10:45:52.203 INFO:tasks.workunit.client.0.smithi094.stdout:3/62: write d0/f11 [5004095,10985] 0 2024-03-24T10:45:52.203 INFO:tasks.workunit.client.0.smithi094.stdout:9/72: dread df/d12/f15 [0,4194304] 0 2024-03-24T10:45:52.203 INFO:tasks.workunit.client.0.smithi094.stdout:9/73: write df/f14 [896232,26621] 0 2024-03-24T10:45:52.204 INFO:tasks.workunit.client.0.smithi094.stdout:7/59: unlink d1/d9/lc 0 2024-03-24T10:45:52.206 INFO:tasks.workunit.client.0.smithi094.stdout:6/48: mkdir d0/d5/da/dd 0 2024-03-24T10:45:52.207 INFO:tasks.workunit.client.0.smithi094.stdout:0/41: mknod c11 0 2024-03-24T10:45:52.210 INFO:tasks.workunit.client.0.smithi094.stdout:2/42: mknod c9 0 2024-03-24T10:45:52.210 INFO:tasks.workunit.client.0.smithi094.stdout:2/43: fsync f8 0 2024-03-24T10:45:52.211 INFO:tasks.workunit.client.0.smithi094.stdout:1/53: mknod d4/cf 0 2024-03-24T10:45:52.212 INFO:tasks.workunit.client.0.smithi094.stdout:3/63: mknod d0/db/dc/c13 0 2024-03-24T10:45:52.214 INFO:tasks.workunit.client.0.smithi094.stdout:9/74: link c5 df/d10/d11/c16 0 2024-03-24T10:45:52.214 INFO:tasks.workunit.client.0.smithi094.stdout:6/49: dread d0/f2 [0,4194304] 0 2024-03-24T10:45:52.214 INFO:tasks.workunit.client.0.smithi094.stdout:6/50: write d0/f3 [703197,104373] 0 2024-03-24T10:45:52.215 INFO:tasks.workunit.client.0.smithi094.stdout:8/80: rename df/d12/f15 to df/f1b 0 2024-03-24T10:45:52.219 INFO:tasks.workunit.client.0.smithi094.stdout:0/42: rename c11 to c12 0 2024-03-24T10:45:52.225 INFO:tasks.workunit.client.0.smithi094.stdout:6/51: creat d0/d5/da/dd/fe x:0 0 0 2024-03-24T10:45:52.226 INFO:tasks.workunit.client.0.smithi094.stdout:6/52: chown d0/cb 1850 1 2024-03-24T10:45:52.226 INFO:tasks.workunit.client.0.smithi094.stdout:6/53: dread - d0/f8 zero size 2024-03-24T10:45:52.226 INFO:tasks.workunit.client.0.smithi094.stdout:6/54: truncate d0/f8 19246 0 2024-03-24T10:45:52.226 INFO:tasks.workunit.client.0.smithi094.stdout:6/55: creat d0/d5/da/dd/ff x:0 0 0 2024-03-24T10:45:52.226 INFO:tasks.workunit.client.0.smithi094.stdout:6/56: dread - d0/fc zero size 2024-03-24T10:45:52.228 INFO:tasks.workunit.client.0.smithi094.stdout:0/43: unlink f8 0 2024-03-24T10:45:52.228 INFO:tasks.workunit.client.0.smithi094.stdout:6/57: write d0/f2 [1545617,74093] 0 2024-03-24T10:45:52.228 INFO:tasks.workunit.client.0.smithi094.stdout:6/58: readlink d0/l6 0 2024-03-24T10:45:52.229 INFO:tasks.workunit.client.0.smithi094.stdout:6/59: rename d0/d5/da to d0/d5/da/dd/d10 22 2024-03-24T10:45:52.229 INFO:tasks.workunit.client.0.smithi094.stdout:4/92: truncate d0/f6 153996 0 2024-03-24T10:45:52.230 INFO:tasks.workunit.client.0.smithi094.stdout:3/64: rmdir d0/db/de 39 2024-03-24T10:45:52.230 INFO:tasks.workunit.client.0.smithi094.stdout:3/65: creat d0/f14 x:0 0 0 2024-03-24T10:45:52.233 INFO:tasks.workunit.client.0.smithi094.stdout:0/44: truncate f0 3543661 0 2024-03-24T10:45:52.233 INFO:tasks.workunit.client.0.smithi094.stdout:0/45: creat f13 x:0 0 0 2024-03-24T10:45:52.233 INFO:tasks.workunit.client.0.smithi094.stdout:0/46: write fd [302974,114061] 0 2024-03-24T10:45:52.233 INFO:tasks.workunit.client.0.smithi094.stdout:0/47: chown f5 13 1 2024-03-24T10:45:52.233 INFO:tasks.workunit.client.0.smithi094.stdout:0/48: write f13 [938145,91585] 0 2024-03-24T10:45:52.235 INFO:tasks.workunit.client.0.smithi094.stdout:3/66: getdents d0/db 0 2024-03-24T10:45:52.235 INFO:tasks.workunit.client.0.smithi094.stdout:3/67: write d0/f10 [822711,84325] 0 2024-03-24T10:45:52.235 INFO:tasks.workunit.client.0.smithi094.stdout:3/68: write d0/f3 [2380078,65690] 0 2024-03-24T10:45:52.235 INFO:tasks.workunit.client.0.smithi094.stdout:3/69: write d0/f3 [1369060,16046] 0 2024-03-24T10:45:52.235 INFO:tasks.workunit.client.0.smithi094.stdout:2/44: dwrite f8 [0,4194304] 0 2024-03-24T10:45:52.235 INFO:tasks.workunit.client.0.smithi094.stdout:2/45: stat l6 0 2024-03-24T10:45:52.236 INFO:tasks.workunit.client.0.smithi094.stdout:9/75: dwrite df/f14 [0,4194304] 0 2024-03-24T10:45:52.236 INFO:tasks.workunit.client.0.smithi094.stdout:9/76: creat df/f17 x:0 0 0 2024-03-24T10:45:52.236 INFO:tasks.workunit.client.0.smithi094.stdout:9/77: write df/f17 [668879,64624] 0 2024-03-24T10:45:52.238 INFO:tasks.workunit.client.0.smithi094.stdout:0/49: mkdir d14 0 2024-03-24T10:45:52.239 INFO:tasks.workunit.client.0.smithi094.stdout:3/70: symlink d0/db/de/l15 0 2024-03-24T10:45:52.240 INFO:tasks.workunit.client.0.smithi094.stdout:8/81: getdents df/d12 0 2024-03-24T10:45:52.242 INFO:tasks.workunit.client.0.smithi094.stdout:2/46: symlink la 0 2024-03-24T10:45:52.242 INFO:tasks.workunit.client.0.smithi094.stdout:2/47: dread - f7 zero size 2024-03-24T10:45:52.246 INFO:tasks.workunit.client.0.smithi094.stdout:6/60: dwrite d0/fc [0,4194304] 0 2024-03-24T10:45:52.246 INFO:tasks.workunit.client.0.smithi094.stdout:9/78: rename c8 to df/d10/d11/c18 0 2024-03-24T10:45:52.247 INFO:tasks.workunit.client.0.smithi094.stdout:0/50: creat d14/f15 x:0 0 0 2024-03-24T10:45:52.247 INFO:tasks.workunit.client.0.smithi094.stdout:0/51: write f13 [1701631,116000] 0 2024-03-24T10:45:52.247 INFO:tasks.workunit.client.0.smithi094.stdout:0/52: creat d14/f16 x:0 0 0 2024-03-24T10:45:52.248 INFO:tasks.workunit.client.0.smithi094.stdout:2/48: write f8 [3466571,37038] 0 2024-03-24T10:45:52.248 INFO:tasks.workunit.client.0.smithi094.stdout:3/71: rename d0/c4 to d0/c16 0 2024-03-24T10:45:52.248 INFO:tasks.workunit.client.0.smithi094.stdout:4/93: write d0/f6 [884221,40826] 0 2024-03-24T10:45:52.248 INFO:tasks.workunit.client.0.smithi094.stdout:4/94: chown d0/d5/ld 90 1 2024-03-24T10:45:52.252 INFO:tasks.workunit.client.0.smithi094.stdout:0/53: fsync f13 0 2024-03-24T10:45:52.259 INFO:tasks.workunit.client.0.smithi094.stdout:6/61: rmdir d0/d5 39 2024-03-24T10:45:52.259 INFO:tasks.workunit.client.0.smithi094.stdout:6/62: chown d0/d5/da 1109207 1 2024-03-24T10:45:52.259 INFO:tasks.workunit.client.0.smithi094.stdout:6/63: write d0/f8 [917051,30026] 0 2024-03-24T10:45:52.259 INFO:tasks.workunit.client.0.smithi094.stdout:6/64: write d0/f3 [1068821,15657] 0 2024-03-24T10:45:52.260 INFO:tasks.workunit.client.0.smithi094.stdout:6/65: read d0/f8 [245944,124015] 0 2024-03-24T10:45:52.260 INFO:tasks.workunit.client.0.smithi094.stdout:9/79: symlink df/d10/d11/l19 0 2024-03-24T10:45:52.262 INFO:tasks.workunit.client.0.smithi094.stdout:2/49: link l3 lb 0 2024-03-24T10:45:52.262 INFO:tasks.workunit.client.0.smithi094.stdout:2/50: readlink l5 0 2024-03-24T10:45:52.264 INFO:tasks.workunit.client.0.smithi094.stdout:4/95: creat d0/d5/f16 x:0 0 0 2024-03-24T10:45:52.268 INFO:tasks.workunit.client.0.smithi094.stdout:8/82: link l7 df/l1c 0 2024-03-24T10:45:52.268 INFO:tasks.workunit.client.0.smithi094.stdout:8/83: chown f5 28 1 2024-03-24T10:45:52.270 INFO:tasks.workunit.client.0.smithi094.stdout:9/80: write fe [5268898,34305] 0 2024-03-24T10:45:52.272 INFO:tasks.workunit.client.0.smithi094.stdout:9/81: dread df/d12/f15 [0,4194304] 0 2024-03-24T10:45:52.272 INFO:tasks.workunit.client.0.smithi094.stdout:6/66: symlink d0/d5/da/dd/l11 0 2024-03-24T10:45:52.273 INFO:tasks.workunit.client.0.smithi094.stdout:2/51: mknod cc 0 2024-03-24T10:45:52.275 INFO:tasks.workunit.client.0.smithi094.stdout:3/72: rmdir d0/db/dc 39 2024-03-24T10:45:52.279 INFO:tasks.workunit.client.0.smithi094.stdout:0/54: dread f5 [0,4194304] 0 2024-03-24T10:45:52.279 INFO:tasks.workunit.client.0.smithi094.stdout:0/55: dread - fc zero size 2024-03-24T10:45:52.282 INFO:tasks.workunit.client.0.smithi094.stdout:6/67: creat d0/d5/da/f12 x:0 0 0 2024-03-24T10:45:52.284 INFO:tasks.workunit.client.0.smithi094.stdout:2/52: dread f8 [0,4194304] 0 2024-03-24T10:45:52.284 INFO:tasks.workunit.client.0.smithi094.stdout:6/68: rename d0/d5/da/dd/ff to d0/d5/da/dd/f13 0 2024-03-24T10:45:52.286 INFO:tasks.workunit.client.0.smithi094.stdout:6/69: mknod d0/c14 0 2024-03-24T10:45:52.286 INFO:tasks.workunit.client.0.smithi094.stdout:6/70: chown d0/fc 4716080 1 2024-03-24T10:45:52.287 INFO:tasks.workunit.client.0.smithi094.stdout:6/71: mknod d0/d5/c15 0 2024-03-24T10:45:52.293 INFO:tasks.workunit.client.0.smithi094.stdout:8/84: dwrite fc [0,4194304] 0 2024-03-24T10:45:52.294 INFO:tasks.workunit.client.0.smithi094.stdout:4/96: dwrite d0/d5/d9/f11 [0,4194304] 0 2024-03-24T10:45:52.298 INFO:tasks.workunit.client.0.smithi094.stdout:4/97: dread d0/d5/d9/f12 [0,4194304] 0 2024-03-24T10:45:52.299 INFO:tasks.workunit.client.0.smithi094.stdout:8/85: getdents df 0 2024-03-24T10:45:52.301 INFO:tasks.workunit.client.0.smithi094.stdout:8/86: dread df/f1b [0,4194304] 0 2024-03-24T10:45:52.301 INFO:tasks.workunit.client.0.smithi094.stdout:8/87: write df/f16 [800854,128016] 0 2024-03-24T10:45:52.301 INFO:tasks.workunit.client.0.smithi094.stdout:8/88: readlink df/l1c 0 2024-03-24T10:45:52.302 INFO:tasks.workunit.client.0.smithi094.stdout:8/89: write f8 [99931,10033] 0 2024-03-24T10:45:52.302 INFO:tasks.workunit.client.0.smithi094.stdout:9/82: dwrite df/d10/d11/f13 [0,4194304] 0 2024-03-24T10:45:52.310 INFO:tasks.workunit.client.0.smithi094.stdout:9/83: write df/d10/d11/f13 [3762987,76039] 0 2024-03-24T10:45:52.311 INFO:tasks.workunit.client.0.smithi094.stdout:9/84: unlink fe 0 2024-03-24T10:45:52.311 INFO:tasks.workunit.client.0.smithi094.stdout:9/85: stat df/f17 0 2024-03-24T10:45:52.312 INFO:tasks.workunit.client.0.smithi094.stdout:9/86: getdents df/d10 0 2024-03-24T10:45:52.312 INFO:tasks.workunit.client.0.smithi094.stdout:9/87: stat c2 0 2024-03-24T10:45:52.313 INFO:tasks.workunit.client.0.smithi094.stdout:9/88: unlink l6 0 2024-03-24T10:45:52.314 INFO:tasks.workunit.client.0.smithi094.stdout:9/89: symlink df/d10/l1a 0 2024-03-24T10:45:52.314 INFO:tasks.workunit.client.0.smithi094.stdout:9/90: chown df/d12/f15 398823 1 2024-03-24T10:45:52.314 INFO:tasks.workunit.client.0.smithi094.stdout:9/91: fdatasync df/f14 0 2024-03-24T10:45:52.314 INFO:tasks.workunit.client.0.smithi094.stdout:9/92: mknod df/d10/c1b 0 2024-03-24T10:45:52.324 INFO:tasks.workunit.client.0.smithi094.stdout:0/56: dwrite f5 [0,4194304] 0 2024-03-24T10:45:52.324 INFO:tasks.workunit.client.0.smithi094.stdout:0/57: read - f3 zero size 2024-03-24T10:45:52.324 INFO:tasks.workunit.client.0.smithi094.stdout:0/58: chown c12 153 1 2024-03-24T10:45:52.324 INFO:tasks.workunit.client.0.smithi094.stdout:0/59: fdatasync fa 0 2024-03-24T10:45:52.324 INFO:tasks.workunit.client.0.smithi094.stdout:0/60: dread - fc zero size 2024-03-24T10:45:52.324 INFO:tasks.workunit.client.0.smithi094.stdout:0/61: truncate fa 776190 0 2024-03-24T10:45:52.324 INFO:tasks.workunit.client.0.smithi094.stdout:0/62: truncate d14/f15 493727 0 2024-03-24T10:45:52.324 INFO:tasks.workunit.client.0.smithi094.stdout:0/63: write d14/f16 [702398,64780] 0 2024-03-24T10:45:52.324 INFO:tasks.workunit.client.0.smithi094.stdout:3/73: dwrite d0/f14 [0,4194304] 0 2024-03-24T10:45:52.325 INFO:tasks.workunit.client.0.smithi094.stdout:2/53: dwrite f7 [0,4194304] 0 2024-03-24T10:45:52.325 INFO:tasks.workunit.client.0.smithi094.stdout:6/72: dwrite d0/d5/da/dd/f13 [0,4194304] 0 2024-03-24T10:45:52.325 INFO:tasks.workunit.client.0.smithi094.stdout:6/73: write d0/d5/da/dd/fe [1023726,36859] 0 2024-03-24T10:45:52.325 INFO:tasks.workunit.client.0.smithi094.stdout:4/98: dwrite d0/d5/d9/f11 [0,4194304] 0 2024-03-24T10:45:52.329 INFO:tasks.workunit.client.0.smithi094.stdout:6/74: dread d0/f3 [0,4194304] 0 2024-03-24T10:45:52.329 INFO:tasks.workunit.client.0.smithi094.stdout:0/64: mknod d14/c17 0 2024-03-24T10:45:52.330 INFO:tasks.workunit.client.0.smithi094.stdout:0/65: read f13 [1250995,61531] 0 2024-03-24T10:45:52.331 INFO:tasks.workunit.client.0.smithi094.stdout:6/75: dread d0/d5/da/dd/fe [0,4194304] 0 2024-03-24T10:45:52.337 INFO:tasks.workunit.client.0.smithi094.stdout:9/93: getdents df/d10 0 2024-03-24T10:45:52.337 INFO:tasks.workunit.client.0.smithi094.stdout:9/94: stat df/d10 0 2024-03-24T10:45:52.341 INFO:tasks.workunit.client.0.smithi094.stdout:0/66: mkdir d14/d18 0 2024-03-24T10:45:52.342 INFO:tasks.workunit.client.0.smithi094.stdout:6/76: mknod d0/d5/da/dd/c16 0 2024-03-24T10:45:52.342 INFO:tasks.workunit.client.0.smithi094.stdout:6/77: write d0/f2 [3055541,75183] 0 2024-03-24T10:45:52.343 INFO:tasks.workunit.client.0.smithi094.stdout:2/54: truncate f7 3099796 0 2024-03-24T10:45:52.344 INFO:tasks.workunit.client.0.smithi094.stdout:9/95: symlink df/l1c 0 2024-03-24T10:45:52.344 INFO:tasks.workunit.client.0.smithi094.stdout:9/96: write df/f17 [1604802,18646] 0 2024-03-24T10:45:52.345 INFO:tasks.workunit.client.0.smithi094.stdout:4/99: mknod d0/c17 0 2024-03-24T10:45:52.346 INFO:tasks.workunit.client.0.smithi094.stdout:4/100: chown d0/d5/d9/ff 8 1 2024-03-24T10:45:52.346 INFO:tasks.workunit.client.0.smithi094.stdout:4/101: chown d0/l14 6 1 2024-03-24T10:45:52.346 INFO:tasks.workunit.client.0.smithi094.stdout:4/102: fdatasync d0/f7 0 2024-03-24T10:45:52.348 INFO:tasks.workunit.client.0.smithi094.stdout:6/78: rename d0/d5/da/dd/fe to d0/f17 0 2024-03-24T10:45:52.351 INFO:tasks.workunit.client.0.smithi094.stdout:4/103: link d0/c2 d0/c18 0 2024-03-24T10:45:52.352 INFO:tasks.workunit.client.0.smithi094.stdout:5/53: sync 2024-03-24T10:45:52.352 INFO:tasks.workunit.client.0.smithi094.stdout:7/60: sync 2024-03-24T10:45:52.352 INFO:tasks.workunit.client.0.smithi094.stdout:5/54: fsync d7/d8/fa 0 2024-03-24T10:45:52.353 INFO:tasks.workunit.client.0.smithi094.stdout:5/55: write d7/d8/fa [2654019,48099] 0 2024-03-24T10:45:52.354 INFO:tasks.workunit.client.0.smithi094.stdout:5/56: symlink d7/d8/d9/l13 0 2024-03-24T10:45:52.354 INFO:tasks.workunit.client.0.smithi094.stdout:5/57: write d7/f10 [1983028,45345] 0 2024-03-24T10:45:52.355 INFO:tasks.workunit.client.0.smithi094.stdout:9/97: write df/f14 [850947,54271] 0 2024-03-24T10:45:52.355 INFO:tasks.workunit.client.0.smithi094.stdout:5/58: stat c5 0 2024-03-24T10:45:52.355 INFO:tasks.workunit.client.0.smithi094.stdout:5/59: write d7/d8/fa [3054495,93417] 0 2024-03-24T10:45:52.356 INFO:tasks.workunit.client.0.smithi094.stdout:5/60: mkdir d7/d8/d14 0 2024-03-24T10:45:52.356 INFO:tasks.workunit.client.0.smithi094.stdout:5/61: write d7/d8/fa [311483,89644] 0 2024-03-24T10:45:52.356 INFO:tasks.workunit.client.0.smithi094.stdout:9/98: read df/d12/f15 [666220,5549] 0 2024-03-24T10:45:52.357 INFO:tasks.workunit.client.0.smithi094.stdout:5/62: symlink d7/dc/d12/l15 0 2024-03-24T10:45:52.364 INFO:tasks.workunit.client.0.smithi094.stdout:7/61: dread f0 [0,4194304] 0 2024-03-24T10:45:52.366 INFO:tasks.workunit.client.0.smithi094.stdout:7/62: symlink d1/d9/d14/l17 0 2024-03-24T10:45:52.366 INFO:tasks.workunit.client.0.smithi094.stdout:7/63: truncate d1/d9/f13 417450 0 2024-03-24T10:45:52.366 INFO:tasks.workunit.client.0.smithi094.stdout:7/64: creat d1/d9/d14/f18 x:0 0 0 2024-03-24T10:45:52.372 INFO:tasks.workunit.client.0.smithi094.stdout:7/65: dread f0 [0,4194304] 0 2024-03-24T10:45:52.374 INFO:tasks.workunit.client.0.smithi094.stdout:7/66: rename d1/d9/la to d1/d9/d16/l19 0 2024-03-24T10:45:52.379 INFO:tasks.workunit.client.0.smithi094.stdout:0/67: dwrite fa [0,4194304] 0 2024-03-24T10:45:52.379 INFO:tasks.workunit.client.0.smithi094.stdout:0/68: fdatasync d14/f15 0 2024-03-24T10:45:52.379 INFO:tasks.workunit.client.0.smithi094.stdout:0/69: truncate f3 351325 0 2024-03-24T10:45:52.379 INFO:tasks.workunit.client.0.smithi094.stdout:0/70: write d14/f15 [1045086,57953] 0 2024-03-24T10:45:52.379 INFO:tasks.workunit.client.0.smithi094.stdout:5/63: dread d7/d8/fa [0,4194304] 0 2024-03-24T10:45:52.380 INFO:tasks.workunit.client.0.smithi094.stdout:6/79: dwrite d0/d5/da/dd/f13 [4194304,4194304] 0 2024-03-24T10:45:52.382 INFO:tasks.workunit.client.0.smithi094.stdout:7/67: dread d1/d9/fe [0,4194304] 0 2024-03-24T10:45:52.385 INFO:tasks.workunit.client.0.smithi094.stdout:2/55: dwrite f7 [0,4194304] 0 2024-03-24T10:45:52.387 INFO:tasks.workunit.client.0.smithi094.stdout:9/99: dread df/f14 [0,4194304] 0 2024-03-24T10:45:52.387 INFO:tasks.workunit.client.0.smithi094.stdout:0/71: mknod d14/c19 0 2024-03-24T10:45:52.389 INFO:tasks.workunit.client.0.smithi094.stdout:2/56: dread f8 [0,4194304] 0 2024-03-24T10:45:52.391 INFO:tasks.workunit.client.0.smithi094.stdout:5/64: creat d7/d8/f16 x:0 0 0 2024-03-24T10:45:52.391 INFO:tasks.workunit.client.0.smithi094.stdout:5/65: read - d7/d8/f16 zero size 2024-03-24T10:45:52.391 INFO:tasks.workunit.client.0.smithi094.stdout:5/66: stat c5 0 2024-03-24T10:45:52.391 INFO:tasks.workunit.client.0.smithi094.stdout:5/67: read d7/d8/fa [2377746,10026] 0 2024-03-24T10:45:52.391 INFO:tasks.workunit.client.0.smithi094.stdout:6/80: link d0/f17 d0/f18 0 2024-03-24T10:45:52.392 INFO:tasks.workunit.client.0.smithi094.stdout:7/68: symlink d1/d9/d16/l1a 0 2024-03-24T10:45:52.396 INFO:tasks.workunit.client.0.smithi094.stdout:0/72: creat d14/d18/f1a x:0 0 0 2024-03-24T10:45:52.396 INFO:tasks.workunit.client.0.smithi094.stdout:0/73: chown f0 51588 1 2024-03-24T10:45:52.396 INFO:tasks.workunit.client.0.smithi094.stdout:0/74: readlink l7 0 2024-03-24T10:45:52.396 INFO:tasks.workunit.client.0.smithi094.stdout:0/75: fdatasync f9 0 2024-03-24T10:45:52.397 INFO:tasks.workunit.client.0.smithi094.stdout:6/81: creat d0/d5/f19 x:0 0 0 2024-03-24T10:45:52.397 INFO:tasks.workunit.client.0.smithi094.stdout:6/82: write d0/f8 [519813,73529] 0 2024-03-24T10:45:52.397 INFO:tasks.workunit.client.0.smithi094.stdout:6/83: write d0/d5/f19 [857026,77372] 0 2024-03-24T10:45:52.400 INFO:tasks.workunit.client.0.smithi094.stdout:7/69: unlink d1/d9/d14/f18 0 2024-03-24T10:45:52.400 INFO:tasks.workunit.client.0.smithi094.stdout:7/70: creat d1/f1b x:0 0 0 2024-03-24T10:45:52.403 INFO:tasks.workunit.client.0.smithi094.stdout:9/100: rmdir df 39 2024-03-24T10:45:52.406 INFO:tasks.workunit.client.0.smithi094.stdout:0/76: unlink l10 0 2024-03-24T10:45:52.406 INFO:tasks.workunit.client.0.smithi094.stdout:1/54: sync 2024-03-24T10:45:52.406 INFO:tasks.workunit.client.0.smithi094.stdout:0/77: write fd [1024175,88720] 0 2024-03-24T10:45:52.407 INFO:tasks.workunit.client.0.smithi094.stdout:0/78: chown d14/d18/f1a 30065 1 2024-03-24T10:45:52.407 INFO:tasks.workunit.client.0.smithi094.stdout:0/79: creat d14/f1b x:0 0 0 2024-03-24T10:45:52.407 INFO:tasks.workunit.client.0.smithi094.stdout:0/80: rename d14 to d14/d1c 22 2024-03-24T10:45:52.407 INFO:tasks.workunit.client.0.smithi094.stdout:0/81: chown d14/c19 268040 1 2024-03-24T10:45:52.407 INFO:tasks.workunit.client.0.smithi094.stdout:0/82: creat d14/d18/f1d x:0 0 0 2024-03-24T10:45:52.407 INFO:tasks.workunit.client.0.smithi094.stdout:0/83: creat d14/f1e x:0 0 0 2024-03-24T10:45:52.407 INFO:tasks.workunit.client.0.smithi094.stdout:0/84: truncate fc 330037 0 2024-03-24T10:45:52.407 INFO:tasks.workunit.client.0.smithi094.stdout:0/85: creat d14/d18/f1f x:0 0 0 2024-03-24T10:45:52.407 INFO:tasks.workunit.client.0.smithi094.stdout:0/86: chown d14/f1e 0 1 2024-03-24T10:45:52.408 INFO:tasks.workunit.client.0.smithi094.stdout:0/87: fdatasync f3 0 2024-03-24T10:45:52.409 INFO:tasks.workunit.client.0.smithi094.stdout:7/71: mknod d1/c1c 0 2024-03-24T10:45:52.409 INFO:tasks.workunit.client.0.smithi094.stdout:7/72: dread - d1/f10 zero size 2024-03-24T10:45:52.409 INFO:tasks.workunit.client.0.smithi094.stdout:7/73: chown d1/f2 19 1 2024-03-24T10:45:52.409 INFO:tasks.workunit.client.0.smithi094.stdout:7/74: chown d1/d9/fd 0 1 2024-03-24T10:45:52.409 INFO:tasks.workunit.client.0.smithi094.stdout:0/88: read d14/f16 [525505,85265] 0 2024-03-24T10:45:52.409 INFO:tasks.workunit.client.0.smithi094.stdout:0/89: dread - d14/f1e zero size 2024-03-24T10:45:52.409 INFO:tasks.workunit.client.0.smithi094.stdout:1/55: creat d4/d9/db/f10 x:0 0 0 2024-03-24T10:45:52.409 INFO:tasks.workunit.client.0.smithi094.stdout:1/56: creat d4/f11 x:0 0 0 2024-03-24T10:45:52.409 INFO:tasks.workunit.client.0.smithi094.stdout:1/57: chown d4/d9/db/f10 530388039 1 2024-03-24T10:45:52.409 INFO:tasks.workunit.client.0.smithi094.stdout:1/58: chown d4/d9 214488810 1 2024-03-24T10:45:52.409 INFO:tasks.workunit.client.0.smithi094.stdout:1/59: rename d4/d9 to d4/d9/de/d12 22 2024-03-24T10:45:52.412 INFO:tasks.workunit.client.0.smithi094.stdout:0/90: symlink d14/l20 0 2024-03-24T10:45:52.415 INFO:tasks.workunit.client.0.smithi094.stdout:5/68: dwrite d7/f10 [0,4194304] 0 2024-03-24T10:45:52.415 INFO:tasks.workunit.client.0.smithi094.stdout:2/57: dwrite f8 [0,4194304] 0 2024-03-24T10:45:52.415 INFO:tasks.workunit.client.0.smithi094.stdout:2/58: chown cc 3110548 1 2024-03-24T10:45:52.416 INFO:tasks.workunit.client.0.smithi094.stdout:0/91: rename d14/f15 to d14/d18/f21 0 2024-03-24T10:45:52.417 INFO:tasks.workunit.client.0.smithi094.stdout:1/60: unlink d4/f11 0 2024-03-24T10:45:52.417 INFO:tasks.workunit.client.0.smithi094.stdout:1/61: chown c3 1387880 1 2024-03-24T10:45:52.417 INFO:tasks.workunit.client.0.smithi094.stdout:1/62: write d4/d9/fa [4472858,95099] 0 2024-03-24T10:45:52.420 INFO:tasks.workunit.client.0.smithi094.stdout:7/75: getdents d1/d9/d14 0 2024-03-24T10:45:52.422 INFO:tasks.workunit.client.0.smithi094.stdout:1/63: link f1 d4/d9/db/f13 0 2024-03-24T10:45:52.422 INFO:tasks.workunit.client.0.smithi094.stdout:2/59: write f8 [2722818,44267] 0 2024-03-24T10:45:52.423 INFO:tasks.workunit.client.0.smithi094.stdout:7/76: mknod d1/d9/d14/c1d 0 2024-03-24T10:45:52.424 INFO:tasks.workunit.client.0.smithi094.stdout:2/60: mknod cd 0 2024-03-24T10:45:52.424 INFO:tasks.workunit.client.0.smithi094.stdout:2/61: chown l6 2047969 1 2024-03-24T10:45:52.425 INFO:tasks.workunit.client.0.smithi094.stdout:2/62: write f7 [4237537,63601] 0 2024-03-24T10:45:52.426 INFO:tasks.workunit.client.0.smithi094.stdout:2/63: creat fe x:0 0 0 2024-03-24T10:45:52.426 INFO:tasks.workunit.client.0.smithi094.stdout:2/64: write f8 [783864,117275] 0 2024-03-24T10:45:52.428 INFO:tasks.workunit.client.0.smithi094.stdout:9/101: dwrite df/f17 [0,4194304] 0 2024-03-24T10:45:52.430 INFO:tasks.workunit.client.0.smithi094.stdout:2/65: unlink l1 0 2024-03-24T10:45:52.434 INFO:tasks.workunit.client.0.smithi094.stdout:6/84: truncate d0/d5/f19 343809 0 2024-03-24T10:45:52.436 INFO:tasks.workunit.client.0.smithi094.stdout:2/66: symlink lf 0 2024-03-24T10:45:52.438 INFO:tasks.workunit.client.0.smithi094.stdout:1/64: write d4/f8 [3453575,35428] 0 2024-03-24T10:45:52.440 INFO:tasks.workunit.client.0.smithi094.stdout:2/67: mknod c10 0 2024-03-24T10:45:52.440 INFO:tasks.workunit.client.0.smithi094.stdout:6/85: dread d0/f2 [0,4194304] 0 2024-03-24T10:45:52.440 INFO:tasks.workunit.client.0.smithi094.stdout:6/86: write d0/f17 [1648776,108722] 0 2024-03-24T10:45:52.440 INFO:tasks.workunit.client.0.smithi094.stdout:6/87: write d0/d5/da/f12 [107737,44673] 0 2024-03-24T10:45:52.440 INFO:tasks.workunit.client.0.smithi094.stdout:6/88: chown d0/d5/da 1220398 1 2024-03-24T10:45:52.440 INFO:tasks.workunit.client.0.smithi094.stdout:6/89: readlink d0/l6 0 2024-03-24T10:45:52.440 INFO:tasks.workunit.client.0.smithi094.stdout:6/90: truncate d0/fc 4349542 0 2024-03-24T10:45:52.440 INFO:tasks.workunit.client.0.smithi094.stdout:1/65: mknod d4/d9/de/c14 0 2024-03-24T10:45:52.445 INFO:tasks.workunit.client.0.smithi094.stdout:1/66: write d4/d9/fd [3580878,88637] 0 2024-03-24T10:45:52.446 INFO:tasks.workunit.client.0.smithi094.stdout:1/67: symlink d4/d9/db/l15 0 2024-03-24T10:45:52.449 INFO:tasks.workunit.client.0.smithi094.stdout:1/68: write d4/f7 [3454126,71443] 0 2024-03-24T10:45:52.449 INFO:tasks.workunit.client.0.smithi094.stdout:1/69: creat d4/f16 x:0 0 0 2024-03-24T10:45:52.450 INFO:tasks.workunit.client.0.smithi094.stdout:1/70: symlink d4/d9/l17 0 2024-03-24T10:45:52.452 INFO:tasks.workunit.client.0.smithi094.stdout:1/71: rename d4/f8 to d4/d9/db/f18 0 2024-03-24T10:45:52.453 INFO:tasks.workunit.client.0.smithi094.stdout:1/72: unlink c3 0 2024-03-24T10:45:52.462 INFO:tasks.workunit.client.0.smithi094.stdout:0/92: dwrite d14/d18/f1a [0,4194304] 0 2024-03-24T10:45:52.462 INFO:tasks.workunit.client.0.smithi094.stdout:7/77: dwrite f0 [0,4194304] 0 2024-03-24T10:45:52.463 INFO:tasks.workunit.client.0.smithi094.stdout:5/69: dwrite d7/d8/f16 [0,4194304] 0 2024-03-24T10:45:52.463 INFO:tasks.workunit.client.0.smithi094.stdout:6/91: fsync d0/f17 0 2024-03-24T10:45:52.463 INFO:tasks.workunit.client.0.smithi094.stdout:9/102: dwrite df/f17 [0,4194304] 0 2024-03-24T10:45:52.465 INFO:tasks.workunit.client.0.smithi094.stdout:0/93: dread fd [0,4194304] 0 2024-03-24T10:45:52.468 INFO:tasks.workunit.client.0.smithi094.stdout:5/70: rename d7/dc/d12/l15 to d7/d8/d14/l17 0 2024-03-24T10:45:52.471 INFO:tasks.workunit.client.0.smithi094.stdout:6/92: unlink d0/cb 0 2024-03-24T10:45:52.478 INFO:tasks.workunit.client.0.smithi094.stdout:7/78: dwrite d1/d9/f12 [0,4194304] 0 2024-03-24T10:45:52.479 INFO:tasks.workunit.client.0.smithi094.stdout:7/79: stat d1/f1b 0 2024-03-24T10:45:52.479 INFO:tasks.workunit.client.0.smithi094.stdout:7/80: write d1/f2 [995525,4650] 0 2024-03-24T10:45:52.479 INFO:tasks.workunit.client.0.smithi094.stdout:7/81: chown d1/d9/d16/l19 5 1 2024-03-24T10:45:52.479 INFO:tasks.workunit.client.0.smithi094.stdout:7/82: write d1/f2 [2025014,113437] 0 2024-03-24T10:45:52.481 INFO:tasks.workunit.client.0.smithi094.stdout:5/71: write d7/d8/f16 [3905365,35107] 0 2024-03-24T10:45:52.483 INFO:tasks.workunit.client.0.smithi094.stdout:9/103: symlink df/d12/l1d 0 2024-03-24T10:45:52.483 INFO:tasks.workunit.client.0.smithi094.stdout:0/94: mkdir d14/d22 0 2024-03-24T10:45:52.486 INFO:tasks.workunit.client.0.smithi094.stdout:6/93: unlink d0/d5/da/dd/c16 0 2024-03-24T10:45:52.486 INFO:tasks.workunit.client.0.smithi094.stdout:6/94: chown d0/f2 42428163 1 2024-03-24T10:45:52.486 INFO:tasks.workunit.client.0.smithi094.stdout:6/95: fdatasync d0/f3 0 2024-03-24T10:45:52.486 INFO:tasks.workunit.client.0.smithi094.stdout:9/104: dread df/d10/d11/f13 [0,4194304] 0 2024-03-24T10:45:52.487 INFO:tasks.workunit.client.0.smithi094.stdout:9/105: write df/d12/f15 [184684,92618] 0 2024-03-24T10:45:52.487 INFO:tasks.workunit.client.0.smithi094.stdout:6/96: dread d0/f18 [0,4194304] 0 2024-03-24T10:45:52.487 INFO:tasks.workunit.client.0.smithi094.stdout:6/97: chown d0/f8 90 1 2024-03-24T10:45:52.493 INFO:tasks.workunit.client.0.smithi094.stdout:5/72: truncate d7/f10 568188 0 2024-03-24T10:45:52.494 INFO:tasks.workunit.client.0.smithi094.stdout:0/95: truncate d14/f16 743044 0 2024-03-24T10:45:52.495 INFO:tasks.workunit.client.0.smithi094.stdout:9/106: creat df/d12/f1e x:0 0 0 2024-03-24T10:45:52.496 INFO:tasks.workunit.client.0.smithi094.stdout:9/107: write df/d12/f1e [393322,110872] 0 2024-03-24T10:45:52.496 INFO:tasks.workunit.client.0.smithi094.stdout:6/98: unlink d0/l6 0 2024-03-24T10:45:52.498 INFO:tasks.workunit.client.0.smithi094.stdout:7/83: symlink d1/d9/d14/l1e 0 2024-03-24T10:45:52.498 INFO:tasks.workunit.client.0.smithi094.stdout:7/84: creat d1/f1f x:0 0 0 2024-03-24T10:45:52.498 INFO:tasks.workunit.client.0.smithi094.stdout:7/85: chown d1/d9/d16 33 1 2024-03-24T10:45:52.498 INFO:tasks.workunit.client.0.smithi094.stdout:7/86: fsync d1/d9/f13 0 2024-03-24T10:45:52.498 INFO:tasks.workunit.client.0.smithi094.stdout:3/74: sync 2024-03-24T10:45:52.498 INFO:tasks.workunit.client.0.smithi094.stdout:4/104: sync 2024-03-24T10:45:52.499 INFO:tasks.workunit.client.0.smithi094.stdout:4/105: chown d0/d5/d9/de 0 1 2024-03-24T10:45:52.499 INFO:tasks.workunit.client.0.smithi094.stdout:4/106: readlink d0/d5/ld 0 2024-03-24T10:45:52.499 INFO:tasks.workunit.client.0.smithi094.stdout:9/108: rename df/d10 to df/d12/d1f 0 2024-03-24T10:45:52.499 INFO:tasks.workunit.client.0.smithi094.stdout:8/90: sync 2024-03-24T10:45:52.502 INFO:tasks.workunit.client.0.smithi094.stdout:7/87: mkdir d1/d9/d16/d20 0 2024-03-24T10:45:52.503 INFO:tasks.workunit.client.0.smithi094.stdout:3/75: mknod d0/db/dc/c17 0 2024-03-24T10:45:52.503 INFO:tasks.workunit.client.0.smithi094.stdout:3/76: stat d0/db/ld 0 2024-03-24T10:45:52.504 INFO:tasks.workunit.client.0.smithi094.stdout:9/109: symlink df/l20 0 2024-03-24T10:45:52.504 INFO:tasks.workunit.client.0.smithi094.stdout:9/110: chown df/f14 121986 1 2024-03-24T10:45:52.505 INFO:tasks.workunit.client.0.smithi094.stdout:8/91: symlink df/l1d 0 2024-03-24T10:45:52.506 INFO:tasks.workunit.client.0.smithi094.stdout:8/92: fsync fc 0 2024-03-24T10:45:52.509 INFO:tasks.workunit.client.0.smithi094.stdout:7/88: mkdir d1/d9/d16/d21 0 2024-03-24T10:45:52.510 INFO:tasks.workunit.client.0.smithi094.stdout:7/89: write d1/f1b [910510,111692] 0 2024-03-24T10:45:52.510 INFO:tasks.workunit.client.0.smithi094.stdout:0/96: dwrite d14/d18/f1d [0,4194304] 0 2024-03-24T10:45:52.511 INFO:tasks.workunit.client.0.smithi094.stdout:7/90: truncate d1/d9/fe 3681611 0 2024-03-24T10:45:52.512 INFO:tasks.workunit.client.0.smithi094.stdout:5/73: write d7/f10 [831153,4349] 0 2024-03-24T10:45:52.515 INFO:tasks.workunit.client.0.smithi094.stdout:0/97: symlink d14/d22/l23 0 2024-03-24T10:45:52.519 INFO:tasks.workunit.client.0.smithi094.stdout:5/74: unlink d7/d8/d9/cd 0 2024-03-24T10:45:52.521 INFO:tasks.workunit.client.0.smithi094.stdout:5/75: symlink d7/dc/d12/l18 0 2024-03-24T10:45:52.521 INFO:tasks.workunit.client.0.smithi094.stdout:5/76: write d7/d8/fa [5200956,18082] 0 2024-03-24T10:45:52.522 INFO:tasks.workunit.client.0.smithi094.stdout:0/98: dread fa [0,4194304] 0 2024-03-24T10:45:52.548 INFO:tasks.workunit.client.0.smithi094.stdout:6/99: dwrite d0/d5/da/dd/f13 [4194304,4194304] 0 2024-03-24T10:45:52.549 INFO:tasks.workunit.client.0.smithi094.stdout:8/93: dwrite f4 [0,4194304] 0 2024-03-24T10:45:52.549 INFO:tasks.workunit.client.0.smithi094.stdout:8/94: write f4 [4234124,89517] 0 2024-03-24T10:45:52.549 INFO:tasks.workunit.client.0.smithi094.stdout:9/111: dwrite df/d12/f1e [0,4194304] 0 2024-03-24T10:45:52.550 INFO:tasks.workunit.client.0.smithi094.stdout:4/107: dwrite d0/d5/f16 [0,4194304] 0 2024-03-24T10:45:52.551 INFO:tasks.workunit.client.0.smithi094.stdout:3/77: dwrite d0/db/de/ff [0,4194304] 0 2024-03-24T10:45:52.551 INFO:tasks.workunit.client.0.smithi094.stdout:0/99: dread d14/f16 [0,4194304] 0 2024-03-24T10:45:52.551 INFO:tasks.workunit.client.0.smithi094.stdout:3/78: fdatasync d0/db/dc/f12 0 2024-03-24T10:45:52.551 INFO:tasks.workunit.client.0.smithi094.stdout:6/100: mkdir d0/d1a 0 2024-03-24T10:45:52.553 INFO:tasks.workunit.client.0.smithi094.stdout:8/95: mknod df/c1e 0 2024-03-24T10:45:52.553 INFO:tasks.workunit.client.0.smithi094.stdout:8/96: fsync fd 0 2024-03-24T10:45:52.556 INFO:tasks.workunit.client.0.smithi094.stdout:9/112: rename df/l1c to df/l21 0 2024-03-24T10:45:52.558 INFO:tasks.workunit.client.0.smithi094.stdout:7/91: getdents d1 0 2024-03-24T10:45:52.558 INFO:tasks.workunit.client.0.smithi094.stdout:7/92: chown d1/d9/d16/d20 2 1 2024-03-24T10:45:52.558 INFO:tasks.workunit.client.0.smithi094.stdout:7/93: write d1/f2 [722455,69326] 0 2024-03-24T10:45:52.560 INFO:tasks.workunit.client.0.smithi094.stdout:4/108: symlink d0/d5/l19 0 2024-03-24T10:45:52.560 INFO:tasks.workunit.client.0.smithi094.stdout:4/109: chown d0/d5/d9/c10 359405213 1 2024-03-24T10:45:52.562 INFO:tasks.workunit.client.0.smithi094.stdout:5/77: dwrite d7/d8/fa [4194304,4194304] 0 2024-03-24T10:45:52.562 INFO:tasks.workunit.client.0.smithi094.stdout:5/78: write d7/d8/f16 [3238226,126990] 0 2024-03-24T10:45:52.569 INFO:tasks.workunit.client.0.smithi094.stdout:9/113: dwrite df/d12/d1f/d11/f13 [4194304,4194304] 0 2024-03-24T10:45:52.569 INFO:tasks.workunit.client.0.smithi094.stdout:9/114: getdents df/d12/d1f 0 2024-03-24T10:45:52.572 INFO:tasks.workunit.client.0.smithi094.stdout:0/100: getdents d14/d22 0 2024-03-24T10:45:52.573 INFO:tasks.workunit.client.0.smithi094.stdout:3/79: truncate d0/f11 4392705 0 2024-03-24T10:45:52.573 INFO:tasks.workunit.client.0.smithi094.stdout:3/80: chown d0/db/ca 1105053 1 2024-03-24T10:45:52.574 INFO:tasks.workunit.client.0.smithi094.stdout:6/101: mknod d0/d5/c1b 0 2024-03-24T10:45:52.574 INFO:tasks.workunit.client.0.smithi094.stdout:6/102: chown d0 989264 1 2024-03-24T10:45:52.578 INFO:tasks.workunit.client.0.smithi094.stdout:8/97: symlink df/d12/d18/l1f 0 2024-03-24T10:45:52.578 INFO:tasks.workunit.client.0.smithi094.stdout:4/110: mknod d0/d5/d9/c1a 0 2024-03-24T10:45:52.578 INFO:tasks.workunit.client.0.smithi094.stdout:5/79: creat d7/dc/f19 x:0 0 0 2024-03-24T10:45:52.579 INFO:tasks.workunit.client.0.smithi094.stdout:9/115: creat df/f22 x:0 0 0 2024-03-24T10:45:52.580 INFO:tasks.workunit.client.0.smithi094.stdout:6/103: unlink d0/c14 0 2024-03-24T10:45:52.581 INFO:tasks.workunit.client.0.smithi094.stdout:8/98: creat df/d12/d18/f20 x:0 0 0 2024-03-24T10:45:52.582 INFO:tasks.workunit.client.0.smithi094.stdout:8/99: readlink df/d12/d18/l1f 0 2024-03-24T10:45:52.582 INFO:tasks.workunit.client.0.smithi094.stdout:8/100: chown df/d12/d18 2793 1 2024-03-24T10:45:52.583 INFO:tasks.workunit.client.0.smithi094.stdout:4/111: mkdir d0/d5/d1b 0 2024-03-24T10:45:52.583 INFO:tasks.workunit.client.0.smithi094.stdout:4/112: rename d0/d5/d9/de to d0/d5/d9/de/d1c 22 2024-03-24T10:45:52.584 INFO:tasks.workunit.client.0.smithi094.stdout:6/104: symlink d0/d1a/l1c 0 2024-03-24T10:45:52.586 INFO:tasks.workunit.client.0.smithi094.stdout:8/101: creat df/d12/f21 x:0 0 0 2024-03-24T10:45:52.586 INFO:tasks.workunit.client.0.smithi094.stdout:8/102: fdatasync fd 0 2024-03-24T10:45:52.586 INFO:tasks.workunit.client.0.smithi094.stdout:7/94: dwrite d1/f2 [0,4194304] 0 2024-03-24T10:45:52.586 INFO:tasks.workunit.client.0.smithi094.stdout:7/95: creat d1/d9/f22 x:0 0 0 2024-03-24T10:45:52.587 INFO:tasks.workunit.client.0.smithi094.stdout:0/101: dwrite fc [0,4194304] 0 2024-03-24T10:45:52.587 INFO:tasks.workunit.client.0.smithi094.stdout:4/113: unlink d0/d5/d9/lc 0 2024-03-24T10:45:52.588 INFO:tasks.workunit.client.0.smithi094.stdout:3/81: dread d0/db/de/ff [0,4194304] 0 2024-03-24T10:45:52.588 INFO:tasks.workunit.client.0.smithi094.stdout:6/105: truncate d0/f2 2115506 0 2024-03-24T10:45:52.588 INFO:tasks.workunit.client.0.smithi094.stdout:6/106: fsync d0/f3 0 2024-03-24T10:45:52.588 INFO:tasks.workunit.client.0.smithi094.stdout:6/107: write d0/d5/da/f12 [330598,42740] 0 2024-03-24T10:45:52.589 INFO:tasks.workunit.client.0.smithi094.stdout:5/80: write d7/d8/fa [6475344,112949] 0 2024-03-24T10:45:52.590 INFO:tasks.workunit.client.0.smithi094.stdout:5/81: chown c5 17 1 2024-03-24T10:45:52.590 INFO:tasks.workunit.client.0.smithi094.stdout:8/103: mkdir df/d12/d18/d22 0 2024-03-24T10:45:52.590 INFO:tasks.workunit.client.0.smithi094.stdout:8/104: dread - df/d12/d18/f20 zero size 2024-03-24T10:45:52.590 INFO:tasks.workunit.client.0.smithi094.stdout:8/105: write f6 [871591,130063] 0 2024-03-24T10:45:52.590 INFO:tasks.workunit.client.0.smithi094.stdout:9/116: dread df/d12/f15 [0,4194304] 0 2024-03-24T10:45:52.592 INFO:tasks.workunit.client.0.smithi094.stdout:7/96: rename d1/f1b to d1/d9/f23 0 2024-03-24T10:45:52.592 INFO:tasks.workunit.client.0.smithi094.stdout:7/97: chown d1/f2 3778882 1 2024-03-24T10:45:52.592 INFO:tasks.workunit.client.0.smithi094.stdout:8/106: dread df/f16 [0,4194304] 0 2024-03-24T10:45:52.592 INFO:tasks.workunit.client.0.smithi094.stdout:8/107: creat df/d12/d18/f23 x:0 0 0 2024-03-24T10:45:52.594 INFO:tasks.workunit.client.0.smithi094.stdout:4/114: symlink d0/d5/d9/l1d 0 2024-03-24T10:45:52.594 INFO:tasks.workunit.client.0.smithi094.stdout:4/115: chown d0/c2 2537 1 2024-03-24T10:45:52.595 INFO:tasks.workunit.client.0.smithi094.stdout:0/102: write fc [893923,38196] 0 2024-03-24T10:45:52.596 INFO:tasks.workunit.client.0.smithi094.stdout:8/108: dread df/f1b [0,4194304] 0 2024-03-24T10:45:52.596 INFO:tasks.workunit.client.0.smithi094.stdout:8/109: truncate df/d12/f21 94437 0 2024-03-24T10:45:52.596 INFO:tasks.workunit.client.0.smithi094.stdout:6/108: truncate d0/f18 927321 0 2024-03-24T10:45:52.596 INFO:tasks.workunit.client.0.smithi094.stdout:6/109: write d0/d5/f19 [281567,74271] 0 2024-03-24T10:45:52.597 INFO:tasks.workunit.client.0.smithi094.stdout:5/82: creat d7/d8/d14/f1a x:0 0 0 2024-03-24T10:45:52.599 INFO:tasks.workunit.client.0.smithi094.stdout:2/68: sync 2024-03-24T10:45:52.599 INFO:tasks.workunit.client.0.smithi094.stdout:1/73: sync 2024-03-24T10:45:52.600 INFO:tasks.workunit.client.0.smithi094.stdout:7/98: mknod d1/d9/d16/c24 0 2024-03-24T10:45:52.600 INFO:tasks.workunit.client.0.smithi094.stdout:7/99: dread - d1/d9/f22 zero size 2024-03-24T10:45:52.600 INFO:tasks.workunit.client.0.smithi094.stdout:7/100: getdents d1/d9/d16/d21 0 2024-03-24T10:45:52.602 INFO:tasks.workunit.client.0.smithi094.stdout:4/116: creat d0/d5/d1b/f1e x:0 0 0 2024-03-24T10:45:52.603 INFO:tasks.workunit.client.0.smithi094.stdout:0/103: read d14/d18/f1a [3314871,19088] 0 2024-03-24T10:45:52.603 INFO:tasks.workunit.client.0.smithi094.stdout:5/83: mkdir d7/d8/d1b 0 2024-03-24T10:45:52.603 INFO:tasks.workunit.client.0.smithi094.stdout:5/84: chown d7/d8/d14/f1a 784 1 2024-03-24T10:45:52.605 INFO:tasks.workunit.client.0.smithi094.stdout:9/117: getdents df/d12 0 2024-03-24T10:45:52.606 INFO:tasks.workunit.client.0.smithi094.stdout:9/118: creat df/f23 x:0 0 0 2024-03-24T10:45:52.606 INFO:tasks.workunit.client.0.smithi094.stdout:9/119: stat df/d12 0 2024-03-24T10:45:52.606 INFO:tasks.workunit.client.0.smithi094.stdout:9/120: dread - df/f22 zero size 2024-03-24T10:45:52.606 INFO:tasks.workunit.client.0.smithi094.stdout:9/121: chown df/f14 684387447 1 2024-03-24T10:45:52.606 INFO:tasks.workunit.client.0.smithi094.stdout:9/122: rename df to df/d12/d1f/d11/d24 22 2024-03-24T10:45:52.606 INFO:tasks.workunit.client.0.smithi094.stdout:9/123: write df/f23 [953421,22526] 0 2024-03-24T10:45:52.606 INFO:tasks.workunit.client.0.smithi094.stdout:9/124: fsync df/f22 0 2024-03-24T10:45:52.606 INFO:tasks.workunit.client.0.smithi094.stdout:9/125: read df/f14 [3916301,79052] 0 2024-03-24T10:45:52.606 INFO:tasks.workunit.client.0.smithi094.stdout:2/69: dread - fe zero size 2024-03-24T10:45:52.606 INFO:tasks.workunit.client.0.smithi094.stdout:2/70: fsync fe 0 2024-03-24T10:45:52.607 INFO:tasks.workunit.client.0.smithi094.stdout:9/126: dread df/d12/f15 [0,4194304] 0 2024-03-24T10:45:52.607 INFO:tasks.workunit.client.0.smithi094.stdout:1/74: truncate d4/d9/fd 2816598 0 2024-03-24T10:45:52.609 INFO:tasks.workunit.client.0.smithi094.stdout:7/101: mkdir d1/d9/d25 0 2024-03-24T10:45:52.609 INFO:tasks.workunit.client.0.smithi094.stdout:7/102: readlink d1/d9/lf 0 2024-03-24T10:45:52.612 INFO:tasks.workunit.client.0.smithi094.stdout:0/104: symlink d14/d22/l24 0 2024-03-24T10:45:52.612 INFO:tasks.workunit.client.0.smithi094.stdout:6/110: dwrite d0/d5/da/dd/f13 [4194304,4194304] 0 2024-03-24T10:45:52.612 INFO:tasks.workunit.client.0.smithi094.stdout:6/111: chown d0/c4 3 1 2024-03-24T10:45:52.612 INFO:tasks.workunit.client.0.smithi094.stdout:6/112: write d0/d5/f19 [494583,41606] 0 2024-03-24T10:45:52.612 INFO:tasks.workunit.client.0.smithi094.stdout:8/110: getdents df/d12 0 2024-03-24T10:45:52.618 INFO:tasks.workunit.client.0.smithi094.stdout:9/127: symlink df/d12/d1f/l25 0 2024-03-24T10:45:52.619 INFO:tasks.workunit.client.0.smithi094.stdout:1/75: link d4/d9/db/l15 d4/l19 0 2024-03-24T10:45:52.621 INFO:tasks.workunit.client.0.smithi094.stdout:5/85: dwrite d7/d8/d14/f1a [0,4194304] 0 2024-03-24T10:45:52.622 INFO:tasks.workunit.client.0.smithi094.stdout:0/105: rename d14/d18/f1a to d14/d18/f25 0 2024-03-24T10:45:52.622 INFO:tasks.workunit.client.0.smithi094.stdout:9/128: dread df/f23 [0,4194304] 0 2024-03-24T10:45:52.623 INFO:tasks.workunit.client.0.smithi094.stdout:2/71: mknod c11 0 2024-03-24T10:45:52.626 INFO:tasks.workunit.client.0.smithi094.stdout:1/76: rename d4/d9/db/f13 to d4/d9/db/f1a 0 2024-03-24T10:45:52.626 INFO:tasks.workunit.client.0.smithi094.stdout:1/77: truncate d4/d9/db/f1a 4663993 0 2024-03-24T10:45:52.626 INFO:tasks.workunit.client.0.smithi094.stdout:1/78: stat d4/cf 0 2024-03-24T10:45:52.631 INFO:tasks.workunit.client.0.smithi094.stdout:0/106: dread f0 [0,4194304] 0 2024-03-24T10:45:52.631 INFO:tasks.workunit.client.0.smithi094.stdout:5/86: rmdir d7/dc/d12 39 2024-03-24T10:45:52.631 INFO:tasks.workunit.client.0.smithi094.stdout:6/113: write d0/f17 [136901,24853] 0 2024-03-24T10:45:52.634 INFO:tasks.workunit.client.0.smithi094.stdout:8/111: dwrite df/f16 [0,4194304] 0 2024-03-24T10:45:52.634 INFO:tasks.workunit.client.0.smithi094.stdout:4/117: fsync d0/d5/d1b/f1e 0 2024-03-24T10:45:52.634 INFO:tasks.workunit.client.0.smithi094.stdout:9/129: rename lb to df/d12/d1f/d11/l26 0 2024-03-24T10:45:52.635 INFO:tasks.workunit.client.0.smithi094.stdout:0/107: dread d14/d18/f25 [0,4194304] 0 2024-03-24T10:45:52.637 INFO:tasks.workunit.client.0.smithi094.stdout:1/79: creat d4/d9/de/f1b x:0 0 0 2024-03-24T10:45:52.638 INFO:tasks.workunit.client.0.smithi094.stdout:5/87: creat d7/d8/d1b/f1c x:0 0 0 2024-03-24T10:45:52.644 INFO:tasks.workunit.client.0.smithi094.stdout:8/112: rename df/l1d to df/d12/d18/l24 0 2024-03-24T10:45:52.644 INFO:tasks.workunit.client.0.smithi094.stdout:8/113: readlink l7 0 2024-03-24T10:45:52.648 INFO:tasks.workunit.client.0.smithi094.stdout:4/118: write d0/d5/d9/f11 [1276585,16075] 0 2024-03-24T10:45:52.652 INFO:tasks.workunit.client.0.smithi094.stdout:5/88: link d7/d8/d14/l17 d7/dc/d12/l1d 0 2024-03-24T10:45:52.655 INFO:tasks.workunit.client.0.smithi094.stdout:9/130: dwrite df/f23 [0,4194304] 0 2024-03-24T10:45:52.656 INFO:tasks.workunit.client.0.smithi094.stdout:7/103: truncate d1/f2 885176 0 2024-03-24T10:45:52.656 INFO:tasks.workunit.client.0.smithi094.stdout:7/104: fdatasync d1/f10 0 2024-03-24T10:45:52.669 INFO:tasks.workunit.client.0.smithi094.stdout:0/108: rename d14/c17 to d14/d18/c26 0 2024-03-24T10:45:52.669 INFO:tasks.workunit.client.0.smithi094.stdout:0/109: chown le 21921 1 2024-03-24T10:45:52.669 INFO:tasks.workunit.client.0.smithi094.stdout:0/110: creat d14/f27 x:0 0 0 2024-03-24T10:45:52.671 INFO:tasks.workunit.client.0.smithi094.stdout:9/131: mknod df/c27 0 2024-03-24T10:45:52.672 INFO:tasks.workunit.client.0.smithi094.stdout:7/105: creat d1/d9/d16/d20/f26 x:0 0 0 2024-03-24T10:45:52.673 INFO:tasks.workunit.client.0.smithi094.stdout:3/82: truncate d0/f11 515575 0 2024-03-24T10:45:52.675 INFO:tasks.workunit.client.0.smithi094.stdout:0/111: mkdir d14/d22/d28 0 2024-03-24T10:45:52.676 INFO:tasks.workunit.client.0.smithi094.stdout:0/112: fsync fa 0 2024-03-24T10:45:52.677 INFO:tasks.workunit.client.0.smithi094.stdout:4/119: mkdir d0/d1f 0 2024-03-24T10:45:52.678 INFO:tasks.workunit.client.0.smithi094.stdout:3/83: symlink d0/db/l18 0 2024-03-24T10:45:52.680 INFO:tasks.workunit.client.0.smithi094.stdout:4/120: symlink d0/l20 0 2024-03-24T10:45:52.682 INFO:tasks.workunit.client.0.smithi094.stdout:0/113: dread d14/d18/f1d [0,4194304] 0 2024-03-24T10:45:52.682 INFO:tasks.workunit.client.0.smithi094.stdout:0/114: stat f9 0 2024-03-24T10:45:52.682 INFO:tasks.workunit.client.0.smithi094.stdout:0/115: stat d14/l20 0 2024-03-24T10:45:52.682 INFO:tasks.workunit.client.0.smithi094.stdout:0/116: chown le 965731258 1 2024-03-24T10:45:52.683 INFO:tasks.workunit.client.0.smithi094.stdout:4/121: dread d0/d5/d9/ff [0,4194304] 0 2024-03-24T10:45:52.683 INFO:tasks.workunit.client.0.smithi094.stdout:0/117: link f5 d14/d22/f29 0 2024-03-24T10:45:52.684 INFO:tasks.workunit.client.0.smithi094.stdout:4/122: dread d0/f7 [0,4194304] 0 2024-03-24T10:45:52.685 INFO:tasks.workunit.client.0.smithi094.stdout:4/123: mknod d0/d5/d9/de/c21 0 2024-03-24T10:45:52.688 INFO:tasks.workunit.client.0.smithi094.stdout:4/124: write d0/d5/d9/f12 [405290,125184] 0 2024-03-24T10:45:52.690 INFO:tasks.workunit.client.0.smithi094.stdout:2/72: dwrite f7 [0,4194304] 0 2024-03-24T10:45:52.690 INFO:tasks.workunit.client.0.smithi094.stdout:8/114: dwrite f8 [0,4194304] 0 2024-03-24T10:45:52.691 INFO:tasks.workunit.client.0.smithi094.stdout:1/80: dwrite d4/f16 [0,4194304] 0 2024-03-24T10:45:52.691 INFO:tasks.workunit.client.0.smithi094.stdout:1/81: fdatasync d4/d9/fd 0 2024-03-24T10:45:52.691 INFO:tasks.workunit.client.0.smithi094.stdout:5/89: dwrite d7/d8/fa [0,4194304] 0 2024-03-24T10:45:52.692 INFO:tasks.workunit.client.0.smithi094.stdout:4/125: fdatasync d0/d5/d9/f12 0 2024-03-24T10:45:52.692 INFO:tasks.workunit.client.0.smithi094.stdout:4/126: write d0/f7 [1449807,32727] 0 2024-03-24T10:45:52.692 INFO:tasks.workunit.client.0.smithi094.stdout:4/127: write d0/d5/d9/f11 [4515516,34122] 0 2024-03-24T10:45:52.694 INFO:tasks.workunit.client.0.smithi094.stdout:8/115: mkdir df/d25 0 2024-03-24T10:45:52.697 INFO:tasks.workunit.client.0.smithi094.stdout:5/90: dread d7/f10 [0,4194304] 0 2024-03-24T10:45:52.697 INFO:tasks.workunit.client.0.smithi094.stdout:5/91: write d7/d8/d1b/f1c [594184,121918] 0 2024-03-24T10:45:52.697 INFO:tasks.workunit.client.0.smithi094.stdout:5/92: chown d7/dc 0 1 2024-03-24T10:45:52.698 INFO:tasks.workunit.client.0.smithi094.stdout:5/93: rename d7 to d7/dc/d12/d1e 22 2024-03-24T10:45:52.698 INFO:tasks.workunit.client.0.smithi094.stdout:5/94: write d7/d8/d1b/f1c [270057,129683] 0 2024-03-24T10:45:52.704 INFO:tasks.workunit.client.0.smithi094.stdout:8/116: mknod df/d12/c26 0 2024-03-24T10:45:52.707 INFO:tasks.workunit.client.0.smithi094.stdout:7/106: getdents d1/d9/d16/d20 0 2024-03-24T10:45:52.708 INFO:tasks.workunit.client.0.smithi094.stdout:3/84: dwrite d0/db/de/ff [0,4194304] 0 2024-03-24T10:45:52.710 INFO:tasks.workunit.client.0.smithi094.stdout:7/107: truncate f0 771298 0 2024-03-24T10:45:52.710 INFO:tasks.workunit.client.0.smithi094.stdout:7/108: chown d1/d9/d16/d20 34 1 2024-03-24T10:45:52.711 INFO:tasks.workunit.client.0.smithi094.stdout:8/117: dread df/f11 [0,4194304] 0 2024-03-24T10:45:52.712 INFO:tasks.workunit.client.0.smithi094.stdout:8/118: stat df/d12/d18/f20 0 2024-03-24T10:45:52.714 INFO:tasks.workunit.client.0.smithi094.stdout:9/132: truncate df/f23 1052219 0 2024-03-24T10:45:52.716 INFO:tasks.workunit.client.0.smithi094.stdout:3/85: unlink d0/f10 0 2024-03-24T10:45:52.716 INFO:tasks.workunit.client.0.smithi094.stdout:2/73: dwrite fe [0,4194304] 0 2024-03-24T10:45:52.716 INFO:tasks.workunit.client.0.smithi094.stdout:5/95: dwrite d7/f10 [0,4194304] 0 2024-03-24T10:45:52.719 INFO:tasks.workunit.client.0.smithi094.stdout:9/133: mkdir df/d28 0 2024-03-24T10:45:52.719 INFO:tasks.workunit.client.0.smithi094.stdout:9/134: read - df/f22 zero size 2024-03-24T10:45:52.720 INFO:tasks.workunit.client.0.smithi094.stdout:3/86: mknod d0/db/dc/c19 0 2024-03-24T10:45:52.720 INFO:tasks.workunit.client.0.smithi094.stdout:3/87: read - d0/db/dc/f12 zero size 2024-03-24T10:45:52.723 INFO:tasks.workunit.client.0.smithi094.stdout:5/96: link d7/d8/d1b/f1c d7/dc/d12/f1f 0 2024-03-24T10:45:52.725 INFO:tasks.workunit.client.0.smithi094.stdout:7/109: symlink d1/d9/d14/l27 0 2024-03-24T10:45:52.726 INFO:tasks.workunit.client.0.smithi094.stdout:4/128: dwrite d0/f6 [0,4194304] 0 2024-03-24T10:45:52.726 INFO:tasks.workunit.client.0.smithi094.stdout:3/88: mknod d0/db/de/c1a 0 2024-03-24T10:45:52.729 INFO:tasks.workunit.client.0.smithi094.stdout:2/74: dread fe [0,4194304] 0 2024-03-24T10:45:52.729 INFO:tasks.workunit.client.0.smithi094.stdout:5/97: rename d7/d8/d14/l17 to d7/d8/d14/l20 0 2024-03-24T10:45:52.730 INFO:tasks.workunit.client.0.smithi094.stdout:8/119: dwrite f6 [0,4194304] 0 2024-03-24T10:45:52.730 INFO:tasks.workunit.client.0.smithi094.stdout:8/120: dread - df/d12/d18/f20 zero size 2024-03-24T10:45:52.730 INFO:tasks.workunit.client.0.smithi094.stdout:8/121: truncate f4 4427088 0 2024-03-24T10:45:52.730 INFO:tasks.workunit.client.0.smithi094.stdout:9/135: write df/d12/f1e [32725,89393] 0 2024-03-24T10:45:52.731 INFO:tasks.workunit.client.0.smithi094.stdout:1/82: truncate d4/f16 2222595 0 2024-03-24T10:45:52.733 INFO:tasks.workunit.client.0.smithi094.stdout:2/75: creat f12 x:0 0 0 2024-03-24T10:45:52.738 INFO:tasks.workunit.client.0.smithi094.stdout:9/136: mkdir df/d12/d1f/d29 0 2024-03-24T10:45:52.741 INFO:tasks.workunit.client.0.smithi094.stdout:1/83: symlink d4/l1c 0 2024-03-24T10:45:52.746 INFO:tasks.workunit.client.0.smithi094.stdout:5/98: write d7/d8/d14/f1a [916316,67680] 0 2024-03-24T10:45:52.746 INFO:tasks.workunit.client.0.smithi094.stdout:2/76: symlink l13 0 2024-03-24T10:45:52.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:52 smithi094 ceph-mon[29446]: pgmap v239: 65 pgs: 65 active+clean; 271 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 4.0 MiB/s rd, 12 MiB/s wr, 534 op/s 2024-03-24T10:45:52.748 INFO:tasks.workunit.client.0.smithi094.stdout:4/129: rename d0/d5/d9 to d0/d22 0 2024-03-24T10:45:52.749 INFO:tasks.workunit.client.0.smithi094.stdout:1/84: rmdir d4/d9/de 39 2024-03-24T10:45:52.751 INFO:tasks.workunit.client.0.smithi094.stdout:5/99: symlink d7/dc/d12/l21 0 2024-03-24T10:45:52.753 INFO:tasks.workunit.client.0.smithi094.stdout:5/100: symlink d7/de/l22 0 2024-03-24T10:45:52.754 INFO:tasks.workunit.client.0.smithi094.stdout:4/130: rename d0/c2 to d0/c23 0 2024-03-24T10:45:52.754 INFO:tasks.workunit.client.0.smithi094.stdout:4/131: chown d0/d5/f16 3 1 2024-03-24T10:45:52.754 INFO:tasks.workunit.client.0.smithi094.stdout:4/132: fdatasync d0/f7 0 2024-03-24T10:45:52.755 INFO:tasks.workunit.client.0.smithi094.stdout:7/110: dwrite d1/f2 [0,4194304] 0 2024-03-24T10:45:52.756 INFO:tasks.workunit.client.0.smithi094.stdout:8/122: dwrite df/f1b [0,4194304] 0 2024-03-24T10:45:52.756 INFO:tasks.workunit.client.0.smithi094.stdout:3/89: dwrite d0/f3 [0,4194304] 0 2024-03-24T10:45:52.756 INFO:tasks.workunit.client.0.smithi094.stdout:6/114: sync 2024-03-24T10:45:52.756 INFO:tasks.workunit.client.0.smithi094.stdout:6/115: write d0/f8 [404991,20730] 0 2024-03-24T10:45:52.759 INFO:tasks.workunit.client.0.smithi094.stdout:9/137: dread df/f23 [0,4194304] 0 2024-03-24T10:45:52.759 INFO:tasks.workunit.client.0.smithi094.stdout:9/138: creat df/f2a x:0 0 0 2024-03-24T10:45:52.759 INFO:tasks.workunit.client.0.smithi094.stdout:9/139: chown cc 808088 1 2024-03-24T10:45:52.759 INFO:tasks.workunit.client.0.smithi094.stdout:9/140: read df/d12/f15 [559457,105533] 0 2024-03-24T10:45:52.759 INFO:tasks.workunit.client.0.smithi094.stdout:9/141: write df/d12/f1e [4840529,44484] 0 2024-03-24T10:45:52.760 INFO:tasks.workunit.client.0.smithi094.stdout:3/90: dread d0/f11 [0,4194304] 0 2024-03-24T10:45:52.760 INFO:tasks.workunit.client.0.smithi094.stdout:3/91: creat d0/f1b x:0 0 0 2024-03-24T10:45:52.761 INFO:tasks.workunit.client.0.smithi094.stdout:6/116: dread d0/f18 [0,4194304] 0 2024-03-24T10:45:52.761 INFO:tasks.workunit.client.0.smithi094.stdout:6/117: fdatasync d0/f2 0 2024-03-24T10:45:52.761 INFO:tasks.workunit.client.0.smithi094.stdout:6/118: fsync d0/d5/f19 0 2024-03-24T10:45:52.762 INFO:tasks.workunit.client.0.smithi094.stdout:7/111: creat d1/d9/d16/d20/f28 x:0 0 0 2024-03-24T10:45:52.763 INFO:tasks.workunit.client.0.smithi094.stdout:8/123: symlink df/d12/d18/d22/l27 0 2024-03-24T10:45:52.764 INFO:tasks.workunit.client.0.smithi094.stdout:8/124: write df/d12/d18/f23 [47413,81848] 0 2024-03-24T10:45:52.765 INFO:tasks.workunit.client.0.smithi094.stdout:9/142: dread df/f14 [0,4194304] 0 2024-03-24T10:45:52.766 INFO:tasks.workunit.client.0.smithi094.stdout:9/143: stat df/f22 0 2024-03-24T10:45:52.778 INFO:tasks.workunit.client.0.smithi094.stdout:7/112: getdents d1/d9/d14 0 2024-03-24T10:45:52.779 INFO:tasks.workunit.client.0.smithi094.stdout:8/125: mkdir df/d25/d28 0 2024-03-24T10:45:52.780 INFO:tasks.workunit.client.0.smithi094.stdout:9/144: rename df/f23 to df/f2b 0 2024-03-24T10:45:52.781 INFO:tasks.workunit.client.0.smithi094.stdout:3/92: unlink d0/f1b 0 2024-03-24T10:45:52.782 INFO:tasks.workunit.client.0.smithi094.stdout:8/126: link df/d12/f21 df/d25/f29 0 2024-03-24T10:45:52.782 INFO:tasks.workunit.client.0.smithi094.stdout:8/127: write df/d12/d18/f20 [964251,104983] 0 2024-03-24T10:45:52.783 INFO:tasks.workunit.client.0.smithi094.stdout:9/145: truncate df/f2b 999675 0 2024-03-24T10:45:52.785 INFO:tasks.workunit.client.0.smithi094.stdout:3/93: mkdir d0/db/d1c 0 2024-03-24T10:45:52.786 INFO:tasks.workunit.client.0.smithi094.stdout:6/119: rename d0/d5/da/dd to d0/d1a/d1d 0 2024-03-24T10:45:52.786 INFO:tasks.workunit.client.0.smithi094.stdout:6/120: chown d0/d5 7 1 2024-03-24T10:45:52.787 INFO:tasks.workunit.client.0.smithi094.stdout:8/128: link df/d12/c19 df/c2a 0 2024-03-24T10:45:52.787 INFO:tasks.workunit.client.0.smithi094.stdout:8/129: write df/d12/f21 [442765,29407] 0 2024-03-24T10:45:52.788 INFO:tasks.workunit.client.0.smithi094.stdout:9/146: rename df/d12/d1f/d29 to df/d28/d2c 0 2024-03-24T10:45:52.793 INFO:tasks.workunit.client.0.smithi094.stdout:8/130: dread fc [0,4194304] 0 2024-03-24T10:45:52.794 INFO:tasks.workunit.client.0.smithi094.stdout:8/131: link df/f1b df/f2b 0 2024-03-24T10:45:52.795 INFO:tasks.workunit.client.0.smithi094.stdout:8/132: creat df/d25/d28/f2c x:0 0 0 2024-03-24T10:45:52.796 INFO:tasks.workunit.client.0.smithi094.stdout:8/133: unlink df/f11 0 2024-03-24T10:45:52.798 INFO:tasks.workunit.client.0.smithi094.stdout:8/134: creat df/d25/f2d x:0 0 0 2024-03-24T10:45:52.800 INFO:tasks.workunit.client.0.smithi094.stdout:3/94: write d0/f2 [608829,48413] 0 2024-03-24T10:45:52.800 INFO:tasks.workunit.client.0.smithi094.stdout:3/95: write d0/f2 [4804801,81361] 0 2024-03-24T10:45:52.800 INFO:tasks.workunit.client.0.smithi094.stdout:1/85: dwrite d4/f16 [0,4194304] 0 2024-03-24T10:45:52.800 INFO:tasks.workunit.client.0.smithi094.stdout:1/86: creat d4/d9/db/f1d x:0 0 0 2024-03-24T10:45:52.800 INFO:tasks.workunit.client.0.smithi094.stdout:1/87: fdatasync d4/d9/db/f1a 0 2024-03-24T10:45:52.800 INFO:tasks.workunit.client.0.smithi094.stdout:1/88: creat d4/f1e x:0 0 0 2024-03-24T10:45:52.801 INFO:tasks.workunit.client.0.smithi094.stdout:5/101: dwrite d7/dc/d12/f1f [0,4194304] 0 2024-03-24T10:45:52.801 INFO:tasks.workunit.client.0.smithi094.stdout:4/133: dwrite d0/f6 [0,4194304] 0 2024-03-24T10:45:52.801 INFO:tasks.workunit.client.0.smithi094.stdout:4/134: chown d0/d5/l19 6755 1 2024-03-24T10:45:52.801 INFO:tasks.workunit.client.0.smithi094.stdout:4/135: write d0/d22/f12 [1430846,44621] 0 2024-03-24T10:45:52.801 INFO:tasks.workunit.client.0.smithi094.stdout:4/136: stat d0/d5/f16 0 2024-03-24T10:45:52.801 INFO:tasks.workunit.client.0.smithi094.stdout:8/135: symlink df/d25/d28/l2e 0 2024-03-24T10:45:52.801 INFO:tasks.workunit.client.0.smithi094.stdout:7/113: dwrite f0 [0,4194304] 0 2024-03-24T10:45:52.801 INFO:tasks.workunit.client.0.smithi094.stdout:3/96: symlink d0/db/d1c/l1d 0 2024-03-24T10:45:52.806 INFO:tasks.workunit.client.0.smithi094.stdout:5/102: mkdir d7/d23 0 2024-03-24T10:45:52.806 INFO:tasks.workunit.client.0.smithi094.stdout:5/103: chown d7/d8/d9/l13 1067456 1 2024-03-24T10:45:52.807 INFO:tasks.workunit.client.0.smithi094.stdout:8/136: rmdir df/d12/d18/d22 39 2024-03-24T10:45:52.807 INFO:tasks.workunit.client.0.smithi094.stdout:8/137: chown df/l1c 747571626 1 2024-03-24T10:45:52.808 INFO:tasks.workunit.client.0.smithi094.stdout:7/114: symlink d1/l29 0 2024-03-24T10:45:52.808 INFO:tasks.workunit.client.0.smithi094.stdout:7/115: dread - d1/d9/f22 zero size 2024-03-24T10:45:52.808 INFO:tasks.workunit.client.0.smithi094.stdout:7/116: fdatasync d1/f7 0 2024-03-24T10:45:52.808 INFO:tasks.workunit.client.0.smithi094.stdout:7/117: truncate d1/d9/fe 3902371 0 2024-03-24T10:45:52.809 INFO:tasks.workunit.client.0.smithi094.stdout:0/118: sync 2024-03-24T10:45:52.809 INFO:tasks.workunit.client.0.smithi094.stdout:8/138: creat df/d12/f2f x:0 0 0 2024-03-24T10:45:52.809 INFO:tasks.workunit.client.0.smithi094.stdout:8/139: creat df/d25/f30 x:0 0 0 2024-03-24T10:45:52.811 INFO:tasks.workunit.client.0.smithi094.stdout:6/121: dwrite d0/d1a/d1d/f13 [4194304,4194304] 0 2024-03-24T10:45:52.811 INFO:tasks.workunit.client.0.smithi094.stdout:6/122: creat d0/f1e x:0 0 0 2024-03-24T10:45:52.811 INFO:tasks.workunit.client.0.smithi094.stdout:6/123: fdatasync d0/d1a/d1d/f13 0 2024-03-24T10:45:52.811 INFO:tasks.workunit.client.0.smithi094.stdout:6/124: dread - d0/f1e zero size 2024-03-24T10:45:52.811 INFO:tasks.workunit.client.0.smithi094.stdout:6/125: chown d0/f2 28530491 1 2024-03-24T10:45:52.811 INFO:tasks.workunit.client.0.smithi094.stdout:7/118: symlink d1/d9/d25/l2a 0 2024-03-24T10:45:52.812 INFO:tasks.workunit.client.0.smithi094.stdout:0/119: link d14/d18/f21 d14/f2a 0 2024-03-24T10:45:52.815 INFO:tasks.workunit.client.0.smithi094.stdout:1/89: dwrite d4/d9/db/f10 [0,4194304] 0 2024-03-24T10:45:52.815 INFO:tasks.workunit.client.0.smithi094.stdout:1/90: chown f1 3983 1 2024-03-24T10:45:52.815 INFO:tasks.workunit.client.0.smithi094.stdout:1/91: dread - d4/f1e zero size 2024-03-24T10:45:52.815 INFO:tasks.workunit.client.0.smithi094.stdout:8/140: mkdir df/d25/d28/d31 0 2024-03-24T10:45:52.815 INFO:tasks.workunit.client.0.smithi094.stdout:4/137: read d0/d5/f16 [3188224,98320] 0 2024-03-24T10:45:52.818 INFO:tasks.workunit.client.0.smithi094.stdout:5/104: read d7/dc/d12/f1f [1632781,25459] 0 2024-03-24T10:45:52.824 INFO:tasks.workunit.client.0.smithi094.stdout:9/147: rename df/d12 to df/d2d 0 2024-03-24T10:45:52.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:52 smithi096 ceph-mon[38097]: pgmap v239: 65 pgs: 65 active+clean; 271 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 4.0 MiB/s rd, 12 MiB/s wr, 534 op/s 2024-03-24T10:45:52.830 INFO:tasks.workunit.client.0.smithi094.stdout:6/126: mknod d0/d5/da/c1f 0 2024-03-24T10:45:52.830 INFO:tasks.workunit.client.0.smithi094.stdout:6/127: fdatasync d0/f17 0 2024-03-24T10:45:52.831 INFO:tasks.workunit.client.0.smithi094.stdout:7/119: mknod d1/d9/d25/c2b 0 2024-03-24T10:45:52.831 INFO:tasks.workunit.client.0.smithi094.stdout:7/120: write d1/d9/f23 [1541177,121453] 0 2024-03-24T10:45:52.831 INFO:tasks.workunit.client.0.smithi094.stdout:7/121: creat d1/d9/f2c x:0 0 0 2024-03-24T10:45:52.831 INFO:tasks.workunit.client.0.smithi094.stdout:7/122: chown d1/l5 681121484 1 2024-03-24T10:45:52.831 INFO:tasks.workunit.client.0.smithi094.stdout:0/120: mkdir d14/d22/d28/d2b 0 2024-03-24T10:45:52.832 INFO:tasks.workunit.client.0.smithi094.stdout:0/121: creat d14/f2c x:0 0 0 2024-03-24T10:45:52.832 INFO:tasks.workunit.client.0.smithi094.stdout:0/122: fsync d14/d18/f25 0 2024-03-24T10:45:52.833 INFO:tasks.workunit.client.0.smithi094.stdout:3/97: dwrite d0/f3 [0,4194304] 0 2024-03-24T10:45:52.834 INFO:tasks.workunit.client.0.smithi094.stdout:1/92: link d4/d9/fa d4/d9/db/f1f 0 2024-03-24T10:45:52.835 INFO:tasks.workunit.client.0.smithi094.stdout:8/141: creat df/d25/d28/d31/f32 x:0 0 0 2024-03-24T10:45:52.836 INFO:tasks.workunit.client.0.smithi094.stdout:4/138: mkdir d0/d5/d1b/d24 0 2024-03-24T10:45:52.838 INFO:tasks.workunit.client.0.smithi094.stdout:9/148: creat df/d28/f2e x:0 0 0 2024-03-24T10:45:52.839 INFO:tasks.workunit.client.0.smithi094.stdout:6/128: unlink d0/f17 0 2024-03-24T10:45:52.841 INFO:tasks.workunit.client.0.smithi094.stdout:7/123: mknod d1/c2d 0 2024-03-24T10:45:52.841 INFO:tasks.workunit.client.0.smithi094.stdout:7/124: creat d1/d9/f2e x:0 0 0 2024-03-24T10:45:52.841 INFO:tasks.workunit.client.0.smithi094.stdout:7/125: dread - d1/f1f zero size 2024-03-24T10:45:52.842 INFO:tasks.workunit.client.0.smithi094.stdout:0/123: mknod d14/d18/c2d 0 2024-03-24T10:45:52.842 INFO:tasks.workunit.client.0.smithi094.stdout:0/124: truncate d14/f2a 1301881 0 2024-03-24T10:45:52.842 INFO:tasks.workunit.client.0.smithi094.stdout:0/125: readlink d14/d22/l24 0 2024-03-24T10:45:52.843 INFO:tasks.workunit.client.0.smithi094.stdout:0/126: creat d14/d18/f2e x:0 0 0 2024-03-24T10:45:52.845 INFO:tasks.workunit.client.0.smithi094.stdout:7/126: write d1/d9/fe [2959188,71375] 0 2024-03-24T10:45:52.845 INFO:tasks.workunit.client.0.smithi094.stdout:7/127: write d1/f1f [134384,96393] 0 2024-03-24T10:45:52.845 INFO:tasks.workunit.client.0.smithi094.stdout:7/128: fdatasync d1/d9/fd 0 2024-03-24T10:45:52.845 INFO:tasks.workunit.client.0.smithi094.stdout:7/129: truncate d1/f1f 1050415 0 2024-03-24T10:45:52.845 INFO:tasks.workunit.client.0.smithi094.stdout:4/139: mknod d0/d5/c25 0 2024-03-24T10:45:52.845 INFO:tasks.workunit.client.0.smithi094.stdout:7/130: write d1/f10 [870645,129983] 0 2024-03-24T10:45:52.845 INFO:tasks.workunit.client.0.smithi094.stdout:5/105: unlink d7/d8/d1b/f1c 0 2024-03-24T10:45:52.846 INFO:tasks.workunit.client.0.smithi094.stdout:5/106: creat d7/f24 x:0 0 0 2024-03-24T10:45:52.846 INFO:tasks.workunit.client.0.smithi094.stdout:9/149: mknod df/d2d/d1f/d11/c2f 0 2024-03-24T10:45:52.846 INFO:tasks.workunit.client.0.smithi094.stdout:9/150: truncate df/d2d/f15 1432167 0 2024-03-24T10:45:52.846 INFO:tasks.workunit.client.0.smithi094.stdout:9/151: creat df/d28/f30 x:0 0 0 2024-03-24T10:45:52.849 INFO:tasks.workunit.client.0.smithi094.stdout:6/129: rename d0/f3 to d0/d5/da/f20 0 2024-03-24T10:45:52.849 INFO:tasks.workunit.client.0.smithi094.stdout:6/130: chown d0/d5/da/f12 2071 1 2024-03-24T10:45:52.849 INFO:tasks.workunit.client.0.smithi094.stdout:6/131: fdatasync d0/d5/da/f12 0 2024-03-24T10:45:52.851 INFO:tasks.workunit.client.0.smithi094.stdout:8/142: dwrite f5 [0,4194304] 0 2024-03-24T10:45:52.853 INFO:tasks.workunit.client.0.smithi094.stdout:0/127: link f9 d14/d22/f2f 0 2024-03-24T10:45:52.853 INFO:tasks.workunit.client.0.smithi094.stdout:0/128: dread - d14/f27 zero size 2024-03-24T10:45:52.855 INFO:tasks.workunit.client.0.smithi094.stdout:1/93: truncate d4/f16 1457465 0 2024-03-24T10:45:52.855 INFO:tasks.workunit.client.0.smithi094.stdout:2/77: sync 2024-03-24T10:45:52.858 INFO:tasks.workunit.client.0.smithi094.stdout:7/131: unlink d1/c15 0 2024-03-24T10:45:52.860 INFO:tasks.workunit.client.0.smithi094.stdout:6/132: write d0/fc [3246123,120868] 0 2024-03-24T10:45:52.861 INFO:tasks.workunit.client.0.smithi094.stdout:5/107: dwrite d7/f24 [0,4194304] 0 2024-03-24T10:45:52.861 INFO:tasks.workunit.client.0.smithi094.stdout:6/133: dread d0/d5/f19 [0,4194304] 0 2024-03-24T10:45:52.862 INFO:tasks.workunit.client.0.smithi094.stdout:1/94: fsync d4/d9/db/f1f 0 2024-03-24T10:45:52.862 INFO:tasks.workunit.client.0.smithi094.stdout:1/95: chown d4/f16 308 1 2024-03-24T10:45:52.865 INFO:tasks.workunit.client.0.smithi094.stdout:0/129: symlink d14/d22/l30 0 2024-03-24T10:45:52.867 INFO:tasks.workunit.client.0.smithi094.stdout:2/78: creat f14 x:0 0 0 2024-03-24T10:45:52.868 INFO:tasks.workunit.client.0.smithi094.stdout:0/130: dread d14/d18/f1d [0,4194304] 0 2024-03-24T10:45:52.870 INFO:tasks.workunit.client.0.smithi094.stdout:7/132: creat d1/d9/d16/f2f x:0 0 0 2024-03-24T10:45:52.871 INFO:tasks.workunit.client.0.smithi094.stdout:4/140: getdents d0/d5 0 2024-03-24T10:45:52.879 INFO:tasks.workunit.client.0.smithi094.stdout:3/98: truncate d0/f2 3355082 0 2024-03-24T10:45:52.880 INFO:tasks.workunit.client.0.smithi094.stdout:3/99: write d0/f11 [575051,116794] 0 2024-03-24T10:45:52.880 INFO:tasks.workunit.client.0.smithi094.stdout:3/100: creat d0/f1e x:0 0 0 2024-03-24T10:45:52.880 INFO:tasks.workunit.client.0.smithi094.stdout:5/108: stat d7/d8/d14/l20 0 2024-03-24T10:45:52.880 INFO:tasks.workunit.client.0.smithi094.stdout:5/109: dread - d7/dc/f19 zero size 2024-03-24T10:45:52.882 INFO:tasks.workunit.client.0.smithi094.stdout:1/96: truncate f1 3965808 0 2024-03-24T10:45:52.886 INFO:tasks.workunit.client.0.smithi094.stdout:8/143: truncate f3 3171244 0 2024-03-24T10:45:52.886 INFO:tasks.workunit.client.0.smithi094.stdout:2/79: dwrite fe [4194304,4194304] 0 2024-03-24T10:45:52.887 INFO:tasks.workunit.client.0.smithi094.stdout:0/131: rmdir d14/d22/d28 39 2024-03-24T10:45:52.887 INFO:tasks.workunit.client.0.smithi094.stdout:0/132: chown d14/d22/d28/d2b 37 1 2024-03-24T10:45:52.891 INFO:tasks.workunit.client.0.smithi094.stdout:2/80: dread fe [0,4194304] 0 2024-03-24T10:45:52.893 INFO:tasks.workunit.client.0.smithi094.stdout:0/133: creat d14/d22/d28/f31 x:0 0 0 2024-03-24T10:45:52.895 INFO:tasks.workunit.client.0.smithi094.stdout:1/97: write d4/d9/fa [1962198,69582] 0 2024-03-24T10:45:52.896 INFO:tasks.workunit.client.0.smithi094.stdout:7/133: unlink d1/d9/f2e 0 2024-03-24T10:45:52.896 INFO:tasks.workunit.client.0.smithi094.stdout:7/134: creat d1/d9/f30 x:0 0 0 2024-03-24T10:45:52.897 INFO:tasks.workunit.client.0.smithi094.stdout:8/144: write fc [2171869,35299] 0 2024-03-24T10:45:52.897 INFO:tasks.workunit.client.0.smithi094.stdout:8/145: fsync df/d12/f21 0 2024-03-24T10:45:52.898 INFO:tasks.workunit.client.0.smithi094.stdout:4/141: unlink d0/la 0 2024-03-24T10:45:52.899 INFO:tasks.workunit.client.0.smithi094.stdout:1/98: symlink d4/d9/db/l20 0 2024-03-24T10:45:52.899 INFO:tasks.workunit.client.0.smithi094.stdout:1/99: truncate d4/f1e 88225 0 2024-03-24T10:45:52.899 INFO:tasks.workunit.client.0.smithi094.stdout:2/81: dread f7 [0,4194304] 0 2024-03-24T10:45:52.902 INFO:tasks.workunit.client.0.smithi094.stdout:2/82: write f7 [621607,98428] 0 2024-03-24T10:45:52.910 INFO:tasks.workunit.client.0.smithi094.stdout:7/135: write f0 [579485,22537] 0 2024-03-24T10:45:52.910 INFO:tasks.workunit.client.0.smithi094.stdout:7/136: truncate d1/d9/d16/d20/f26 575963 0 2024-03-24T10:45:52.910 INFO:tasks.workunit.client.0.smithi094.stdout:7/137: write d1/d9/d16/d20/f26 [1612311,16641] 0 2024-03-24T10:45:52.910 INFO:tasks.workunit.client.0.smithi094.stdout:5/110: fsync d7/dc/d12/f1f 0 2024-03-24T10:45:52.917 INFO:tasks.workunit.client.0.smithi094.stdout:3/101: dread d0/f3 [0,4194304] 0 2024-03-24T10:45:52.917 INFO:tasks.workunit.client.0.smithi094.stdout:0/134: dwrite f3 [0,4194304] 0 2024-03-24T10:45:52.917 INFO:tasks.workunit.client.0.smithi094.stdout:5/111: truncate d7/f24 2760789 0 2024-03-24T10:45:52.917 INFO:tasks.workunit.client.0.smithi094.stdout:5/112: write d7/d8/fa [4399665,10616] 0 2024-03-24T10:45:52.919 INFO:tasks.workunit.client.0.smithi094.stdout:1/100: dread f1 [0,4194304] 0 2024-03-24T10:45:52.919 INFO:tasks.workunit.client.0.smithi094.stdout:1/101: stat d4/d9/db/f1f 0 2024-03-24T10:45:52.921 INFO:tasks.workunit.client.0.smithi094.stdout:3/102: rename d0/db/dc/c17 to d0/db/de/c1f 0 2024-03-24T10:45:52.921 INFO:tasks.workunit.client.0.smithi094.stdout:5/113: rename d7/d8/d1b to d7/d8/d1b/d25 22 2024-03-24T10:45:52.921 INFO:tasks.workunit.client.0.smithi094.stdout:4/142: rmdir d0 39 2024-03-24T10:45:52.924 INFO:tasks.workunit.client.0.smithi094.stdout:8/146: dwrite f3 [0,4194304] 0 2024-03-24T10:45:52.926 INFO:tasks.workunit.client.0.smithi094.stdout:0/135: mkdir d14/d22/d32 0 2024-03-24T10:45:52.926 INFO:tasks.workunit.client.0.smithi094.stdout:3/103: dread d0/f2 [0,4194304] 0 2024-03-24T10:45:52.927 INFO:tasks.workunit.client.0.smithi094.stdout:2/83: rename fe to f15 0 2024-03-24T10:45:52.927 INFO:tasks.workunit.client.0.smithi094.stdout:5/114: fdatasync d7/dc/d12/f1f 0 2024-03-24T10:45:52.928 INFO:tasks.workunit.client.0.smithi094.stdout:4/143: creat d0/d5/d1b/d24/f26 x:0 0 0 2024-03-24T10:45:52.930 INFO:tasks.workunit.client.0.smithi094.stdout:7/138: dwrite d1/f7 [4194304,4194304] 0 2024-03-24T10:45:52.931 INFO:tasks.workunit.client.0.smithi094.stdout:8/147: write df/f16 [821594,21505] 0 2024-03-24T10:45:52.934 INFO:tasks.workunit.client.0.smithi094.stdout:0/136: creat d14/d22/f33 x:0 0 0 2024-03-24T10:45:52.937 INFO:tasks.workunit.client.0.smithi094.stdout:3/104: mkdir d0/db/d1c/d20 0 2024-03-24T10:45:52.937 INFO:tasks.workunit.client.0.smithi094.stdout:0/137: read d14/d18/f1d [1625818,51405] 0 2024-03-24T10:45:52.938 INFO:tasks.workunit.client.0.smithi094.stdout:7/139: dread d1/f7 [0,4194304] 0 2024-03-24T10:45:52.938 INFO:tasks.workunit.client.0.smithi094.stdout:5/115: rename d7/de to d7/d26 0 2024-03-24T10:45:52.938 INFO:tasks.workunit.client.0.smithi094.stdout:5/116: fsync d7/f10 0 2024-03-24T10:45:52.939 INFO:tasks.workunit.client.0.smithi094.stdout:4/144: symlink d0/d22/l27 0 2024-03-24T10:45:52.944 INFO:tasks.workunit.client.0.smithi094.stdout:8/148: mkdir df/d25/d28/d33 0 2024-03-24T10:45:52.946 INFO:tasks.workunit.client.0.smithi094.stdout:2/84: dwrite f7 [4194304,4194304] 0 2024-03-24T10:45:52.946 INFO:tasks.workunit.client.0.smithi094.stdout:0/138: unlink d14/d22/l23 0 2024-03-24T10:45:52.946 INFO:tasks.workunit.client.0.smithi094.stdout:3/105: dread d0/f14 [0,4194304] 0 2024-03-24T10:45:52.947 INFO:tasks.workunit.client.0.smithi094.stdout:7/140: getdents d1 0 2024-03-24T10:45:52.947 INFO:tasks.workunit.client.0.smithi094.stdout:7/141: write d1/f10 [1480025,10930] 0 2024-03-24T10:45:52.949 INFO:tasks.workunit.client.0.smithi094.stdout:4/145: mkdir d0/d22/de/d28 0 2024-03-24T10:45:52.950 INFO:tasks.workunit.client.0.smithi094.stdout:8/149: mknod df/c34 0 2024-03-24T10:45:52.952 INFO:tasks.workunit.client.0.smithi094.stdout:4/146: dread d0/d22/f12 [0,4194304] 0 2024-03-24T10:45:52.953 INFO:tasks.workunit.client.0.smithi094.stdout:0/139: symlink d14/l34 0 2024-03-24T10:45:52.954 INFO:tasks.workunit.client.0.smithi094.stdout:9/152: sync 2024-03-24T10:45:52.954 INFO:tasks.workunit.client.0.smithi094.stdout:3/106: mknod d0/c21 0 2024-03-24T10:45:52.955 INFO:tasks.workunit.client.0.smithi094.stdout:7/142: mkdir d1/d9/d31 0 2024-03-24T10:45:52.956 INFO:tasks.workunit.client.0.smithi094.stdout:3/107: dread d0/f11 [0,4194304] 0 2024-03-24T10:45:52.956 INFO:tasks.workunit.client.0.smithi094.stdout:3/108: fsync d0/db/de/ff 0 2024-03-24T10:45:52.956 INFO:tasks.workunit.client.0.smithi094.stdout:3/109: dread - d0/db/dc/f12 zero size 2024-03-24T10:45:52.956 INFO:tasks.workunit.client.0.smithi094.stdout:5/117: rmdir d7/d8/d14 39 2024-03-24T10:45:52.958 INFO:tasks.workunit.client.0.smithi094.stdout:8/150: link df/c2a df/d12/d18/c35 0 2024-03-24T10:45:52.960 INFO:tasks.workunit.client.0.smithi094.stdout:4/147: truncate d0/d22/ff 1171042 0 2024-03-24T10:45:52.960 INFO:tasks.workunit.client.0.smithi094.stdout:3/110: dread d0/f2 [0,4194304] 0 2024-03-24T10:45:52.961 INFO:tasks.workunit.client.0.smithi094.stdout:2/85: truncate f8 1198692 0 2024-03-24T10:45:52.962 INFO:tasks.workunit.client.0.smithi094.stdout:9/153: symlink df/d2d/d1f/l31 0 2024-03-24T10:45:52.963 INFO:tasks.workunit.client.0.smithi094.stdout:7/143: symlink d1/d9/d16/l32 0 2024-03-24T10:45:52.966 INFO:tasks.workunit.client.0.smithi094.stdout:5/118: creat d7/d8/d14/f27 x:0 0 0 2024-03-24T10:45:52.967 INFO:tasks.workunit.client.0.smithi094.stdout:8/151: truncate fa 4169854 0 2024-03-24T10:45:52.967 INFO:tasks.workunit.client.0.smithi094.stdout:8/152: chown df/c13 14 1 2024-03-24T10:45:52.967 INFO:tasks.workunit.client.0.smithi094.stdout:8/153: stat df/d12/d18/d22/l27 0 2024-03-24T10:45:52.967 INFO:tasks.workunit.client.0.smithi094.stdout:9/154: dread df/d2d/f1e [0,4194304] 0 2024-03-24T10:45:52.968 INFO:tasks.workunit.client.0.smithi094.stdout:9/155: fsync df/f22 0 2024-03-24T10:45:52.968 INFO:tasks.workunit.client.0.smithi094.stdout:9/156: truncate df/f14 5124682 0 2024-03-24T10:45:52.968 INFO:tasks.workunit.client.0.smithi094.stdout:9/157: getdents df/d28/d2c 0 2024-03-24T10:45:52.968 INFO:tasks.workunit.client.0.smithi094.stdout:3/111: mknod d0/db/dc/c22 0 2024-03-24T10:45:52.973 INFO:tasks.workunit.client.0.smithi094.stdout:8/154: mkdir df/d25/d36 0 2024-03-24T10:45:52.973 INFO:tasks.workunit.client.0.smithi094.stdout:8/155: fdatasync df/d25/f29 0 2024-03-24T10:45:52.973 INFO:tasks.workunit.client.0.smithi094.stdout:8/156: fsync df/d25/d28/d31/f32 0 2024-03-24T10:45:52.976 INFO:tasks.workunit.client.0.smithi094.stdout:9/158: unlink df/d2d/d1f/l25 0 2024-03-24T10:45:52.976 INFO:tasks.workunit.client.0.smithi094.stdout:9/159: chown df/d2d/d1f/d11/c2f 1139801 1 2024-03-24T10:45:52.977 INFO:tasks.workunit.client.0.smithi094.stdout:2/86: mknod c16 0 2024-03-24T10:45:52.978 INFO:tasks.workunit.client.0.smithi094.stdout:8/157: rmdir df/d12/d18/d22 39 2024-03-24T10:45:52.979 INFO:tasks.workunit.client.0.smithi094.stdout:8/158: stat df/d25/d28/l2e 0 2024-03-24T10:45:52.979 INFO:tasks.workunit.client.0.smithi094.stdout:5/119: getdents d7/d8/d14 0 2024-03-24T10:45:52.979 INFO:tasks.workunit.client.0.smithi094.stdout:5/120: creat d7/d8/f28 x:0 0 0 2024-03-24T10:45:52.981 INFO:tasks.workunit.client.0.smithi094.stdout:9/160: mknod df/d28/d2c/c32 0 2024-03-24T10:45:52.983 INFO:tasks.workunit.client.0.smithi094.stdout:8/159: rmdir df/d25/d28/d33 0 2024-03-24T10:45:52.983 INFO:tasks.workunit.client.0.smithi094.stdout:8/160: dread - df/d12/f2f zero size 2024-03-24T10:45:52.983 INFO:tasks.workunit.client.0.smithi094.stdout:8/161: readlink lb 0 2024-03-24T10:45:52.985 INFO:tasks.workunit.client.0.smithi094.stdout:5/121: rename d7/d8/d14/f1a to d7/d8/d9/f29 0 2024-03-24T10:45:52.985 INFO:tasks.workunit.client.0.smithi094.stdout:5/122: dread - d7/d8/f28 zero size 2024-03-24T10:45:52.986 INFO:tasks.workunit.client.0.smithi094.stdout:7/144: dwrite d1/d9/d16/f2f [0,4194304] 0 2024-03-24T10:45:52.987 INFO:tasks.workunit.client.0.smithi094.stdout:2/87: symlink l17 0 2024-03-24T10:45:52.987 INFO:tasks.workunit.client.0.smithi094.stdout:2/88: truncate f14 355874 0 2024-03-24T10:45:52.987 INFO:tasks.workunit.client.0.smithi094.stdout:2/89: truncate f7 8749240 0 2024-03-24T10:45:52.987 INFO:tasks.workunit.client.0.smithi094.stdout:3/112: dwrite d0/f1e [0,4194304] 0 2024-03-24T10:45:52.988 INFO:tasks.workunit.client.0.smithi094.stdout:8/162: symlink df/d12/l37 0 2024-03-24T10:45:52.988 INFO:tasks.workunit.client.0.smithi094.stdout:5/123: truncate d7/dc/d12/f1f 2012785 0 2024-03-24T10:45:52.993 INFO:tasks.workunit.client.0.smithi094.stdout:7/145: creat d1/d9/d16/d21/f33 x:0 0 0 2024-03-24T10:45:52.993 INFO:tasks.workunit.client.0.smithi094.stdout:7/146: stat d1/l8 0 2024-03-24T10:45:52.994 INFO:tasks.workunit.client.0.smithi094.stdout:9/161: getdents df/d28 0 2024-03-24T10:45:52.994 INFO:tasks.workunit.client.0.smithi094.stdout:9/162: chown df/d2d/d1f/d11/l26 4587818 1 2024-03-24T10:45:52.994 INFO:tasks.workunit.client.0.smithi094.stdout:9/163: read - df/d28/f2e zero size 2024-03-24T10:45:52.994 INFO:tasks.workunit.client.0.smithi094.stdout:9/164: chown c5 6113 1 2024-03-24T10:45:52.994 INFO:tasks.workunit.client.0.smithi094.stdout:9/165: chown df/d2d/d1f/c1b 24537732 1 2024-03-24T10:45:53.001 INFO:tasks.workunit.client.0.smithi094.stdout:3/113: mkdir d0/db/d1c/d20/d23 0 2024-03-24T10:45:53.002 INFO:tasks.workunit.client.0.smithi094.stdout:8/163: creat df/d12/d18/d22/f38 x:0 0 0 2024-03-24T10:45:53.002 INFO:tasks.workunit.client.0.smithi094.stdout:8/164: chown df/d12/l37 133079 1 2024-03-24T10:45:53.002 INFO:tasks.workunit.client.0.smithi094.stdout:8/165: chown df/d12/l1a 523322657 1 2024-03-24T10:45:53.002 INFO:tasks.workunit.client.0.smithi094.stdout:8/166: read df/d12/d18/f23 [13024,5880] 0 2024-03-24T10:45:53.002 INFO:tasks.workunit.client.0.smithi094.stdout:7/147: mkdir d1/d9/d14/d34 0 2024-03-24T10:45:53.002 INFO:tasks.workunit.client.0.smithi094.stdout:2/90: mkdir d18 0 2024-03-24T10:45:53.003 INFO:tasks.workunit.client.0.smithi094.stdout:5/124: dwrite d7/f24 [0,4194304] 0 2024-03-24T10:45:53.004 INFO:tasks.workunit.client.0.smithi094.stdout:7/148: mkdir d1/d9/d35 0 2024-03-24T10:45:53.004 INFO:tasks.workunit.client.0.smithi094.stdout:8/167: dread df/d25/f29 [0,4194304] 0 2024-03-24T10:45:53.004 INFO:tasks.workunit.client.0.smithi094.stdout:2/91: creat d18/f19 x:0 0 0 2024-03-24T10:45:53.006 INFO:tasks.workunit.client.0.smithi094.stdout:5/125: creat d7/d8/d14/f2a x:0 0 0 2024-03-24T10:45:53.006 INFO:tasks.workunit.client.0.smithi094.stdout:5/126: chown d7/d23 5273160 1 2024-03-24T10:45:53.008 INFO:tasks.workunit.client.0.smithi094.stdout:2/92: mknod d18/c1a 0 2024-03-24T10:45:53.008 INFO:tasks.workunit.client.0.smithi094.stdout:2/93: creat d18/f1b x:0 0 0 2024-03-24T10:45:53.011 INFO:tasks.workunit.client.0.smithi094.stdout:9/166: dwrite df/d2d/f1e [4194304,4194304] 0 2024-03-24T10:45:53.013 INFO:tasks.workunit.client.0.smithi094.stdout:9/167: symlink df/d28/d2c/l33 0 2024-03-24T10:45:53.013 INFO:tasks.workunit.client.0.smithi094.stdout:9/168: write df/f22 [1036211,95084] 0 2024-03-24T10:45:53.013 INFO:tasks.workunit.client.0.smithi094.stdout:9/169: fdatasync df/d2d/f1e 0 2024-03-24T10:45:53.015 INFO:tasks.workunit.client.0.smithi094.stdout:8/168: rmdir df/d12 39 2024-03-24T10:45:53.015 INFO:tasks.workunit.client.0.smithi094.stdout:8/169: chown df/d12/d18/f20 31 1 2024-03-24T10:45:53.016 INFO:tasks.workunit.client.0.smithi094.stdout:8/170: creat df/d12/f39 x:0 0 0 2024-03-24T10:45:53.017 INFO:tasks.workunit.client.0.smithi094.stdout:8/171: symlink df/d25/d28/l3a 0 2024-03-24T10:45:53.017 INFO:tasks.workunit.client.0.smithi094.stdout:8/172: truncate df/d12/d18/f23 377685 0 2024-03-24T10:45:53.017 INFO:tasks.workunit.client.0.smithi094.stdout:5/127: write d7/f10 [3220622,1981] 0 2024-03-24T10:45:53.018 INFO:tasks.workunit.client.0.smithi094.stdout:5/128: mkdir d7/d23/d2b 0 2024-03-24T10:45:53.018 INFO:tasks.workunit.client.0.smithi094.stdout:5/129: read d7/d8/f16 [245618,72977] 0 2024-03-24T10:45:53.019 INFO:tasks.workunit.client.0.smithi094.stdout:2/94: rmdir d18 39 2024-03-24T10:45:53.037 INFO:tasks.workunit.client.0.smithi094.stdout:9/170: dwrite df/f2a [0,4194304] 0 2024-03-24T10:45:53.037 INFO:tasks.workunit.client.0.smithi094.stdout:9/171: write df/d28/f2e [1015469,26091] 0 2024-03-24T10:45:53.038 INFO:tasks.workunit.client.0.smithi094.stdout:5/130: dread d7/f10 [0,4194304] 0 2024-03-24T10:45:53.040 INFO:tasks.workunit.client.0.smithi094.stdout:9/172: unlink df/d2d/d1f/d11/l26 0 2024-03-24T10:45:53.040 INFO:tasks.workunit.client.0.smithi094.stdout:5/131: symlink d7/d26/l2c 0 2024-03-24T10:45:53.042 INFO:tasks.workunit.client.0.smithi094.stdout:9/173: unlink df/d2d/d1f/c1b 0 2024-03-24T10:45:53.042 INFO:tasks.workunit.client.0.smithi094.stdout:9/174: write df/d28/f2e [1403162,22691] 0 2024-03-24T10:45:53.042 INFO:tasks.workunit.client.0.smithi094.stdout:2/95: dwrite f7 [0,4194304] 0 2024-03-24T10:45:53.042 INFO:tasks.workunit.client.0.smithi094.stdout:5/132: symlink d7/d26/l2d 0 2024-03-24T10:45:53.044 INFO:tasks.workunit.client.0.smithi094.stdout:9/175: rename df/d28 to df/d2d/d34 0 2024-03-24T10:45:53.044 INFO:tasks.workunit.client.0.smithi094.stdout:9/176: chown df/d2d/d1f/d11/c18 15 1 2024-03-24T10:45:53.045 INFO:tasks.workunit.client.0.smithi094.stdout:2/96: mknod d18/c1c 0 2024-03-24T10:45:53.046 INFO:tasks.workunit.client.0.smithi094.stdout:5/133: chown d7/dc/d12/l18 1493814 1 2024-03-24T10:45:53.046 INFO:tasks.workunit.client.0.smithi094.stdout:2/97: dread f14 [0,4194304] 0 2024-03-24T10:45:53.049 INFO:tasks.workunit.client.0.smithi094.stdout:9/177: rename c5 to df/d2d/d1f/d11/c35 0 2024-03-24T10:45:53.049 INFO:tasks.workunit.client.0.smithi094.stdout:9/178: chown df/d2d/d1f/d11/c18 1931 1 2024-03-24T10:45:53.049 INFO:tasks.workunit.client.0.smithi094.stdout:9/179: readlink df/d2d/d1f/l31 0 2024-03-24T10:45:53.049 INFO:tasks.workunit.client.0.smithi094.stdout:5/134: truncate d7/d8/fa 3026613 0 2024-03-24T10:45:53.050 INFO:tasks.workunit.client.0.smithi094.stdout:8/173: dwrite df/f1b [4194304,4194304] 0 2024-03-24T10:45:53.053 INFO:tasks.workunit.client.0.smithi094.stdout:9/180: dread df/f14 [0,4194304] 0 2024-03-24T10:45:53.053 INFO:tasks.workunit.client.0.smithi094.stdout:9/181: creat df/f36 x:0 0 0 2024-03-24T10:45:53.058 INFO:tasks.workunit.client.0.smithi094.stdout:8/174: mknod df/d12/c3b 0 2024-03-24T10:45:53.058 INFO:tasks.workunit.client.0.smithi094.stdout:8/175: write df/d12/f2f [52311,89620] 0 2024-03-24T10:45:53.058 INFO:tasks.workunit.client.0.smithi094.stdout:8/176: dread - fd zero size 2024-03-24T10:45:53.061 INFO:tasks.workunit.client.0.smithi094.stdout:9/182: symlink df/d2d/d1f/l37 0 2024-03-24T10:45:53.061 INFO:tasks.workunit.client.0.smithi094.stdout:9/183: fdatasync df/d2d/d34/f30 0 2024-03-24T10:45:53.062 INFO:tasks.workunit.client.0.smithi094.stdout:8/177: mkdir df/d25/d28/d31/d3c 0 2024-03-24T10:45:53.062 INFO:tasks.workunit.client.0.smithi094.stdout:8/178: chown df/d12/f39 47972 1 2024-03-24T10:45:53.063 INFO:tasks.workunit.client.0.smithi094.stdout:8/179: readlink df/l1c 0 2024-03-24T10:45:53.063 INFO:tasks.workunit.client.0.smithi094.stdout:9/184: symlink df/d2d/l38 0 2024-03-24T10:45:53.065 INFO:tasks.workunit.client.0.smithi094.stdout:9/185: creat df/d2d/d34/f39 x:0 0 0 2024-03-24T10:45:53.067 INFO:tasks.workunit.client.0.smithi094.stdout:9/186: dread df/d2d/f15 [0,4194304] 0 2024-03-24T10:45:53.067 INFO:tasks.workunit.client.0.smithi094.stdout:9/187: dread - df/f36 zero size 2024-03-24T10:45:53.067 INFO:tasks.workunit.client.0.smithi094.stdout:9/188: dread - df/f36 zero size 2024-03-24T10:45:53.067 INFO:tasks.workunit.client.0.smithi094.stdout:9/189: readlink df/d2d/l1d 0 2024-03-24T10:45:53.067 INFO:tasks.workunit.client.0.smithi094.stdout:6/134: sync 2024-03-24T10:45:53.068 INFO:tasks.workunit.client.0.smithi094.stdout:9/190: mkdir df/d2d/d34/d2c/d3a 0 2024-03-24T10:45:53.068 INFO:tasks.workunit.client.0.smithi094.stdout:5/135: dwrite d7/d8/f16 [0,4194304] 0 2024-03-24T10:45:53.069 INFO:tasks.workunit.client.0.smithi094.stdout:6/135: truncate d0/f2 1351051 0 2024-03-24T10:45:53.070 INFO:tasks.workunit.client.0.smithi094.stdout:5/136: fdatasync d7/d8/d14/f27 0 2024-03-24T10:45:53.070 INFO:tasks.workunit.client.0.smithi094.stdout:5/137: write d7/d8/d9/f29 [3143090,73029] 0 2024-03-24T10:45:53.070 INFO:tasks.workunit.client.0.smithi094.stdout:5/138: creat d7/d8/f2e x:0 0 0 2024-03-24T10:45:53.072 INFO:tasks.workunit.client.0.smithi094.stdout:8/180: write f6 [2198002,111145] 0 2024-03-24T10:45:53.075 INFO:tasks.workunit.client.0.smithi094.stdout:5/139: creat d7/d23/d2b/f2f x:0 0 0 2024-03-24T10:45:53.079 INFO:tasks.workunit.client.0.smithi094.stdout:6/136: rename d0/d1a/d1d to d0/d21 0 2024-03-24T10:45:53.080 INFO:tasks.workunit.client.0.smithi094.stdout:6/137: fdatasync d0/f1e 0 2024-03-24T10:45:53.081 INFO:tasks.workunit.client.0.smithi094.stdout:6/138: chown d0/d5 361 1 2024-03-24T10:45:53.084 INFO:tasks.workunit.client.0.smithi094.stdout:6/139: symlink d0/l22 0 2024-03-24T10:45:53.085 INFO:tasks.workunit.client.0.smithi094.stdout:6/140: mkdir d0/d5/da/d23 0 2024-03-24T10:45:53.085 INFO:tasks.workunit.client.0.smithi094.stdout:6/141: creat d0/d5/da/f24 x:0 0 0 2024-03-24T10:45:53.121 INFO:tasks.workunit.client.0.smithi094.stdout:6/142: dread d0/f2 [0,4194304] 0 2024-03-24T10:45:53.122 INFO:tasks.workunit.client.0.smithi094.stdout:5/140: write d7/d8/fa [569530,90846] 0 2024-03-24T10:45:53.123 INFO:tasks.workunit.client.0.smithi094.stdout:6/143: mknod d0/d21/c25 0 2024-03-24T10:45:53.123 INFO:tasks.workunit.client.0.smithi094.stdout:6/144: write d0/d5/da/f24 [718002,66580] 0 2024-03-24T10:45:53.124 INFO:tasks.workunit.client.0.smithi094.stdout:8/181: dwrite df/f2b [4194304,4194304] 0 2024-03-24T10:45:53.124 INFO:tasks.workunit.client.0.smithi094.stdout:9/191: dwrite df/d2d/f1e [4194304,4194304] 0 2024-03-24T10:45:53.124 INFO:tasks.workunit.client.0.smithi094.stdout:9/192: creat df/f3b x:0 0 0 2024-03-24T10:45:53.124 INFO:tasks.workunit.client.0.smithi094.stdout:9/193: dread - df/d2d/d34/f30 zero size 2024-03-24T10:45:53.124 INFO:tasks.workunit.client.0.smithi094.stdout:5/141: symlink d7/d23/d2b/l30 0 2024-03-24T10:45:53.128 INFO:tasks.workunit.client.0.smithi094.stdout:5/142: mknod d7/d23/d2b/c31 0 2024-03-24T10:45:53.128 INFO:tasks.workunit.client.0.smithi094.stdout:8/182: getdents df/d25/d28 0 2024-03-24T10:45:53.131 INFO:tasks.workunit.client.0.smithi094.stdout:9/194: getdents df 0 2024-03-24T10:45:53.137 INFO:tasks.workunit.client.0.smithi094.stdout:6/145: dwrite d0/f1e [0,4194304] 0 2024-03-24T10:45:53.140 INFO:tasks.workunit.client.0.smithi094.stdout:8/183: dread f8 [0,4194304] 0 2024-03-24T10:45:53.141 INFO:tasks.workunit.client.0.smithi094.stdout:6/146: rename d0/d5/da/f24 to d0/d5/da/d23/f26 0 2024-03-24T10:45:53.141 INFO:tasks.workunit.client.0.smithi094.stdout:6/147: fsync d0/d21/f13 0 2024-03-24T10:45:53.143 INFO:tasks.workunit.client.0.smithi094.stdout:6/148: mkdir d0/d5/da/d27 0 2024-03-24T10:45:53.143 INFO:tasks.workunit.client.0.smithi094.stdout:6/149: read d0/f8 [738005,88641] 0 2024-03-24T10:45:53.145 INFO:tasks.workunit.client.0.smithi094.stdout:6/150: link d0/f2 d0/d5/da/d27/f28 0 2024-03-24T10:45:53.145 INFO:tasks.workunit.client.0.smithi094.stdout:6/151: write d0/f18 [393924,106921] 0 2024-03-24T10:45:53.145 INFO:tasks.workunit.client.0.smithi094.stdout:6/152: creat d0/d5/f29 x:0 0 0 2024-03-24T10:45:53.145 INFO:tasks.workunit.client.0.smithi094.stdout:6/153: write d0/f8 [342919,74756] 0 2024-03-24T10:45:53.147 INFO:tasks.workunit.client.0.smithi094.stdout:8/184: dread df/f1b [0,4194304] 0 2024-03-24T10:45:53.147 INFO:tasks.workunit.client.0.smithi094.stdout:8/185: write df/d25/d28/d31/f32 [457675,72019] 0 2024-03-24T10:45:53.148 INFO:tasks.workunit.client.0.smithi094.stdout:8/186: creat df/d12/d18/d22/f3d x:0 0 0 2024-03-24T10:45:53.149 INFO:tasks.workunit.client.0.smithi094.stdout:8/187: creat df/d25/d28/d31/d3c/f3e x:0 0 0 2024-03-24T10:45:53.149 INFO:tasks.workunit.client.0.smithi094.stdout:8/188: dread - df/d25/d28/d31/d3c/f3e zero size 2024-03-24T10:45:53.150 INFO:tasks.workunit.client.0.smithi094.stdout:8/189: write df/d12/f2f [369206,25813] 0 2024-03-24T10:45:53.150 INFO:tasks.workunit.client.0.smithi094.stdout:8/190: write df/d12/d18/f20 [1343079,23387] 0 2024-03-24T10:45:53.150 INFO:tasks.workunit.client.0.smithi094.stdout:8/191: truncate fd 615896 0 2024-03-24T10:45:53.150 INFO:tasks.workunit.client.0.smithi094.stdout:8/192: mkdir df/d25/d36/d3f 0 2024-03-24T10:45:53.151 INFO:tasks.workunit.client.0.smithi094.stdout:8/193: write df/f1b [8961739,49401] 0 2024-03-24T10:45:53.151 INFO:tasks.workunit.client.0.smithi094.stdout:8/194: creat df/d25/d28/d31/f40 x:0 0 0 2024-03-24T10:45:53.151 INFO:tasks.workunit.client.0.smithi094.stdout:9/195: dwrite df/f36 [0,4194304] 0 2024-03-24T10:45:53.151 INFO:tasks.workunit.client.0.smithi094.stdout:6/154: dread d0/d21/f13 [0,4194304] 0 2024-03-24T10:45:53.152 INFO:tasks.workunit.client.0.smithi094.stdout:8/195: unlink ce 0 2024-03-24T10:45:53.157 INFO:tasks.workunit.client.0.smithi094.stdout:6/155: mkdir d0/d21/d2a 0 2024-03-24T10:45:53.157 INFO:tasks.workunit.client.0.smithi094.stdout:6/156: chown d0 38645 1 2024-03-24T10:45:53.158 INFO:tasks.workunit.client.0.smithi094.stdout:6/157: write d0/fc [2254533,24332] 0 2024-03-24T10:45:53.161 INFO:tasks.workunit.client.0.smithi094.stdout:6/158: unlink d0/f18 0 2024-03-24T10:45:53.163 INFO:tasks.workunit.client.0.smithi094.stdout:6/159: dread d0/d5/da/d23/f26 [0,4194304] 0 2024-03-24T10:45:53.163 INFO:tasks.workunit.client.0.smithi094.stdout:6/160: rmdir d0/d21 39 2024-03-24T10:45:53.165 INFO:tasks.workunit.client.0.smithi094.stdout:6/161: getdents d0/d5 0 2024-03-24T10:45:53.166 INFO:tasks.workunit.client.0.smithi094.stdout:6/162: creat d0/d21/f2b x:0 0 0 2024-03-24T10:45:53.167 INFO:tasks.workunit.client.0.smithi094.stdout:6/163: mknod d0/d5/da/c2c 0 2024-03-24T10:45:53.168 INFO:tasks.workunit.client.0.smithi094.stdout:6/164: chown d0/f8 1103 1 2024-03-24T10:45:53.168 INFO:tasks.workunit.client.0.smithi094.stdout:6/165: creat d0/d21/f2d x:0 0 0 2024-03-24T10:45:53.171 INFO:tasks.workunit.client.0.smithi094.stdout:8/196: truncate df/f2b 4359069 0 2024-03-24T10:45:53.188 INFO:tasks.workunit.client.0.smithi094.stdout:1/102: sync 2024-03-24T10:45:53.188 INFO:tasks.workunit.client.0.smithi094.stdout:1/103: chown d4/d9/fd 993 1 2024-03-24T10:45:53.189 INFO:tasks.workunit.client.0.smithi094.stdout:1/104: rename d4/d9/de/f1b to d4/d9/db/f21 0 2024-03-24T10:45:53.190 INFO:tasks.workunit.client.0.smithi094.stdout:8/197: dwrite df/d12/d18/d22/f3d [0,4194304] 0 2024-03-24T10:45:53.190 INFO:tasks.workunit.client.0.smithi094.stdout:8/198: creat df/d25/d28/d31/d3c/f41 x:0 0 0 2024-03-24T10:45:53.190 INFO:tasks.workunit.client.0.smithi094.stdout:8/199: read - df/d25/d28/d31/d3c/f41 zero size 2024-03-24T10:45:53.192 INFO:tasks.workunit.client.0.smithi094.stdout:8/200: creat df/d25/d28/f42 x:0 0 0 2024-03-24T10:45:53.193 INFO:tasks.workunit.client.0.smithi094.stdout:8/201: read - df/d12/d18/d22/f38 zero size 2024-03-24T10:45:53.194 INFO:tasks.workunit.client.0.smithi094.stdout:8/202: unlink df/c2a 0 2024-03-24T10:45:53.194 INFO:tasks.workunit.client.0.smithi094.stdout:8/203: write df/d25/d28/d31/f32 [1045014,70048] 0 2024-03-24T10:45:53.196 INFO:tasks.workunit.client.0.smithi094.stdout:1/105: write d4/d9/db/f10 [259520,24814] 0 2024-03-24T10:45:53.196 INFO:tasks.workunit.client.0.smithi094.stdout:1/106: write d4/f1e [119574,129656] 0 2024-03-24T10:45:53.209 INFO:tasks.workunit.client.0.smithi094.stdout:1/107: dwrite d4/d9/db/f21 [0,4194304] 0 2024-03-24T10:45:53.209 INFO:tasks.workunit.client.0.smithi094.stdout:1/108: creat d4/d9/db/f22 x:0 0 0 2024-03-24T10:45:53.224 INFO:tasks.workunit.client.0.smithi094.stdout:1/109: dwrite d4/f1e [0,4194304] 0 2024-03-24T10:45:53.224 INFO:tasks.workunit.client.0.smithi094.stdout:1/110: dread - d4/d9/db/f22 zero size 2024-03-24T10:45:53.224 INFO:tasks.workunit.client.0.smithi094.stdout:1/111: write d4/d9/db/f18 [1438680,43881] 0 2024-03-24T10:45:53.226 INFO:tasks.workunit.client.0.smithi094.stdout:1/112: symlink d4/d9/de/l23 0 2024-03-24T10:45:53.238 INFO:tasks.workunit.client.0.smithi094.stdout:1/113: dwrite d4/d9/db/f1f [4194304,4194304] 0 2024-03-24T10:45:53.263 INFO:tasks.workunit.client.0.smithi094.stdout:4/148: sync 2024-03-24T10:45:53.263 INFO:tasks.workunit.client.0.smithi094.stdout:0/140: sync 2024-03-24T10:45:53.263 INFO:tasks.workunit.client.0.smithi094.stdout:4/149: fsync d0/d5/d1b/f1e 0 2024-03-24T10:45:53.263 INFO:tasks.workunit.client.0.smithi094.stdout:0/141: dread - d14/d18/f1f zero size 2024-03-24T10:45:53.263 INFO:tasks.workunit.client.0.smithi094.stdout:0/142: chown d14/f1b 48 1 2024-03-24T10:45:53.263 INFO:tasks.workunit.client.0.smithi094.stdout:4/150: chown d0/l14 7076367 1 2024-03-24T10:45:53.263 INFO:tasks.workunit.client.0.smithi094.stdout:4/151: stat d0/d22/ff 0 2024-03-24T10:45:53.263 INFO:tasks.workunit.client.0.smithi094.stdout:4/152: truncate d0/d22/ff 1284425 0 2024-03-24T10:45:53.263 INFO:tasks.workunit.client.0.smithi094.stdout:4/153: read d0/d22/ff [60113,1896] 0 2024-03-24T10:45:53.264 INFO:tasks.workunit.client.0.smithi094.stdout:0/143: creat d14/d22/f35 x:0 0 0 2024-03-24T10:45:53.264 INFO:tasks.workunit.client.0.smithi094.stdout:0/144: truncate d14/d22/d28/f31 300847 0 2024-03-24T10:45:53.265 INFO:tasks.workunit.client.0.smithi094.stdout:0/145: link d14/c19 d14/d22/d28/c36 0 2024-03-24T10:45:53.267 INFO:tasks.workunit.client.0.smithi094.stdout:0/146: rename f5 to d14/d18/f37 0 2024-03-24T10:45:53.267 INFO:tasks.workunit.client.0.smithi094.stdout:0/147: write d14/d22/f2f [737431,49351] 0 2024-03-24T10:45:53.267 INFO:tasks.workunit.client.0.smithi094.stdout:0/148: write d14/f1e [818951,60812] 0 2024-03-24T10:45:53.268 INFO:tasks.workunit.client.0.smithi094.stdout:4/154: write d0/f7 [1434752,68670] 0 2024-03-24T10:45:53.268 INFO:tasks.workunit.client.0.smithi094.stdout:4/155: dread - d0/d5/d1b/f1e zero size 2024-03-24T10:45:53.269 INFO:tasks.workunit.client.0.smithi094.stdout:4/156: symlink d0/d5/l29 0 2024-03-24T10:45:53.270 INFO:tasks.workunit.client.0.smithi094.stdout:4/157: creat d0/d5/d1b/f2a x:0 0 0 2024-03-24T10:45:53.271 INFO:tasks.workunit.client.0.smithi094.stdout:4/158: mkdir d0/d22/de/d28/d2b 0 2024-03-24T10:45:53.271 INFO:tasks.workunit.client.0.smithi094.stdout:0/149: write f0 [2952297,124547] 0 2024-03-24T10:45:53.273 INFO:tasks.workunit.client.0.smithi094.stdout:4/159: symlink d0/l2c 0 2024-03-24T10:45:53.273 INFO:tasks.workunit.client.0.smithi094.stdout:4/160: fsync d0/d5/d1b/d24/f26 0 2024-03-24T10:45:53.273 INFO:tasks.workunit.client.0.smithi094.stdout:4/161: mknod d0/d1f/c2d 0 2024-03-24T10:45:53.273 INFO:tasks.workunit.client.0.smithi094.stdout:4/162: creat d0/d5/d1b/f2e x:0 0 0 2024-03-24T10:45:53.275 INFO:tasks.workunit.client.0.smithi094.stdout:4/163: link d0/d22/de/c21 d0/d5/d1b/c2f 0 2024-03-24T10:45:53.276 INFO:tasks.workunit.client.0.smithi094.stdout:4/164: creat d0/d22/de/f30 x:0 0 0 2024-03-24T10:45:53.277 INFO:tasks.workunit.client.0.smithi094.stdout:4/165: mkdir d0/d5/d1b/d24/d31 0 2024-03-24T10:45:53.277 INFO:tasks.workunit.client.0.smithi094.stdout:4/166: dread - d0/d5/d1b/d24/f26 zero size 2024-03-24T10:45:53.278 INFO:tasks.workunit.client.0.smithi094.stdout:4/167: link d0/c17 d0/d1f/c32 0 2024-03-24T10:45:53.297 INFO:tasks.workunit.client.0.smithi094.stdout:0/150: dwrite f9 [0,4194304] 0 2024-03-24T10:45:53.298 INFO:tasks.workunit.client.0.smithi094.stdout:4/168: dwrite d0/f6 [0,4194304] 0 2024-03-24T10:45:53.298 INFO:tasks.workunit.client.0.smithi094.stdout:4/169: readlink d0/d5/ld 0 2024-03-24T10:45:53.298 INFO:tasks.workunit.client.0.smithi094.stdout:4/170: write d0/d22/de/f30 [128067,86714] 0 2024-03-24T10:45:53.299 INFO:tasks.workunit.client.0.smithi094.stdout:0/151: unlink d14/d22/f29 0 2024-03-24T10:45:53.300 INFO:tasks.workunit.client.0.smithi094.stdout:4/171: mkdir d0/d22/de/d28/d2b/d33 0 2024-03-24T10:45:53.301 INFO:tasks.workunit.client.0.smithi094.stdout:0/152: dread fd [0,4194304] 0 2024-03-24T10:45:53.303 INFO:tasks.workunit.client.0.smithi094.stdout:4/172: rename d0/d5/d1b/d24/d31 to d0/d34 0 2024-03-24T10:45:53.305 INFO:tasks.workunit.client.0.smithi094.stdout:4/173: symlink d0/d22/de/d28/d2b/l35 0 2024-03-24T10:45:53.305 INFO:tasks.workunit.client.0.smithi094.stdout:4/174: fdatasync d0/f6 0 2024-03-24T10:45:53.306 INFO:tasks.workunit.client.0.smithi094.stdout:0/153: getdents d14 0 2024-03-24T10:45:53.306 INFO:tasks.workunit.client.0.smithi094.stdout:0/154: fsync d14/f27 0 2024-03-24T10:45:53.306 INFO:tasks.workunit.client.0.smithi094.stdout:0/155: chown d14/d22 104467832 1 2024-03-24T10:45:53.308 INFO:tasks.workunit.client.0.smithi094.stdout:0/156: creat d14/d22/d28/d2b/f38 x:0 0 0 2024-03-24T10:45:53.308 INFO:tasks.workunit.client.0.smithi094.stdout:4/175: dread d0/d22/de/f30 [0,4194304] 0 2024-03-24T10:45:53.311 INFO:tasks.workunit.client.0.smithi094.stdout:0/157: unlink d14/d22/l30 0 2024-03-24T10:45:53.326 INFO:tasks.workunit.client.0.smithi094.stdout:4/176: dwrite d0/d22/ff [0,4194304] 0 2024-03-24T10:45:53.328 INFO:tasks.workunit.client.0.smithi094.stdout:4/177: mknod d0/d22/c36 0 2024-03-24T10:45:53.329 INFO:tasks.workunit.client.0.smithi094.stdout:4/178: unlink d0/d5/d1b/d24/f26 0 2024-03-24T10:45:53.330 INFO:tasks.workunit.client.0.smithi094.stdout:4/179: creat d0/d22/de/d28/f37 x:0 0 0 2024-03-24T10:45:53.330 INFO:tasks.workunit.client.0.smithi094.stdout:4/180: stat d0/d22/f11 0 2024-03-24T10:45:53.341 INFO:tasks.workunit.client.0.smithi094.stdout:4/181: dread d0/d22/ff [0,4194304] 0 2024-03-24T10:45:53.341 INFO:tasks.workunit.client.0.smithi094.stdout:0/158: dwrite d14/d18/f37 [0,4194304] 0 2024-03-24T10:45:53.343 INFO:tasks.workunit.client.0.smithi094.stdout:0/159: symlink d14/l39 0 2024-03-24T10:45:53.344 INFO:tasks.workunit.client.0.smithi094.stdout:4/182: symlink d0/d22/de/d28/d2b/l38 0 2024-03-24T10:45:53.345 INFO:tasks.workunit.client.0.smithi094.stdout:0/160: symlink d14/d22/d32/l3a 0 2024-03-24T10:45:53.345 INFO:tasks.workunit.client.0.smithi094.stdout:0/161: creat d14/d22/d28/d2b/f3b x:0 0 0 2024-03-24T10:45:53.345 INFO:tasks.workunit.client.0.smithi094.stdout:0/162: rename d14/d22/d28 to d14/d22/d28/d2b/d3c 22 2024-03-24T10:45:53.345 INFO:tasks.workunit.client.0.smithi094.stdout:0/163: creat d14/d22/d28/d2b/f3d x:0 0 0 2024-03-24T10:45:53.345 INFO:tasks.workunit.client.0.smithi094.stdout:0/164: creat d14/d18/f3e x:0 0 0 2024-03-24T10:45:53.350 INFO:tasks.workunit.client.0.smithi094.stdout:0/165: unlink fc 0 2024-03-24T10:45:53.350 INFO:tasks.workunit.client.0.smithi094.stdout:0/166: dread - d14/d22/d28/d2b/f3d zero size 2024-03-24T10:45:53.351 INFO:tasks.workunit.client.0.smithi094.stdout:0/167: truncate d14/d22/d28/d2b/f3b 988884 0 2024-03-24T10:45:53.351 INFO:tasks.workunit.client.0.smithi094.stdout:0/168: write fd [134254,4576] 0 2024-03-24T10:45:53.351 INFO:tasks.workunit.client.0.smithi094.stdout:7/149: sync 2024-03-24T10:45:53.351 INFO:tasks.workunit.client.0.smithi094.stdout:3/114: sync 2024-03-24T10:45:53.351 INFO:tasks.workunit.client.0.smithi094.stdout:2/98: sync 2024-03-24T10:45:53.351 INFO:tasks.workunit.client.0.smithi094.stdout:3/115: chown d0/db 584 1 2024-03-24T10:45:53.352 INFO:tasks.workunit.client.0.smithi094.stdout:7/150: chown d1/d9/d35 22009875 1 2024-03-24T10:45:53.352 INFO:tasks.workunit.client.0.smithi094.stdout:0/169: mknod d14/d22/d32/c3f 0 2024-03-24T10:45:53.354 INFO:tasks.workunit.client.0.smithi094.stdout:2/99: rename f14 to d18/f1d 0 2024-03-24T10:45:53.355 INFO:tasks.workunit.client.0.smithi094.stdout:7/151: creat d1/d9/d14/f36 x:0 0 0 2024-03-24T10:45:53.356 INFO:tasks.workunit.client.0.smithi094.stdout:3/116: creat d0/db/d1c/d20/f24 x:0 0 0 2024-03-24T10:45:53.357 INFO:tasks.workunit.client.0.smithi094.stdout:2/100: mkdir d18/d1e 0 2024-03-24T10:45:53.359 INFO:tasks.workunit.client.0.smithi094.stdout:7/152: unlink d1/c1c 0 2024-03-24T10:45:53.359 INFO:tasks.workunit.client.0.smithi094.stdout:7/153: write d1/f2 [5143903,18074] 0 2024-03-24T10:45:53.360 INFO:tasks.workunit.client.0.smithi094.stdout:2/101: mkdir d18/d1f 0 2024-03-24T10:45:53.360 INFO:tasks.workunit.client.0.smithi094.stdout:2/102: write d18/f19 [899318,24576] 0 2024-03-24T10:45:53.360 INFO:tasks.workunit.client.0.smithi094.stdout:7/154: mknod d1/d9/d14/c37 0 2024-03-24T10:45:53.361 INFO:tasks.workunit.client.0.smithi094.stdout:4/183: dwrite d0/d22/f11 [0,4194304] 0 2024-03-24T10:45:53.361 INFO:tasks.workunit.client.0.smithi094.stdout:4/184: truncate d0/d5/d1b/f2e 4766 0 2024-03-24T10:45:53.362 INFO:tasks.workunit.client.0.smithi094.stdout:2/103: rmdir d18/d1f 0 2024-03-24T10:45:53.362 INFO:tasks.workunit.client.0.smithi094.stdout:3/117: write d0/f2 [1686247,34840] 0 2024-03-24T10:45:53.363 INFO:tasks.workunit.client.0.smithi094.stdout:4/185: unlink d0/d22/c10 0 2024-03-24T10:45:53.364 INFO:tasks.workunit.client.0.smithi094.stdout:3/118: mkdir d0/db/de/d25 0 2024-03-24T10:45:53.365 INFO:tasks.workunit.client.0.smithi094.stdout:3/119: mkdir d0/db/d1c/d20/d23/d26 0 2024-03-24T10:45:53.368 INFO:tasks.workunit.client.0.smithi094.stdout:0/170: dwrite d14/f27 [0,4194304] 0 2024-03-24T10:45:53.368 INFO:tasks.workunit.client.0.smithi094.stdout:0/171: truncate d14/f2a 2004902 0 2024-03-24T10:45:53.368 INFO:tasks.workunit.client.0.smithi094.stdout:0/172: truncate d14/d18/f25 5083213 0 2024-03-24T10:45:53.369 INFO:tasks.workunit.client.0.smithi094.stdout:3/120: rename d0/db/de/l15 to d0/db/l27 0 2024-03-24T10:45:53.374 INFO:tasks.workunit.client.0.smithi094.stdout:3/121: rmdir d0/db 39 2024-03-24T10:45:53.375 INFO:tasks.workunit.client.0.smithi094.stdout:0/173: dread d14/d18/f21 [0,4194304] 0 2024-03-24T10:45:53.377 INFO:tasks.workunit.client.0.smithi094.stdout:4/186: rmdir d0/d5/d1b/d24 0 2024-03-24T10:45:53.377 INFO:tasks.workunit.client.0.smithi094.stdout:4/187: creat d0/d5/d1b/f39 x:0 0 0 2024-03-24T10:45:53.379 INFO:tasks.workunit.client.0.smithi094.stdout:3/122: rmdir d0/db/d1c/d20/d23/d26 0 2024-03-24T10:45:53.382 INFO:tasks.workunit.client.0.smithi094.stdout:4/188: read d0/d22/ff [2888980,83949] 0 2024-03-24T10:45:53.382 INFO:tasks.workunit.client.0.smithi094.stdout:4/189: creat d0/d22/de/d28/f3a x:0 0 0 2024-03-24T10:45:53.382 INFO:tasks.workunit.client.0.smithi094.stdout:4/190: write d0/d22/ff [5182474,89179] 0 2024-03-24T10:45:53.382 INFO:tasks.workunit.client.0.smithi094.stdout:4/191: truncate d0/d5/d1b/f39 638109 0 2024-03-24T10:45:53.384 INFO:tasks.workunit.client.0.smithi094.stdout:4/192: mknod d0/d5/d1b/c3b 0 2024-03-24T10:45:53.386 INFO:tasks.workunit.client.0.smithi094.stdout:4/193: rename d0/l2c to d0/d34/l3c 0 2024-03-24T10:45:53.389 INFO:tasks.workunit.client.0.smithi094.stdout:4/194: mkdir d0/d22/de/d28/d3d 0 2024-03-24T10:45:53.389 INFO:tasks.workunit.client.0.smithi094.stdout:7/155: dwrite d1/f10 [0,4194304] 0 2024-03-24T10:45:53.389 INFO:tasks.workunit.client.0.smithi094.stdout:0/174: dread f9 [0,4194304] 0 2024-03-24T10:45:53.389 INFO:tasks.workunit.client.0.smithi094.stdout:7/156: truncate d1/d9/d16/d20/f26 1878131 0 2024-03-24T10:45:53.390 INFO:tasks.workunit.client.0.smithi094.stdout:2/104: dwrite d18/f1d [0,4194304] 0 2024-03-24T10:45:53.390 INFO:tasks.workunit.client.0.smithi094.stdout:2/105: fdatasync f15 0 2024-03-24T10:45:53.393 INFO:tasks.workunit.client.0.smithi094.stdout:4/195: rename d0/d22/de/d28/d2b/l35 to d0/d5/l3e 0 2024-03-24T10:45:53.394 INFO:tasks.workunit.client.0.smithi094.stdout:7/157: dread f0 [0,4194304] 0 2024-03-24T10:45:53.394 INFO:tasks.workunit.client.0.smithi094.stdout:7/158: truncate d1/d9/f12 4561123 0 2024-03-24T10:45:53.394 INFO:tasks.workunit.client.0.smithi094.stdout:7/159: fsync d1/d9/d16/d20/f26 0 2024-03-24T10:45:53.395 INFO:tasks.workunit.client.0.smithi094.stdout:2/106: symlink d18/d1e/l20 0 2024-03-24T10:45:53.396 INFO:tasks.workunit.client.0.smithi094.stdout:3/123: dwrite d0/f11 [0,4194304] 0 2024-03-24T10:45:53.397 INFO:tasks.workunit.client.0.smithi094.stdout:4/196: rename d0/d5/d1b/f1e to d0/d22/de/d28/f3f 0 2024-03-24T10:45:53.400 INFO:tasks.workunit.client.0.smithi094.stdout:0/175: write d14/d18/f37 [1950140,103516] 0 2024-03-24T10:45:53.400 INFO:tasks.workunit.client.0.smithi094.stdout:0/176: stat d14/d22/d32/c3f 0 2024-03-24T10:45:53.400 INFO:tasks.workunit.client.0.smithi094.stdout:0/177: readlink d14/d22/l24 0 2024-03-24T10:45:53.401 INFO:tasks.workunit.client.0.smithi094.stdout:7/160: mknod d1/d9/d31/c38 0 2024-03-24T10:45:53.401 INFO:tasks.workunit.client.0.smithi094.stdout:7/161: readlink d1/d9/d16/l1a 0 2024-03-24T10:45:53.401 INFO:tasks.workunit.client.0.smithi094.stdout:2/107: rmdir d18/d1e 39 2024-03-24T10:45:53.403 INFO:tasks.workunit.client.0.smithi094.stdout:3/124: dread d0/f2 [0,4194304] 0 2024-03-24T10:45:53.412 INFO:tasks.workunit.client.0.smithi094.stdout:0/178: rename f13 to d14/d18/f40 0 2024-03-24T10:45:53.412 INFO:tasks.workunit.client.0.smithi094.stdout:7/162: rmdir d1/d9/d31 39 2024-03-24T10:45:53.413 INFO:tasks.workunit.client.0.smithi094.stdout:2/108: symlink d18/d1e/l21 0 2024-03-24T10:45:53.413 INFO:tasks.workunit.client.0.smithi094.stdout:2/109: dread - d18/f1b zero size 2024-03-24T10:45:53.414 INFO:tasks.workunit.client.0.smithi094.stdout:3/125: mkdir d0/db/dc/d28 0 2024-03-24T10:45:53.414 INFO:tasks.workunit.client.0.smithi094.stdout:4/197: truncate d0/d22/ff 1356789 0 2024-03-24T10:45:53.415 INFO:tasks.workunit.client.0.smithi094.stdout:4/198: dread - d0/d5/d1b/f2a zero size 2024-03-24T10:45:53.416 INFO:tasks.workunit.client.0.smithi094.stdout:0/179: mkdir d14/d22/d28/d2b/d41 0 2024-03-24T10:45:53.417 INFO:tasks.workunit.client.0.smithi094.stdout:7/163: mkdir d1/d9/d31/d39 0 2024-03-24T10:45:53.417 INFO:tasks.workunit.client.0.smithi094.stdout:7/164: getdents d1/d9/d31/d39 0 2024-03-24T10:45:53.417 INFO:tasks.workunit.client.0.smithi094.stdout:7/165: fdatasync d1/d9/f22 0 2024-03-24T10:45:53.418 INFO:tasks.workunit.client.0.smithi094.stdout:2/110: mkdir d18/d1e/d22 0 2024-03-24T10:45:53.418 INFO:tasks.workunit.client.0.smithi094.stdout:2/111: write d18/f19 [599374,29323] 0 2024-03-24T10:45:53.420 INFO:tasks.workunit.client.0.smithi094.stdout:4/199: link d0/f6 d0/d22/de/d28/d2b/d33/f40 0 2024-03-24T10:45:53.422 INFO:tasks.workunit.client.0.smithi094.stdout:7/166: rename d1/l5 to d1/d9/d16/d21/l3a 0 2024-03-24T10:45:53.424 INFO:tasks.workunit.client.0.smithi094.stdout:4/200: creat d0/d5/f41 x:0 0 0 2024-03-24T10:45:53.428 INFO:tasks.workunit.client.0.smithi094.stdout:2/112: dread d18/f19 [0,4194304] 0 2024-03-24T10:45:53.428 INFO:tasks.workunit.client.0.smithi094.stdout:6/166: sync 2024-03-24T10:45:53.428 INFO:tasks.workunit.client.0.smithi094.stdout:9/196: sync 2024-03-24T10:45:53.429 INFO:tasks.workunit.client.0.smithi094.stdout:6/167: dread - d0/d21/f2d zero size 2024-03-24T10:45:53.429 INFO:tasks.workunit.client.0.smithi094.stdout:1/114: sync 2024-03-24T10:45:53.429 INFO:tasks.workunit.client.0.smithi094.stdout:8/204: sync 2024-03-24T10:45:53.429 INFO:tasks.workunit.client.0.smithi094.stdout:5/143: sync 2024-03-24T10:45:53.429 INFO:tasks.workunit.client.0.smithi094.stdout:5/144: creat d7/d8/f32 x:0 0 0 2024-03-24T10:45:53.430 INFO:tasks.workunit.client.0.smithi094.stdout:5/145: truncate d7/dc/f19 183392 0 2024-03-24T10:45:53.430 INFO:tasks.workunit.client.0.smithi094.stdout:2/113: stat lf 0 2024-03-24T10:45:53.431 INFO:tasks.workunit.client.0.smithi094.stdout:8/205: unlink df/f16 0 2024-03-24T10:45:53.431 INFO:tasks.workunit.client.0.smithi094.stdout:8/206: fdatasync df/d25/d28/d31/f32 0 2024-03-24T10:45:53.432 INFO:tasks.workunit.client.0.smithi094.stdout:5/146: symlink d7/l33 0 2024-03-24T10:45:53.432 INFO:tasks.workunit.client.0.smithi094.stdout:5/147: creat d7/f34 x:0 0 0 2024-03-24T10:45:53.432 INFO:tasks.workunit.client.0.smithi094.stdout:7/167: dread f0 [0,4194304] 0 2024-03-24T10:45:53.433 INFO:tasks.workunit.client.0.smithi094.stdout:5/148: creat d7/d8/d14/f35 x:0 0 0 2024-03-24T10:45:53.436 INFO:tasks.workunit.client.0.smithi094.stdout:5/149: mkdir d7/d8/d14/d36 0 2024-03-24T10:45:53.439 INFO:tasks.workunit.client.0.smithi094.stdout:1/115: write d4/d9/db/f21 [1393069,74229] 0 2024-03-24T10:45:53.440 INFO:tasks.workunit.client.0.smithi094.stdout:6/168: dread d0/f1e [0,4194304] 0 2024-03-24T10:45:53.440 INFO:tasks.workunit.client.0.smithi094.stdout:1/116: mknod d4/d9/db/c24 0 2024-03-24T10:45:53.441 INFO:tasks.workunit.client.0.smithi094.stdout:8/207: read df/f1b [3511535,114982] 0 2024-03-24T10:45:53.449 INFO:tasks.workunit.client.0.smithi094.stdout:0/180: dwrite fa [4194304,4194304] 0 2024-03-24T10:45:53.449 INFO:tasks.workunit.client.0.smithi094.stdout:3/126: dwrite d0/f3 [0,4194304] 0 2024-03-24T10:45:53.449 INFO:tasks.workunit.client.0.smithi094.stdout:0/181: readlink d14/l20 0 2024-03-24T10:45:53.449 INFO:tasks.workunit.client.0.smithi094.stdout:2/114: dread f7 [4194304,4194304] 0 2024-03-24T10:45:53.449 INFO:tasks.workunit.client.0.smithi094.stdout:2/115: fdatasync f15 0 2024-03-24T10:45:53.449 INFO:tasks.workunit.client.0.smithi094.stdout:2/116: truncate f12 447892 0 2024-03-24T10:45:53.451 INFO:tasks.workunit.client.0.smithi094.stdout:3/127: creat d0/db/d1c/d20/d23/f29 x:0 0 0 2024-03-24T10:45:53.451 INFO:tasks.workunit.client.0.smithi094.stdout:5/150: write d7/d8/f16 [261722,51470] 0 2024-03-24T10:45:53.452 INFO:tasks.workunit.client.0.smithi094.stdout:5/151: creat d7/d8/d1b/f37 x:0 0 0 2024-03-24T10:45:53.452 INFO:tasks.workunit.client.0.smithi094.stdout:5/152: write d7/dc/d12/f1f [1392730,100809] 0 2024-03-24T10:45:53.452 INFO:tasks.workunit.client.0.smithi094.stdout:0/182: symlink d14/d22/d28/d2b/d41/l42 0 2024-03-24T10:45:53.453 INFO:tasks.workunit.client.0.smithi094.stdout:2/117: creat d18/f23 x:0 0 0 2024-03-24T10:45:53.454 INFO:tasks.workunit.client.0.smithi094.stdout:3/128: mkdir d0/db/d1c/d20/d23/d2a 0 2024-03-24T10:45:53.454 INFO:tasks.workunit.client.0.smithi094.stdout:3/129: chown d0/db/dc 24 1 2024-03-24T10:45:53.456 INFO:tasks.workunit.client.0.smithi094.stdout:5/153: symlink d7/d23/d2b/l38 0 2024-03-24T10:45:53.456 INFO:tasks.workunit.client.0.smithi094.stdout:0/183: dread d14/d18/f25 [0,4194304] 0 2024-03-24T10:45:53.457 INFO:tasks.workunit.client.0.smithi094.stdout:5/154: creat d7/dc/d12/f39 x:0 0 0 2024-03-24T10:45:53.457 INFO:tasks.workunit.client.0.smithi094.stdout:0/184: symlink d14/d18/l43 0 2024-03-24T10:45:53.458 INFO:tasks.workunit.client.0.smithi094.stdout:0/185: creat d14/d22/d28/d2b/f44 x:0 0 0 2024-03-24T10:45:53.460 INFO:tasks.workunit.client.0.smithi094.stdout:7/168: dread d1/f2 [0,4194304] 0 2024-03-24T10:45:53.460 INFO:tasks.workunit.client.0.smithi094.stdout:0/186: link d14/d18/f21 d14/d22/d28/d2b/f45 0 2024-03-24T10:45:53.461 INFO:tasks.workunit.client.0.smithi094.stdout:7/169: rename d1/d9/d25/c2b to d1/d9/c3b 0 2024-03-24T10:45:53.461 INFO:tasks.workunit.client.0.smithi094.stdout:7/170: creat d1/d9/f3c x:0 0 0 2024-03-24T10:45:53.472 INFO:tasks.workunit.client.0.smithi094.stdout:3/130: write d0/f2 [2200816,57201] 0 2024-03-24T10:45:53.473 INFO:tasks.workunit.client.0.smithi094.stdout:4/201: dwrite d0/d22/de/d28/d2b/d33/f40 [0,4194304] 0 2024-03-24T10:45:53.474 INFO:tasks.workunit.client.0.smithi094.stdout:4/202: write d0/d22/de/d28/f37 [376742,49211] 0 2024-03-24T10:45:53.474 INFO:tasks.workunit.client.0.smithi094.stdout:9/197: dwrite df/f17 [0,4194304] 0 2024-03-24T10:45:53.475 INFO:tasks.workunit.client.0.smithi094.stdout:9/198: write df/f14 [2165555,103435] 0 2024-03-24T10:45:53.475 INFO:tasks.workunit.client.0.smithi094.stdout:4/203: mkdir d0/d5/d42 0 2024-03-24T10:45:53.478 INFO:tasks.workunit.client.0.smithi094.stdout:9/199: symlink df/d2d/d34/l3c 0 2024-03-24T10:45:53.478 INFO:tasks.workunit.client.0.smithi094.stdout:8/208: write f4 [3030687,66009] 0 2024-03-24T10:45:53.478 INFO:tasks.workunit.client.0.smithi094.stdout:8/209: rename df/d25 to df/d25/d43 22 2024-03-24T10:45:53.478 INFO:tasks.workunit.client.0.smithi094.stdout:8/210: readlink df/d25/d28/l3a 0 2024-03-24T10:45:53.478 INFO:tasks.workunit.client.0.smithi094.stdout:8/211: chown df/d25/d28/d31/f40 502 1 2024-03-24T10:45:53.479 INFO:tasks.workunit.client.0.smithi094.stdout:8/212: creat df/d25/f44 x:0 0 0 2024-03-24T10:45:53.479 INFO:tasks.workunit.client.0.smithi094.stdout:9/200: creat df/d2d/d1f/f3d x:0 0 0 2024-03-24T10:45:53.480 INFO:tasks.workunit.client.0.smithi094.stdout:8/213: symlink df/d25/d36/d3f/l45 0 2024-03-24T10:45:53.481 INFO:tasks.workunit.client.0.smithi094.stdout:9/201: rename c9 to df/d2d/d34/d2c/c3e 0 2024-03-24T10:45:53.482 INFO:tasks.workunit.client.0.smithi094.stdout:2/118: read f15 [6686333,21119] 0 2024-03-24T10:45:53.482 INFO:tasks.workunit.client.0.smithi094.stdout:8/214: symlink df/d25/d28/l46 0 2024-03-24T10:45:53.482 INFO:tasks.workunit.client.0.smithi094.stdout:6/169: dwrite d0/d5/da/f20 [0,4194304] 0 2024-03-24T10:45:53.484 INFO:tasks.workunit.client.0.smithi094.stdout:2/119: creat d18/d1e/f24 x:0 0 0 2024-03-24T10:45:53.484 INFO:tasks.workunit.client.0.smithi094.stdout:2/120: write d18/f23 [764562,39732] 0 2024-03-24T10:45:53.484 INFO:tasks.workunit.client.0.smithi094.stdout:2/121: creat d18/d1e/f25 x:0 0 0 2024-03-24T10:45:53.488 INFO:tasks.workunit.client.0.smithi094.stdout:1/117: dwrite d4/f1e [0,4194304] 0 2024-03-24T10:45:53.489 INFO:tasks.workunit.client.0.smithi094.stdout:6/170: rmdir d0/d5/da/d27 39 2024-03-24T10:45:53.489 INFO:tasks.workunit.client.0.smithi094.stdout:6/171: write d0/d21/f13 [9146126,43665] 0 2024-03-24T10:45:53.489 INFO:tasks.workunit.client.0.smithi094.stdout:0/187: dwrite d14/d18/f25 [4194304,4194304] 0 2024-03-24T10:45:53.489 INFO:tasks.workunit.client.0.smithi094.stdout:0/188: read d14/d18/f37 [4009615,75766] 0 2024-03-24T10:45:53.489 INFO:tasks.workunit.client.0.smithi094.stdout:0/189: creat d14/d22/d28/f46 x:0 0 0 2024-03-24T10:45:53.493 INFO:tasks.workunit.client.0.smithi094.stdout:8/215: rename df/d12/d18 to df/d25/d28/d31/d47 0 2024-03-24T10:45:53.494 INFO:tasks.workunit.client.0.smithi094.stdout:8/216: creat df/d25/f48 x:0 0 0 2024-03-24T10:45:53.496 INFO:tasks.workunit.client.0.smithi094.stdout:7/171: dwrite d1/d9/f3c [0,4194304] 0 2024-03-24T10:45:53.498 INFO:tasks.workunit.client.0.smithi094.stdout:3/131: dwrite d0/db/d1c/d20/f24 [0,4194304] 0 2024-03-24T10:45:53.511 INFO:tasks.workunit.client.0.smithi094.stdout:5/155: fsync d7/d8/d1b/f37 0 2024-03-24T10:45:53.511 INFO:tasks.workunit.client.0.smithi094.stdout:5/156: fsync d7/d8/fa 0 2024-03-24T10:45:53.512 INFO:tasks.workunit.client.0.smithi094.stdout:5/157: fsync d7/d8/d14/f2a 0 2024-03-24T10:45:53.512 INFO:tasks.workunit.client.0.smithi094.stdout:1/118: dwrite d4/d9/db/f1d [0,4194304] 0 2024-03-24T10:45:53.512 INFO:tasks.workunit.client.0.smithi094.stdout:1/119: read d4/f7 [6542267,42999] 0 2024-03-24T10:45:53.514 INFO:tasks.workunit.client.0.smithi094.stdout:3/132: creat d0/db/f2b x:0 0 0 2024-03-24T10:45:53.514 INFO:tasks.workunit.client.0.smithi094.stdout:3/133: stat d0/db 0 2024-03-24T10:45:53.514 INFO:tasks.workunit.client.0.smithi094.stdout:3/134: fdatasync d0/db/dc/f12 0 2024-03-24T10:45:53.520 INFO:tasks.workunit.client.0.smithi094.stdout:0/190: dwrite d14/d22/f2f [0,4194304] 0 2024-03-24T10:45:53.521 INFO:tasks.workunit.client.0.smithi094.stdout:7/172: dwrite d1/d9/d16/d21/f33 [0,4194304] 0 2024-03-24T10:45:53.521 INFO:tasks.workunit.client.0.smithi094.stdout:7/173: fdatasync d1/f10 0 2024-03-24T10:45:53.521 INFO:tasks.workunit.client.0.smithi094.stdout:8/217: dwrite df/d12/f39 [0,4194304] 0 2024-03-24T10:45:53.521 INFO:tasks.workunit.client.0.smithi094.stdout:8/218: write df/d25/d28/d31/d3c/f3e [520188,118635] 0 2024-03-24T10:45:53.526 INFO:tasks.workunit.client.0.smithi094.stdout:3/135: mknod d0/db/dc/c2c 0 2024-03-24T10:45:53.527 INFO:tasks.workunit.client.0.smithi094.stdout:1/120: dwrite d4/d9/db/f1f [4194304,4194304] 0 2024-03-24T10:45:53.527 INFO:tasks.workunit.client.0.smithi094.stdout:1/121: fdatasync d4/d9/fa 0 2024-03-24T10:45:53.530 INFO:tasks.workunit.client.0.smithi094.stdout:7/174: unlink d1/d9/d14/l27 0 2024-03-24T10:45:53.534 INFO:tasks.workunit.client.0.smithi094.stdout:8/219: link df/d25/d36/d3f/l45 df/d25/d36/d3f/l49 0 2024-03-24T10:45:53.535 INFO:tasks.workunit.client.0.smithi094.stdout:5/158: rmdir d7/d8/d1b 39 2024-03-24T10:45:53.535 INFO:tasks.workunit.client.0.smithi094.stdout:5/159: chown d7/d23/d2b/l30 16 1 2024-03-24T10:45:53.537 INFO:tasks.workunit.client.0.smithi094.stdout:3/136: creat d0/db/dc/f2d x:0 0 0 2024-03-24T10:45:53.537 INFO:tasks.workunit.client.0.smithi094.stdout:3/137: truncate d0/db/f2b 155751 0 2024-03-24T10:45:53.537 INFO:tasks.workunit.client.0.smithi094.stdout:3/138: rename d0/db/d1c/d20/d23 to d0/db/d1c/d20/d23/d2a/d2e 22 2024-03-24T10:45:53.537 INFO:tasks.workunit.client.0.smithi094.stdout:2/122: truncate d18/f19 122363 0 2024-03-24T10:45:53.537 INFO:tasks.workunit.client.0.smithi094.stdout:2/123: dread - d18/f1b zero size 2024-03-24T10:45:53.538 INFO:tasks.workunit.client.0.smithi094.stdout:0/191: rmdir d14/d18 39 2024-03-24T10:45:53.538 INFO:tasks.workunit.client.0.smithi094.stdout:0/192: creat d14/d22/d28/f47 x:0 0 0 2024-03-24T10:45:53.540 INFO:tasks.workunit.client.0.smithi094.stdout:1/122: rmdir d4/d9 39 2024-03-24T10:45:53.542 INFO:tasks.workunit.client.0.smithi094.stdout:5/160: mkdir d7/d8/d14/d36/d3a 0 2024-03-24T10:45:53.545 INFO:tasks.workunit.client.0.smithi094.stdout:2/124: symlink d18/l26 0 2024-03-24T10:45:53.545 INFO:tasks.workunit.client.0.smithi094.stdout:2/125: creat d18/f27 x:0 0 0 2024-03-24T10:45:53.545 INFO:tasks.workunit.client.0.smithi094.stdout:2/126: chown d18/d1e/f25 5 1 2024-03-24T10:45:53.545 INFO:tasks.workunit.client.0.smithi094.stdout:2/127: write d18/d1e/f24 [206880,35848] 0 2024-03-24T10:45:53.545 INFO:tasks.workunit.client.0.smithi094.stdout:2/128: creat d18/d1e/f28 x:0 0 0 2024-03-24T10:45:53.545 INFO:tasks.workunit.client.0.smithi094.stdout:2/129: fdatasync d18/f27 0 2024-03-24T10:45:53.546 INFO:tasks.workunit.client.0.smithi094.stdout:7/175: dwrite d1/d9/fd [0,4194304] 0 2024-03-24T10:45:53.546 INFO:tasks.workunit.client.0.smithi094.stdout:7/176: chown d1/d9/d16/l19 109245810 1 2024-03-24T10:45:53.546 INFO:tasks.workunit.client.0.smithi094.stdout:7/177: truncate d1/d9/f30 685818 0 2024-03-24T10:45:53.549 INFO:tasks.workunit.client.0.smithi094.stdout:1/123: rename d4/d9/db/f22 to d4/d9/f25 0 2024-03-24T10:45:53.549 INFO:tasks.workunit.client.0.smithi094.stdout:5/161: rename d7/d8/d1b to d7/d8/d1b/d3b 22 2024-03-24T10:45:53.550 INFO:tasks.workunit.client.0.smithi094.stdout:5/162: dread - d7/dc/d12/f39 zero size 2024-03-24T10:45:53.550 INFO:tasks.workunit.client.0.smithi094.stdout:0/193: dread f3 [0,4194304] 0 2024-03-24T10:45:53.550 INFO:tasks.workunit.client.0.smithi094.stdout:0/194: write d14/f1b [617976,87067] 0 2024-03-24T10:45:53.552 INFO:tasks.workunit.client.0.smithi094.stdout:2/130: rmdir d18/d1e/d22 0 2024-03-24T10:45:53.558 INFO:tasks.workunit.client.0.smithi094.stdout:7/178: mkdir d1/d9/d25/d3d 0 2024-03-24T10:45:53.560 INFO:tasks.workunit.client.0.smithi094.stdout:0/195: write d14/d22/f2f [1681744,47856] 0 2024-03-24T10:45:53.560 INFO:tasks.workunit.client.0.smithi094.stdout:0/196: read - d14/d18/f3e zero size 2024-03-24T10:45:53.561 INFO:tasks.workunit.client.0.smithi094.stdout:2/131: chown l3 38178 1 2024-03-24T10:45:53.561 INFO:tasks.workunit.client.0.smithi094.stdout:2/132: creat d18/f29 x:0 0 0 2024-03-24T10:45:53.563 INFO:tasks.workunit.client.0.smithi094.stdout:7/179: rename d1/d9/d14/c37 to d1/d9/d16/d20/c3e 0 2024-03-24T10:45:53.563 INFO:tasks.workunit.client.0.smithi094.stdout:7/180: creat d1/d9/d16/d21/f3f x:0 0 0 2024-03-24T10:45:53.566 INFO:tasks.workunit.client.0.smithi094.stdout:0/197: unlink lf 0 2024-03-24T10:45:53.568 INFO:tasks.workunit.client.0.smithi094.stdout:7/181: creat d1/d9/d25/d3d/f40 x:0 0 0 2024-03-24T10:45:53.568 INFO:tasks.workunit.client.0.smithi094.stdout:7/182: dread - d1/d9/d16/d20/f28 zero size 2024-03-24T10:45:53.569 INFO:tasks.workunit.client.0.smithi094.stdout:0/198: mkdir d14/d22/d28/d48 0 2024-03-24T10:45:53.570 INFO:tasks.workunit.client.0.smithi094.stdout:7/183: mknod d1/d9/d31/d39/c41 0 2024-03-24T10:45:53.570 INFO:tasks.workunit.client.0.smithi094.stdout:7/184: write d1/d9/f23 [1714872,30636] 0 2024-03-24T10:45:53.570 INFO:tasks.workunit.client.0.smithi094.stdout:7/185: fsync d1/d9/f2c 0 2024-03-24T10:45:53.570 INFO:tasks.workunit.client.0.smithi094.stdout:7/186: creat d1/d9/d14/f42 x:0 0 0 2024-03-24T10:45:53.571 INFO:tasks.workunit.client.0.smithi094.stdout:0/199: creat d14/f49 x:0 0 0 2024-03-24T10:45:53.571 INFO:tasks.workunit.client.0.smithi094.stdout:0/200: dread - d14/d18/f3e zero size 2024-03-24T10:45:53.572 INFO:tasks.workunit.client.0.smithi094.stdout:3/139: getdents d0/db/dc 0 2024-03-24T10:45:53.572 INFO:tasks.workunit.client.0.smithi094.stdout:3/140: chown d0/db/d1c/l1d 181 1 2024-03-24T10:45:53.572 INFO:tasks.workunit.client.0.smithi094.stdout:3/141: chown d0/db/de/c1f 120 1 2024-03-24T10:45:53.572 INFO:tasks.workunit.client.0.smithi094.stdout:3/142: fsync d0/db/f2b 0 2024-03-24T10:45:53.574 INFO:tasks.workunit.client.0.smithi094.stdout:7/187: unlink d1/d9/lf 0 2024-03-24T10:45:53.574 INFO:tasks.workunit.client.0.smithi094.stdout:5/163: dwrite d7/d8/d14/f35 [0,4194304] 0 2024-03-24T10:45:53.575 INFO:tasks.workunit.client.0.smithi094.stdout:0/201: stat d14/d22/d28/c36 0 2024-03-24T10:45:53.575 INFO:tasks.workunit.client.0.smithi094.stdout:2/133: dread d18/f19 [0,4194304] 0 2024-03-24T10:45:53.575 INFO:tasks.workunit.client.0.smithi094.stdout:1/124: dwrite d4/f7 [4194304,4194304] 0 2024-03-24T10:45:53.575 INFO:tasks.workunit.client.0.smithi094.stdout:7/188: write d1/d9/d16/d20/f26 [237620,74086] 0 2024-03-24T10:45:53.575 INFO:tasks.workunit.client.0.smithi094.stdout:7/189: truncate d1/d9/f13 629785 0 2024-03-24T10:45:53.576 INFO:tasks.workunit.client.0.smithi094.stdout:3/143: symlink d0/db/dc/l2f 0 2024-03-24T10:45:53.580 INFO:tasks.workunit.client.0.smithi094.stdout:0/202: creat d14/d22/d28/d48/f4a x:0 0 0 2024-03-24T10:45:53.582 INFO:tasks.workunit.client.0.smithi094.stdout:2/134: chown lb 811191 1 2024-03-24T10:45:53.582 INFO:tasks.workunit.client.0.smithi094.stdout:0/203: dread fd [0,4194304] 0 2024-03-24T10:45:53.584 INFO:tasks.workunit.client.0.smithi094.stdout:7/190: rename d1/d9/d14/l1e to d1/d9/d25/d3d/l43 0 2024-03-24T10:45:53.586 INFO:tasks.workunit.client.0.smithi094.stdout:3/144: link d0/db/ld d0/l30 0 2024-03-24T10:45:53.588 INFO:tasks.workunit.client.0.smithi094.stdout:5/164: getdents d7/d23/d2b 0 2024-03-24T10:45:53.588 INFO:tasks.workunit.client.0.smithi094.stdout:5/165: stat d7/d8/d14/f27 0 2024-03-24T10:45:53.588 INFO:tasks.workunit.client.0.smithi094.stdout:4/204: sync 2024-03-24T10:45:53.588 INFO:tasks.workunit.client.0.smithi094.stdout:6/172: sync 2024-03-24T10:45:53.588 INFO:tasks.workunit.client.0.smithi094.stdout:9/202: sync 2024-03-24T10:45:53.588 INFO:tasks.workunit.client.0.smithi094.stdout:9/203: dread - df/d2d/d34/f39 zero size 2024-03-24T10:45:53.589 INFO:tasks.workunit.client.0.smithi094.stdout:2/135: mknod d18/d1e/c2a 0 2024-03-24T10:45:53.589 INFO:tasks.workunit.client.0.smithi094.stdout:2/136: chown l13 1579691 1 2024-03-24T10:45:53.589 INFO:tasks.workunit.client.0.smithi094.stdout:2/137: write d18/f19 [924148,36846] 0 2024-03-24T10:45:53.590 INFO:tasks.workunit.client.0.smithi094.stdout:1/125: dwrite d4/fc [0,4194304] 0 2024-03-24T10:45:53.590 INFO:tasks.workunit.client.0.smithi094.stdout:1/126: fsync d4/fc 0 2024-03-24T10:45:53.591 INFO:tasks.workunit.client.0.smithi094.stdout:7/191: mkdir d1/d9/d31/d39/d44 0 2024-03-24T10:45:53.593 INFO:tasks.workunit.client.0.smithi094.stdout:3/145: mknod d0/db/de/d25/c31 0 2024-03-24T10:45:53.596 INFO:tasks.workunit.client.0.smithi094.stdout:5/166: symlink d7/d23/l3c 0 2024-03-24T10:45:53.596 INFO:tasks.workunit.client.0.smithi094.stdout:5/167: write d7/d8/fa [3812620,15181] 0 2024-03-24T10:45:53.596 INFO:tasks.workunit.client.0.smithi094.stdout:5/168: write d7/f34 [98582,2939] 0 2024-03-24T10:45:53.597 INFO:tasks.workunit.client.0.smithi094.stdout:7/192: dread d1/d9/f23 [0,4194304] 0 2024-03-24T10:45:53.597 INFO:tasks.workunit.client.0.smithi094.stdout:4/205: creat d0/d22/de/d28/d2b/f43 x:0 0 0 2024-03-24T10:45:53.597 INFO:tasks.workunit.client.0.smithi094.stdout:4/206: readlink d0/d22/de/d28/d2b/l38 0 2024-03-24T10:45:53.598 INFO:tasks.workunit.client.0.smithi094.stdout:4/207: fsync d0/d5/f41 0 2024-03-24T10:45:53.598 INFO:tasks.workunit.client.0.smithi094.stdout:4/208: chown d0/c18 504340 1 2024-03-24T10:45:53.598 INFO:tasks.workunit.client.0.smithi094.stdout:4/209: rename d0/d22 to d0/d22/de/d28/d2b/d33/d44 22 2024-03-24T10:45:53.599 INFO:tasks.workunit.client.0.smithi094.stdout:2/138: link d18/f1d d18/d1e/f2b 0 2024-03-24T10:45:53.602 INFO:tasks.workunit.client.0.smithi094.stdout:1/127: link d4/d9/db/l20 d4/d9/l26 0 2024-03-24T10:45:53.602 INFO:tasks.workunit.client.0.smithi094.stdout:1/128: chown d4/d9/db/f1f 0 1 2024-03-24T10:45:53.602 INFO:tasks.workunit.client.0.smithi094.stdout:1/129: truncate d4/d9/db/f1d 4633132 0 2024-03-24T10:45:53.605 INFO:tasks.workunit.client.0.smithi094.stdout:3/146: mknod d0/db/c32 0 2024-03-24T10:45:53.606 INFO:tasks.workunit.client.0.smithi094.stdout:0/204: dwrite d14/f1b [0,4194304] 0 2024-03-24T10:45:53.606 INFO:tasks.workunit.client.0.smithi094.stdout:0/205: chown d14/d18 3 1 2024-03-24T10:45:53.607 INFO:tasks.workunit.client.0.smithi094.stdout:0/206: chown d14/d18/f40 127665009 1 2024-03-24T10:45:53.607 INFO:tasks.workunit.client.0.smithi094.stdout:0/207: write d14/d18/f1f [550757,61685] 0 2024-03-24T10:45:53.607 INFO:tasks.workunit.client.0.smithi094.stdout:9/204: dwrite df/d2d/d1f/d11/f13 [4194304,4194304] 0 2024-03-24T10:45:53.607 INFO:tasks.workunit.client.0.smithi094.stdout:5/169: mkdir d7/d26/d3d 0 2024-03-24T10:45:53.608 INFO:tasks.workunit.client.0.smithi094.stdout:4/210: stat d0/c18 0 2024-03-24T10:45:53.608 INFO:tasks.workunit.client.0.smithi094.stdout:7/193: read d1/f7 [1669299,26340] 0 2024-03-24T10:45:53.609 INFO:tasks.workunit.client.0.smithi094.stdout:2/139: unlink d18/d1e/f28 0 2024-03-24T10:45:53.610 INFO:tasks.workunit.client.0.smithi094.stdout:1/130: creat d4/d9/f27 x:0 0 0 2024-03-24T10:45:53.610 INFO:tasks.workunit.client.0.smithi094.stdout:1/131: creat d4/d9/f28 x:0 0 0 2024-03-24T10:45:53.611 INFO:tasks.workunit.client.0.smithi094.stdout:3/147: creat d0/db/d1c/d20/d23/d2a/f33 x:0 0 0 2024-03-24T10:45:53.613 INFO:tasks.workunit.client.0.smithi094.stdout:0/208: rmdir d14/d22/d32 39 2024-03-24T10:45:53.614 INFO:tasks.workunit.client.0.smithi094.stdout:9/205: creat df/f3f x:0 0 0 2024-03-24T10:45:53.615 INFO:tasks.workunit.client.0.smithi094.stdout:5/170: creat d7/dc/d12/f3e x:0 0 0 2024-03-24T10:45:53.615 INFO:tasks.workunit.client.0.smithi094.stdout:5/171: chown d7/d8/d14 1 1 2024-03-24T10:45:53.616 INFO:tasks.workunit.client.0.smithi094.stdout:4/211: mknod d0/d22/de/d28/d2b/d33/c45 0 2024-03-24T10:45:53.617 INFO:tasks.workunit.client.0.smithi094.stdout:7/194: symlink d1/d9/d31/l45 0 2024-03-24T10:45:53.617 INFO:tasks.workunit.client.0.smithi094.stdout:7/195: write d1/d9/f22 [949474,106516] 0 2024-03-24T10:45:53.617 INFO:tasks.workunit.client.0.smithi094.stdout:7/196: fsync d1/d9/d25/d3d/f40 0 2024-03-24T10:45:53.617 INFO:tasks.workunit.client.0.smithi094.stdout:7/197: fdatasync d1/d9/fe 0 2024-03-24T10:45:53.617 INFO:tasks.workunit.client.0.smithi094.stdout:7/198: fdatasync d1/d9/d16/d20/f26 0 2024-03-24T10:45:53.617 INFO:tasks.workunit.client.0.smithi094.stdout:7/199: fdatasync d1/d9/f30 0 2024-03-24T10:45:53.618 INFO:tasks.workunit.client.0.smithi094.stdout:2/140: readlink l5 0 2024-03-24T10:45:53.620 INFO:tasks.workunit.client.0.smithi094.stdout:1/132: symlink d4/d9/de/l29 0 2024-03-24T10:45:53.620 INFO:tasks.workunit.client.0.smithi094.stdout:1/133: creat d4/f2a x:0 0 0 2024-03-24T10:45:53.621 INFO:tasks.workunit.client.0.smithi094.stdout:9/206: dread df/f36 [0,4194304] 0 2024-03-24T10:45:53.621 INFO:tasks.workunit.client.0.smithi094.stdout:3/148: mkdir d0/db/de/d25/d34 0 2024-03-24T10:45:53.633 INFO:tasks.workunit.client.0.smithi094.stdout:4/212: mkdir d0/d22/de/d28/d2b/d46 0 2024-03-24T10:45:53.634 INFO:tasks.workunit.client.0.smithi094.stdout:4/213: truncate d0/d5/f41 901361 0 2024-03-24T10:45:53.634 INFO:tasks.workunit.client.0.smithi094.stdout:4/214: dread d0/d5/d1b/f39 [0,4194304] 0 2024-03-24T10:45:53.636 INFO:tasks.workunit.client.0.smithi094.stdout:1/134: dwrite d4/d9/db/f1d [0,4194304] 0 2024-03-24T10:45:53.637 INFO:tasks.workunit.client.0.smithi094.stdout:9/207: rename df/f17 to df/d2d/d34/d2c/f40 0 2024-03-24T10:45:53.637 INFO:tasks.workunit.client.0.smithi094.stdout:9/208: creat df/f41 x:0 0 0 2024-03-24T10:45:53.637 INFO:tasks.workunit.client.0.smithi094.stdout:9/209: read - df/d2d/d34/f39 zero size 2024-03-24T10:45:53.637 INFO:tasks.workunit.client.0.smithi094.stdout:3/149: symlink d0/db/l35 0 2024-03-24T10:45:53.637 INFO:tasks.workunit.client.0.smithi094.stdout:0/209: getdents d14/d22/d32 0 2024-03-24T10:45:53.639 INFO:tasks.workunit.client.0.smithi094.stdout:5/172: rmdir d7/dc 39 2024-03-24T10:45:53.639 INFO:tasks.workunit.client.0.smithi094.stdout:5/173: chown d7/d8/d14/d36/d3a 2428982 1 2024-03-24T10:45:53.641 INFO:tasks.workunit.client.0.smithi094.stdout:4/215: rmdir d0/d22/de/d28/d2b/d46 0 2024-03-24T10:45:53.643 INFO:tasks.workunit.client.0.smithi094.stdout:1/135: rename d4/fc to d4/d9/de/f2b 0 2024-03-24T10:45:53.643 INFO:tasks.workunit.client.0.smithi094.stdout:1/136: chown d4/l19 158 1 2024-03-24T10:45:53.643 INFO:tasks.workunit.client.0.smithi094.stdout:1/137: write d4/d9/fa [3570262,112581] 0 2024-03-24T10:45:53.643 INFO:tasks.workunit.client.0.smithi094.stdout:1/138: creat d4/d9/f2c x:0 0 0 2024-03-24T10:45:53.645 INFO:tasks.workunit.client.0.smithi094.stdout:9/210: truncate df/f2a 382783 0 2024-03-24T10:45:53.645 INFO:tasks.workunit.client.0.smithi094.stdout:2/141: truncate d18/f19 444617 0 2024-03-24T10:45:53.647 INFO:tasks.workunit.client.0.smithi094.stdout:3/150: mknod d0/db/c36 0 2024-03-24T10:45:53.647 INFO:tasks.workunit.client.0.smithi094.stdout:3/151: chown d0/db/f2b 18962 1 2024-03-24T10:45:53.648 INFO:tasks.workunit.client.0.smithi094.stdout:0/210: symlink d14/d22/d28/d48/l4b 0 2024-03-24T10:45:53.649 INFO:tasks.workunit.client.0.smithi094.stdout:0/211: chown d14/d18/f1d 15 1 2024-03-24T10:45:53.649 INFO:tasks.workunit.client.0.smithi094.stdout:0/212: write f0 [12235,5376] 0 2024-03-24T10:45:53.649 INFO:tasks.workunit.client.0.smithi094.stdout:0/213: chown fd 55 1 2024-03-24T10:45:53.649 INFO:tasks.workunit.client.0.smithi094.stdout:0/214: chown f3 197 1 2024-03-24T10:45:53.649 INFO:tasks.workunit.client.0.smithi094.stdout:5/174: mknod d7/d8/d14/d36/d3a/c3f 0 2024-03-24T10:45:53.654 INFO:tasks.workunit.client.0.smithi094.stdout:1/139: rename d4/d9/db/f1a to d4/d9/f2d 0 2024-03-24T10:45:53.655 INFO:tasks.workunit.client.0.smithi094.stdout:2/142: chown c9 255299918 1 2024-03-24T10:45:53.656 INFO:tasks.workunit.client.0.smithi094.stdout:0/215: write f3 [2794706,44618] 0 2024-03-24T10:45:53.658 INFO:tasks.workunit.client.0.smithi094.stdout:3/152: symlink d0/db/d1c/d20/d23/l37 0 2024-03-24T10:45:53.660 INFO:tasks.workunit.client.0.smithi094.stdout:4/216: dwrite d0/f6 [4194304,4194304] 0 2024-03-24T10:45:53.661 INFO:tasks.workunit.client.0.smithi094.stdout:1/140: mkdir d4/d9/de/d2e 0 2024-03-24T10:45:53.664 INFO:tasks.workunit.client.0.smithi094.stdout:9/211: rmdir df/d2d/d1f 39 2024-03-24T10:45:53.664 INFO:tasks.workunit.client.0.smithi094.stdout:9/212: chown df 0 1 2024-03-24T10:45:53.667 INFO:tasks.workunit.client.0.smithi094.stdout:9/213: read df/f22 [202817,71728] 0 2024-03-24T10:45:53.669 INFO:tasks.workunit.client.0.smithi094.stdout:2/143: truncate d18/f19 1026992 0 2024-03-24T10:45:53.669 INFO:tasks.workunit.client.0.smithi094.stdout:2/144: chown lf 98 1 2024-03-24T10:45:53.669 INFO:tasks.workunit.client.0.smithi094.stdout:2/145: readlink d18/l26 0 2024-03-24T10:45:53.669 INFO:tasks.workunit.client.0.smithi094.stdout:2/146: chown l5 63416091 1 2024-03-24T10:45:53.669 INFO:tasks.workunit.client.0.smithi094.stdout:9/214: dread df/d2d/f15 [0,4194304] 0 2024-03-24T10:45:53.670 INFO:tasks.workunit.client.0.smithi094.stdout:3/153: truncate d0/db/dc/f2d 937289 0 2024-03-24T10:45:53.673 INFO:tasks.workunit.client.0.smithi094.stdout:5/175: link d7/dc/d12/l1d d7/d8/d9/l40 0 2024-03-24T10:45:53.673 INFO:tasks.workunit.client.0.smithi094.stdout:5/176: chown d7/dc/f19 202 1 2024-03-24T10:45:53.673 INFO:tasks.workunit.client.0.smithi094.stdout:5/177: write d7/d8/f28 [485979,89459] 0 2024-03-24T10:45:53.673 INFO:tasks.workunit.client.0.smithi094.stdout:5/178: dread - d7/d8/f32 zero size 2024-03-24T10:45:53.673 INFO:tasks.workunit.client.0.smithi094.stdout:5/179: chown d7/d8/d9/f29 110 1 2024-03-24T10:45:53.678 INFO:tasks.workunit.client.0.smithi094.stdout:9/215: mknod df/c42 0 2024-03-24T10:45:53.678 INFO:tasks.workunit.client.0.smithi094.stdout:9/216: fsync df/d2d/d34/f30 0 2024-03-24T10:45:53.679 INFO:tasks.workunit.client.0.smithi094.stdout:9/217: write df/d2d/f15 [2160549,45313] 0 2024-03-24T10:45:53.681 INFO:tasks.workunit.client.0.smithi094.stdout:9/218: unlink df/c27 0 2024-03-24T10:45:53.683 INFO:tasks.workunit.client.0.smithi094.stdout:1/141: getdents d4/d9/de 0 2024-03-24T10:45:53.684 INFO:tasks.workunit.client.0.smithi094.stdout:1/142: mkdir d4/d2f 0 2024-03-24T10:45:53.685 INFO:tasks.workunit.client.0.smithi094.stdout:1/143: creat d4/d9/db/f30 x:0 0 0 2024-03-24T10:45:53.685 INFO:tasks.workunit.client.0.smithi094.stdout:1/144: stat d4/l6 0 2024-03-24T10:45:53.686 INFO:tasks.workunit.client.0.smithi094.stdout:1/145: mkdir d4/d9/db/d31 0 2024-03-24T10:45:53.687 INFO:tasks.workunit.client.0.smithi094.stdout:1/146: fsync d4/f1e 0 2024-03-24T10:45:53.688 INFO:tasks.workunit.client.0.smithi094.stdout:1/147: rmdir d4/d2f 0 2024-03-24T10:45:53.689 INFO:tasks.workunit.client.0.smithi094.stdout:9/219: dread df/d2d/d34/f2e [0,4194304] 0 2024-03-24T10:45:53.691 INFO:tasks.workunit.client.0.smithi094.stdout:9/220: dread df/f2b [0,4194304] 0 2024-03-24T10:45:53.691 INFO:tasks.workunit.client.0.smithi094.stdout:9/221: chown df/d2d/f15 34788 1 2024-03-24T10:45:53.692 INFO:tasks.workunit.client.0.smithi094.stdout:9/222: mkdir df/d2d/d1f/d11/d43 0 2024-03-24T10:45:53.692 INFO:tasks.workunit.client.0.smithi094.stdout:3/154: dread d0/db/de/ff [0,4194304] 0 2024-03-24T10:45:53.693 INFO:tasks.workunit.client.0.smithi094.stdout:4/217: dwrite d0/f7 [8388608,4194304] 0 2024-03-24T10:45:53.694 INFO:tasks.workunit.client.0.smithi094.stdout:3/155: mkdir d0/db/dc/d28/d38 0 2024-03-24T10:45:53.694 INFO:tasks.workunit.client.0.smithi094.stdout:3/156: write d0/db/d1c/d20/d23/d2a/f33 [53575,68883] 0 2024-03-24T10:45:53.695 INFO:tasks.workunit.client.0.smithi094.stdout:2/147: dwrite f7 [4194304,4194304] 0 2024-03-24T10:45:53.696 INFO:tasks.workunit.client.0.smithi094.stdout:4/218: creat d0/f47 x:0 0 0 2024-03-24T10:45:53.696 INFO:tasks.workunit.client.0.smithi094.stdout:1/148: dread d4/f1e [0,4194304] 0 2024-03-24T10:45:53.696 INFO:tasks.workunit.client.0.smithi094.stdout:1/149: creat d4/d9/f32 x:0 0 0 2024-03-24T10:45:53.696 INFO:tasks.workunit.client.0.smithi094.stdout:1/150: dread - d4/d9/f32 zero size 2024-03-24T10:45:53.696 INFO:tasks.workunit.client.0.smithi094.stdout:1/151: write d4/d9/f25 [969937,49968] 0 2024-03-24T10:45:53.696 INFO:tasks.workunit.client.0.smithi094.stdout:9/223: dread df/f36 [0,4194304] 0 2024-03-24T10:45:53.697 INFO:tasks.workunit.client.0.smithi094.stdout:0/216: dwrite d14/d22/d28/d2b/f3d [0,4194304] 0 2024-03-24T10:45:53.697 INFO:tasks.workunit.client.0.smithi094.stdout:5/180: dwrite d7/dc/d12/f3e [0,4194304] 0 2024-03-24T10:45:53.697 INFO:tasks.workunit.client.0.smithi094.stdout:5/181: fsync d7/d8/f32 0 2024-03-24T10:45:53.697 INFO:tasks.workunit.client.0.smithi094.stdout:5/182: chown d7/d23/d2b 0 1 2024-03-24T10:45:53.698 INFO:tasks.workunit.client.0.smithi094.stdout:2/148: symlink d18/l2c 0 2024-03-24T10:45:53.698 INFO:tasks.workunit.client.0.smithi094.stdout:2/149: write d18/f19 [1870965,62803] 0 2024-03-24T10:45:53.699 INFO:tasks.workunit.client.0.smithi094.stdout:4/219: rename d0/d22/de/d28/f37 to d0/d34/f48 0 2024-03-24T10:45:53.699 INFO:tasks.workunit.client.0.smithi094.stdout:3/157: dread d0/db/d1c/d20/d23/d2a/f33 [0,4194304] 0 2024-03-24T10:45:53.701 INFO:tasks.workunit.client.0.smithi094.stdout:8/220: sync 2024-03-24T10:45:53.703 INFO:tasks.workunit.client.0.smithi094.stdout:1/152: write d4/d9/db/f1f [7820113,70819] 0 2024-03-24T10:45:53.703 INFO:tasks.workunit.client.0.smithi094.stdout:9/224: creat df/d2d/f44 x:0 0 0 2024-03-24T10:45:53.703 INFO:tasks.workunit.client.0.smithi094.stdout:9/225: chown df/f41 771263 1 2024-03-24T10:45:53.706 INFO:tasks.workunit.client.0.smithi094.stdout:8/221: write f8 [169302,47752] 0 2024-03-24T10:45:53.714 INFO:tasks.workunit.client.0.smithi094.stdout:2/150: symlink d18/d1e/l2d 0 2024-03-24T10:45:53.715 INFO:tasks.workunit.client.0.smithi094.stdout:4/220: mknod d0/d5/d42/c49 0 2024-03-24T10:45:53.718 INFO:tasks.workunit.client.0.smithi094.stdout:1/153: mkdir d4/d9/de/d2e/d33 0 2024-03-24T10:45:53.719 INFO:tasks.workunit.client.0.smithi094.stdout:9/226: symlink df/d2d/d1f/d11/d43/l45 0 2024-03-24T10:45:53.719 INFO:tasks.workunit.client.0.smithi094.stdout:9/227: chown df/d2d/l38 3993537 1 2024-03-24T10:45:53.719 INFO:tasks.workunit.client.0.smithi094.stdout:9/228: chown df/d2d/d34/f2e 846008 1 2024-03-24T10:45:53.720 INFO:tasks.workunit.client.0.smithi094.stdout:0/217: rmdir d14/d22/d28/d2b 39 2024-03-24T10:45:53.722 INFO:tasks.workunit.client.0.smithi094.stdout:8/222: creat df/d25/d28/d31/d47/f4a x:0 0 0 2024-03-24T10:45:53.723 INFO:tasks.workunit.client.0.smithi094.stdout:6/173: sync 2024-03-24T10:45:53.724 INFO:tasks.workunit.client.0.smithi094.stdout:4/221: dread d0/d22/f11 [0,4194304] 0 2024-03-24T10:45:53.724 INFO:tasks.workunit.client.0.smithi094.stdout:4/222: truncate d0/d22/de/f30 1223123 0 2024-03-24T10:45:53.724 INFO:tasks.workunit.client.0.smithi094.stdout:4/223: dread - d0/d22/de/d28/f3a zero size 2024-03-24T10:45:53.724 INFO:tasks.workunit.client.0.smithi094.stdout:3/158: rename d0/db/dc/c19 to d0/db/de/d25/c39 0 2024-03-24T10:45:53.725 INFO:tasks.workunit.client.0.smithi094.stdout:5/183: truncate d7/dc/d12/f3e 4119123 0 2024-03-24T10:45:53.725 INFO:tasks.workunit.client.0.smithi094.stdout:9/229: rmdir df/d2d/d1f/d11 39 2024-03-24T10:45:53.725 INFO:tasks.workunit.client.0.smithi094.stdout:9/230: write df/f41 [77507,39943] 0 2024-03-24T10:45:53.728 INFO:tasks.workunit.client.0.smithi094.stdout:8/223: rename df/d25/d28/l3a to df/d25/d28/l4b 0 2024-03-24T10:45:53.728 INFO:tasks.workunit.client.0.smithi094.stdout:8/224: readlink df/d25/d36/d3f/l45 0 2024-03-24T10:45:53.728 INFO:tasks.workunit.client.0.smithi094.stdout:8/225: truncate df/d25/d28/f42 210953 0 2024-03-24T10:45:53.728 INFO:tasks.workunit.client.0.smithi094.stdout:8/226: chown f3 106121 1 2024-03-24T10:45:53.728 INFO:tasks.workunit.client.0.smithi094.stdout:8/227: chown df/d25/d28/f2c 18928 1 2024-03-24T10:45:53.728 INFO:tasks.workunit.client.0.smithi094.stdout:6/174: rmdir d0/d5/da/d23 39 2024-03-24T10:45:53.730 INFO:tasks.workunit.client.0.smithi094.stdout:2/151: rmdir d18 39 2024-03-24T10:45:53.730 INFO:tasks.workunit.client.0.smithi094.stdout:4/224: truncate d0/d22/ff 302153 0 2024-03-24T10:45:53.732 INFO:tasks.workunit.client.0.smithi094.stdout:1/154: rename d4/d9/de/l29 to d4/d9/db/l34 0 2024-03-24T10:45:53.733 INFO:tasks.workunit.client.0.smithi094.stdout:7/200: sync 2024-03-24T10:45:53.733 INFO:tasks.workunit.client.0.smithi094.stdout:7/201: creat d1/d9/d25/d3d/f46 x:0 0 0 2024-03-24T10:45:53.733 INFO:tasks.workunit.client.0.smithi094.stdout:7/202: fsync d1/d9/d16/d20/f26 0 2024-03-24T10:45:53.734 INFO:tasks.workunit.client.0.smithi094.stdout:5/184: unlink d7/d8/f2e 0 2024-03-24T10:45:53.736 INFO:tasks.workunit.client.0.smithi094.stdout:9/231: truncate df/d2d/d1f/d11/f13 3660426 0 2024-03-24T10:45:53.738 INFO:tasks.workunit.client.0.smithi094.stdout:0/218: getdents d14/d22/d28 0 2024-03-24T10:45:53.738 INFO:tasks.workunit.client.0.smithi094.stdout:0/219: truncate d14/f49 603420 0 2024-03-24T10:45:53.739 INFO:tasks.workunit.client.0.smithi094.stdout:6/175: creat d0/d5/da/f2e x:0 0 0 2024-03-24T10:45:53.740 INFO:tasks.workunit.client.0.smithi094.stdout:2/152: creat d18/f2e x:0 0 0 2024-03-24T10:45:53.743 INFO:tasks.workunit.client.0.smithi094.stdout:4/225: mkdir d0/d5/d42/d4a 0 2024-03-24T10:45:53.743 INFO:tasks.workunit.client.0.smithi094.stdout:4/226: readlink d0/d22/l1d 0 2024-03-24T10:45:53.743 INFO:tasks.workunit.client.0.smithi094.stdout:9/232: write df/d2d/d34/d2c/f40 [3854850,92463] 0 2024-03-24T10:45:53.747 INFO:tasks.workunit.client.0.smithi094.stdout:3/159: rename d0/db/de to d0/db/d1c/d20/d23/d2a/d3a 0 2024-03-24T10:45:53.748 INFO:tasks.workunit.client.0.smithi094.stdout:1/155: symlink d4/d9/l35 0 2024-03-24T10:45:53.748 INFO:tasks.workunit.client.0.smithi094.stdout:1/156: dread - d4/d9/f2c zero size 2024-03-24T10:45:53.750 INFO:tasks.workunit.client.0.smithi094.stdout:7/203: stat d1/d9/d16/d20/c3e 0 2024-03-24T10:45:53.750 INFO:tasks.workunit.client.0.smithi094.stdout:7/204: chown d1/d9/d25/d3d/l43 16 1 2024-03-24T10:45:53.752 INFO:tasks.workunit.client.0.smithi094.stdout:0/220: symlink d14/d22/d28/l4c 0 2024-03-24T10:45:53.752 INFO:tasks.workunit.client.0.smithi094.stdout:3/160: dread d0/db/d1c/d20/d23/d2a/d3a/ff [0,4194304] 0 2024-03-24T10:45:53.752 INFO:tasks.workunit.client.0.smithi094.stdout:3/161: write d0/f14 [4937270,121086] 0 2024-03-24T10:45:53.753 INFO:tasks.workunit.client.0.smithi094.stdout:6/176: mkdir d0/d21/d2a/d2f 0 2024-03-24T10:45:53.755 INFO:tasks.workunit.client.0.smithi094.stdout:4/227: unlink d0/d5/f41 0 2024-03-24T10:45:53.755 INFO:tasks.workunit.client.0.smithi094.stdout:4/228: write d0/d22/ff [228828,26040] 0 2024-03-24T10:45:53.755 INFO:tasks.workunit.client.0.smithi094.stdout:0/221: dread d14/d18/f21 [0,4194304] 0 2024-03-24T10:45:53.756 INFO:tasks.workunit.client.0.smithi094.stdout:1/157: dread d4/d9/fd [0,4194304] 0 2024-03-24T10:45:53.758 INFO:tasks.workunit.client.0.smithi094.stdout:9/233: dwrite df/f3f [0,4194304] 0 2024-03-24T10:45:53.761 INFO:tasks.workunit.client.0.smithi094.stdout:8/228: rename df/d25/d28/d31/d47/c35 to df/d25/d28/d31/c4c 0 2024-03-24T10:45:53.761 INFO:tasks.workunit.client.0.smithi094.stdout:5/185: rename d7/d8/d14/d36/d3a to d7/d8/d14/d36/d3a/d41 22 2024-03-24T10:45:53.762 INFO:tasks.workunit.client.0.smithi094.stdout:7/205: symlink d1/d9/l47 0 2024-03-24T10:45:53.764 INFO:tasks.workunit.client.0.smithi094.stdout:3/162: creat d0/db/dc/f3b x:0 0 0 2024-03-24T10:45:53.764 INFO:tasks.workunit.client.0.smithi094.stdout:3/163: dread - d0/db/dc/f3b zero size 2024-03-24T10:45:53.766 INFO:tasks.workunit.client.0.smithi094.stdout:0/222: write d14/d22/d28/d2b/f3b [1854599,56611] 0 2024-03-24T10:45:53.766 INFO:tasks.workunit.client.0.smithi094.stdout:7/206: dread d1/d9/f30 [0,4194304] 0 2024-03-24T10:45:53.770 INFO:tasks.workunit.client.0.smithi094.stdout:1/158: dread d4/d9/db/f1d [0,4194304] 0 2024-03-24T10:45:53.770 INFO:tasks.workunit.client.0.smithi094.stdout:1/159: fsync d4/d9/f28 0 2024-03-24T10:45:53.772 INFO:tasks.workunit.client.0.smithi094.stdout:0/223: unlink d14/f16 0 2024-03-24T10:45:53.776 INFO:tasks.workunit.client.0.smithi094.stdout:6/177: rename d0/d5/da/d27 to d0/d30 0 2024-03-24T10:45:53.776 INFO:tasks.workunit.client.0.smithi094.stdout:9/234: rename df/d2d/d34 to df/d2d/d34/d2c/d46 22 2024-03-24T10:45:53.776 INFO:tasks.workunit.client.0.smithi094.stdout:9/235: truncate df/d2d/d34/f30 61009 0 2024-03-24T10:45:53.776 INFO:tasks.workunit.client.0.smithi094.stdout:9/236: truncate df/d2d/f44 276932 0 2024-03-24T10:45:53.776 INFO:tasks.workunit.client.0.smithi094.stdout:9/237: creat df/d2d/d34/f47 x:0 0 0 2024-03-24T10:45:53.780 INFO:tasks.workunit.client.0.smithi094.stdout:7/207: rmdir d1/d9/d14 39 2024-03-24T10:45:53.780 INFO:tasks.workunit.client.0.smithi094.stdout:7/208: fdatasync d1/d9/f23 0 2024-03-24T10:45:53.780 INFO:tasks.workunit.client.0.smithi094.stdout:7/209: read d1/d9/fe [2480815,128298] 0 2024-03-24T10:45:53.781 INFO:tasks.workunit.client.0.smithi094.stdout:3/164: write d0/f1e [2302952,37621] 0 2024-03-24T10:45:53.781 INFO:tasks.workunit.client.0.smithi094.stdout:4/229: dwrite d0/d34/f48 [0,4194304] 0 2024-03-24T10:45:53.781 INFO:tasks.workunit.client.0.smithi094.stdout:4/230: chown d0/d5/d1b 233485276 1 2024-03-24T10:45:53.783 INFO:tasks.workunit.client.0.smithi094.stdout:9/238: rename df/d2d/d34/l3c to df/d2d/d34/d2c/l48 0 2024-03-24T10:45:53.783 INFO:tasks.workunit.client.0.smithi094.stdout:1/160: dread d4/f7 [4194304,4194304] 0 2024-03-24T10:45:53.788 INFO:tasks.workunit.client.0.smithi094.stdout:8/229: dwrite df/d25/d28/d31/d47/f23 [0,4194304] 0 2024-03-24T10:45:53.789 INFO:tasks.workunit.client.0.smithi094.stdout:5/186: dwrite d7/d8/fa [0,4194304] 0 2024-03-24T10:45:53.789 INFO:tasks.workunit.client.0.smithi094.stdout:5/187: stat d7/d26/l22 0 2024-03-24T10:45:53.790 INFO:tasks.workunit.client.0.smithi094.stdout:7/210: unlink d1/f1f 0 2024-03-24T10:45:53.794 INFO:tasks.workunit.client.0.smithi094.stdout:9/239: creat df/d2d/d1f/f49 x:0 0 0 2024-03-24T10:45:53.794 INFO:tasks.workunit.client.0.smithi094.stdout:9/240: creat df/d2d/d1f/f4a x:0 0 0 2024-03-24T10:45:53.794 INFO:tasks.workunit.client.0.smithi094.stdout:9/241: chown df/d2d/d1f 1517837 1 2024-03-24T10:45:53.795 INFO:tasks.workunit.client.0.smithi094.stdout:1/161: mkdir d4/d36 0 2024-03-24T10:45:53.795 INFO:tasks.workunit.client.0.smithi094.stdout:1/162: truncate d4/d9/f32 127767 0 2024-03-24T10:45:53.795 INFO:tasks.workunit.client.0.smithi094.stdout:1/163: chown d4/d9/l35 6 1 2024-03-24T10:45:53.795 INFO:tasks.workunit.client.0.smithi094.stdout:9/242: read df/d2d/d34/f2e [1317051,114730] 0 2024-03-24T10:45:53.795 INFO:tasks.workunit.client.0.smithi094.stdout:9/243: write df/f2b [1641909,39741] 0 2024-03-24T10:45:53.795 INFO:tasks.workunit.client.0.smithi094.stdout:0/224: getdents d14/d22/d28/d48 0 2024-03-24T10:45:53.799 INFO:tasks.workunit.client.0.smithi094.stdout:5/188: creat d7/d8/d1b/f42 x:0 0 0 2024-03-24T10:45:53.802 INFO:tasks.workunit.client.0.smithi094.stdout:6/178: dwrite d0/f2 [0,4194304] 0 2024-03-24T10:45:53.802 INFO:tasks.workunit.client.0.smithi094.stdout:6/179: chown d0/d21/f2b 103745367 1 2024-03-24T10:45:53.802 INFO:tasks.workunit.client.0.smithi094.stdout:4/231: dwrite d0/d5/f16 [0,4194304] 0 2024-03-24T10:45:53.803 INFO:tasks.workunit.client.0.smithi094.stdout:1/164: rename d4/d9/de/d2e to d4/d9/d37 0 2024-03-24T10:45:53.803 INFO:tasks.workunit.client.0.smithi094.stdout:1/165: dread - d4/f2a zero size 2024-03-24T10:45:53.804 INFO:tasks.workunit.client.0.smithi094.stdout:0/225: dread - d14/d22/d28/f46 zero size 2024-03-24T10:45:53.805 INFO:tasks.workunit.client.0.smithi094.stdout:8/230: rmdir df/d25/d28/d31/d47 39 2024-03-24T10:45:53.807 INFO:tasks.workunit.client.0.smithi094.stdout:5/189: creat d7/d23/d2b/f43 x:0 0 0 2024-03-24T10:45:53.809 INFO:tasks.workunit.client.0.smithi094.stdout:7/211: getdents d1/d9 0 2024-03-24T10:45:53.811 INFO:tasks.workunit.client.0.smithi094.stdout:6/180: link d0/d1a/l1c d0/d1a/l31 0 2024-03-24T10:45:53.812 INFO:tasks.workunit.client.0.smithi094.stdout:4/232: creat d0/f4b x:0 0 0 2024-03-24T10:45:53.814 INFO:tasks.workunit.client.0.smithi094.stdout:5/190: dread d7/d8/fa [0,4194304] 0 2024-03-24T10:45:53.815 INFO:tasks.workunit.client.0.smithi094.stdout:1/166: unlink d4/l1c 0 2024-03-24T10:45:53.815 INFO:tasks.workunit.client.0.smithi094.stdout:1/167: write d4/f2a [915834,40615] 0 2024-03-24T10:45:53.816 INFO:tasks.workunit.client.0.smithi094.stdout:0/226: symlink d14/d22/d28/d2b/l4d 0 2024-03-24T10:45:53.817 INFO:tasks.workunit.client.0.smithi094.stdout:8/231: symlink df/d12/l4d 0 2024-03-24T10:45:53.820 INFO:tasks.workunit.client.0.smithi094.stdout:7/212: unlink d1/d9/d16/d20/f26 0 2024-03-24T10:45:53.822 INFO:tasks.workunit.client.0.smithi094.stdout:5/191: creat d7/d23/f44 x:0 0 0 2024-03-24T10:45:53.825 INFO:tasks.workunit.client.0.smithi094.stdout:8/232: unlink df/c13 0 2024-03-24T10:45:53.825 INFO:tasks.workunit.client.0.smithi094.stdout:8/233: dread - df/d25/d28/d31/d47/f4a zero size 2024-03-24T10:45:53.827 INFO:tasks.workunit.client.0.smithi094.stdout:7/213: creat d1/d9/d31/d39/f48 x:0 0 0 2024-03-24T10:45:53.830 INFO:tasks.workunit.client.0.smithi094.stdout:5/192: rmdir d7/d26/d3d 0 2024-03-24T10:45:53.830 INFO:tasks.workunit.client.0.smithi094.stdout:5/193: write d7/d8/d1b/f37 [87784,19350] 0 2024-03-24T10:45:53.830 INFO:tasks.workunit.client.0.smithi094.stdout:5/194: write d7/d8/d1b/f37 [1139158,45082] 0 2024-03-24T10:45:53.831 INFO:tasks.workunit.client.0.smithi094.stdout:6/181: dwrite d0/d5/da/f20 [0,4194304] 0 2024-03-24T10:45:53.831 INFO:tasks.workunit.client.0.smithi094.stdout:6/182: fsync d0/f8 0 2024-03-24T10:45:53.832 INFO:tasks.workunit.client.0.smithi094.stdout:7/214: unlink d1/f7 0 2024-03-24T10:45:53.832 INFO:tasks.workunit.client.0.smithi094.stdout:7/215: write d1/d9/fe [3752026,100390] 0 2024-03-24T10:45:53.832 INFO:tasks.workunit.client.0.smithi094.stdout:6/183: truncate d0/d5/da/f12 372829 0 2024-03-24T10:45:53.834 INFO:tasks.workunit.client.0.smithi094.stdout:7/216: mkdir d1/d9/d16/d21/d49 0 2024-03-24T10:45:53.834 INFO:tasks.workunit.client.0.smithi094.stdout:7/217: chown d1/d9/lb 313 1 2024-03-24T10:45:53.834 INFO:tasks.workunit.client.0.smithi094.stdout:7/218: stat d1/d9/d25/d3d 0 2024-03-24T10:45:53.834 INFO:tasks.workunit.client.0.smithi094.stdout:7/219: write f0 [3063797,84591] 0 2024-03-24T10:45:53.834 INFO:tasks.workunit.client.0.smithi094.stdout:7/220: stat d1/d9/d16/l32 0 2024-03-24T10:45:53.834 INFO:tasks.workunit.client.0.smithi094.stdout:7/221: write d1/d9/f23 [2566401,97648] 0 2024-03-24T10:45:53.834 INFO:tasks.workunit.client.0.smithi094.stdout:7/222: chown d1/d9/d25/d3d/f40 0 1 2024-03-24T10:45:53.834 INFO:tasks.workunit.client.0.smithi094.stdout:5/195: dread d7/d8/fa [0,4194304] 0 2024-03-24T10:45:53.834 INFO:tasks.workunit.client.0.smithi094.stdout:6/184: mknod d0/d21/d2a/d2f/c32 0 2024-03-24T10:45:53.835 INFO:tasks.workunit.client.0.smithi094.stdout:7/223: link d1/l8 d1/d9/d14/l4a 0 2024-03-24T10:45:53.836 INFO:tasks.workunit.client.0.smithi094.stdout:5/196: symlink d7/d23/d2b/l45 0 2024-03-24T10:45:53.836 INFO:tasks.workunit.client.0.smithi094.stdout:6/185: creat d0/f33 x:0 0 0 2024-03-24T10:45:53.838 INFO:tasks.workunit.client.0.smithi094.stdout:7/224: link d1/d9/f23 d1/d9/d14/f4b 0 2024-03-24T10:45:53.839 INFO:tasks.workunit.client.0.smithi094.stdout:5/197: rename d7/d23/d2b/l30 to d7/d8/l46 0 2024-03-24T10:45:53.840 INFO:tasks.workunit.client.0.smithi094.stdout:6/186: mkdir d0/d5/da/d23/d34 0 2024-03-24T10:45:53.841 INFO:tasks.workunit.client.0.smithi094.stdout:5/198: unlink d7/d23/d2b/f2f 0 2024-03-24T10:45:53.841 INFO:tasks.workunit.client.0.smithi094.stdout:5/199: chown d7/d8 3104782 1 2024-03-24T10:45:53.841 INFO:tasks.workunit.client.0.smithi094.stdout:6/187: mkdir d0/d1a/d35 0 2024-03-24T10:45:53.843 INFO:tasks.workunit.client.0.smithi094.stdout:5/200: rename d7/d23/d2b/f43 to d7/d8/d14/d36/f47 0 2024-03-24T10:45:53.843 INFO:tasks.workunit.client.0.smithi094.stdout:5/201: chown d7/d23/d2b/l45 1741 1 2024-03-24T10:45:53.843 INFO:tasks.workunit.client.0.smithi094.stdout:4/233: dwrite d0/f7 [4194304,4194304] 0 2024-03-24T10:45:53.843 INFO:tasks.workunit.client.0.smithi094.stdout:1/168: dwrite d4/d9/f25 [0,4194304] 0 2024-03-24T10:45:53.844 INFO:tasks.workunit.client.0.smithi094.stdout:6/188: creat d0/d21/d2a/d2f/f36 x:0 0 0 2024-03-24T10:45:53.844 INFO:tasks.workunit.client.0.smithi094.stdout:6/189: read - d0/d21/f2d zero size 2024-03-24T10:45:53.844 INFO:tasks.workunit.client.0.smithi094.stdout:6/190: dread - d0/d5/f29 zero size 2024-03-24T10:45:53.847 INFO:tasks.workunit.client.0.smithi094.stdout:7/225: dread d1/d9/f23 [0,4194304] 0 2024-03-24T10:45:53.852 INFO:tasks.workunit.client.0.smithi094.stdout:1/169: dread d4/d9/db/f18 [0,4194304] 0 2024-03-24T10:45:53.852 INFO:tasks.workunit.client.0.smithi094.stdout:0/227: getdents d14/d22/d28/d2b 0 2024-03-24T10:45:53.852 INFO:tasks.workunit.client.0.smithi094.stdout:0/228: truncate d14/d22/d28/d48/f4a 969479 0 2024-03-24T10:45:53.853 INFO:tasks.workunit.client.0.smithi094.stdout:5/202: mkdir d7/d8/d9/d48 0 2024-03-24T10:45:53.854 INFO:tasks.workunit.client.0.smithi094.stdout:4/234: symlink d0/d5/d42/l4c 0 2024-03-24T10:45:53.854 INFO:tasks.workunit.client.0.smithi094.stdout:4/235: fdatasync d0/f7 0 2024-03-24T10:45:53.854 INFO:tasks.workunit.client.0.smithi094.stdout:4/236: chown d0/d1f 7413032 1 2024-03-24T10:45:53.854 INFO:tasks.workunit.client.0.smithi094.stdout:6/191: write d0/d5/da/f20 [2008116,57947] 0 2024-03-24T10:45:53.855 INFO:tasks.workunit.client.0.smithi094.stdout:1/170: dread d4/d9/fa [4194304,4194304] 0 2024-03-24T10:45:53.855 INFO:tasks.workunit.client.0.smithi094.stdout:1/171: readlink d4/l6 0 2024-03-24T10:45:53.857 INFO:tasks.workunit.client.0.smithi094.stdout:7/226: rename d1/d9/d14/f42 to d1/d9/d14/f4c 0 2024-03-24T10:45:53.858 INFO:tasks.workunit.client.0.smithi094.stdout:1/172: write d4/f1e [1911361,5773] 0 2024-03-24T10:45:53.859 INFO:tasks.workunit.client.0.smithi094.stdout:0/229: link d14/c19 d14/d22/d28/d2b/c4e 0 2024-03-24T10:45:53.860 INFO:tasks.workunit.client.0.smithi094.stdout:7/227: dread d1/d9/f22 [0,4194304] 0 2024-03-24T10:45:53.860 INFO:tasks.workunit.client.0.smithi094.stdout:7/228: dread - d1/d9/d14/f36 zero size 2024-03-24T10:45:53.863 INFO:tasks.workunit.client.0.smithi094.stdout:5/203: unlink d7/f24 0 2024-03-24T10:45:53.863 INFO:tasks.workunit.client.0.smithi094.stdout:0/230: write d14/d22/d28/d2b/f3b [1851018,116441] 0 2024-03-24T10:45:53.864 INFO:tasks.workunit.client.0.smithi094.stdout:6/192: mknod d0/d5/da/d23/d34/c37 0 2024-03-24T10:45:53.864 INFO:tasks.workunit.client.0.smithi094.stdout:4/237: dread d0/d34/f48 [0,4194304] 0 2024-03-24T10:45:53.865 INFO:tasks.workunit.client.0.smithi094.stdout:4/238: dread - d0/d22/de/d28/f3f zero size 2024-03-24T10:45:53.865 INFO:tasks.workunit.client.0.smithi094.stdout:1/173: truncate d4/f7 1218 0 2024-03-24T10:45:53.867 INFO:tasks.workunit.client.0.smithi094.stdout:7/229: mknod d1/d9/d14/d34/c4d 0 2024-03-24T10:45:53.867 INFO:tasks.workunit.client.0.smithi094.stdout:7/230: readlink d1/d9/d16/l32 0 2024-03-24T10:45:53.870 INFO:tasks.workunit.client.0.smithi094.stdout:0/231: creat d14/d18/f4f x:0 0 0 2024-03-24T10:45:53.876 INFO:tasks.workunit.client.0.smithi094.stdout:4/239: write d0/f7 [7902613,69880] 0 2024-03-24T10:45:53.877 INFO:tasks.workunit.client.0.smithi094.stdout:0/232: symlink d14/d22/d32/l50 0 2024-03-24T10:45:53.877 INFO:tasks.workunit.client.0.smithi094.stdout:0/233: chown d14/f1e 7254795 1 2024-03-24T10:45:53.882 INFO:tasks.workunit.client.0.smithi094.stdout:4/240: mknod d0/d22/c4d 0 2024-03-24T10:45:53.882 INFO:tasks.workunit.client.0.smithi094.stdout:4/241: creat d0/d5/d1b/f4e x:0 0 0 2024-03-24T10:45:53.885 INFO:tasks.workunit.client.0.smithi094.stdout:4/242: mkdir d0/d5/d1b/d4f 0 2024-03-24T10:45:53.892 INFO:tasks.workunit.client.0.smithi094.stdout:4/243: write d0/d22/de/d28/d2b/d33/f40 [1328872,21301] 0 2024-03-24T10:45:53.892 INFO:tasks.workunit.client.0.smithi094.stdout:4/244: fsync d0/d5/d1b/f39 0 2024-03-24T10:45:53.897 INFO:tasks.workunit.client.0.smithi094.stdout:6/193: dwrite d0/d21/d2a/d2f/f36 [0,4194304] 0 2024-03-24T10:45:53.900 INFO:tasks.workunit.client.0.smithi094.stdout:6/194: creat d0/d5/da/d23/d34/f38 x:0 0 0 2024-03-24T10:45:53.900 INFO:tasks.workunit.client.0.smithi094.stdout:6/195: truncate d0/d5/f19 270893 0 2024-03-24T10:45:53.902 INFO:tasks.workunit.client.0.smithi094.stdout:7/231: dwrite d1/d9/d14/f36 [0,4194304] 0 2024-03-24T10:45:53.902 INFO:tasks.workunit.client.0.smithi094.stdout:7/232: getdents d1/d9 0 2024-03-24T10:45:53.907 INFO:tasks.workunit.client.0.smithi094.stdout:4/245: dwrite d0/d22/de/f30 [0,4194304] 0 2024-03-24T10:45:53.907 INFO:tasks.workunit.client.0.smithi094.stdout:6/196: dread d0/d30/f28 [0,4194304] 0 2024-03-24T10:45:53.907 INFO:tasks.workunit.client.0.smithi094.stdout:0/234: dwrite d14/d18/f2e [0,4194304] 0 2024-03-24T10:45:53.913 INFO:tasks.workunit.client.0.smithi094.stdout:6/197: creat d0/d1a/f39 x:0 0 0 2024-03-24T10:45:53.913 INFO:tasks.workunit.client.0.smithi094.stdout:6/198: stat d0/d21/f2d 0 2024-03-24T10:45:53.913 INFO:tasks.workunit.client.0.smithi094.stdout:6/199: dread - d0/d21/f2d zero size 2024-03-24T10:45:53.916 INFO:tasks.workunit.client.0.smithi094.stdout:6/200: dread d0/d5/da/d23/f26 [0,4194304] 0 2024-03-24T10:45:53.922 INFO:tasks.workunit.client.0.smithi094.stdout:7/233: dwrite d1/d9/d16/d21/f33 [0,4194304] 0 2024-03-24T10:45:53.923 INFO:tasks.workunit.client.0.smithi094.stdout:7/234: truncate d1/d9/f30 1647887 0 2024-03-24T10:45:53.923 INFO:tasks.workunit.client.0.smithi094.stdout:7/235: truncate d1/d9/d14/f4b 3693159 0 2024-03-24T10:45:53.923 INFO:tasks.workunit.client.0.smithi094.stdout:7/236: dread d1/d9/f30 [0,4194304] 0 2024-03-24T10:45:53.924 INFO:tasks.workunit.client.0.smithi094.stdout:7/237: fdatasync d1/d9/fd 0 2024-03-24T10:45:53.926 INFO:tasks.workunit.client.0.smithi094.stdout:7/238: creat d1/d9/d31/d39/d44/f4e x:0 0 0 2024-03-24T10:45:53.926 INFO:tasks.workunit.client.0.smithi094.stdout:7/239: chown d1/d9/f13 392752 1 2024-03-24T10:45:53.926 INFO:tasks.workunit.client.0.smithi094.stdout:2/153: sync 2024-03-24T10:45:53.927 INFO:tasks.workunit.client.0.smithi094.stdout:3/165: sync 2024-03-24T10:45:53.927 INFO:tasks.workunit.client.0.smithi094.stdout:2/154: truncate d18/f1b 354563 0 2024-03-24T10:45:53.927 INFO:tasks.workunit.client.0.smithi094.stdout:2/155: dread - d18/d1e/f25 zero size 2024-03-24T10:45:53.928 INFO:tasks.workunit.client.0.smithi094.stdout:3/166: mkdir d0/db/dc/d28/d3c 0 2024-03-24T10:45:53.929 INFO:tasks.workunit.client.0.smithi094.stdout:3/167: mknod d0/db/d1c/d20/d23/d2a/c3d 0 2024-03-24T10:45:53.931 INFO:tasks.workunit.client.0.smithi094.stdout:3/168: symlink d0/db/dc/d28/l3e 0 2024-03-24T10:45:53.931 INFO:tasks.workunit.client.0.smithi094.stdout:3/169: creat d0/db/d1c/d20/d23/f3f x:0 0 0 2024-03-24T10:45:53.932 INFO:tasks.workunit.client.0.smithi094.stdout:2/156: dread d18/f23 [0,4194304] 0 2024-03-24T10:45:53.933 INFO:tasks.workunit.client.0.smithi094.stdout:3/170: read d0/db/dc/f2d [29583,50038] 0 2024-03-24T10:45:53.934 INFO:tasks.workunit.client.0.smithi094.stdout:3/171: write d0/db/dc/f12 [365297,90684] 0 2024-03-24T10:45:53.934 INFO:tasks.workunit.client.0.smithi094.stdout:3/172: write d0/db/d1c/d20/d23/f3f [471604,97958] 0 2024-03-24T10:45:53.936 INFO:tasks.workunit.client.0.smithi094.stdout:3/173: symlink d0/db/dc/l40 0 2024-03-24T10:45:53.936 INFO:tasks.workunit.client.0.smithi094.stdout:3/174: fdatasync d0/db/d1c/d20/f24 0 2024-03-24T10:45:53.937 INFO:tasks.workunit.client.0.smithi094.stdout:3/175: link d0/db/d1c/d20/d23/d2a/d3a/c1a d0/db/c41 0 2024-03-24T10:45:53.972 INFO:tasks.workunit.client.0.smithi094.stdout:4/246: dwrite d0/d34/f48 [4194304,4194304] 0 2024-03-24T10:45:53.972 INFO:tasks.workunit.client.0.smithi094.stdout:0/235: dwrite d14/d22/d28/d2b/f45 [0,4194304] 0 2024-03-24T10:45:53.972 INFO:tasks.workunit.client.0.smithi094.stdout:0/236: fdatasync d14/d18/f40 0 2024-03-24T10:45:53.972 INFO:tasks.workunit.client.0.smithi094.stdout:0/237: dread - d14/d18/f4f zero size 2024-03-24T10:45:53.972 INFO:tasks.workunit.client.0.smithi094.stdout:7/240: dwrite d1/f2 [4194304,4194304] 0 2024-03-24T10:45:53.972 INFO:tasks.workunit.client.0.smithi094.stdout:2/157: dwrite d18/f29 [0,4194304] 0 2024-03-24T10:45:53.972 INFO:tasks.workunit.client.0.smithi094.stdout:3/176: dwrite d0/f2 [0,4194304] 0 2024-03-24T10:45:53.972 INFO:tasks.workunit.client.0.smithi094.stdout:6/201: dwrite d0/d5/da/f20 [0,4194304] 0 2024-03-24T10:45:53.972 INFO:tasks.workunit.client.0.smithi094.stdout:6/202: truncate d0/d5/da/f12 1157695 0 2024-03-24T10:45:53.972 INFO:tasks.workunit.client.0.smithi094.stdout:6/203: creat d0/d21/f3a x:0 0 0 2024-03-24T10:45:53.972 INFO:tasks.workunit.client.0.smithi094.stdout:6/204: chown d0/d21/d2a/d2f/c32 0 1 2024-03-24T10:45:53.977 INFO:tasks.workunit.client.0.smithi094.stdout:4/247: write d0/d22/f11 [3569317,32474] 0 2024-03-24T10:45:53.977 INFO:tasks.workunit.client.0.smithi094.stdout:0/238: mknod d14/d22/c51 0 2024-03-24T10:45:53.980 INFO:tasks.workunit.client.0.smithi094.stdout:7/241: mknod d1/d9/d31/d39/d44/c4f 0 2024-03-24T10:45:53.993 INFO:tasks.workunit.client.0.smithi094.stdout:7/242: rename d1/d9/d16/d20/f28 to d1/d9/d25/f50 0 2024-03-24T10:45:53.993 INFO:tasks.workunit.client.0.smithi094.stdout:6/205: mkdir d0/d3b 0 2024-03-24T10:45:53.995 INFO:tasks.workunit.client.0.smithi094.stdout:0/239: getdents d14 0 2024-03-24T10:45:53.995 INFO:tasks.workunit.client.0.smithi094.stdout:0/240: readlink d14/l39 0 2024-03-24T10:45:53.995 INFO:tasks.workunit.client.0.smithi094.stdout:7/243: mkdir d1/d9/d16/d21/d51 0 2024-03-24T10:45:53.997 INFO:tasks.workunit.client.0.smithi094.stdout:0/241: creat d14/f52 x:0 0 0 2024-03-24T10:45:53.998 INFO:tasks.workunit.client.0.smithi094.stdout:7/244: unlink d1/d9/d16/l32 0 2024-03-24T10:45:53.998 INFO:tasks.workunit.client.0.smithi094.stdout:6/206: rmdir d0/d1a 39 2024-03-24T10:45:53.999 INFO:tasks.workunit.client.0.smithi094.stdout:0/242: readlink d14/d22/d28/l4c 0 2024-03-24T10:45:53.999 INFO:tasks.workunit.client.0.smithi094.stdout:7/245: mknod d1/d9/d16/d20/c52 0 2024-03-24T10:45:54.000 INFO:tasks.workunit.client.0.smithi094.stdout:6/207: creat d0/f3c x:0 0 0 2024-03-24T10:45:54.001 INFO:tasks.workunit.client.0.smithi094.stdout:7/246: symlink d1/d9/l53 0 2024-03-24T10:45:54.002 INFO:tasks.workunit.client.0.smithi094.stdout:6/208: mkdir d0/d5/da/d23/d34/d3d 0 2024-03-24T10:45:54.002 INFO:tasks.workunit.client.0.smithi094.stdout:7/247: symlink d1/d9/d31/l54 0 2024-03-24T10:45:54.003 INFO:tasks.workunit.client.0.smithi094.stdout:6/209: symlink d0/d1a/l3e 0 2024-03-24T10:45:54.004 INFO:tasks.workunit.client.0.smithi094.stdout:6/210: symlink d0/d5/da/l3f 0 2024-03-24T10:45:54.006 INFO:tasks.workunit.client.0.smithi094.stdout:3/177: dwrite d0/f2 [0,4194304] 0 2024-03-24T10:45:54.006 INFO:tasks.workunit.client.0.smithi094.stdout:4/248: dwrite d0/f47 [0,4194304] 0 2024-03-24T10:45:54.006 INFO:tasks.workunit.client.0.smithi094.stdout:3/178: write d0/db/f2b [238425,70093] 0 2024-03-24T10:45:54.006 INFO:tasks.workunit.client.0.smithi094.stdout:3/179: fdatasync d0/f2 0 2024-03-24T10:45:54.006 INFO:tasks.workunit.client.0.smithi094.stdout:3/180: creat d0/f42 x:0 0 0 2024-03-24T10:45:54.009 INFO:tasks.workunit.client.0.smithi094.stdout:4/249: symlink d0/d5/d1b/l50 0 2024-03-24T10:45:54.009 INFO:tasks.workunit.client.0.smithi094.stdout:4/250: write d0/d22/de/d28/f3a [262739,38524] 0 2024-03-24T10:45:54.009 INFO:tasks.workunit.client.0.smithi094.stdout:4/251: fdatasync d0/d5/d1b/f39 0 2024-03-24T10:45:54.010 INFO:tasks.workunit.client.0.smithi094.stdout:2/158: dwrite d18/f1d [0,4194304] 0 2024-03-24T10:45:54.010 INFO:tasks.workunit.client.0.smithi094.stdout:6/211: write d0/d21/f13 [2787702,102630] 0 2024-03-24T10:45:54.010 INFO:tasks.workunit.client.0.smithi094.stdout:3/181: symlink d0/db/d1c/d20/d23/d2a/l43 0 2024-03-24T10:45:54.014 INFO:tasks.workunit.client.0.smithi094.stdout:4/252: link d0/d5/f16 d0/d22/de/d28/d2b/f51 0 2024-03-24T10:45:54.023 INFO:tasks.workunit.client.0.smithi094.stdout:6/212: rename d0/d5/da/d23/d34/f38 to d0/d21/d2a/f40 0 2024-03-24T10:45:54.023 INFO:tasks.workunit.client.0.smithi094.stdout:9/244: sync 2024-03-24T10:45:54.023 INFO:tasks.workunit.client.0.smithi094.stdout:8/234: sync 2024-03-24T10:45:54.023 INFO:tasks.workunit.client.0.smithi094.stdout:8/235: dread - df/d25/d28/d31/d47/d22/f38 zero size 2024-03-24T10:45:54.023 INFO:tasks.workunit.client.0.smithi094.stdout:8/236: readlink lb 0 2024-03-24T10:45:54.023 INFO:tasks.workunit.client.0.smithi094.stdout:9/245: write df/d2d/d34/f2e [1197463,65907] 0 2024-03-24T10:45:54.026 INFO:tasks.workunit.client.0.smithi094.stdout:6/213: link d0/d5/da/d23/d34/c37 d0/d5/da/d23/d34/d3d/c41 0 2024-03-24T10:45:54.026 INFO:tasks.workunit.client.0.smithi094.stdout:0/243: dwrite d14/d18/f37 [0,4194304] 0 2024-03-24T10:45:54.026 INFO:tasks.workunit.client.0.smithi094.stdout:0/244: chown d14/d22/d28/f46 236138175 1 2024-03-24T10:45:54.028 INFO:tasks.workunit.client.0.smithi094.stdout:9/246: creat df/d2d/d34/d2c/d3a/f4b x:0 0 0 2024-03-24T10:45:54.029 INFO:tasks.workunit.client.0.smithi094.stdout:7/248: dwrite d1/d9/d25/d3d/f46 [0,4194304] 0 2024-03-24T10:45:54.029 INFO:tasks.workunit.client.0.smithi094.stdout:7/249: dread - d1/d9/d16/d21/f3f zero size 2024-03-24T10:45:54.029 INFO:tasks.workunit.client.0.smithi094.stdout:7/250: chown d1/d9/d31/l45 26 1 2024-03-24T10:45:54.029 INFO:tasks.workunit.client.0.smithi094.stdout:7/251: write d1/d9/d14/f4c [154146,3171] 0 2024-03-24T10:45:54.029 INFO:tasks.workunit.client.0.smithi094.stdout:7/252: creat d1/d9/d25/d3d/f55 x:0 0 0 2024-03-24T10:45:54.030 INFO:tasks.workunit.client.0.smithi094.stdout:7/253: readlink d1/d9/d14/l4a 0 2024-03-24T10:45:54.030 INFO:tasks.workunit.client.0.smithi094.stdout:7/254: write d1/d9/d14/f4c [476887,11449] 0 2024-03-24T10:45:54.030 INFO:tasks.workunit.client.0.smithi094.stdout:7/255: dread - d1/d9/d25/d3d/f40 zero size 2024-03-24T10:45:54.030 INFO:tasks.workunit.client.0.smithi094.stdout:7/256: creat d1/d9/d31/d39/d44/f56 x:0 0 0 2024-03-24T10:45:54.031 INFO:tasks.workunit.client.0.smithi094.stdout:0/245: mkdir d14/d22/d28/d2b/d41/d53 0 2024-03-24T10:45:54.031 INFO:tasks.workunit.client.0.smithi094.stdout:0/246: chown d14/d22/f33 890 1 2024-03-24T10:45:54.032 INFO:tasks.workunit.client.0.smithi094.stdout:9/247: creat df/d2d/d34/d2c/f4c x:0 0 0 2024-03-24T10:45:54.037 INFO:tasks.workunit.client.0.smithi094.stdout:7/257: mknod d1/d9/d31/c57 0 2024-03-24T10:45:54.042 INFO:tasks.workunit.client.0.smithi094.stdout:9/248: creat df/d2d/d34/d2c/f4d x:0 0 0 2024-03-24T10:45:54.042 INFO:tasks.workunit.client.0.smithi094.stdout:9/249: write df/d2d/d1f/f49 [326612,72460] 0 2024-03-24T10:45:54.044 INFO:tasks.workunit.client.0.smithi094.stdout:9/250: creat df/d2d/d1f/d11/f4e x:0 0 0 2024-03-24T10:45:54.047 INFO:tasks.workunit.client.0.smithi094.stdout:1/174: sync 2024-03-24T10:45:54.048 INFO:tasks.workunit.client.0.smithi094.stdout:5/204: sync 2024-03-24T10:45:54.048 INFO:tasks.workunit.client.0.smithi094.stdout:1/175: chown d4/f2a 263673217 1 2024-03-24T10:45:54.049 INFO:tasks.workunit.client.0.smithi094.stdout:0/247: dread d14/d18/f21 [0,4194304] 0 2024-03-24T10:45:54.049 INFO:tasks.workunit.client.0.smithi094.stdout:5/205: creat d7/d8/d9/f49 x:0 0 0 2024-03-24T10:45:54.050 INFO:tasks.workunit.client.0.smithi094.stdout:2/159: dwrite d18/d1e/f25 [0,4194304] 0 2024-03-24T10:45:54.051 INFO:tasks.workunit.client.0.smithi094.stdout:0/248: link d14/d22/f33 d14/d18/f54 0 2024-03-24T10:45:54.051 INFO:tasks.workunit.client.0.smithi094.stdout:0/249: write d14/d22/d28/d2b/f38 [70137,6853] 0 2024-03-24T10:45:54.052 INFO:tasks.workunit.client.0.smithi094.stdout:9/251: dread df/d2d/f1e [0,4194304] 0 2024-03-24T10:45:54.053 INFO:tasks.workunit.client.0.smithi094.stdout:5/206: symlink d7/d8/d9/l4a 0 2024-03-24T10:45:54.053 INFO:tasks.workunit.client.0.smithi094.stdout:5/207: creat d7/d23/d2b/f4b x:0 0 0 2024-03-24T10:45:54.053 INFO:tasks.workunit.client.0.smithi094.stdout:5/208: chown d7/dc/d12/l18 854 1 2024-03-24T10:45:54.053 INFO:tasks.workunit.client.0.smithi094.stdout:5/209: truncate d7/dc/f19 669954 0 2024-03-24T10:45:54.053 INFO:tasks.workunit.client.0.smithi094.stdout:5/210: chown d7/d8/fa 122361 1 2024-03-24T10:45:54.053 INFO:tasks.workunit.client.0.smithi094.stdout:5/211: chown d7/d23 38732000 1 2024-03-24T10:45:54.053 INFO:tasks.workunit.client.0.smithi094.stdout:2/160: dread d18/d1e/f24 [0,4194304] 0 2024-03-24T10:45:54.053 INFO:tasks.workunit.client.0.smithi094.stdout:2/161: fsync d18/f27 0 2024-03-24T10:45:54.053 INFO:tasks.workunit.client.0.smithi094.stdout:2/162: dread - d18/f2e zero size 2024-03-24T10:45:54.054 INFO:tasks.workunit.client.0.smithi094.stdout:0/250: mknod d14/d22/d28/d2b/d41/c55 0 2024-03-24T10:45:54.056 INFO:tasks.workunit.client.0.smithi094.stdout:1/176: dread d4/d9/f2d [0,4194304] 0 2024-03-24T10:45:54.056 INFO:tasks.workunit.client.0.smithi094.stdout:1/177: chown d4 1931951 1 2024-03-24T10:45:54.057 INFO:tasks.workunit.client.0.smithi094.stdout:9/252: dread df/f36 [0,4194304] 0 2024-03-24T10:45:54.060 INFO:tasks.workunit.client.0.smithi094.stdout:8/237: dwrite df/d12/f21 [0,4194304] 0 2024-03-24T10:45:54.060 INFO:tasks.workunit.client.0.smithi094.stdout:8/238: read f4 [1338507,80501] 0 2024-03-24T10:45:54.061 INFO:tasks.workunit.client.0.smithi094.stdout:7/258: dwrite d1/d9/d16/f2f [0,4194304] 0 2024-03-24T10:45:54.061 INFO:tasks.workunit.client.0.smithi094.stdout:7/259: dread - d1/d9/d25/f50 zero size 2024-03-24T10:45:54.061 INFO:tasks.workunit.client.0.smithi094.stdout:7/260: write d1/d9/d25/f50 [788969,128924] 0 2024-03-24T10:45:54.064 INFO:tasks.workunit.client.0.smithi094.stdout:6/214: dwrite d0/f33 [0,4194304] 0 2024-03-24T10:45:54.065 INFO:tasks.workunit.client.0.smithi094.stdout:9/253: read df/d2d/d34/f2e [733248,95205] 0 2024-03-24T10:45:54.070 INFO:tasks.workunit.client.0.smithi094.stdout:6/215: truncate d0/d30/f28 3308318 0 2024-03-24T10:45:54.070 INFO:tasks.workunit.client.0.smithi094.stdout:2/163: dread d18/d1e/f2b [0,4194304] 0 2024-03-24T10:45:54.071 INFO:tasks.workunit.client.0.smithi094.stdout:4/253: dwrite d0/d22/de/d28/d2b/d33/f40 [4194304,4194304] 0 2024-03-24T10:45:54.071 INFO:tasks.workunit.client.0.smithi094.stdout:4/254: stat d0/f47 0 2024-03-24T10:45:54.071 INFO:tasks.workunit.client.0.smithi094.stdout:8/239: dread df/d25/d28/d31/f32 [0,4194304] 0 2024-03-24T10:45:54.071 INFO:tasks.workunit.client.0.smithi094.stdout:4/255: creat d0/d22/de/d28/f52 x:0 0 0 2024-03-24T10:45:54.073 INFO:tasks.workunit.client.0.smithi094.stdout:2/164: symlink d18/l2f 0 2024-03-24T10:45:54.073 INFO:tasks.workunit.client.0.smithi094.stdout:2/165: fdatasync d18/f1b 0 2024-03-24T10:45:54.073 INFO:tasks.workunit.client.0.smithi094.stdout:2/166: stat d18/d1e/f24 0 2024-03-24T10:45:54.073 INFO:tasks.workunit.client.0.smithi094.stdout:2/167: chown l3 676980 1 2024-03-24T10:45:54.076 INFO:tasks.workunit.client.0.smithi094.stdout:4/256: unlink d0/d22/de/d28/d2b/l38 0 2024-03-24T10:45:54.077 INFO:tasks.workunit.client.0.smithi094.stdout:5/212: dwrite d7/f34 [0,4194304] 0 2024-03-24T10:45:54.077 INFO:tasks.workunit.client.0.smithi094.stdout:1/178: dwrite d4/d9/db/f10 [0,4194304] 0 2024-03-24T10:45:54.078 INFO:tasks.workunit.client.0.smithi094.stdout:6/216: link d0/f3c d0/f42 0 2024-03-24T10:45:54.079 INFO:tasks.workunit.client.0.smithi094.stdout:6/217: write d0/d5/da/f12 [77157,97775] 0 2024-03-24T10:45:54.079 INFO:tasks.workunit.client.0.smithi094.stdout:6/218: chown d0/d21/f2d 27 1 2024-03-24T10:45:54.081 INFO:tasks.workunit.client.0.smithi094.stdout:5/213: dread d7/d8/d1b/f37 [0,4194304] 0 2024-03-24T10:45:54.081 INFO:tasks.workunit.client.0.smithi094.stdout:5/214: creat d7/f4c x:0 0 0 2024-03-24T10:45:54.081 INFO:tasks.workunit.client.0.smithi094.stdout:5/215: fsync d7/d8/d14/f2a 0 2024-03-24T10:45:54.081 INFO:tasks.workunit.client.0.smithi094.stdout:4/257: unlink d0/f7 0 2024-03-24T10:45:54.086 INFO:tasks.workunit.client.0.smithi094.stdout:1/179: mkdir d4/d36/d38 0 2024-03-24T10:45:54.088 INFO:tasks.workunit.client.0.smithi094.stdout:2/168: unlink cc 0 2024-03-24T10:45:54.088 INFO:tasks.workunit.client.0.smithi094.stdout:2/169: fdatasync d18/d1e/f2b 0 2024-03-24T10:45:54.089 INFO:tasks.workunit.client.0.smithi094.stdout:2/170: stat f12 0 2024-03-24T10:45:54.089 INFO:tasks.workunit.client.0.smithi094.stdout:6/219: symlink d0/d1a/l43 0 2024-03-24T10:45:54.089 INFO:tasks.workunit.client.0.smithi094.stdout:6/220: stat d0/d5/f29 0 2024-03-24T10:45:54.090 INFO:tasks.workunit.client.0.smithi094.stdout:5/216: dread d7/d8/d14/f35 [0,4194304] 0 2024-03-24T10:45:54.090 INFO:tasks.workunit.client.0.smithi094.stdout:4/258: symlink d0/d22/l53 0 2024-03-24T10:45:54.091 INFO:tasks.workunit.client.0.smithi094.stdout:1/180: mkdir d4/d9/db/d39 0 2024-03-24T10:45:54.093 INFO:tasks.workunit.client.0.smithi094.stdout:2/171: rename d18/c1a to d18/d1e/c30 0 2024-03-24T10:45:54.093 INFO:tasks.workunit.client.0.smithi094.stdout:2/172: creat d18/f31 x:0 0 0 2024-03-24T10:45:54.093 INFO:tasks.workunit.client.0.smithi094.stdout:5/217: creat d7/d8/d14/d36/f4d x:0 0 0 2024-03-24T10:45:54.095 INFO:tasks.workunit.client.0.smithi094.stdout:4/259: creat d0/d22/de/d28/d2b/d33/f54 x:0 0 0 2024-03-24T10:45:54.095 INFO:tasks.workunit.client.0.smithi094.stdout:4/260: chown d0/d5/l29 424973143 1 2024-03-24T10:45:54.096 INFO:tasks.workunit.client.0.smithi094.stdout:1/181: creat d4/d9/d37/d33/f3a x:0 0 0 2024-03-24T10:45:54.096 INFO:tasks.workunit.client.0.smithi094.stdout:1/182: readlink d4/d9/l17 0 2024-03-24T10:45:54.099 INFO:tasks.workunit.client.0.smithi094.stdout:9/254: dwrite df/d2d/f1e [4194304,4194304] 0 2024-03-24T10:45:54.099 INFO:tasks.workunit.client.0.smithi094.stdout:2/173: creat d18/d1e/f32 x:0 0 0 2024-03-24T10:45:54.100 INFO:tasks.workunit.client.0.smithi094.stdout:8/240: dwrite f8 [0,4194304] 0 2024-03-24T10:45:54.101 INFO:tasks.workunit.client.0.smithi094.stdout:4/261: stat d0/d1f/c32 0 2024-03-24T10:45:54.101 INFO:tasks.workunit.client.0.smithi094.stdout:4/262: write d0/d22/de/d28/d2b/d33/f54 [449425,53566] 0 2024-03-24T10:45:54.102 INFO:tasks.workunit.client.0.smithi094.stdout:4/263: read - d0/d5/d1b/f4e zero size 2024-03-24T10:45:54.102 INFO:tasks.workunit.client.0.smithi094.stdout:4/264: truncate d0/d22/ff 801627 0 2024-03-24T10:45:54.104 INFO:tasks.workunit.client.0.smithi094.stdout:2/174: unlink d18/d1e/f2b 0 2024-03-24T10:45:54.108 INFO:tasks.workunit.client.0.smithi094.stdout:4/265: truncate d0/d22/de/f30 2479751 0 2024-03-24T10:45:54.108 INFO:tasks.workunit.client.0.smithi094.stdout:4/266: truncate d0/d22/de/d28/d2b/f43 974903 0 2024-03-24T10:45:54.109 INFO:tasks.workunit.client.0.smithi094.stdout:9/255: rmdir df/d2d/d34/d2c/d3a 39 2024-03-24T10:45:54.109 INFO:tasks.workunit.client.0.smithi094.stdout:9/256: write df/d2d/d34/f2e [1385471,87405] 0 2024-03-24T10:45:54.109 INFO:tasks.workunit.client.0.smithi094.stdout:9/257: dread - df/d2d/d34/d2c/f4c zero size 2024-03-24T10:45:54.114 INFO:tasks.workunit.client.0.smithi094.stdout:9/258: rmdir df/d2d/d34/d2c/d3a 39 2024-03-24T10:45:54.114 INFO:tasks.workunit.client.0.smithi094.stdout:9/259: chown df/d2d/d1f/l37 50148 1 2024-03-24T10:45:54.116 INFO:tasks.workunit.client.0.smithi094.stdout:4/267: getdents d0/d22/de/d28/d2b 0 2024-03-24T10:45:54.118 INFO:tasks.workunit.client.0.smithi094.stdout:9/260: stat df/d2d/d1f/d11/c18 0 2024-03-24T10:45:54.119 INFO:tasks.workunit.client.0.smithi094.stdout:9/261: rename df/f2b to df/d2d/d1f/d11/f4f 0 2024-03-24T10:45:54.119 INFO:tasks.workunit.client.0.smithi094.stdout:9/262: chown df/d2d/d34/f39 98059938 1 2024-03-24T10:45:54.121 INFO:tasks.workunit.client.0.smithi094.stdout:9/263: rmdir df/d2d/d1f/d11/d43 39 2024-03-24T10:45:54.122 INFO:tasks.workunit.client.0.smithi094.stdout:6/221: dwrite d0/d21/d2a/d2f/f36 [0,4194304] 0 2024-03-24T10:45:54.122 INFO:tasks.workunit.client.0.smithi094.stdout:6/222: chown d0/d21/d2a/f40 2552 1 2024-03-24T10:45:54.122 INFO:tasks.workunit.client.0.smithi094.stdout:6/223: dread - d0/d5/da/f2e zero size 2024-03-24T10:45:54.122 INFO:tasks.workunit.client.0.smithi094.stdout:6/224: dread - d0/d5/f29 zero size 2024-03-24T10:45:54.122 INFO:tasks.workunit.client.0.smithi094.stdout:6/225: fsync d0/f3c 0 2024-03-24T10:45:54.122 INFO:tasks.workunit.client.0.smithi094.stdout:6/226: stat d0/d5/da/f2e 0 2024-03-24T10:45:54.137 INFO:tasks.workunit.client.0.smithi094.stdout:1/183: dwrite d4/d9/f2d [0,4194304] 0 2024-03-24T10:45:54.138 INFO:tasks.workunit.client.0.smithi094.stdout:5/218: dwrite d7/dc/d12/f1f [0,4194304] 0 2024-03-24T10:45:54.139 INFO:tasks.workunit.client.0.smithi094.stdout:5/219: chown d7/d8/d1b 21570959 1 2024-03-24T10:45:54.141 INFO:tasks.workunit.client.0.smithi094.stdout:8/241: dwrite df/f1b [0,4194304] 0 2024-03-24T10:45:54.141 INFO:tasks.workunit.client.0.smithi094.stdout:8/242: truncate df/d25/f30 87095 0 2024-03-24T10:45:54.144 INFO:tasks.workunit.client.0.smithi094.stdout:4/268: dwrite d0/d22/de/d28/f52 [0,4194304] 0 2024-03-24T10:45:54.145 INFO:tasks.workunit.client.0.smithi094.stdout:6/227: dwrite d0/d21/f2d [0,4194304] 0 2024-03-24T10:45:54.145 INFO:tasks.workunit.client.0.smithi094.stdout:6/228: fdatasync d0/f2 0 2024-03-24T10:45:54.146 INFO:tasks.workunit.client.0.smithi094.stdout:2/175: dwrite f7 [4194304,4194304] 0 2024-03-24T10:45:54.146 INFO:tasks.workunit.client.0.smithi094.stdout:2/176: creat d18/d1e/f33 x:0 0 0 2024-03-24T10:45:54.146 INFO:tasks.workunit.client.0.smithi094.stdout:2/177: chown c9 255061 1 2024-03-24T10:45:54.148 INFO:tasks.workunit.client.0.smithi094.stdout:8/243: creat df/d25/d36/d3f/f4e x:0 0 0 2024-03-24T10:45:54.151 INFO:tasks.workunit.client.0.smithi094.stdout:1/184: write d4/d9/f25 [845375,68517] 0 2024-03-24T10:45:54.152 INFO:tasks.workunit.client.0.smithi094.stdout:4/269: rename d0/d22/f12 to d0/d1f/f55 0 2024-03-24T10:45:54.155 INFO:tasks.workunit.client.0.smithi094.stdout:6/229: creat d0/d5/f44 x:0 0 0 2024-03-24T10:45:54.158 INFO:tasks.workunit.client.0.smithi094.stdout:5/220: dwrite d7/dc/d12/f39 [0,4194304] 0 2024-03-24T10:45:54.159 INFO:tasks.workunit.client.0.smithi094.stdout:1/185: creat d4/d9/de/f3b x:0 0 0 2024-03-24T10:45:54.160 INFO:tasks.workunit.client.0.smithi094.stdout:9/264: rmdir df/d2d/d1f 39 2024-03-24T10:45:54.160 INFO:tasks.workunit.client.0.smithi094.stdout:9/265: chown df/d2d/d34/d2c/f4c 278 1 2024-03-24T10:45:54.160 INFO:tasks.workunit.client.0.smithi094.stdout:9/266: truncate df/d2d/d1f/f49 776416 0 2024-03-24T10:45:54.160 INFO:tasks.workunit.client.0.smithi094.stdout:9/267: write df/d2d/d34/f47 [594457,89526] 0 2024-03-24T10:45:54.168 INFO:tasks.workunit.client.0.smithi094.stdout:8/244: dwrite f3 [0,4194304] 0 2024-03-24T10:45:54.169 INFO:tasks.workunit.client.0.smithi094.stdout:1/186: mkdir d4/d9/db/d31/d3c 0 2024-03-24T10:45:54.170 INFO:tasks.workunit.client.0.smithi094.stdout:9/268: mkdir df/d2d/d34/d2c/d3a/d50 0 2024-03-24T10:45:54.174 INFO:tasks.workunit.client.0.smithi094.stdout:8/245: rename df/d25/d28/f2c to df/d25/d36/d3f/f4f 0 2024-03-24T10:45:54.176 INFO:tasks.workunit.client.0.smithi094.stdout:1/187: mkdir d4/d9/de/d3d 0 2024-03-24T10:45:54.177 INFO:tasks.workunit.client.0.smithi094.stdout:9/269: symlink df/d2d/d1f/d11/d43/l51 0 2024-03-24T10:45:54.177 INFO:tasks.workunit.client.0.smithi094.stdout:9/270: dread - df/d2d/d34/f39 zero size 2024-03-24T10:45:54.177 INFO:tasks.workunit.client.0.smithi094.stdout:9/271: creat df/f52 x:0 0 0 2024-03-24T10:45:54.177 INFO:tasks.workunit.client.0.smithi094.stdout:4/270: dwrite d0/d22/de/d28/d2b/d33/f54 [0,4194304] 0 2024-03-24T10:45:54.179 INFO:tasks.workunit.client.0.smithi094.stdout:5/221: dwrite d7/d8/d9/f29 [0,4194304] 0 2024-03-24T10:45:54.179 INFO:tasks.workunit.client.0.smithi094.stdout:5/222: write d7/dc/f19 [1683502,37509] 0 2024-03-24T10:45:54.180 INFO:tasks.workunit.client.0.smithi094.stdout:8/246: read df/d25/d28/d31/d47/d22/f3d [760776,550] 0 2024-03-24T10:45:54.183 INFO:tasks.workunit.client.0.smithi094.stdout:2/178: truncate f7 2908190 0 2024-03-24T10:45:54.184 INFO:tasks.workunit.client.0.smithi094.stdout:4/271: mknod d0/d5/c56 0 2024-03-24T10:45:54.184 INFO:tasks.workunit.client.0.smithi094.stdout:4/272: chown d0/d5/l3e 1443928041 1 2024-03-24T10:45:54.187 INFO:tasks.workunit.client.0.smithi094.stdout:5/223: mknod d7/d8/d1b/c4e 0 2024-03-24T10:45:54.190 INFO:tasks.workunit.client.0.smithi094.stdout:4/273: read d0/d34/f48 [538357,41337] 0 2024-03-24T10:45:54.190 INFO:tasks.workunit.client.0.smithi094.stdout:4/274: dread - d0/d5/d1b/f4e zero size 2024-03-24T10:45:54.191 INFO:tasks.workunit.client.0.smithi094.stdout:1/188: rmdir d4/d9/db 39 2024-03-24T10:45:54.191 INFO:tasks.workunit.client.0.smithi094.stdout:1/189: read - d4/d9/d37/d33/f3a zero size 2024-03-24T10:45:54.191 INFO:tasks.workunit.client.0.smithi094.stdout:1/190: chown d4/d9/db/d31/d3c 306484484 1 2024-03-24T10:45:54.191 INFO:tasks.workunit.client.0.smithi094.stdout:1/191: truncate d4/d9/de/f3b 591594 0 2024-03-24T10:45:54.191 INFO:tasks.workunit.client.0.smithi094.stdout:1/192: fdatasync d4/d9/f25 0 2024-03-24T10:45:54.191 INFO:tasks.workunit.client.0.smithi094.stdout:5/224: dread d7/d8/fa [0,4194304] 0 2024-03-24T10:45:54.193 INFO:tasks.workunit.client.0.smithi094.stdout:4/275: link d0/d22/l27 d0/d1f/l57 0 2024-03-24T10:45:54.193 INFO:tasks.workunit.client.0.smithi094.stdout:4/276: write d0/d5/d1b/f39 [1034492,114312] 0 2024-03-24T10:45:54.193 INFO:tasks.workunit.client.0.smithi094.stdout:4/277: truncate d0/d5/d1b/f4e 568402 0 2024-03-24T10:45:54.193 INFO:tasks.workunit.client.0.smithi094.stdout:4/278: chown d0/d5/l29 818 1 2024-03-24T10:45:54.193 INFO:tasks.workunit.client.0.smithi094.stdout:4/279: write d0/d5/d1b/f39 [1574138,60060] 0 2024-03-24T10:45:54.194 INFO:tasks.workunit.client.0.smithi094.stdout:8/247: write f8 [3136545,127196] 0 2024-03-24T10:45:54.194 INFO:tasks.workunit.client.0.smithi094.stdout:8/248: stat df/d25/d28/d31/d47/d22 0 2024-03-24T10:45:54.194 INFO:tasks.workunit.client.0.smithi094.stdout:4/280: symlink d0/d22/l58 0 2024-03-24T10:45:54.194 INFO:tasks.workunit.client.0.smithi094.stdout:4/281: creat d0/d22/de/d28/f59 x:0 0 0 2024-03-24T10:45:54.195 INFO:tasks.workunit.client.0.smithi094.stdout:8/249: mknod df/d25/d36/d3f/c50 0 2024-03-24T10:45:54.195 INFO:tasks.workunit.client.0.smithi094.stdout:8/250: chown df/c1e 7 1 2024-03-24T10:45:54.196 INFO:tasks.workunit.client.0.smithi094.stdout:5/225: dread d7/d8/d14/f35 [0,4194304] 0 2024-03-24T10:45:54.196 INFO:tasks.workunit.client.0.smithi094.stdout:5/226: chown d7/d8/d14/d36/d3a/c3f 187415170 1 2024-03-24T10:45:54.197 INFO:tasks.workunit.client.0.smithi094.stdout:4/282: mknod d0/d5/d1b/c5a 0 2024-03-24T10:45:54.198 INFO:tasks.workunit.client.0.smithi094.stdout:8/251: mknod df/d12/c51 0 2024-03-24T10:45:54.198 INFO:tasks.workunit.client.0.smithi094.stdout:8/252: write df/d25/d28/d31/f40 [176397,27192] 0 2024-03-24T10:45:54.198 INFO:tasks.workunit.client.0.smithi094.stdout:8/253: creat df/d12/f52 x:0 0 0 2024-03-24T10:45:54.199 INFO:tasks.workunit.client.0.smithi094.stdout:9/272: getdents df/d2d/d1f/d11/d43 0 2024-03-24T10:45:54.202 INFO:tasks.workunit.client.0.smithi094.stdout:8/254: symlink df/d12/l53 0 2024-03-24T10:45:54.204 INFO:tasks.workunit.client.0.smithi094.stdout:9/273: mkdir df/d2d/d1f/d11/d53 0 2024-03-24T10:45:54.207 INFO:tasks.workunit.client.0.smithi094.stdout:3/182: sync 2024-03-24T10:45:54.210 INFO:tasks.workunit.client.0.smithi094.stdout:4/283: dread d0/f47 [0,4194304] 0 2024-03-24T10:45:54.210 INFO:tasks.workunit.client.0.smithi094.stdout:4/284: stat d0/d5/d1b/d4f 0 2024-03-24T10:45:54.211 INFO:tasks.workunit.client.0.smithi094.stdout:9/274: dread df/d2d/f1e [0,4194304] 0 2024-03-24T10:45:54.212 INFO:tasks.workunit.client.0.smithi094.stdout:4/285: rename d0/l20 to d0/d34/l5b 0 2024-03-24T10:45:54.213 INFO:tasks.workunit.client.0.smithi094.stdout:1/193: dwrite d4/f2a [0,4194304] 0 2024-03-24T10:45:54.213 INFO:tasks.workunit.client.0.smithi094.stdout:1/194: truncate d4/d9/db/f18 4625442 0 2024-03-24T10:45:54.214 INFO:tasks.workunit.client.0.smithi094.stdout:5/227: dwrite d7/d8/d1b/f37 [0,4194304] 0 2024-03-24T10:45:54.214 INFO:tasks.workunit.client.0.smithi094.stdout:5/228: fdatasync d7/f34 0 2024-03-24T10:45:54.214 INFO:tasks.workunit.client.0.smithi094.stdout:9/275: truncate df/f22 1008199 0 2024-03-24T10:45:54.218 INFO:tasks.workunit.client.0.smithi094.stdout:1/195: rename d4/d9/db/c24 to d4/c3e 0 2024-03-24T10:45:54.220 INFO:tasks.workunit.client.0.smithi094.stdout:9/276: symlink df/d2d/l54 0 2024-03-24T10:45:54.220 INFO:tasks.workunit.client.0.smithi094.stdout:9/277: chown df/d2d/d1f 6816 1 2024-03-24T10:45:54.220 INFO:tasks.workunit.client.0.smithi094.stdout:9/278: readlink df/d2d/d1f/l37 0 2024-03-24T10:45:54.220 INFO:tasks.workunit.client.0.smithi094.stdout:9/279: fsync df/d2d/f1e 0 2024-03-24T10:45:54.223 INFO:tasks.workunit.client.0.smithi094.stdout:1/196: creat d4/d36/f3f x:0 0 0 2024-03-24T10:45:54.223 INFO:tasks.workunit.client.0.smithi094.stdout:1/197: chown d4/f1e 159 1 2024-03-24T10:45:54.226 INFO:tasks.workunit.client.0.smithi094.stdout:1/198: link d4/d9/db/l34 d4/l40 0 2024-03-24T10:45:54.228 INFO:tasks.workunit.client.0.smithi094.stdout:4/286: dread d0/d22/de/d28/d2b/f51 [0,4194304] 0 2024-03-24T10:45:54.231 INFO:tasks.workunit.client.0.smithi094.stdout:1/199: dread d4/d9/fa [4194304,4194304] 0 2024-03-24T10:45:54.231 INFO:tasks.workunit.client.0.smithi094.stdout:1/200: creat d4/f41 x:0 0 0 2024-03-24T10:45:54.231 INFO:tasks.workunit.client.0.smithi094.stdout:1/201: dread - d4/d9/db/f30 zero size 2024-03-24T10:45:54.231 INFO:tasks.workunit.client.0.smithi094.stdout:1/202: fsync d4/d9/db/f10 0 2024-03-24T10:45:54.231 INFO:tasks.workunit.client.0.smithi094.stdout:1/203: stat d4/d9/d37 0 2024-03-24T10:45:54.238 INFO:tasks.workunit.client.0.smithi094.stdout:1/204: dread d4/d9/db/f21 [0,4194304] 0 2024-03-24T10:45:54.238 INFO:tasks.workunit.client.0.smithi094.stdout:1/205: chown d4/d9/db/l34 106 1 2024-03-24T10:45:54.239 INFO:tasks.workunit.client.0.smithi094.stdout:1/206: creat d4/d9/db/f42 x:0 0 0 2024-03-24T10:45:54.239 INFO:tasks.workunit.client.0.smithi094.stdout:1/207: dread - d4/d9/f27 zero size 2024-03-24T10:45:54.239 INFO:tasks.workunit.client.0.smithi094.stdout:1/208: truncate d4/f41 828163 0 2024-03-24T10:45:54.239 INFO:tasks.workunit.client.0.smithi094.stdout:1/209: write d4/d9/fa [1508202,16878] 0 2024-03-24T10:45:54.240 INFO:tasks.workunit.client.0.smithi094.stdout:1/210: mkdir d4/d9/db/d39/d43 0 2024-03-24T10:45:54.240 INFO:tasks.workunit.client.0.smithi094.stdout:1/211: creat d4/d36/f44 x:0 0 0 2024-03-24T10:45:54.241 INFO:tasks.workunit.client.0.smithi094.stdout:1/212: mknod d4/d36/d38/c45 0 2024-03-24T10:45:54.241 INFO:tasks.workunit.client.0.smithi094.stdout:1/213: creat d4/f46 x:0 0 0 2024-03-24T10:45:54.244 INFO:tasks.workunit.client.0.smithi094.stdout:4/287: dwrite d0/f4b [0,4194304] 0 2024-03-24T10:45:54.244 INFO:tasks.workunit.client.0.smithi094.stdout:4/288: chown d0/c8 3085 1 2024-03-24T10:45:54.244 INFO:tasks.workunit.client.0.smithi094.stdout:8/255: dwrite fa [0,4194304] 0 2024-03-24T10:45:54.244 INFO:tasks.workunit.client.0.smithi094.stdout:8/256: chown df/l1c 1213192 1 2024-03-24T10:45:54.245 INFO:tasks.workunit.client.0.smithi094.stdout:8/257: creat df/d25/d36/d3f/f54 x:0 0 0 2024-03-24T10:45:54.245 INFO:tasks.workunit.client.0.smithi094.stdout:8/258: stat df/d25/f48 0 2024-03-24T10:45:54.246 INFO:tasks.workunit.client.0.smithi094.stdout:4/289: creat d0/d22/de/d28/d3d/f5c x:0 0 0 2024-03-24T10:45:54.246 INFO:tasks.workunit.client.0.smithi094.stdout:4/290: write d0/d5/d1b/f39 [1957236,87953] 0 2024-03-24T10:45:54.251 INFO:tasks.workunit.client.0.smithi094.stdout:3/183: dwrite d0/db/d1c/d20/f24 [0,4194304] 0 2024-03-24T10:45:54.251 INFO:tasks.workunit.client.0.smithi094.stdout:3/184: chown d0/db/ld 4118747 1 2024-03-24T10:45:54.251 INFO:tasks.workunit.client.0.smithi094.stdout:5/229: dwrite d7/d8/f16 [0,4194304] 0 2024-03-24T10:45:54.251 INFO:tasks.workunit.client.0.smithi094.stdout:5/230: truncate d7/d8/d14/d36/f4d 595994 0 2024-03-24T10:45:54.251 INFO:tasks.workunit.client.0.smithi094.stdout:5/231: write d7/d8/d14/f2a [503982,88871] 0 2024-03-24T10:45:54.251 INFO:tasks.workunit.client.0.smithi094.stdout:5/232: truncate d7/d23/d2b/f4b 44891 0 2024-03-24T10:45:54.251 INFO:tasks.workunit.client.0.smithi094.stdout:5/233: creat d7/d8/d9/f4f x:0 0 0 2024-03-24T10:45:54.251 INFO:tasks.workunit.client.0.smithi094.stdout:1/214: fdatasync d4/d9/db/f1f 0 2024-03-24T10:45:54.251 INFO:tasks.workunit.client.0.smithi094.stdout:9/280: dwrite df/d2d/d34/f2e [0,4194304] 0 2024-03-24T10:45:54.251 INFO:tasks.workunit.client.0.smithi094.stdout:4/291: read d0/d5/f16 [1960148,64316] 0 2024-03-24T10:45:54.256 INFO:tasks.workunit.client.0.smithi094.stdout:8/259: dread df/d25/f29 [0,4194304] 0 2024-03-24T10:45:54.262 INFO:tasks.workunit.client.0.smithi094.stdout:3/185: unlink d0/l30 0 2024-03-24T10:45:54.262 INFO:tasks.workunit.client.0.smithi094.stdout:3/186: readlink d0/db/ld 0 2024-03-24T10:45:54.265 INFO:tasks.workunit.client.0.smithi094.stdout:5/234: rename d7/d8/d14/f2a to d7/d8/d9/d48/f50 0 2024-03-24T10:45:54.268 INFO:tasks.workunit.client.0.smithi094.stdout:4/292: mkdir d0/d22/de/d28/d2b/d33/d5d 0 2024-03-24T10:45:54.272 INFO:tasks.workunit.client.0.smithi094.stdout:5/235: dread d7/d8/d1b/f37 [0,4194304] 0 2024-03-24T10:45:54.272 INFO:tasks.workunit.client.0.smithi094.stdout:5/236: truncate d7/dc/d12/f1f 4540430 0 2024-03-24T10:45:54.272 INFO:tasks.workunit.client.0.smithi094.stdout:9/281: dwrite df/f36 [0,4194304] 0 2024-03-24T10:45:54.272 INFO:tasks.workunit.client.0.smithi094.stdout:8/260: mkdir df/d25/d28/d31/d47/d22/d55 0 2024-03-24T10:45:54.273 INFO:tasks.workunit.client.0.smithi094.stdout:8/261: write df/d25/f2d [615358,11486] 0 2024-03-24T10:45:54.274 INFO:tasks.workunit.client.0.smithi094.stdout:3/187: mknod d0/db/d1c/d20/d23/d2a/d3a/c44 0 2024-03-24T10:45:54.275 INFO:tasks.workunit.client.0.smithi094.stdout:4/293: creat d0/f5e x:0 0 0 2024-03-24T10:45:54.276 INFO:tasks.workunit.client.0.smithi094.stdout:4/294: write d0/d22/de/d28/d2b/f43 [372721,100396] 0 2024-03-24T10:45:54.276 INFO:tasks.workunit.client.0.smithi094.stdout:5/237: mkdir d7/d8/d9/d48/d51 0 2024-03-24T10:45:54.279 INFO:tasks.workunit.client.0.smithi094.stdout:8/262: dread f3 [0,4194304] 0 2024-03-24T10:45:54.279 INFO:tasks.workunit.client.0.smithi094.stdout:8/263: dread - df/d25/d36/d3f/f54 zero size 2024-03-24T10:45:54.279 INFO:tasks.workunit.client.0.smithi094.stdout:8/264: chown df/d12/l37 282087 1 2024-03-24T10:45:54.281 INFO:tasks.workunit.client.0.smithi094.stdout:9/282: rename df/d2d/d1f/l37 to df/d2d/l55 0 2024-03-24T10:45:54.283 INFO:tasks.workunit.client.0.smithi094.stdout:4/295: creat d0/f5f x:0 0 0 2024-03-24T10:45:54.283 INFO:tasks.workunit.client.0.smithi094.stdout:4/296: write d0/d22/de/d28/f52 [4230688,92138] 0 2024-03-24T10:45:54.283 INFO:tasks.workunit.client.0.smithi094.stdout:4/297: chown d0/f5f 7925 1 2024-03-24T10:45:54.283 INFO:tasks.workunit.client.0.smithi094.stdout:4/298: dread - d0/d22/de/d28/f3f zero size 2024-03-24T10:45:54.284 INFO:tasks.workunit.client.0.smithi094.stdout:5/238: creat d7/d8/d9/d48/d51/f52 x:0 0 0 2024-03-24T10:45:54.284 INFO:tasks.workunit.client.0.smithi094.stdout:5/239: truncate d7/d23/f44 181841 0 2024-03-24T10:45:54.284 INFO:tasks.workunit.client.0.smithi094.stdout:8/265: creat df/f56 x:0 0 0 2024-03-24T10:45:54.285 INFO:tasks.workunit.client.0.smithi094.stdout:9/283: link df/f36 df/f56 0 2024-03-24T10:45:54.286 INFO:tasks.workunit.client.0.smithi094.stdout:3/188: mkdir d0/db/dc/d45 0 2024-03-24T10:45:54.287 INFO:tasks.workunit.client.0.smithi094.stdout:5/240: write d7/d8/fa [4112666,102735] 0 2024-03-24T10:45:54.287 INFO:tasks.workunit.client.0.smithi094.stdout:5/241: write d7/d8/f32 [76074,68014] 0 2024-03-24T10:45:54.287 INFO:tasks.workunit.client.0.smithi094.stdout:5/242: write d7/d8/d1b/f42 [924747,37830] 0 2024-03-24T10:45:54.287 INFO:tasks.workunit.client.0.smithi094.stdout:5/243: chown d7/d23/d2b/l45 401282275 1 2024-03-24T10:45:54.287 INFO:tasks.workunit.client.0.smithi094.stdout:5/244: fsync d7/dc/f19 0 2024-03-24T10:45:54.288 INFO:tasks.workunit.client.0.smithi094.stdout:4/299: creat d0/f60 x:0 0 0 2024-03-24T10:45:54.288 INFO:tasks.workunit.client.0.smithi094.stdout:4/300: chown d0/d5/d1b/f2a 337183 1 2024-03-24T10:45:54.288 INFO:tasks.workunit.client.0.smithi094.stdout:4/301: chown d0/l14 838415 1 2024-03-24T10:45:54.288 INFO:tasks.workunit.client.0.smithi094.stdout:8/266: symlink df/d25/d28/d31/d47/d22/l57 0 2024-03-24T10:45:54.290 INFO:tasks.workunit.client.0.smithi094.stdout:4/302: rename d0/d22/de/d28/d2b/d33/f40 to d0/d5/d42/d4a/f61 0 2024-03-24T10:45:54.290 INFO:tasks.workunit.client.0.smithi094.stdout:4/303: readlink d0/d22/l53 0 2024-03-24T10:45:54.290 INFO:tasks.workunit.client.0.smithi094.stdout:4/304: fdatasync d0/f6 0 2024-03-24T10:45:54.293 INFO:tasks.workunit.client.0.smithi094.stdout:8/267: getdents df/d25/d28/d31 0 2024-03-24T10:45:54.294 INFO:tasks.workunit.client.0.smithi094.stdout:3/189: dread d0/f11 [0,4194304] 0 2024-03-24T10:45:54.294 INFO:tasks.workunit.client.0.smithi094.stdout:3/190: fsync d0/f14 0 2024-03-24T10:45:54.294 INFO:tasks.workunit.client.0.smithi094.stdout:3/191: write d0/f1e [1259486,130443] 0 2024-03-24T10:45:54.294 INFO:tasks.workunit.client.0.smithi094.stdout:3/192: fsync d0/db/f2b 0 2024-03-24T10:45:54.294 INFO:tasks.workunit.client.0.smithi094.stdout:3/193: stat d0/db/dc/d28/l3e 0 2024-03-24T10:45:54.295 INFO:tasks.workunit.client.0.smithi094.stdout:3/194: stat d0/db/d1c/d20 0 2024-03-24T10:45:54.298 INFO:tasks.workunit.client.0.smithi094.stdout:0/251: sync 2024-03-24T10:45:54.298 INFO:tasks.workunit.client.0.smithi094.stdout:5/245: dread d7/d8/d9/f29 [0,4194304] 0 2024-03-24T10:45:54.301 INFO:tasks.workunit.client.0.smithi094.stdout:9/284: dwrite df/d2d/d34/d2c/d3a/f4b [0,4194304] 0 2024-03-24T10:45:54.301 INFO:tasks.workunit.client.0.smithi094.stdout:5/246: dread d7/d23/f44 [0,4194304] 0 2024-03-24T10:45:54.303 INFO:tasks.workunit.client.0.smithi094.stdout:0/252: mknod d14/d18/c56 0 2024-03-24T10:45:54.303 INFO:tasks.workunit.client.0.smithi094.stdout:0/253: readlink d14/d22/d28/d2b/l4d 0 2024-03-24T10:45:54.306 INFO:tasks.workunit.client.0.smithi094.stdout:6/230: sync 2024-03-24T10:45:54.306 INFO:tasks.workunit.client.0.smithi094.stdout:7/261: sync 2024-03-24T10:45:54.306 INFO:tasks.workunit.client.0.smithi094.stdout:7/262: chown d1/d9/d14/d34 65 1 2024-03-24T10:45:54.308 INFO:tasks.workunit.client.0.smithi094.stdout:5/247: creat d7/d26/f53 x:0 0 0 2024-03-24T10:45:54.309 INFO:tasks.workunit.client.0.smithi094.stdout:0/254: rename d14/d22/d32/l3a to d14/d22/l57 0 2024-03-24T10:45:54.311 INFO:tasks.workunit.client.0.smithi094.stdout:6/231: symlink d0/d5/da/d23/d34/l45 0 2024-03-24T10:45:54.313 INFO:tasks.workunit.client.0.smithi094.stdout:6/232: truncate d0/f2 2329964 0 2024-03-24T10:45:54.313 INFO:tasks.workunit.client.0.smithi094.stdout:6/233: creat d0/d21/d2a/d2f/f46 x:0 0 0 2024-03-24T10:45:54.318 INFO:tasks.workunit.client.0.smithi094.stdout:8/268: rmdir df 39 2024-03-24T10:45:54.320 INFO:tasks.workunit.client.0.smithi094.stdout:8/269: creat df/d25/f58 x:0 0 0 2024-03-24T10:45:54.321 INFO:tasks.workunit.client.0.smithi094.stdout:5/248: dread d7/dc/d12/f1f [0,4194304] 0 2024-03-24T10:45:54.321 INFO:tasks.workunit.client.0.smithi094.stdout:5/249: chown d7/d8/fa 256578379 1 2024-03-24T10:45:54.321 INFO:tasks.workunit.client.0.smithi094.stdout:5/250: fdatasync d7/d8/f28 0 2024-03-24T10:45:54.321 INFO:tasks.workunit.client.0.smithi094.stdout:5/251: write d7/d8/d14/d36/f47 [202315,44815] 0 2024-03-24T10:45:54.322 INFO:tasks.workunit.client.0.smithi094.stdout:5/252: unlink d7/d8/f32 0 2024-03-24T10:45:54.322 INFO:tasks.workunit.client.0.smithi094.stdout:5/253: creat d7/d8/d9/f54 x:0 0 0 2024-03-24T10:45:54.325 INFO:tasks.workunit.client.0.smithi094.stdout:4/305: dwrite d0/d5/d1b/f39 [0,4194304] 0 2024-03-24T10:45:54.326 INFO:tasks.workunit.client.0.smithi094.stdout:8/270: dread fa [0,4194304] 0 2024-03-24T10:45:54.328 INFO:tasks.workunit.client.0.smithi094.stdout:4/306: creat d0/d22/f62 x:0 0 0 2024-03-24T10:45:54.328 INFO:tasks.workunit.client.0.smithi094.stdout:7/263: dwrite d1/d9/d25/d3d/f46 [0,4194304] 0 2024-03-24T10:45:54.329 INFO:tasks.workunit.client.0.smithi094.stdout:9/285: dwrite df/d2d/d34/d2c/f4c [0,4194304] 0 2024-03-24T10:45:54.329 INFO:tasks.workunit.client.0.smithi094.stdout:3/195: dwrite d0/f2 [0,4194304] 0 2024-03-24T10:45:54.329 INFO:tasks.workunit.client.0.smithi094.stdout:5/254: fdatasync d7/d8/d1b/f42 0 2024-03-24T10:45:54.329 INFO:tasks.workunit.client.0.smithi094.stdout:8/271: rmdir df 39 2024-03-24T10:45:54.329 INFO:tasks.workunit.client.0.smithi094.stdout:8/272: truncate df/d25/d36/d3f/f4f 1005484 0 2024-03-24T10:45:54.329 INFO:tasks.workunit.client.0.smithi094.stdout:8/273: truncate df/d12/f52 705033 0 2024-03-24T10:45:54.332 INFO:tasks.workunit.client.0.smithi094.stdout:4/307: dread d0/d22/de/f30 [0,4194304] 0 2024-03-24T10:45:54.333 INFO:tasks.workunit.client.0.smithi094.stdout:0/255: rename d14/d22/l57 to d14/d22/d28/d2b/d41/d53/l58 0 2024-03-24T10:45:54.335 INFO:tasks.workunit.client.0.smithi094.stdout:3/196: symlink d0/db/dc/d45/l46 0 2024-03-24T10:45:54.338 INFO:tasks.workunit.client.0.smithi094.stdout:9/286: dread df/d2d/d34/d2c/d3a/f4b [0,4194304] 0 2024-03-24T10:45:54.338 INFO:tasks.workunit.client.0.smithi094.stdout:9/287: dread - df/d2d/d1f/f4a zero size 2024-03-24T10:45:54.338 INFO:tasks.workunit.client.0.smithi094.stdout:7/264: dread d1/d9/d25/d3d/f46 [0,4194304] 0 2024-03-24T10:45:54.338 INFO:tasks.workunit.client.0.smithi094.stdout:7/265: chown d1/d9/d31/d39/d44/f4e 39 1 2024-03-24T10:45:54.338 INFO:tasks.workunit.client.0.smithi094.stdout:7/266: fdatasync d1/d9/f13 0 2024-03-24T10:45:54.338 INFO:tasks.workunit.client.0.smithi094.stdout:6/234: rmdir d0/d21/d2a/d2f 39 2024-03-24T10:45:54.339 INFO:tasks.workunit.client.0.smithi094.stdout:6/235: truncate d0/d5/f19 1173867 0 2024-03-24T10:45:54.342 INFO:tasks.workunit.client.0.smithi094.stdout:3/197: write d0/db/d1c/d20/d23/d2a/d3a/ff [444821,39630] 0 2024-03-24T10:45:54.346 INFO:tasks.workunit.client.0.smithi094.stdout:8/274: creat df/d25/d36/d3f/f59 x:0 0 0 2024-03-24T10:45:54.346 INFO:tasks.workunit.client.0.smithi094.stdout:8/275: truncate df/d25/d28/d31/d47/d22/f38 391542 0 2024-03-24T10:45:54.348 INFO:tasks.workunit.client.0.smithi094.stdout:0/256: symlink d14/d22/l59 0 2024-03-24T10:45:54.348 INFO:tasks.workunit.client.0.smithi094.stdout:0/257: chown d14 65 1 2024-03-24T10:45:54.349 INFO:tasks.workunit.client.0.smithi094.stdout:0/258: write d14/d18/f40 [1290886,33193] 0 2024-03-24T10:45:54.349 INFO:tasks.workunit.client.0.smithi094.stdout:0/259: truncate d14/d18/f4f 724551 0 2024-03-24T10:45:54.349 INFO:tasks.workunit.client.0.smithi094.stdout:0/260: fsync d14/d22/d28/d2b/f38 0 2024-03-24T10:45:54.349 INFO:tasks.workunit.client.0.smithi094.stdout:0/261: dread - d14/d18/f54 zero size 2024-03-24T10:45:54.349 INFO:tasks.workunit.client.0.smithi094.stdout:7/267: rename d1/d9/d31/d39/d44/c4f to d1/c58 0 2024-03-24T10:45:54.350 INFO:tasks.workunit.client.0.smithi094.stdout:3/198: mknod d0/db/dc/d28/c47 0 2024-03-24T10:45:54.352 INFO:tasks.workunit.client.0.smithi094.stdout:8/276: creat df/d25/d28/d31/d47/f5a x:0 0 0 2024-03-24T10:45:54.354 INFO:tasks.workunit.client.0.smithi094.stdout:7/268: dread d1/d9/d14/f4c [0,4194304] 0 2024-03-24T10:45:54.355 INFO:tasks.workunit.client.0.smithi094.stdout:0/262: creat d14/d22/d28/d2b/d41/d53/f5a x:0 0 0 2024-03-24T10:45:54.356 INFO:tasks.workunit.client.0.smithi094.stdout:3/199: chown d0/db/dc/c22 196 1 2024-03-24T10:45:54.356 INFO:tasks.workunit.client.0.smithi094.stdout:3/200: creat d0/db/f48 x:0 0 0 2024-03-24T10:45:54.356 INFO:tasks.workunit.client.0.smithi094.stdout:3/201: read d0/db/dc/f12 [414959,22403] 0 2024-03-24T10:45:54.357 INFO:tasks.workunit.client.0.smithi094.stdout:8/277: mkdir df/d12/d5b 0 2024-03-24T10:45:54.359 INFO:tasks.workunit.client.0.smithi094.stdout:4/308: dwrite d0/d5/d1b/f4e [0,4194304] 0 2024-03-24T10:45:54.359 INFO:tasks.workunit.client.0.smithi094.stdout:7/269: truncate d1/d9/fe 1128047 0 2024-03-24T10:45:54.359 INFO:tasks.workunit.client.0.smithi094.stdout:4/309: dread - d0/f60 zero size 2024-03-24T10:45:54.359 INFO:tasks.workunit.client.0.smithi094.stdout:7/270: chown d1/d9/d31/d39/f48 170 1 2024-03-24T10:45:54.359 INFO:tasks.workunit.client.0.smithi094.stdout:7/271: write d1/d9/f30 [1964824,35175] 0 2024-03-24T10:45:54.359 INFO:tasks.workunit.client.0.smithi094.stdout:9/288: dwrite df/d2d/d1f/f49 [0,4194304] 0 2024-03-24T10:45:54.360 INFO:tasks.workunit.client.0.smithi094.stdout:9/289: stat df/d2d/d1f 0 2024-03-24T10:45:54.360 INFO:tasks.workunit.client.0.smithi094.stdout:5/255: truncate d7/d8/fa 1458798 0 2024-03-24T10:45:54.360 INFO:tasks.workunit.client.0.smithi094.stdout:5/256: readlink d7/d26/l22 0 2024-03-24T10:45:54.363 INFO:tasks.workunit.client.0.smithi094.stdout:8/278: mknod df/d12/d5b/c5c 0 2024-03-24T10:45:54.363 INFO:tasks.workunit.client.0.smithi094.stdout:8/279: rename df/d25/d28/d31/d47/d22/d55 to df/d25/d28/d31/d47/d22/d55/d5d 22 2024-03-24T10:45:54.363 INFO:tasks.workunit.client.0.smithi094.stdout:2/179: sync 2024-03-24T10:45:54.363 INFO:tasks.workunit.client.0.smithi094.stdout:2/180: readlink d18/d1e/l20 0 2024-03-24T10:45:54.366 INFO:tasks.workunit.client.0.smithi094.stdout:7/272: getdents d1/d9/d14 0 2024-03-24T10:45:54.366 INFO:tasks.workunit.client.0.smithi094.stdout:7/273: fdatasync d1/d9/d14/f4b 0 2024-03-24T10:45:54.366 INFO:tasks.workunit.client.0.smithi094.stdout:7/274: readlink d1/d9/lb 0 2024-03-24T10:45:54.370 INFO:tasks.workunit.client.0.smithi094.stdout:8/280: write df/f1b [2048579,114527] 0 2024-03-24T10:45:54.376 INFO:tasks.workunit.client.0.smithi094.stdout:0/263: dwrite f9 [0,4194304] 0 2024-03-24T10:45:54.376 INFO:tasks.workunit.client.0.smithi094.stdout:0/264: getdents d14/d22/d28/d48 0 2024-03-24T10:45:54.377 INFO:tasks.workunit.client.0.smithi094.stdout:9/290: link df/d2d/d34/d2c/d3a/f4b df/d2d/d34/d2c/f57 0 2024-03-24T10:45:54.378 INFO:tasks.workunit.client.0.smithi094.stdout:9/291: truncate df/d2d/d1f/f4a 201598 0 2024-03-24T10:45:54.378 INFO:tasks.workunit.client.0.smithi094.stdout:8/281: dread df/d25/d28/d31/f40 [0,4194304] 0 2024-03-24T10:45:54.378 INFO:tasks.workunit.client.0.smithi094.stdout:5/257: mkdir d7/d26/d55 0 2024-03-24T10:45:54.378 INFO:tasks.workunit.client.0.smithi094.stdout:5/258: readlink d7/dc/d12/l18 0 2024-03-24T10:45:54.378 INFO:tasks.workunit.client.0.smithi094.stdout:5/259: write d7/d8/d14/d36/f47 [572387,55514] 0 2024-03-24T10:45:54.378 INFO:tasks.workunit.client.0.smithi094.stdout:2/181: mkdir d18/d1e/d34 0 2024-03-24T10:45:54.379 INFO:tasks.workunit.client.0.smithi094.stdout:2/182: chown d18 106696 1 2024-03-24T10:45:54.379 INFO:tasks.workunit.client.0.smithi094.stdout:4/310: mknod d0/c63 0 2024-03-24T10:45:54.379 INFO:tasks.workunit.client.0.smithi094.stdout:4/311: chown d0/d5/l19 1464 1 2024-03-24T10:45:54.379 INFO:tasks.workunit.client.0.smithi094.stdout:4/312: stat d0/f5e 0 2024-03-24T10:45:54.379 INFO:tasks.workunit.client.0.smithi094.stdout:4/313: chown d0/d22/de/d28/d2b/f51 121 1 2024-03-24T10:45:54.381 INFO:tasks.workunit.client.0.smithi094.stdout:8/282: read df/d12/f21 [181148,12045] 0 2024-03-24T10:45:54.382 INFO:tasks.workunit.client.0.smithi094.stdout:8/283: write df/d12/f2f [766685,55599] 0 2024-03-24T10:45:54.383 INFO:tasks.workunit.client.0.smithi094.stdout:2/183: write f7 [2093981,61088] 0 2024-03-24T10:45:54.383 INFO:tasks.workunit.client.0.smithi094.stdout:0/265: mkdir d14/d22/d5b 0 2024-03-24T10:45:54.384 INFO:tasks.workunit.client.0.smithi094.stdout:9/292: truncate df/d2d/d1f/d11/f4f 830103 0 2024-03-24T10:45:54.385 INFO:tasks.workunit.client.0.smithi094.stdout:3/202: dwrite d0/db/f2b [0,4194304] 0 2024-03-24T10:45:54.386 INFO:tasks.workunit.client.0.smithi094.stdout:3/203: truncate d0/db/d1c/d20/d23/d2a/f33 233816 0 2024-03-24T10:45:54.386 INFO:tasks.workunit.client.0.smithi094.stdout:3/204: chown d0/db/d1c/d20/d23/d2a/d3a/ff 3998409 1 2024-03-24T10:45:54.386 INFO:tasks.workunit.client.0.smithi094.stdout:7/275: dread d1/f10 [0,4194304] 0 2024-03-24T10:45:54.388 INFO:tasks.workunit.client.0.smithi094.stdout:5/260: mkdir d7/dc/d56 0 2024-03-24T10:45:54.390 INFO:tasks.workunit.client.0.smithi094.stdout:4/314: unlink d0/c8 0 2024-03-24T10:45:54.390 INFO:tasks.workunit.client.0.smithi094.stdout:4/315: truncate d0/d5/d1b/f2a 701482 0 2024-03-24T10:45:54.392 INFO:tasks.workunit.client.0.smithi094.stdout:8/284: symlink df/d12/d5b/l5e 0 2024-03-24T10:45:54.393 INFO:tasks.workunit.client.0.smithi094.stdout:8/285: write fc [2147815,45411] 0 2024-03-24T10:45:54.393 INFO:tasks.workunit.client.0.smithi094.stdout:9/293: mkdir df/d2d/d1f/d58 0 2024-03-24T10:45:54.399 INFO:tasks.workunit.client.0.smithi094.stdout:1/215: sync 2024-03-24T10:45:54.399 INFO:tasks.workunit.client.0.smithi094.stdout:1/216: fdatasync d4/d9/db/f21 0 2024-03-24T10:45:54.399 INFO:tasks.workunit.client.0.smithi094.stdout:5/261: mkdir d7/d8/d14/d36/d3a/d57 0 2024-03-24T10:45:54.401 INFO:tasks.workunit.client.0.smithi094.stdout:4/316: unlink d0/d5/l3e 0 2024-03-24T10:45:54.401 INFO:tasks.workunit.client.0.smithi094.stdout:4/317: readlink d0/d34/l5b 0 2024-03-24T10:45:54.403 INFO:tasks.workunit.client.0.smithi094.stdout:8/286: rename fc to df/d25/d28/d31/d3c/f5f 0 2024-03-24T10:45:54.403 INFO:tasks.workunit.client.0.smithi094.stdout:8/287: readlink df/d25/d28/l4b 0 2024-03-24T10:45:54.404 INFO:tasks.workunit.client.0.smithi094.stdout:8/288: fsync df/d25/d28/d31/f32 0 2024-03-24T10:45:54.404 INFO:tasks.workunit.client.0.smithi094.stdout:8/289: read f4 [502593,89849] 0 2024-03-24T10:45:54.404 INFO:tasks.workunit.client.0.smithi094.stdout:9/294: rmdir df/d2d/d1f/d11 39 2024-03-24T10:45:54.404 INFO:tasks.workunit.client.0.smithi094.stdout:9/295: read df/d2d/f44 [127322,31762] 0 2024-03-24T10:45:54.404 INFO:tasks.workunit.client.0.smithi094.stdout:9/296: chown df/d2d/d34/d2c/d3a 390 1 2024-03-24T10:45:54.408 INFO:tasks.workunit.client.0.smithi094.stdout:3/205: rmdir d0/db 39 2024-03-24T10:45:54.409 INFO:tasks.workunit.client.0.smithi094.stdout:2/184: dwrite d18/d1e/f33 [0,4194304] 0 2024-03-24T10:45:54.409 INFO:tasks.workunit.client.0.smithi094.stdout:7/276: dwrite d1/d9/f23 [0,4194304] 0 2024-03-24T10:45:54.409 INFO:tasks.workunit.client.0.smithi094.stdout:7/277: readlink d1/d9/d14/l4a 0 2024-03-24T10:45:54.410 INFO:tasks.workunit.client.0.smithi094.stdout:1/217: mkdir d4/d9/d47 0 2024-03-24T10:45:54.410 INFO:tasks.workunit.client.0.smithi094.stdout:1/218: chown d4/f2a 45346 1 2024-03-24T10:45:54.410 INFO:tasks.workunit.client.0.smithi094.stdout:1/219: chown d4/d9/db/d31/d3c 726190574 1 2024-03-24T10:45:54.414 INFO:tasks.workunit.client.0.smithi094.stdout:4/318: symlink d0/d34/l64 0 2024-03-24T10:45:54.414 INFO:tasks.workunit.client.0.smithi094.stdout:4/319: dread - d0/d22/de/d28/f3f zero size 2024-03-24T10:45:54.414 INFO:tasks.workunit.client.0.smithi094.stdout:3/206: stat d0/db/dc/c2c 0 2024-03-24T10:45:54.416 INFO:tasks.workunit.client.0.smithi094.stdout:9/297: dread df/f14 [0,4194304] 0 2024-03-24T10:45:54.417 INFO:tasks.workunit.client.0.smithi094.stdout:1/220: dread d4/f1e [0,4194304] 0 2024-03-24T10:45:54.418 INFO:tasks.workunit.client.0.smithi094.stdout:1/221: write d4/f16 [2250758,23244] 0 2024-03-24T10:45:54.418 INFO:tasks.workunit.client.0.smithi094.stdout:2/185: creat d18/d1e/d34/f35 x:0 0 0 2024-03-24T10:45:54.419 INFO:tasks.workunit.client.0.smithi094.stdout:7/278: mkdir d1/d59 0 2024-03-24T10:45:54.420 INFO:tasks.workunit.client.0.smithi094.stdout:2/186: dread f8 [0,4194304] 0 2024-03-24T10:45:54.420 INFO:tasks.workunit.client.0.smithi094.stdout:2/187: stat d18/d1e/l20 0 2024-03-24T10:45:54.424 INFO:tasks.workunit.client.0.smithi094.stdout:8/290: dwrite df/d25/d28/d31/f32 [0,4194304] 0 2024-03-24T10:45:54.425 INFO:tasks.workunit.client.0.smithi094.stdout:8/291: truncate df/d25/d28/d31/d47/f5a 878117 0 2024-03-24T10:45:54.430 INFO:tasks.workunit.client.0.smithi094.stdout:5/262: rmdir d7/d8 39 2024-03-24T10:45:54.432 INFO:tasks.workunit.client.0.smithi094.stdout:3/207: rename d0/db/c36 to d0/db/d1c/d20/d23/d2a/d3a/d25/c49 0 2024-03-24T10:45:54.433 INFO:tasks.workunit.client.0.smithi094.stdout:1/222: dwrite d4/d9/f32 [0,4194304] 0 2024-03-24T10:45:54.434 INFO:tasks.workunit.client.0.smithi094.stdout:7/279: link d1/d9/d16/f2f d1/d9/d25/d3d/f5a 0 2024-03-24T10:45:54.434 INFO:tasks.workunit.client.0.smithi094.stdout:7/280: chown d1/d9/d31/d39/d44/f56 4 1 2024-03-24T10:45:54.434 INFO:tasks.workunit.client.0.smithi094.stdout:7/281: chown d1/d9/d16/l1a 1946692 1 2024-03-24T10:45:54.434 INFO:tasks.workunit.client.0.smithi094.stdout:2/188: symlink d18/l36 0 2024-03-24T10:45:54.439 INFO:tasks.workunit.client.0.smithi094.stdout:5/263: creat d7/d8/f58 x:0 0 0 2024-03-24T10:45:54.440 INFO:tasks.workunit.client.0.smithi094.stdout:4/320: symlink d0/l65 0 2024-03-24T10:45:54.440 INFO:tasks.workunit.client.0.smithi094.stdout:4/321: truncate d0/d22/de/d28/f3f 142457 0 2024-03-24T10:45:54.442 INFO:tasks.workunit.client.0.smithi094.stdout:3/208: rename d0/db/ld to d0/db/d1c/d20/d23/d2a/l4a 0 2024-03-24T10:45:54.447 INFO:tasks.workunit.client.0.smithi094.stdout:1/223: creat d4/d9/db/d31/f48 x:0 0 0 2024-03-24T10:45:54.447 INFO:tasks.workunit.client.0.smithi094.stdout:2/189: truncate d18/d1e/f33 1853268 0 2024-03-24T10:45:54.448 INFO:tasks.workunit.client.0.smithi094.stdout:2/190: read d18/d1e/f24 [144350,100338] 0 2024-03-24T10:45:54.449 INFO:tasks.workunit.client.0.smithi094.stdout:8/292: truncate df/d25/d28/d31/d3c/f5f 1001631 0 2024-03-24T10:45:54.449 INFO:tasks.workunit.client.0.smithi094.stdout:8/293: chown df/d25/d28/f42 65741313 1 2024-03-24T10:45:54.449 INFO:tasks.workunit.client.0.smithi094.stdout:5/264: creat d7/dc/d56/f59 x:0 0 0 2024-03-24T10:45:54.450 INFO:tasks.workunit.client.0.smithi094.stdout:5/265: readlink d7/l33 0 2024-03-24T10:45:54.450 INFO:tasks.workunit.client.0.smithi094.stdout:5/266: dread - d7/d8/f58 zero size 2024-03-24T10:45:54.450 INFO:tasks.workunit.client.0.smithi094.stdout:3/209: symlink d0/l4b 0 2024-03-24T10:45:54.451 INFO:tasks.workunit.client.0.smithi094.stdout:1/224: creat d4/d9/db/d39/f49 x:0 0 0 2024-03-24T10:45:54.453 INFO:tasks.workunit.client.0.smithi094.stdout:2/191: mknod d18/c37 0 2024-03-24T10:45:54.453 INFO:tasks.workunit.client.0.smithi094.stdout:2/192: chown d18/l36 279859614 1 2024-03-24T10:45:54.453 INFO:tasks.workunit.client.0.smithi094.stdout:7/282: dwrite d1/d9/d31/d39/d44/f56 [0,4194304] 0 2024-03-24T10:45:54.453 INFO:tasks.workunit.client.0.smithi094.stdout:8/294: mknod df/d12/d5b/c60 0 2024-03-24T10:45:54.453 INFO:tasks.workunit.client.0.smithi094.stdout:8/295: stat df/d25/d28/d31/f40 0 2024-03-24T10:45:54.453 INFO:tasks.workunit.client.0.smithi094.stdout:8/296: creat df/d25/f61 x:0 0 0 2024-03-24T10:45:54.454 INFO:tasks.workunit.client.0.smithi094.stdout:8/297: chown df/d12/d5b 0 1 2024-03-24T10:45:54.455 INFO:tasks.workunit.client.0.smithi094.stdout:5/267: rename d7/d8/d9/f54 to d7/d23/d2b/f5a 0 2024-03-24T10:45:54.455 INFO:tasks.workunit.client.0.smithi094.stdout:5/268: fsync d7/d26/f53 0 2024-03-24T10:45:54.455 INFO:tasks.workunit.client.0.smithi094.stdout:4/322: dwrite d0/d5/d1b/f2e [0,4194304] 0 2024-03-24T10:45:54.456 INFO:tasks.workunit.client.0.smithi094.stdout:3/210: creat d0/db/d1c/d20/d23/d2a/f4c x:0 0 0 2024-03-24T10:45:54.462 INFO:tasks.workunit.client.0.smithi094.stdout:4/323: write d0/d5/d42/d4a/f61 [9469812,130235] 0 2024-03-24T10:45:54.462 INFO:tasks.workunit.client.0.smithi094.stdout:4/324: write d0/d1f/f55 [215197,10049] 0 2024-03-24T10:45:54.462 INFO:tasks.workunit.client.0.smithi094.stdout:4/325: chown d0/d22/de/d28/d2b/d33/c45 53718 1 2024-03-24T10:45:54.464 INFO:tasks.workunit.client.0.smithi094.stdout:7/283: unlink d1/d9/d25/l2a 0 2024-03-24T10:45:54.469 INFO:tasks.workunit.client.0.smithi094.stdout:1/225: dwrite d4/d9/f25 [0,4194304] 0 2024-03-24T10:45:54.469 INFO:tasks.workunit.client.0.smithi094.stdout:1/226: write d4/d9/db/f42 [585096,80647] 0 2024-03-24T10:45:54.469 INFO:tasks.workunit.client.0.smithi094.stdout:1/227: read - d4/d9/f28 zero size 2024-03-24T10:45:54.470 INFO:tasks.workunit.client.0.smithi094.stdout:3/211: mknod d0/db/d1c/d20/d23/d2a/d3a/d25/c4d 0 2024-03-24T10:45:54.472 INFO:tasks.workunit.client.0.smithi094.stdout:7/284: rename d1/d9/d16/d21/l3a to d1/d9/d16/d21/d51/l5b 0 2024-03-24T10:45:54.473 INFO:tasks.workunit.client.0.smithi094.stdout:8/298: rmdir df/d25/d36/d3f 39 2024-03-24T10:45:54.476 INFO:tasks.workunit.client.0.smithi094.stdout:1/228: symlink d4/d9/d37/l4a 0 2024-03-24T10:45:54.477 INFO:tasks.workunit.client.0.smithi094.stdout:4/326: link d0/c23 d0/c66 0 2024-03-24T10:45:54.477 INFO:tasks.workunit.client.0.smithi094.stdout:4/327: stat d0/d22/de/d28/d3d/f5c 0 2024-03-24T10:45:54.478 INFO:tasks.workunit.client.0.smithi094.stdout:6/236: sync 2024-03-24T10:45:54.478 INFO:tasks.workunit.client.0.smithi094.stdout:6/237: fdatasync d0/d21/f3a 0 2024-03-24T10:45:54.479 INFO:tasks.workunit.client.0.smithi094.stdout:7/285: symlink d1/d9/d25/d3d/l5c 0 2024-03-24T10:45:54.479 INFO:tasks.workunit.client.0.smithi094.stdout:7/286: truncate d1/d9/f12 4787069 0 2024-03-24T10:45:54.479 INFO:tasks.workunit.client.0.smithi094.stdout:7/287: getdents d1/d9/d35 0 2024-03-24T10:45:54.480 INFO:tasks.workunit.client.0.smithi094.stdout:8/299: symlink df/d25/d28/d31/l62 0 2024-03-24T10:45:54.483 INFO:tasks.workunit.client.0.smithi094.stdout:3/212: link d0/db/c32 d0/db/dc/c4e 0 2024-03-24T10:45:54.483 INFO:tasks.workunit.client.0.smithi094.stdout:3/213: chown d0/db/dc 932635 1 2024-03-24T10:45:54.484 INFO:tasks.workunit.client.0.smithi094.stdout:3/214: fdatasync d0/db/d1c/d20/d23/f29 0 2024-03-24T10:45:54.485 INFO:tasks.workunit.client.0.smithi094.stdout:1/229: dread d4/d9/f32 [0,4194304] 0 2024-03-24T10:45:54.485 INFO:tasks.workunit.client.0.smithi094.stdout:1/230: creat d4/f4b x:0 0 0 2024-03-24T10:45:54.485 INFO:tasks.workunit.client.0.smithi094.stdout:1/231: truncate d4/d36/f3f 691458 0 2024-03-24T10:45:54.485 INFO:tasks.workunit.client.0.smithi094.stdout:1/232: stat d4/f7 0 2024-03-24T10:45:54.485 INFO:tasks.workunit.client.0.smithi094.stdout:1/233: read - d4/d9/db/f30 zero size 2024-03-24T10:45:54.485 INFO:tasks.workunit.client.0.smithi094.stdout:1/234: chown d4/d9/db/f10 8360502 1 2024-03-24T10:45:54.485 INFO:tasks.workunit.client.0.smithi094.stdout:1/235: stat d4/f4b 0 2024-03-24T10:45:54.487 INFO:tasks.workunit.client.0.smithi094.stdout:8/300: mkdir df/d12/d5b/d63 0 2024-03-24T10:45:54.488 INFO:tasks.workunit.client.0.smithi094.stdout:7/288: dread d1/d9/d31/d39/d44/f56 [0,4194304] 0 2024-03-24T10:45:54.488 INFO:tasks.workunit.client.0.smithi094.stdout:7/289: dread - d1/d9/d25/d3d/f40 zero size 2024-03-24T10:45:54.488 INFO:tasks.workunit.client.0.smithi094.stdout:3/215: mkdir d0/db/d1c/d20/d23/d2a/d3a/d25/d4f 0 2024-03-24T10:45:54.489 INFO:tasks.workunit.client.0.smithi094.stdout:3/216: truncate d0/db/d1c/d20/d23/d2a/f33 1056879 0 2024-03-24T10:45:54.492 INFO:tasks.workunit.client.0.smithi094.stdout:1/236: creat d4/d9/d37/f4c x:0 0 0 2024-03-24T10:45:54.492 INFO:tasks.workunit.client.0.smithi094.stdout:1/237: fsync d4/d9/f32 0 2024-03-24T10:45:54.492 INFO:tasks.workunit.client.0.smithi094.stdout:6/238: rmdir d0/d5/da 39 2024-03-24T10:45:54.496 INFO:tasks.workunit.client.0.smithi094.stdout:7/290: rename d1/d9/d25/d3d/f40 to d1/f5d 0 2024-03-24T10:45:54.497 INFO:tasks.workunit.client.0.smithi094.stdout:1/238: mknod d4/d9/de/d3d/c4d 0 2024-03-24T10:45:54.498 INFO:tasks.workunit.client.0.smithi094.stdout:1/239: write d4/f46 [604482,9352] 0 2024-03-24T10:45:54.498 INFO:tasks.workunit.client.0.smithi094.stdout:1/240: write d4/f16 [2946550,91634] 0 2024-03-24T10:45:54.498 INFO:tasks.workunit.client.0.smithi094.stdout:4/328: dwrite d0/d22/de/d28/f59 [0,4194304] 0 2024-03-24T10:45:54.498 INFO:tasks.workunit.client.0.smithi094.stdout:4/329: fsync d0/d22/f11 0 2024-03-24T10:45:54.498 INFO:tasks.workunit.client.0.smithi094.stdout:4/330: write d0/d22/f62 [467590,48940] 0 2024-03-24T10:45:54.498 INFO:tasks.workunit.client.0.smithi094.stdout:7/291: creat d1/d9/d16/d21/d51/f5e x:0 0 0 2024-03-24T10:45:54.501 INFO:tasks.workunit.client.0.smithi094.stdout:4/331: rmdir d0/d34 39 2024-03-24T10:45:54.506 INFO:tasks.workunit.client.0.smithi094.stdout:4/332: dread d0/d22/ff [0,4194304] 0 2024-03-24T10:45:54.506 INFO:tasks.workunit.client.0.smithi094.stdout:4/333: stat d0/f60 0 2024-03-24T10:45:54.516 INFO:tasks.workunit.client.0.smithi094.stdout:8/301: dwrite f4 [0,4194304] 0 2024-03-24T10:45:54.517 INFO:tasks.workunit.client.0.smithi094.stdout:1/241: dwrite d4/f1e [0,4194304] 0 2024-03-24T10:45:54.517 INFO:tasks.workunit.client.0.smithi094.stdout:1/242: readlink d4/d9/db/l20 0 2024-03-24T10:45:54.519 INFO:tasks.workunit.client.0.smithi094.stdout:3/217: dwrite d0/db/dc/f3b [0,4194304] 0 2024-03-24T10:45:54.522 INFO:tasks.workunit.client.0.smithi094.stdout:7/292: rmdir d1/d9/d25 39 2024-03-24T10:45:54.524 INFO:tasks.workunit.client.0.smithi094.stdout:8/302: write df/d12/f39 [2198615,96172] 0 2024-03-24T10:45:54.524 INFO:tasks.workunit.client.0.smithi094.stdout:6/239: dwrite d0/d30/f28 [0,4194304] 0 2024-03-24T10:45:54.524 INFO:tasks.workunit.client.0.smithi094.stdout:6/240: dread - d0/d21/f2b zero size 2024-03-24T10:45:54.524 INFO:tasks.workunit.client.0.smithi094.stdout:6/241: write d0/d21/f3a [245405,25795] 0 2024-03-24T10:45:54.527 INFO:tasks.workunit.client.0.smithi094.stdout:3/218: mkdir d0/db/dc/d28/d50 0 2024-03-24T10:45:54.527 INFO:tasks.workunit.client.0.smithi094.stdout:7/293: creat d1/d9/d25/f5f x:0 0 0 2024-03-24T10:45:54.529 INFO:tasks.workunit.client.0.smithi094.stdout:8/303: symlink df/d12/l64 0 2024-03-24T10:45:54.530 INFO:tasks.workunit.client.0.smithi094.stdout:8/304: truncate df/d25/f48 266525 0 2024-03-24T10:45:54.530 INFO:tasks.workunit.client.0.smithi094.stdout:8/305: fsync df/f1b 0 2024-03-24T10:45:54.530 INFO:tasks.workunit.client.0.smithi094.stdout:8/306: readlink df/d12/l53 0 2024-03-24T10:45:54.530 INFO:tasks.workunit.client.0.smithi094.stdout:1/243: dwrite d4/f4b [0,4194304] 0 2024-03-24T10:45:54.531 INFO:tasks.workunit.client.0.smithi094.stdout:1/244: chown d4/d9/de/c14 339910580 1 2024-03-24T10:45:54.531 INFO:tasks.workunit.client.0.smithi094.stdout:6/242: symlink d0/d5/da/d23/d34/l47 0 2024-03-24T10:45:54.532 INFO:tasks.workunit.client.0.smithi094.stdout:3/219: symlink d0/db/dc/d28/d50/l51 0 2024-03-24T10:45:54.532 INFO:tasks.workunit.client.0.smithi094.stdout:3/220: dread - d0/db/d1c/d20/d23/f29 zero size 2024-03-24T10:45:54.534 INFO:tasks.workunit.client.0.smithi094.stdout:1/245: creat d4/d36/d38/f4e x:0 0 0 2024-03-24T10:45:54.536 INFO:tasks.workunit.client.0.smithi094.stdout:6/243: mknod d0/d5/da/d23/d34/d3d/c48 0 2024-03-24T10:45:54.536 INFO:tasks.workunit.client.0.smithi094.stdout:6/244: chown d0/d5/da/d23/f26 1851 1 2024-03-24T10:45:54.545 INFO:tasks.workunit.client.0.smithi094.stdout:9/298: sync 2024-03-24T10:45:54.545 INFO:tasks.workunit.client.0.smithi094.stdout:2/193: sync 2024-03-24T10:45:54.545 INFO:tasks.workunit.client.0.smithi094.stdout:5/269: sync 2024-03-24T10:45:54.545 INFO:tasks.workunit.client.0.smithi094.stdout:0/266: sync 2024-03-24T10:45:54.545 INFO:tasks.workunit.client.0.smithi094.stdout:5/270: chown d7/d8/d9/f4f 20219704 1 2024-03-24T10:45:54.545 INFO:tasks.workunit.client.0.smithi094.stdout:0/267: write d14/d22/d28/d48/f4a [1848811,91970] 0 2024-03-24T10:45:54.545 INFO:tasks.workunit.client.0.smithi094.stdout:0/268: fdatasync d14/d18/f1f 0 2024-03-24T10:45:54.550 INFO:tasks.workunit.client.0.smithi094.stdout:3/221: rename d0/db/dc/c13 to d0/db/dc/c52 0 2024-03-24T10:45:54.550 INFO:tasks.workunit.client.0.smithi094.stdout:1/246: rmdir d4 39 2024-03-24T10:45:54.550 INFO:tasks.workunit.client.0.smithi094.stdout:1/247: chown d4/d9/db/d31/d3c 524477450 1 2024-03-24T10:45:54.552 INFO:tasks.workunit.client.0.smithi094.stdout:9/299: mknod df/d2d/d34/c59 0 2024-03-24T10:45:54.552 INFO:tasks.workunit.client.0.smithi094.stdout:2/194: dread d18/f19 [0,4194304] 0 2024-03-24T10:45:54.552 INFO:tasks.workunit.client.0.smithi094.stdout:2/195: creat d18/f38 x:0 0 0 2024-03-24T10:45:54.553 INFO:tasks.workunit.client.0.smithi094.stdout:0/269: mknod d14/d22/c5c 0 2024-03-24T10:45:54.553 INFO:tasks.workunit.client.0.smithi094.stdout:3/222: mkdir d0/db/dc/d28/d38/d53 0 2024-03-24T10:45:54.554 INFO:tasks.workunit.client.0.smithi094.stdout:1/248: creat d4/d9/d47/f4f x:0 0 0 2024-03-24T10:45:54.556 INFO:tasks.workunit.client.0.smithi094.stdout:9/300: rename df/d2d/d34/d2c/d3a to df/d2d/d1f/d58/d5a 0 2024-03-24T10:45:54.556 INFO:tasks.workunit.client.0.smithi094.stdout:7/294: dwrite d1/d9/d25/d3d/f55 [0,4194304] 0 2024-03-24T10:45:54.556 INFO:tasks.workunit.client.0.smithi094.stdout:8/307: dwrite df/d25/d28/d31/f40 [0,4194304] 0 2024-03-24T10:45:54.556 INFO:tasks.workunit.client.0.smithi094.stdout:8/308: chown df/d25/f44 1060418390 1 2024-03-24T10:45:54.556 INFO:tasks.workunit.client.0.smithi094.stdout:8/309: truncate df/d25/f30 1051198 0 2024-03-24T10:45:54.557 INFO:tasks.workunit.client.0.smithi094.stdout:5/271: getdents d7/d8 0 2024-03-24T10:45:54.557 INFO:tasks.workunit.client.0.smithi094.stdout:6/245: dwrite d0/f42 [0,4194304] 0 2024-03-24T10:45:54.559 INFO:tasks.workunit.client.0.smithi094.stdout:3/223: symlink d0/db/dc/d28/d3c/l54 0 2024-03-24T10:45:54.559 INFO:tasks.workunit.client.0.smithi094.stdout:0/270: write d14/f1e [723984,111906] 0 2024-03-24T10:45:54.560 INFO:tasks.workunit.client.0.smithi094.stdout:1/249: link d4/d9/db/f1f d4/d9/db/d31/d3c/f50 0 2024-03-24T10:45:54.560 INFO:tasks.workunit.client.0.smithi094.stdout:1/250: chown d4/d9/db/l34 104 1 2024-03-24T10:45:54.562 INFO:tasks.workunit.client.0.smithi094.stdout:2/196: rename d18/l2f to d18/d1e/l39 0 2024-03-24T10:45:54.563 INFO:tasks.workunit.client.0.smithi094.stdout:0/271: dread f0 [0,4194304] 0 2024-03-24T10:45:54.565 INFO:tasks.workunit.client.0.smithi094.stdout:8/310: mknod df/d12/d5b/c65 0 2024-03-24T10:45:54.567 INFO:tasks.workunit.client.0.smithi094.stdout:5/272: symlink d7/dc/l5b 0 2024-03-24T10:45:54.570 INFO:tasks.workunit.client.0.smithi094.stdout:6/246: getdents d0/d5/da/d23 0 2024-03-24T10:45:54.571 INFO:tasks.workunit.client.0.smithi094.stdout:3/224: unlink d0/db/dc/f3b 0 2024-03-24T10:45:54.573 INFO:tasks.workunit.client.0.smithi094.stdout:1/251: creat d4/d9/d37/d33/f51 x:0 0 0 2024-03-24T10:45:54.575 INFO:tasks.workunit.client.0.smithi094.stdout:9/301: rename df/d2d/d1f/l1a to df/d2d/d1f/d58/l5b 0 2024-03-24T10:45:54.576 INFO:tasks.workunit.client.0.smithi094.stdout:2/197: mknod d18/d1e/c3a 0 2024-03-24T10:45:54.578 INFO:tasks.workunit.client.0.smithi094.stdout:0/272: link d14/d18/f21 d14/d22/d32/f5d 0 2024-03-24T10:45:54.580 INFO:tasks.workunit.client.0.smithi094.stdout:8/311: mkdir df/d25/d28/d31/d47/d66 0 2024-03-24T10:45:54.581 INFO:tasks.workunit.client.0.smithi094.stdout:1/252: truncate d4/f2a 3191900 0 2024-03-24T10:45:54.585 INFO:tasks.workunit.client.0.smithi094.stdout:9/302: unlink df/d2d/l38 0 2024-03-24T10:45:54.585 INFO:tasks.workunit.client.0.smithi094.stdout:0/273: truncate d14/d22/d28/d2b/f3d 726215 0 2024-03-24T10:45:54.587 INFO:tasks.workunit.client.0.smithi094.stdout:5/273: dwrite d7/d26/f53 [0,4194304] 0 2024-03-24T10:45:54.587 INFO:tasks.workunit.client.0.smithi094.stdout:5/274: stat d7/d26/l2d 0 2024-03-24T10:45:54.588 INFO:tasks.workunit.client.0.smithi094.stdout:5/275: chown d7/d8/d9/l13 33 1 2024-03-24T10:45:54.588 INFO:tasks.workunit.client.0.smithi094.stdout:5/276: creat d7/d26/f5c x:0 0 0 2024-03-24T10:45:54.588 INFO:tasks.workunit.client.0.smithi094.stdout:5/277: fsync d7/d8/d9/d48/d51/f52 0 2024-03-24T10:45:54.588 INFO:tasks.workunit.client.0.smithi094.stdout:7/295: rename d1/d9/d31 to d1/d9/d16/d20/d60 0 2024-03-24T10:45:54.589 INFO:tasks.workunit.client.0.smithi094.stdout:9/303: mknod df/d2d/d34/c5c 0 2024-03-24T10:45:54.589 INFO:tasks.workunit.client.0.smithi094.stdout:9/304: readlink df/d2d/l54 0 2024-03-24T10:45:54.590 INFO:tasks.workunit.client.0.smithi094.stdout:9/305: read - df/d2d/d1f/f3d zero size 2024-03-24T10:45:54.591 INFO:tasks.workunit.client.0.smithi094.stdout:5/278: rmdir d7/d8/d9/d48/d51 39 2024-03-24T10:45:54.592 INFO:tasks.workunit.client.0.smithi094.stdout:5/279: stat d7/d8/d9/f49 0 2024-03-24T10:45:54.592 INFO:tasks.workunit.client.0.smithi094.stdout:5/280: dread - d7/d8/f58 zero size 2024-03-24T10:45:54.593 INFO:tasks.workunit.client.0.smithi094.stdout:3/225: rename d0/db/dc/d45 to d0/db/dc/d28/d3c/d55 0 2024-03-24T10:45:54.593 INFO:tasks.workunit.client.0.smithi094.stdout:3/226: truncate d0/f11 4476878 0 2024-03-24T10:45:54.593 INFO:tasks.workunit.client.0.smithi094.stdout:9/306: symlink df/d2d/d1f/l5d 0 2024-03-24T10:45:54.593 INFO:tasks.workunit.client.0.smithi094.stdout:9/307: write df/d2d/d34/f47 [1630664,82746] 0 2024-03-24T10:45:54.594 INFO:tasks.workunit.client.0.smithi094.stdout:0/274: dread d14/d22/d32/f5d [0,4194304] 0 2024-03-24T10:45:54.594 INFO:tasks.workunit.client.0.smithi094.stdout:5/281: mknod d7/d8/d14/d36/d3a/c5d 0 2024-03-24T10:45:54.595 INFO:tasks.workunit.client.0.smithi094.stdout:3/227: truncate d0/f2 1773963 0 2024-03-24T10:45:54.597 INFO:tasks.workunit.client.0.smithi094.stdout:9/308: rename df/d2d/l54 to df/d2d/d1f/d11/l5e 0 2024-03-24T10:45:54.598 INFO:tasks.workunit.client.0.smithi094.stdout:5/282: mknod d7/d8/d9/d48/d51/c5e 0 2024-03-24T10:45:54.599 INFO:tasks.workunit.client.0.smithi094.stdout:5/283: dread d7/d8/d14/d36/f4d [0,4194304] 0 2024-03-24T10:45:54.600 INFO:tasks.workunit.client.0.smithi094.stdout:0/275: rename d14/d22/d32/l50 to d14/d22/d28/d2b/d41/l5e 0 2024-03-24T10:45:54.601 INFO:tasks.workunit.client.0.smithi094.stdout:1/253: dwrite d4/f46 [0,4194304] 0 2024-03-24T10:45:54.603 INFO:tasks.workunit.client.0.smithi094.stdout:6/247: dwrite d0/d5/da/d23/f26 [0,4194304] 0 2024-03-24T10:45:54.603 INFO:tasks.workunit.client.0.smithi094.stdout:0/276: truncate d14/f2a 4273746 0 2024-03-24T10:45:54.606 INFO:tasks.workunit.client.0.smithi094.stdout:2/198: dwrite f12 [0,4194304] 0 2024-03-24T10:45:54.606 INFO:tasks.workunit.client.0.smithi094.stdout:2/199: fdatasync d18/f38 0 2024-03-24T10:45:54.606 INFO:tasks.workunit.client.0.smithi094.stdout:2/200: write f7 [1329651,26817] 0 2024-03-24T10:45:54.607 INFO:tasks.workunit.client.0.smithi094.stdout:1/254: symlink d4/d9/d37/l52 0 2024-03-24T10:45:54.607 INFO:tasks.workunit.client.0.smithi094.stdout:1/255: truncate d4/d9/db/d31/f48 573027 0 2024-03-24T10:45:54.607 INFO:tasks.workunit.client.0.smithi094.stdout:9/309: write df/f56 [995409,42455] 0 2024-03-24T10:45:54.612 INFO:tasks.workunit.client.0.smithi094.stdout:6/248: getdents d0/d5/da/d23/d34/d3d 0 2024-03-24T10:45:54.612 INFO:tasks.workunit.client.0.smithi094.stdout:6/249: truncate d0/d21/f2b 553126 0 2024-03-24T10:45:54.613 INFO:tasks.workunit.client.0.smithi094.stdout:6/250: write d0/d21/f2b [380996,28017] 0 2024-03-24T10:45:54.613 INFO:tasks.workunit.client.0.smithi094.stdout:0/277: symlink d14/d22/d32/l5f 0 2024-03-24T10:45:54.615 INFO:tasks.workunit.client.0.smithi094.stdout:8/312: dwrite df/d12/f39 [0,4194304] 0 2024-03-24T10:45:54.618 INFO:tasks.workunit.client.0.smithi094.stdout:3/228: dwrite d0/db/d1c/d20/d23/f29 [0,4194304] 0 2024-03-24T10:45:54.618 INFO:tasks.workunit.client.0.smithi094.stdout:3/229: fdatasync d0/db/d1c/d20/d23/d2a/f33 0 2024-03-24T10:45:54.624 INFO:tasks.workunit.client.0.smithi094.stdout:5/284: rename d7/d8 to d7/d23/d5f 0 2024-03-24T10:45:54.627 INFO:tasks.workunit.client.0.smithi094.stdout:9/310: creat df/d2d/d34/d2c/f5f x:0 0 0 2024-03-24T10:45:54.628 INFO:tasks.workunit.client.0.smithi094.stdout:0/278: mknod d14/d18/c60 0 2024-03-24T10:45:54.631 INFO:tasks.workunit.client.0.smithi094.stdout:6/251: dwrite d0/d21/d2a/f40 [0,4194304] 0 2024-03-24T10:45:54.631 INFO:tasks.workunit.client.0.smithi094.stdout:6/252: write d0/d5/f19 [1557832,38805] 0 2024-03-24T10:45:54.631 INFO:tasks.workunit.client.0.smithi094.stdout:8/313: dwrite df/d25/d36/d3f/f4f [0,4194304] 0 2024-03-24T10:45:54.633 INFO:tasks.workunit.client.0.smithi094.stdout:1/256: rename d4/d9/db/l15 to d4/d9/db/d31/l53 0 2024-03-24T10:45:54.634 INFO:tasks.workunit.client.0.smithi094.stdout:3/230: dwrite d0/db/d1c/d20/d23/d2a/f33 [0,4194304] 0 2024-03-24T10:45:54.636 INFO:tasks.workunit.client.0.smithi094.stdout:2/201: getdents d18 0 2024-03-24T10:45:54.638 INFO:tasks.workunit.client.0.smithi094.stdout:0/279: symlink d14/d22/d28/d2b/l61 0 2024-03-24T10:45:54.640 INFO:tasks.workunit.client.0.smithi094.stdout:6/253: creat d0/f49 x:0 0 0 2024-03-24T10:45:54.643 INFO:tasks.workunit.client.0.smithi094.stdout:1/257: getdents d4/d9/d37 0 2024-03-24T10:45:54.646 INFO:tasks.workunit.client.0.smithi094.stdout:9/311: dwrite df/d2d/d1f/d58/d5a/f4b [4194304,4194304] 0 2024-03-24T10:45:54.647 INFO:tasks.workunit.client.0.smithi094.stdout:3/231: mkdir d0/db/d1c/d20/d23/d2a/d3a/d56 0 2024-03-24T10:45:54.648 INFO:tasks.workunit.client.0.smithi094.stdout:8/314: dwrite f6 [0,4194304] 0 2024-03-24T10:45:54.648 INFO:tasks.workunit.client.0.smithi094.stdout:8/315: fdatasync f5 0 2024-03-24T10:45:54.650 INFO:tasks.workunit.client.0.smithi094.stdout:4/334: sync 2024-03-24T10:45:54.650 INFO:tasks.workunit.client.0.smithi094.stdout:4/335: write d0/f60 [214928,124104] 0 2024-03-24T10:45:54.650 INFO:tasks.workunit.client.0.smithi094.stdout:5/285: rmdir d7/d23 39 2024-03-24T10:45:54.650 INFO:tasks.workunit.client.0.smithi094.stdout:5/286: fdatasync d7/dc/d12/f39 0 2024-03-24T10:45:54.650 INFO:tasks.workunit.client.0.smithi094.stdout:6/254: read d0/f33 [1049978,67730] 0 2024-03-24T10:45:54.652 INFO:tasks.workunit.client.0.smithi094.stdout:2/202: rename d18/d1e/f32 to d18/f3b 0 2024-03-24T10:45:54.653 INFO:tasks.workunit.client.0.smithi094.stdout:3/232: rename d0/db/d1c to d0/db/d1c/d20/d23/d2a/d57 22 2024-03-24T10:45:54.653 INFO:tasks.workunit.client.0.smithi094.stdout:3/233: fsync d0/db/d1c/d20/f24 0 2024-03-24T10:45:54.653 INFO:tasks.workunit.client.0.smithi094.stdout:3/234: fsync d0/f11 0 2024-03-24T10:45:54.657 INFO:tasks.workunit.client.0.smithi094.stdout:8/316: symlink df/d25/l67 0 2024-03-24T10:45:54.657 INFO:tasks.workunit.client.0.smithi094.stdout:8/317: truncate df/d25/d28/d31/d47/f5a 986484 0 2024-03-24T10:45:54.657 INFO:tasks.workunit.client.0.smithi094.stdout:8/318: stat df/d25/d28/d31/l62 0 2024-03-24T10:45:54.658 INFO:tasks.workunit.client.0.smithi094.stdout:8/319: dread df/d12/f52 [0,4194304] 0 2024-03-24T10:45:54.658 INFO:tasks.workunit.client.0.smithi094.stdout:8/320: truncate df/d25/f58 262529 0 2024-03-24T10:45:54.659 INFO:tasks.workunit.client.0.smithi094.stdout:4/336: creat d0/d5/d42/f67 x:0 0 0 2024-03-24T10:45:54.661 INFO:tasks.workunit.client.0.smithi094.stdout:5/287: unlink d7/d26/l2c 0 2024-03-24T10:45:54.662 INFO:tasks.workunit.client.0.smithi094.stdout:2/203: dread f15 [4194304,4194304] 0 2024-03-24T10:45:54.662 INFO:tasks.workunit.client.0.smithi094.stdout:8/321: write f3 [1043003,9228] 0 2024-03-24T10:45:54.662 INFO:tasks.workunit.client.0.smithi094.stdout:5/288: read d7/d23/d2b/f4b [23525,4140] 0 2024-03-24T10:45:54.662 INFO:tasks.workunit.client.0.smithi094.stdout:6/255: symlink d0/d21/l4a 0 2024-03-24T10:45:54.662 INFO:tasks.workunit.client.0.smithi094.stdout:6/256: write d0/fc [2798415,69040] 0 2024-03-24T10:45:54.663 INFO:tasks.workunit.client.0.smithi094.stdout:3/235: creat d0/db/dc/d28/d3c/f58 x:0 0 0 2024-03-24T10:45:54.663 INFO:tasks.workunit.client.0.smithi094.stdout:3/236: readlink d0/db/dc/d28/l3e 0 2024-03-24T10:45:54.664 INFO:tasks.workunit.client.0.smithi094.stdout:0/280: unlink d14/f27 0 2024-03-24T10:45:54.666 INFO:tasks.workunit.client.0.smithi094.stdout:8/322: mknod df/d25/c68 0 2024-03-24T10:45:54.667 INFO:tasks.workunit.client.0.smithi094.stdout:5/289: rmdir d7/d23/d5f/d9 39 2024-03-24T10:45:54.668 INFO:tasks.workunit.client.0.smithi094.stdout:4/337: dread d0/d34/f48 [4194304,4194304] 0 2024-03-24T10:45:54.668 INFO:tasks.workunit.client.0.smithi094.stdout:4/338: readlink d0/d22/l58 0 2024-03-24T10:45:54.668 INFO:tasks.workunit.client.0.smithi094.stdout:2/204: link l6 d18/d1e/l3c 0 2024-03-24T10:45:54.670 INFO:tasks.workunit.client.0.smithi094.stdout:8/323: link df/d25/d28/d31/d47/d22/f3d df/d25/d28/d31/d47/f69 0 2024-03-24T10:45:54.674 INFO:tasks.workunit.client.0.smithi094.stdout:5/290: unlink d7/dc/d56/f59 0 2024-03-24T10:45:54.674 INFO:tasks.workunit.client.0.smithi094.stdout:9/312: dwrite df/d2d/d34/f2e [0,4194304] 0 2024-03-24T10:45:54.675 INFO:tasks.workunit.client.0.smithi094.stdout:3/237: dread d0/db/d1c/d20/f24 [0,4194304] 0 2024-03-24T10:45:54.675 INFO:tasks.workunit.client.0.smithi094.stdout:3/238: fsync d0/f42 0 2024-03-24T10:45:54.675 INFO:tasks.workunit.client.0.smithi094.stdout:3/239: dread - d0/db/d1c/d20/d23/d2a/f4c zero size 2024-03-24T10:45:54.675 INFO:tasks.workunit.client.0.smithi094.stdout:3/240: truncate d0/f2 1926081 0 2024-03-24T10:45:54.676 INFO:tasks.workunit.client.0.smithi094.stdout:2/205: link d18/f23 d18/d1e/d34/f3d 0 2024-03-24T10:45:54.676 INFO:tasks.workunit.client.0.smithi094.stdout:2/206: write d18/f19 [2449768,101163] 0 2024-03-24T10:45:54.678 INFO:tasks.workunit.client.0.smithi094.stdout:8/324: creat df/d12/f6a x:0 0 0 2024-03-24T10:45:54.680 INFO:tasks.workunit.client.0.smithi094.stdout:5/291: creat d7/d23/d5f/d1b/f60 x:0 0 0 2024-03-24T10:45:54.680 INFO:tasks.workunit.client.0.smithi094.stdout:5/292: chown c3 34072758 1 2024-03-24T10:45:54.681 INFO:tasks.workunit.client.0.smithi094.stdout:1/258: dwrite d4/d9/db/f18 [0,4194304] 0 2024-03-24T10:45:54.681 INFO:tasks.workunit.client.0.smithi094.stdout:0/281: dread fa [4194304,4194304] 0 2024-03-24T10:45:54.682 INFO:tasks.workunit.client.0.smithi094.stdout:1/259: dread d4/d9/db/f18 [4194304,4194304] 0 2024-03-24T10:45:54.682 INFO:tasks.workunit.client.0.smithi094.stdout:1/260: chown d4/d9/f2c 155918 1 2024-03-24T10:45:54.688 INFO:tasks.workunit.client.0.smithi094.stdout:3/241: creat d0/db/dc/d28/d38/d53/f59 x:0 0 0 2024-03-24T10:45:54.690 INFO:tasks.workunit.client.0.smithi094.stdout:8/325: rename df/d12/f52 to df/d25/d28/f6b 0 2024-03-24T10:45:54.691 INFO:tasks.workunit.client.0.smithi094.stdout:0/282: creat d14/f62 x:0 0 0 2024-03-24T10:45:54.691 INFO:tasks.workunit.client.0.smithi094.stdout:1/261: mknod d4/d9/db/d39/c54 0 2024-03-24T10:45:54.692 INFO:tasks.workunit.client.0.smithi094.stdout:3/242: symlink d0/db/d1c/d20/d23/d2a/d3a/l5a 0 2024-03-24T10:45:54.693 INFO:tasks.workunit.client.0.smithi094.stdout:5/293: write d7/f34 [3991232,64960] 0 2024-03-24T10:45:54.693 INFO:tasks.workunit.client.0.smithi094.stdout:8/326: rename df/d25/d36/d3f/l45 to df/d25/l6c 0 2024-03-24T10:45:54.695 INFO:tasks.workunit.client.0.smithi094.stdout:5/294: write d7/dc/f19 [1000272,20248] 0 2024-03-24T10:45:54.696 INFO:tasks.workunit.client.0.smithi094.stdout:8/327: creat df/d25/d28/d31/d47/d22/d55/f6d x:0 0 0 2024-03-24T10:45:54.696 INFO:tasks.workunit.client.0.smithi094.stdout:0/283: getdents d14/d18 0 2024-03-24T10:45:54.699 INFO:tasks.workunit.client.0.smithi094.stdout:0/284: mknod d14/d22/d28/d2b/c63 0 2024-03-24T10:45:54.699 INFO:tasks.workunit.client.0.smithi094.stdout:0/285: readlink d14/d22/d32/l5f 0 2024-03-24T10:45:54.701 INFO:tasks.workunit.client.0.smithi094.stdout:0/286: link d14/d18/l43 d14/d22/d28/d2b/d41/d53/l64 0 2024-03-24T10:45:54.701 INFO:tasks.workunit.client.0.smithi094.stdout:0/287: chown d14/d18/f3e 15 1 2024-03-24T10:45:54.703 INFO:tasks.workunit.client.0.smithi094.stdout:0/288: mkdir d14/d22/d32/d65 0 2024-03-24T10:45:54.704 INFO:tasks.workunit.client.0.smithi094.stdout:6/257: dwrite d0/fc [0,4194304] 0 2024-03-24T10:45:54.704 INFO:tasks.workunit.client.0.smithi094.stdout:6/258: chown d0/d21/c25 20 1 2024-03-24T10:45:54.705 INFO:tasks.workunit.client.0.smithi094.stdout:9/313: dwrite df/d2d/d1f/f3d [0,4194304] 0 2024-03-24T10:45:54.705 INFO:tasks.workunit.client.0.smithi094.stdout:3/243: dread d0/f2 [0,4194304] 0 2024-03-24T10:45:54.706 INFO:tasks.workunit.client.0.smithi094.stdout:6/259: rename d0/l22 to d0/d5/l4b 0 2024-03-24T10:45:54.712 INFO:tasks.workunit.client.0.smithi094.stdout:9/314: unlink df/f3b 0 2024-03-24T10:45:54.714 INFO:tasks.workunit.client.0.smithi094.stdout:3/244: rename d0/db/dc/d28/d3c/f58 to d0/db/d1c/d20/d23/d2a/f5b 0 2024-03-24T10:45:54.715 INFO:tasks.workunit.client.0.smithi094.stdout:3/245: getdents d0/db/d1c/d20/d23/d2a/d3a/d25/d4f 0 2024-03-24T10:45:54.716 INFO:tasks.workunit.client.0.smithi094.stdout:6/260: symlink d0/d5/da/d23/d34/l4c 0 2024-03-24T10:45:54.718 INFO:tasks.workunit.client.0.smithi094.stdout:9/315: link c2 df/d2d/d34/c60 0 2024-03-24T10:45:54.718 INFO:tasks.workunit.client.0.smithi094.stdout:9/316: read df/f41 [80248,123471] 0 2024-03-24T10:45:54.719 INFO:tasks.workunit.client.0.smithi094.stdout:3/246: mknod d0/db/dc/d28/d50/c5c 0 2024-03-24T10:45:54.721 INFO:tasks.workunit.client.0.smithi094.stdout:6/261: mkdir d0/d5/da/d23/d34/d3d/d4d 0 2024-03-24T10:45:54.722 INFO:tasks.workunit.client.0.smithi094.stdout:6/262: rmdir d0/d21/d2a/d2f 39 2024-03-24T10:45:54.723 INFO:tasks.workunit.client.0.smithi094.stdout:1/262: dwrite d4/f16 [0,4194304] 0 2024-03-24T10:45:54.726 INFO:tasks.workunit.client.0.smithi094.stdout:2/207: getdents d18/d1e/d34 0 2024-03-24T10:45:54.729 INFO:tasks.workunit.client.0.smithi094.stdout:2/208: unlink f12 0 2024-03-24T10:45:54.729 INFO:tasks.workunit.client.0.smithi094.stdout:6/263: dread d0/d5/da/f12 [0,4194304] 0 2024-03-24T10:45:54.742 INFO:tasks.workunit.client.0.smithi094.stdout:6/264: read d0/d5/da/d23/f26 [157351,61831] 0 2024-03-24T10:45:54.743 INFO:tasks.workunit.client.0.smithi094.stdout:5/295: dwrite d7/d23/f44 [0,4194304] 0 2024-03-24T10:45:54.743 INFO:tasks.workunit.client.0.smithi094.stdout:0/289: dwrite d14/f1b [0,4194304] 0 2024-03-24T10:45:54.743 INFO:tasks.workunit.client.0.smithi094.stdout:8/328: dwrite fd [0,4194304] 0 2024-03-24T10:45:54.743 INFO:tasks.workunit.client.0.smithi094.stdout:3/247: dwrite d0/db/f2b [0,4194304] 0 2024-03-24T10:45:54.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:54 smithi094 ceph-mon[29446]: pgmap v240: 65 pgs: 65 active+clean; 304 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 4.8 MiB/s rd, 17 MiB/s wr, 594 op/s 2024-03-24T10:45:54.747 INFO:tasks.workunit.client.0.smithi094.stdout:0/290: dread d14/f1e [0,4194304] 0 2024-03-24T10:45:54.749 INFO:tasks.workunit.client.0.smithi094.stdout:6/265: dread d0/d21/d2a/d2f/f36 [0,4194304] 0 2024-03-24T10:45:54.749 INFO:tasks.workunit.client.0.smithi094.stdout:6/266: stat d0/d5/c15 0 2024-03-24T10:45:54.755 INFO:tasks.workunit.client.0.smithi094.stdout:5/296: creat d7/dc/d12/f61 x:0 0 0 2024-03-24T10:45:54.757 INFO:tasks.workunit.client.0.smithi094.stdout:3/248: unlink d0/db/dc/d28/d38/d53/f59 0 2024-03-24T10:45:54.757 INFO:tasks.workunit.client.0.smithi094.stdout:6/267: write d0/d21/f13 [5050829,124666] 0 2024-03-24T10:45:54.757 INFO:tasks.workunit.client.0.smithi094.stdout:6/268: fsync d0/d5/da/d23/f26 0 2024-03-24T10:45:54.757 INFO:tasks.workunit.client.0.smithi094.stdout:3/249: write d0/db/f48 [409442,848] 0 2024-03-24T10:45:54.757 INFO:tasks.workunit.client.0.smithi094.stdout:0/291: mknod d14/d18/c66 0 2024-03-24T10:45:54.758 INFO:tasks.workunit.client.0.smithi094.stdout:5/297: mkdir d7/dc/d62 0 2024-03-24T10:45:54.760 INFO:tasks.workunit.client.0.smithi094.stdout:3/250: symlink d0/db/d1c/d20/d23/d2a/d3a/d25/l5d 0 2024-03-24T10:45:54.760 INFO:tasks.workunit.client.0.smithi094.stdout:3/251: chown d0/db/dc/l2f 70 1 2024-03-24T10:45:54.760 INFO:tasks.workunit.client.0.smithi094.stdout:3/252: dread - d0/f42 zero size 2024-03-24T10:45:54.760 INFO:tasks.workunit.client.0.smithi094.stdout:8/329: dwrite df/d25/d36/d3f/f59 [0,4194304] 0 2024-03-24T10:45:54.761 INFO:tasks.workunit.client.0.smithi094.stdout:0/292: creat d14/d18/f67 x:0 0 0 2024-03-24T10:45:54.762 INFO:tasks.workunit.client.0.smithi094.stdout:5/298: mkdir d7/dc/d62/d63 0 2024-03-24T10:45:54.765 INFO:tasks.workunit.client.0.smithi094.stdout:5/299: mkdir d7/d26/d55/d64 0 2024-03-24T10:45:54.766 INFO:tasks.workunit.client.0.smithi094.stdout:7/296: sync 2024-03-24T10:45:54.769 INFO:tasks.workunit.client.0.smithi094.stdout:7/297: rmdir d1/d59 0 2024-03-24T10:45:54.769 INFO:tasks.workunit.client.0.smithi094.stdout:8/330: dread df/f2b [0,4194304] 0 2024-03-24T10:45:54.770 INFO:tasks.workunit.client.0.smithi094.stdout:7/298: mkdir d1/d9/d35/d61 0 2024-03-24T10:45:54.770 INFO:tasks.workunit.client.0.smithi094.stdout:3/253: dread d0/f14 [0,4194304] 0 2024-03-24T10:45:54.772 INFO:tasks.workunit.client.0.smithi094.stdout:6/269: dwrite d0/f49 [0,4194304] 0 2024-03-24T10:45:54.772 INFO:tasks.workunit.client.0.smithi094.stdout:6/270: dread - d0/d5/da/f2e zero size 2024-03-24T10:45:54.772 INFO:tasks.workunit.client.0.smithi094.stdout:8/331: rename df/d12/l37 to df/d25/d36/l6e 0 2024-03-24T10:45:54.772 INFO:tasks.workunit.client.0.smithi094.stdout:8/332: truncate df/d25/d28/d31/d47/f69 5047937 0 2024-03-24T10:45:54.772 INFO:tasks.workunit.client.0.smithi094.stdout:5/300: dread d7/d23/d5f/d14/f35 [0,4194304] 0 2024-03-24T10:45:54.772 INFO:tasks.workunit.client.0.smithi094.stdout:5/301: creat d7/d26/f65 x:0 0 0 2024-03-24T10:45:54.773 INFO:tasks.workunit.client.0.smithi094.stdout:7/299: symlink d1/d9/d35/l62 0 2024-03-24T10:45:54.773 INFO:tasks.workunit.client.0.smithi094.stdout:3/254: dread d0/f14 [0,4194304] 0 2024-03-24T10:45:54.773 INFO:tasks.workunit.client.0.smithi094.stdout:3/255: readlink d0/db/d1c/d20/d23/d2a/l43 0 2024-03-24T10:45:54.774 INFO:tasks.workunit.client.0.smithi094.stdout:3/256: truncate d0/db/d1c/d20/d23/d2a/f5b 719271 0 2024-03-24T10:45:54.776 INFO:tasks.workunit.client.0.smithi094.stdout:6/271: mkdir d0/d5/da/d23/d34/d4e 0 2024-03-24T10:45:54.778 INFO:tasks.workunit.client.0.smithi094.stdout:6/272: dread d0/d5/da/f12 [0,4194304] 0 2024-03-24T10:45:54.778 INFO:tasks.workunit.client.0.smithi094.stdout:8/333: creat df/d25/d28/f6f x:0 0 0 2024-03-24T10:45:54.778 INFO:tasks.workunit.client.0.smithi094.stdout:8/334: write df/d25/f48 [355589,52831] 0 2024-03-24T10:45:54.778 INFO:tasks.workunit.client.0.smithi094.stdout:8/335: chown f4 100 1 2024-03-24T10:45:54.781 INFO:tasks.workunit.client.0.smithi094.stdout:0/293: dwrite d14/d22/d28/d2b/f3d [0,4194304] 0 2024-03-24T10:45:54.785 INFO:tasks.workunit.client.0.smithi094.stdout:7/300: rename d1/d9/d16/d20/c3e to d1/d9/d14/d34/c63 0 2024-03-24T10:45:54.786 INFO:tasks.workunit.client.0.smithi094.stdout:8/336: creat df/d25/d28/d31/d47/d22/f70 x:0 0 0 2024-03-24T10:45:54.787 INFO:tasks.workunit.client.0.smithi094.stdout:0/294: truncate d14/d18/f4f 98368 0 2024-03-24T10:45:54.792 INFO:tasks.workunit.client.0.smithi094.stdout:7/301: truncate d1/d9/d25/d3d/f5a 2131659 0 2024-03-24T10:45:54.792 INFO:tasks.workunit.client.0.smithi094.stdout:7/302: write d1/d9/d16/d21/f33 [4699362,5763] 0 2024-03-24T10:45:54.793 INFO:tasks.workunit.client.0.smithi094.stdout:6/273: dwrite d0/d5/da/f20 [0,4194304] 0 2024-03-24T10:45:54.794 INFO:tasks.workunit.client.0.smithi094.stdout:6/274: readlink d0/d5/da/l3f 0 2024-03-24T10:45:54.794 INFO:tasks.workunit.client.0.smithi094.stdout:6/275: creat d0/d21/f4f x:0 0 0 2024-03-24T10:45:54.794 INFO:tasks.workunit.client.0.smithi094.stdout:6/276: write d0/d21/d2a/d2f/f46 [1009840,74829] 0 2024-03-24T10:45:54.794 INFO:tasks.workunit.client.0.smithi094.stdout:6/277: readlink d0/d1a/l31 0 2024-03-24T10:45:54.794 INFO:tasks.workunit.client.0.smithi094.stdout:0/295: mkdir d14/d22/d28/d2b/d68 0 2024-03-24T10:45:54.794 INFO:tasks.workunit.client.0.smithi094.stdout:0/296: creat d14/d18/f69 x:0 0 0 2024-03-24T10:45:54.795 INFO:tasks.workunit.client.0.smithi094.stdout:7/303: symlink d1/d9/d35/l64 0 2024-03-24T10:45:54.796 INFO:tasks.workunit.client.0.smithi094.stdout:5/302: getdents d7/d26 0 2024-03-24T10:45:54.796 INFO:tasks.workunit.client.0.smithi094.stdout:5/303: chown d7/dc/d62 25880 1 2024-03-24T10:45:54.796 INFO:tasks.workunit.client.0.smithi094.stdout:5/304: chown d7/d23/d5f/d9/d48/d51/f52 32651714 1 2024-03-24T10:45:54.796 INFO:tasks.workunit.client.0.smithi094.stdout:6/278: write d0/d21/d2a/d2f/f36 [3203004,68331] 0 2024-03-24T10:45:54.796 INFO:tasks.workunit.client.0.smithi094.stdout:6/279: write d0/f33 [692398,79258] 0 2024-03-24T10:45:54.800 INFO:tasks.workunit.client.0.smithi094.stdout:0/297: symlink d14/d22/d28/d2b/d41/l6a 0 2024-03-24T10:45:54.800 INFO:tasks.workunit.client.0.smithi094.stdout:0/298: chown d14/d22/d32/f5d 4111308 1 2024-03-24T10:45:54.801 INFO:tasks.workunit.client.0.smithi094.stdout:7/304: mkdir d1/d9/d35/d65 0 2024-03-24T10:45:54.802 INFO:tasks.workunit.client.0.smithi094.stdout:5/305: getdents d7/dc/d62 0 2024-03-24T10:45:54.805 INFO:tasks.workunit.client.0.smithi094.stdout:6/280: rename d0/d5/da/d23/d34/d3d to d0/d30/d50 0 2024-03-24T10:45:54.806 INFO:tasks.workunit.client.0.smithi094.stdout:0/299: mkdir d14/d22/d28/d2b/d41/d6b 0 2024-03-24T10:45:54.806 INFO:tasks.workunit.client.0.smithi094.stdout:0/300: read - d14/d22/d28/d2b/d41/d53/f5a zero size 2024-03-24T10:45:54.806 INFO:tasks.workunit.client.0.smithi094.stdout:0/301: write d14/d18/f1d [3294261,42206] 0 2024-03-24T10:45:54.806 INFO:tasks.workunit.client.0.smithi094.stdout:4/339: sync 2024-03-24T10:45:54.807 INFO:tasks.workunit.client.0.smithi094.stdout:4/340: dread d0/d5/d1b/f2a [0,4194304] 0 2024-03-24T10:45:54.810 INFO:tasks.workunit.client.0.smithi094.stdout:7/305: getdents d1 0 2024-03-24T10:45:54.812 INFO:tasks.workunit.client.0.smithi094.stdout:6/281: dread d0/f49 [0,4194304] 0 2024-03-24T10:45:54.814 INFO:tasks.workunit.client.0.smithi094.stdout:5/306: rmdir d7 39 2024-03-24T10:45:54.816 INFO:tasks.workunit.client.0.smithi094.stdout:0/302: link d14/d18/f40 d14/d22/d28/f6c 0 2024-03-24T10:45:54.816 INFO:tasks.workunit.client.0.smithi094.stdout:0/303: write d14/f1e [646333,74576] 0 2024-03-24T10:45:54.817 INFO:tasks.workunit.client.0.smithi094.stdout:6/282: symlink d0/d21/l51 0 2024-03-24T10:45:54.818 INFO:tasks.workunit.client.0.smithi094.stdout:7/306: write d1/d9/f23 [1868661,85897] 0 2024-03-24T10:45:54.819 INFO:tasks.workunit.client.0.smithi094.stdout:0/304: symlink d14/d22/d28/d2b/l6d 0 2024-03-24T10:45:54.821 INFO:tasks.workunit.client.0.smithi094.stdout:5/307: mknod d7/dc/d12/c66 0 2024-03-24T10:45:54.822 INFO:tasks.workunit.client.0.smithi094.stdout:7/307: creat d1/d9/d35/d65/f66 x:0 0 0 2024-03-24T10:45:54.824 INFO:tasks.workunit.client.0.smithi094.stdout:6/283: unlink d0/c4 0 2024-03-24T10:45:54.824 INFO:tasks.workunit.client.0.smithi094.stdout:6/284: write d0/d5/da/f12 [1464931,106214] 0 2024-03-24T10:45:54.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:54 smithi096 ceph-mon[38097]: pgmap v240: 65 pgs: 65 active+clean; 304 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 4.8 MiB/s rd, 17 MiB/s wr, 594 op/s 2024-03-24T10:45:54.825 INFO:tasks.workunit.client.0.smithi094.stdout:6/285: rmdir d0/d5/da/d23/d34 39 2024-03-24T10:45:54.826 INFO:tasks.workunit.client.0.smithi094.stdout:6/286: symlink d0/d21/d2a/l52 0 2024-03-24T10:45:54.828 INFO:tasks.workunit.client.0.smithi094.stdout:6/287: creat d0/d21/d2a/d2f/f53 x:0 0 0 2024-03-24T10:45:54.828 INFO:tasks.workunit.client.0.smithi094.stdout:0/305: write f9 [2360485,126795] 0 2024-03-24T10:45:54.829 INFO:tasks.workunit.client.0.smithi094.stdout:7/308: dread d1/d9/d14/f4c [0,4194304] 0 2024-03-24T10:45:54.829 INFO:tasks.workunit.client.0.smithi094.stdout:7/309: write d1/d9/d16/d20/d60/d39/f48 [688955,1812] 0 2024-03-24T10:45:54.830 INFO:tasks.workunit.client.0.smithi094.stdout:6/288: rename d0/d21/d2a/d2f/f53 to d0/d1a/f54 0 2024-03-24T10:45:54.830 INFO:tasks.workunit.client.0.smithi094.stdout:6/289: fsync d0/d21/d2a/d2f/f46 0 2024-03-24T10:45:54.832 INFO:tasks.workunit.client.0.smithi094.stdout:0/306: unlink d14/d22/l59 0 2024-03-24T10:45:54.832 INFO:tasks.workunit.client.0.smithi094.stdout:0/307: readlink d14/d22/d28/l4c 0 2024-03-24T10:45:54.832 INFO:tasks.workunit.client.0.smithi094.stdout:0/308: read - d14/d18/f69 zero size 2024-03-24T10:45:54.833 INFO:tasks.workunit.client.0.smithi094.stdout:0/309: mknod d14/d22/d28/d2b/c6e 0 2024-03-24T10:45:54.834 INFO:tasks.workunit.client.0.smithi094.stdout:0/310: mknod d14/d22/d28/d2b/d41/d53/c6f 0 2024-03-24T10:45:54.834 INFO:tasks.workunit.client.0.smithi094.stdout:0/311: stat d14/d22/d28/d2b/d41/d53/l64 0 2024-03-24T10:45:54.834 INFO:tasks.workunit.client.0.smithi094.stdout:0/312: dread - d14/d18/f67 zero size 2024-03-24T10:45:54.834 INFO:tasks.workunit.client.0.smithi094.stdout:0/313: stat d14/d22/d28/d2b/d41/d53/c6f 0 2024-03-24T10:45:54.835 INFO:tasks.workunit.client.0.smithi094.stdout:0/314: mkdir d14/d22/d32/d70 0 2024-03-24T10:45:54.835 INFO:tasks.workunit.client.0.smithi094.stdout:0/315: fdatasync d14/d22/d28/f31 0 2024-03-24T10:45:54.837 INFO:tasks.workunit.client.0.smithi094.stdout:0/316: symlink d14/d22/d28/d48/l71 0 2024-03-24T10:45:54.838 INFO:tasks.workunit.client.0.smithi094.stdout:6/290: dread d0/d5/da/f12 [0,4194304] 0 2024-03-24T10:45:54.840 INFO:tasks.workunit.client.0.smithi094.stdout:6/291: unlink d0/d5/da/f12 0 2024-03-24T10:45:54.841 INFO:tasks.workunit.client.0.smithi094.stdout:6/292: mkdir d0/d21/d2a/d55 0 2024-03-24T10:45:54.841 INFO:tasks.workunit.client.0.smithi094.stdout:7/310: dread d1/d9/d14/f4b [0,4194304] 0 2024-03-24T10:45:54.841 INFO:tasks.workunit.client.0.smithi094.stdout:7/311: mknod d1/d9/d35/c67 0 2024-03-24T10:45:54.842 INFO:tasks.workunit.client.0.smithi094.stdout:7/312: fdatasync d1/d9/f13 0 2024-03-24T10:45:54.845 INFO:tasks.workunit.client.0.smithi094.stdout:0/317: dread d14/d18/f21 [0,4194304] 0 2024-03-24T10:45:54.845 INFO:tasks.workunit.client.0.smithi094.stdout:5/308: dwrite d7/d26/f53 [4194304,4194304] 0 2024-03-24T10:45:54.845 INFO:tasks.workunit.client.0.smithi094.stdout:5/309: fdatasync d7/d23/d5f/d9/d48/f50 0 2024-03-24T10:45:54.846 INFO:tasks.workunit.client.0.smithi094.stdout:6/293: write d0/d21/d2a/d2f/f46 [624262,53071] 0 2024-03-24T10:45:54.848 INFO:tasks.workunit.client.0.smithi094.stdout:0/318: creat d14/d22/d28/d2b/d68/f72 x:0 0 0 2024-03-24T10:45:54.850 INFO:tasks.workunit.client.0.smithi094.stdout:5/310: dread d7/dc/d12/f1f [0,4194304] 0 2024-03-24T10:45:54.851 INFO:tasks.workunit.client.0.smithi094.stdout:5/311: mknod d7/dc/d56/c67 0 2024-03-24T10:45:54.852 INFO:tasks.workunit.client.0.smithi094.stdout:5/312: read d7/d23/d5f/fa [65880,36148] 0 2024-03-24T10:45:54.852 INFO:tasks.workunit.client.0.smithi094.stdout:5/313: write d7/d23/d5f/d9/f29 [5062242,89285] 0 2024-03-24T10:45:54.852 INFO:tasks.workunit.client.0.smithi094.stdout:5/314: chown d7/d23/d5f/d9/d48/d51/f52 73 1 2024-03-24T10:45:54.856 INFO:tasks.workunit.client.0.smithi094.stdout:6/294: dread d0/fc [0,4194304] 0 2024-03-24T10:45:54.856 INFO:tasks.workunit.client.0.smithi094.stdout:7/313: dwrite d1/d9/fd [0,4194304] 0 2024-03-24T10:45:54.858 INFO:tasks.workunit.client.0.smithi094.stdout:6/295: truncate d0/f33 225403 0 2024-03-24T10:45:54.858 INFO:tasks.workunit.client.0.smithi094.stdout:6/296: read - d0/d5/f29 zero size 2024-03-24T10:45:54.863 INFO:tasks.workunit.client.0.smithi094.stdout:2/209: sync 2024-03-24T10:45:54.863 INFO:tasks.workunit.client.0.smithi094.stdout:7/314: dread d1/d9/f12 [0,4194304] 0 2024-03-24T10:45:54.863 INFO:tasks.workunit.client.0.smithi094.stdout:0/319: write d14/d22/d32/f5d [2540860,27834] 0 2024-03-24T10:45:54.864 INFO:tasks.workunit.client.0.smithi094.stdout:0/320: mknod d14/d22/c73 0 2024-03-24T10:45:54.865 INFO:tasks.workunit.client.0.smithi094.stdout:1/263: sync 2024-03-24T10:45:54.865 INFO:tasks.workunit.client.0.smithi094.stdout:9/317: sync 2024-03-24T10:45:54.867 INFO:tasks.workunit.client.0.smithi094.stdout:1/264: truncate d4/d9/f2d 4164357 0 2024-03-24T10:45:54.867 INFO:tasks.workunit.client.0.smithi094.stdout:1/265: chown d4/d9/db/f30 1725503 1 2024-03-24T10:45:54.867 INFO:tasks.workunit.client.0.smithi094.stdout:1/266: write d4/d9/d37/d33/f51 [597129,121458] 0 2024-03-24T10:45:54.867 INFO:tasks.workunit.client.0.smithi094.stdout:9/318: rmdir df/d2d/d1f/d11/d43 39 2024-03-24T10:45:54.869 INFO:tasks.workunit.client.0.smithi094.stdout:1/267: mknod d4/d9/db/d39/d43/c55 0 2024-03-24T10:45:54.870 INFO:tasks.workunit.client.0.smithi094.stdout:6/297: dread d0/f1e [0,4194304] 0 2024-03-24T10:45:54.870 INFO:tasks.workunit.client.0.smithi094.stdout:6/298: write d0/d5/da/f2e [697817,70600] 0 2024-03-24T10:45:54.870 INFO:tasks.workunit.client.0.smithi094.stdout:6/299: readlink d0/d5/da/d23/d34/l47 0 2024-03-24T10:45:54.870 INFO:tasks.workunit.client.0.smithi094.stdout:6/300: creat d0/d21/f56 x:0 0 0 2024-03-24T10:45:54.870 INFO:tasks.workunit.client.0.smithi094.stdout:9/319: mkdir df/d2d/d1f/d58/d5a/d61 0 2024-03-24T10:45:54.871 INFO:tasks.workunit.client.0.smithi094.stdout:9/320: creat df/f62 x:0 0 0 2024-03-24T10:45:54.872 INFO:tasks.workunit.client.0.smithi094.stdout:1/268: truncate d4/d9/db/f18 926510 0 2024-03-24T10:45:54.872 INFO:tasks.workunit.client.0.smithi094.stdout:2/210: dread f7 [0,4194304] 0 2024-03-24T10:45:54.872 INFO:tasks.workunit.client.0.smithi094.stdout:2/211: write d18/f2e [645483,100531] 0 2024-03-24T10:45:54.876 INFO:tasks.workunit.client.0.smithi094.stdout:9/321: creat df/d2d/d1f/d58/d5a/f63 x:0 0 0 2024-03-24T10:45:54.878 INFO:tasks.workunit.client.0.smithi094.stdout:2/212: creat d18/d1e/d34/f3e x:0 0 0 2024-03-24T10:45:54.878 INFO:tasks.workunit.client.0.smithi094.stdout:2/213: truncate d18/f27 57952 0 2024-03-24T10:45:54.880 INFO:tasks.workunit.client.0.smithi094.stdout:9/322: rename df/d2d/d34/c60 to df/d2d/d1f/d58/d5a/c64 0 2024-03-24T10:45:54.880 INFO:tasks.workunit.client.0.smithi094.stdout:9/323: chown df/d2d/d34/f39 157606 1 2024-03-24T10:45:54.880 INFO:tasks.workunit.client.0.smithi094.stdout:5/315: dwrite d7/d23/d5f/d14/d36/f4d [0,4194304] 0 2024-03-24T10:45:54.885 INFO:tasks.workunit.client.0.smithi094.stdout:7/315: dwrite d1/d9/d16/d21/d51/f5e [0,4194304] 0 2024-03-24T10:45:54.886 INFO:tasks.workunit.client.0.smithi094.stdout:0/321: dwrite d14/d18/f54 [0,4194304] 0 2024-03-24T10:45:54.886 INFO:tasks.workunit.client.0.smithi094.stdout:0/322: fsync d14/d22/d28/f6c 0 2024-03-24T10:45:54.891 INFO:tasks.workunit.client.0.smithi094.stdout:5/316: rename d7/dc/d62 to d7/d23/d2b/d68 0 2024-03-24T10:45:54.894 INFO:tasks.workunit.client.0.smithi094.stdout:7/316: read d1/d9/fd [3727520,47074] 0 2024-03-24T10:45:54.894 INFO:tasks.workunit.client.0.smithi094.stdout:7/317: readlink d1/d9/d14/l17 0 2024-03-24T10:45:54.897 INFO:tasks.workunit.client.0.smithi094.stdout:0/323: rmdir d14/d22/d28/d2b 39 2024-03-24T10:45:54.897 INFO:tasks.workunit.client.0.smithi094.stdout:0/324: write d14/d22/f35 [627717,109115] 0 2024-03-24T10:45:54.899 INFO:tasks.workunit.client.0.smithi094.stdout:0/325: getdents d14 0 2024-03-24T10:45:54.899 INFO:tasks.workunit.client.0.smithi094.stdout:7/318: write d1/d9/d16/d20/d60/d39/d44/f56 [2615746,56687] 0 2024-03-24T10:45:54.900 INFO:tasks.workunit.client.0.smithi094.stdout:0/326: unlink d14/d22/d28/d2b/d41/l5e 0 2024-03-24T10:45:54.902 INFO:tasks.workunit.client.0.smithi094.stdout:7/319: symlink d1/d9/d14/l68 0 2024-03-24T10:45:54.902 INFO:tasks.workunit.client.0.smithi094.stdout:7/320: creat d1/d9/d16/d20/f69 x:0 0 0 2024-03-24T10:45:54.902 INFO:tasks.workunit.client.0.smithi094.stdout:7/321: chown d1/d9/f3c 51035979 1 2024-03-24T10:45:54.904 INFO:tasks.workunit.client.0.smithi094.stdout:9/324: dwrite df/f41 [0,4194304] 0 2024-03-24T10:45:54.904 INFO:tasks.workunit.client.0.smithi094.stdout:9/325: chown df/d2d/l1d 815 1 2024-03-24T10:45:54.904 INFO:tasks.workunit.client.0.smithi094.stdout:9/326: truncate df/d2d/f44 353006 0 2024-03-24T10:45:54.905 INFO:tasks.workunit.client.0.smithi094.stdout:2/214: dwrite d18/d1e/d34/f3e [0,4194304] 0 2024-03-24T10:45:54.905 INFO:tasks.workunit.client.0.smithi094.stdout:2/215: write d18/f38 [119796,43834] 0 2024-03-24T10:45:54.905 INFO:tasks.workunit.client.0.smithi094.stdout:2/216: fsync d18/f3b 0 2024-03-24T10:45:54.905 INFO:tasks.workunit.client.0.smithi094.stdout:1/269: truncate f1 3225628 0 2024-03-24T10:45:54.905 INFO:tasks.workunit.client.0.smithi094.stdout:1/270: stat d4/d9/f2d 0 2024-03-24T10:45:54.909 INFO:tasks.workunit.client.0.smithi094.stdout:9/327: stat cc 0 2024-03-24T10:45:54.910 INFO:tasks.workunit.client.0.smithi094.stdout:9/328: dread - df/d2d/d1f/d58/d5a/f63 zero size 2024-03-24T10:45:54.914 INFO:tasks.workunit.client.0.smithi094.stdout:1/271: symlink d4/d9/db/d31/l56 0 2024-03-24T10:45:54.915 INFO:tasks.workunit.client.0.smithi094.stdout:9/329: mkdir df/d2d/d1f/d58/d5a/d65 0 2024-03-24T10:45:54.916 INFO:tasks.workunit.client.0.smithi094.stdout:2/217: getdents d18/d1e/d34 0 2024-03-24T10:45:54.918 INFO:tasks.workunit.client.0.smithi094.stdout:9/330: unlink cc 0 2024-03-24T10:45:54.919 INFO:tasks.workunit.client.0.smithi094.stdout:1/272: write d4/d9/db/f1d [1510362,108231] 0 2024-03-24T10:45:54.919 INFO:tasks.workunit.client.0.smithi094.stdout:4/341: sync 2024-03-24T10:45:54.919 INFO:tasks.workunit.client.0.smithi094.stdout:3/257: sync 2024-03-24T10:45:54.919 INFO:tasks.workunit.client.0.smithi094.stdout:8/337: sync 2024-03-24T10:45:54.919 INFO:tasks.workunit.client.0.smithi094.stdout:3/258: truncate d0/db/dc/f12 1383448 0 2024-03-24T10:45:54.920 INFO:tasks.workunit.client.0.smithi094.stdout:2/218: creat d18/f3f x:0 0 0 2024-03-24T10:45:54.921 INFO:tasks.workunit.client.0.smithi094.stdout:1/273: mknod d4/d9/d47/c57 0 2024-03-24T10:45:54.921 INFO:tasks.workunit.client.0.smithi094.stdout:1/274: chown d4/d9/db/f1d 3 1 2024-03-24T10:45:54.921 INFO:tasks.workunit.client.0.smithi094.stdout:1/275: write d4/d9/db/f42 [1677889,99831] 0 2024-03-24T10:45:54.923 INFO:tasks.workunit.client.0.smithi094.stdout:4/342: rename d0/d22/l1d to d0/d22/de/d28/d2b/d33/l68 0 2024-03-24T10:45:54.925 INFO:tasks.workunit.client.0.smithi094.stdout:3/259: mkdir d0/db/dc/d28/d50/d5e 0 2024-03-24T10:45:54.925 INFO:tasks.workunit.client.0.smithi094.stdout:7/322: dwrite d1/d9/fd [0,4194304] 0 2024-03-24T10:45:54.926 INFO:tasks.workunit.client.0.smithi094.stdout:0/327: dwrite d14/d22/d28/d2b/f45 [0,4194304] 0 2024-03-24T10:45:54.926 INFO:tasks.workunit.client.0.smithi094.stdout:2/219: dread d18/f2e [0,4194304] 0 2024-03-24T10:45:54.926 INFO:tasks.workunit.client.0.smithi094.stdout:0/328: write fd [63227,36854] 0 2024-03-24T10:45:54.926 INFO:tasks.workunit.client.0.smithi094.stdout:1/276: unlink d4/l6 0 2024-03-24T10:45:54.926 INFO:tasks.workunit.client.0.smithi094.stdout:0/329: chown d14/d22/d28/d2b/d41/d53/l64 122000 1 2024-03-24T10:45:54.926 INFO:tasks.workunit.client.0.smithi094.stdout:2/220: write d18/d1e/f25 [4636319,121882] 0 2024-03-24T10:45:54.927 INFO:tasks.workunit.client.0.smithi094.stdout:8/338: dread f6 [0,4194304] 0 2024-03-24T10:45:54.929 INFO:tasks.workunit.client.0.smithi094.stdout:1/277: getdents d4/d9/db/d31/d3c 0 2024-03-24T10:45:54.929 INFO:tasks.workunit.client.0.smithi094.stdout:7/323: symlink d1/d9/d14/l6a 0 2024-03-24T10:45:54.930 INFO:tasks.workunit.client.0.smithi094.stdout:0/330: symlink d14/d22/d28/d2b/d41/d53/l74 0 2024-03-24T10:45:54.932 INFO:tasks.workunit.client.0.smithi094.stdout:4/343: dread d0/f4b [0,4194304] 0 2024-03-24T10:45:54.932 INFO:tasks.workunit.client.0.smithi094.stdout:4/344: write d0/d22/de/d28/d3d/f5c [547904,111296] 0 2024-03-24T10:45:54.932 INFO:tasks.workunit.client.0.smithi094.stdout:4/345: chown d0/d22/de/d28/d2b/f51 88 1 2024-03-24T10:45:54.932 INFO:tasks.workunit.client.0.smithi094.stdout:7/324: truncate d1/d9/f3c 1326713 0 2024-03-24T10:45:54.932 INFO:tasks.workunit.client.0.smithi094.stdout:7/325: stat d1/d9 0 2024-03-24T10:45:54.933 INFO:tasks.workunit.client.0.smithi094.stdout:1/278: mkdir d4/d9/d37/d33/d58 0 2024-03-24T10:45:54.934 INFO:tasks.workunit.client.0.smithi094.stdout:4/346: rename d0/f6 to d0/d22/de/d28/d2b/f69 0 2024-03-24T10:45:54.934 INFO:tasks.workunit.client.0.smithi094.stdout:4/347: chown d0/d34/l5b 34237425 1 2024-03-24T10:45:54.935 INFO:tasks.workunit.client.0.smithi094.stdout:4/348: fdatasync d0/d22/de/d28/d2b/d33/f54 0 2024-03-24T10:45:54.935 INFO:tasks.workunit.client.0.smithi094.stdout:7/326: creat d1/d9/d35/f6b x:0 0 0 2024-03-24T10:45:54.935 INFO:tasks.workunit.client.0.smithi094.stdout:7/327: readlink d1/d9/d16/l1a 0 2024-03-24T10:45:54.935 INFO:tasks.workunit.client.0.smithi094.stdout:7/328: stat d1/d9/d16/d20 0 2024-03-24T10:45:54.936 INFO:tasks.workunit.client.0.smithi094.stdout:1/279: symlink d4/d9/d47/l59 0 2024-03-24T10:45:54.936 INFO:tasks.workunit.client.0.smithi094.stdout:3/260: dread d0/db/d1c/d20/d23/f29 [0,4194304] 0 2024-03-24T10:45:54.937 INFO:tasks.workunit.client.0.smithi094.stdout:1/280: mknod d4/d9/d37/d33/d58/c5a 0 2024-03-24T10:45:54.937 INFO:tasks.workunit.client.0.smithi094.stdout:1/281: creat d4/d9/d37/d33/f5b x:0 0 0 2024-03-24T10:45:54.940 INFO:tasks.workunit.client.0.smithi094.stdout:9/331: dwrite df/d2d/d34/f39 [0,4194304] 0 2024-03-24T10:45:54.940 INFO:tasks.workunit.client.0.smithi094.stdout:9/332: write df/d2d/d1f/d11/f13 [3340886,64222] 0 2024-03-24T10:45:54.941 INFO:tasks.workunit.client.0.smithi094.stdout:1/282: symlink d4/d9/d37/l5c 0 2024-03-24T10:45:54.945 INFO:tasks.workunit.client.0.smithi094.stdout:0/331: dread d14/d22/d28/d2b/f45 [0,4194304] 0 2024-03-24T10:45:54.949 INFO:tasks.workunit.client.0.smithi094.stdout:9/333: unlink df/f41 0 2024-03-24T10:45:54.950 INFO:tasks.workunit.client.0.smithi094.stdout:1/283: mkdir d4/d9/db/d5d 0 2024-03-24T10:45:54.955 INFO:tasks.workunit.client.0.smithi094.stdout:4/349: rmdir d0/d22/de 39 2024-03-24T10:45:54.958 INFO:tasks.workunit.client.0.smithi094.stdout:9/334: rmdir df/d2d/d1f/d58/d5a/d65 0 2024-03-24T10:45:54.959 INFO:tasks.workunit.client.0.smithi094.stdout:1/284: mknod d4/d9/db/d31/c5e 0 2024-03-24T10:45:54.960 INFO:tasks.workunit.client.0.smithi094.stdout:4/350: rmdir d0/d22/de/d28/d2b 39 2024-03-24T10:45:54.961 INFO:tasks.workunit.client.0.smithi094.stdout:4/351: mknod d0/d22/c6a 0 2024-03-24T10:45:54.965 INFO:tasks.workunit.client.0.smithi094.stdout:0/332: dwrite d14/d18/f40 [0,4194304] 0 2024-03-24T10:45:54.969 INFO:tasks.workunit.client.0.smithi094.stdout:0/333: creat d14/f75 x:0 0 0 2024-03-24T10:45:54.970 INFO:tasks.workunit.client.0.smithi094.stdout:0/334: mknod d14/d22/d28/d2b/d68/c76 0 2024-03-24T10:45:54.971 INFO:tasks.workunit.client.0.smithi094.stdout:0/335: mknod d14/d22/d32/c77 0 2024-03-24T10:45:54.977 INFO:tasks.workunit.client.0.smithi094.stdout:7/329: dwrite d1/d9/fd [0,4194304] 0 2024-03-24T10:45:54.978 INFO:tasks.workunit.client.0.smithi094.stdout:4/352: dwrite d0/d22/f62 [0,4194304] 0 2024-03-24T10:45:54.988 INFO:tasks.workunit.client.0.smithi094.stdout:6/301: sync 2024-03-24T10:45:54.989 INFO:tasks.workunit.client.0.smithi094.stdout:6/302: symlink d0/d5/da/l57 0 2024-03-24T10:45:54.991 INFO:tasks.workunit.client.0.smithi094.stdout:0/336: dwrite d14/d22/d28/d2b/f3b [0,4194304] 0 2024-03-24T10:45:54.991 INFO:tasks.workunit.client.0.smithi094.stdout:6/303: rename d0/f42 to d0/d5/f58 0 2024-03-24T10:45:54.994 INFO:tasks.workunit.client.0.smithi094.stdout:6/304: symlink d0/d5/da/d23/l59 0 2024-03-24T10:45:55.001 INFO:tasks.workunit.client.0.smithi094.stdout:0/337: dread d14/d22/d28/d2b/f3b [0,4194304] 0 2024-03-24T10:45:55.002 INFO:tasks.workunit.client.0.smithi094.stdout:0/338: chown d14/d18/c26 444 1 2024-03-24T10:45:55.002 INFO:tasks.workunit.client.0.smithi094.stdout:0/339: write f0 [3290974,51989] 0 2024-03-24T10:45:55.002 INFO:tasks.workunit.client.0.smithi094.stdout:6/305: read d0/f3c [3323806,123523] 0 2024-03-24T10:45:55.002 INFO:tasks.workunit.client.0.smithi094.stdout:6/306: creat d0/d5/f5a x:0 0 0 2024-03-24T10:45:55.002 INFO:tasks.workunit.client.0.smithi094.stdout:7/330: dwrite d1/d9/d16/d21/d51/f5e [4194304,4194304] 0 2024-03-24T10:45:55.003 INFO:tasks.workunit.client.0.smithi094.stdout:4/353: dwrite d0/d22/f62 [0,4194304] 0 2024-03-24T10:45:55.003 INFO:tasks.workunit.client.0.smithi094.stdout:0/340: creat d14/d22/d28/d48/f78 x:0 0 0 2024-03-24T10:45:55.005 INFO:tasks.workunit.client.0.smithi094.stdout:6/307: creat d0/d30/f5b x:0 0 0 2024-03-24T10:45:55.008 INFO:tasks.workunit.client.0.smithi094.stdout:7/331: mkdir d1/d6c 0 2024-03-24T10:45:55.009 INFO:tasks.workunit.client.0.smithi094.stdout:4/354: write d0/d5/d1b/f39 [644146,70068] 0 2024-03-24T10:45:55.009 INFO:tasks.workunit.client.0.smithi094.stdout:4/355: write d0/d22/f62 [4651991,99851] 0 2024-03-24T10:45:55.012 INFO:tasks.workunit.client.0.smithi094.stdout:6/308: dread d0/d21/f2d [0,4194304] 0 2024-03-24T10:45:55.012 INFO:tasks.workunit.client.0.smithi094.stdout:6/309: creat d0/d21/f5c x:0 0 0 2024-03-24T10:45:55.016 INFO:tasks.workunit.client.0.smithi094.stdout:6/310: dread d0/d21/f2d [0,4194304] 0 2024-03-24T10:45:55.017 INFO:tasks.workunit.client.0.smithi094.stdout:6/311: mkdir d0/d5/da/d23/d34/d4e/d5d 0 2024-03-24T10:45:55.018 INFO:tasks.workunit.client.0.smithi094.stdout:6/312: unlink d0/d5/da/d23/l59 0 2024-03-24T10:45:55.018 INFO:tasks.workunit.client.0.smithi094.stdout:6/313: fsync d0/d1a/f39 0 2024-03-24T10:45:55.019 INFO:tasks.workunit.client.0.smithi094.stdout:6/314: mknod d0/d5/c5e 0 2024-03-24T10:45:55.025 INFO:tasks.workunit.client.0.smithi094.stdout:4/356: dwrite d0/d22/de/d28/f52 [0,4194304] 0 2024-03-24T10:45:55.031 INFO:tasks.workunit.client.0.smithi094.stdout:6/315: dwrite d0/d1a/f54 [0,4194304] 0 2024-03-24T10:45:55.036 INFO:tasks.workunit.client.0.smithi094.stdout:6/316: rmdir d0/d5/da 39 2024-03-24T10:45:55.036 INFO:tasks.workunit.client.0.smithi094.stdout:4/357: write d0/d22/f62 [3294922,60790] 0 2024-03-24T10:45:55.037 INFO:tasks.workunit.client.0.smithi094.stdout:4/358: dread d0/d22/de/d28/d3d/f5c [0,4194304] 0 2024-03-24T10:45:55.041 INFO:tasks.workunit.client.0.smithi094.stdout:4/359: dread d0/d34/f48 [4194304,4194304] 0 2024-03-24T10:45:55.041 INFO:tasks.workunit.client.0.smithi094.stdout:4/360: fsync d0/d5/d1b/f2e 0 2024-03-24T10:45:55.041 INFO:tasks.workunit.client.0.smithi094.stdout:4/361: truncate d0/d1f/f55 951583 0 2024-03-24T10:45:55.056 INFO:tasks.workunit.client.0.smithi094.stdout:4/362: dwrite d0/d5/d42/d4a/f61 [4194304,4194304] 0 2024-03-24T10:45:55.057 INFO:tasks.workunit.client.0.smithi094.stdout:4/363: rmdir d0/d22/de/d28 39 2024-03-24T10:45:55.057 INFO:tasks.workunit.client.0.smithi094.stdout:4/364: truncate d0/d22/ff 1981259 0 2024-03-24T10:45:55.058 INFO:tasks.workunit.client.0.smithi094.stdout:4/365: creat d0/d22/de/d28/f6b x:0 0 0 2024-03-24T10:45:55.059 INFO:tasks.workunit.client.0.smithi094.stdout:4/366: mknod d0/c6c 0 2024-03-24T10:45:55.059 INFO:tasks.workunit.client.0.smithi094.stdout:4/367: mkdir d0/d22/de/d28/d2b/d33/d5d/d6d 0 2024-03-24T10:45:55.061 INFO:tasks.workunit.client.0.smithi094.stdout:4/368: mknod d0/c6e 0 2024-03-24T10:45:55.062 INFO:tasks.workunit.client.0.smithi094.stdout:4/369: mknod d0/d5/d42/d4a/c6f 0 2024-03-24T10:45:55.063 INFO:tasks.workunit.client.0.smithi094.stdout:4/370: dread d0/d5/d1b/f2a [0,4194304] 0 2024-03-24T10:45:55.066 INFO:tasks.workunit.client.0.smithi094.stdout:4/371: rename d0/d22/de/d28/d2b/d33/d5d to d0/d5/d1b/d4f/d70 0 2024-03-24T10:45:55.066 INFO:tasks.workunit.client.0.smithi094.stdout:4/372: dread - d0/f5f zero size 2024-03-24T10:45:55.072 INFO:tasks.workunit.client.0.smithi094.stdout:4/373: write d0/d22/de/d28/f59 [580051,14211] 0 2024-03-24T10:45:55.077 INFO:tasks.workunit.client.0.smithi094.stdout:4/374: write d0/f47 [2246274,38857] 0 2024-03-24T10:45:55.077 INFO:tasks.workunit.client.0.smithi094.stdout:4/375: write d0/d22/de/d28/f59 [2721832,37988] 0 2024-03-24T10:45:55.079 INFO:tasks.workunit.client.0.smithi094.stdout:4/376: rename d0/d5/c56 to d0/c71 0 2024-03-24T10:45:55.079 INFO:tasks.workunit.client.0.smithi094.stdout:4/377: fdatasync d0/d1f/f55 0 2024-03-24T10:45:55.084 INFO:tasks.workunit.client.0.smithi094.stdout:4/378: dread d0/d22/de/d28/f59 [0,4194304] 0 2024-03-24T10:45:55.091 INFO:tasks.workunit.client.0.smithi094.stdout:5/317: sync 2024-03-24T10:45:55.106 INFO:tasks.workunit.client.0.smithi094.stdout:5/318: dwrite d7/d26/f53 [0,4194304] 0 2024-03-24T10:45:55.106 INFO:tasks.workunit.client.0.smithi094.stdout:5/319: fsync d7/f10 0 2024-03-24T10:45:55.106 INFO:tasks.workunit.client.0.smithi094.stdout:5/320: mknod d7/d26/c69 0 2024-03-24T10:45:55.107 INFO:tasks.workunit.client.0.smithi094.stdout:5/321: creat d7/d23/d5f/d9/d48/d51/f6a x:0 0 0 2024-03-24T10:45:55.118 INFO:tasks.workunit.client.0.smithi094.stdout:5/322: write d7/d23/d5f/d14/d36/f4d [927963,18600] 0 2024-03-24T10:45:55.120 INFO:tasks.workunit.client.0.smithi094.stdout:5/323: chown d7/d23/d5f/d14/l20 514460756 1 2024-03-24T10:45:55.120 INFO:tasks.workunit.client.0.smithi094.stdout:5/324: write d7/d23/d5f/d9/d48/d51/f6a [624083,59312] 0 2024-03-24T10:45:55.120 INFO:tasks.workunit.client.0.smithi094.stdout:5/325: truncate d7/d26/f5c 312592 0 2024-03-24T10:45:55.120 INFO:tasks.workunit.client.0.smithi094.stdout:5/326: chown d7/d23/d5f/d9/d48/d51 1002 1 2024-03-24T10:45:55.120 INFO:tasks.workunit.client.0.smithi094.stdout:5/327: fsync d7/d23/d2b/f5a 0 2024-03-24T10:45:55.120 INFO:tasks.workunit.client.0.smithi094.stdout:5/328: creat d7/d23/d5f/d9/d48/f6b x:0 0 0 2024-03-24T10:45:55.138 INFO:tasks.workunit.client.0.smithi094.stdout:5/329: dwrite d7/d23/d5f/d14/d36/f47 [0,4194304] 0 2024-03-24T10:45:55.142 INFO:tasks.workunit.client.0.smithi094.stdout:5/330: mknod d7/c6c 0 2024-03-24T10:45:55.147 INFO:tasks.workunit.client.0.smithi094.stdout:5/331: dread d7/d23/d5f/d9/f29 [0,4194304] 0 2024-03-24T10:45:55.147 INFO:tasks.workunit.client.0.smithi094.stdout:5/332: readlink d7/dc/l5b 0 2024-03-24T10:45:55.147 INFO:tasks.workunit.client.0.smithi094.stdout:2/221: sync 2024-03-24T10:45:55.148 INFO:tasks.workunit.client.0.smithi094.stdout:5/333: creat d7/d23/d5f/d14/f6d x:0 0 0 2024-03-24T10:45:55.149 INFO:tasks.workunit.client.0.smithi094.stdout:5/334: creat d7/dc/d56/f6e x:0 0 0 2024-03-24T10:45:55.149 INFO:tasks.workunit.client.0.smithi094.stdout:5/335: fdatasync d7/d23/d2b/f5a 0 2024-03-24T10:45:55.149 INFO:tasks.workunit.client.0.smithi094.stdout:5/336: write d7/dc/d56/f6e [227914,49426] 0 2024-03-24T10:45:55.150 INFO:tasks.workunit.client.0.smithi094.stdout:2/222: dread d18/d1e/f33 [0,4194304] 0 2024-03-24T10:45:55.156 INFO:tasks.workunit.client.0.smithi094.stdout:2/223: write d18/d1e/f25 [2954633,22151] 0 2024-03-24T10:45:55.156 INFO:tasks.workunit.client.0.smithi094.stdout:2/224: write d18/d1e/d34/f35 [51312,83885] 0 2024-03-24T10:45:55.158 INFO:tasks.workunit.client.0.smithi094.stdout:2/225: dread d18/d1e/f33 [0,4194304] 0 2024-03-24T10:45:55.165 INFO:tasks.workunit.client.0.smithi094.stdout:5/337: dwrite d7/d23/d2b/f5a [0,4194304] 0 2024-03-24T10:45:55.165 INFO:tasks.workunit.client.0.smithi094.stdout:5/338: chown d7/dc/d12/f1f 3 1 2024-03-24T10:45:55.165 INFO:tasks.workunit.client.0.smithi094.stdout:5/339: chown d7/d23/d5f/d1b 243529777 1 2024-03-24T10:45:55.168 INFO:tasks.workunit.client.0.smithi094.stdout:5/340: mkdir d7/d23/d5f/d14/d6f 0 2024-03-24T10:45:55.168 INFO:tasks.workunit.client.0.smithi094.stdout:5/341: dread - d7/d26/f65 zero size 2024-03-24T10:45:55.169 INFO:tasks.workunit.client.0.smithi094.stdout:5/342: symlink d7/d23/d5f/d14/d36/l70 0 2024-03-24T10:45:55.170 INFO:tasks.workunit.client.0.smithi094.stdout:5/343: read - d7/dc/d12/f61 zero size 2024-03-24T10:45:55.172 INFO:tasks.workunit.client.0.smithi094.stdout:2/226: dwrite d18/d1e/d34/f3d [0,4194304] 0 2024-03-24T10:45:55.174 INFO:tasks.workunit.client.0.smithi094.stdout:2/227: write d18/f2e [536319,62678] 0 2024-03-24T10:45:55.176 INFO:tasks.workunit.client.0.smithi094.stdout:2/228: rename d18/d1e/l21 to d18/d1e/l40 0 2024-03-24T10:45:55.180 INFO:tasks.workunit.client.0.smithi094.stdout:2/229: rename cd to d18/d1e/c41 0 2024-03-24T10:45:55.181 INFO:tasks.workunit.client.0.smithi094.stdout:2/230: mknod d18/d1e/d34/c42 0 2024-03-24T10:45:55.183 INFO:tasks.workunit.client.0.smithi094.stdout:2/231: creat d18/d1e/d34/f43 x:0 0 0 2024-03-24T10:45:55.183 INFO:tasks.workunit.client.0.smithi094.stdout:5/344: dwrite d7/d23/d5f/d1b/f37 [4194304,4194304] 0 2024-03-24T10:45:55.184 INFO:tasks.workunit.client.0.smithi094.stdout:2/232: link d18/f19 d18/d1e/d34/f44 0 2024-03-24T10:45:55.185 INFO:tasks.workunit.client.0.smithi094.stdout:9/335: sync 2024-03-24T10:45:55.185 INFO:tasks.workunit.client.0.smithi094.stdout:4/379: sync 2024-03-24T10:45:55.185 INFO:tasks.workunit.client.0.smithi094.stdout:7/332: sync 2024-03-24T10:45:55.185 INFO:tasks.workunit.client.0.smithi094.stdout:1/285: sync 2024-03-24T10:45:55.185 INFO:tasks.workunit.client.0.smithi094.stdout:8/339: sync 2024-03-24T10:45:55.185 INFO:tasks.workunit.client.0.smithi094.stdout:7/333: chown d1/d9/d14/l6a 5281539 1 2024-03-24T10:45:55.185 INFO:tasks.workunit.client.0.smithi094.stdout:8/340: readlink df/d25/d28/l2e 0 2024-03-24T10:45:55.185 INFO:tasks.workunit.client.0.smithi094.stdout:3/261: sync 2024-03-24T10:45:55.185 INFO:tasks.workunit.client.0.smithi094.stdout:4/380: creat d0/d22/de/d28/f72 x:0 0 0 2024-03-24T10:45:55.185 INFO:tasks.workunit.client.0.smithi094.stdout:0/341: sync 2024-03-24T10:45:55.185 INFO:tasks.workunit.client.0.smithi094.stdout:6/317: sync 2024-03-24T10:45:55.187 INFO:tasks.workunit.client.0.smithi094.stdout:9/336: unlink df/d2d/d1f/d58/l5b 0 2024-03-24T10:45:55.187 INFO:tasks.workunit.client.0.smithi094.stdout:9/337: dread - df/d2d/d34/d2c/f5f zero size 2024-03-24T10:45:55.187 INFO:tasks.workunit.client.0.smithi094.stdout:9/338: write df/d2d/f15 [437005,2710] 0 2024-03-24T10:45:55.188 INFO:tasks.workunit.client.0.smithi094.stdout:1/286: mkdir d4/d36/d38/d5f 0 2024-03-24T10:45:55.190 INFO:tasks.workunit.client.0.smithi094.stdout:8/341: mknod df/d25/d36/d3f/c71 0 2024-03-24T10:45:55.191 INFO:tasks.workunit.client.0.smithi094.stdout:7/334: creat d1/d9/d25/d3d/f6d x:0 0 0 2024-03-24T10:45:55.191 INFO:tasks.workunit.client.0.smithi094.stdout:5/345: read d7/d23/d5f/d14/d36/f47 [3453522,111881] 0 2024-03-24T10:45:55.191 INFO:tasks.workunit.client.0.smithi094.stdout:5/346: write d7/d23/d5f/d9/d48/d51/f52 [597091,32034] 0 2024-03-24T10:45:55.192 INFO:tasks.workunit.client.0.smithi094.stdout:4/381: mkdir d0/d34/d73 0 2024-03-24T10:45:55.192 INFO:tasks.workunit.client.0.smithi094.stdout:4/382: fsync d0/d22/de/f30 0 2024-03-24T10:45:55.194 INFO:tasks.workunit.client.0.smithi094.stdout:3/262: creat d0/db/d1c/d20/d23/f5f x:0 0 0 2024-03-24T10:45:55.197 INFO:tasks.workunit.client.0.smithi094.stdout:6/318: creat d0/d3b/f5f x:0 0 0 2024-03-24T10:45:55.197 INFO:tasks.workunit.client.0.smithi094.stdout:6/319: fdatasync d0/d21/d2a/d2f/f36 0 2024-03-24T10:45:55.199 INFO:tasks.workunit.client.0.smithi094.stdout:1/287: unlink d4/f46 0 2024-03-24T10:45:55.201 INFO:tasks.workunit.client.0.smithi094.stdout:8/342: rename f5 to df/d25/d28/d31/f72 0 2024-03-24T10:45:55.201 INFO:tasks.workunit.client.0.smithi094.stdout:7/335: creat d1/d9/d14/f6e x:0 0 0 2024-03-24T10:45:55.202 INFO:tasks.workunit.client.0.smithi094.stdout:5/347: symlink d7/d23/d5f/l71 0 2024-03-24T10:45:55.203 INFO:tasks.workunit.client.0.smithi094.stdout:9/339: dread df/d2d/d34/d2c/f57 [4194304,4194304] 0 2024-03-24T10:45:55.203 INFO:tasks.workunit.client.0.smithi094.stdout:6/320: dread d0/d21/f2b [0,4194304] 0 2024-03-24T10:45:55.203 INFO:tasks.workunit.client.0.smithi094.stdout:4/383: symlink d0/d34/d73/l74 0 2024-03-24T10:45:55.203 INFO:tasks.workunit.client.0.smithi094.stdout:4/384: fsync d0/d22/ff 0 2024-03-24T10:45:55.204 INFO:tasks.workunit.client.0.smithi094.stdout:3/263: link d0/db/d1c/d20/d23/f29 d0/db/dc/d28/d50/f60 0 2024-03-24T10:45:55.205 INFO:tasks.workunit.client.0.smithi094.stdout:8/343: getdents df/d25/d36 0 2024-03-24T10:45:55.205 INFO:tasks.workunit.client.0.smithi094.stdout:8/344: readlink lb 0 2024-03-24T10:45:55.205 INFO:tasks.workunit.client.0.smithi094.stdout:8/345: chown f3 5664389 1 2024-03-24T10:45:55.206 INFO:tasks.workunit.client.0.smithi094.stdout:5/348: mknod d7/d23/d5f/c72 0 2024-03-24T10:45:55.206 INFO:tasks.workunit.client.0.smithi094.stdout:5/349: write d7/d26/f53 [9342609,70418] 0 2024-03-24T10:45:55.206 INFO:tasks.workunit.client.0.smithi094.stdout:9/340: truncate df/d2d/d34/d2c/f4c 2839116 0 2024-03-24T10:45:55.207 INFO:tasks.workunit.client.0.smithi094.stdout:9/341: read - df/f62 zero size 2024-03-24T10:45:55.207 INFO:tasks.workunit.client.0.smithi094.stdout:9/342: chown df/f14 13121 1 2024-03-24T10:45:55.208 INFO:tasks.workunit.client.0.smithi094.stdout:6/321: mknod d0/d5/c60 0 2024-03-24T10:45:55.208 INFO:tasks.workunit.client.0.smithi094.stdout:6/322: dread - d0/d21/f5c zero size 2024-03-24T10:45:55.208 INFO:tasks.workunit.client.0.smithi094.stdout:6/323: chown d0/d1a/l3e 48103070 1 2024-03-24T10:45:55.208 INFO:tasks.workunit.client.0.smithi094.stdout:6/324: truncate d0/d21/d2a/d2f/f46 1204565 0 2024-03-24T10:45:55.209 INFO:tasks.workunit.client.0.smithi094.stdout:3/264: stat d0/db/c32 0 2024-03-24T10:45:55.210 INFO:tasks.workunit.client.0.smithi094.stdout:5/350: truncate d7/d23/d5f/fa 790976 0 2024-03-24T10:45:55.210 INFO:tasks.workunit.client.0.smithi094.stdout:7/336: write d1/d9/d14/f4c [310353,42412] 0 2024-03-24T10:45:55.210 INFO:tasks.workunit.client.0.smithi094.stdout:4/385: dread d0/d5/d1b/f2e [0,4194304] 0 2024-03-24T10:45:55.211 INFO:tasks.workunit.client.0.smithi094.stdout:9/343: mkdir df/d2d/d1f/d11/d66 0 2024-03-24T10:45:55.212 INFO:tasks.workunit.client.0.smithi094.stdout:6/325: creat d0/f61 x:0 0 0 2024-03-24T10:45:55.212 INFO:tasks.workunit.client.0.smithi094.stdout:6/326: readlink d0/d5/da/d23/d34/l47 0 2024-03-24T10:45:55.212 INFO:tasks.workunit.client.0.smithi094.stdout:6/327: fdatasync d0/f2 0 2024-03-24T10:45:55.212 INFO:tasks.workunit.client.0.smithi094.stdout:6/328: chown d0/d1a/f54 2 1 2024-03-24T10:45:55.212 INFO:tasks.workunit.client.0.smithi094.stdout:3/265: rmdir d0/db/d1c/d20/d23/d2a/d3a 39 2024-03-24T10:45:55.214 INFO:tasks.workunit.client.0.smithi094.stdout:7/337: unlink d1/d9/d16/l1a 0 2024-03-24T10:45:55.214 INFO:tasks.workunit.client.0.smithi094.stdout:7/338: creat d1/d9/d16/d20/f6f x:0 0 0 2024-03-24T10:45:55.215 INFO:tasks.workunit.client.0.smithi094.stdout:5/351: dread d7/d23/d5f/d9/d48/f50 [0,4194304] 0 2024-03-24T10:45:55.215 INFO:tasks.workunit.client.0.smithi094.stdout:4/386: truncate d0/d5/d1b/f2e 1204064 0 2024-03-24T10:45:55.216 INFO:tasks.workunit.client.0.smithi094.stdout:9/344: link df/d2d/d1f/l5d df/d2d/d1f/l67 0 2024-03-24T10:45:55.217 INFO:tasks.workunit.client.0.smithi094.stdout:6/329: mknod d0/d30/d50/d4d/c62 0 2024-03-24T10:45:55.217 INFO:tasks.workunit.client.0.smithi094.stdout:6/330: read d0/d21/d2a/d2f/f36 [2233815,32737] 0 2024-03-24T10:45:55.218 INFO:tasks.workunit.client.0.smithi094.stdout:3/266: unlink d0/db/dc/c4e 0 2024-03-24T10:45:55.221 INFO:tasks.workunit.client.0.smithi094.stdout:5/352: unlink d7/d23/d5f/d14/l20 0 2024-03-24T10:45:55.221 INFO:tasks.workunit.client.0.smithi094.stdout:5/353: readlink d7/d26/l22 0 2024-03-24T10:45:55.222 INFO:tasks.workunit.client.0.smithi094.stdout:6/331: mkdir d0/d21/d2a/d55/d63 0 2024-03-24T10:45:55.222 INFO:tasks.workunit.client.0.smithi094.stdout:5/354: mkdir d7/d23/d5f/d9/d48/d73 0 2024-03-24T10:45:55.222 INFO:tasks.workunit.client.0.smithi094.stdout:5/355: dread - d7/d23/d5f/d9/f4f zero size 2024-03-24T10:45:55.223 INFO:tasks.workunit.client.0.smithi094.stdout:6/332: mknod d0/d5/c64 0 2024-03-24T10:45:55.225 INFO:tasks.workunit.client.0.smithi094.stdout:6/333: symlink d0/d1a/l65 0 2024-03-24T10:45:55.225 INFO:tasks.workunit.client.0.smithi094.stdout:6/334: dread - d0/d21/f56 zero size 2024-03-24T10:45:55.228 INFO:tasks.workunit.client.0.smithi094.stdout:8/346: dwrite df/d25/d36/d3f/f4f [0,4194304] 0 2024-03-24T10:45:55.229 INFO:tasks.workunit.client.0.smithi094.stdout:2/233: dwrite d18/f27 [0,4194304] 0 2024-03-24T10:45:55.229 INFO:tasks.workunit.client.0.smithi094.stdout:2/234: write d18/d1e/d34/f35 [406532,26394] 0 2024-03-24T10:45:55.229 INFO:tasks.workunit.client.0.smithi094.stdout:2/235: creat d18/f45 x:0 0 0 2024-03-24T10:45:55.229 INFO:tasks.workunit.client.0.smithi094.stdout:2/236: creat d18/d1e/d34/f46 x:0 0 0 2024-03-24T10:45:55.229 INFO:tasks.workunit.client.0.smithi094.stdout:9/345: write df/f14 [5067722,24144] 0 2024-03-24T10:45:55.230 INFO:tasks.workunit.client.0.smithi094.stdout:9/346: fsync df/f36 0 2024-03-24T10:45:55.230 INFO:tasks.workunit.client.0.smithi094.stdout:1/288: dwrite d4/d9/db/d39/f49 [0,4194304] 0 2024-03-24T10:45:55.233 INFO:tasks.workunit.client.0.smithi094.stdout:5/356: dread d7/dc/d12/f1f [0,4194304] 0 2024-03-24T10:45:55.233 INFO:tasks.workunit.client.0.smithi094.stdout:0/342: dwrite d14/d22/d28/d2b/d68/f72 [0,4194304] 0 2024-03-24T10:45:55.233 INFO:tasks.workunit.client.0.smithi094.stdout:5/357: creat d7/d23/d5f/d9/d48/f74 x:0 0 0 2024-03-24T10:45:55.233 INFO:tasks.workunit.client.0.smithi094.stdout:5/358: dread - d7/f4c zero size 2024-03-24T10:45:55.242 INFO:tasks.workunit.client.0.smithi094.stdout:2/237: rename d18/f29 to d18/f47 0 2024-03-24T10:45:55.242 INFO:tasks.workunit.client.0.smithi094.stdout:2/238: truncate d18/d1e/d34/f35 980749 0 2024-03-24T10:45:55.247 INFO:tasks.workunit.client.0.smithi094.stdout:1/289: symlink d4/d9/de/l60 0 2024-03-24T10:45:55.247 INFO:tasks.workunit.client.0.smithi094.stdout:4/387: dwrite d0/f5f [0,4194304] 0 2024-03-24T10:45:55.250 INFO:tasks.workunit.client.0.smithi094.stdout:4/388: dread d0/f4b [0,4194304] 0 2024-03-24T10:45:55.251 INFO:tasks.workunit.client.0.smithi094.stdout:2/239: mkdir d18/d48 0 2024-03-24T10:45:55.251 INFO:tasks.workunit.client.0.smithi094.stdout:2/240: fsync d18/f19 0 2024-03-24T10:45:55.251 INFO:tasks.workunit.client.0.smithi094.stdout:2/241: creat d18/d1e/d34/f49 x:0 0 0 2024-03-24T10:45:55.252 INFO:tasks.workunit.client.0.smithi094.stdout:9/347: rmdir df/d2d/d1f/d58 39 2024-03-24T10:45:55.254 INFO:tasks.workunit.client.0.smithi094.stdout:1/290: rename d4/d9/db/l20 to d4/d9/db/l61 0 2024-03-24T10:45:55.255 INFO:tasks.workunit.client.0.smithi094.stdout:1/291: fsync d4/d36/d38/f4e 0 2024-03-24T10:45:55.255 INFO:tasks.workunit.client.0.smithi094.stdout:4/389: dread d0/d22/de/f30 [0,4194304] 0 2024-03-24T10:45:55.258 INFO:tasks.workunit.client.0.smithi094.stdout:2/242: mkdir d18/d1e/d34/d4a 0 2024-03-24T10:45:55.259 INFO:tasks.workunit.client.0.smithi094.stdout:2/243: chown d18/d1e/d34 1 1 2024-03-24T10:45:55.259 INFO:tasks.workunit.client.0.smithi094.stdout:2/244: chown d18/d1e/c41 106 1 2024-03-24T10:45:55.259 INFO:tasks.workunit.client.0.smithi094.stdout:2/245: fsync f8 0 2024-03-24T10:45:55.259 INFO:tasks.workunit.client.0.smithi094.stdout:2/246: chown d18/f47 535236 1 2024-03-24T10:45:55.259 INFO:tasks.workunit.client.0.smithi094.stdout:2/247: dread - d18/f45 zero size 2024-03-24T10:45:55.259 INFO:tasks.workunit.client.0.smithi094.stdout:2/248: read f8 [663883,86174] 0 2024-03-24T10:45:55.259 INFO:tasks.workunit.client.0.smithi094.stdout:2/249: chown d18/d1e/d34/c42 220840473 1 2024-03-24T10:45:55.259 INFO:tasks.workunit.client.0.smithi094.stdout:2/250: write d18/f31 [412350,56654] 0 2024-03-24T10:45:55.259 INFO:tasks.workunit.client.0.smithi094.stdout:1/292: truncate d4/d9/db/d31/d3c/f50 6254884 0 2024-03-24T10:45:55.259 INFO:tasks.workunit.client.0.smithi094.stdout:1/293: chown d4/d9/d47/c57 1294795 1 2024-03-24T10:45:55.260 INFO:tasks.workunit.client.0.smithi094.stdout:2/251: truncate d18/f19 427082 0 2024-03-24T10:45:55.263 INFO:tasks.workunit.client.0.smithi094.stdout:2/252: symlink d18/d48/l4b 0 2024-03-24T10:45:55.263 INFO:tasks.workunit.client.0.smithi094.stdout:3/267: dwrite d0/f2 [0,4194304] 0 2024-03-24T10:45:55.264 INFO:tasks.workunit.client.0.smithi094.stdout:6/335: dwrite d0/f3c [0,4194304] 0 2024-03-24T10:45:55.264 INFO:tasks.workunit.client.0.smithi094.stdout:6/336: dread - d0/d30/f5b zero size 2024-03-24T10:45:55.264 INFO:tasks.workunit.client.0.smithi094.stdout:7/339: dwrite d1/d9/d35/f6b [0,4194304] 0 2024-03-24T10:45:55.265 INFO:tasks.workunit.client.0.smithi094.stdout:2/253: symlink d18/l4c 0 2024-03-24T10:45:55.268 INFO:tasks.workunit.client.0.smithi094.stdout:3/268: mkdir d0/db/d1c/d20/d23/d2a/d3a/d56/d61 0 2024-03-24T10:45:55.268 INFO:tasks.workunit.client.0.smithi094.stdout:2/254: dread d18/d1e/f33 [0,4194304] 0 2024-03-24T10:45:55.269 INFO:tasks.workunit.client.0.smithi094.stdout:0/343: dwrite d14/d22/d28/d2b/f38 [0,4194304] 0 2024-03-24T10:45:55.270 INFO:tasks.workunit.client.0.smithi094.stdout:6/337: dread d0/d21/d2a/d2f/f36 [0,4194304] 0 2024-03-24T10:45:55.271 INFO:tasks.workunit.client.0.smithi094.stdout:3/269: symlink d0/db/dc/d28/d3c/d55/l62 0 2024-03-24T10:45:55.273 INFO:tasks.workunit.client.0.smithi094.stdout:8/347: dwrite df/d25/d36/d3f/f4f [0,4194304] 0 2024-03-24T10:45:55.274 INFO:tasks.workunit.client.0.smithi094.stdout:0/344: symlink d14/d22/d28/d48/l79 0 2024-03-24T10:45:55.274 INFO:tasks.workunit.client.0.smithi094.stdout:0/345: chown d14/d22/c73 0 1 2024-03-24T10:45:55.277 INFO:tasks.workunit.client.0.smithi094.stdout:6/338: mkdir d0/d5/d66 0 2024-03-24T10:45:55.278 INFO:tasks.workunit.client.0.smithi094.stdout:8/348: getdents df/d25/d36/d3f 0 2024-03-24T10:45:55.278 INFO:tasks.workunit.client.0.smithi094.stdout:8/349: write df/d25/f61 [295084,54900] 0 2024-03-24T10:45:55.279 INFO:tasks.workunit.client.0.smithi094.stdout:3/270: dread d0/db/d1c/d20/d23/d2a/d3a/ff [0,4194304] 0 2024-03-24T10:45:55.282 INFO:tasks.workunit.client.0.smithi094.stdout:6/339: truncate d0/d5/da/f20 2098522 0 2024-03-24T10:45:55.283 INFO:tasks.workunit.client.0.smithi094.stdout:8/350: mknod df/d25/d28/d31/d47/d22/c73 0 2024-03-24T10:45:55.284 INFO:tasks.workunit.client.0.smithi094.stdout:3/271: read d0/db/d1c/d20/d23/f29 [2364340,104401] 0 2024-03-24T10:45:55.289 INFO:tasks.workunit.client.0.smithi094.stdout:4/390: dwrite d0/f5e [0,4194304] 0 2024-03-24T10:45:55.289 INFO:tasks.workunit.client.0.smithi094.stdout:9/348: dwrite df/d2d/d1f/d58/d5a/f63 [0,4194304] 0 2024-03-24T10:45:55.289 INFO:tasks.workunit.client.0.smithi094.stdout:9/349: chown df/d2d/d34/f47 113726590 1 2024-03-24T10:45:55.290 INFO:tasks.workunit.client.0.smithi094.stdout:5/359: dwrite d7/d23/d5f/d14/d36/f4d [4194304,4194304] 0 2024-03-24T10:45:55.292 INFO:tasks.workunit.client.0.smithi094.stdout:7/340: dwrite d1/d9/d35/f6b [0,4194304] 0 2024-03-24T10:45:55.292 INFO:tasks.workunit.client.0.smithi094.stdout:7/341: read - d1/d9/d35/d65/f66 zero size 2024-03-24T10:45:55.292 INFO:tasks.workunit.client.0.smithi094.stdout:1/294: dwrite d4/d9/db/d31/f48 [0,4194304] 0 2024-03-24T10:45:55.292 INFO:tasks.workunit.client.0.smithi094.stdout:1/295: chown d4/d9/fd 0 1 2024-03-24T10:45:55.297 INFO:tasks.workunit.client.0.smithi094.stdout:4/391: mknod d0/d5/d1b/d4f/d70/d6d/c75 0 2024-03-24T10:45:55.297 INFO:tasks.workunit.client.0.smithi094.stdout:4/392: chown d0/c71 17 1 2024-03-24T10:45:55.297 INFO:tasks.workunit.client.0.smithi094.stdout:4/393: creat d0/d22/f76 x:0 0 0 2024-03-24T10:45:55.299 INFO:tasks.workunit.client.0.smithi094.stdout:5/360: write d7/d23/d5f/d14/f35 [2884217,95443] 0 2024-03-24T10:45:55.306 INFO:tasks.workunit.client.0.smithi094.stdout:9/350: dread df/d2d/d34/f39 [0,4194304] 0 2024-03-24T10:45:55.306 INFO:tasks.workunit.client.0.smithi094.stdout:7/342: mknod d1/d6c/c70 0 2024-03-24T10:45:55.308 INFO:tasks.workunit.client.0.smithi094.stdout:4/394: mknod d0/d34/d73/c77 0 2024-03-24T10:45:55.312 INFO:tasks.workunit.client.0.smithi094.stdout:2/255: getdents d18/d48 0 2024-03-24T10:45:55.312 INFO:tasks.workunit.client.0.smithi094.stdout:1/296: read d4/d9/db/d39/f49 [1985384,20985] 0 2024-03-24T10:45:55.312 INFO:tasks.workunit.client.0.smithi094.stdout:4/395: symlink d0/d22/de/l78 0 2024-03-24T10:45:55.313 INFO:tasks.workunit.client.0.smithi094.stdout:5/361: read d7/d23/d5f/d9/d48/d51/f52 [8952,88248] 0 2024-03-24T10:45:55.313 INFO:tasks.workunit.client.0.smithi094.stdout:5/362: chown d7/d23/d5f/d14/d36/l70 8451661 1 2024-03-24T10:45:55.313 INFO:tasks.workunit.client.0.smithi094.stdout:2/256: creat d18/d48/f4d x:0 0 0 2024-03-24T10:45:55.314 INFO:tasks.workunit.client.0.smithi094.stdout:2/257: creat d18/f4e x:0 0 0 2024-03-24T10:45:55.314 INFO:tasks.workunit.client.0.smithi094.stdout:2/258: getdents d18/d1e/d34/d4a 0 2024-03-24T10:45:55.314 INFO:tasks.workunit.client.0.smithi094.stdout:2/259: truncate d18/d1e/d34/f49 662797 0 2024-03-24T10:45:55.315 INFO:tasks.workunit.client.0.smithi094.stdout:1/297: mkdir d4/d36/d38/d62 0 2024-03-24T10:45:55.315 INFO:tasks.workunit.client.0.smithi094.stdout:1/298: chown d4/d9/db 0 1 2024-03-24T10:45:55.316 INFO:tasks.workunit.client.0.smithi094.stdout:8/351: dwrite df/d25/d28/d31/d47/f20 [0,4194304] 0 2024-03-24T10:45:55.317 INFO:tasks.workunit.client.0.smithi094.stdout:6/340: dwrite d0/d1a/f54 [0,4194304] 0 2024-03-24T10:45:55.319 INFO:tasks.workunit.client.0.smithi094.stdout:4/396: rename d0/d5/d42/c49 to d0/d34/d73/c79 0 2024-03-24T10:45:55.319 INFO:tasks.workunit.client.0.smithi094.stdout:4/397: readlink d0/d5/ld 0 2024-03-24T10:45:55.324 INFO:tasks.workunit.client.0.smithi094.stdout:3/272: dwrite d0/f3 [4194304,4194304] 0 2024-03-24T10:45:55.325 INFO:tasks.workunit.client.0.smithi094.stdout:8/352: symlink df/d25/l74 0 2024-03-24T10:45:55.326 INFO:tasks.workunit.client.0.smithi094.stdout:4/398: dread d0/d22/ff [0,4194304] 0 2024-03-24T10:45:55.327 INFO:tasks.workunit.client.0.smithi094.stdout:9/351: dwrite df/d2d/d34/d2c/f4c [0,4194304] 0 2024-03-24T10:45:55.327 INFO:tasks.workunit.client.0.smithi094.stdout:5/363: mknod d7/d23/d5f/d14/c75 0 2024-03-24T10:45:55.329 INFO:tasks.workunit.client.0.smithi094.stdout:0/346: getdents d14/d22/d28/d48 0 2024-03-24T10:45:55.330 INFO:tasks.workunit.client.0.smithi094.stdout:3/273: symlink d0/db/dc/d28/d50/d5e/l63 0 2024-03-24T10:45:55.330 INFO:tasks.workunit.client.0.smithi094.stdout:3/274: chown d0/db/d1c/d20/d23/d2a/d3a/d25/c31 94287 1 2024-03-24T10:45:55.330 INFO:tasks.workunit.client.0.smithi094.stdout:3/275: creat d0/db/d1c/d20/d23/d2a/f64 x:0 0 0 2024-03-24T10:45:55.334 INFO:tasks.workunit.client.0.smithi094.stdout:8/353: mkdir df/d12/d5b/d63/d75 0 2024-03-24T10:45:55.334 INFO:tasks.workunit.client.0.smithi094.stdout:8/354: write df/d25/d28/d31/d3c/f5f [1603642,39651] 0 2024-03-24T10:45:55.334 INFO:tasks.workunit.client.0.smithi094.stdout:8/355: creat df/d25/d28/d31/d47/d22/d55/f76 x:0 0 0 2024-03-24T10:45:55.334 INFO:tasks.workunit.client.0.smithi094.stdout:8/356: creat df/d25/d28/d31/d47/f77 x:0 0 0 2024-03-24T10:45:55.334 INFO:tasks.workunit.client.0.smithi094.stdout:4/399: mkdir d0/d22/de/d28/d2b/d7a 0 2024-03-24T10:45:55.334 INFO:tasks.workunit.client.0.smithi094.stdout:4/400: fsync d0/d22/f11 0 2024-03-24T10:45:55.335 INFO:tasks.workunit.client.0.smithi094.stdout:4/401: read - d0/d5/d42/f67 zero size 2024-03-24T10:45:55.340 INFO:tasks.workunit.client.0.smithi094.stdout:9/352: rename df/d2d/d34/f2e to df/d2d/d1f/d11/d43/f68 0 2024-03-24T10:45:55.341 INFO:tasks.workunit.client.0.smithi094.stdout:0/347: mknod d14/d18/c7a 0 2024-03-24T10:45:55.347 INFO:tasks.workunit.client.0.smithi094.stdout:8/357: symlink df/d25/d28/d31/d47/d22/l78 0 2024-03-24T10:45:55.347 INFO:tasks.workunit.client.0.smithi094.stdout:8/358: write df/d25/d36/d3f/f4e [993689,97972] 0 2024-03-24T10:45:55.348 INFO:tasks.workunit.client.0.smithi094.stdout:4/402: truncate d0/d22/de/d28/d2b/d33/f54 1466767 0 2024-03-24T10:45:55.350 INFO:tasks.workunit.client.0.smithi094.stdout:8/359: creat df/d25/d28/d31/f79 x:0 0 0 2024-03-24T10:45:55.350 INFO:tasks.workunit.client.0.smithi094.stdout:8/360: chown df/d25/d36/d3f/l49 3345374 1 2024-03-24T10:45:55.351 INFO:tasks.workunit.client.0.smithi094.stdout:5/364: read d7/d23/d2b/f5a [3992734,32847] 0 2024-03-24T10:45:55.351 INFO:tasks.workunit.client.0.smithi094.stdout:5/365: write d7/d23/d2b/f4b [908949,42836] 0 2024-03-24T10:45:55.351 INFO:tasks.workunit.client.0.smithi094.stdout:5/366: fsync d7/f10 0 2024-03-24T10:45:55.352 INFO:tasks.workunit.client.0.smithi094.stdout:8/361: rename df/d25/f2d to df/d25/d28/d31/d47/d22/d55/f7a 0 2024-03-24T10:45:55.354 INFO:tasks.workunit.client.0.smithi094.stdout:5/367: getdents d7/d23/d5f/d14/d36/d3a 0 2024-03-24T10:45:55.356 INFO:tasks.workunit.client.0.smithi094.stdout:8/362: creat df/d12/d5b/d63/d75/f7b x:0 0 0 2024-03-24T10:45:55.357 INFO:tasks.workunit.client.0.smithi094.stdout:8/363: creat df/d25/d28/d31/d47/f7c x:0 0 0 2024-03-24T10:45:55.357 INFO:tasks.workunit.client.0.smithi094.stdout:8/364: truncate df/d12/d5b/d63/d75/f7b 123466 0 2024-03-24T10:45:55.357 INFO:tasks.workunit.client.0.smithi094.stdout:8/365: chown df/d25/d28/d31/c4c 4134 1 2024-03-24T10:45:55.378 INFO:tasks.workunit.client.0.smithi094.stdout:7/343: dwrite d1/d9/d25/f50 [0,4194304] 0 2024-03-24T10:45:55.378 INFO:tasks.workunit.client.0.smithi094.stdout:2/260: dwrite d18/f45 [0,4194304] 0 2024-03-24T10:45:55.378 INFO:tasks.workunit.client.0.smithi094.stdout:2/261: dread - d18/f4e zero size 2024-03-24T10:45:55.380 INFO:tasks.workunit.client.0.smithi094.stdout:7/344: link d1/d9/f30 d1/d6c/f71 0 2024-03-24T10:45:55.382 INFO:tasks.workunit.client.0.smithi094.stdout:7/345: creat d1/d9/d16/d20/d60/f72 x:0 0 0 2024-03-24T10:45:55.382 INFO:tasks.workunit.client.0.smithi094.stdout:7/346: creat d1/d9/d25/d3d/f73 x:0 0 0 2024-03-24T10:45:55.383 INFO:tasks.workunit.client.0.smithi094.stdout:7/347: chown d1/l8 668089 1 2024-03-24T10:45:55.384 INFO:tasks.workunit.client.0.smithi094.stdout:7/348: creat d1/d9/d35/d61/f74 x:0 0 0 2024-03-24T10:45:55.386 INFO:tasks.workunit.client.0.smithi094.stdout:7/349: symlink d1/d9/d16/d21/d49/l75 0 2024-03-24T10:45:55.392 INFO:tasks.workunit.client.0.smithi094.stdout:7/350: dread d1/d9/d35/f6b [0,4194304] 0 2024-03-24T10:45:55.393 INFO:tasks.workunit.client.0.smithi094.stdout:6/341: dwrite d0/d1a/f54 [0,4194304] 0 2024-03-24T10:45:55.393 INFO:tasks.workunit.client.0.smithi094.stdout:1/299: dwrite d4/d9/f2c [0,4194304] 0 2024-03-24T10:45:55.395 INFO:tasks.workunit.client.0.smithi094.stdout:1/300: mkdir d4/d9/db/d5d/d63 0 2024-03-24T10:45:55.396 INFO:tasks.workunit.client.0.smithi094.stdout:1/301: fsync d4/d9/db/d31/d3c/f50 0 2024-03-24T10:45:55.396 INFO:tasks.workunit.client.0.smithi094.stdout:3/276: dwrite d0/db/d1c/d20/d23/f5f [0,4194304] 0 2024-03-24T10:45:55.397 INFO:tasks.workunit.client.0.smithi094.stdout:1/302: creat d4/d9/db/d5d/f64 x:0 0 0 2024-03-24T10:45:55.399 INFO:tasks.workunit.client.0.smithi094.stdout:7/351: write d1/d9/d25/f50 [1404815,108710] 0 2024-03-24T10:45:55.399 INFO:tasks.workunit.client.0.smithi094.stdout:3/277: truncate d0/db/d1c/d20/d23/d2a/d3a/ff 4003689 0 2024-03-24T10:45:55.399 INFO:tasks.workunit.client.0.smithi094.stdout:3/278: write d0/f1e [1206625,123365] 0 2024-03-24T10:45:55.401 INFO:tasks.workunit.client.0.smithi094.stdout:1/303: rename d4/d9/d37/d33/f3a to d4/d9/de/d3d/f65 0 2024-03-24T10:45:55.402 INFO:tasks.workunit.client.0.smithi094.stdout:3/279: mknod d0/db/d1c/d20/c65 0 2024-03-24T10:45:55.402 INFO:tasks.workunit.client.0.smithi094.stdout:9/353: dwrite df/d2d/d1f/d11/f4e [0,4194304] 0 2024-03-24T10:45:55.403 INFO:tasks.workunit.client.0.smithi094.stdout:0/348: dwrite d14/f1b [4194304,4194304] 0 2024-03-24T10:45:55.404 INFO:tasks.workunit.client.0.smithi094.stdout:3/280: mknod d0/db/dc/d28/d50/c66 0 2024-03-24T10:45:55.404 INFO:tasks.workunit.client.0.smithi094.stdout:3/281: chown d0/db/d1c/d20/d23/d2a/f4c 1 1 2024-03-24T10:45:55.404 INFO:tasks.workunit.client.0.smithi094.stdout:4/403: dwrite d0/d5/d42/d4a/f61 [4194304,4194304] 0 2024-03-24T10:45:55.408 INFO:tasks.workunit.client.0.smithi094.stdout:0/349: mkdir d14/d22/d28/d48/d7b 0 2024-03-24T10:45:55.408 INFO:tasks.workunit.client.0.smithi094.stdout:0/350: rename d14/d22/d28/d2b/d41/d6b to d14/d22/d28/d2b/d41/d6b/d7c 22 2024-03-24T10:45:55.409 INFO:tasks.workunit.client.0.smithi094.stdout:4/404: truncate d0/f47 1960671 0 2024-03-24T10:45:55.409 INFO:tasks.workunit.client.0.smithi094.stdout:4/405: stat d0/d22/de/d28/d2b/d33/c45 0 2024-03-24T10:45:55.409 INFO:tasks.workunit.client.0.smithi094.stdout:4/406: chown d0/d1f/l57 10027635 1 2024-03-24T10:45:55.411 INFO:tasks.workunit.client.0.smithi094.stdout:0/351: rmdir d14/d22/d28/d2b/d41/d6b 0 2024-03-24T10:45:55.411 INFO:tasks.workunit.client.0.smithi094.stdout:0/352: readlink d14/d22/d28/d2b/d41/d53/l58 0 2024-03-24T10:45:55.413 INFO:tasks.workunit.client.0.smithi094.stdout:4/407: creat d0/d22/de/d28/d2b/d7a/f7b x:0 0 0 2024-03-24T10:45:55.413 INFO:tasks.workunit.client.0.smithi094.stdout:4/408: chown d0/d22/de/d28/f59 203 1 2024-03-24T10:45:55.415 INFO:tasks.workunit.client.0.smithi094.stdout:0/353: mkdir d14/d22/d7d 0 2024-03-24T10:45:55.416 INFO:tasks.workunit.client.0.smithi094.stdout:2/262: dwrite d18/d1e/d34/f43 [0,4194304] 0 2024-03-24T10:45:55.416 INFO:tasks.workunit.client.0.smithi094.stdout:7/352: dread d1/d9/d14/f36 [0,4194304] 0 2024-03-24T10:45:55.416 INFO:tasks.workunit.client.0.smithi094.stdout:2/263: write d18/d1e/f25 [233279,48525] 0 2024-03-24T10:45:55.416 INFO:tasks.workunit.client.0.smithi094.stdout:7/353: chown d1/d9/d25/d3d/f5a 180866634 1 2024-03-24T10:45:55.417 INFO:tasks.workunit.client.0.smithi094.stdout:4/409: link d0/d5/d1b/f4e d0/d5/d1b/d4f/d70/f7c 0 2024-03-24T10:45:55.419 INFO:tasks.workunit.client.0.smithi094.stdout:9/354: dwrite df/d2d/d34/d2c/f4d [0,4194304] 0 2024-03-24T10:45:55.419 INFO:tasks.workunit.client.0.smithi094.stdout:6/342: dwrite d0/d21/f2b [0,4194304] 0 2024-03-24T10:45:55.419 INFO:tasks.workunit.client.0.smithi094.stdout:8/366: dwrite df/d25/d28/d31/d3c/f3e [0,4194304] 0 2024-03-24T10:45:55.419 INFO:tasks.workunit.client.0.smithi094.stdout:1/304: dread d4/d9/f32 [0,4194304] 0 2024-03-24T10:45:55.419 INFO:tasks.workunit.client.0.smithi094.stdout:5/368: dwrite d7/f10 [4194304,4194304] 0 2024-03-24T10:45:55.422 INFO:tasks.workunit.client.0.smithi094.stdout:3/282: dwrite d0/db/d1c/d20/d23/d2a/f33 [0,4194304] 0 2024-03-24T10:45:55.422 INFO:tasks.workunit.client.0.smithi094.stdout:3/283: chown d0/db/dc/l2f 93666 1 2024-03-24T10:45:55.423 INFO:tasks.workunit.client.0.smithi094.stdout:7/354: unlink d1/d9/d35/d61/f74 0 2024-03-24T10:45:55.425 INFO:tasks.workunit.client.0.smithi094.stdout:2/264: mknod d18/d1e/d34/d4a/c4f 0 2024-03-24T10:45:55.429 INFO:tasks.workunit.client.0.smithi094.stdout:6/343: dread d0/d21/f13 [0,4194304] 0 2024-03-24T10:45:55.429 INFO:tasks.workunit.client.0.smithi094.stdout:9/355: mkdir df/d2d/d1f/d58/d5a/d69 0 2024-03-24T10:45:55.429 INFO:tasks.workunit.client.0.smithi094.stdout:9/356: chown df/d2d/d1f/l31 24388465 1 2024-03-24T10:45:55.434 INFO:tasks.workunit.client.0.smithi094.stdout:4/410: write d0/d5/d1b/d4f/d70/f7c [2325687,5944] 0 2024-03-24T10:45:55.436 INFO:tasks.workunit.client.0.smithi094.stdout:4/411: write d0/d22/de/f30 [774755,66152] 0 2024-03-24T10:45:55.438 INFO:tasks.workunit.client.0.smithi094.stdout:7/355: dread d1/d9/d16/f2f [0,4194304] 0 2024-03-24T10:45:55.438 INFO:tasks.workunit.client.0.smithi094.stdout:7/356: dread - d1/d9/d16/d20/d60/f72 zero size 2024-03-24T10:45:55.438 INFO:tasks.workunit.client.0.smithi094.stdout:8/367: truncate df/d25/d28/d31/d47/f69 2955168 0 2024-03-24T10:45:55.440 INFO:tasks.workunit.client.0.smithi094.stdout:5/369: symlink d7/d23/d5f/d14/d36/l76 0 2024-03-24T10:45:55.440 INFO:tasks.workunit.client.0.smithi094.stdout:5/370: write d7/d23/d5f/d1b/f60 [884832,63395] 0 2024-03-24T10:45:55.443 INFO:tasks.workunit.client.0.smithi094.stdout:3/284: mknod d0/db/dc/d28/d50/c67 0 2024-03-24T10:45:55.451 INFO:tasks.workunit.client.0.smithi094.stdout:9/357: creat df/d2d/d1f/d58/d5a/f6a x:0 0 0 2024-03-24T10:45:55.454 INFO:tasks.workunit.client.0.smithi094.stdout:2/265: dwrite d18/f27 [0,4194304] 0 2024-03-24T10:45:55.461 INFO:tasks.workunit.client.0.smithi094.stdout:4/412: mkdir d0/d5/d7d 0 2024-03-24T10:45:55.467 INFO:tasks.workunit.client.0.smithi094.stdout:6/344: getdents d0/d21/d2a/d2f 0 2024-03-24T10:45:55.469 INFO:tasks.workunit.client.0.smithi094.stdout:0/354: dwrite d14/d18/f2e [0,4194304] 0 2024-03-24T10:45:55.469 INFO:tasks.workunit.client.0.smithi094.stdout:2/266: write d18/f45 [2275892,123326] 0 2024-03-24T10:45:55.469 INFO:tasks.workunit.client.0.smithi094.stdout:8/368: dwrite df/d25/d36/d3f/f4e [0,4194304] 0 2024-03-24T10:45:55.470 INFO:tasks.workunit.client.0.smithi094.stdout:8/369: rename df/d25 to df/d25/d28/d31/d47/d22/d7d 22 2024-03-24T10:45:55.470 INFO:tasks.workunit.client.0.smithi094.stdout:7/357: dwrite d1/d9/d14/f4b [0,4194304] 0 2024-03-24T10:45:55.470 INFO:tasks.workunit.client.0.smithi094.stdout:9/358: mkdir df/d2d/d1f/d11/d43/d6b 0 2024-03-24T10:45:55.474 INFO:tasks.workunit.client.0.smithi094.stdout:5/371: link d7/d23/d5f/d9/f49 d7/d23/f77 0 2024-03-24T10:45:55.476 INFO:tasks.workunit.client.0.smithi094.stdout:6/345: mknod d0/d21/d2a/d2f/c67 0 2024-03-24T10:45:55.476 INFO:tasks.workunit.client.0.smithi094.stdout:6/346: chown d0/d30/d50/d4d 13 1 2024-03-24T10:45:55.478 INFO:tasks.workunit.client.0.smithi094.stdout:0/355: unlink d14/d18/f3e 0 2024-03-24T10:45:55.480 INFO:tasks.workunit.client.0.smithi094.stdout:2/267: rmdir d18/d1e/d34/d4a 39 2024-03-24T10:45:55.481 INFO:tasks.workunit.client.0.smithi094.stdout:2/268: dread d18/d1e/f24 [0,4194304] 0 2024-03-24T10:45:55.482 INFO:tasks.workunit.client.0.smithi094.stdout:2/269: dread d18/f19 [0,4194304] 0 2024-03-24T10:45:55.496 INFO:tasks.workunit.client.0.smithi094.stdout:9/359: rmdir df/d2d/d1f/d11/d43/d6b 0 2024-03-24T10:45:55.498 INFO:tasks.workunit.client.0.smithi094.stdout:4/413: truncate d0/d5/d1b/d4f/d70/f7c 3818774 0 2024-03-24T10:45:55.499 INFO:tasks.workunit.client.0.smithi094.stdout:8/370: dwrite df/d25/f29 [0,4194304] 0 2024-03-24T10:45:55.500 INFO:tasks.workunit.client.0.smithi094.stdout:6/347: truncate d0/d30/f28 1295358 0 2024-03-24T10:45:55.501 INFO:tasks.workunit.client.0.smithi094.stdout:0/356: dwrite d14/d22/f2f [4194304,4194304] 0 2024-03-24T10:45:55.501 INFO:tasks.workunit.client.0.smithi094.stdout:7/358: getdents d1/d9/d14 0 2024-03-24T10:45:55.504 INFO:tasks.workunit.client.0.smithi094.stdout:9/360: rename df/d2d/d1f/d11/l5e to df/d2d/d34/d2c/l6c 0 2024-03-24T10:45:55.505 INFO:tasks.workunit.client.0.smithi094.stdout:4/414: link d0/d34/d73/c79 d0/d22/de/d28/d3d/c7e 0 2024-03-24T10:45:55.505 INFO:tasks.workunit.client.0.smithi094.stdout:4/415: fsync d0/d22/de/d28/f72 0 2024-03-24T10:45:55.506 INFO:tasks.workunit.client.0.smithi094.stdout:5/372: truncate d7/d23/d5f/d9/d48/d51/f52 518987 0 2024-03-24T10:45:55.509 INFO:tasks.workunit.client.0.smithi094.stdout:8/371: link df/d25/d28/d31/d3c/f41 df/d25/d28/d31/d47/f7e 0 2024-03-24T10:45:55.512 INFO:tasks.workunit.client.0.smithi094.stdout:0/357: creat d14/d22/d28/d2b/d41/f7e x:0 0 0 2024-03-24T10:45:55.512 INFO:tasks.workunit.client.0.smithi094.stdout:0/358: creat d14/f7f x:0 0 0 2024-03-24T10:45:55.514 INFO:tasks.workunit.client.0.smithi094.stdout:2/270: unlink c10 0 2024-03-24T10:45:55.514 INFO:tasks.workunit.client.0.smithi094.stdout:2/271: write f7 [3169881,97738] 0 2024-03-24T10:45:55.516 INFO:tasks.workunit.client.0.smithi094.stdout:6/348: rename d0/d1a/f54 to d0/d5/da/d23/f68 0 2024-03-24T10:45:55.516 INFO:tasks.workunit.client.0.smithi094.stdout:6/349: readlink d0/d1a/l1c 0 2024-03-24T10:45:55.517 INFO:tasks.workunit.client.0.smithi094.stdout:9/361: creat df/d2d/d1f/d11/d43/f6d x:0 0 0 2024-03-24T10:45:55.518 INFO:tasks.workunit.client.0.smithi094.stdout:9/362: chown df/d2d/d34/d2c/f5f 3302 1 2024-03-24T10:45:55.519 INFO:tasks.workunit.client.0.smithi094.stdout:5/373: creat d7/d23/d5f/d9/f78 x:0 0 0 2024-03-24T10:45:55.520 INFO:tasks.workunit.client.0.smithi094.stdout:4/416: dwrite d0/d22/f76 [0,4194304] 0 2024-03-24T10:45:55.520 INFO:tasks.workunit.client.0.smithi094.stdout:4/417: fdatasync d0/d22/de/d28/f52 0 2024-03-24T10:45:55.520 INFO:tasks.workunit.client.0.smithi094.stdout:4/418: fsync d0/d22/de/d28/f72 0 2024-03-24T10:45:55.520 INFO:tasks.workunit.client.0.smithi094.stdout:4/419: write d0/f60 [1254246,27397] 0 2024-03-24T10:45:55.520 INFO:tasks.workunit.client.0.smithi094.stdout:4/420: fdatasync d0/d34/f48 0 2024-03-24T10:45:55.521 INFO:tasks.workunit.client.0.smithi094.stdout:8/372: symlink df/d25/d28/d31/d47/d66/l7f 0 2024-03-24T10:45:55.521 INFO:tasks.workunit.client.0.smithi094.stdout:8/373: readlink df/d25/d28/d31/d47/d22/l27 0 2024-03-24T10:45:55.522 INFO:tasks.workunit.client.0.smithi094.stdout:8/374: chown df/d12/c17 145734778 1 2024-03-24T10:45:55.523 INFO:tasks.workunit.client.0.smithi094.stdout:0/359: creat d14/d22/d32/f80 x:0 0 0 2024-03-24T10:45:55.525 INFO:tasks.workunit.client.0.smithi094.stdout:2/272: link d18/f1d d18/d1e/d34/f50 0 2024-03-24T10:45:55.526 INFO:tasks.workunit.client.0.smithi094.stdout:2/273: creat d18/d1e/d34/f51 x:0 0 0 2024-03-24T10:45:55.526 INFO:tasks.workunit.client.0.smithi094.stdout:2/274: fdatasync d18/f23 0 2024-03-24T10:45:55.526 INFO:tasks.workunit.client.0.smithi094.stdout:2/275: chown d18/d1e/l2d 7588 1 2024-03-24T10:45:55.526 INFO:tasks.workunit.client.0.smithi094.stdout:2/276: chown d18/d1e 961199 1 2024-03-24T10:45:55.526 INFO:tasks.workunit.client.0.smithi094.stdout:2/277: fdatasync d18/d1e/d34/f46 0 2024-03-24T10:45:55.526 INFO:tasks.workunit.client.0.smithi094.stdout:2/278: write d18/f3b [711094,18900] 0 2024-03-24T10:45:55.526 INFO:tasks.workunit.client.0.smithi094.stdout:2/279: creat d18/d48/f52 x:0 0 0 2024-03-24T10:45:55.527 INFO:tasks.workunit.client.0.smithi094.stdout:5/374: write d7/d23/d5f/d1b/f37 [5441707,71184] 0 2024-03-24T10:45:55.527 INFO:tasks.workunit.client.0.smithi094.stdout:7/359: rename d1/d9/d25/d3d/l5c to d1/d9/d35/d65/l76 0 2024-03-24T10:45:55.528 INFO:tasks.workunit.client.0.smithi094.stdout:7/360: readlink d1/d9/d14/l17 0 2024-03-24T10:45:55.529 INFO:tasks.workunit.client.0.smithi094.stdout:9/363: unlink df/d2d/d1f/l31 0 2024-03-24T10:45:55.533 INFO:tasks.workunit.client.0.smithi094.stdout:8/375: mknod df/d25/d28/d31/d47/c80 0 2024-03-24T10:45:55.536 INFO:tasks.workunit.client.0.smithi094.stdout:0/360: creat d14/d22/d28/d48/f81 x:0 0 0 2024-03-24T10:45:55.536 INFO:tasks.workunit.client.0.smithi094.stdout:0/361: truncate d14/f62 619980 0 2024-03-24T10:45:55.536 INFO:tasks.workunit.client.0.smithi094.stdout:0/362: write d14/d22/d28/d48/f78 [424064,32446] 0 2024-03-24T10:45:55.538 INFO:tasks.workunit.client.0.smithi094.stdout:6/350: dwrite d0/d21/f4f [0,4194304] 0 2024-03-24T10:45:55.540 INFO:tasks.workunit.client.0.smithi094.stdout:5/375: creat d7/d23/d5f/d14/d36/d3a/f79 x:0 0 0 2024-03-24T10:45:55.540 INFO:tasks.workunit.client.0.smithi094.stdout:9/364: creat df/d2d/d1f/f6e x:0 0 0 2024-03-24T10:45:55.540 INFO:tasks.workunit.client.0.smithi094.stdout:9/365: creat df/d2d/d1f/d58/d5a/f6f x:0 0 0 2024-03-24T10:45:55.540 INFO:tasks.workunit.client.0.smithi094.stdout:8/376: mknod df/d12/d5b/d63/d75/c81 0 2024-03-24T10:45:55.541 INFO:tasks.workunit.client.0.smithi094.stdout:0/363: unlink d14/f52 0 2024-03-24T10:45:55.546 INFO:tasks.workunit.client.0.smithi094.stdout:5/376: rename d7/d23/d5f/d14/d36/d3a/c5d to d7/d26/d55/c7a 0 2024-03-24T10:45:55.546 INFO:tasks.workunit.client.0.smithi094.stdout:5/377: stat d7/c6c 0 2024-03-24T10:45:55.549 INFO:tasks.workunit.client.0.smithi094.stdout:8/377: unlink df/d25/d28/d31/d47/d22/l57 0 2024-03-24T10:45:55.550 INFO:tasks.workunit.client.0.smithi094.stdout:0/364: fdatasync d14/d22/d28/f47 0 2024-03-24T10:45:55.552 INFO:tasks.workunit.client.0.smithi094.stdout:5/378: rename c5 to d7/d23/d5f/d14/d6f/c7b 0 2024-03-24T10:45:55.553 INFO:tasks.workunit.client.0.smithi094.stdout:7/361: dwrite d1/d9/d16/d21/d51/f5e [0,4194304] 0 2024-03-24T10:45:55.553 INFO:tasks.workunit.client.0.smithi094.stdout:7/362: chown d1/d9/d16/d20/f69 1071925881 1 2024-03-24T10:45:55.553 INFO:tasks.workunit.client.0.smithi094.stdout:2/280: dwrite d18/f27 [0,4194304] 0 2024-03-24T10:45:55.553 INFO:tasks.workunit.client.0.smithi094.stdout:2/281: fsync d18/d1e/f33 0 2024-03-24T10:45:55.554 INFO:tasks.workunit.client.0.smithi094.stdout:4/421: dwrite d0/d22/de/d28/d2b/f51 [0,4194304] 0 2024-03-24T10:45:55.555 INFO:tasks.workunit.client.0.smithi094.stdout:6/351: dwrite d0/d21/f13 [0,4194304] 0 2024-03-24T10:45:55.556 INFO:tasks.workunit.client.0.smithi094.stdout:5/379: truncate d7/d23/d2b/f4b 361113 0 2024-03-24T10:45:55.556 INFO:tasks.workunit.client.0.smithi094.stdout:5/380: chown d7/dc/d12/f39 8585 1 2024-03-24T10:45:55.557 INFO:tasks.workunit.client.0.smithi094.stdout:7/363: link d1/d9/d16/d20/d60/d39/d44/f4e d1/d9/d14/f77 0 2024-03-24T10:45:55.558 INFO:tasks.workunit.client.0.smithi094.stdout:8/378: dread f6 [0,4194304] 0 2024-03-24T10:45:55.572 INFO:tasks.workunit.client.0.smithi094.stdout:2/282: dread d18/f27 [0,4194304] 0 2024-03-24T10:45:55.572 INFO:tasks.workunit.client.0.smithi094.stdout:6/352: rmdir d0/d21/d2a/d55/d63 0 2024-03-24T10:45:55.574 INFO:tasks.workunit.client.0.smithi094.stdout:7/364: rename d1/d9/d16/d21/f33 to d1/d9/d16/d21/d49/f78 0 2024-03-24T10:45:55.577 INFO:tasks.workunit.client.0.smithi094.stdout:6/353: link d0/d5/f29 d0/d3b/f69 0 2024-03-24T10:45:55.577 INFO:tasks.workunit.client.0.smithi094.stdout:6/354: write d0/d30/f5b [499819,46304] 0 2024-03-24T10:45:55.577 INFO:tasks.workunit.client.0.smithi094.stdout:7/365: symlink d1/d9/d16/d21/l79 0 2024-03-24T10:45:55.579 INFO:tasks.workunit.client.0.smithi094.stdout:7/366: unlink d1/d9/d14/f77 0 2024-03-24T10:45:55.580 INFO:tasks.workunit.client.0.smithi094.stdout:7/367: mknod d1/d9/d16/d20/d60/c7a 0 2024-03-24T10:45:55.580 INFO:tasks.workunit.client.0.smithi094.stdout:2/283: dread d18/d1e/d34/f35 [0,4194304] 0 2024-03-24T10:45:55.581 INFO:tasks.workunit.client.0.smithi094.stdout:7/368: symlink d1/d9/d16/l7b 0 2024-03-24T10:45:55.582 INFO:tasks.workunit.client.0.smithi094.stdout:7/369: truncate d1/d9/d25/f50 2354125 0 2024-03-24T10:45:55.583 INFO:tasks.workunit.client.0.smithi094.stdout:7/370: dread - d1/d9/d16/d20/d60/d39/d44/f4e zero size 2024-03-24T10:45:55.584 INFO:tasks.workunit.client.0.smithi094.stdout:7/371: readlink d1/d9/d35/l62 0 2024-03-24T10:45:55.586 INFO:tasks.workunit.client.0.smithi094.stdout:7/372: rename d1/d9/d16/c24 to d1/d9/d25/d3d/c7c 0 2024-03-24T10:45:55.594 INFO:tasks.workunit.client.0.smithi094.stdout:5/381: dwrite d7/f34 [0,4194304] 0 2024-03-24T10:45:55.596 INFO:tasks.workunit.client.0.smithi094.stdout:5/382: creat d7/d23/d5f/d14/f7c x:0 0 0 2024-03-24T10:45:55.600 INFO:tasks.workunit.client.0.smithi094.stdout:6/355: dwrite d0/f3c [0,4194304] 0 2024-03-24T10:45:55.609 INFO:tasks.workunit.client.0.smithi094.stdout:8/379: dwrite df/d25/d28/d31/d3c/f5f [0,4194304] 0 2024-03-24T10:45:55.610 INFO:tasks.workunit.client.0.smithi094.stdout:2/284: dwrite d18/f45 [0,4194304] 0 2024-03-24T10:45:55.610 INFO:tasks.workunit.client.0.smithi094.stdout:5/383: write d7/d23/d5f/f16 [1712987,120806] 0 2024-03-24T10:45:55.611 INFO:tasks.workunit.client.0.smithi094.stdout:8/380: link df/d25/d28/d31/l62 df/l82 0 2024-03-24T10:45:55.612 INFO:tasks.workunit.client.0.smithi094.stdout:8/381: write df/d25/d28/d31/d47/d22/f3d [1207642,76910] 0 2024-03-24T10:45:55.612 INFO:tasks.workunit.client.0.smithi094.stdout:8/382: chown df/d12/d5b/l5e 3899 1 2024-03-24T10:45:55.614 INFO:tasks.workunit.client.0.smithi094.stdout:8/383: rename df/d25/d28/d31/d47/d22/f3d to df/d12/d5b/f83 0 2024-03-24T10:45:55.614 INFO:tasks.workunit.client.0.smithi094.stdout:8/384: dread - df/d25/f44 zero size 2024-03-24T10:45:55.616 INFO:tasks.workunit.client.0.smithi094.stdout:0/365: truncate f9 870704 0 2024-03-24T10:45:55.619 INFO:tasks.workunit.client.0.smithi094.stdout:0/366: symlink d14/d22/d5b/l82 0 2024-03-24T10:45:55.619 INFO:tasks.workunit.client.0.smithi094.stdout:7/373: getdents d1/d9/d16/d21/d49 0 2024-03-24T10:45:55.620 INFO:tasks.workunit.client.0.smithi094.stdout:7/374: chown d1/d9/l47 15 1 2024-03-24T10:45:55.620 INFO:tasks.workunit.client.0.smithi094.stdout:7/375: creat d1/d9/d35/d65/f7d x:0 0 0 2024-03-24T10:45:55.620 INFO:tasks.workunit.client.0.smithi094.stdout:7/376: read - d1/d9/d16/d20/f6f zero size 2024-03-24T10:45:55.620 INFO:tasks.workunit.client.0.smithi094.stdout:7/377: creat d1/d9/d16/d20/f7e x:0 0 0 2024-03-24T10:45:55.625 INFO:tasks.workunit.client.0.smithi094.stdout:2/285: read d18/d1e/d34/f43 [3317172,105870] 0 2024-03-24T10:45:55.625 INFO:tasks.workunit.client.0.smithi094.stdout:2/286: write f15 [4622577,33232] 0 2024-03-24T10:45:55.625 INFO:tasks.workunit.client.0.smithi094.stdout:7/378: truncate f0 2958238 0 2024-03-24T10:45:55.626 INFO:tasks.workunit.client.0.smithi094.stdout:0/367: getdents d14/d22/d28/d48 0 2024-03-24T10:45:55.627 INFO:tasks.workunit.client.0.smithi094.stdout:2/287: dread d18/d1e/d34/f35 [0,4194304] 0 2024-03-24T10:45:55.627 INFO:tasks.workunit.client.0.smithi094.stdout:2/288: truncate d18/d48/f4d 962724 0 2024-03-24T10:45:55.627 INFO:tasks.workunit.client.0.smithi094.stdout:7/379: write d1/d9/f30 [1505662,102602] 0 2024-03-24T10:45:55.628 INFO:tasks.workunit.client.0.smithi094.stdout:5/384: dwrite d7/d23/d5f/d14/d36/f4d [8388608,4194304] 0 2024-03-24T10:45:55.629 INFO:tasks.workunit.client.0.smithi094.stdout:0/368: symlink d14/d22/d28/d2b/d41/d53/l83 0 2024-03-24T10:45:55.629 INFO:tasks.workunit.client.0.smithi094.stdout:0/369: readlink d14/l34 0 2024-03-24T10:45:55.629 INFO:tasks.workunit.client.0.smithi094.stdout:0/370: chown d14/d18/f25 7383206 1 2024-03-24T10:45:55.629 INFO:tasks.workunit.client.0.smithi094.stdout:6/356: dwrite d0/d21/f2b [0,4194304] 0 2024-03-24T10:45:55.629 INFO:tasks.workunit.client.0.smithi094.stdout:8/385: dwrite df/d25/f44 [0,4194304] 0 2024-03-24T10:45:55.631 INFO:tasks.workunit.client.0.smithi094.stdout:5/385: getdents d7/d23 0 2024-03-24T10:45:55.631 INFO:tasks.workunit.client.0.smithi094.stdout:5/386: stat d7/d23 0 2024-03-24T10:45:55.633 INFO:tasks.workunit.client.0.smithi094.stdout:0/371: mknod d14/d22/d5b/c84 0 2024-03-24T10:45:55.633 INFO:tasks.workunit.client.0.smithi094.stdout:0/372: chown d14/d22/d28/d2b/c6e 438 1 2024-03-24T10:45:55.637 INFO:tasks.workunit.client.0.smithi094.stdout:2/289: dread f15 [4194304,4194304] 0 2024-03-24T10:45:55.637 INFO:tasks.workunit.client.0.smithi094.stdout:2/290: write f15 [6718925,50451] 0 2024-03-24T10:45:55.637 INFO:tasks.workunit.client.0.smithi094.stdout:2/291: creat d18/d1e/f53 x:0 0 0 2024-03-24T10:45:55.637 INFO:tasks.workunit.client.0.smithi094.stdout:2/292: chown lb 0 1 2024-03-24T10:45:55.638 INFO:tasks.workunit.client.0.smithi094.stdout:5/387: rename d7/d23/d5f/f16 to d7/d26/d55/d64/f7d 0 2024-03-24T10:45:55.638 INFO:tasks.workunit.client.0.smithi094.stdout:5/388: read - d7/d26/f65 zero size 2024-03-24T10:45:55.640 INFO:tasks.workunit.client.0.smithi094.stdout:0/373: mkdir d14/d18/d85 0 2024-03-24T10:45:55.640 INFO:tasks.workunit.client.0.smithi094.stdout:7/380: dwrite d1/d9/d16/d20/d60/d39/f48 [0,4194304] 0 2024-03-24T10:45:55.641 INFO:tasks.workunit.client.0.smithi094.stdout:6/357: truncate d0/d21/d2a/d2f/f46 436513 0 2024-03-24T10:45:55.641 INFO:tasks.workunit.client.0.smithi094.stdout:6/358: write d0/d21/d2a/f40 [4870746,129657] 0 2024-03-24T10:45:55.641 INFO:tasks.workunit.client.0.smithi094.stdout:6/359: creat d0/d3b/f6a x:0 0 0 2024-03-24T10:45:55.642 INFO:tasks.workunit.client.0.smithi094.stdout:8/386: fsync df/d12/d5b/f83 0 2024-03-24T10:45:55.645 INFO:tasks.workunit.client.0.smithi094.stdout:6/360: dread d0/d30/f28 [0,4194304] 0 2024-03-24T10:45:55.645 INFO:tasks.workunit.client.0.smithi094.stdout:3/285: sync 2024-03-24T10:45:55.645 INFO:tasks.workunit.client.0.smithi094.stdout:1/305: sync 2024-03-24T10:45:55.645 INFO:tasks.workunit.client.0.smithi094.stdout:1/306: truncate f1 4147924 0 2024-03-24T10:45:55.645 INFO:tasks.workunit.client.0.smithi094.stdout:1/307: getdents d4/d36/d38/d62 0 2024-03-24T10:45:55.645 INFO:tasks.workunit.client.0.smithi094.stdout:2/293: mkdir d18/d54 0 2024-03-24T10:45:55.645 INFO:tasks.workunit.client.0.smithi094.stdout:0/374: write d14/d22/d28/d2b/f45 [2834159,68929] 0 2024-03-24T10:45:55.646 INFO:tasks.workunit.client.0.smithi094.stdout:2/294: truncate d18/f45 4571128 0 2024-03-24T10:45:55.647 INFO:tasks.workunit.client.0.smithi094.stdout:8/387: dread df/f1b [0,4194304] 0 2024-03-24T10:45:55.647 INFO:tasks.workunit.client.0.smithi094.stdout:5/389: mknod d7/d23/d5f/c7e 0 2024-03-24T10:45:55.647 INFO:tasks.workunit.client.0.smithi094.stdout:8/388: write df/d25/d28/d31/d47/d22/d55/f6d [587252,87836] 0 2024-03-24T10:45:55.647 INFO:tasks.workunit.client.0.smithi094.stdout:5/390: read - d7/d23/d5f/d14/f27 zero size 2024-03-24T10:45:55.647 INFO:tasks.workunit.client.0.smithi094.stdout:5/391: truncate d7/d23/d5f/d9/d48/d51/f52 723516 0 2024-03-24T10:45:55.647 INFO:tasks.workunit.client.0.smithi094.stdout:5/392: fdatasync d7/d26/f65 0 2024-03-24T10:45:55.647 INFO:tasks.workunit.client.0.smithi094.stdout:5/393: chown d7/dc/d56 352299 1 2024-03-24T10:45:55.647 INFO:tasks.workunit.client.0.smithi094.stdout:5/394: creat d7/d23/d5f/d14/d36/d3a/f7f x:0 0 0 2024-03-24T10:45:55.647 INFO:tasks.workunit.client.0.smithi094.stdout:5/395: readlink d7/d23/d5f/l71 0 2024-03-24T10:45:55.657 INFO:tasks.workunit.client.0.smithi094.stdout:6/361: dwrite d0/d5/f44 [0,4194304] 0 2024-03-24T10:45:55.657 INFO:tasks.workunit.client.0.smithi094.stdout:6/362: fdatasync d0/d21/f13 0 2024-03-24T10:45:55.657 INFO:tasks.workunit.client.0.smithi094.stdout:6/363: write d0/f33 [1218719,11011] 0 2024-03-24T10:45:55.657 INFO:tasks.workunit.client.0.smithi094.stdout:7/381: truncate d1/f10 4142144 0 2024-03-24T10:45:55.659 INFO:tasks.workunit.client.0.smithi094.stdout:8/389: dread df/d25/d28/d31/d47/f20 [0,4194304] 0 2024-03-24T10:45:55.660 INFO:tasks.workunit.client.0.smithi094.stdout:3/286: symlink d0/db/d1c/d20/d23/l68 0 2024-03-24T10:45:55.661 INFO:tasks.workunit.client.0.smithi094.stdout:1/308: mkdir d4/d9/de/d3d/d66 0 2024-03-24T10:45:55.661 INFO:tasks.workunit.client.0.smithi094.stdout:2/295: rmdir d18/d1e/d34/d4a 39 2024-03-24T10:45:55.662 INFO:tasks.workunit.client.0.smithi094.stdout:0/375: mkdir d14/d22/d5b/d86 0 2024-03-24T10:45:55.663 INFO:tasks.workunit.client.0.smithi094.stdout:0/376: write d14/d18/f67 [713611,110292] 0 2024-03-24T10:45:55.665 INFO:tasks.workunit.client.0.smithi094.stdout:5/396: rmdir d7/d26/d55 39 2024-03-24T10:45:55.668 INFO:tasks.workunit.client.0.smithi094.stdout:6/364: mkdir d0/d30/d50/d4d/d6b 0 2024-03-24T10:45:55.669 INFO:tasks.workunit.client.0.smithi094.stdout:7/382: mkdir d1/d9/d14/d34/d7f 0 2024-03-24T10:45:55.670 INFO:tasks.workunit.client.0.smithi094.stdout:8/390: creat df/d25/d28/d31/d3c/f84 x:0 0 0 2024-03-24T10:45:55.671 INFO:tasks.workunit.client.0.smithi094.stdout:3/287: mkdir d0/db/d1c/d20/d23/d2a/d3a/d56/d61/d69 0 2024-03-24T10:45:55.671 INFO:tasks.workunit.client.0.smithi094.stdout:9/366: sync 2024-03-24T10:45:55.671 INFO:tasks.workunit.client.0.smithi094.stdout:4/422: sync 2024-03-24T10:45:55.671 INFO:tasks.workunit.client.0.smithi094.stdout:9/367: stat df/d2d/d34/f30 0 2024-03-24T10:45:55.672 INFO:tasks.workunit.client.0.smithi094.stdout:8/391: dread df/d25/f58 [0,4194304] 0 2024-03-24T10:45:55.672 INFO:tasks.workunit.client.0.smithi094.stdout:9/368: fdatasync df/d2d/d34/d2c/f4d 0 2024-03-24T10:45:55.672 INFO:tasks.workunit.client.0.smithi094.stdout:1/309: truncate d4/d9/f2c 1787088 0 2024-03-24T10:45:55.672 INFO:tasks.workunit.client.0.smithi094.stdout:1/310: write d4/f16 [4780997,5616] 0 2024-03-24T10:45:55.674 INFO:tasks.workunit.client.0.smithi094.stdout:2/296: rename d18/d1e/d34/f3e to d18/d1e/d34/d4a/f55 0 2024-03-24T10:45:55.674 INFO:tasks.workunit.client.0.smithi094.stdout:5/397: creat d7/d26/f80 x:0 0 0 2024-03-24T10:45:55.675 INFO:tasks.workunit.client.0.smithi094.stdout:5/398: fdatasync d7/d23/d5f/d1b/f42 0 2024-03-24T10:45:55.675 INFO:tasks.workunit.client.0.smithi094.stdout:2/297: dread f8 [0,4194304] 0 2024-03-24T10:45:55.681 INFO:tasks.workunit.client.0.smithi094.stdout:6/365: creat d0/d5/da/d23/f6c x:0 0 0 2024-03-24T10:45:55.681 INFO:tasks.workunit.client.0.smithi094.stdout:6/366: write d0/d5/f29 [413999,107217] 0 2024-03-24T10:45:55.682 INFO:tasks.workunit.client.0.smithi094.stdout:7/383: truncate d1/d9/d14/f4c 286603 0 2024-03-24T10:45:55.683 INFO:tasks.workunit.client.0.smithi094.stdout:3/288: mkdir d0/db/dc/d28/d38/d6a 0 2024-03-24T10:45:55.684 INFO:tasks.workunit.client.0.smithi094.stdout:3/289: readlink d0/db/d1c/d20/d23/l37 0 2024-03-24T10:45:55.685 INFO:tasks.workunit.client.0.smithi094.stdout:4/423: unlink d0/d22/de/d28/f52 0 2024-03-24T10:45:55.686 INFO:tasks.workunit.client.0.smithi094.stdout:9/369: mkdir df/d2d/d1f/d58/d70 0 2024-03-24T10:45:55.688 INFO:tasks.workunit.client.0.smithi094.stdout:1/311: creat d4/d9/db/f67 x:0 0 0 2024-03-24T10:45:55.688 INFO:tasks.workunit.client.0.smithi094.stdout:1/312: write d4/d9/db/f42 [2653943,89005] 0 2024-03-24T10:45:55.690 INFO:tasks.workunit.client.0.smithi094.stdout:8/392: rename df/d25/d36/d3f/f4e to df/d25/d28/d31/d47/f85 0 2024-03-24T10:45:55.690 INFO:tasks.workunit.client.0.smithi094.stdout:8/393: stat df/d25/d28/d31/f40 0 2024-03-24T10:45:55.693 INFO:tasks.workunit.client.0.smithi094.stdout:5/399: mkdir d7/d23/d2b/d68/d81 0 2024-03-24T10:45:55.693 INFO:tasks.workunit.client.0.smithi094.stdout:5/400: write d7/d26/d55/d64/f7d [3302330,22091] 0 2024-03-24T10:45:55.693 INFO:tasks.workunit.client.0.smithi094.stdout:5/401: fsync d7/d23/d5f/d9/d48/d51/f6a 0 2024-03-24T10:45:55.693 INFO:tasks.workunit.client.0.smithi094.stdout:5/402: write d7/d23/d5f/d1b/f42 [1994852,5531] 0 2024-03-24T10:45:55.693 INFO:tasks.workunit.client.0.smithi094.stdout:5/403: truncate d7/dc/d56/f6e 506586 0 2024-03-24T10:45:55.694 INFO:tasks.workunit.client.0.smithi094.stdout:7/384: dread d1/d9/f23 [0,4194304] 0 2024-03-24T10:45:55.694 INFO:tasks.workunit.client.0.smithi094.stdout:2/298: truncate f8 875421 0 2024-03-24T10:45:55.694 INFO:tasks.workunit.client.0.smithi094.stdout:2/299: truncate d18/d1e/f24 915053 0 2024-03-24T10:45:55.696 INFO:tasks.workunit.client.0.smithi094.stdout:6/367: symlink d0/d30/d50/d4d/l6d 0 2024-03-24T10:45:55.696 INFO:tasks.workunit.client.0.smithi094.stdout:6/368: readlink d0/d1a/l3e 0 2024-03-24T10:45:55.698 INFO:tasks.workunit.client.0.smithi094.stdout:4/424: truncate d0/d5/d1b/f2a 256526 0 2024-03-24T10:45:55.699 INFO:tasks.workunit.client.0.smithi094.stdout:8/394: mknod df/d25/d28/d31/c86 0 2024-03-24T10:45:55.699 INFO:tasks.workunit.client.0.smithi094.stdout:8/395: creat df/d25/d28/f87 x:0 0 0 2024-03-24T10:45:55.701 INFO:tasks.workunit.client.0.smithi094.stdout:9/370: write df/d2d/d1f/f49 [1955348,109778] 0 2024-03-24T10:45:55.702 INFO:tasks.workunit.client.0.smithi094.stdout:5/404: rename d7/d26/l2d to d7/d23/d2b/d68/d81/l82 0 2024-03-24T10:45:55.703 INFO:tasks.workunit.client.0.smithi094.stdout:5/405: truncate d7/d26/f80 1022977 0 2024-03-24T10:45:55.703 INFO:tasks.workunit.client.0.smithi094.stdout:7/385: read d1/d9/d14/f4b [1965598,109817] 0 2024-03-24T10:45:55.703 INFO:tasks.workunit.client.0.smithi094.stdout:7/386: chown d1/c11 265 1 2024-03-24T10:45:55.703 INFO:tasks.workunit.client.0.smithi094.stdout:7/387: write d1/d9/d25/d3d/f46 [4354083,81002] 0 2024-03-24T10:45:55.703 INFO:tasks.workunit.client.0.smithi094.stdout:2/300: mkdir d18/d48/d56 0 2024-03-24T10:45:55.705 INFO:tasks.workunit.client.0.smithi094.stdout:4/425: link d0/f5f d0/d22/de/d28/f7f 0 2024-03-24T10:45:55.705 INFO:tasks.workunit.client.0.smithi094.stdout:4/426: write d0/d22/de/d28/d2b/f43 [1006488,99405] 0 2024-03-24T10:45:55.705 INFO:tasks.workunit.client.0.smithi094.stdout:3/290: dwrite d0/db/d1c/d20/d23/d2a/d3a/ff [0,4194304] 0 2024-03-24T10:45:55.706 INFO:tasks.workunit.client.0.smithi094.stdout:1/313: dwrite d4/d9/de/f3b [0,4194304] 0 2024-03-24T10:45:55.709 INFO:tasks.workunit.client.0.smithi094.stdout:8/396: rename df/d25/f48 to df/d25/d28/d31/f88 0 2024-03-24T10:45:55.710 INFO:tasks.workunit.client.0.smithi094.stdout:9/371: rename df/d2d to df/d2d/d1f/d58/d5a/d50/d71 22 2024-03-24T10:45:55.711 INFO:tasks.workunit.client.0.smithi094.stdout:5/406: creat d7/d23/d5f/f83 x:0 0 0 2024-03-24T10:45:55.711 INFO:tasks.workunit.client.0.smithi094.stdout:5/407: dread - d7/d23/d5f/d9/f49 zero size 2024-03-24T10:45:55.713 INFO:tasks.workunit.client.0.smithi094.stdout:6/369: dwrite d0/d5/da/d23/f6c [0,4194304] 0 2024-03-24T10:45:55.713 INFO:tasks.workunit.client.0.smithi094.stdout:6/370: creat d0/d3b/f6e x:0 0 0 2024-03-24T10:45:55.715 INFO:tasks.workunit.client.0.smithi094.stdout:2/301: symlink d18/d1e/d34/d4a/l57 0 2024-03-24T10:45:55.715 INFO:tasks.workunit.client.0.smithi094.stdout:2/302: stat d18/d1e/d34/d4a 0 2024-03-24T10:45:55.715 INFO:tasks.workunit.client.0.smithi094.stdout:2/303: chown d18/l4c 2005 1 2024-03-24T10:45:55.715 INFO:tasks.workunit.client.0.smithi094.stdout:2/304: fdatasync d18/f45 0 2024-03-24T10:45:55.715 INFO:tasks.workunit.client.0.smithi094.stdout:3/291: link d0/db/dc/d28/d50/c67 d0/db/dc/d28/d38/d53/c6b 0 2024-03-24T10:45:55.717 INFO:tasks.workunit.client.0.smithi094.stdout:2/305: dread d18/f2e [0,4194304] 0 2024-03-24T10:45:55.719 INFO:tasks.workunit.client.0.smithi094.stdout:8/397: dread df/d12/f39 [0,4194304] 0 2024-03-24T10:45:55.719 INFO:tasks.workunit.client.0.smithi094.stdout:1/314: creat d4/d36/d38/d5f/f68 x:0 0 0 2024-03-24T10:45:55.723 INFO:tasks.workunit.client.0.smithi094.stdout:1/315: fsync d4/f16 0 2024-03-24T10:45:55.725 INFO:tasks.workunit.client.0.smithi094.stdout:3/292: read d0/db/d1c/d20/d23/f5f [3180972,105184] 0 2024-03-24T10:45:55.726 INFO:tasks.workunit.client.0.smithi094.stdout:9/372: dwrite df/d2d/d1f/d58/d5a/f6a [0,4194304] 0 2024-03-24T10:45:55.726 INFO:tasks.workunit.client.0.smithi094.stdout:2/306: creat d18/d48/d56/f58 x:0 0 0 2024-03-24T10:45:55.727 INFO:tasks.workunit.client.0.smithi094.stdout:8/398: creat df/f89 x:0 0 0 2024-03-24T10:45:55.727 INFO:tasks.workunit.client.0.smithi094.stdout:8/399: creat df/d25/d28/d31/d47/d22/d55/f8a x:0 0 0 2024-03-24T10:45:55.730 INFO:tasks.workunit.client.0.smithi094.stdout:6/371: dwrite d0/f1e [0,4194304] 0 2024-03-24T10:45:55.732 INFO:tasks.workunit.client.0.smithi094.stdout:5/408: getdents d7/d23/d5f/d14 0 2024-03-24T10:45:55.735 INFO:tasks.workunit.client.0.smithi094.stdout:1/316: mknod d4/d9/d47/c69 0 2024-03-24T10:45:55.736 INFO:tasks.workunit.client.0.smithi094.stdout:3/293: rmdir d0/db/dc/d28 39 2024-03-24T10:45:55.742 INFO:tasks.workunit.client.0.smithi094.stdout:1/317: dread d4/d9/db/f1f [0,4194304] 0 2024-03-24T10:45:55.751 INFO:tasks.workunit.client.0.smithi094.stdout:3/294: dread d0/db/d1c/d20/d23/f29 [0,4194304] 0 2024-03-24T10:45:55.752 INFO:tasks.workunit.client.0.smithi094.stdout:7/388: rename d1/d9/d16/d20 to d1/d6c/d80 0 2024-03-24T10:45:55.754 INFO:tasks.workunit.client.0.smithi094.stdout:6/372: creat d0/d1a/f6f x:0 0 0 2024-03-24T10:45:55.755 INFO:tasks.workunit.client.0.smithi094.stdout:3/295: mknod d0/c6c 0 2024-03-24T10:45:55.755 INFO:tasks.workunit.client.0.smithi094.stdout:1/318: write d4/d9/de/f2b [1875340,84315] 0 2024-03-24T10:45:55.756 INFO:tasks.workunit.client.0.smithi094.stdout:4/427: rename d0/l14 to d0/d5/d42/l80 0 2024-03-24T10:45:55.756 INFO:tasks.workunit.client.0.smithi094.stdout:7/389: mkdir d1/d6c/d80/d81 0 2024-03-24T10:45:55.756 INFO:tasks.workunit.client.0.smithi094.stdout:7/390: chown d1/d9/d25/d3d/f46 229483 1 2024-03-24T10:45:55.757 INFO:tasks.workunit.client.0.smithi094.stdout:3/296: mknod d0/c6d 0 2024-03-24T10:45:55.759 INFO:tasks.workunit.client.0.smithi094.stdout:4/428: read d0/d5/d1b/f2e [538433,102766] 0 2024-03-24T10:45:55.759 INFO:tasks.workunit.client.0.smithi094.stdout:4/429: fsync d0/f4b 0 2024-03-24T10:45:55.759 INFO:tasks.workunit.client.0.smithi094.stdout:1/319: creat d4/d9/db/d5d/d63/f6a x:0 0 0 2024-03-24T10:45:55.759 INFO:tasks.workunit.client.0.smithi094.stdout:1/320: write d4/d9/db/f1f [4358074,111689] 0 2024-03-24T10:45:55.759 INFO:tasks.workunit.client.0.smithi094.stdout:4/430: getdents d0/d22/de 0 2024-03-24T10:45:55.764 INFO:tasks.workunit.client.0.smithi094.stdout:7/391: dread d1/d9/fd [0,4194304] 0 2024-03-24T10:45:55.766 INFO:tasks.workunit.client.0.smithi094.stdout:7/392: creat d1/d6c/f82 x:0 0 0 2024-03-24T10:45:55.768 INFO:tasks.workunit.client.0.smithi094.stdout:7/393: getdents d1/d9/d25/d3d 0 2024-03-24T10:45:55.768 INFO:tasks.workunit.client.0.smithi094.stdout:7/394: chown d1/l29 1 1 2024-03-24T10:45:55.770 INFO:tasks.workunit.client.0.smithi094.stdout:9/373: dwrite df/f36 [0,4194304] 0 2024-03-24T10:45:55.770 INFO:tasks.workunit.client.0.smithi094.stdout:4/431: dread d0/d22/de/d28/f3f [0,4194304] 0 2024-03-24T10:45:55.770 INFO:tasks.workunit.client.0.smithi094.stdout:5/409: dwrite d7/d23/d5f/d9/d48/d51/f52 [0,4194304] 0 2024-03-24T10:45:55.770 INFO:tasks.workunit.client.0.smithi094.stdout:8/400: dwrite df/f2b [4194304,4194304] 0 2024-03-24T10:45:55.770 INFO:tasks.workunit.client.0.smithi094.stdout:5/410: creat d7/d26/f84 x:0 0 0 2024-03-24T10:45:55.770 INFO:tasks.workunit.client.0.smithi094.stdout:5/411: write d7/d26/f84 [798670,41646] 0 2024-03-24T10:45:55.771 INFO:tasks.workunit.client.0.smithi094.stdout:6/373: dread d0/d21/f2b [0,4194304] 0 2024-03-24T10:45:55.773 INFO:tasks.workunit.client.0.smithi094.stdout:2/307: rmdir d18/d48 39 2024-03-24T10:45:55.776 INFO:tasks.workunit.client.0.smithi094.stdout:8/401: symlink df/d12/l8b 0 2024-03-24T10:45:55.776 INFO:tasks.workunit.client.0.smithi094.stdout:8/402: write df/d12/d5b/d63/d75/f7b [501838,80726] 0 2024-03-24T10:45:55.776 INFO:tasks.workunit.client.0.smithi094.stdout:8/403: chown df/d25/d28/d31/d47/d22/d55 69696664 1 2024-03-24T10:45:55.777 INFO:tasks.workunit.client.0.smithi094.stdout:8/404: write df/d25/d28/d31/d47/f5a [163922,44881] 0 2024-03-24T10:45:55.777 INFO:tasks.workunit.client.0.smithi094.stdout:8/405: readlink df/d12/l8b 0 2024-03-24T10:45:55.777 INFO:tasks.workunit.client.0.smithi094.stdout:5/412: creat d7/d23/d5f/d14/f85 x:0 0 0 2024-03-24T10:45:55.777 INFO:tasks.workunit.client.0.smithi094.stdout:5/413: write d7/d26/d55/d64/f7d [1628897,108564] 0 2024-03-24T10:45:55.777 INFO:tasks.workunit.client.0.smithi094.stdout:9/374: rmdir df/d2d/d1f/d58/d5a 39 2024-03-24T10:45:55.780 INFO:tasks.workunit.client.0.smithi094.stdout:6/374: link d0/d5/da/l57 d0/d30/d50/l70 0 2024-03-24T10:45:55.781 INFO:tasks.workunit.client.0.smithi094.stdout:2/308: getdents d18/d48/d56 0 2024-03-24T10:45:55.782 INFO:tasks.workunit.client.0.smithi094.stdout:8/406: mkdir df/d8c 0 2024-03-24T10:45:55.782 INFO:tasks.workunit.client.0.smithi094.stdout:8/407: creat df/d25/d28/d31/d3c/f8d x:0 0 0 2024-03-24T10:45:55.782 INFO:tasks.workunit.client.0.smithi094.stdout:8/408: write df/f89 [532155,20507] 0 2024-03-24T10:45:55.783 INFO:tasks.workunit.client.0.smithi094.stdout:5/414: symlink d7/d23/d5f/d1b/l86 0 2024-03-24T10:45:55.783 INFO:tasks.workunit.client.0.smithi094.stdout:5/415: truncate d7/d23/d5f/d9/f49 184692 0 2024-03-24T10:45:55.784 INFO:tasks.workunit.client.0.smithi094.stdout:9/375: mkdir df/d2d/d72 0 2024-03-24T10:45:55.784 INFO:tasks.workunit.client.0.smithi094.stdout:9/376: stat df/d2d/d1f/d11/f4e 0 2024-03-24T10:45:55.784 INFO:tasks.workunit.client.0.smithi094.stdout:4/432: dread d0/d22/de/d28/d2b/f51 [0,4194304] 0 2024-03-24T10:45:55.784 INFO:tasks.workunit.client.0.smithi094.stdout:4/433: fsync d0/d34/f48 0 2024-03-24T10:45:55.787 INFO:tasks.workunit.client.0.smithi094.stdout:8/409: creat df/d25/d36/d3f/f8e x:0 0 0 2024-03-24T10:45:55.787 INFO:tasks.workunit.client.0.smithi094.stdout:7/395: dwrite d1/d9/d25/f50 [0,4194304] 0 2024-03-24T10:45:55.788 INFO:tasks.workunit.client.0.smithi094.stdout:1/321: dwrite d4/d9/db/f1d [4194304,4194304] 0 2024-03-24T10:45:55.788 INFO:tasks.workunit.client.0.smithi094.stdout:5/416: write d7/dc/d12/f1f [1792725,9648] 0 2024-03-24T10:45:55.789 INFO:tasks.workunit.client.0.smithi094.stdout:3/297: dwrite d0/db/f48 [0,4194304] 0 2024-03-24T10:45:55.789 INFO:tasks.workunit.client.0.smithi094.stdout:6/375: dread d0/f8 [0,4194304] 0 2024-03-24T10:45:55.789 INFO:tasks.workunit.client.0.smithi094.stdout:6/376: stat d0/d1a/f39 0 2024-03-24T10:45:55.790 INFO:tasks.workunit.client.0.smithi094.stdout:9/377: link df/d2d/d1f/d58/d5a/f6f df/d2d/d34/d2c/f73 0 2024-03-24T10:45:55.792 INFO:tasks.workunit.client.0.smithi094.stdout:7/396: write d1/d9/d25/d3d/f46 [231319,7019] 0 2024-03-24T10:45:55.792 INFO:tasks.workunit.client.0.smithi094.stdout:4/434: truncate d0/f5e 1672798 0 2024-03-24T10:45:55.792 INFO:tasks.workunit.client.0.smithi094.stdout:8/410: mkdir df/d8c/d8f 0 2024-03-24T10:45:55.795 INFO:tasks.workunit.client.0.smithi094.stdout:0/377: sync 2024-03-24T10:45:55.796 INFO:tasks.workunit.client.0.smithi094.stdout:5/417: mknod d7/d23/d5f/d9/d48/d73/c87 0 2024-03-24T10:45:55.798 INFO:tasks.workunit.client.0.smithi094.stdout:5/418: dread d7/d23/d5f/d9/d48/f50 [0,4194304] 0 2024-03-24T10:45:55.800 INFO:tasks.workunit.client.0.smithi094.stdout:3/298: truncate d0/db/d1c/d20/d23/d2a/f5b 379104 0 2024-03-24T10:45:55.802 INFO:tasks.workunit.client.0.smithi094.stdout:9/378: read df/d2d/d1f/d11/f4e [442065,103478] 0 2024-03-24T10:45:55.806 INFO:tasks.workunit.client.0.smithi094.stdout:6/377: unlink d0/fc 0 2024-03-24T10:45:55.808 INFO:tasks.workunit.client.0.smithi094.stdout:2/309: dwrite d18/d1e/f24 [0,4194304] 0 2024-03-24T10:45:55.809 INFO:tasks.workunit.client.0.smithi094.stdout:7/397: unlink d1/d9/d16/d21/d49/f78 0 2024-03-24T10:45:55.810 INFO:tasks.workunit.client.0.smithi094.stdout:4/435: symlink d0/d22/de/d28/d2b/d7a/l81 0 2024-03-24T10:45:55.813 INFO:tasks.workunit.client.0.smithi094.stdout:0/378: rename d14/d18/f67 to d14/d18/d85/f87 0 2024-03-24T10:45:55.813 INFO:tasks.workunit.client.0.smithi094.stdout:0/379: chown d14/d22/d28/d2b/d41/d53/l74 1975 1 2024-03-24T10:45:55.814 INFO:tasks.workunit.client.0.smithi094.stdout:9/379: mkdir df/d2d/d34/d2c/d74 0 2024-03-24T10:45:55.816 INFO:tasks.workunit.client.0.smithi094.stdout:6/378: mknod d0/d30/c71 0 2024-03-24T10:45:55.817 INFO:tasks.workunit.client.0.smithi094.stdout:6/379: write d0/d30/f5b [344460,87737] 0 2024-03-24T10:45:55.819 INFO:tasks.workunit.client.0.smithi094.stdout:2/310: symlink d18/d48/l59 0 2024-03-24T10:45:55.821 INFO:tasks.workunit.client.0.smithi094.stdout:4/436: creat d0/d34/f82 x:0 0 0 2024-03-24T10:45:55.821 INFO:tasks.workunit.client.0.smithi094.stdout:4/437: truncate d0/d5/d1b/f2e 1868055 0 2024-03-24T10:45:55.824 INFO:tasks.workunit.client.0.smithi094.stdout:7/398: rename d1/d6c/d80/d60 to d1/d9/d16/d21/d51/d83 0 2024-03-24T10:45:55.826 INFO:tasks.workunit.client.0.smithi094.stdout:1/322: dwrite f1 [0,4194304] 0 2024-03-24T10:45:55.826 INFO:tasks.workunit.client.0.smithi094.stdout:6/380: dread d0/d5/da/d23/f6c [0,4194304] 0 2024-03-24T10:45:55.826 INFO:tasks.workunit.client.0.smithi094.stdout:0/380: mknod d14/d22/d5b/c88 0 2024-03-24T10:45:55.831 INFO:tasks.workunit.client.0.smithi094.stdout:6/381: dread d0/f8 [0,4194304] 0 2024-03-24T10:45:55.832 INFO:tasks.workunit.client.0.smithi094.stdout:4/438: creat d0/d22/de/d28/d2b/f83 x:0 0 0 2024-03-24T10:45:55.833 INFO:tasks.workunit.client.0.smithi094.stdout:4/439: chown d0/d22/de/d28/d2b/d7a 4510955 1 2024-03-24T10:45:55.833 INFO:tasks.workunit.client.0.smithi094.stdout:4/440: creat d0/d22/de/d28/d2b/d7a/f84 x:0 0 0 2024-03-24T10:45:55.834 INFO:tasks.workunit.client.0.smithi094.stdout:6/382: write d0/d21/f13 [8495932,19224] 0 2024-03-24T10:45:55.834 INFO:tasks.workunit.client.0.smithi094.stdout:6/383: truncate d0/d3b/f5f 204035 0 2024-03-24T10:45:55.834 INFO:tasks.workunit.client.0.smithi094.stdout:3/299: dwrite d0/db/d1c/d20/d23/d2a/d3a/ff [0,4194304] 0 2024-03-24T10:45:55.834 INFO:tasks.workunit.client.0.smithi094.stdout:3/300: write d0/db/dc/d28/d50/f60 [2727393,14799] 0 2024-03-24T10:45:55.836 INFO:tasks.workunit.client.0.smithi094.stdout:7/399: rename d1/d9/d16/d21/d51/f5e to d1/d9/d16/d21/d49/f84 0 2024-03-24T10:45:55.837 INFO:tasks.workunit.client.0.smithi094.stdout:8/411: dwrite df/d25/f29 [0,4194304] 0 2024-03-24T10:45:55.838 INFO:tasks.workunit.client.0.smithi094.stdout:1/323: mkdir d4/d9/d37/d33/d58/d6b 0 2024-03-24T10:45:55.840 INFO:tasks.workunit.client.0.smithi094.stdout:9/380: dwrite df/d2d/d1f/d58/d5a/f63 [0,4194304] 0 2024-03-24T10:45:55.840 INFO:tasks.workunit.client.0.smithi094.stdout:3/301: dread d0/db/d1c/d20/f24 [0,4194304] 0 2024-03-24T10:45:55.840 INFO:tasks.workunit.client.0.smithi094.stdout:3/302: fdatasync d0/db/d1c/d20/d23/d2a/f4c 0 2024-03-24T10:45:55.840 INFO:tasks.workunit.client.0.smithi094.stdout:3/303: fdatasync d0/db/d1c/d20/d23/d2a/d3a/ff 0 2024-03-24T10:45:55.846 INFO:tasks.workunit.client.0.smithi094.stdout:2/311: dwrite d18/d48/f4d [0,4194304] 0 2024-03-24T10:45:55.847 INFO:tasks.workunit.client.0.smithi094.stdout:0/381: dwrite d14/d18/f1d [4194304,4194304] 0 2024-03-24T10:45:55.854 INFO:tasks.workunit.client.0.smithi094.stdout:4/441: creat d0/d5/d1b/d4f/f85 x:0 0 0 2024-03-24T10:45:55.854 INFO:tasks.workunit.client.0.smithi094.stdout:4/442: stat d0/d1f 0 2024-03-24T10:45:55.857 INFO:tasks.workunit.client.0.smithi094.stdout:6/384: creat d0/d1a/d35/f72 x:0 0 0 2024-03-24T10:45:55.858 INFO:tasks.workunit.client.0.smithi094.stdout:7/400: rename d1/l8 to d1/d9/d16/d21/d51/d83/l85 0 2024-03-24T10:45:55.863 INFO:tasks.workunit.client.0.smithi094.stdout:1/324: link d4/d9/l17 d4/d9/db/d5d/d63/l6c 0 2024-03-24T10:45:55.863 INFO:tasks.workunit.client.0.smithi094.stdout:1/325: chown d4/d9/db/d39 912646 1 2024-03-24T10:45:55.863 INFO:tasks.workunit.client.0.smithi094.stdout:1/326: fsync d4/d9/d37/d33/f5b 0 2024-03-24T10:45:55.863 INFO:tasks.workunit.client.0.smithi094.stdout:1/327: dread d4/f7 [0,4194304] 0 2024-03-24T10:45:55.864 INFO:tasks.workunit.client.0.smithi094.stdout:9/381: dwrite df/f56 [0,4194304] 0 2024-03-24T10:45:55.865 INFO:tasks.workunit.client.0.smithi094.stdout:3/304: symlink d0/db/dc/d28/d50/l6e 0 2024-03-24T10:45:55.865 INFO:tasks.workunit.client.0.smithi094.stdout:3/305: creat d0/db/dc/d28/d3c/f6f x:0 0 0 2024-03-24T10:45:55.867 INFO:tasks.workunit.client.0.smithi094.stdout:1/328: dread d4/d9/db/f42 [0,4194304] 0 2024-03-24T10:45:55.867 INFO:tasks.workunit.client.0.smithi094.stdout:1/329: fdatasync d4/d36/d38/f4e 0 2024-03-24T10:45:55.869 INFO:tasks.workunit.client.0.smithi094.stdout:2/312: mkdir d18/d1e/d5a 0 2024-03-24T10:45:55.877 INFO:tasks.workunit.client.0.smithi094.stdout:7/401: creat d1/d9/d16/d21/d51/d83/d39/f86 x:0 0 0 2024-03-24T10:45:55.879 INFO:tasks.workunit.client.0.smithi094.stdout:8/412: getdents df/d25/d36/d3f 0 2024-03-24T10:45:55.879 INFO:tasks.workunit.client.0.smithi094.stdout:8/413: stat df/d12/f21 0 2024-03-24T10:45:55.879 INFO:tasks.workunit.client.0.smithi094.stdout:8/414: creat df/f90 x:0 0 0 2024-03-24T10:45:55.879 INFO:tasks.workunit.client.0.smithi094.stdout:4/443: dwrite d0/d5/d1b/f39 [4194304,4194304] 0 2024-03-24T10:45:55.883 INFO:tasks.workunit.client.0.smithi094.stdout:3/306: unlink d0/db/dc/d28/d3c/l54 0 2024-03-24T10:45:55.884 INFO:tasks.workunit.client.0.smithi094.stdout:2/313: dwrite f8 [0,4194304] 0 2024-03-24T10:45:55.884 INFO:tasks.workunit.client.0.smithi094.stdout:1/330: unlink d4/d9/fa 0 2024-03-24T10:45:55.885 INFO:tasks.workunit.client.0.smithi094.stdout:6/385: dwrite d0/d3b/f6e [0,4194304] 0 2024-03-24T10:45:55.885 INFO:tasks.workunit.client.0.smithi094.stdout:7/402: mknod d1/c87 0 2024-03-24T10:45:55.889 INFO:tasks.workunit.client.0.smithi094.stdout:8/415: rename df/d25/d28/d31/d47/d66/l7f to df/d25/d36/l91 0 2024-03-24T10:45:55.889 INFO:tasks.workunit.client.0.smithi094.stdout:8/416: write df/d25/d36/d3f/f4f [4516901,37697] 0 2024-03-24T10:45:55.889 INFO:tasks.workunit.client.0.smithi094.stdout:8/417: read - df/d25/d28/d31/d47/f7e zero size 2024-03-24T10:45:55.889 INFO:tasks.workunit.client.0.smithi094.stdout:4/444: mknod d0/d5/d1b/c86 0 2024-03-24T10:45:55.890 INFO:tasks.workunit.client.0.smithi094.stdout:4/445: fsync d0/d22/ff 0 2024-03-24T10:45:55.890 INFO:tasks.workunit.client.0.smithi094.stdout:4/446: stat d0/d34 0 2024-03-24T10:45:55.891 INFO:tasks.workunit.client.0.smithi094.stdout:3/307: creat d0/db/dc/d28/d3c/d55/f70 x:0 0 0 2024-03-24T10:45:55.894 INFO:tasks.workunit.client.0.smithi094.stdout:1/331: unlink d4/d9/de/d3d/f65 0 2024-03-24T10:45:55.896 INFO:tasks.workunit.client.0.smithi094.stdout:2/314: dwrite d18/f1b [0,4194304] 0 2024-03-24T10:45:55.898 INFO:tasks.workunit.client.0.smithi094.stdout:6/386: mknod d0/d1a/c73 0 2024-03-24T10:45:55.902 INFO:tasks.workunit.client.0.smithi094.stdout:2/315: dread d18/f27 [0,4194304] 0 2024-03-24T10:45:55.902 INFO:tasks.workunit.client.0.smithi094.stdout:2/316: write f7 [3263555,101747] 0 2024-03-24T10:45:55.902 INFO:tasks.workunit.client.0.smithi094.stdout:4/447: unlink d0/d34/d73/c79 0 2024-03-24T10:45:55.902 INFO:tasks.workunit.client.0.smithi094.stdout:4/448: chown d0/d34/d73 640949135 1 2024-03-24T10:45:55.907 INFO:tasks.workunit.client.0.smithi094.stdout:6/387: symlink d0/d5/da/d23/d34/d4e/l74 0 2024-03-24T10:45:55.907 INFO:tasks.workunit.client.0.smithi094.stdout:6/388: stat d0/d5/da/d23/d34 0 2024-03-24T10:45:55.908 INFO:tasks.workunit.client.0.smithi094.stdout:6/389: fsync d0/d21/d2a/d2f/f46 0 2024-03-24T10:45:55.909 INFO:tasks.workunit.client.0.smithi094.stdout:2/317: mknod d18/d48/c5b 0 2024-03-24T10:45:55.909 INFO:tasks.workunit.client.0.smithi094.stdout:2/318: readlink l3 0 2024-03-24T10:45:55.909 INFO:tasks.workunit.client.0.smithi094.stdout:2/319: dread - d18/f4e zero size 2024-03-24T10:45:55.909 INFO:tasks.workunit.client.0.smithi094.stdout:2/320: dread - d18/d1e/d34/f51 zero size 2024-03-24T10:45:55.909 INFO:tasks.workunit.client.0.smithi094.stdout:2/321: chown d18/d1e/f24 24060 1 2024-03-24T10:45:55.912 INFO:tasks.workunit.client.0.smithi094.stdout:2/322: unlink d18/d1e/f24 0 2024-03-24T10:45:55.912 INFO:tasks.workunit.client.0.smithi094.stdout:2/323: truncate d18/f4e 551266 0 2024-03-24T10:45:55.912 INFO:tasks.workunit.client.0.smithi094.stdout:5/419: sync 2024-03-24T10:45:55.912 INFO:tasks.workunit.client.0.smithi094.stdout:3/308: dwrite d0/db/dc/d28/d3c/f6f [0,4194304] 0 2024-03-24T10:45:55.913 INFO:tasks.workunit.client.0.smithi094.stdout:4/449: rename d0/d34/l5b to d0/l87 0 2024-03-24T10:45:55.914 INFO:tasks.workunit.client.0.smithi094.stdout:7/403: getdents d1/d9/d16/d21/d51/d83/d39 0 2024-03-24T10:45:55.914 INFO:tasks.workunit.client.0.smithi094.stdout:7/404: chown d1/d9/d16/d21/l79 71 1 2024-03-24T10:45:55.915 INFO:tasks.workunit.client.0.smithi094.stdout:6/390: write d0/d5/da/d23/f68 [104933,37759] 0 2024-03-24T10:45:55.915 INFO:tasks.workunit.client.0.smithi094.stdout:6/391: chown d0/d21/d2a/l52 337 1 2024-03-24T10:45:55.915 INFO:tasks.workunit.client.0.smithi094.stdout:6/392: creat d0/d5/da/d23/f75 x:0 0 0 2024-03-24T10:45:55.918 INFO:tasks.workunit.client.0.smithi094.stdout:6/393: dread d0/d5/f29 [0,4194304] 0 2024-03-24T10:45:55.918 INFO:tasks.workunit.client.0.smithi094.stdout:6/394: chown d0/d3b 368063116 1 2024-03-24T10:45:55.918 INFO:tasks.workunit.client.0.smithi094.stdout:6/395: readlink d0/d5/l4b 0 2024-03-24T10:45:55.918 INFO:tasks.workunit.client.0.smithi094.stdout:6/396: creat d0/d30/f76 x:0 0 0 2024-03-24T10:45:55.918 INFO:tasks.workunit.client.0.smithi094.stdout:6/397: stat d0/d30 0 2024-03-24T10:45:55.918 INFO:tasks.workunit.client.0.smithi094.stdout:6/398: truncate d0/d30/f76 948149 0 2024-03-24T10:45:55.918 INFO:tasks.workunit.client.0.smithi094.stdout:6/399: truncate d0/d21/f5c 596552 0 2024-03-24T10:45:55.918 INFO:tasks.workunit.client.0.smithi094.stdout:5/420: getdents d7/d26/d55/d64 0 2024-03-24T10:45:55.918 INFO:tasks.workunit.client.0.smithi094.stdout:5/421: chown d7/d26/f5c 18279 1 2024-03-24T10:45:55.921 INFO:tasks.workunit.client.0.smithi094.stdout:2/324: rename d18/d48/f4d to d18/d54/f5c 0 2024-03-24T10:45:55.921 INFO:tasks.workunit.client.0.smithi094.stdout:2/325: read d18/f2e [602352,108061] 0 2024-03-24T10:45:55.922 INFO:tasks.workunit.client.0.smithi094.stdout:7/405: truncate d1/f10 720113 0 2024-03-24T10:45:55.922 INFO:tasks.workunit.client.0.smithi094.stdout:7/406: creat d1/d6c/f88 x:0 0 0 2024-03-24T10:45:55.922 INFO:tasks.workunit.client.0.smithi094.stdout:7/407: stat d1/d6c/c70 0 2024-03-24T10:45:55.922 INFO:tasks.workunit.client.0.smithi094.stdout:7/408: creat d1/d9/d35/d65/f89 x:0 0 0 2024-03-24T10:45:55.927 INFO:tasks.workunit.client.0.smithi094.stdout:6/400: mkdir d0/d5/da/d77 0 2024-03-24T10:45:55.927 INFO:tasks.workunit.client.0.smithi094.stdout:6/401: fsync d0/f1e 0 2024-03-24T10:45:55.927 INFO:tasks.workunit.client.0.smithi094.stdout:6/402: chown d0/d3b/f6a 6 1 2024-03-24T10:45:55.927 INFO:tasks.workunit.client.0.smithi094.stdout:6/403: write d0/d5/da/d23/f68 [3927093,86067] 0 2024-03-24T10:45:55.929 INFO:tasks.workunit.client.0.smithi094.stdout:5/422: truncate d7/dc/d56/f6e 170580 0 2024-03-24T10:45:55.932 INFO:tasks.workunit.client.0.smithi094.stdout:3/309: dwrite d0/db/dc/f2d [0,4194304] 0 2024-03-24T10:45:55.932 INFO:tasks.workunit.client.0.smithi094.stdout:6/404: write d0/d21/f2b [3523947,2183] 0 2024-03-24T10:45:55.932 INFO:tasks.workunit.client.0.smithi094.stdout:4/450: dwrite d0/d5/d1b/f2a [0,4194304] 0 2024-03-24T10:45:55.932 INFO:tasks.workunit.client.0.smithi094.stdout:5/423: write d7/d23/d5f/d1b/f60 [516822,112506] 0 2024-03-24T10:45:55.932 INFO:tasks.workunit.client.0.smithi094.stdout:5/424: readlink d7/d23/d5f/l71 0 2024-03-24T10:45:55.933 INFO:tasks.workunit.client.0.smithi094.stdout:4/451: dread d0/d22/ff [0,4194304] 0 2024-03-24T10:45:55.936 INFO:tasks.workunit.client.0.smithi094.stdout:7/409: creat d1/d9/d16/d21/d51/d83/d39/d44/f8a x:0 0 0 2024-03-24T10:45:55.944 INFO:tasks.workunit.client.0.smithi094.stdout:2/326: rmdir d18 39 2024-03-24T10:45:55.948 INFO:tasks.workunit.client.0.smithi094.stdout:3/310: unlink d0/db/dc/d28/d50/f60 0 2024-03-24T10:45:55.948 INFO:tasks.workunit.client.0.smithi094.stdout:3/311: truncate d0/f1e 5089970 0 2024-03-24T10:45:55.956 INFO:tasks.workunit.client.0.smithi094.stdout:5/425: symlink d7/d23/d5f/d14/d6f/l88 0 2024-03-24T10:45:55.956 INFO:tasks.workunit.client.0.smithi094.stdout:5/426: readlink d7/dc/d12/l1d 0 2024-03-24T10:45:55.957 INFO:tasks.workunit.client.0.smithi094.stdout:4/452: truncate d0/d22/f11 3045942 0 2024-03-24T10:45:55.957 INFO:tasks.workunit.client.0.smithi094.stdout:4/453: readlink d0/d34/l3c 0 2024-03-24T10:45:55.959 INFO:tasks.workunit.client.0.smithi094.stdout:2/327: write d18/f47 [2500492,82280] 0 2024-03-24T10:45:55.962 INFO:tasks.workunit.client.0.smithi094.stdout:7/410: mknod d1/d9/d35/d61/c8b 0 2024-03-24T10:45:55.962 INFO:tasks.workunit.client.0.smithi094.stdout:7/411: truncate d1/d9/d35/d65/f7d 1018792 0 2024-03-24T10:45:55.962 INFO:tasks.workunit.client.0.smithi094.stdout:3/312: dread d0/f3 [0,4194304] 0 2024-03-24T10:45:55.963 INFO:tasks.workunit.client.0.smithi094.stdout:3/313: truncate d0/db/dc/f12 1659033 0 2024-03-24T10:45:55.966 INFO:tasks.workunit.client.0.smithi094.stdout:0/382: sync 2024-03-24T10:45:55.966 INFO:tasks.workunit.client.0.smithi094.stdout:9/382: sync 2024-03-24T10:45:55.966 INFO:tasks.workunit.client.0.smithi094.stdout:9/383: chown df/d2d/d34/d2c/c3e 3733 1 2024-03-24T10:45:55.968 INFO:tasks.workunit.client.0.smithi094.stdout:5/427: creat d7/d23/d5f/d14/d6f/f89 x:0 0 0 2024-03-24T10:45:55.969 INFO:tasks.workunit.client.0.smithi094.stdout:5/428: dread - d7/d23/d5f/d14/f85 zero size 2024-03-24T10:45:55.969 INFO:tasks.workunit.client.0.smithi094.stdout:4/454: dread d0/d5/d1b/f39 [4194304,4194304] 0 2024-03-24T10:45:55.969 INFO:tasks.workunit.client.0.smithi094.stdout:0/383: dread d14/d22/d28/d48/f78 [0,4194304] 0 2024-03-24T10:45:55.969 INFO:tasks.workunit.client.0.smithi094.stdout:5/429: dread d7/dc/d56/f6e [0,4194304] 0 2024-03-24T10:45:55.970 INFO:tasks.workunit.client.0.smithi094.stdout:2/328: rename d18/f38 to d18/d48/f5d 0 2024-03-24T10:45:55.972 INFO:tasks.workunit.client.0.smithi094.stdout:7/412: mknod d1/d9/c8c 0 2024-03-24T10:45:55.972 INFO:tasks.workunit.client.0.smithi094.stdout:7/413: chown d1/d9/d16/d21/d49/f84 3 1 2024-03-24T10:45:55.972 INFO:tasks.workunit.client.0.smithi094.stdout:7/414: fsync d1/d9/f3c 0 2024-03-24T10:45:55.977 INFO:tasks.workunit.client.0.smithi094.stdout:1/332: sync 2024-03-24T10:45:55.978 INFO:tasks.workunit.client.0.smithi094.stdout:8/418: sync 2024-03-24T10:45:55.978 INFO:tasks.workunit.client.0.smithi094.stdout:8/419: write df/d25/d28/d31/d47/f77 [648844,73316] 0 2024-03-24T10:45:55.981 INFO:tasks.workunit.client.0.smithi094.stdout:4/455: creat d0/d5/d7d/f88 x:0 0 0 2024-03-24T10:45:55.981 INFO:tasks.workunit.client.0.smithi094.stdout:9/384: dwrite df/f52 [0,4194304] 0 2024-03-24T10:45:55.983 INFO:tasks.workunit.client.0.smithi094.stdout:0/384: symlink d14/d18/d85/l89 0 2024-03-24T10:45:55.983 INFO:tasks.workunit.client.0.smithi094.stdout:0/385: rename d14 to d14/d22/d28/d2b/d68/d8a 22 2024-03-24T10:45:55.984 INFO:tasks.workunit.client.0.smithi094.stdout:5/430: mknod d7/d23/d5f/d14/d36/d3a/d57/c8a 0 2024-03-24T10:45:55.984 INFO:tasks.workunit.client.0.smithi094.stdout:5/431: write d7/d23/d5f/d1b/f42 [2248223,35095] 0 2024-03-24T10:45:55.984 INFO:tasks.workunit.client.0.smithi094.stdout:5/432: write d7/d23/d2b/f5a [465083,86382] 0 2024-03-24T10:45:55.984 INFO:tasks.workunit.client.0.smithi094.stdout:5/433: creat d7/d23/d5f/f8b x:0 0 0 2024-03-24T10:45:55.985 INFO:tasks.workunit.client.0.smithi094.stdout:2/329: link d18/d1e/d34/f51 d18/d1e/d5a/f5e 0 2024-03-24T10:45:55.987 INFO:tasks.workunit.client.0.smithi094.stdout:6/405: rename d0/d5 to d0/d78 0 2024-03-24T10:45:55.987 INFO:tasks.workunit.client.0.smithi094.stdout:6/406: fdatasync d0/d30/f76 0 2024-03-24T10:45:55.988 INFO:tasks.workunit.client.0.smithi094.stdout:9/385: dread df/d2d/d34/d2c/f4c [0,4194304] 0 2024-03-24T10:45:55.989 INFO:tasks.workunit.client.0.smithi094.stdout:1/333: link d4/d9/db/l34 d4/d9/db/d5d/d63/l6d 0 2024-03-24T10:45:55.989 INFO:tasks.workunit.client.0.smithi094.stdout:7/415: dwrite d1/d9/d14/f36 [0,4194304] 0 2024-03-24T10:45:55.990 INFO:tasks.workunit.client.0.smithi094.stdout:8/420: creat df/d25/d36/d3f/f92 x:0 0 0 2024-03-24T10:45:55.991 INFO:tasks.workunit.client.0.smithi094.stdout:4/456: creat d0/d22/de/f89 x:0 0 0 2024-03-24T10:45:55.994 INFO:tasks.workunit.client.0.smithi094.stdout:0/386: symlink d14/d22/d28/d2b/l8b 0 2024-03-24T10:45:55.995 INFO:tasks.workunit.client.0.smithi094.stdout:5/434: symlink d7/d23/d5f/d14/d36/d3a/d57/l8c 0 2024-03-24T10:45:55.995 INFO:tasks.workunit.client.0.smithi094.stdout:5/435: dread - d7/d23/d5f/d14/d6f/f89 zero size 2024-03-24T10:45:55.995 INFO:tasks.workunit.client.0.smithi094.stdout:5/436: write d7/d23/d5f/d14/f85 [891330,101305] 0 2024-03-24T10:45:55.996 INFO:tasks.workunit.client.0.smithi094.stdout:2/330: mkdir d18/d5f 0 2024-03-24T10:45:55.997 INFO:tasks.workunit.client.0.smithi094.stdout:6/407: symlink d0/d1a/d35/l79 0 2024-03-24T10:45:55.997 INFO:tasks.workunit.client.0.smithi094.stdout:6/408: chown d0/d1a/l65 37891574 1 2024-03-24T10:45:55.997 INFO:tasks.workunit.client.0.smithi094.stdout:9/386: creat df/d2d/d1f/d11/f75 x:0 0 0 2024-03-24T10:45:55.998 INFO:tasks.workunit.client.0.smithi094.stdout:1/334: symlink d4/d9/d47/l6e 0 2024-03-24T10:45:56.004 INFO:tasks.workunit.client.0.smithi094.stdout:0/387: creat d14/d18/d85/f8c x:0 0 0 2024-03-24T10:45:56.005 INFO:tasks.workunit.client.0.smithi094.stdout:8/421: dread fa [4194304,4194304] 0 2024-03-24T10:45:56.006 INFO:tasks.workunit.client.0.smithi094.stdout:5/437: mknod d7/d23/d5f/d1b/c8d 0 2024-03-24T10:45:56.008 INFO:tasks.workunit.client.0.smithi094.stdout:2/331: creat d18/d5f/f60 x:0 0 0 2024-03-24T10:45:56.009 INFO:tasks.workunit.client.0.smithi094.stdout:5/438: write d7/d23/d5f/d1b/f42 [647110,91793] 0 2024-03-24T10:45:56.009 INFO:tasks.workunit.client.0.smithi094.stdout:6/409: mkdir d0/d78/d66/d7a 0 2024-03-24T10:45:56.012 INFO:tasks.workunit.client.0.smithi094.stdout:0/388: symlink d14/d22/d5b/l8d 0 2024-03-24T10:45:56.014 INFO:tasks.workunit.client.0.smithi094.stdout:8/422: symlink df/d25/d28/d31/d3c/l93 0 2024-03-24T10:45:56.014 INFO:tasks.workunit.client.0.smithi094.stdout:2/332: mkdir d18/d1e/d5a/d61 0 2024-03-24T10:45:56.015 INFO:tasks.workunit.client.0.smithi094.stdout:5/439: symlink d7/d23/d5f/d14/l8e 0 2024-03-24T10:45:56.016 INFO:tasks.workunit.client.0.smithi094.stdout:0/389: creat d14/d22/d28/d48/d7b/f8e x:0 0 0 2024-03-24T10:45:56.017 INFO:tasks.workunit.client.0.smithi094.stdout:8/423: link df/d25/f29 df/d12/d5b/d63/f94 0 2024-03-24T10:45:56.018 INFO:tasks.workunit.client.0.smithi094.stdout:2/333: getdents d18/d48 0 2024-03-24T10:45:56.018 INFO:tasks.workunit.client.0.smithi094.stdout:2/334: write d18/d48/f52 [197815,29039] 0 2024-03-24T10:45:56.018 INFO:tasks.workunit.client.0.smithi094.stdout:6/410: write d0/d21/d2a/f40 [3738297,119041] 0 2024-03-24T10:45:56.019 INFO:tasks.workunit.client.0.smithi094.stdout:2/335: rename d18/d1e/l40 to d18/d48/l62 0 2024-03-24T10:45:56.028 INFO:tasks.workunit.client.0.smithi094.stdout:9/387: dwrite df/f3f [4194304,4194304] 0 2024-03-24T10:45:56.028 INFO:tasks.workunit.client.0.smithi094.stdout:9/388: read df/f14 [383368,39572] 0 2024-03-24T10:45:56.029 INFO:tasks.workunit.client.0.smithi094.stdout:1/335: dwrite d4/f16 [0,4194304] 0 2024-03-24T10:45:56.029 INFO:tasks.workunit.client.0.smithi094.stdout:1/336: truncate d4/d9/db/d5d/f64 72369 0 2024-03-24T10:45:56.030 INFO:tasks.workunit.client.0.smithi094.stdout:6/411: read d0/d21/f2d [106150,102461] 0 2024-03-24T10:45:56.030 INFO:tasks.workunit.client.0.smithi094.stdout:4/457: dwrite d0/d22/de/d28/d3d/f5c [0,4194304] 0 2024-03-24T10:45:56.035 INFO:tasks.workunit.client.0.smithi094.stdout:6/412: link d0/d21/f2b d0/d1a/f7b 0 2024-03-24T10:45:56.036 INFO:tasks.workunit.client.0.smithi094.stdout:9/389: dread df/f3f [0,4194304] 0 2024-03-24T10:45:56.036 INFO:tasks.workunit.client.0.smithi094.stdout:9/390: readlink df/d2d/l55 0 2024-03-24T10:45:56.039 INFO:tasks.workunit.client.0.smithi094.stdout:4/458: truncate d0/d22/f11 258657 0 2024-03-24T10:45:56.040 INFO:tasks.workunit.client.0.smithi094.stdout:6/413: mkdir d0/d78/da/d23/d7c 0 2024-03-24T10:45:56.041 INFO:tasks.workunit.client.0.smithi094.stdout:9/391: rename df/f14 to df/d2d/d1f/d11/d43/f76 0 2024-03-24T10:45:56.042 INFO:tasks.workunit.client.0.smithi094.stdout:1/337: write f1 [2820107,109904] 0 2024-03-24T10:45:56.044 INFO:tasks.workunit.client.0.smithi094.stdout:4/459: unlink d0/d5/d42/l80 0 2024-03-24T10:45:56.044 INFO:tasks.workunit.client.0.smithi094.stdout:4/460: creat d0/d5/d7d/f8a x:0 0 0 2024-03-24T10:45:56.044 INFO:tasks.workunit.client.0.smithi094.stdout:4/461: chown d0/d22/de/d28/f3a 3153 1 2024-03-24T10:45:56.044 INFO:tasks.workunit.client.0.smithi094.stdout:4/462: chown d0/d22/de/d28/d2b/d7a/f7b 786063 1 2024-03-24T10:45:56.044 INFO:tasks.workunit.client.0.smithi094.stdout:6/414: symlink d0/d78/da/d23/d34/d4e/l7d 0 2024-03-24T10:45:56.047 INFO:tasks.workunit.client.0.smithi094.stdout:6/415: dread d0/d3b/f69 [0,4194304] 0 2024-03-24T10:45:56.049 INFO:tasks.workunit.client.0.smithi094.stdout:2/336: dwrite f8 [0,4194304] 0 2024-03-24T10:45:56.049 INFO:tasks.workunit.client.0.smithi094.stdout:0/390: dwrite d14/d22/d32/f80 [0,4194304] 0 2024-03-24T10:45:56.049 INFO:tasks.workunit.client.0.smithi094.stdout:0/391: readlink d14/d22/d28/d2b/l6d 0 2024-03-24T10:45:56.049 INFO:tasks.workunit.client.0.smithi094.stdout:0/392: chown d14/d18/c7a 374 1 2024-03-24T10:45:56.049 INFO:tasks.workunit.client.0.smithi094.stdout:5/440: dwrite d7/d23/d5f/f83 [0,4194304] 0 2024-03-24T10:45:56.050 INFO:tasks.workunit.client.0.smithi094.stdout:8/424: dwrite df/d25/d36/d3f/f54 [0,4194304] 0 2024-03-24T10:45:56.051 INFO:tasks.workunit.client.0.smithi094.stdout:4/463: creat d0/d5/d1b/d4f/d70/d6d/f8b x:0 0 0 2024-03-24T10:45:56.051 INFO:tasks.workunit.client.0.smithi094.stdout:4/464: readlink d0/d5/l29 0 2024-03-24T10:45:56.052 INFO:tasks.workunit.client.0.smithi094.stdout:2/337: dread d18/d1e/f33 [0,4194304] 0 2024-03-24T10:45:56.052 INFO:tasks.workunit.client.0.smithi094.stdout:2/338: readlink d18/d1e/l20 0 2024-03-24T10:45:56.053 INFO:tasks.workunit.client.0.smithi094.stdout:1/338: rename d4/d36/d38 to d4/d9/de/d6f 0 2024-03-24T10:45:56.056 INFO:tasks.workunit.client.0.smithi094.stdout:9/392: dwrite df/d2d/f1e [4194304,4194304] 0 2024-03-24T10:45:56.056 INFO:tasks.workunit.client.0.smithi094.stdout:9/393: truncate df/d2d/d1f/f4a 1096817 0 2024-03-24T10:45:56.056 INFO:tasks.workunit.client.0.smithi094.stdout:9/394: write df/d2d/d1f/d11/f4e [556849,81573] 0 2024-03-24T10:45:56.061 INFO:tasks.workunit.client.0.smithi094.stdout:5/441: getdents d7/d23/d5f/d14/d36 0 2024-03-24T10:45:56.066 INFO:tasks.workunit.client.0.smithi094.stdout:6/416: dwrite d0/f49 [0,4194304] 0 2024-03-24T10:45:56.066 INFO:tasks.workunit.client.0.smithi094.stdout:6/417: fsync d0/d1a/f7b 0 2024-03-24T10:45:56.076 INFO:tasks.workunit.client.0.smithi094.stdout:8/425: mkdir df/d25/d28/d31/d95 0 2024-03-24T10:45:56.076 INFO:tasks.workunit.client.0.smithi094.stdout:1/339: creat d4/d36/f70 x:0 0 0 2024-03-24T10:45:56.078 INFO:tasks.workunit.client.0.smithi094.stdout:0/393: rename d14/f7f to d14/d22/d5b/d86/f8f 0 2024-03-24T10:45:56.081 INFO:tasks.workunit.client.0.smithi094.stdout:6/418: creat d0/d78/da/d23/d34/d4e/f7e x:0 0 0 2024-03-24T10:45:56.084 INFO:tasks.workunit.client.0.smithi094.stdout:0/394: dread d14/d22/f2f [0,4194304] 0 2024-03-24T10:45:56.088 INFO:tasks.workunit.client.0.smithi094.stdout:6/419: write d0/f1e [2488189,15357] 0 2024-03-24T10:45:56.091 INFO:tasks.workunit.client.0.smithi094.stdout:8/426: dread df/d25/d36/d3f/f54 [0,4194304] 0 2024-03-24T10:45:56.091 INFO:tasks.workunit.client.0.smithi094.stdout:8/427: write df/d12/d5b/d63/f94 [4567210,64175] 0 2024-03-24T10:45:56.091 INFO:tasks.workunit.client.0.smithi094.stdout:8/428: read f3 [4065926,34200] 0 2024-03-24T10:45:56.092 INFO:tasks.workunit.client.0.smithi094.stdout:8/429: readlink df/d12/l4d 0 2024-03-24T10:45:56.092 INFO:tasks.workunit.client.0.smithi094.stdout:2/339: dwrite d18/d5f/f60 [0,4194304] 0 2024-03-24T10:45:56.092 INFO:tasks.workunit.client.0.smithi094.stdout:1/340: write d4/f2a [1985763,127597] 0 2024-03-24T10:45:56.094 INFO:tasks.workunit.client.0.smithi094.stdout:5/442: dwrite d7/d23/d5f/d1b/f42 [0,4194304] 0 2024-03-24T10:45:56.094 INFO:tasks.workunit.client.0.smithi094.stdout:5/443: fsync d7/d23/d5f/f58 0 2024-03-24T10:45:56.094 INFO:tasks.workunit.client.0.smithi094.stdout:9/395: dwrite df/f3f [4194304,4194304] 0 2024-03-24T10:45:56.095 INFO:tasks.workunit.client.0.smithi094.stdout:9/396: stat df/c42 0 2024-03-24T10:45:56.095 INFO:tasks.workunit.client.0.smithi094.stdout:8/430: mkdir df/d12/d5b/d63/d75/d96 0 2024-03-24T10:45:56.098 INFO:tasks.workunit.client.0.smithi094.stdout:2/340: mknod d18/d1e/d34/d4a/c63 0 2024-03-24T10:45:56.099 INFO:tasks.workunit.client.0.smithi094.stdout:2/341: write d18/d1e/d34/f49 [300929,77503] 0 2024-03-24T10:45:56.099 INFO:tasks.workunit.client.0.smithi094.stdout:2/342: chown d18/d1e/d5a/f5e 144245 1 2024-03-24T10:45:56.099 INFO:tasks.workunit.client.0.smithi094.stdout:1/341: symlink d4/d9/db/d39/l71 0 2024-03-24T10:45:56.099 INFO:tasks.workunit.client.0.smithi094.stdout:1/342: dread - d4/d9/de/d6f/d5f/f68 zero size 2024-03-24T10:45:56.100 INFO:tasks.workunit.client.0.smithi094.stdout:5/444: creat d7/d23/d2b/d68/f8f x:0 0 0 2024-03-24T10:45:56.100 INFO:tasks.workunit.client.0.smithi094.stdout:5/445: fdatasync d7/dc/d12/f39 0 2024-03-24T10:45:56.100 INFO:tasks.workunit.client.0.smithi094.stdout:3/314: sync 2024-03-24T10:45:56.102 INFO:tasks.workunit.client.0.smithi094.stdout:8/431: write df/d25/d28/d31/d3c/f5f [11661,101527] 0 2024-03-24T10:45:56.102 INFO:tasks.workunit.client.0.smithi094.stdout:8/432: write df/d25/d28/d31/d47/d22/d55/f7a [1557007,13864] 0 2024-03-24T10:45:56.104 INFO:tasks.workunit.client.0.smithi094.stdout:2/343: dread d18/f27 [0,4194304] 0 2024-03-24T10:45:56.104 INFO:tasks.workunit.client.0.smithi094.stdout:1/343: mknod d4/c72 0 2024-03-24T10:45:56.104 INFO:tasks.workunit.client.0.smithi094.stdout:1/344: creat d4/d36/f73 x:0 0 0 2024-03-24T10:45:56.105 INFO:tasks.workunit.client.0.smithi094.stdout:5/446: mknod d7/c90 0 2024-03-24T10:45:56.105 INFO:tasks.workunit.client.0.smithi094.stdout:5/447: readlink d7/d23/d2b/l38 0 2024-03-24T10:45:56.105 INFO:tasks.workunit.client.0.smithi094.stdout:5/448: creat d7/d23/d5f/d14/d6f/f91 x:0 0 0 2024-03-24T10:45:56.105 INFO:tasks.workunit.client.0.smithi094.stdout:5/449: stat d7/d23/d5f/fa 0 2024-03-24T10:45:56.105 INFO:tasks.workunit.client.0.smithi094.stdout:5/450: chown d7/d26 387433 1 2024-03-24T10:45:56.105 INFO:tasks.workunit.client.0.smithi094.stdout:5/451: dread - d7/d23/d5f/d14/f6d zero size 2024-03-24T10:45:56.106 INFO:tasks.workunit.client.0.smithi094.stdout:2/344: mknod d18/d48/d56/c64 0 2024-03-24T10:45:56.108 INFO:tasks.workunit.client.0.smithi094.stdout:0/395: dwrite d14/d18/f69 [0,4194304] 0 2024-03-24T10:45:56.110 INFO:tasks.workunit.client.0.smithi094.stdout:2/345: mkdir d18/d1e/d65 0 2024-03-24T10:45:56.112 INFO:tasks.workunit.client.0.smithi094.stdout:1/345: mknod d4/d9/db/d31/d3c/c74 0 2024-03-24T10:45:56.113 INFO:tasks.workunit.client.0.smithi094.stdout:1/346: dread - d4/d9/f28 zero size 2024-03-24T10:45:56.113 INFO:tasks.workunit.client.0.smithi094.stdout:8/433: write df/d25/d36/d3f/f54 [3240555,50179] 0 2024-03-24T10:45:56.113 INFO:tasks.workunit.client.0.smithi094.stdout:8/434: creat df/d25/f97 x:0 0 0 2024-03-24T10:45:56.114 INFO:tasks.workunit.client.0.smithi094.stdout:8/435: chown df/d25/d36/d3f/c71 0 1 2024-03-24T10:45:56.114 INFO:tasks.workunit.client.0.smithi094.stdout:5/452: dread d7/d23/d5f/d1b/f37 [4194304,4194304] 0 2024-03-24T10:45:56.114 INFO:tasks.workunit.client.0.smithi094.stdout:8/436: dread df/d25/d28/d31/d47/d22/f38 [0,4194304] 0 2024-03-24T10:45:56.115 INFO:tasks.workunit.client.0.smithi094.stdout:8/437: readlink l7 0 2024-03-24T10:45:56.115 INFO:tasks.workunit.client.0.smithi094.stdout:8/438: write df/d25/d36/d3f/f54 [639891,79865] 0 2024-03-24T10:45:56.115 INFO:tasks.workunit.client.0.smithi094.stdout:0/396: mknod d14/d18/d85/c90 0 2024-03-24T10:45:56.115 INFO:tasks.workunit.client.0.smithi094.stdout:0/397: chown d14/c19 65313 1 2024-03-24T10:45:56.116 INFO:tasks.workunit.client.0.smithi094.stdout:3/315: dwrite d0/db/d1c/d20/d23/f3f [0,4194304] 0 2024-03-24T10:45:56.117 INFO:tasks.workunit.client.0.smithi094.stdout:2/346: mkdir d18/d66 0 2024-03-24T10:45:56.119 INFO:tasks.workunit.client.0.smithi094.stdout:1/347: mkdir d4/d9/db/d39/d75 0 2024-03-24T10:45:56.119 INFO:tasks.workunit.client.0.smithi094.stdout:1/348: write d4/d9/de/f2b [1264176,53664] 0 2024-03-24T10:45:56.123 INFO:tasks.workunit.client.0.smithi094.stdout:5/453: creat d7/d23/d5f/d9/f92 x:0 0 0 2024-03-24T10:45:56.123 INFO:tasks.workunit.client.0.smithi094.stdout:5/454: creat d7/dc/d56/f93 x:0 0 0 2024-03-24T10:45:56.123 INFO:tasks.workunit.client.0.smithi094.stdout:5/455: creat d7/d23/d5f/d14/d6f/f94 x:0 0 0 2024-03-24T10:45:56.123 INFO:tasks.workunit.client.0.smithi094.stdout:5/456: truncate d7/d23/d5f/d14/d6f/f91 39277 0 2024-03-24T10:45:56.123 INFO:tasks.workunit.client.0.smithi094.stdout:5/457: write d7/d23/d5f/d1b/f37 [5166708,95223] 0 2024-03-24T10:45:56.123 INFO:tasks.workunit.client.0.smithi094.stdout:5/458: fdatasync d7/d23/d5f/d14/d36/f47 0 2024-03-24T10:45:56.124 INFO:tasks.workunit.client.0.smithi094.stdout:8/439: mkdir df/d25/d36/d98 0 2024-03-24T10:45:56.124 INFO:tasks.workunit.client.0.smithi094.stdout:8/440: truncate df/d25/d28/d31/d47/f7c 524396 0 2024-03-24T10:45:56.126 INFO:tasks.workunit.client.0.smithi094.stdout:0/398: symlink d14/d22/d7d/l91 0 2024-03-24T10:45:56.131 INFO:tasks.workunit.client.0.smithi094.stdout:2/347: creat d18/d1e/d65/f67 x:0 0 0 2024-03-24T10:45:56.132 INFO:tasks.workunit.client.0.smithi094.stdout:7/416: sync 2024-03-24T10:45:56.134 INFO:tasks.workunit.client.0.smithi094.stdout:9/397: truncate df/f3f 7753370 0 2024-03-24T10:45:56.135 INFO:tasks.workunit.client.0.smithi094.stdout:7/417: dread d1/d6c/f71 [0,4194304] 0 2024-03-24T10:45:56.138 INFO:tasks.workunit.client.0.smithi094.stdout:0/399: getdents d14/d22/d5b 0 2024-03-24T10:45:56.139 INFO:tasks.workunit.client.0.smithi094.stdout:2/348: getdents d18/d1e/d34 0 2024-03-24T10:45:56.140 INFO:tasks.workunit.client.0.smithi094.stdout:2/349: rename d18/d1e/d5a/d61 to d18/d1e/d5a/d61/d68 22 2024-03-24T10:45:56.141 INFO:tasks.workunit.client.0.smithi094.stdout:9/398: creat df/d2d/d1f/d58/d5a/f77 x:0 0 0 2024-03-24T10:45:56.143 INFO:tasks.workunit.client.0.smithi094.stdout:7/418: truncate d1/d9/f22 481694 0 2024-03-24T10:45:56.143 INFO:tasks.workunit.client.0.smithi094.stdout:7/419: write d1/d6c/f71 [366934,75707] 0 2024-03-24T10:45:56.143 INFO:tasks.workunit.client.0.smithi094.stdout:7/420: fdatasync d1/d6c/d80/f69 0 2024-03-24T10:45:56.143 INFO:tasks.workunit.client.0.smithi094.stdout:7/421: dread - d1/f5d zero size 2024-03-24T10:45:56.143 INFO:tasks.workunit.client.0.smithi094.stdout:7/422: chown d1/d9/f30 263309813 1 2024-03-24T10:45:56.143 INFO:tasks.workunit.client.0.smithi094.stdout:7/423: truncate d1/d9/f2c 456989 0 2024-03-24T10:45:56.144 INFO:tasks.workunit.client.0.smithi094.stdout:7/424: write f0 [2386686,66796] 0 2024-03-24T10:45:56.144 INFO:tasks.workunit.client.0.smithi094.stdout:7/425: truncate d1/d9/d16/f2f 2876177 0 2024-03-24T10:45:56.144 INFO:tasks.workunit.client.0.smithi094.stdout:7/426: fdatasync d1/d9/d25/d3d/f73 0 2024-03-24T10:45:56.144 INFO:tasks.workunit.client.0.smithi094.stdout:2/350: mkdir d18/d1e/d34/d69 0 2024-03-24T10:45:56.144 INFO:tasks.workunit.client.0.smithi094.stdout:2/351: write d18/f3f [623246,97499] 0 2024-03-24T10:45:56.144 INFO:tasks.workunit.client.0.smithi094.stdout:0/400: read d14/d22/d28/d2b/f38 [4165394,32067] 0 2024-03-24T10:45:56.145 INFO:tasks.workunit.client.0.smithi094.stdout:9/399: rename df/d2d/f44 to df/d2d/d1f/d58/d5a/d69/f78 0 2024-03-24T10:45:56.148 INFO:tasks.workunit.client.0.smithi094.stdout:7/427: symlink d1/d6c/d80/d81/l8d 0 2024-03-24T10:45:56.148 INFO:tasks.workunit.client.0.smithi094.stdout:7/428: fsync d1/f5d 0 2024-03-24T10:45:56.148 INFO:tasks.workunit.client.0.smithi094.stdout:7/429: fsync d1/f10 0 2024-03-24T10:45:56.148 INFO:tasks.workunit.client.0.smithi094.stdout:7/430: write d1/d9/d25/d3d/f55 [4708924,107213] 0 2024-03-24T10:45:56.148 INFO:tasks.workunit.client.0.smithi094.stdout:7/431: write d1/d6c/d80/f7e [739616,60396] 0 2024-03-24T10:45:56.148 INFO:tasks.workunit.client.0.smithi094.stdout:7/432: creat d1/d9/d16/d21/d51/f8e x:0 0 0 2024-03-24T10:45:56.148 INFO:tasks.workunit.client.0.smithi094.stdout:7/433: dread - d1/d6c/d80/f69 zero size 2024-03-24T10:45:56.148 INFO:tasks.workunit.client.0.smithi094.stdout:2/352: dread - d18/d1e/d34/f46 zero size 2024-03-24T10:45:56.149 INFO:tasks.workunit.client.0.smithi094.stdout:9/400: mkdir df/d2d/d1f/d58/d5a/d79 0 2024-03-24T10:45:56.151 INFO:tasks.workunit.client.0.smithi094.stdout:7/434: mkdir d1/d9/d14/d8f 0 2024-03-24T10:45:56.153 INFO:tasks.workunit.client.0.smithi094.stdout:9/401: mkdir df/d2d/d1f/d58/d5a/d69/d7a 0 2024-03-24T10:45:56.158 INFO:tasks.workunit.client.0.smithi094.stdout:1/349: dwrite d4/d36/f73 [0,4194304] 0 2024-03-24T10:45:56.158 INFO:tasks.workunit.client.0.smithi094.stdout:5/459: dwrite d7/d26/f80 [0,4194304] 0 2024-03-24T10:45:56.158 INFO:tasks.workunit.client.0.smithi094.stdout:5/460: fdatasync d7/d26/f53 0 2024-03-24T10:45:56.158 INFO:tasks.workunit.client.0.smithi094.stdout:5/461: chown d7/d23/d5f/d14/d36/l76 205768 1 2024-03-24T10:45:56.158 INFO:tasks.workunit.client.0.smithi094.stdout:5/462: write d7/d23/d5f/fa [1671520,24906] 0 2024-03-24T10:45:56.158 INFO:tasks.workunit.client.0.smithi094.stdout:5/463: chown d7/d23/d2b/l45 425771 1 2024-03-24T10:45:56.158 INFO:tasks.workunit.client.0.smithi094.stdout:5/464: chown d7/d23/d2b 258676142 1 2024-03-24T10:45:56.158 INFO:tasks.workunit.client.0.smithi094.stdout:8/441: dwrite f6 [0,4194304] 0 2024-03-24T10:45:56.159 INFO:tasks.workunit.client.0.smithi094.stdout:8/442: stat f6 0 2024-03-24T10:45:56.160 INFO:tasks.workunit.client.0.smithi094.stdout:0/401: read d14/d18/f54 [289148,55813] 0 2024-03-24T10:45:56.161 INFO:tasks.workunit.client.0.smithi094.stdout:1/350: mkdir d4/d9/d37/d33/d76 0 2024-03-24T10:45:56.164 INFO:tasks.workunit.client.0.smithi094.stdout:5/465: mknod d7/d23/d5f/d14/d6f/c95 0 2024-03-24T10:45:56.164 INFO:tasks.workunit.client.0.smithi094.stdout:5/466: chown d7/d23/d5f/d14/d6f/f94 209 1 2024-03-24T10:45:56.165 INFO:tasks.workunit.client.0.smithi094.stdout:8/443: getdents df/d8c 0 2024-03-24T10:45:56.165 INFO:tasks.workunit.client.0.smithi094.stdout:8/444: dread - df/d25/d28/d31/d47/d22/d55/f76 zero size 2024-03-24T10:45:56.165 INFO:tasks.workunit.client.0.smithi094.stdout:8/445: chown df/d8c 7503 1 2024-03-24T10:45:56.165 INFO:tasks.workunit.client.0.smithi094.stdout:4/465: sync 2024-03-24T10:45:56.167 INFO:tasks.workunit.client.0.smithi094.stdout:4/466: dread d0/d22/f11 [0,4194304] 0 2024-03-24T10:45:56.167 INFO:tasks.workunit.client.0.smithi094.stdout:4/467: readlink d0/d22/l53 0 2024-03-24T10:45:56.168 INFO:tasks.workunit.client.0.smithi094.stdout:2/353: dwrite d18/f23 [0,4194304] 0 2024-03-24T10:45:56.168 INFO:tasks.workunit.client.0.smithi094.stdout:2/354: creat d18/d1e/d65/f6a x:0 0 0 2024-03-24T10:45:56.172 INFO:tasks.workunit.client.0.smithi094.stdout:0/402: symlink d14/d18/d85/l92 0 2024-03-24T10:45:56.173 INFO:tasks.workunit.client.0.smithi094.stdout:1/351: rename d4/d9/db/f21 to d4/d9/f77 0 2024-03-24T10:45:56.173 INFO:tasks.workunit.client.0.smithi094.stdout:1/352: chown d4/d9/db/d5d 6175 1 2024-03-24T10:45:56.173 INFO:tasks.workunit.client.0.smithi094.stdout:1/353: chown d4/d9/db/d5d/f64 3664953 1 2024-03-24T10:45:56.176 INFO:tasks.workunit.client.0.smithi094.stdout:5/467: unlink d7/d23/d5f/fa 0 2024-03-24T10:45:56.176 INFO:tasks.workunit.client.0.smithi094.stdout:5/468: dread - d7/d23/d5f/d14/f6d zero size 2024-03-24T10:45:56.176 INFO:tasks.workunit.client.0.smithi094.stdout:5/469: fsync d7/d23/d5f/d14/d36/f47 0 2024-03-24T10:45:56.176 INFO:tasks.workunit.client.0.smithi094.stdout:5/470: write d7/d23/f77 [970099,21758] 0 2024-03-24T10:45:56.177 INFO:tasks.workunit.client.0.smithi094.stdout:4/468: getdents d0/d22/de/d28/d3d 0 2024-03-24T10:45:56.180 INFO:tasks.workunit.client.0.smithi094.stdout:0/403: truncate d14/d22/d28/d48/f4a 1593965 0 2024-03-24T10:45:56.180 INFO:tasks.workunit.client.0.smithi094.stdout:0/404: write d14/d22/f2f [105857,428] 0 2024-03-24T10:45:56.181 INFO:tasks.workunit.client.0.smithi094.stdout:8/446: dwrite df/d25/d36/d3f/f8e [0,4194304] 0 2024-03-24T10:45:56.181 INFO:tasks.workunit.client.0.smithi094.stdout:2/355: write d18/f23 [3973967,5421] 0 2024-03-24T10:45:56.182 INFO:tasks.workunit.client.0.smithi094.stdout:1/354: symlink d4/d9/de/d3d/d66/l78 0 2024-03-24T10:45:56.182 INFO:tasks.workunit.client.0.smithi094.stdout:1/355: readlink d4/d9/de/l60 0 2024-03-24T10:45:56.182 INFO:tasks.workunit.client.0.smithi094.stdout:1/356: read d4/f2a [281916,82298] 0 2024-03-24T10:45:56.183 INFO:tasks.workunit.client.0.smithi094.stdout:5/471: symlink d7/d23/d5f/d14/l96 0 2024-03-24T10:45:56.183 INFO:tasks.workunit.client.0.smithi094.stdout:4/469: dread d0/d5/f16 [0,4194304] 0 2024-03-24T10:45:56.184 INFO:tasks.workunit.client.0.smithi094.stdout:4/470: fdatasync d0/d22/de/d28/f72 0 2024-03-24T10:45:56.184 INFO:tasks.workunit.client.0.smithi094.stdout:4/471: truncate d0/d22/f76 4702998 0 2024-03-24T10:45:56.184 INFO:tasks.workunit.client.0.smithi094.stdout:4/472: chown d0/d5/d42/d4a/c6f 38389845 1 2024-03-24T10:45:56.185 INFO:tasks.workunit.client.0.smithi094.stdout:0/405: dread d14/d22/f35 [0,4194304] 0 2024-03-24T10:45:56.185 INFO:tasks.workunit.client.0.smithi094.stdout:9/402: getdents df/d2d/d1f/d58/d5a 0 2024-03-24T10:45:56.185 INFO:tasks.workunit.client.0.smithi094.stdout:9/403: write df/d2d/d1f/d58/d5a/f77 [377627,15581] 0 2024-03-24T10:45:56.185 INFO:tasks.workunit.client.0.smithi094.stdout:9/404: write df/d2d/d34/d2c/f73 [121059,46217] 0 2024-03-24T10:45:56.185 INFO:tasks.workunit.client.0.smithi094.stdout:9/405: stat df/d2d/d34/d2c/d74 0 2024-03-24T10:45:56.185 INFO:tasks.workunit.client.0.smithi094.stdout:9/406: write df/d2d/d1f/f6e [589699,97492] 0 2024-03-24T10:45:56.188 INFO:tasks.workunit.client.0.smithi094.stdout:7/435: truncate d1/d9/d25/d3d/f55 4684948 0 2024-03-24T10:45:56.188 INFO:tasks.workunit.client.0.smithi094.stdout:7/436: dread - d1/d9/d16/d21/d51/f8e zero size 2024-03-24T10:45:56.191 INFO:tasks.workunit.client.0.smithi094.stdout:8/447: rename df/d12/d5b/c65 to df/d25/d28/d31/d47/c99 0 2024-03-24T10:45:56.191 INFO:tasks.workunit.client.0.smithi094.stdout:0/406: rename d14/d18/d85 to d14/d18/d85/d93 22 2024-03-24T10:45:56.191 INFO:tasks.workunit.client.0.smithi094.stdout:8/448: write df/d25/d36/d3f/f4f [5312864,85008] 0 2024-03-24T10:45:56.195 INFO:tasks.workunit.client.0.smithi094.stdout:4/473: mkdir d0/d22/de/d28/d3d/d8c 0 2024-03-24T10:45:56.195 INFO:tasks.workunit.client.0.smithi094.stdout:4/474: creat d0/d5/d7d/f8d x:0 0 0 2024-03-24T10:45:56.197 INFO:tasks.workunit.client.0.smithi094.stdout:9/407: mkdir df/d7b 0 2024-03-24T10:45:56.198 INFO:tasks.workunit.client.0.smithi094.stdout:7/437: getdents d1/d9/d14/d34 0 2024-03-24T10:45:56.198 INFO:tasks.workunit.client.0.smithi094.stdout:7/438: write d1/d6c/d80/f6f [994552,33513] 0 2024-03-24T10:45:56.200 INFO:tasks.workunit.client.0.smithi094.stdout:0/407: truncate d14/d22/d28/d2b/d68/f72 1181182 0 2024-03-24T10:45:56.200 INFO:tasks.workunit.client.0.smithi094.stdout:0/408: creat d14/d22/d28/d48/d7b/f94 x:0 0 0 2024-03-24T10:45:56.205 INFO:tasks.workunit.client.0.smithi094.stdout:9/408: symlink df/d2d/d34/d2c/d74/l7c 0 2024-03-24T10:45:56.205 INFO:tasks.workunit.client.0.smithi094.stdout:7/439: creat d1/d9/d16/d21/d51/d83/f90 x:0 0 0 2024-03-24T10:45:56.208 INFO:tasks.workunit.client.0.smithi094.stdout:9/409: creat df/d2d/d1f/d58/d5a/d69/d7a/f7d x:0 0 0 2024-03-24T10:45:56.209 INFO:tasks.workunit.client.0.smithi094.stdout:5/472: dwrite d7/d23/d5f/d9/f49 [0,4194304] 0 2024-03-24T10:45:56.210 INFO:tasks.workunit.client.0.smithi094.stdout:5/473: creat d7/d23/d5f/f97 x:0 0 0 2024-03-24T10:45:56.213 INFO:tasks.workunit.client.0.smithi094.stdout:9/410: mkdir df/d2d/d1f/d7e 0 2024-03-24T10:45:56.217 INFO:tasks.workunit.client.0.smithi094.stdout:8/449: dwrite fa [4194304,4194304] 0 2024-03-24T10:45:56.217 INFO:tasks.workunit.client.0.smithi094.stdout:5/474: mknod d7/d23/d5f/d14/d6f/c98 0 2024-03-24T10:45:56.217 INFO:tasks.workunit.client.0.smithi094.stdout:5/475: chown d7/d23/d5f/d14/d36/f4d 30324 1 2024-03-24T10:45:56.218 INFO:tasks.workunit.client.0.smithi094.stdout:5/476: creat d7/d23/d5f/f99 x:0 0 0 2024-03-24T10:45:56.219 INFO:tasks.workunit.client.0.smithi094.stdout:8/450: stat df/d25/d36/l91 0 2024-03-24T10:45:56.219 INFO:tasks.workunit.client.0.smithi094.stdout:8/451: chown df/d8c/d8f 9884 1 2024-03-24T10:45:56.221 INFO:tasks.workunit.client.0.smithi094.stdout:5/477: readlink d7/d23/d5f/d9/l40 0 2024-03-24T10:45:56.221 INFO:tasks.workunit.client.0.smithi094.stdout:5/478: truncate d7/d23/d5f/f99 804571 0 2024-03-24T10:45:56.221 INFO:tasks.workunit.client.0.smithi094.stdout:5/479: fdatasync d7/d23/d5f/d14/f35 0 2024-03-24T10:45:56.221 INFO:tasks.workunit.client.0.smithi094.stdout:7/440: write d1/d9/d16/d21/d51/d83/d39/f48 [63628,6797] 0 2024-03-24T10:45:56.221 INFO:tasks.workunit.client.0.smithi094.stdout:7/441: write d1/d6c/f71 [1068885,75628] 0 2024-03-24T10:45:56.222 INFO:tasks.workunit.client.0.smithi094.stdout:8/452: mknod df/d25/d28/d31/d47/c9a 0 2024-03-24T10:45:56.225 INFO:tasks.workunit.client.0.smithi094.stdout:7/442: rename d1/d9/d25/f5f to d1/d9/d14/d8f/f91 0 2024-03-24T10:45:56.228 INFO:tasks.workunit.client.0.smithi094.stdout:6/420: sync 2024-03-24T10:45:56.228 INFO:tasks.workunit.client.0.smithi094.stdout:3/316: sync 2024-03-24T10:45:56.228 INFO:tasks.workunit.client.0.smithi094.stdout:9/411: write df/d2d/d1f/f3d [386500,71043] 0 2024-03-24T10:45:56.228 INFO:tasks.workunit.client.0.smithi094.stdout:7/443: symlink d1/d9/d14/d34/l92 0 2024-03-24T10:45:56.229 INFO:tasks.workunit.client.0.smithi094.stdout:9/412: getdents df/d2d/d1f/d58/d5a 0 2024-03-24T10:45:56.229 INFO:tasks.workunit.client.0.smithi094.stdout:8/453: rmdir df/d25/d36/d3f 39 2024-03-24T10:45:56.231 INFO:tasks.workunit.client.0.smithi094.stdout:6/421: creat d0/d78/da/d23/d34/d4e/d5d/f7f x:0 0 0 2024-03-24T10:45:56.233 INFO:tasks.workunit.client.0.smithi094.stdout:9/413: creat df/d2d/d1f/d11/d53/f7f x:0 0 0 2024-03-24T10:45:56.234 INFO:tasks.workunit.client.0.smithi094.stdout:8/454: rename df/d25/d28/d31/d47/d22/d55/f76 to df/d25/d36/f9b 0 2024-03-24T10:45:56.235 INFO:tasks.workunit.client.0.smithi094.stdout:9/414: mkdir df/d2d/d1f/d11/d66/d80 0 2024-03-24T10:45:56.235 INFO:tasks.workunit.client.0.smithi094.stdout:8/455: write df/d12/f2f [503874,20613] 0 2024-03-24T10:45:56.237 INFO:tasks.workunit.client.0.smithi094.stdout:2/356: dwrite d18/d1e/d34/f51 [0,4194304] 0 2024-03-24T10:45:56.237 INFO:tasks.workunit.client.0.smithi094.stdout:2/357: stat d18/f3f 0 2024-03-24T10:45:56.237 INFO:tasks.workunit.client.0.smithi094.stdout:6/422: rename d0/d21/d2a/d2f/f46 to d0/d78/da/d23/d34/d4e/f80 0 2024-03-24T10:45:56.238 INFO:tasks.workunit.client.0.smithi094.stdout:3/317: dread d0/db/dc/d28/d3c/f6f [0,4194304] 0 2024-03-24T10:45:56.240 INFO:tasks.workunit.client.0.smithi094.stdout:2/358: mkdir d18/d48/d6b 0 2024-03-24T10:45:56.241 INFO:tasks.workunit.client.0.smithi094.stdout:2/359: write d18/f4e [244195,101596] 0 2024-03-24T10:45:56.241 INFO:tasks.workunit.client.0.smithi094.stdout:1/357: dwrite d4/d9/db/d31/d3c/f50 [4194304,4194304] 0 2024-03-24T10:45:56.243 INFO:tasks.workunit.client.0.smithi094.stdout:6/423: unlink d0/d78/da/c1f 0 2024-03-24T10:45:56.243 INFO:tasks.workunit.client.0.smithi094.stdout:6/424: truncate d0/d21/f2d 4871477 0 2024-03-24T10:45:56.243 INFO:tasks.workunit.client.0.smithi094.stdout:6/425: write d0/d1a/d35/f72 [766163,8420] 0 2024-03-24T10:45:56.245 INFO:tasks.workunit.client.0.smithi094.stdout:9/415: getdents df 0 2024-03-24T10:45:56.245 INFO:tasks.workunit.client.0.smithi094.stdout:9/416: chown df/d2d 0 1 2024-03-24T10:45:56.245 INFO:tasks.workunit.client.0.smithi094.stdout:9/417: creat df/d2d/d1f/d11/d53/f81 x:0 0 0 2024-03-24T10:45:56.247 INFO:tasks.workunit.client.0.smithi094.stdout:0/409: dwrite d14/d18/f54 [0,4194304] 0 2024-03-24T10:45:56.248 INFO:tasks.workunit.client.0.smithi094.stdout:4/475: dwrite d0/d5/d7d/f8a [0,4194304] 0 2024-03-24T10:45:56.251 INFO:tasks.workunit.client.0.smithi094.stdout:6/426: creat d0/d30/d50/f81 x:0 0 0 2024-03-24T10:45:56.251 INFO:tasks.workunit.client.0.smithi094.stdout:6/427: stat d0/d3b 0 2024-03-24T10:45:56.252 INFO:tasks.workunit.client.0.smithi094.stdout:8/456: write fd [1734754,85226] 0 2024-03-24T10:45:56.252 INFO:tasks.workunit.client.0.smithi094.stdout:8/457: chown df/d25/f58 0 1 2024-03-24T10:45:56.254 INFO:tasks.workunit.client.0.smithi094.stdout:0/410: mknod d14/d18/d85/c95 0 2024-03-24T10:45:56.254 INFO:tasks.workunit.client.0.smithi094.stdout:0/411: chown d14/d22/d28/f6c 547 1 2024-03-24T10:45:56.258 INFO:tasks.workunit.client.0.smithi094.stdout:2/360: creat d18/d1e/d34/d4a/f6c x:0 0 0 2024-03-24T10:45:56.258 INFO:tasks.workunit.client.0.smithi094.stdout:2/361: creat d18/d1e/d34/d4a/f6d x:0 0 0 2024-03-24T10:45:56.259 INFO:tasks.workunit.client.0.smithi094.stdout:4/476: dread d0/d22/de/d28/d2b/f69 [0,4194304] 0 2024-03-24T10:45:56.261 INFO:tasks.workunit.client.0.smithi094.stdout:3/318: dwrite d0/db/dc/f12 [0,4194304] 0 2024-03-24T10:45:56.261 INFO:tasks.workunit.client.0.smithi094.stdout:3/319: chown d0/db/d1c/d20/d23/d2a/d3a/c44 221747 1 2024-03-24T10:45:56.261 INFO:tasks.workunit.client.0.smithi094.stdout:5/480: getdents d7/d23/d5f/d14/d6f 0 2024-03-24T10:45:56.261 INFO:tasks.workunit.client.0.smithi094.stdout:5/481: creat d7/d23/d5f/d14/f9a x:0 0 0 2024-03-24T10:45:56.267 INFO:tasks.workunit.client.0.smithi094.stdout:8/458: creat df/d12/d5b/d63/f9c x:0 0 0 2024-03-24T10:45:56.270 INFO:tasks.workunit.client.0.smithi094.stdout:0/412: mknod d14/d22/d7d/c96 0 2024-03-24T10:45:56.272 INFO:tasks.workunit.client.0.smithi094.stdout:2/362: mknod d18/d1e/d34/d4a/c6e 0 2024-03-24T10:45:56.273 INFO:tasks.workunit.client.0.smithi094.stdout:2/363: truncate d18/d1e/d34/d4a/f6c 241090 0 2024-03-24T10:45:56.273 INFO:tasks.workunit.client.0.smithi094.stdout:2/364: chown d18/d1e/c3a 33565651 1 2024-03-24T10:45:56.273 INFO:tasks.workunit.client.0.smithi094.stdout:9/418: dwrite df/d2d/d34/d2c/f4c [0,4194304] 0 2024-03-24T10:45:56.274 INFO:tasks.workunit.client.0.smithi094.stdout:7/444: dwrite f0 [0,4194304] 0 2024-03-24T10:45:56.274 INFO:tasks.workunit.client.0.smithi094.stdout:7/445: chown d1/d9/d25/f50 3001406 1 2024-03-24T10:45:56.274 INFO:tasks.workunit.client.0.smithi094.stdout:4/477: mknod d0/d5/d7d/c8e 0 2024-03-24T10:45:56.276 INFO:tasks.workunit.client.0.smithi094.stdout:4/478: write d0/d22/de/d28/f3a [238445,127188] 0 2024-03-24T10:45:56.276 INFO:tasks.workunit.client.0.smithi094.stdout:4/479: truncate d0/d5/d7d/f88 827516 0 2024-03-24T10:45:56.276 INFO:tasks.workunit.client.0.smithi094.stdout:6/428: rmdir d0/d21 39 2024-03-24T10:45:56.276 INFO:tasks.workunit.client.0.smithi094.stdout:6/429: dread - d0/d78/f5a zero size 2024-03-24T10:45:56.281 INFO:tasks.workunit.client.0.smithi094.stdout:3/320: mkdir d0/db/d1c/d20/d23/d2a/d3a/d25/d34/d71 0 2024-03-24T10:45:56.283 INFO:tasks.workunit.client.0.smithi094.stdout:5/482: mknod d7/dc/d56/c9b 0 2024-03-24T10:45:56.283 INFO:tasks.workunit.client.0.smithi094.stdout:5/483: read - d7/d23/d5f/d9/d48/f74 zero size 2024-03-24T10:45:56.284 INFO:tasks.workunit.client.0.smithi094.stdout:5/484: chown d7/d23/d2b/d68/d81/l82 3985 1 2024-03-24T10:45:56.285 INFO:tasks.workunit.client.0.smithi094.stdout:8/459: mkdir df/d25/d9d 0 2024-03-24T10:45:56.285 INFO:tasks.workunit.client.0.smithi094.stdout:8/460: dread - df/d25/d36/d3f/f92 zero size 2024-03-24T10:45:56.285 INFO:tasks.workunit.client.0.smithi094.stdout:0/413: dwrite d14/d18/f25 [4194304,4194304] 0 2024-03-24T10:45:56.285 INFO:tasks.workunit.client.0.smithi094.stdout:8/461: readlink df/d12/l53 0 2024-03-24T10:45:56.285 INFO:tasks.workunit.client.0.smithi094.stdout:0/414: fsync d14/d22/d28/d2b/d68/f72 0 2024-03-24T10:45:56.285 INFO:tasks.workunit.client.0.smithi094.stdout:0/415: chown d14/d18/f69 533071254 1 2024-03-24T10:45:56.286 INFO:tasks.workunit.client.0.smithi094.stdout:2/365: creat d18/d1e/d34/f6f x:0 0 0 2024-03-24T10:45:56.286 INFO:tasks.workunit.client.0.smithi094.stdout:2/366: getdents d18/d1e/d34/d69 0 2024-03-24T10:45:56.287 INFO:tasks.workunit.client.0.smithi094.stdout:3/321: dread d0/f3 [0,4194304] 0 2024-03-24T10:45:56.287 INFO:tasks.workunit.client.0.smithi094.stdout:9/419: truncate df/d2d/d34/d2c/f40 435595 0 2024-03-24T10:45:56.287 INFO:tasks.workunit.client.0.smithi094.stdout:9/420: fdatasync df/f62 0 2024-03-24T10:45:56.287 INFO:tasks.workunit.client.0.smithi094.stdout:9/421: chown df/l20 170432 1 2024-03-24T10:45:56.287 INFO:tasks.workunit.client.0.smithi094.stdout:7/446: mknod d1/d9/d16/c93 0 2024-03-24T10:45:56.289 INFO:tasks.workunit.client.0.smithi094.stdout:6/430: dwrite d0/d1a/f39 [0,4194304] 0 2024-03-24T10:45:56.289 INFO:tasks.workunit.client.0.smithi094.stdout:4/480: getdents d0/d22/de/d28/d2b/d33 0 2024-03-24T10:45:56.290 INFO:tasks.workunit.client.0.smithi094.stdout:4/481: write d0/d22/ff [1964456,116901] 0 2024-03-24T10:45:56.290 INFO:tasks.workunit.client.0.smithi094.stdout:4/482: chown d0/d22/l53 60426 1 2024-03-24T10:45:56.291 INFO:tasks.workunit.client.0.smithi094.stdout:6/431: dread d0/f33 [0,4194304] 0 2024-03-24T10:45:56.297 INFO:tasks.workunit.client.0.smithi094.stdout:5/485: symlink d7/d23/d5f/d9/d48/d51/l9c 0 2024-03-24T10:45:56.298 INFO:tasks.workunit.client.0.smithi094.stdout:8/462: getdents df/d12/d5b 0 2024-03-24T10:45:56.300 INFO:tasks.workunit.client.0.smithi094.stdout:0/416: mkdir d14/d22/d5b/d86/d97 0 2024-03-24T10:45:56.302 INFO:tasks.workunit.client.0.smithi094.stdout:2/367: rename d18/d5f/f60 to d18/d54/f70 0 2024-03-24T10:45:56.302 INFO:tasks.workunit.client.0.smithi094.stdout:3/322: write d0/db/d1c/d20/d23/f29 [2973666,57658] 0 2024-03-24T10:45:56.303 INFO:tasks.workunit.client.0.smithi094.stdout:9/422: mkdir df/d2d/d1f/d58/d82 0 2024-03-24T10:45:56.305 INFO:tasks.workunit.client.0.smithi094.stdout:4/483: dwrite d0/d5/d7d/f8d [0,4194304] 0 2024-03-24T10:45:56.305 INFO:tasks.workunit.client.0.smithi094.stdout:7/447: creat d1/d9/d16/d21/d51/d83/d39/f94 x:0 0 0 2024-03-24T10:45:56.306 INFO:tasks.workunit.client.0.smithi094.stdout:8/463: write df/d25/d36/d3f/f8e [2394400,102965] 0 2024-03-24T10:45:56.308 INFO:tasks.workunit.client.0.smithi094.stdout:6/432: symlink d0/d30/l82 0 2024-03-24T10:45:56.310 INFO:tasks.workunit.client.0.smithi094.stdout:0/417: rmdir d14/d22/d5b/d86/d97 0 2024-03-24T10:45:56.313 INFO:tasks.workunit.client.0.smithi094.stdout:4/484: fdatasync d0/d22/ff 0 2024-03-24T10:45:56.314 INFO:tasks.workunit.client.0.smithi094.stdout:3/323: creat d0/db/dc/d28/d38/f72 x:0 0 0 2024-03-24T10:45:56.314 INFO:tasks.workunit.client.0.smithi094.stdout:3/324: fdatasync d0/f2 0 2024-03-24T10:45:56.316 INFO:tasks.workunit.client.0.smithi094.stdout:6/433: rmdir d0/d30/d50/d4d 39 2024-03-24T10:45:56.319 INFO:tasks.workunit.client.0.smithi094.stdout:2/368: mkdir d18/d1e/d34/d71 0 2024-03-24T10:45:56.319 INFO:tasks.workunit.client.0.smithi094.stdout:2/369: readlink l6 0 2024-03-24T10:45:56.322 INFO:tasks.workunit.client.0.smithi094.stdout:3/325: mkdir d0/db/d1c/d20/d23/d2a/d3a/d56/d61/d69/d73 0 2024-03-24T10:45:56.323 INFO:tasks.workunit.client.0.smithi094.stdout:0/418: getdents d14/d22/d28/d2b/d41/d53 0 2024-03-24T10:45:56.323 INFO:tasks.workunit.client.0.smithi094.stdout:5/486: dwrite d7/dc/d12/f3e [0,4194304] 0 2024-03-24T10:45:56.325 INFO:tasks.workunit.client.0.smithi094.stdout:2/370: unlink d18/d1e/l20 0 2024-03-24T10:45:56.326 INFO:tasks.workunit.client.0.smithi094.stdout:3/326: mkdir d0/db/d1c/d20/d23/d2a/d3a/d25/d4f/d74 0 2024-03-24T10:45:56.328 INFO:tasks.workunit.client.0.smithi094.stdout:0/419: mknod d14/d18/c98 0 2024-03-24T10:45:56.331 INFO:tasks.workunit.client.0.smithi094.stdout:5/487: rename d7/dc/d12/c66 to d7/d23/d2b/d68/d63/c9d 0 2024-03-24T10:45:56.331 INFO:tasks.workunit.client.0.smithi094.stdout:5/488: chown d7/d23/d5f/c7e 288 1 2024-03-24T10:45:56.332 INFO:tasks.workunit.client.0.smithi094.stdout:8/464: dwrite df/d25/d28/d31/d47/d22/d55/f7a [0,4194304] 0 2024-03-24T10:45:56.333 INFO:tasks.workunit.client.0.smithi094.stdout:3/327: creat d0/db/d1c/d20/d23/d2a/d3a/d56/d61/d69/d73/f75 x:0 0 0 2024-03-24T10:45:56.335 INFO:tasks.workunit.client.0.smithi094.stdout:0/420: symlink d14/d22/d5b/d86/l99 0 2024-03-24T10:45:56.335 INFO:tasks.workunit.client.0.smithi094.stdout:0/421: write d14/d22/d28/d48/f78 [1157725,118272] 0 2024-03-24T10:45:56.337 INFO:tasks.workunit.client.0.smithi094.stdout:9/423: dwrite df/d2d/d1f/d58/d5a/f63 [0,4194304] 0 2024-03-24T10:45:56.337 INFO:tasks.workunit.client.0.smithi094.stdout:9/424: creat df/d2d/f83 x:0 0 0 2024-03-24T10:45:56.338 INFO:tasks.workunit.client.0.smithi094.stdout:9/425: creat df/d2d/f84 x:0 0 0 2024-03-24T10:45:56.338 INFO:tasks.workunit.client.0.smithi094.stdout:9/426: stat df/d2d/d34/d2c/f5f 0 2024-03-24T10:45:56.340 INFO:tasks.workunit.client.0.smithi094.stdout:8/465: link df/d25/d28/d31/d47/f4a df/d8c/f9e 0 2024-03-24T10:45:56.340 INFO:tasks.workunit.client.0.smithi094.stdout:8/466: creat df/d25/d28/d31/f9f x:0 0 0 2024-03-24T10:45:56.346 INFO:tasks.workunit.client.0.smithi094.stdout:8/467: symlink df/d25/d28/d31/d47/d22/d55/la0 0 2024-03-24T10:45:56.346 INFO:tasks.workunit.client.0.smithi094.stdout:8/468: creat df/d25/d28/d31/d47/d22/fa1 x:0 0 0 2024-03-24T10:45:56.346 INFO:tasks.workunit.client.0.smithi094.stdout:8/469: write df/d25/d28/d31/d3c/f84 [869257,86153] 0 2024-03-24T10:45:56.350 INFO:tasks.workunit.client.0.smithi094.stdout:3/328: write d0/db/f48 [198727,63276] 0 2024-03-24T10:45:56.352 INFO:tasks.workunit.client.0.smithi094.stdout:3/329: symlink d0/db/d1c/d20/d23/d2a/d3a/d56/d61/d69/d73/l76 0 2024-03-24T10:45:56.353 INFO:tasks.workunit.client.0.smithi094.stdout:3/330: symlink d0/db/dc/d28/d50/l77 0 2024-03-24T10:45:56.353 INFO:tasks.workunit.client.0.smithi094.stdout:3/331: write d0/db/d1c/d20/d23/d2a/f4c [800600,126629] 0 2024-03-24T10:45:56.353 INFO:tasks.workunit.client.0.smithi094.stdout:3/332: chown d0/db/dc/d28/d38 99728141 1 2024-03-24T10:45:56.354 INFO:tasks.workunit.client.0.smithi094.stdout:3/333: creat d0/db/dc/d28/d38/d53/f78 x:0 0 0 2024-03-24T10:45:56.355 INFO:tasks.workunit.client.0.smithi094.stdout:3/334: dread d0/f1e [4194304,4194304] 0 2024-03-24T10:45:56.355 INFO:tasks.workunit.client.0.smithi094.stdout:3/335: chown d0/db/d1c/d20/d23/d2a/d3a/d25 1 1 2024-03-24T10:45:56.357 INFO:tasks.workunit.client.0.smithi094.stdout:3/336: creat d0/db/dc/d28/d38/d6a/f79 x:0 0 0 2024-03-24T10:45:56.357 INFO:tasks.workunit.client.0.smithi094.stdout:3/337: chown d0/db/dc/d28/d38/d6a/f79 24931 1 2024-03-24T10:45:56.358 INFO:tasks.workunit.client.0.smithi094.stdout:0/422: dread d14/d18/f21 [0,4194304] 0 2024-03-24T10:45:56.358 INFO:tasks.workunit.client.0.smithi094.stdout:0/423: dread - d14/d22/d28/f47 zero size 2024-03-24T10:45:56.358 INFO:tasks.workunit.client.0.smithi094.stdout:0/424: stat d14/d22/d28/d2b/c63 0 2024-03-24T10:45:56.359 INFO:tasks.workunit.client.0.smithi094.stdout:0/425: stat d14/d18/f54 0 2024-03-24T10:45:56.359 INFO:tasks.workunit.client.0.smithi094.stdout:7/448: dwrite d1/d9/d16/d21/d51/d83/d39/f94 [0,4194304] 0 2024-03-24T10:45:56.360 INFO:tasks.workunit.client.0.smithi094.stdout:0/426: creat d14/d22/d7d/f9a x:0 0 0 2024-03-24T10:45:56.360 INFO:tasks.workunit.client.0.smithi094.stdout:0/427: creat d14/d22/d32/f9b x:0 0 0 2024-03-24T10:45:56.364 INFO:tasks.workunit.client.0.smithi094.stdout:7/449: getdents d1/d9/d35/d65 0 2024-03-24T10:45:56.364 INFO:tasks.workunit.client.0.smithi094.stdout:7/450: write d1/d9/d16/d21/d51/d83/f72 [782172,130543] 0 2024-03-24T10:45:56.365 INFO:tasks.workunit.client.0.smithi094.stdout:6/434: dwrite d0/d21/f13 [0,4194304] 0 2024-03-24T10:45:56.365 INFO:tasks.workunit.client.0.smithi094.stdout:6/435: getdents d0/d78/da/d77 0 2024-03-24T10:45:56.366 INFO:tasks.workunit.client.0.smithi094.stdout:6/436: mkdir d0/d78/da/d23/d83 0 2024-03-24T10:45:56.368 INFO:tasks.workunit.client.0.smithi094.stdout:6/437: symlink d0/d21/l84 0 2024-03-24T10:45:56.370 INFO:tasks.workunit.client.0.smithi094.stdout:6/438: link d0/d78/f44 d0/d78/da/d23/d83/f85 0 2024-03-24T10:45:56.372 INFO:tasks.workunit.client.0.smithi094.stdout:5/489: dwrite d7/d23/f77 [4194304,4194304] 0 2024-03-24T10:45:56.372 INFO:tasks.workunit.client.0.smithi094.stdout:5/490: chown d7/d23/d5f/d14/d36 124879 1 2024-03-24T10:45:56.372 INFO:tasks.workunit.client.0.smithi094.stdout:5/491: write d7/d23/d5f/d14/d36/f47 [679359,109121] 0 2024-03-24T10:45:56.372 INFO:tasks.workunit.client.0.smithi094.stdout:5/492: dread - d7/d23/d5f/d9/d48/f74 zero size 2024-03-24T10:45:56.373 INFO:tasks.workunit.client.0.smithi094.stdout:5/493: fdatasync d7/d23/d5f/f99 0 2024-03-24T10:45:56.373 INFO:tasks.workunit.client.0.smithi094.stdout:5/494: creat d7/d23/d2b/d68/f9e x:0 0 0 2024-03-24T10:45:56.375 INFO:tasks.workunit.client.0.smithi094.stdout:0/428: write d14/f1b [987071,40644] 0 2024-03-24T10:45:56.375 INFO:tasks.workunit.client.0.smithi094.stdout:0/429: chown d14/d22/d28/l4c 16327115 1 2024-03-24T10:45:56.375 INFO:tasks.workunit.client.0.smithi094.stdout:4/485: dwrite d0/d5/d7d/f88 [0,4194304] 0 2024-03-24T10:45:56.375 INFO:tasks.workunit.client.0.smithi094.stdout:8/470: dwrite df/d25/d28/d31/f88 [0,4194304] 0 2024-03-24T10:45:56.375 INFO:tasks.workunit.client.0.smithi094.stdout:8/471: truncate df/d25/d36/f9b 787916 0 2024-03-24T10:45:56.375 INFO:tasks.workunit.client.0.smithi094.stdout:8/472: write df/d25/d28/d31/d47/f7c [1218032,78413] 0 2024-03-24T10:45:56.380 INFO:tasks.workunit.client.0.smithi094.stdout:9/427: dwrite df/d2d/d1f/d11/f13 [0,4194304] 0 2024-03-24T10:45:56.383 INFO:tasks.workunit.client.0.smithi094.stdout:3/338: dwrite d0/db/dc/f2d [0,4194304] 0 2024-03-24T10:45:56.383 INFO:tasks.workunit.client.0.smithi094.stdout:4/486: dread d0/d22/f62 [0,4194304] 0 2024-03-24T10:45:56.383 INFO:tasks.workunit.client.0.smithi094.stdout:7/451: dwrite d1/d9/d25/f50 [0,4194304] 0 2024-03-24T10:45:56.383 INFO:tasks.workunit.client.0.smithi094.stdout:6/439: write d0/d78/f44 [1096358,82120] 0 2024-03-24T10:45:56.390 INFO:tasks.workunit.client.0.smithi094.stdout:0/430: dwrite d14/d22/d28/d48/d7b/f94 [0,4194304] 0 2024-03-24T10:45:56.390 INFO:tasks.workunit.client.0.smithi094.stdout:0/431: creat d14/d22/d7d/f9c x:0 0 0 2024-03-24T10:45:56.390 INFO:tasks.workunit.client.0.smithi094.stdout:0/432: write d14/d22/d5b/d86/f8f [663040,17901] 0 2024-03-24T10:45:56.392 INFO:tasks.workunit.client.0.smithi094.stdout:8/473: stat df/d25/d36/d3f/l49 0 2024-03-24T10:45:56.392 INFO:tasks.workunit.client.0.smithi094.stdout:8/474: stat df/d25/d28/d31/f88 0 2024-03-24T10:45:56.392 INFO:tasks.workunit.client.0.smithi094.stdout:8/475: write df/f56 [949668,19471] 0 2024-03-24T10:45:56.392 INFO:tasks.workunit.client.0.smithi094.stdout:8/476: creat df/d25/d28/d31/d47/fa2 x:0 0 0 2024-03-24T10:45:56.393 INFO:tasks.workunit.client.0.smithi094.stdout:5/495: getdents d7/d23/d5f/d9 0 2024-03-24T10:45:56.393 INFO:tasks.workunit.client.0.smithi094.stdout:5/496: truncate d7/d23/d5f/d9/f78 1009880 0 2024-03-24T10:45:56.396 INFO:tasks.workunit.client.0.smithi094.stdout:9/428: symlink df/d2d/d1f/d11/d66/l85 0 2024-03-24T10:45:56.398 INFO:tasks.workunit.client.0.smithi094.stdout:0/433: dread d14/d22/d28/d2b/f3b [0,4194304] 0 2024-03-24T10:45:56.398 INFO:tasks.workunit.client.0.smithi094.stdout:0/434: creat d14/d18/f9d x:0 0 0 2024-03-24T10:45:56.399 INFO:tasks.workunit.client.0.smithi094.stdout:3/339: link d0/db/d1c/d20/d23/d2a/d3a/d25/c31 d0/db/d1c/d20/d23/d2a/d3a/d25/d4f/d74/c7a 0 2024-03-24T10:45:56.399 INFO:tasks.workunit.client.0.smithi094.stdout:3/340: dread - d0/db/dc/d28/d38/f72 zero size 2024-03-24T10:45:56.400 INFO:tasks.workunit.client.0.smithi094.stdout:5/497: dread d7/d23/d5f/d1b/f37 [0,4194304] 0 2024-03-24T10:45:56.400 INFO:tasks.workunit.client.0.smithi094.stdout:4/487: link d0/d22/de/d28/d2b/f83 d0/d22/de/d28/f8f 0 2024-03-24T10:45:56.400 INFO:tasks.workunit.client.0.smithi094.stdout:4/488: readlink d0/d22/de/l78 0 2024-03-24T10:45:56.402 INFO:tasks.workunit.client.0.smithi094.stdout:0/435: dread fd [0,4194304] 0 2024-03-24T10:45:56.402 INFO:tasks.workunit.client.0.smithi094.stdout:0/436: chown d14/d18/f1f 8 1 2024-03-24T10:45:56.408 INFO:tasks.workunit.client.0.smithi094.stdout:8/477: mkdir df/d25/d28/d31/d47/d66/da3 0 2024-03-24T10:45:56.408 INFO:tasks.workunit.client.0.smithi094.stdout:8/478: fdatasync df/d25/d36/d3f/f92 0 2024-03-24T10:45:56.409 INFO:tasks.workunit.client.0.smithi094.stdout:9/429: mknod df/d2d/d1f/d58/d5a/d50/c86 0 2024-03-24T10:45:56.411 INFO:tasks.workunit.client.0.smithi094.stdout:5/498: mkdir d7/d23/d5f/d14/d9f 0 2024-03-24T10:45:56.413 INFO:tasks.workunit.client.0.smithi094.stdout:4/489: unlink d0/f4b 0 2024-03-24T10:45:56.415 INFO:tasks.workunit.client.0.smithi094.stdout:7/452: dwrite d1/d9/d35/f6b [0,4194304] 0 2024-03-24T10:45:56.415 INFO:tasks.workunit.client.0.smithi094.stdout:7/453: chown d1/d9/d16/d21/d51/d83/f90 3196 1 2024-03-24T10:45:56.415 INFO:tasks.workunit.client.0.smithi094.stdout:0/437: rename d14/d22/d5b/d86/l99 to d14/d18/d85/l9e 0 2024-03-24T10:45:56.416 INFO:tasks.workunit.client.0.smithi094.stdout:6/440: dwrite d0/d21/f2b [4194304,4194304] 0 2024-03-24T10:45:56.416 INFO:tasks.workunit.client.0.smithi094.stdout:9/430: mkdir df/d2d/d1f/d58/d5a/d61/d87 0 2024-03-24T10:45:56.418 INFO:tasks.workunit.client.0.smithi094.stdout:6/441: dread d0/d21/f5c [0,4194304] 0 2024-03-24T10:45:56.420 INFO:tasks.workunit.client.0.smithi094.stdout:9/431: dread df/f2a [0,4194304] 0 2024-03-24T10:45:56.420 INFO:tasks.workunit.client.0.smithi094.stdout:9/432: truncate df/d2d/f1e 9031216 0 2024-03-24T10:45:56.422 INFO:tasks.workunit.client.0.smithi094.stdout:5/499: rename d7/d23/d5f/d9/d48/d73/c87 to d7/d23/d5f/ca0 0 2024-03-24T10:45:56.424 INFO:tasks.workunit.client.0.smithi094.stdout:0/438: symlink d14/d22/d28/d2b/l9f 0 2024-03-24T10:45:56.430 INFO:tasks.workunit.client.0.smithi094.stdout:6/442: mkdir d0/d30/d50/d4d/d6b/d86 0 2024-03-24T10:45:56.431 INFO:tasks.workunit.client.0.smithi094.stdout:6/443: dread d0/d78/da/d23/d34/d4e/f80 [0,4194304] 0 2024-03-24T10:45:56.431 INFO:tasks.workunit.client.0.smithi094.stdout:9/433: mkdir df/d2d/d1f/d11/d53/d88 0 2024-03-24T10:45:56.433 INFO:tasks.workunit.client.0.smithi094.stdout:5/500: getdents d7/dc 0 2024-03-24T10:45:56.434 INFO:tasks.workunit.client.0.smithi094.stdout:6/444: getdents d0/d21/d2a/d2f 0 2024-03-24T10:45:56.434 INFO:tasks.workunit.client.0.smithi094.stdout:8/479: dwrite df/d25/d28/d31/d47/f69 [0,4194304] 0 2024-03-24T10:45:56.436 INFO:tasks.workunit.client.0.smithi094.stdout:4/490: dwrite d0/d22/de/d28/d2b/f69 [8388608,4194304] 0 2024-03-24T10:45:56.437 INFO:tasks.workunit.client.0.smithi094.stdout:9/434: rename df/d2d/d34/f47 to df/d2d/d1f/d11/d53/d88/f89 0 2024-03-24T10:45:56.437 INFO:tasks.workunit.client.0.smithi094.stdout:9/435: dread - df/d2d/f84 zero size 2024-03-24T10:45:56.440 INFO:tasks.workunit.client.0.smithi094.stdout:5/501: link d7/d26/l22 d7/d23/d5f/d14/d9f/la1 0 2024-03-24T10:45:56.440 INFO:tasks.workunit.client.0.smithi094.stdout:7/454: dwrite d1/d9/f12 [0,4194304] 0 2024-03-24T10:45:56.441 INFO:tasks.workunit.client.0.smithi094.stdout:8/480: rename df/d25/d28/d31/d47/d22/fa1 to df/d25/d28/d31/d47/d66/da3/fa4 0 2024-03-24T10:45:56.441 INFO:tasks.workunit.client.0.smithi094.stdout:8/481: creat df/d25/d28/d31/d47/d22/d55/fa5 x:0 0 0 2024-03-24T10:45:56.441 INFO:tasks.workunit.client.0.smithi094.stdout:8/482: write df/d25/d28/d31/f72 [279567,127335] 0 2024-03-24T10:45:56.441 INFO:tasks.workunit.client.0.smithi094.stdout:8/483: readlink df/d12/d5b/l5e 0 2024-03-24T10:45:56.444 INFO:tasks.workunit.client.0.smithi094.stdout:5/502: truncate d7/d23/d5f/d1b/f42 649728 0 2024-03-24T10:45:56.448 INFO:tasks.workunit.client.0.smithi094.stdout:6/445: dread d0/d21/f4f [0,4194304] 0 2024-03-24T10:45:56.450 INFO:tasks.workunit.client.0.smithi094.stdout:5/503: creat d7/dc/fa2 x:0 0 0 2024-03-24T10:45:56.451 INFO:tasks.workunit.client.0.smithi094.stdout:6/446: dread d0/d1a/d35/f72 [0,4194304] 0 2024-03-24T10:45:56.453 INFO:tasks.workunit.client.0.smithi094.stdout:0/439: getdents d14/d22/d28/d2b 0 2024-03-24T10:45:56.460 INFO:tasks.workunit.client.0.smithi094.stdout:1/358: sync 2024-03-24T10:45:56.475 INFO:tasks.workunit.client.0.smithi094.stdout:4/491: dwrite d0/d22/de/d28/f8f [0,4194304] 0 2024-03-24T10:45:56.493 INFO:tasks.workunit.client.0.smithi094.stdout:4/492: dread d0/d5/d7d/f8a [0,4194304] 0 2024-03-24T10:45:56.495 INFO:tasks.workunit.client.0.smithi094.stdout:4/493: unlink d0/d22/l27 0 2024-03-24T10:45:56.499 INFO:tasks.workunit.client.0.smithi094.stdout:7/455: dwrite f0 [0,4194304] 0 2024-03-24T10:45:56.499 INFO:tasks.workunit.client.0.smithi094.stdout:8/484: dwrite df/d25/d28/d31/d47/d22/d55/f7a [0,4194304] 0 2024-03-24T10:45:56.499 INFO:tasks.workunit.client.0.smithi094.stdout:7/456: write d1/d9/f23 [280031,38517] 0 2024-03-24T10:45:56.499 INFO:tasks.workunit.client.0.smithi094.stdout:8/485: creat df/d12/d5b/d63/d75/fa6 x:0 0 0 2024-03-24T10:45:56.499 INFO:tasks.workunit.client.0.smithi094.stdout:4/494: rmdir d0/d22/de/d28/d2b 39 2024-03-24T10:45:56.499 INFO:tasks.workunit.client.0.smithi094.stdout:4/495: creat d0/d5/d1b/f90 x:0 0 0 2024-03-24T10:45:56.499 INFO:tasks.workunit.client.0.smithi094.stdout:6/447: dwrite d0/f61 [0,4194304] 0 2024-03-24T10:45:56.499 INFO:tasks.workunit.client.0.smithi094.stdout:9/436: dwrite df/d2d/d34/d2c/f40 [0,4194304] 0 2024-03-24T10:45:56.499 INFO:tasks.workunit.client.0.smithi094.stdout:9/437: chown df 44036 1 2024-03-24T10:45:56.500 INFO:tasks.workunit.client.0.smithi094.stdout:8/486: read df/f89 [306926,67254] 0 2024-03-24T10:45:56.500 INFO:tasks.workunit.client.0.smithi094.stdout:0/440: dwrite d14/d22/d28/f31 [0,4194304] 0 2024-03-24T10:45:56.500 INFO:tasks.workunit.client.0.smithi094.stdout:0/441: fsync d14/d18/f40 0 2024-03-24T10:45:56.500 INFO:tasks.workunit.client.0.smithi094.stdout:0/442: fsync d14/d22/d28/f46 0 2024-03-24T10:45:56.500 INFO:tasks.workunit.client.0.smithi094.stdout:1/359: dwrite f1 [0,4194304] 0 2024-03-24T10:45:56.501 INFO:tasks.workunit.client.0.smithi094.stdout:7/457: mknod d1/d9/d16/c95 0 2024-03-24T10:45:56.502 INFO:tasks.workunit.client.0.smithi094.stdout:4/496: mkdir d0/d22/de/d28/d3d/d91 0 2024-03-24T10:45:56.505 INFO:tasks.workunit.client.0.smithi094.stdout:9/438: write df/f36 [27052,102322] 0 2024-03-24T10:45:56.505 INFO:tasks.workunit.client.0.smithi094.stdout:9/439: chown df/d2d/d1f/d11/c18 70607 1 2024-03-24T10:45:56.505 INFO:tasks.workunit.client.0.smithi094.stdout:9/440: getdents df 0 2024-03-24T10:45:56.509 INFO:tasks.workunit.client.0.smithi094.stdout:6/448: symlink d0/d78/da/d77/l87 0 2024-03-24T10:45:56.509 INFO:tasks.workunit.client.0.smithi094.stdout:6/449: write d0/d78/f5a [379784,16023] 0 2024-03-24T10:45:56.509 INFO:tasks.workunit.client.0.smithi094.stdout:6/450: fsync d0/d21/f3a 0 2024-03-24T10:45:56.509 INFO:tasks.workunit.client.0.smithi094.stdout:6/451: truncate d0/f33 1669712 0 2024-03-24T10:45:56.512 INFO:tasks.workunit.client.0.smithi094.stdout:8/487: write df/d25/f29 [2389135,112428] 0 2024-03-24T10:45:56.512 INFO:tasks.workunit.client.0.smithi094.stdout:0/443: truncate f0 2062242 0 2024-03-24T10:45:56.516 INFO:tasks.workunit.client.0.smithi094.stdout:7/458: dread d1/d9/d14/f36 [0,4194304] 0 2024-03-24T10:45:56.517 INFO:tasks.workunit.client.0.smithi094.stdout:1/360: symlink d4/d9/db/d39/d43/l79 0 2024-03-24T10:45:56.517 INFO:tasks.workunit.client.0.smithi094.stdout:1/361: readlink d4/d9/d47/l59 0 2024-03-24T10:45:56.518 INFO:tasks.workunit.client.0.smithi094.stdout:4/497: symlink d0/d22/de/d28/d3d/d91/l92 0 2024-03-24T10:45:56.521 INFO:tasks.workunit.client.0.smithi094.stdout:9/441: rename df/d2d/d1f/f6e to df/d2d/d1f/d58/d5a/d61/f8a 0 2024-03-24T10:45:56.522 INFO:tasks.workunit.client.0.smithi094.stdout:6/452: mknod d0/d21/d2a/c88 0 2024-03-24T10:45:56.522 INFO:tasks.workunit.client.0.smithi094.stdout:6/453: chown d0/d1a/l43 478 1 2024-03-24T10:45:56.523 INFO:tasks.workunit.client.0.smithi094.stdout:9/442: dread df/d2d/d1f/d58/d5a/f77 [0,4194304] 0 2024-03-24T10:45:56.523 INFO:tasks.workunit.client.0.smithi094.stdout:9/443: chown df/d2d/d34/f39 14391 1 2024-03-24T10:45:56.523 INFO:tasks.workunit.client.0.smithi094.stdout:9/444: fdatasync df/d2d/d1f/d58/d5a/d69/d7a/f7d 0 2024-03-24T10:45:56.525 INFO:tasks.workunit.client.0.smithi094.stdout:8/488: link df/d25/d28/d31/d47/d22/d55/f8a df/d25/d28/fa7 0 2024-03-24T10:45:56.525 INFO:tasks.workunit.client.0.smithi094.stdout:8/489: chown df/d12/l1a 2619749 1 2024-03-24T10:45:56.526 INFO:tasks.workunit.client.0.smithi094.stdout:5/504: truncate d7/d23/d5f/d1b/f42 196993 0 2024-03-24T10:45:56.530 INFO:tasks.workunit.client.0.smithi094.stdout:5/505: dread d7/d23/d5f/d9/d48/d51/f6a [0,4194304] 0 2024-03-24T10:45:56.530 INFO:tasks.workunit.client.0.smithi094.stdout:7/459: mkdir d1/d9/d96 0 2024-03-24T10:45:56.530 INFO:tasks.workunit.client.0.smithi094.stdout:5/506: write d7/dc/d56/f6e [76495,128800] 0 2024-03-24T10:45:56.533 INFO:tasks.workunit.client.0.smithi094.stdout:0/444: dwrite d14/f1b [4194304,4194304] 0 2024-03-24T10:45:56.533 INFO:tasks.workunit.client.0.smithi094.stdout:0/445: fdatasync d14/d18/f25 0 2024-03-24T10:45:56.533 INFO:tasks.workunit.client.0.smithi094.stdout:0/446: dread - d14/d22/d7d/f9a zero size 2024-03-24T10:45:56.533 INFO:tasks.workunit.client.0.smithi094.stdout:0/447: creat d14/d18/fa0 x:0 0 0 2024-03-24T10:45:56.533 INFO:tasks.workunit.client.0.smithi094.stdout:0/448: stat d14/d22/d28/d48/l71 0 2024-03-24T10:45:56.533 INFO:tasks.workunit.client.0.smithi094.stdout:0/449: fdatasync d14/d22/d28/f31 0 2024-03-24T10:45:56.533 INFO:tasks.workunit.client.0.smithi094.stdout:0/450: dread - d14/d22/d7d/f9a zero size 2024-03-24T10:45:56.533 INFO:tasks.workunit.client.0.smithi094.stdout:0/451: fsync d14/d22/d28/d48/f4a 0 2024-03-24T10:45:56.533 INFO:tasks.workunit.client.0.smithi094.stdout:0/452: stat d14/d22/d28/d2b/d41/d53/l64 0 2024-03-24T10:45:56.533 INFO:tasks.workunit.client.0.smithi094.stdout:0/453: fdatasync d14/d22/d7d/f9c 0 2024-03-24T10:45:56.533 INFO:tasks.workunit.client.0.smithi094.stdout:1/362: mknod d4/d9/c7a 0 2024-03-24T10:45:56.534 INFO:tasks.workunit.client.0.smithi094.stdout:5/507: dread d7/d23/d5f/f28 [0,4194304] 0 2024-03-24T10:45:56.536 INFO:tasks.workunit.client.0.smithi094.stdout:9/445: rename df/d2d/d1f/d11/l19 to df/l8b 0 2024-03-24T10:45:56.536 INFO:tasks.workunit.client.0.smithi094.stdout:9/446: fsync df/d2d/f83 0 2024-03-24T10:45:56.538 INFO:tasks.workunit.client.0.smithi094.stdout:8/490: mkdir df/d25/d28/d31/d47/d66/da3/da8 0 2024-03-24T10:45:56.541 INFO:tasks.workunit.client.0.smithi094.stdout:0/454: rmdir d14/d22/d28 39 2024-03-24T10:45:56.541 INFO:tasks.workunit.client.0.smithi094.stdout:0/455: chown d14/d22/d28/f31 31 1 2024-03-24T10:45:56.543 INFO:tasks.workunit.client.0.smithi094.stdout:1/363: unlink d4/d9/db/d39/c54 0 2024-03-24T10:45:56.544 INFO:tasks.workunit.client.0.smithi094.stdout:6/454: dwrite d0/d78/da/d23/f26 [0,4194304] 0 2024-03-24T10:45:56.545 INFO:tasks.workunit.client.0.smithi094.stdout:4/498: rmdir d0/d5/d1b/d4f 39 2024-03-24T10:45:56.547 INFO:tasks.workunit.client.0.smithi094.stdout:9/447: creat df/d2d/d34/d2c/f8c x:0 0 0 2024-03-24T10:45:56.547 INFO:tasks.workunit.client.0.smithi094.stdout:9/448: read df/f22 [851051,84314] 0 2024-03-24T10:45:56.547 INFO:tasks.workunit.client.0.smithi094.stdout:9/449: readlink df/l21 0 2024-03-24T10:45:56.553 INFO:tasks.workunit.client.0.smithi094.stdout:5/508: dwrite d7/dc/d12/f61 [0,4194304] 0 2024-03-24T10:45:56.553 INFO:tasks.workunit.client.0.smithi094.stdout:5/509: write d7/d23/d5f/f8b [101760,85749] 0 2024-03-24T10:45:56.554 INFO:tasks.workunit.client.0.smithi094.stdout:5/510: readlink d7/d23/d5f/d9/l40 0 2024-03-24T10:45:56.554 INFO:tasks.workunit.client.0.smithi094.stdout:8/491: rename df/d25/d28/d31/f9f to df/d12/d5b/d63/fa9 0 2024-03-24T10:45:56.554 INFO:tasks.workunit.client.0.smithi094.stdout:8/492: chown df/d25/d36/d3f/f4f 15589 1 2024-03-24T10:45:56.554 INFO:tasks.workunit.client.0.smithi094.stdout:8/493: truncate df/d25/d28/d31/d47/f77 864184 0 2024-03-24T10:45:56.554 INFO:tasks.workunit.client.0.smithi094.stdout:1/364: symlink d4/d9/db/d5d/l7b 0 2024-03-24T10:45:56.555 INFO:tasks.workunit.client.0.smithi094.stdout:0/456: write d14/d18/f69 [1072177,9782] 0 2024-03-24T10:45:56.555 INFO:tasks.workunit.client.0.smithi094.stdout:0/457: fdatasync d14/d18/f2e 0 2024-03-24T10:45:56.556 INFO:tasks.workunit.client.0.smithi094.stdout:4/499: mkdir d0/d34/d73/d93 0 2024-03-24T10:45:56.560 INFO:tasks.workunit.client.0.smithi094.stdout:8/494: dread f6 [0,4194304] 0 2024-03-24T10:45:56.561 INFO:tasks.workunit.client.0.smithi094.stdout:6/455: rename d0/d30/d50/c48 to d0/d78/d66/d7a/c89 0 2024-03-24T10:45:56.564 INFO:tasks.workunit.client.0.smithi094.stdout:1/365: creat d4/d9/db/d31/d3c/f7c x:0 0 0 2024-03-24T10:45:56.564 INFO:tasks.workunit.client.0.smithi094.stdout:1/366: dread - d4/d9/de/d6f/f4e zero size 2024-03-24T10:45:56.566 INFO:tasks.workunit.client.0.smithi094.stdout:8/495: creat df/d25/d36/d3f/faa x:0 0 0 2024-03-24T10:45:56.567 INFO:tasks.workunit.client.0.smithi094.stdout:3/341: sync 2024-03-24T10:45:56.567 INFO:tasks.workunit.client.0.smithi094.stdout:2/371: sync 2024-03-24T10:45:56.567 INFO:tasks.workunit.client.0.smithi094.stdout:2/372: dread - d18/d48/d56/f58 zero size 2024-03-24T10:45:56.567 INFO:tasks.workunit.client.0.smithi094.stdout:3/342: write d0/db/dc/d28/d38/f72 [165360,89815] 0 2024-03-24T10:45:56.568 INFO:tasks.workunit.client.0.smithi094.stdout:9/450: rename df/d2d/d1f/d11/f13 to df/d2d/f8d 0 2024-03-24T10:45:56.569 INFO:tasks.workunit.client.0.smithi094.stdout:6/456: getdents d0/d78/d66 0 2024-03-24T10:45:56.570 INFO:tasks.workunit.client.0.smithi094.stdout:3/343: stat d0/db/d1c/d20/d23/d2a/d3a/d25/d4f/d74/c7a 0 2024-03-24T10:45:56.570 INFO:tasks.workunit.client.0.smithi094.stdout:3/344: fsync d0/f14 0 2024-03-24T10:45:56.570 INFO:tasks.workunit.client.0.smithi094.stdout:3/345: fdatasync d0/db/d1c/d20/f24 0 2024-03-24T10:45:56.573 INFO:tasks.workunit.client.0.smithi094.stdout:5/511: rename d7/d23/d5f/d14/d6f/c95 to d7/d23/d5f/d14/d9f/ca3 0 2024-03-24T10:45:56.574 INFO:tasks.workunit.client.0.smithi094.stdout:6/457: symlink d0/d78/da/d77/l8a 0 2024-03-24T10:45:56.574 INFO:tasks.workunit.client.0.smithi094.stdout:6/458: fdatasync d0/d78/da/d23/f6c 0 2024-03-24T10:45:56.574 INFO:tasks.workunit.client.0.smithi094.stdout:4/500: dwrite d0/d22/ff [0,4194304] 0 2024-03-24T10:45:56.577 INFO:tasks.workunit.client.0.smithi094.stdout:3/346: creat d0/db/d1c/d20/d23/d2a/d3a/d56/f7b x:0 0 0 2024-03-24T10:45:56.580 INFO:tasks.workunit.client.0.smithi094.stdout:0/458: rename le to d14/d22/d5b/la1 0 2024-03-24T10:45:56.581 INFO:tasks.workunit.client.0.smithi094.stdout:5/512: stat d7/d26/l22 0 2024-03-24T10:45:56.583 INFO:tasks.workunit.client.0.smithi094.stdout:3/347: mkdir d0/db/dc/d28/d3c/d7c 0 2024-03-24T10:45:56.586 INFO:tasks.workunit.client.0.smithi094.stdout:6/459: rename d0/d3b/f5f to d0/d78/d66/d7a/f8b 0 2024-03-24T10:45:56.586 INFO:tasks.workunit.client.0.smithi094.stdout:5/513: link d7/d23/d5f/d1b/f42 d7/d23/d5f/d1b/fa4 0 2024-03-24T10:45:56.587 INFO:tasks.workunit.client.0.smithi094.stdout:3/348: rmdir d0/db/d1c/d20/d23/d2a/d3a 39 2024-03-24T10:45:56.588 INFO:tasks.workunit.client.0.smithi094.stdout:6/460: rename d0/d78/da/d23/d34/l4c to d0/d78/d66/d7a/l8c 0 2024-03-24T10:45:56.589 INFO:tasks.workunit.client.0.smithi094.stdout:5/514: mknod d7/d23/d5f/d9/d48/d73/ca5 0 2024-03-24T10:45:56.589 INFO:tasks.workunit.client.0.smithi094.stdout:5/515: dread - d7/f4c zero size 2024-03-24T10:45:56.590 INFO:tasks.workunit.client.0.smithi094.stdout:5/516: chown d7/d23/d5f/d14/d9f/la1 96492 1 2024-03-24T10:45:56.590 INFO:tasks.workunit.client.0.smithi094.stdout:8/496: write df/d25/d28/d31/d47/f85 [1204786,1846] 0 2024-03-24T10:45:56.590 INFO:tasks.workunit.client.0.smithi094.stdout:2/373: dwrite d18/d1e/d34/d4a/f6d [0,4194304] 0 2024-03-24T10:45:56.591 INFO:tasks.workunit.client.0.smithi094.stdout:6/461: dread d0/d78/f29 [0,4194304] 0 2024-03-24T10:45:56.591 INFO:tasks.workunit.client.0.smithi094.stdout:3/349: mknod d0/db/dc/d28/c7d 0 2024-03-24T10:45:56.591 INFO:tasks.workunit.client.0.smithi094.stdout:8/497: link df/d12/d5b/d63/d75/c81 df/d25/d28/d31/d47/cab 0 2024-03-24T10:45:56.591 INFO:tasks.workunit.client.0.smithi094.stdout:8/498: chown df/d12/c51 8 1 2024-03-24T10:45:56.594 INFO:tasks.workunit.client.0.smithi094.stdout:0/459: dread d14/d18/d85/f87 [0,4194304] 0 2024-03-24T10:45:56.594 INFO:tasks.workunit.client.0.smithi094.stdout:0/460: truncate d14/d22/d28/d48/f81 354593 0 2024-03-24T10:45:56.594 INFO:tasks.workunit.client.0.smithi094.stdout:0/461: dread - d14/d22/d28/f47 zero size 2024-03-24T10:45:56.595 INFO:tasks.workunit.client.0.smithi094.stdout:6/462: symlink d0/d30/d50/d4d/d6b/l8d 0 2024-03-24T10:45:56.596 INFO:tasks.workunit.client.0.smithi094.stdout:5/517: dread d7/d23/d5f/d14/d36/f47 [0,4194304] 0 2024-03-24T10:45:56.598 INFO:tasks.workunit.client.0.smithi094.stdout:8/499: rename df/d25/d28/d31/d3c/l93 to df/d12/d5b/d63/d75/d96/lac 0 2024-03-24T10:45:56.598 INFO:tasks.workunit.client.0.smithi094.stdout:8/500: write df/d25/d28/f6b [755934,90259] 0 2024-03-24T10:45:56.598 INFO:tasks.workunit.client.0.smithi094.stdout:8/501: readlink df/d25/d36/d3f/l49 0 2024-03-24T10:45:56.601 INFO:tasks.workunit.client.0.smithi094.stdout:0/462: truncate d14/d18/f4f 63455 0 2024-03-24T10:45:56.601 INFO:tasks.workunit.client.0.smithi094.stdout:6/463: symlink d0/d30/d50/d4d/l8e 0 2024-03-24T10:45:56.602 INFO:tasks.workunit.client.0.smithi094.stdout:9/451: dwrite df/d2d/d1f/d11/f4e [0,4194304] 0 2024-03-24T10:45:56.604 INFO:tasks.workunit.client.0.smithi094.stdout:6/464: symlink d0/d78/l8f 0 2024-03-24T10:45:56.607 INFO:tasks.workunit.client.0.smithi094.stdout:0/463: mkdir d14/d22/d28/da2 0 2024-03-24T10:45:56.607 INFO:tasks.workunit.client.0.smithi094.stdout:0/464: creat d14/d18/d85/fa3 x:0 0 0 2024-03-24T10:45:56.608 INFO:tasks.workunit.client.0.smithi094.stdout:0/465: dread d14/f49 [0,4194304] 0 2024-03-24T10:45:56.608 INFO:tasks.workunit.client.0.smithi094.stdout:6/465: dread d0/d78/da/d23/d83/f85 [0,4194304] 0 2024-03-24T10:45:56.608 INFO:tasks.workunit.client.0.smithi094.stdout:6/466: write d0/d78/da/d23/f68 [16057,5621] 0 2024-03-24T10:45:56.609 INFO:tasks.workunit.client.0.smithi094.stdout:2/374: dwrite d18/d48/f5d [0,4194304] 0 2024-03-24T10:45:56.609 INFO:tasks.workunit.client.0.smithi094.stdout:2/375: creat d18/d1e/d65/f72 x:0 0 0 2024-03-24T10:45:56.610 INFO:tasks.workunit.client.0.smithi094.stdout:4/501: dwrite d0/d22/de/d28/f7f [0,4194304] 0 2024-03-24T10:45:56.610 INFO:tasks.workunit.client.0.smithi094.stdout:9/452: rename df/d2d/l55 to df/d2d/d1f/d11/d66/d80/l8e 0 2024-03-24T10:45:56.610 INFO:tasks.workunit.client.0.smithi094.stdout:9/453: fsync df/d2d/d1f/f3d 0 2024-03-24T10:45:56.610 INFO:tasks.workunit.client.0.smithi094.stdout:9/454: chown df/d2d/d1f/d58/d5a/d69/d7a 0 1 2024-03-24T10:45:56.614 INFO:tasks.workunit.client.0.smithi094.stdout:0/466: truncate d14/d22/d28/d2b/d68/f72 758156 0 2024-03-24T10:45:56.641 INFO:tasks.workunit.client.0.smithi094.stdout:3/350: dwrite d0/db/dc/d28/d38/d53/f78 [0,4194304] 0 2024-03-24T10:45:56.644 INFO:tasks.workunit.client.0.smithi094.stdout:4/502: dwrite d0/d22/de/d28/f3a [0,4194304] 0 2024-03-24T10:45:56.646 INFO:tasks.workunit.client.0.smithi094.stdout:3/351: write d0/db/d1c/d20/f24 [3971120,88845] 0 2024-03-24T10:45:56.646 INFO:tasks.workunit.client.0.smithi094.stdout:6/467: dwrite d0/f61 [0,4194304] 0 2024-03-24T10:45:56.647 INFO:tasks.workunit.client.0.smithi094.stdout:5/518: dwrite d7/d23/d5f/d9/d48/f6b [0,4194304] 0 2024-03-24T10:45:56.651 INFO:tasks.workunit.client.0.smithi094.stdout:4/503: dread d0/d5/d1b/f2a [0,4194304] 0 2024-03-24T10:45:56.655 INFO:tasks.workunit.client.0.smithi094.stdout:3/352: dread d0/f14 [0,4194304] 0 2024-03-24T10:45:56.655 INFO:tasks.workunit.client.0.smithi094.stdout:5/519: write d7/d23/d5f/f83 [3083906,59862] 0 2024-03-24T10:45:56.681 INFO:tasks.workunit.client.0.smithi094.stdout:9/455: truncate df/f22 278518 0 2024-03-24T10:45:56.682 INFO:tasks.workunit.client.0.smithi094.stdout:0/467: stat d14/c19 0 2024-03-24T10:45:56.685 INFO:tasks.workunit.client.0.smithi094.stdout:9/456: dread df/f36 [0,4194304] 0 2024-03-24T10:45:56.685 INFO:tasks.workunit.client.0.smithi094.stdout:9/457: creat df/d2d/d1f/d58/d5a/d69/d7a/f8f x:0 0 0 2024-03-24T10:45:56.698 INFO:tasks.workunit.client.0.smithi094.stdout:6/468: creat d0/d30/d50/d4d/d6b/f90 x:0 0 0 2024-03-24T10:45:56.702 INFO:tasks.workunit.client.0.smithi094.stdout:3/353: creat d0/db/dc/d28/d50/d5e/f7e x:0 0 0 2024-03-24T10:45:56.702 INFO:tasks.workunit.client.0.smithi094.stdout:3/354: chown d0/db/dc/d28/d50/l77 60 1 2024-03-24T10:45:56.704 INFO:tasks.workunit.client.0.smithi094.stdout:5/520: rename d7/d23/d5f/d14/l8e to d7/d23/d5f/d9/d48/la6 0 2024-03-24T10:45:56.704 INFO:tasks.workunit.client.0.smithi094.stdout:5/521: creat d7/d23/d5f/d9/d48/fa7 x:0 0 0 2024-03-24T10:45:56.711 INFO:tasks.workunit.client.0.smithi094.stdout:6/469: creat d0/d78/da/d23/d34/f91 x:0 0 0 2024-03-24T10:45:56.712 INFO:tasks.workunit.client.0.smithi094.stdout:4/504: dread d0/d1f/f55 [0,4194304] 0 2024-03-24T10:45:56.715 INFO:tasks.workunit.client.0.smithi094.stdout:5/522: dread d7/d23/d5f/d9/d48/d51/f52 [0,4194304] 0 2024-03-24T10:45:56.715 INFO:tasks.workunit.client.0.smithi094.stdout:5/523: write d7/d23/d5f/d14/f85 [1424438,111562] 0 2024-03-24T10:45:56.715 INFO:tasks.workunit.client.0.smithi094.stdout:5/524: write d7/d23/d5f/d14/d36/d3a/f7f [346088,4165] 0 2024-03-24T10:45:56.720 INFO:tasks.workunit.client.0.smithi094.stdout:2/376: rmdir d18/d1e/d34/d69 0 2024-03-24T10:45:56.720 INFO:tasks.workunit.client.0.smithi094.stdout:2/377: write d18/d1e/d65/f6a [229206,56385] 0 2024-03-24T10:45:56.722 INFO:tasks.workunit.client.0.smithi094.stdout:9/458: getdents df/d2d/d1f/d11/d53 0 2024-03-24T10:45:56.722 INFO:tasks.workunit.client.0.smithi094.stdout:9/459: truncate df/d2d/f83 1045012 0 2024-03-24T10:45:56.722 INFO:tasks.workunit.client.0.smithi094.stdout:9/460: creat df/d2d/d1f/d11/f90 x:0 0 0 2024-03-24T10:45:56.722 INFO:tasks.workunit.client.0.smithi094.stdout:9/461: chown df/d7b 4630 1 2024-03-24T10:45:56.722 INFO:tasks.workunit.client.0.smithi094.stdout:9/462: dread - df/f62 zero size 2024-03-24T10:45:56.722 INFO:tasks.workunit.client.0.smithi094.stdout:9/463: truncate df/d2d/d1f/d58/d5a/f6a 5166482 0 2024-03-24T10:45:56.724 INFO:tasks.workunit.client.0.smithi094.stdout:6/470: dwrite d0/d21/f3a [0,4194304] 0 2024-03-24T10:45:56.726 INFO:tasks.workunit.client.0.smithi094.stdout:6/471: read d0/d30/f28 [956036,22075] 0 2024-03-24T10:45:56.727 INFO:tasks.workunit.client.0.smithi094.stdout:6/472: chown d0/d78/da/d23 111670 1 2024-03-24T10:45:56.727 INFO:tasks.workunit.client.0.smithi094.stdout:3/355: rename d0/db/d1c to d0/db/dc/d28/d3c/d55/d7f 0 2024-03-24T10:45:56.728 INFO:tasks.workunit.client.0.smithi094.stdout:4/505: mkdir d0/d34/d73/d94 0 2024-03-24T10:45:56.728 INFO:tasks.workunit.client.0.smithi094.stdout:4/506: write d0/d22/ff [4897215,27360] 0 2024-03-24T10:45:56.732 INFO:tasks.workunit.client.0.smithi094.stdout:9/464: creat df/d2d/d1f/d7e/f91 x:0 0 0 2024-03-24T10:45:56.737 INFO:tasks.workunit.client.0.smithi094.stdout:6/473: symlink d0/d78/da/d23/d83/l92 0 2024-03-24T10:45:56.737 INFO:tasks.workunit.client.0.smithi094.stdout:2/378: dread d18/d1e/d34/f35 [0,4194304] 0 2024-03-24T10:45:56.738 INFO:tasks.workunit.client.0.smithi094.stdout:4/507: stat d0/d22/de/d28/d3d/c7e 0 2024-03-24T10:45:56.740 INFO:tasks.workunit.client.0.smithi094.stdout:9/465: truncate df/d2d/f15 1448879 0 2024-03-24T10:45:56.741 INFO:tasks.workunit.client.0.smithi094.stdout:6/474: getdents d0/d21/d2a/d55 0 2024-03-24T10:45:56.742 INFO:tasks.workunit.client.0.smithi094.stdout:2/379: mkdir d18/d48/d56/d73 0 2024-03-24T10:45:56.743 INFO:tasks.workunit.client.0.smithi094.stdout:4/508: mkdir d0/d5/d1b/d4f/d70/d6d/d95 0 2024-03-24T10:45:56.744 INFO:tasks.workunit.client.0.smithi094.stdout:9/466: rename df/c42 to df/c92 0 2024-03-24T10:45:56.745 INFO:tasks.workunit.client.0.smithi094.stdout:9/467: getdents df/d2d/d1f/d58/d82 0 2024-03-24T10:45:56.745 INFO:tasks.workunit.client.0.smithi094.stdout:9/468: readlink df/d2d/d1f/d11/d43/l45 0 2024-03-24T10:45:56.746 INFO:tasks.workunit.client.0.smithi094.stdout:3/356: dwrite d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/d73/f75 [0,4194304] 0 2024-03-24T10:45:56.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:56 smithi094 ceph-mon[29446]: pgmap v241: 65 pgs: 65 active+clean; 402 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 17 MiB/s rd, 33 MiB/s wr, 606 op/s 2024-03-24T10:45:56.747 INFO:tasks.workunit.client.0.smithi094.stdout:9/469: rename df/d2d/d34/d2c/l6c to df/d2d/d34/d2c/l93 0 2024-03-24T10:45:56.749 INFO:tasks.workunit.client.0.smithi094.stdout:6/475: getdents d0/d1a/d35 0 2024-03-24T10:45:56.750 INFO:tasks.workunit.client.0.smithi094.stdout:4/509: getdents d0 0 2024-03-24T10:45:56.755 INFO:tasks.workunit.client.0.smithi094.stdout:9/470: symlink df/l94 0 2024-03-24T10:45:56.755 INFO:tasks.workunit.client.0.smithi094.stdout:9/471: write df/f56 [2645239,90440] 0 2024-03-24T10:45:56.757 INFO:tasks.workunit.client.0.smithi094.stdout:6/476: creat d0/d21/d2a/d55/f93 x:0 0 0 2024-03-24T10:45:56.757 INFO:tasks.workunit.client.0.smithi094.stdout:2/380: dwrite d18/f1b [0,4194304] 0 2024-03-24T10:45:56.759 INFO:tasks.workunit.client.0.smithi094.stdout:9/472: rmdir df/d2d/d1f/d58/d5a/d79 0 2024-03-24T10:45:56.760 INFO:tasks.workunit.client.0.smithi094.stdout:4/510: dread d0/d22/ff [0,4194304] 0 2024-03-24T10:45:56.760 INFO:tasks.workunit.client.0.smithi094.stdout:4/511: chown d0/d22/de/d28/d3d 6421718 1 2024-03-24T10:45:56.760 INFO:tasks.workunit.client.0.smithi094.stdout:6/477: link d0/d78/c15 d0/d21/d2a/d2f/c94 0 2024-03-24T10:45:56.762 INFO:tasks.workunit.client.0.smithi094.stdout:2/381: mkdir d18/d1e/d5a/d61/d74 0 2024-03-24T10:45:56.763 INFO:tasks.workunit.client.0.smithi094.stdout:9/473: symlink df/d2d/d34/d2c/d74/l95 0 2024-03-24T10:45:56.764 INFO:tasks.workunit.client.0.smithi094.stdout:4/512: creat d0/d34/d73/d93/f96 x:0 0 0 2024-03-24T10:45:56.764 INFO:tasks.workunit.client.0.smithi094.stdout:4/513: write d0/d5/d1b/f90 [412938,17428] 0 2024-03-24T10:45:56.764 INFO:tasks.workunit.client.0.smithi094.stdout:3/357: dwrite d0/db/dc/d28/d38/f72 [0,4194304] 0 2024-03-24T10:45:56.764 INFO:tasks.workunit.client.0.smithi094.stdout:3/358: chown d0/f11 5372105 1 2024-03-24T10:45:56.765 INFO:tasks.workunit.client.0.smithi094.stdout:2/382: dread d18/d1e/d65/f6a [0,4194304] 0 2024-03-24T10:45:56.765 INFO:tasks.workunit.client.0.smithi094.stdout:2/383: fdatasync d18/d1e/d34/f51 0 2024-03-24T10:45:56.766 INFO:tasks.workunit.client.0.smithi094.stdout:2/384: chown d18/d48/l62 23 1 2024-03-24T10:45:56.770 INFO:tasks.workunit.client.0.smithi094.stdout:9/474: dread df/d2d/d1f/f49 [0,4194304] 0 2024-03-24T10:45:56.777 INFO:tasks.workunit.client.0.smithi094.stdout:3/359: creat d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/d73/f80 x:0 0 0 2024-03-24T10:45:56.777 INFO:tasks.workunit.client.0.smithi094.stdout:3/360: readlink d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/l5a 0 2024-03-24T10:45:56.777 INFO:tasks.workunit.client.0.smithi094.stdout:3/361: write d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/f64 [131327,57812] 0 2024-03-24T10:45:56.777 INFO:tasks.workunit.client.0.smithi094.stdout:6/478: rename d0/d21/l11 to d0/l95 0 2024-03-24T10:45:56.779 INFO:tasks.workunit.client.0.smithi094.stdout:9/475: unlink df/d2d/d34/d2c/f57 0 2024-03-24T10:45:56.779 INFO:tasks.workunit.client.0.smithi094.stdout:9/476: write df/d2d/d34/d2c/f4d [4436083,17866] 0 2024-03-24T10:45:56.781 INFO:tasks.workunit.client.0.smithi094.stdout:6/479: mkdir d0/d1a/d35/d96 0 2024-03-24T10:45:56.781 INFO:tasks.workunit.client.0.smithi094.stdout:6/480: fdatasync d0/d1a/d35/f72 0 2024-03-24T10:45:56.783 INFO:tasks.workunit.client.0.smithi094.stdout:2/385: dwrite d18/f23 [0,4194304] 0 2024-03-24T10:45:56.783 INFO:tasks.workunit.client.0.smithi094.stdout:9/477: truncate df/d2d/d1f/d11/d43/f76 3132724 0 2024-03-24T10:45:56.783 INFO:tasks.workunit.client.0.smithi094.stdout:9/478: write df/d2d/d1f/d11/f75 [696367,117800] 0 2024-03-24T10:45:56.783 INFO:tasks.workunit.client.0.smithi094.stdout:9/479: fdatasync df/d2d/d1f/d11/d53/f81 0 2024-03-24T10:45:56.787 INFO:tasks.workunit.client.0.smithi094.stdout:2/386: mknod d18/d48/d56/d73/c75 0 2024-03-24T10:45:56.788 INFO:tasks.workunit.client.0.smithi094.stdout:9/480: truncate df/d2d/d1f/f49 3363753 0 2024-03-24T10:45:56.790 INFO:tasks.workunit.client.0.smithi094.stdout:4/514: rename d0/d5/d42 to d0/d5/d7d/d97 0 2024-03-24T10:45:56.791 INFO:tasks.workunit.client.0.smithi094.stdout:6/481: mknod d0/c97 0 2024-03-24T10:45:56.792 INFO:tasks.workunit.client.0.smithi094.stdout:9/481: truncate df/d2d/d1f/d58/d5a/f6f 148694 0 2024-03-24T10:45:56.792 INFO:tasks.workunit.client.0.smithi094.stdout:9/482: fdatasync df/d2d/d1f/d11/d43/f76 0 2024-03-24T10:45:56.794 INFO:tasks.workunit.client.0.smithi094.stdout:6/482: rename d0/d78/da/l57 to d0/d21/d2a/l98 0 2024-03-24T10:45:56.795 INFO:tasks.workunit.client.0.smithi094.stdout:2/387: dread d18/f1b [0,4194304] 0 2024-03-24T10:45:56.795 INFO:tasks.workunit.client.0.smithi094.stdout:3/362: dwrite d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/f7b [0,4194304] 0 2024-03-24T10:45:56.796 INFO:tasks.workunit.client.0.smithi094.stdout:9/483: mkdir df/d2d/d1f/d11/d96 0 2024-03-24T10:45:56.796 INFO:tasks.workunit.client.0.smithi094.stdout:9/484: readlink df/d2d/d1f/d11/d43/l45 0 2024-03-24T10:45:56.796 INFO:tasks.workunit.client.0.smithi094.stdout:9/485: getdents df/d2d/d1f/d11/d53 0 2024-03-24T10:45:56.796 INFO:tasks.workunit.client.0.smithi094.stdout:5/525: getdents d7/d23/d5f/d14 0 2024-03-24T10:45:56.798 INFO:tasks.workunit.client.0.smithi094.stdout:6/483: dread d0/d78/da/f2e [0,4194304] 0 2024-03-24T10:45:56.798 INFO:tasks.workunit.client.0.smithi094.stdout:6/484: chown d0/d21/l4a 402 1 2024-03-24T10:45:56.802 INFO:tasks.workunit.client.0.smithi094.stdout:3/363: rename d0/db/dc/d28/d50/c66 to d0/db/dc/d28/d3c/d55/c81 0 2024-03-24T10:45:56.802 INFO:tasks.workunit.client.0.smithi094.stdout:2/388: read f8 [3458119,130465] 0 2024-03-24T10:45:56.803 INFO:tasks.workunit.client.0.smithi094.stdout:3/364: rename d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/f33 to d0/db/dc/d28/d38/d6a/f82 0 2024-03-24T10:45:56.803 INFO:tasks.workunit.client.0.smithi094.stdout:3/365: fdatasync d0/f2 0 2024-03-24T10:45:56.805 INFO:tasks.workunit.client.0.smithi094.stdout:2/389: creat d18/d1e/f76 x:0 0 0 2024-03-24T10:45:56.805 INFO:tasks.workunit.client.0.smithi094.stdout:2/390: getdents d18/d1e/d5a/d61/d74 0 2024-03-24T10:45:56.805 INFO:tasks.workunit.client.0.smithi094.stdout:2/391: readlink d18/d1e/l2d 0 2024-03-24T10:45:56.806 INFO:tasks.workunit.client.0.smithi094.stdout:1/367: sync 2024-03-24T10:45:56.806 INFO:tasks.workunit.client.0.smithi094.stdout:7/460: sync 2024-03-24T10:45:56.806 INFO:tasks.workunit.client.0.smithi094.stdout:8/502: sync 2024-03-24T10:45:56.806 INFO:tasks.workunit.client.0.smithi094.stdout:7/461: fsync d1/d9/f3c 0 2024-03-24T10:45:56.807 INFO:tasks.workunit.client.0.smithi094.stdout:4/515: dwrite d0/d22/de/d28/f59 [0,4194304] 0 2024-03-24T10:45:56.808 INFO:tasks.workunit.client.0.smithi094.stdout:8/503: mknod df/d25/d36/d3f/cad 0 2024-03-24T10:45:56.811 INFO:tasks.workunit.client.0.smithi094.stdout:7/462: rmdir d1/d9/d14/d34/d7f 0 2024-03-24T10:45:56.812 INFO:tasks.workunit.client.0.smithi094.stdout:7/463: chown d1/d6c 2036116 1 2024-03-24T10:45:56.815 INFO:tasks.workunit.client.0.smithi094.stdout:4/516: unlink d0/d22/de/d28/d2b/d33/c45 0 2024-03-24T10:45:56.815 INFO:tasks.workunit.client.0.smithi094.stdout:4/517: stat d0/d5/d1b/l50 0 2024-03-24T10:45:56.817 INFO:tasks.workunit.client.0.smithi094.stdout:1/368: dread d4/d9/fd [0,4194304] 0 2024-03-24T10:45:56.817 INFO:tasks.workunit.client.0.smithi094.stdout:1/369: readlink d4/d9/d37/l4a 0 2024-03-24T10:45:56.819 INFO:tasks.workunit.client.0.smithi094.stdout:4/518: mknod d0/d5/d1b/d4f/c98 0 2024-03-24T10:45:56.825 INFO:tasks.workunit.client.0.smithi094.stdout:1/370: rename d4/d9/d47 to d4/d9/d7d 0 2024-03-24T10:45:56.825 INFO:tasks.workunit.client.0.smithi094.stdout:7/464: dread d1/f2 [0,4194304] 0 2024-03-24T10:45:56.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:56 smithi096 ceph-mon[38097]: pgmap v241: 65 pgs: 65 active+clean; 402 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 17 MiB/s rd, 33 MiB/s wr, 606 op/s 2024-03-24T10:45:56.827 INFO:tasks.workunit.client.0.smithi094.stdout:7/465: symlink d1/d9/d35/d61/l97 0 2024-03-24T10:45:56.830 INFO:tasks.workunit.client.0.smithi094.stdout:7/466: rename d1/f2 to d1/d9/d25/d3d/f98 0 2024-03-24T10:45:56.830 INFO:tasks.workunit.client.0.smithi094.stdout:7/467: dread - d1/d9/d16/d21/d51/d83/d39/d44/f8a zero size 2024-03-24T10:45:56.832 INFO:tasks.workunit.client.0.smithi094.stdout:7/468: link d1/d9/d35/c67 d1/d9/d35/c99 0 2024-03-24T10:45:56.832 INFO:tasks.workunit.client.0.smithi094.stdout:7/469: write d1/d9/d25/d3d/f73 [633018,93554] 0 2024-03-24T10:45:56.832 INFO:tasks.workunit.client.0.smithi094.stdout:7/470: write d1/d9/d14/f4b [4477652,101269] 0 2024-03-24T10:45:56.834 INFO:tasks.workunit.client.0.smithi094.stdout:7/471: creat d1/d9/d96/f9a x:0 0 0 2024-03-24T10:45:56.834 INFO:tasks.workunit.client.0.smithi094.stdout:7/472: write d1/d9/d25/d3d/f73 [901004,56756] 0 2024-03-24T10:45:56.835 INFO:tasks.workunit.client.0.smithi094.stdout:7/473: chown d1/d9/d14/f4b 128 1 2024-03-24T10:45:56.835 INFO:tasks.workunit.client.0.smithi094.stdout:7/474: fdatasync d1/d9/d16/d21/d51/f8e 0 2024-03-24T10:45:56.836 INFO:tasks.workunit.client.0.smithi094.stdout:3/366: dwrite d0/db/dc/f2d [0,4194304] 0 2024-03-24T10:45:56.836 INFO:tasks.workunit.client.0.smithi094.stdout:3/367: dread - d0/db/dc/d28/d50/d5e/f7e zero size 2024-03-24T10:45:56.836 INFO:tasks.workunit.client.0.smithi094.stdout:3/368: stat d0/db/dc/d28/d3c/d55 0 2024-03-24T10:45:56.836 INFO:tasks.workunit.client.0.smithi094.stdout:3/369: creat d0/db/dc/d28/d3c/f83 x:0 0 0 2024-03-24T10:45:56.837 INFO:tasks.workunit.client.0.smithi094.stdout:2/392: dwrite d18/d1e/f25 [0,4194304] 0 2024-03-24T10:45:56.839 INFO:tasks.workunit.client.0.smithi094.stdout:3/370: unlink d0/db/dc/d28/c7d 0 2024-03-24T10:45:56.841 INFO:tasks.workunit.client.0.smithi094.stdout:2/393: mkdir d18/d77 0 2024-03-24T10:45:56.841 INFO:tasks.workunit.client.0.smithi094.stdout:2/394: readlink l13 0 2024-03-24T10:45:56.841 INFO:tasks.workunit.client.0.smithi094.stdout:2/395: write d18/d1e/d65/f6a [630072,21131] 0 2024-03-24T10:45:56.842 INFO:tasks.workunit.client.0.smithi094.stdout:7/475: dread d1/d9/d35/f6b [0,4194304] 0 2024-03-24T10:45:56.844 INFO:tasks.workunit.client.0.smithi094.stdout:7/476: write d1/d9/d25/d3d/f5a [80785,8811] 0 2024-03-24T10:45:56.847 INFO:tasks.workunit.client.0.smithi094.stdout:2/396: creat d18/d77/f78 x:0 0 0 2024-03-24T10:45:56.848 INFO:tasks.workunit.client.0.smithi094.stdout:8/504: dwrite fd [0,4194304] 0 2024-03-24T10:45:56.849 INFO:tasks.workunit.client.0.smithi094.stdout:4/519: dwrite d0/d5/d7d/f88 [0,4194304] 0 2024-03-24T10:45:56.850 INFO:tasks.workunit.client.0.smithi094.stdout:7/477: mkdir d1/d9b 0 2024-03-24T10:45:56.851 INFO:tasks.workunit.client.0.smithi094.stdout:2/397: rmdir d18/d1e/d5a/d61 39 2024-03-24T10:45:56.852 INFO:tasks.workunit.client.0.smithi094.stdout:1/371: dwrite f1 [4194304,4194304] 0 2024-03-24T10:45:56.852 INFO:tasks.workunit.client.0.smithi094.stdout:1/372: creat d4/d9/d37/d33/f7e x:0 0 0 2024-03-24T10:45:56.852 INFO:tasks.workunit.client.0.smithi094.stdout:1/373: stat d4/d9/db/d5d/d63 0 2024-03-24T10:45:56.852 INFO:tasks.workunit.client.0.smithi094.stdout:1/374: fdatasync d4/d9/f32 0 2024-03-24T10:45:56.855 INFO:tasks.workunit.client.0.smithi094.stdout:8/505: mkdir df/d25/d36/d98/dae 0 2024-03-24T10:45:56.858 INFO:tasks.workunit.client.0.smithi094.stdout:2/398: truncate d18/d1e/d5a/f5e 1516934 0 2024-03-24T10:45:56.858 INFO:tasks.workunit.client.0.smithi094.stdout:2/399: chown d18/l2c 0 1 2024-03-24T10:45:56.858 INFO:tasks.workunit.client.0.smithi094.stdout:3/371: dwrite d0/db/f2b [0,4194304] 0 2024-03-24T10:45:56.862 INFO:tasks.workunit.client.0.smithi094.stdout:1/375: mknod d4/d9/db/d5d/d63/c7f 0 2024-03-24T10:45:56.862 INFO:tasks.workunit.client.0.smithi094.stdout:1/376: stat d4/d9/l35 0 2024-03-24T10:45:56.864 INFO:tasks.workunit.client.0.smithi094.stdout:3/372: truncate d0/db/dc/d28/d38/d53/f78 1868499 0 2024-03-24T10:45:56.864 INFO:tasks.workunit.client.0.smithi094.stdout:3/373: fsync d0/db/dc/d28/d38/d6a/f79 0 2024-03-24T10:45:56.865 INFO:tasks.workunit.client.0.smithi094.stdout:3/374: symlink d0/db/dc/d28/d38/d6a/l84 0 2024-03-24T10:45:56.868 INFO:tasks.workunit.client.0.smithi094.stdout:0/468: sync 2024-03-24T10:45:56.868 INFO:tasks.workunit.client.0.smithi094.stdout:0/469: chown d14/d22/d28/d2b/f3b 37931 1 2024-03-24T10:45:56.868 INFO:tasks.workunit.client.0.smithi094.stdout:0/470: creat d14/d22/d7d/fa4 x:0 0 0 2024-03-24T10:45:56.872 INFO:tasks.workunit.client.0.smithi094.stdout:0/471: mkdir d14/d22/d28/d48/da5 0 2024-03-24T10:45:56.877 INFO:tasks.workunit.client.0.smithi094.stdout:4/520: dwrite d0/d22/de/d28/d2b/d33/f54 [0,4194304] 0 2024-03-24T10:45:56.877 INFO:tasks.workunit.client.0.smithi094.stdout:4/521: creat d0/d5/d7d/f99 x:0 0 0 2024-03-24T10:45:56.878 INFO:tasks.workunit.client.0.smithi094.stdout:2/400: dread d18/d1e/d34/d4a/f6d [0,4194304] 0 2024-03-24T10:45:56.881 INFO:tasks.workunit.client.0.smithi094.stdout:4/522: truncate d0/f47 768025 0 2024-03-24T10:45:56.881 INFO:tasks.workunit.client.0.smithi094.stdout:1/377: dwrite d4/f4b [4194304,4194304] 0 2024-03-24T10:45:56.881 INFO:tasks.workunit.client.0.smithi094.stdout:3/375: dread d0/f3 [4194304,4194304] 0 2024-03-24T10:45:56.881 INFO:tasks.workunit.client.0.smithi094.stdout:1/378: chown d4/d9/de/c14 44 1 2024-03-24T10:45:56.881 INFO:tasks.workunit.client.0.smithi094.stdout:7/478: dwrite d1/d9/f12 [4194304,4194304] 0 2024-03-24T10:45:56.881 INFO:tasks.workunit.client.0.smithi094.stdout:2/401: mknod d18/d77/c79 0 2024-03-24T10:45:56.882 INFO:tasks.workunit.client.0.smithi094.stdout:2/402: truncate d18/d1e/f33 2566856 0 2024-03-24T10:45:56.882 INFO:tasks.workunit.client.0.smithi094.stdout:8/506: dwrite df/d12/d5b/f83 [4194304,4194304] 0 2024-03-24T10:45:56.882 INFO:tasks.workunit.client.0.smithi094.stdout:8/507: chown df/d12/l53 4 1 2024-03-24T10:45:56.885 INFO:tasks.workunit.client.0.smithi094.stdout:1/379: mkdir d4/d9/de/d6f/d62/d80 0 2024-03-24T10:45:56.887 INFO:tasks.workunit.client.0.smithi094.stdout:7/479: write d1/d9/d14/f36 [3234808,55186] 0 2024-03-24T10:45:56.887 INFO:tasks.workunit.client.0.smithi094.stdout:7/480: read - d1/d6c/f82 zero size 2024-03-24T10:45:56.887 INFO:tasks.workunit.client.0.smithi094.stdout:7/481: fdatasync d1/d9/d16/d21/d51/d83/d39/d44/f4e 0 2024-03-24T10:45:56.887 INFO:tasks.workunit.client.0.smithi094.stdout:3/376: symlink d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/l85 0 2024-03-24T10:45:56.888 INFO:tasks.workunit.client.0.smithi094.stdout:2/403: creat d18/d48/d56/d73/f7a x:0 0 0 2024-03-24T10:45:56.891 INFO:tasks.workunit.client.0.smithi094.stdout:8/508: mkdir df/d25/daf 0 2024-03-24T10:45:56.891 INFO:tasks.workunit.client.0.smithi094.stdout:0/472: dwrite d14/d22/d28/d2b/d41/d53/f5a [0,4194304] 0 2024-03-24T10:45:56.892 INFO:tasks.workunit.client.0.smithi094.stdout:0/473: creat d14/d22/d7d/fa6 x:0 0 0 2024-03-24T10:45:56.892 INFO:tasks.workunit.client.0.smithi094.stdout:4/523: rename d0/d34/d73/l74 to d0/d1f/l9a 0 2024-03-24T10:45:56.892 INFO:tasks.workunit.client.0.smithi094.stdout:4/524: fsync d0/d22/de/d28/d2b/d33/f54 0 2024-03-24T10:45:56.893 INFO:tasks.workunit.client.0.smithi094.stdout:1/380: getdents d4/d9/de/d3d/d66 0 2024-03-24T10:45:56.893 INFO:tasks.workunit.client.0.smithi094.stdout:1/381: stat d4/d9/db/d31/f48 0 2024-03-24T10:45:56.895 INFO:tasks.workunit.client.0.smithi094.stdout:7/482: symlink d1/d9/d14/l9c 0 2024-03-24T10:45:56.895 INFO:tasks.workunit.client.0.smithi094.stdout:7/483: truncate d1/d9/d25/d3d/f6d 437540 0 2024-03-24T10:45:56.897 INFO:tasks.workunit.client.0.smithi094.stdout:3/377: mkdir d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d86 0 2024-03-24T10:45:56.899 INFO:tasks.workunit.client.0.smithi094.stdout:2/404: unlink f8 0 2024-03-24T10:45:56.899 INFO:tasks.workunit.client.0.smithi094.stdout:2/405: readlink d18/l2c 0 2024-03-24T10:45:56.899 INFO:tasks.workunit.client.0.smithi094.stdout:2/406: stat c9 0 2024-03-24T10:45:56.899 INFO:tasks.workunit.client.0.smithi094.stdout:2/407: chown d18/d1e/c41 164993 1 2024-03-24T10:45:56.899 INFO:tasks.workunit.client.0.smithi094.stdout:2/408: truncate d18/d77/f78 718580 0 2024-03-24T10:45:56.899 INFO:tasks.workunit.client.0.smithi094.stdout:9/486: dwrite df/d2d/d34/d2c/f73 [0,4194304] 0 2024-03-24T10:45:56.899 INFO:tasks.workunit.client.0.smithi094.stdout:4/525: dread d0/d22/f62 [0,4194304] 0 2024-03-24T10:45:56.899 INFO:tasks.workunit.client.0.smithi094.stdout:4/526: fsync d0/d22/de/d28/d2b/f51 0 2024-03-24T10:45:56.899 INFO:tasks.workunit.client.0.smithi094.stdout:4/527: truncate d0/d34/f82 814613 0 2024-03-24T10:45:56.900 INFO:tasks.workunit.client.0.smithi094.stdout:8/509: creat df/d25/daf/fb0 x:0 0 0 2024-03-24T10:45:56.906 INFO:tasks.workunit.client.0.smithi094.stdout:0/474: creat d14/d22/d28/d48/da5/fa7 x:0 0 0 2024-03-24T10:45:56.910 INFO:tasks.workunit.client.0.smithi094.stdout:3/378: mkdir d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d87 0 2024-03-24T10:45:56.912 INFO:tasks.workunit.client.0.smithi094.stdout:2/409: write d18/f45 [377180,79599] 0 2024-03-24T10:45:56.913 INFO:tasks.workunit.client.0.smithi094.stdout:9/487: mkdir df/d2d/d34/d2c/d97 0 2024-03-24T10:45:56.913 INFO:tasks.workunit.client.0.smithi094.stdout:9/488: readlink df/l20 0 2024-03-24T10:45:56.915 INFO:tasks.workunit.client.0.smithi094.stdout:4/528: symlink d0/d5/l9b 0 2024-03-24T10:45:56.917 INFO:tasks.workunit.client.0.smithi094.stdout:3/379: creat d0/db/dc/d28/f88 x:0 0 0 2024-03-24T10:45:56.918 INFO:tasks.workunit.client.0.smithi094.stdout:9/489: creat df/d2d/d1f/d58/d5a/d69/f98 x:0 0 0 2024-03-24T10:45:56.921 INFO:tasks.workunit.client.0.smithi094.stdout:9/490: rmdir df/d2d/d72 0 2024-03-24T10:45:56.921 INFO:tasks.workunit.client.0.smithi094.stdout:0/475: write d14/d18/f2e [3695802,85955] 0 2024-03-24T10:45:56.921 INFO:tasks.workunit.client.0.smithi094.stdout:0/476: creat d14/d22/d28/d48/d7b/fa8 x:0 0 0 2024-03-24T10:45:56.923 INFO:tasks.workunit.client.0.smithi094.stdout:9/491: symlink df/d2d/d1f/d58/d5a/d69/d7a/l99 0 2024-03-24T10:45:56.923 INFO:tasks.workunit.client.0.smithi094.stdout:9/492: write df/d2d/d1f/d58/d5a/d69/d7a/f7d [273500,110600] 0 2024-03-24T10:45:56.923 INFO:tasks.workunit.client.0.smithi094.stdout:9/493: stat df/d2d/d34 0 2024-03-24T10:45:56.923 INFO:tasks.workunit.client.0.smithi094.stdout:9/494: write df/d2d/f83 [1179818,66855] 0 2024-03-24T10:45:56.925 INFO:tasks.workunit.client.0.smithi094.stdout:0/477: rename d14/d22/d7d/c96 to d14/d22/d28/d48/d7b/ca9 0 2024-03-24T10:45:56.927 INFO:tasks.workunit.client.0.smithi094.stdout:9/495: write df/d2d/d1f/d11/f4f [248425,84623] 0 2024-03-24T10:45:56.928 INFO:tasks.workunit.client.0.smithi094.stdout:0/478: creat d14/d22/d28/d48/faa x:0 0 0 2024-03-24T10:45:56.930 INFO:tasks.workunit.client.0.smithi094.stdout:3/380: read d0/db/dc/f12 [3592594,60108] 0 2024-03-24T10:45:56.930 INFO:tasks.workunit.client.0.smithi094.stdout:1/382: dwrite d4/d36/f70 [0,4194304] 0 2024-03-24T10:45:56.930 INFO:tasks.workunit.client.0.smithi094.stdout:1/383: chown d4/d9/db/d5d/d63/c7f 3 1 2024-03-24T10:45:56.930 INFO:tasks.workunit.client.0.smithi094.stdout:3/381: creat d0/db/dc/d28/d38/d53/f89 x:0 0 0 2024-03-24T10:45:56.930 INFO:tasks.workunit.client.0.smithi094.stdout:1/384: write d4/d9/db/d31/d3c/f50 [3556838,123439] 0 2024-03-24T10:45:56.934 INFO:tasks.workunit.client.0.smithi094.stdout:8/510: dwrite df/d25/d28/d31/f79 [0,4194304] 0 2024-03-24T10:45:56.935 INFO:tasks.workunit.client.0.smithi094.stdout:1/385: mknod d4/d9/de/d3d/d66/c81 0 2024-03-24T10:45:56.936 INFO:tasks.workunit.client.0.smithi094.stdout:2/410: dwrite d18/d1e/d34/d4a/f6c [0,4194304] 0 2024-03-24T10:45:56.936 INFO:tasks.workunit.client.0.smithi094.stdout:2/411: truncate d18/d1e/d34/f6f 414244 0 2024-03-24T10:45:56.936 INFO:tasks.workunit.client.0.smithi094.stdout:7/484: dwrite d1/d9/d16/d21/d51/d83/d39/d44/f8a [0,4194304] 0 2024-03-24T10:45:56.936 INFO:tasks.workunit.client.0.smithi094.stdout:0/479: dread d14/d18/f40 [0,4194304] 0 2024-03-24T10:45:56.942 INFO:tasks.workunit.client.0.smithi094.stdout:9/496: dwrite df/f52 [0,4194304] 0 2024-03-24T10:45:56.942 INFO:tasks.workunit.client.0.smithi094.stdout:9/497: chown df/d2d/d1f/d58/d5a/d69/d7a 172943 1 2024-03-24T10:45:56.943 INFO:tasks.workunit.client.0.smithi094.stdout:1/386: getdents d4/d9/de 0 2024-03-24T10:45:56.949 INFO:tasks.workunit.client.0.smithi094.stdout:3/382: dread d0/db/dc/d28/d3c/d55/d7f/d20/d23/f3f [0,4194304] 0 2024-03-24T10:45:56.956 INFO:tasks.workunit.client.0.smithi094.stdout:2/412: symlink d18/d48/l7b 0 2024-03-24T10:45:56.962 INFO:tasks.workunit.client.0.smithi094.stdout:8/511: dwrite df/d25/d28/d31/d47/f85 [0,4194304] 0 2024-03-24T10:45:56.963 INFO:tasks.workunit.client.0.smithi094.stdout:7/485: dwrite d1/d9/d14/f6e [0,4194304] 0 2024-03-24T10:45:56.968 INFO:tasks.workunit.client.0.smithi094.stdout:0/480: creat d14/d22/d32/d65/fab x:0 0 0 2024-03-24T10:45:56.971 INFO:tasks.workunit.client.0.smithi094.stdout:2/413: dwrite d18/d1e/d65/f6a [0,4194304] 0 2024-03-24T10:45:56.971 INFO:tasks.workunit.client.0.smithi094.stdout:1/387: rmdir d4/d9/de 39 2024-03-24T10:45:56.973 INFO:tasks.workunit.client.0.smithi094.stdout:3/383: mknod d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d87/c8a 0 2024-03-24T10:45:56.983 INFO:tasks.workunit.client.0.smithi094.stdout:9/498: rmdir df/d2d/d1f/d11 39 2024-03-24T10:45:56.987 INFO:tasks.workunit.client.0.smithi094.stdout:2/414: symlink d18/d48/d56/l7c 0 2024-03-24T10:45:56.989 INFO:tasks.workunit.client.0.smithi094.stdout:1/388: mknod d4/d9/de/d6f/d5f/c82 0 2024-03-24T10:45:56.990 INFO:tasks.workunit.client.0.smithi094.stdout:9/499: mknod df/d2d/d34/d2c/d97/c9a 0 2024-03-24T10:45:56.990 INFO:tasks.workunit.client.0.smithi094.stdout:9/500: chown df/f3f 62047327 1 2024-03-24T10:45:56.990 INFO:tasks.workunit.client.0.smithi094.stdout:9/501: chown df/d2d/d34/d2c/d97 29 1 2024-03-24T10:45:56.990 INFO:tasks.workunit.client.0.smithi094.stdout:2/415: stat l6 0 2024-03-24T10:45:56.991 INFO:tasks.workunit.client.0.smithi094.stdout:1/389: mknod d4/d9/d37/c83 0 2024-03-24T10:45:56.992 INFO:tasks.workunit.client.0.smithi094.stdout:9/502: mknod df/d2d/d1f/d11/c9b 0 2024-03-24T10:45:56.993 INFO:tasks.workunit.client.0.smithi094.stdout:2/416: unlink d18/f2e 0 2024-03-24T10:45:56.993 INFO:tasks.workunit.client.0.smithi094.stdout:2/417: truncate d18/d1e/f53 727139 0 2024-03-24T10:45:56.994 INFO:tasks.workunit.client.0.smithi094.stdout:1/390: mkdir d4/d9/de/d6f/d62/d84 0 2024-03-24T10:45:56.997 INFO:tasks.workunit.client.0.smithi094.stdout:9/503: getdents df/d2d 0 2024-03-24T10:45:57.004 INFO:tasks.workunit.client.0.smithi094.stdout:7/486: dwrite d1/d9/d16/d21/d51/d83/d39/d44/f56 [0,4194304] 0 2024-03-24T10:45:57.004 INFO:tasks.workunit.client.0.smithi094.stdout:7/487: fsync d1/d9/d25/f50 0 2024-03-24T10:45:57.005 INFO:tasks.workunit.client.0.smithi094.stdout:7/488: truncate d1/d9/d25/f50 4120134 0 2024-03-24T10:45:57.006 INFO:tasks.workunit.client.0.smithi094.stdout:6/485: sync 2024-03-24T10:45:57.006 INFO:tasks.workunit.client.0.smithi094.stdout:5/526: sync 2024-03-24T10:45:57.007 INFO:tasks.workunit.client.0.smithi094.stdout:6/486: mknod d0/d30/d50/d4d/c99 0 2024-03-24T10:45:57.007 INFO:tasks.workunit.client.0.smithi094.stdout:6/487: stat d0/d30/l82 0 2024-03-24T10:45:57.007 INFO:tasks.workunit.client.0.smithi094.stdout:0/481: dwrite d14/d22/d28/f46 [0,4194304] 0 2024-03-24T10:45:57.007 INFO:tasks.workunit.client.0.smithi094.stdout:3/384: dwrite d0/db/dc/d28/d38/d53/f78 [0,4194304] 0 2024-03-24T10:45:57.008 INFO:tasks.workunit.client.0.smithi094.stdout:7/489: getdents d1/d9/d16/d21/d51 0 2024-03-24T10:45:57.009 INFO:tasks.workunit.client.0.smithi094.stdout:8/512: dwrite df/d12/d5b/d63/f94 [0,4194304] 0 2024-03-24T10:45:57.009 INFO:tasks.workunit.client.0.smithi094.stdout:9/504: dread df/d2d/d1f/d58/d5a/f63 [0,4194304] 0 2024-03-24T10:45:57.011 INFO:tasks.workunit.client.0.smithi094.stdout:0/482: unlink d14/c19 0 2024-03-24T10:45:57.013 INFO:tasks.workunit.client.0.smithi094.stdout:2/418: dwrite d18/d48/f5d [0,4194304] 0 2024-03-24T10:45:57.013 INFO:tasks.workunit.client.0.smithi094.stdout:3/385: dread d0/f2 [4194304,4194304] 0 2024-03-24T10:45:57.013 INFO:tasks.workunit.client.0.smithi094.stdout:2/419: chown d18/d54 100 1 2024-03-24T10:45:57.013 INFO:tasks.workunit.client.0.smithi094.stdout:7/490: mkdir d1/d9/d16/d21/d51/d83/d9d 0 2024-03-24T10:45:57.013 INFO:tasks.workunit.client.0.smithi094.stdout:7/491: chown d1/d9/fd 29 1 2024-03-24T10:45:57.014 INFO:tasks.workunit.client.0.smithi094.stdout:7/492: creat d1/d9/d35/f9e x:0 0 0 2024-03-24T10:45:57.015 INFO:tasks.workunit.client.0.smithi094.stdout:9/505: dread df/d2d/d1f/d58/d5a/d61/f8a [0,4194304] 0 2024-03-24T10:45:57.015 INFO:tasks.workunit.client.0.smithi094.stdout:6/488: symlink d0/l9a 0 2024-03-24T10:45:57.015 INFO:tasks.workunit.client.0.smithi094.stdout:6/489: write d0/d3b/f6a [1021195,56849] 0 2024-03-24T10:45:57.015 INFO:tasks.workunit.client.0.smithi094.stdout:6/490: stat d0/d3b/f69 0 2024-03-24T10:45:57.015 INFO:tasks.workunit.client.0.smithi094.stdout:6/491: fdatasync d0/d78/f44 0 2024-03-24T10:45:57.016 INFO:tasks.workunit.client.0.smithi094.stdout:8/513: mknod df/d25/d36/cb1 0 2024-03-24T10:45:57.025 INFO:tasks.workunit.client.0.smithi094.stdout:3/386: mkdir d0/db/dc/d28/d3c/d55/d7f/d20/d23/d8b 0 2024-03-24T10:45:57.027 INFO:tasks.workunit.client.0.smithi094.stdout:9/506: mknod df/d2d/d1f/d58/d5a/c9c 0 2024-03-24T10:45:57.027 INFO:tasks.workunit.client.0.smithi094.stdout:9/507: fsync df/d2d/d1f/d58/d5a/f6f 0 2024-03-24T10:45:57.028 INFO:tasks.workunit.client.0.smithi094.stdout:8/514: rmdir df/d8c 39 2024-03-24T10:45:57.029 INFO:tasks.workunit.client.0.smithi094.stdout:2/420: mkdir d18/d1e/d34/d7d 0 2024-03-24T10:45:57.032 INFO:tasks.workunit.client.0.smithi094.stdout:8/515: mkdir df/d25/d36/db2 0 2024-03-24T10:45:57.032 INFO:tasks.workunit.client.0.smithi094.stdout:8/516: chown df/d25/d28/d31/l62 2 1 2024-03-24T10:45:57.032 INFO:tasks.workunit.client.0.smithi094.stdout:8/517: readlink df/d25/l6c 0 2024-03-24T10:45:57.033 INFO:tasks.workunit.client.0.smithi094.stdout:3/387: mknod d0/db/dc/c8c 0 2024-03-24T10:45:57.034 INFO:tasks.workunit.client.0.smithi094.stdout:8/518: mkdir df/d8c/db3 0 2024-03-24T10:45:57.034 INFO:tasks.workunit.client.0.smithi094.stdout:8/519: chown df/d12/d5b/d63/d75/d96 374 1 2024-03-24T10:45:57.035 INFO:tasks.workunit.client.0.smithi094.stdout:9/508: dread df/d2d/f15 [0,4194304] 0 2024-03-24T10:45:57.037 INFO:tasks.workunit.client.0.smithi094.stdout:3/388: unlink d0/db/dc/d28/d50/l77 0 2024-03-24T10:45:57.037 INFO:tasks.workunit.client.0.smithi094.stdout:4/529: sync 2024-03-24T10:45:57.038 INFO:tasks.workunit.client.0.smithi094.stdout:9/509: getdents df/d2d/d1f/d58/d5a/d61 0 2024-03-24T10:45:57.040 INFO:tasks.workunit.client.0.smithi094.stdout:3/389: rename d0/db/dc/d28/c47 to d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/c8d 0 2024-03-24T10:45:57.041 INFO:tasks.workunit.client.0.smithi094.stdout:3/390: dread - d0/db/dc/d28/d38/d6a/f79 zero size 2024-03-24T10:45:57.041 INFO:tasks.workunit.client.0.smithi094.stdout:3/391: write d0/db/dc/d28/d3c/f83 [225767,86171] 0 2024-03-24T10:45:57.042 INFO:tasks.workunit.client.0.smithi094.stdout:7/493: dwrite d1/d9/f2c [0,4194304] 0 2024-03-24T10:45:57.046 INFO:tasks.workunit.client.0.smithi094.stdout:7/494: write d1/d9/d35/f6b [3365274,28727] 0 2024-03-24T10:45:57.047 INFO:tasks.workunit.client.0.smithi094.stdout:7/495: mknod d1/d9/d16/d21/d51/d83/d39/c9f 0 2024-03-24T10:45:57.049 INFO:tasks.workunit.client.0.smithi094.stdout:6/492: dwrite d0/d78/da/f20 [0,4194304] 0 2024-03-24T10:45:57.049 INFO:tasks.workunit.client.0.smithi094.stdout:8/520: dread fa [4194304,4194304] 0 2024-03-24T10:45:57.049 INFO:tasks.workunit.client.0.smithi094.stdout:8/521: readlink df/d12/l4d 0 2024-03-24T10:45:57.049 INFO:tasks.workunit.client.0.smithi094.stdout:8/522: chown df/d25/d28/d31/d47/cab 35 1 2024-03-24T10:45:57.049 INFO:tasks.workunit.client.0.smithi094.stdout:7/496: mknod d1/ca0 0 2024-03-24T10:45:57.052 INFO:tasks.workunit.client.0.smithi094.stdout:6/493: creat d0/d78/da/d23/f9b x:0 0 0 2024-03-24T10:45:57.052 INFO:tasks.workunit.client.0.smithi094.stdout:6/494: write d0/f1e [2776349,43380] 0 2024-03-24T10:45:57.052 INFO:tasks.workunit.client.0.smithi094.stdout:6/495: write d0/d30/f28 [1562490,21660] 0 2024-03-24T10:45:57.052 INFO:tasks.workunit.client.0.smithi094.stdout:6/496: stat d0/d78/c1b 0 2024-03-24T10:45:57.052 INFO:tasks.workunit.client.0.smithi094.stdout:6/497: chown d0/d1a/d35/l79 5562 1 2024-03-24T10:45:57.053 INFO:tasks.workunit.client.0.smithi094.stdout:7/497: truncate d1/d9/d16/f2f 881063 0 2024-03-24T10:45:57.054 INFO:tasks.workunit.client.0.smithi094.stdout:6/498: mkdir d0/d30/d50/d4d/d6b/d86/d9c 0 2024-03-24T10:45:57.055 INFO:tasks.workunit.client.0.smithi094.stdout:9/510: write df/d2d/d1f/d58/d5a/f6a [2492852,12450] 0 2024-03-24T10:45:57.056 INFO:tasks.workunit.client.0.smithi094.stdout:7/498: mkdir d1/d9/d35/d65/da1 0 2024-03-24T10:45:57.058 INFO:tasks.workunit.client.0.smithi094.stdout:6/499: truncate d0/d78/f58 2993883 0 2024-03-24T10:45:57.060 INFO:tasks.workunit.client.0.smithi094.stdout:7/499: mkdir d1/d6c/d80/d81/da2 0 2024-03-24T10:45:57.061 INFO:tasks.workunit.client.0.smithi094.stdout:7/500: fdatasync d1/f5d 0 2024-03-24T10:45:57.063 INFO:tasks.workunit.client.0.smithi094.stdout:7/501: unlink d1/d6c/f82 0 2024-03-24T10:45:57.063 INFO:tasks.workunit.client.0.smithi094.stdout:3/392: dwrite d0/db/dc/d28/d38/d6a/f79 [0,4194304] 0 2024-03-24T10:45:57.063 INFO:tasks.workunit.client.0.smithi094.stdout:3/393: dread - d0/db/dc/d28/d3c/d55/f70 zero size 2024-03-24T10:45:57.065 INFO:tasks.workunit.client.0.smithi094.stdout:4/530: dwrite d0/d22/de/d28/d3d/f5c [0,4194304] 0 2024-03-24T10:45:57.065 INFO:tasks.workunit.client.0.smithi094.stdout:3/394: getdents d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69 0 2024-03-24T10:45:57.065 INFO:tasks.workunit.client.0.smithi094.stdout:3/395: chown d0/db/dc/c8c 2839210 1 2024-03-24T10:45:57.068 INFO:tasks.workunit.client.0.smithi094.stdout:4/531: mkdir d0/d5/d7d/d9c 0 2024-03-24T10:45:57.069 INFO:tasks.workunit.client.0.smithi094.stdout:3/396: creat d0/db/dc/d28/d3c/d7c/f8e x:0 0 0 2024-03-24T10:45:57.071 INFO:tasks.workunit.client.0.smithi094.stdout:6/500: write d0/d78/da/f20 [3654075,70683] 0 2024-03-24T10:45:57.072 INFO:tasks.workunit.client.0.smithi094.stdout:3/397: rename d0/db/dc/d28/d3c/d55/d7f/d20/d23/l37 to d0/db/dc/d28/d38/d6a/l8f 0 2024-03-24T10:45:57.075 INFO:tasks.workunit.client.0.smithi094.stdout:6/501: rename d0/d30/d50/d4d/d6b/l8d to d0/d78/d66/l9d 0 2024-03-24T10:45:57.075 INFO:tasks.workunit.client.0.smithi094.stdout:6/502: chown d0/d78/da/d77/l8a 20 1 2024-03-24T10:45:57.075 INFO:tasks.workunit.client.0.smithi094.stdout:6/503: chown d0/d21/d2a/d2f/c94 25585797 1 2024-03-24T10:45:57.075 INFO:tasks.workunit.client.0.smithi094.stdout:3/398: rmdir d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d87 39 2024-03-24T10:45:57.075 INFO:tasks.workunit.client.0.smithi094.stdout:6/504: write d0/d21/d2a/f40 [4406604,91640] 0 2024-03-24T10:45:57.076 INFO:tasks.workunit.client.0.smithi094.stdout:3/399: symlink d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/l90 0 2024-03-24T10:45:57.077 INFO:tasks.workunit.client.0.smithi094.stdout:6/505: link d0/d78/da/d23/f6c d0/d30/d50/d4d/d6b/d86/f9e 0 2024-03-24T10:45:57.079 INFO:tasks.workunit.client.0.smithi094.stdout:3/400: mknod d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d86/c91 0 2024-03-24T10:45:57.081 INFO:tasks.workunit.client.0.smithi094.stdout:3/401: chown d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/d73/l76 51016590 1 2024-03-24T10:45:57.082 INFO:tasks.workunit.client.0.smithi094.stdout:6/506: dread d0/f2 [0,4194304] 0 2024-03-24T10:45:57.082 INFO:tasks.workunit.client.0.smithi094.stdout:3/402: symlink d0/db/dc/d28/d38/l92 0 2024-03-24T10:45:57.082 INFO:tasks.workunit.client.0.smithi094.stdout:3/403: truncate d0/f42 993003 0 2024-03-24T10:45:57.082 INFO:tasks.workunit.client.0.smithi094.stdout:6/507: getdents d0/d21 0 2024-03-24T10:45:57.087 INFO:tasks.workunit.client.0.smithi094.stdout:3/404: dread d0/db/dc/d28/d38/d6a/f82 [0,4194304] 0 2024-03-24T10:45:57.087 INFO:tasks.workunit.client.0.smithi094.stdout:3/405: chown d0/db/f2b 101992761 1 2024-03-24T10:45:57.091 INFO:tasks.workunit.client.0.smithi094.stdout:4/532: read d0/d22/de/d28/d2b/d33/f54 [3283576,29561] 0 2024-03-24T10:45:57.091 INFO:tasks.workunit.client.0.smithi094.stdout:4/533: write d0/d5/d1b/f39 [2712084,73433] 0 2024-03-24T10:45:57.091 INFO:tasks.workunit.client.0.smithi094.stdout:4/534: readlink d0/d5/l9b 0 2024-03-24T10:45:57.091 INFO:tasks.workunit.client.0.smithi094.stdout:9/511: dwrite df/d2d/d1f/d11/d43/f6d [0,4194304] 0 2024-03-24T10:45:57.093 INFO:tasks.workunit.client.0.smithi094.stdout:9/512: dread df/d2d/d1f/d11/f4f [0,4194304] 0 2024-03-24T10:45:57.094 INFO:tasks.workunit.client.0.smithi094.stdout:9/513: chown df/d2d/d1f/d58/d5a/d69/d7a/f8f 0 1 2024-03-24T10:45:57.096 INFO:tasks.workunit.client.0.smithi094.stdout:3/406: write d0/db/dc/d28/d38/f72 [2878942,99793] 0 2024-03-24T10:45:57.097 INFO:tasks.workunit.client.0.smithi094.stdout:9/514: write df/d2d/d1f/d58/d5a/f6a [4210489,114132] 0 2024-03-24T10:45:57.098 INFO:tasks.workunit.client.0.smithi094.stdout:3/407: rmdir d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d86 39 2024-03-24T10:45:57.098 INFO:tasks.workunit.client.0.smithi094.stdout:3/408: dread - d0/db/dc/d28/d38/d53/f89 zero size 2024-03-24T10:45:57.099 INFO:tasks.workunit.client.0.smithi094.stdout:6/508: dwrite d0/d78/da/d23/d83/f85 [0,4194304] 0 2024-03-24T10:45:57.099 INFO:tasks.workunit.client.0.smithi094.stdout:6/509: write d0/f1e [3563752,60184] 0 2024-03-24T10:45:57.099 INFO:tasks.workunit.client.0.smithi094.stdout:6/510: truncate d0/d78/da/d23/f9b 782326 0 2024-03-24T10:45:57.100 INFO:tasks.workunit.client.0.smithi094.stdout:9/515: rename df/d2d/d34/d2c/l93 to df/d2d/d1f/d7e/l9d 0 2024-03-24T10:45:57.101 INFO:tasks.workunit.client.0.smithi094.stdout:9/516: fdatasync df/d2d/d1f/d7e/f91 0 2024-03-24T10:45:57.101 INFO:tasks.workunit.client.0.smithi094.stdout:9/517: write df/d2d/d1f/f4a [160457,2650] 0 2024-03-24T10:45:57.101 INFO:tasks.workunit.client.0.smithi094.stdout:3/409: creat d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/f93 x:0 0 0 2024-03-24T10:45:57.101 INFO:tasks.workunit.client.0.smithi094.stdout:9/518: chown df/d2d/d1f/d11/d43/l51 791344987 1 2024-03-24T10:45:57.101 INFO:tasks.workunit.client.0.smithi094.stdout:3/410: read d0/db/dc/d28/d38/f72 [156953,18893] 0 2024-03-24T10:45:57.101 INFO:tasks.workunit.client.0.smithi094.stdout:4/535: dread d0/d22/de/d28/d3d/f5c [0,4194304] 0 2024-03-24T10:45:57.101 INFO:tasks.workunit.client.0.smithi094.stdout:4/536: read d0/d22/f62 [363822,121083] 0 2024-03-24T10:45:57.101 INFO:tasks.workunit.client.0.smithi094.stdout:6/511: dread d0/d1a/d35/f72 [0,4194304] 0 2024-03-24T10:45:57.101 INFO:tasks.workunit.client.0.smithi094.stdout:9/519: getdents df/d2d/d1f/d11/d66 0 2024-03-24T10:45:57.102 INFO:tasks.workunit.client.0.smithi094.stdout:4/537: stat d0/d34/l3c 0 2024-03-24T10:45:57.107 INFO:tasks.workunit.client.0.smithi094.stdout:9/520: unlink df/d2d/d34/d2c/f4d 0 2024-03-24T10:45:57.107 INFO:tasks.workunit.client.0.smithi094.stdout:4/538: link d0/d5/c25 d0/d22/c9d 0 2024-03-24T10:45:57.108 INFO:tasks.workunit.client.0.smithi094.stdout:6/512: rmdir d0/d78/da/d23/d34/d4e 39 2024-03-24T10:45:57.112 INFO:tasks.workunit.client.0.smithi094.stdout:4/539: rename d0/d5/d7d/f8a to d0/d22/f9e 0 2024-03-24T10:45:57.112 INFO:tasks.workunit.client.0.smithi094.stdout:6/513: dread d0/d78/da/f2e [0,4194304] 0 2024-03-24T10:45:57.113 INFO:tasks.workunit.client.0.smithi094.stdout:9/521: read df/d2d/d1f/d58/d5a/f6f [1848227,90445] 0 2024-03-24T10:45:57.126 INFO:tasks.workunit.client.0.smithi094.stdout:3/411: dwrite d0/db/dc/d28/d38/f72 [0,4194304] 0 2024-03-24T10:45:57.127 INFO:tasks.workunit.client.0.smithi094.stdout:4/540: dread d0/f5f [0,4194304] 0 2024-03-24T10:45:57.127 INFO:tasks.workunit.client.0.smithi094.stdout:3/412: rename d0/l4b to d0/db/dc/d28/d50/d5e/l94 0 2024-03-24T10:45:57.141 INFO:tasks.workunit.client.0.smithi094.stdout:6/514: dwrite d0/f61 [0,4194304] 0 2024-03-24T10:45:57.143 INFO:tasks.workunit.client.0.smithi094.stdout:6/515: symlink d0/d1a/l9f 0 2024-03-24T10:45:57.143 INFO:tasks.workunit.client.0.smithi094.stdout:6/516: truncate d0/d3b/f6a 1280852 0 2024-03-24T10:45:57.144 INFO:tasks.workunit.client.0.smithi094.stdout:6/517: unlink d0/d30/d50/d4d/c62 0 2024-03-24T10:45:57.145 INFO:tasks.workunit.client.0.smithi094.stdout:6/518: rmdir d0/d78/d66 39 2024-03-24T10:45:57.145 INFO:tasks.workunit.client.0.smithi094.stdout:6/519: chown d0/d30/d50/d4d/d6b 127 1 2024-03-24T10:45:57.145 INFO:tasks.workunit.client.0.smithi094.stdout:4/541: write d0/d5/d7d/d97/d4a/f61 [4359816,37921] 0 2024-03-24T10:45:57.145 INFO:tasks.workunit.client.0.smithi094.stdout:4/542: readlink d0/d5/d7d/d97/l4c 0 2024-03-24T10:45:57.152 INFO:tasks.workunit.client.0.smithi094.stdout:3/413: dwrite d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/f5b [0,4194304] 0 2024-03-24T10:45:57.152 INFO:tasks.workunit.client.0.smithi094.stdout:3/414: chown d0/db/dc/d28/d38/l92 195 1 2024-03-24T10:45:57.152 INFO:tasks.workunit.client.0.smithi094.stdout:4/543: write d0/d22/de/d28/d2b/f69 [12190524,127180] 0 2024-03-24T10:45:57.152 INFO:tasks.workunit.client.0.smithi094.stdout:4/544: write d0/d22/de/d28/d2b/f69 [5470179,30523] 0 2024-03-24T10:45:57.152 INFO:tasks.workunit.client.0.smithi094.stdout:4/545: write d0/d34/f82 [1217378,30567] 0 2024-03-24T10:45:57.152 INFO:tasks.workunit.client.0.smithi094.stdout:9/522: dwrite df/d2d/d1f/f4a [0,4194304] 0 2024-03-24T10:45:57.158 INFO:tasks.workunit.client.0.smithi094.stdout:9/523: mknod df/d2d/d1f/c9e 0 2024-03-24T10:45:57.158 INFO:tasks.workunit.client.0.smithi094.stdout:3/415: rmdir d0/db/dc/d28/d3c/d7c 39 2024-03-24T10:45:57.159 INFO:tasks.workunit.client.0.smithi094.stdout:3/416: write d0/f1e [3848887,870] 0 2024-03-24T10:45:57.159 INFO:tasks.workunit.client.0.smithi094.stdout:3/417: truncate d0/db/dc/d28/d3c/d7c/f8e 269456 0 2024-03-24T10:45:57.159 INFO:tasks.workunit.client.0.smithi094.stdout:3/418: chown d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/l43 4 1 2024-03-24T10:45:57.159 INFO:tasks.workunit.client.0.smithi094.stdout:9/524: creat df/d2d/d1f/d58/d70/f9f x:0 0 0 2024-03-24T10:45:57.160 INFO:tasks.workunit.client.0.smithi094.stdout:3/419: symlink d0/db/dc/d28/d3c/d55/d7f/d20/d23/d8b/l95 0 2024-03-24T10:45:57.160 INFO:tasks.workunit.client.0.smithi094.stdout:3/420: chown d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a 25781711 1 2024-03-24T10:45:57.160 INFO:tasks.workunit.client.0.smithi094.stdout:3/421: chown d0/db/dc/d28/d3c/d55/d7f/d20/d23/l68 596467 1 2024-03-24T10:45:57.160 INFO:tasks.workunit.client.0.smithi094.stdout:3/422: fsync d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/f7b 0 2024-03-24T10:45:57.161 INFO:tasks.workunit.client.0.smithi094.stdout:3/423: symlink d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d4f/l96 0 2024-03-24T10:45:57.162 INFO:tasks.workunit.client.0.smithi094.stdout:3/424: link d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/c39 d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/c97 0 2024-03-24T10:45:57.163 INFO:tasks.workunit.client.0.smithi094.stdout:3/425: truncate d0/db/dc/d28/d3c/f83 199443 0 2024-03-24T10:45:57.164 INFO:tasks.workunit.client.0.smithi094.stdout:3/426: mknod d0/db/dc/d28/d3c/d55/d7f/d20/d23/c98 0 2024-03-24T10:45:57.167 INFO:tasks.workunit.client.0.smithi094.stdout:3/427: creat d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/f99 x:0 0 0 2024-03-24T10:45:57.167 INFO:tasks.workunit.client.0.smithi094.stdout:3/428: truncate d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/f64 916699 0 2024-03-24T10:45:57.167 INFO:tasks.workunit.client.0.smithi094.stdout:6/520: dwrite d0/d21/f5c [0,4194304] 0 2024-03-24T10:45:57.167 INFO:tasks.workunit.client.0.smithi094.stdout:6/521: stat d0/d30/d50/d4d/d6b/d86/d9c 0 2024-03-24T10:45:57.167 INFO:tasks.workunit.client.0.smithi094.stdout:6/522: chown d0/d78/da/d23/d83/f85 56 1 2024-03-24T10:45:57.167 INFO:tasks.workunit.client.0.smithi094.stdout:3/429: creat d0/db/dc/d28/d3c/d55/d7f/d20/d23/f9a x:0 0 0 2024-03-24T10:45:57.167 INFO:tasks.workunit.client.0.smithi094.stdout:3/430: fdatasync d0/db/dc/f2d 0 2024-03-24T10:45:57.167 INFO:tasks.workunit.client.0.smithi094.stdout:4/546: dread d0/d5/d7d/d97/d4a/f61 [8388608,4194304] 0 2024-03-24T10:45:57.167 INFO:tasks.workunit.client.0.smithi094.stdout:9/525: write df/f52 [1076184,77696] 0 2024-03-24T10:45:57.169 INFO:tasks.workunit.client.0.smithi094.stdout:3/431: mkdir d0/db/dc/d28/d38/d6a/d9b 0 2024-03-24T10:45:57.169 INFO:tasks.workunit.client.0.smithi094.stdout:3/432: chown d0/db/dc/d28/d3c/d55/f70 5 1 2024-03-24T10:45:57.170 INFO:tasks.workunit.client.0.smithi094.stdout:9/526: rmdir df/d2d/d1f/d58/d5a/d61 39 2024-03-24T10:45:57.172 INFO:tasks.workunit.client.0.smithi094.stdout:9/527: truncate df/d2d/f8d 2746678 0 2024-03-24T10:45:57.174 INFO:tasks.workunit.client.0.smithi094.stdout:0/483: sync 2024-03-24T10:45:57.175 INFO:tasks.workunit.client.0.smithi094.stdout:5/527: sync 2024-03-24T10:45:57.175 INFO:tasks.workunit.client.0.smithi094.stdout:1/391: sync 2024-03-24T10:45:57.175 INFO:tasks.workunit.client.0.smithi094.stdout:2/421: sync 2024-03-24T10:45:57.175 INFO:tasks.workunit.client.0.smithi094.stdout:8/523: sync 2024-03-24T10:45:57.175 INFO:tasks.workunit.client.0.smithi094.stdout:7/502: sync 2024-03-24T10:45:57.175 INFO:tasks.workunit.client.0.smithi094.stdout:6/523: write d0/d78/da/d23/d83/f85 [2399815,84242] 0 2024-03-24T10:45:57.175 INFO:tasks.workunit.client.0.smithi094.stdout:6/524: creat d0/d30/fa0 x:0 0 0 2024-03-24T10:45:57.176 INFO:tasks.workunit.client.0.smithi094.stdout:6/525: fsync d0/d30/d50/d4d/d6b/f90 0 2024-03-24T10:45:57.178 INFO:tasks.workunit.client.0.smithi094.stdout:5/528: rename d7/d23/d5f/d14/d36/l76 to d7/d23/d5f/d14/d9f/la8 0 2024-03-24T10:45:57.178 INFO:tasks.workunit.client.0.smithi094.stdout:5/529: write d7/d23/d5f/f97 [924666,50754] 0 2024-03-24T10:45:57.182 INFO:tasks.workunit.client.0.smithi094.stdout:1/392: mkdir d4/d9/db/d85 0 2024-03-24T10:45:57.184 INFO:tasks.workunit.client.0.smithi094.stdout:2/422: mkdir d18/d1e/d5a/d61/d74/d7e 0 2024-03-24T10:45:57.185 INFO:tasks.workunit.client.0.smithi094.stdout:8/524: symlink df/d8c/d8f/lb4 0 2024-03-24T10:45:57.188 INFO:tasks.workunit.client.0.smithi094.stdout:7/503: creat d1/d9/d14/d34/fa3 x:0 0 0 2024-03-24T10:45:57.188 INFO:tasks.workunit.client.0.smithi094.stdout:7/504: readlink d1/d9/d16/l7b 0 2024-03-24T10:45:57.191 INFO:tasks.workunit.client.0.smithi094.stdout:0/484: mkdir d14/d22/d28/dac 0 2024-03-24T10:45:57.192 INFO:tasks.workunit.client.0.smithi094.stdout:0/485: chown d14/d22/d32/f80 270098016 1 2024-03-24T10:45:57.192 INFO:tasks.workunit.client.0.smithi094.stdout:0/486: fdatasync d14/d22/f33 0 2024-03-24T10:45:57.194 INFO:tasks.workunit.client.0.smithi094.stdout:1/393: creat d4/d9/db/d39/d75/f86 x:0 0 0 2024-03-24T10:45:57.196 INFO:tasks.workunit.client.0.smithi094.stdout:0/487: truncate d14/f1b 1854062 0 2024-03-24T10:45:57.197 INFO:tasks.workunit.client.0.smithi094.stdout:4/547: dwrite d0/d22/de/d28/f72 [0,4194304] 0 2024-03-24T10:45:57.198 INFO:tasks.workunit.client.0.smithi094.stdout:0/488: mknod d14/d22/d32/d65/cad 0 2024-03-24T10:45:57.198 INFO:tasks.workunit.client.0.smithi094.stdout:0/489: read d14/f2a [842773,96477] 0 2024-03-24T10:45:57.203 INFO:tasks.workunit.client.0.smithi094.stdout:4/548: symlink d0/d1f/l9f 0 2024-03-24T10:45:57.203 INFO:tasks.workunit.client.0.smithi094.stdout:4/549: readlink d0/d22/de/l78 0 2024-03-24T10:45:57.205 INFO:tasks.workunit.client.0.smithi094.stdout:4/550: truncate d0/d22/de/d28/f7f 3292076 0 2024-03-24T10:45:57.205 INFO:tasks.workunit.client.0.smithi094.stdout:4/551: readlink d0/d22/l15 0 2024-03-24T10:45:57.205 INFO:tasks.workunit.client.0.smithi094.stdout:4/552: write d0/d22/de/d28/d2b/d7a/f84 [371049,97757] 0 2024-03-24T10:45:57.215 INFO:tasks.workunit.client.0.smithi094.stdout:3/433: dwrite d0/db/dc/d28/d3c/d55/d7f/d20/d23/f29 [0,4194304] 0 2024-03-24T10:45:57.228 INFO:tasks.workunit.client.0.smithi094.stdout:9/528: dwrite df/d2d/d1f/d58/d5a/f6a [0,4194304] 0 2024-03-24T10:45:57.229 INFO:tasks.workunit.client.0.smithi094.stdout:9/529: truncate df/d2d/f15 1062163 0 2024-03-24T10:45:57.229 INFO:tasks.workunit.client.0.smithi094.stdout:9/530: truncate df/f62 615187 0 2024-03-24T10:45:57.230 INFO:tasks.workunit.client.0.smithi094.stdout:9/531: link df/d2d/d1f/d58/d5a/f63 df/d2d/d1f/d58/d82/fa0 0 2024-03-24T10:45:57.230 INFO:tasks.workunit.client.0.smithi094.stdout:9/532: chown df/d2d/d34/d2c/c3e 54 1 2024-03-24T10:45:57.231 INFO:tasks.workunit.client.0.smithi094.stdout:9/533: creat df/d2d/d1f/d11/d53/fa1 x:0 0 0 2024-03-24T10:45:57.231 INFO:tasks.workunit.client.0.smithi094.stdout:9/534: dread - df/d2d/d1f/d11/d53/f81 zero size 2024-03-24T10:45:57.231 INFO:tasks.workunit.client.0.smithi094.stdout:9/535: mknod df/d7b/ca2 0 2024-03-24T10:45:57.255 INFO:tasks.workunit.client.0.smithi094.stdout:8/525: dwrite df/d25/f30 [0,4194304] 0 2024-03-24T10:45:57.255 INFO:tasks.workunit.client.0.smithi094.stdout:6/526: dwrite d0/d78/da/d23/f75 [0,4194304] 0 2024-03-24T10:45:57.255 INFO:tasks.workunit.client.0.smithi094.stdout:5/530: dwrite d7/d23/d5f/d14/d36/f47 [0,4194304] 0 2024-03-24T10:45:57.255 INFO:tasks.workunit.client.0.smithi094.stdout:5/531: truncate d7/d23/d2b/f4b 1366753 0 2024-03-24T10:45:57.255 INFO:tasks.workunit.client.0.smithi094.stdout:2/423: dwrite d18/d1e/d65/f6a [0,4194304] 0 2024-03-24T10:45:57.255 INFO:tasks.workunit.client.0.smithi094.stdout:1/394: dwrite d4/d9/db/f1f [4194304,4194304] 0 2024-03-24T10:45:57.255 INFO:tasks.workunit.client.0.smithi094.stdout:0/490: dwrite d14/d18/f40 [0,4194304] 0 2024-03-24T10:45:57.256 INFO:tasks.workunit.client.0.smithi094.stdout:4/553: dwrite d0/f5e [0,4194304] 0 2024-03-24T10:45:57.257 INFO:tasks.workunit.client.0.smithi094.stdout:3/434: dwrite d0/db/dc/d28/d3c/d55/d7f/d20/d23/f3f [0,4194304] 0 2024-03-24T10:45:57.257 INFO:tasks.workunit.client.0.smithi094.stdout:3/435: fdatasync d0/f42 0 2024-03-24T10:45:57.268 INFO:tasks.workunit.client.0.smithi094.stdout:8/526: rename df/d8c/db3 to df/d25/d28/d31/d47/d66/da3/db5 0 2024-03-24T10:45:57.272 INFO:tasks.workunit.client.0.smithi094.stdout:6/527: symlink d0/d21/d2a/d55/la1 0 2024-03-24T10:45:57.279 INFO:tasks.workunit.client.0.smithi094.stdout:1/395: symlink d4/d9/d37/d33/d58/l87 0 2024-03-24T10:45:57.279 INFO:tasks.workunit.client.0.smithi094.stdout:1/396: getdents d4/d9/db/d85 0 2024-03-24T10:45:57.280 INFO:tasks.workunit.client.0.smithi094.stdout:4/554: mknod d0/d5/d1b/d4f/d70/d6d/ca0 0 2024-03-24T10:45:57.290 INFO:tasks.workunit.client.0.smithi094.stdout:1/397: creat d4/d9/d37/d33/d58/f88 x:0 0 0 2024-03-24T10:45:57.290 INFO:tasks.workunit.client.0.smithi094.stdout:1/398: creat d4/d36/f89 x:0 0 0 2024-03-24T10:45:57.290 INFO:tasks.workunit.client.0.smithi094.stdout:8/527: dread df/d25/d28/d31/d3c/f3e [0,4194304] 0 2024-03-24T10:45:57.290 INFO:tasks.workunit.client.0.smithi094.stdout:8/528: stat df/d25/d28/d31/d47/d22 0 2024-03-24T10:45:57.290 INFO:tasks.workunit.client.0.smithi094.stdout:8/529: fdatasync df/d25/f61 0 2024-03-24T10:45:57.290 INFO:tasks.workunit.client.0.smithi094.stdout:6/528: getdents d0 0 2024-03-24T10:45:57.290 INFO:tasks.workunit.client.0.smithi094.stdout:4/555: mkdir d0/d22/de/d28/d2b/d33/da1 0 2024-03-24T10:45:57.291 INFO:tasks.workunit.client.0.smithi094.stdout:8/530: symlink df/d25/d28/d31/lb6 0 2024-03-24T10:45:57.292 INFO:tasks.workunit.client.0.smithi094.stdout:6/529: symlink d0/d21/d2a/la2 0 2024-03-24T10:45:57.293 INFO:tasks.workunit.client.0.smithi094.stdout:8/531: creat df/d25/d28/d31/d47/d66/fb7 x:0 0 0 2024-03-24T10:45:57.293 INFO:tasks.workunit.client.0.smithi094.stdout:8/532: chown df/d25/d28/d31/d47 6 1 2024-03-24T10:45:57.294 INFO:tasks.workunit.client.0.smithi094.stdout:6/530: creat d0/d78/da/d77/fa3 x:0 0 0 2024-03-24T10:45:57.295 INFO:tasks.workunit.client.0.smithi094.stdout:4/556: read d0/d22/f9e [1049910,99617] 0 2024-03-24T10:45:57.295 INFO:tasks.workunit.client.0.smithi094.stdout:4/557: write d0/d5/d7d/d97/d4a/f61 [412950,19748] 0 2024-03-24T10:45:57.296 INFO:tasks.workunit.client.0.smithi094.stdout:6/531: dread d0/d3b/f69 [0,4194304] 0 2024-03-24T10:45:57.296 INFO:tasks.workunit.client.0.smithi094.stdout:6/532: write d0/d78/da/d23/f9b [917656,104227] 0 2024-03-24T10:45:57.296 INFO:tasks.workunit.client.0.smithi094.stdout:6/533: chown d0/d1a/c73 960 1 2024-03-24T10:45:57.304 INFO:tasks.workunit.client.0.smithi094.stdout:6/534: write d0/d21/f4f [3293441,36230] 0 2024-03-24T10:45:57.308 INFO:tasks.workunit.client.0.smithi094.stdout:1/399: write d4/d36/f70 [2176182,122049] 0 2024-03-24T10:45:57.370 INFO:tasks.workunit.client.0.smithi094.stdout:5/532: dwrite d7/d23/d5f/d14/f27 [0,4194304] 0 2024-03-24T10:45:57.370 INFO:tasks.workunit.client.0.smithi094.stdout:3/436: dwrite d0/db/f48 [0,4194304] 0 2024-03-24T10:45:57.372 INFO:tasks.workunit.client.0.smithi094.stdout:2/424: dwrite d18/d1e/d65/f72 [0,4194304] 0 2024-03-24T10:45:57.374 INFO:tasks.workunit.client.0.smithi094.stdout:5/533: mkdir d7/d26/d55/da9 0 2024-03-24T10:45:57.375 INFO:tasks.workunit.client.0.smithi094.stdout:3/437: creat d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d87/f9c x:0 0 0 2024-03-24T10:45:57.377 INFO:tasks.workunit.client.0.smithi094.stdout:2/425: getdents d18/d77 0 2024-03-24T10:45:57.377 INFO:tasks.workunit.client.0.smithi094.stdout:2/426: readlink d18/d1e/l2d 0 2024-03-24T10:45:57.379 INFO:tasks.workunit.client.0.smithi094.stdout:2/427: mkdir d18/d7f 0 2024-03-24T10:45:57.381 INFO:tasks.workunit.client.0.smithi094.stdout:2/428: symlink d18/d48/d56/l80 0 2024-03-24T10:45:57.381 INFO:tasks.workunit.client.0.smithi094.stdout:2/429: chown d18/d1e/d5a 743075 1 2024-03-24T10:45:57.381 INFO:tasks.workunit.client.0.smithi094.stdout:8/533: dwrite df/d25/d28/d31/d47/d22/d55/fa5 [0,4194304] 0 2024-03-24T10:45:57.381 INFO:tasks.workunit.client.0.smithi094.stdout:1/400: dwrite d4/d9/f77 [0,4194304] 0 2024-03-24T10:45:57.382 INFO:tasks.workunit.client.0.smithi094.stdout:6/535: write d0/d78/da/d23/f26 [778892,77578] 0 2024-03-24T10:45:57.382 INFO:tasks.workunit.client.0.smithi094.stdout:0/491: dwrite d14/d22/f2f [0,4194304] 0 2024-03-24T10:45:57.382 INFO:tasks.workunit.client.0.smithi094.stdout:0/492: chown d14/d22/d32/d65/fab 268605276 1 2024-03-24T10:45:57.383 INFO:tasks.workunit.client.0.smithi094.stdout:2/430: chown d18/d1e/d34/c42 40541 1 2024-03-24T10:45:57.386 INFO:tasks.workunit.client.0.smithi094.stdout:6/536: fsync d0/d78/da/d23/f26 0 2024-03-24T10:45:57.388 INFO:tasks.workunit.client.0.smithi094.stdout:1/401: symlink d4/d9/de/d6f/l8a 0 2024-03-24T10:45:57.388 INFO:tasks.workunit.client.0.smithi094.stdout:1/402: write d4/d9/d7d/f4f [1009976,33792] 0 2024-03-24T10:45:57.392 INFO:tasks.workunit.client.0.smithi094.stdout:0/493: rmdir d14/d22/d28 39 2024-03-24T10:45:57.393 INFO:tasks.workunit.client.0.smithi094.stdout:2/431: symlink d18/d1e/l81 0 2024-03-24T10:45:57.394 INFO:tasks.workunit.client.0.smithi094.stdout:6/537: creat d0/d30/d50/d4d/d6b/d86/d9c/fa4 x:0 0 0 2024-03-24T10:45:57.395 INFO:tasks.workunit.client.0.smithi094.stdout:1/403: symlink d4/d9/de/d6f/d62/l8b 0 2024-03-24T10:45:57.397 INFO:tasks.workunit.client.0.smithi094.stdout:0/494: rename d14/l39 to d14/d22/d5b/lae 0 2024-03-24T10:45:57.398 INFO:tasks.workunit.client.0.smithi094.stdout:2/432: symlink d18/d1e/d65/l82 0 2024-03-24T10:45:57.398 INFO:tasks.workunit.client.0.smithi094.stdout:2/433: dread - d18/d1e/f76 zero size 2024-03-24T10:45:57.398 INFO:tasks.workunit.client.0.smithi094.stdout:2/434: truncate d18/d1e/d5a/f5e 2511756 0 2024-03-24T10:45:57.398 INFO:tasks.workunit.client.0.smithi094.stdout:2/435: readlink d18/d1e/l81 0 2024-03-24T10:45:57.399 INFO:tasks.workunit.client.0.smithi094.stdout:3/438: dwrite d0/db/dc/d28/d3c/d55/d7f/d20/d23/f9a [0,4194304] 0 2024-03-24T10:45:57.400 INFO:tasks.workunit.client.0.smithi094.stdout:8/534: dwrite df/d25/d28/d31/f88 [0,4194304] 0 2024-03-24T10:45:57.401 INFO:tasks.workunit.client.0.smithi094.stdout:6/538: rename d0/d21/l84 to d0/d78/da/d77/la5 0 2024-03-24T10:45:57.401 INFO:tasks.workunit.client.0.smithi094.stdout:6/539: write d0/f3c [3771623,99250] 0 2024-03-24T10:45:57.401 INFO:tasks.workunit.client.0.smithi094.stdout:0/495: mknod d14/d18/d85/caf 0 2024-03-24T10:45:57.402 INFO:tasks.workunit.client.0.smithi094.stdout:2/436: mknod d18/d7f/c83 0 2024-03-24T10:45:57.403 INFO:tasks.workunit.client.0.smithi094.stdout:1/404: mknod d4/d9/db/d31/d3c/c8c 0 2024-03-24T10:45:57.406 INFO:tasks.workunit.client.0.smithi094.stdout:3/439: creat d0/db/dc/d28/d3c/d7c/f9d x:0 0 0 2024-03-24T10:45:57.409 INFO:tasks.workunit.client.0.smithi094.stdout:7/505: sync 2024-03-24T10:45:57.409 INFO:tasks.workunit.client.0.smithi094.stdout:9/536: sync 2024-03-24T10:45:57.409 INFO:tasks.workunit.client.0.smithi094.stdout:7/506: readlink d1/d9/d16/d21/d51/d83/l54 0 2024-03-24T10:45:57.410 INFO:tasks.workunit.client.0.smithi094.stdout:1/405: read d4/f1e [2866873,52771] 0 2024-03-24T10:45:57.410 INFO:tasks.workunit.client.0.smithi094.stdout:6/540: creat d0/d21/d2a/d2f/fa6 x:0 0 0 2024-03-24T10:45:57.411 INFO:tasks.workunit.client.0.smithi094.stdout:6/541: dread d0/d30/f76 [0,4194304] 0 2024-03-24T10:45:57.412 INFO:tasks.workunit.client.0.smithi094.stdout:2/437: creat d18/d1e/d34/f84 x:0 0 0 2024-03-24T10:45:57.412 INFO:tasks.workunit.client.0.smithi094.stdout:2/438: dread - d18/d1e/d34/f46 zero size 2024-03-24T10:45:57.413 INFO:tasks.workunit.client.0.smithi094.stdout:2/439: dread d18/d1e/d34/f6f [0,4194304] 0 2024-03-24T10:45:57.413 INFO:tasks.workunit.client.0.smithi094.stdout:3/440: rename d0/db/c32 to d0/db/dc/d28/d3c/d7c/c9e 0 2024-03-24T10:45:57.414 INFO:tasks.workunit.client.0.smithi094.stdout:9/537: creat df/d2d/d1f/d11/d66/d80/fa3 x:0 0 0 2024-03-24T10:45:57.414 INFO:tasks.workunit.client.0.smithi094.stdout:9/538: fdatasync df/d2d/d34/d2c/f73 0 2024-03-24T10:45:57.414 INFO:tasks.workunit.client.0.smithi094.stdout:9/539: chown df/d2d/d1f/d11/d66/d80 22261 1 2024-03-24T10:45:57.415 INFO:tasks.workunit.client.0.smithi094.stdout:1/406: mkdir d4/d9/de/d6f/d5f/d8d 0 2024-03-24T10:45:57.415 INFO:tasks.workunit.client.0.smithi094.stdout:1/407: dread - d4/d9/de/d6f/d5f/f68 zero size 2024-03-24T10:45:57.419 INFO:tasks.workunit.client.0.smithi094.stdout:6/542: unlink d0/d1a/f39 0 2024-03-24T10:45:57.421 INFO:tasks.workunit.client.0.smithi094.stdout:2/440: symlink d18/d1e/d5a/l85 0 2024-03-24T10:45:57.421 INFO:tasks.workunit.client.0.smithi094.stdout:8/535: dwrite df/d12/d5b/d63/d75/f7b [0,4194304] 0 2024-03-24T10:45:57.422 INFO:tasks.workunit.client.0.smithi094.stdout:3/441: creat d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d4f/d74/f9f x:0 0 0 2024-03-24T10:45:57.424 INFO:tasks.workunit.client.0.smithi094.stdout:9/540: rename df/d2d/d34/f30 to df/d2d/d1f/fa4 0 2024-03-24T10:45:57.424 INFO:tasks.workunit.client.0.smithi094.stdout:9/541: write df/f62 [640823,86892] 0 2024-03-24T10:45:57.424 INFO:tasks.workunit.client.0.smithi094.stdout:7/507: write d1/d9/d25/d3d/f98 [3546311,5300] 0 2024-03-24T10:45:57.424 INFO:tasks.workunit.client.0.smithi094.stdout:6/543: truncate d0/d78/da/d23/f68 3667645 0 2024-03-24T10:45:57.426 INFO:tasks.workunit.client.0.smithi094.stdout:0/496: dwrite d14/d22/d28/d48/d7b/f94 [0,4194304] 0 2024-03-24T10:45:57.428 INFO:tasks.workunit.client.0.smithi094.stdout:2/441: rmdir d18/d1e/d34/d7d 0 2024-03-24T10:45:57.428 INFO:tasks.workunit.client.0.smithi094.stdout:2/442: truncate d18/f1b 4905379 0 2024-03-24T10:45:57.430 INFO:tasks.workunit.client.0.smithi094.stdout:8/536: symlink df/d12/d5b/d63/d75/d96/lb8 0 2024-03-24T10:45:57.431 INFO:tasks.workunit.client.0.smithi094.stdout:8/537: dread - df/d25/d28/d31/d47/fa2 zero size 2024-03-24T10:45:57.431 INFO:tasks.workunit.client.0.smithi094.stdout:8/538: dread - df/d25/d28/d31/d47/d22/f70 zero size 2024-03-24T10:45:57.431 INFO:tasks.workunit.client.0.smithi094.stdout:8/539: write df/d25/d28/d31/d47/d66/da3/fa4 [722045,86709] 0 2024-03-24T10:45:57.431 INFO:tasks.workunit.client.0.smithi094.stdout:9/542: mkdir df/d2d/d1f/d58/d5a/d50/da5 0 2024-03-24T10:45:57.432 INFO:tasks.workunit.client.0.smithi094.stdout:2/443: dread d18/d1e/f33 [0,4194304] 0 2024-03-24T10:45:57.434 INFO:tasks.workunit.client.0.smithi094.stdout:7/508: link d1/d6c/d80/f69 d1/d9/d16/d21/d49/fa4 0 2024-03-24T10:45:57.436 INFO:tasks.workunit.client.0.smithi094.stdout:1/408: getdents d4/d9/db/d31/d3c 0 2024-03-24T10:45:57.436 INFO:tasks.workunit.client.0.smithi094.stdout:1/409: stat d4/d9/de/d6f 0 2024-03-24T10:45:57.436 INFO:tasks.workunit.client.0.smithi094.stdout:1/410: write d4/d9/db/f67 [308935,77984] 0 2024-03-24T10:45:57.436 INFO:tasks.workunit.client.0.smithi094.stdout:1/411: chown d4/d9/d37/d33/f5b 240046758 1 2024-03-24T10:45:57.436 INFO:tasks.workunit.client.0.smithi094.stdout:1/412: creat d4/d36/f8e x:0 0 0 2024-03-24T10:45:57.437 INFO:tasks.workunit.client.0.smithi094.stdout:6/544: creat d0/d78/da/d23/d34/d4e/fa7 x:0 0 0 2024-03-24T10:45:57.444 INFO:tasks.workunit.client.0.smithi094.stdout:8/540: link df/d25/d28/d31/d3c/f84 df/d8c/d8f/fb9 0 2024-03-24T10:45:57.444 INFO:tasks.workunit.client.0.smithi094.stdout:8/541: write df/d25/d28/d31/d3c/f5f [743944,10822] 0 2024-03-24T10:45:57.445 INFO:tasks.workunit.client.0.smithi094.stdout:2/444: write d18/f23 [2210024,80282] 0 2024-03-24T10:45:57.448 INFO:tasks.workunit.client.0.smithi094.stdout:1/413: creat d4/d9/de/d6f/d62/d80/f8f x:0 0 0 2024-03-24T10:45:57.448 INFO:tasks.workunit.client.0.smithi094.stdout:1/414: dread - d4/d36/f8e zero size 2024-03-24T10:45:57.451 INFO:tasks.workunit.client.0.smithi094.stdout:2/445: symlink d18/d5f/l86 0 2024-03-24T10:45:57.452 INFO:tasks.workunit.client.0.smithi094.stdout:2/446: truncate f7 3843745 0 2024-03-24T10:45:57.452 INFO:tasks.workunit.client.0.smithi094.stdout:2/447: chown d18/d54 3470 1 2024-03-24T10:45:57.455 INFO:tasks.workunit.client.0.smithi094.stdout:9/543: truncate df/d2d/d1f/d58/d5a/f4b 836060 0 2024-03-24T10:45:57.456 INFO:tasks.workunit.client.0.smithi094.stdout:1/415: creat d4/d9/de/d3d/f90 x:0 0 0 2024-03-24T10:45:57.456 INFO:tasks.workunit.client.0.smithi094.stdout:1/416: write d4/d9/db/d39/d75/f86 [58063,44530] 0 2024-03-24T10:45:57.456 INFO:tasks.workunit.client.0.smithi094.stdout:3/442: dwrite d0/db/dc/d28/d3c/d55/d7f/d20/d23/f9a [0,4194304] 0 2024-03-24T10:45:57.456 INFO:tasks.workunit.client.0.smithi094.stdout:0/497: read d14/d22/d28/d2b/d41/d53/f5a [2691944,55428] 0 2024-03-24T10:45:57.457 INFO:tasks.workunit.client.0.smithi094.stdout:9/544: creat df/d2d/d1f/d11/fa6 x:0 0 0 2024-03-24T10:45:57.459 INFO:tasks.workunit.client.0.smithi094.stdout:1/417: creat d4/d9/db/d31/f91 x:0 0 0 2024-03-24T10:45:57.459 INFO:tasks.workunit.client.0.smithi094.stdout:1/418: creat d4/d9/de/d3d/f92 x:0 0 0 2024-03-24T10:45:57.459 INFO:tasks.workunit.client.0.smithi094.stdout:1/419: stat d4/f7 0 2024-03-24T10:45:57.460 INFO:tasks.workunit.client.0.smithi094.stdout:3/443: truncate d0/db/dc/d28/d3c/d55/d7f/d20/f24 3132722 0 2024-03-24T10:45:57.460 INFO:tasks.workunit.client.0.smithi094.stdout:7/509: dwrite d1/d9/f3c [0,4194304] 0 2024-03-24T10:45:57.461 INFO:tasks.workunit.client.0.smithi094.stdout:0/498: rename d14/d18/f2e to d14/d18/fb0 0 2024-03-24T10:45:57.461 INFO:tasks.workunit.client.0.smithi094.stdout:9/545: symlink df/d2d/d1f/d11/d53/d88/la7 0 2024-03-24T10:45:57.467 INFO:tasks.workunit.client.0.smithi094.stdout:1/420: rmdir d4/d9/db/d5d 39 2024-03-24T10:45:57.468 INFO:tasks.workunit.client.0.smithi094.stdout:3/444: getdents d0/db/dc/d28/d3c/d55 0 2024-03-24T10:45:57.470 INFO:tasks.workunit.client.0.smithi094.stdout:1/421: unlink d4/d9/d37/c83 0 2024-03-24T10:45:57.471 INFO:tasks.workunit.client.0.smithi094.stdout:2/448: dread d18/f27 [0,4194304] 0 2024-03-24T10:45:57.471 INFO:tasks.workunit.client.0.smithi094.stdout:3/445: creat d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d34/fa0 x:0 0 0 2024-03-24T10:45:57.472 INFO:tasks.workunit.client.0.smithi094.stdout:3/446: read d0/f42 [913718,22348] 0 2024-03-24T10:45:57.475 INFO:tasks.workunit.client.0.smithi094.stdout:3/447: creat d0/db/dc/d28/d38/d6a/d9b/fa1 x:0 0 0 2024-03-24T10:45:57.476 INFO:tasks.workunit.client.0.smithi094.stdout:8/542: dwrite df/d25/f58 [0,4194304] 0 2024-03-24T10:45:57.476 INFO:tasks.workunit.client.0.smithi094.stdout:6/545: dwrite d0/d1a/f6f [0,4194304] 0 2024-03-24T10:45:57.479 INFO:tasks.workunit.client.0.smithi094.stdout:3/448: link d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/f99 d0/db/dc/d28/d3c/d55/d7f/d20/d23/fa2 0 2024-03-24T10:45:57.488 INFO:tasks.workunit.client.0.smithi094.stdout:8/543: unlink df/d25/d28/d31/d47/d22/l78 0 2024-03-24T10:45:57.495 INFO:tasks.workunit.client.0.smithi094.stdout:8/544: mknod df/d25/cba 0 2024-03-24T10:45:57.498 INFO:tasks.workunit.client.0.smithi094.stdout:8/545: dread df/d25/d28/d31/d47/f77 [0,4194304] 0 2024-03-24T10:45:57.499 INFO:tasks.workunit.client.0.smithi094.stdout:8/546: symlink df/d25/d28/d31/d95/lbb 0 2024-03-24T10:45:57.501 INFO:tasks.workunit.client.0.smithi094.stdout:8/547: link df/d8c/f9e df/d12/d5b/d63/fbc 0 2024-03-24T10:45:57.501 INFO:tasks.workunit.client.0.smithi094.stdout:8/548: write df/d12/d5b/d63/fbc [650327,105535] 0 2024-03-24T10:45:57.501 INFO:tasks.workunit.client.0.smithi094.stdout:8/549: fdatasync f4 0 2024-03-24T10:45:57.502 INFO:tasks.workunit.client.0.smithi094.stdout:9/546: dwrite df/d2d/d1f/d11/f90 [0,4194304] 0 2024-03-24T10:45:57.502 INFO:tasks.workunit.client.0.smithi094.stdout:9/547: chown df/d2d/d1f/d58/d5a/d61/d87 342517 1 2024-03-24T10:45:57.502 INFO:tasks.workunit.client.0.smithi094.stdout:9/548: read - df/d2d/d1f/d7e/f91 zero size 2024-03-24T10:45:57.503 INFO:tasks.workunit.client.0.smithi094.stdout:2/449: dwrite d18/d1e/d34/f46 [0,4194304] 0 2024-03-24T10:45:57.503 INFO:tasks.workunit.client.0.smithi094.stdout:1/422: dwrite d4/d9/db/d5d/d63/f6a [0,4194304] 0 2024-03-24T10:45:57.503 INFO:tasks.workunit.client.0.smithi094.stdout:1/423: creat d4/d9/d37/d33/d58/f93 x:0 0 0 2024-03-24T10:45:57.503 INFO:tasks.workunit.client.0.smithi094.stdout:7/510: dwrite d1/d9/d16/d21/d51/d83/f72 [0,4194304] 0 2024-03-24T10:45:57.503 INFO:tasks.workunit.client.0.smithi094.stdout:7/511: write d1/d9/d14/d34/fa3 [740500,46958] 0 2024-03-24T10:45:57.503 INFO:tasks.workunit.client.0.smithi094.stdout:7/512: readlink d1/d9/d14/l6a 0 2024-03-24T10:45:57.504 INFO:tasks.workunit.client.0.smithi094.stdout:3/449: dwrite d0/db/dc/d28/f88 [0,4194304] 0 2024-03-24T10:45:57.504 INFO:tasks.workunit.client.0.smithi094.stdout:3/450: chown d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a 58 1 2024-03-24T10:45:57.504 INFO:tasks.workunit.client.0.smithi094.stdout:9/549: dread df/d2d/d1f/d11/f4f [0,4194304] 0 2024-03-24T10:45:57.505 INFO:tasks.workunit.client.0.smithi094.stdout:2/450: dread d18/d1e/d5a/f5e [0,4194304] 0 2024-03-24T10:45:57.507 INFO:tasks.workunit.client.0.smithi094.stdout:2/451: write d18/d1e/d5a/f5e [2437093,113777] 0 2024-03-24T10:45:57.507 INFO:tasks.workunit.client.0.smithi094.stdout:2/452: fsync d18/d1e/d34/d4a/f6d 0 2024-03-24T10:45:57.507 INFO:tasks.workunit.client.0.smithi094.stdout:2/453: write d18/d77/f78 [891608,121966] 0 2024-03-24T10:45:57.511 INFO:tasks.workunit.client.0.smithi094.stdout:8/550: write fd [2465106,104772] 0 2024-03-24T10:45:57.512 INFO:tasks.workunit.client.0.smithi094.stdout:1/424: mknod d4/d9/d7d/c94 0 2024-03-24T10:45:57.512 INFO:tasks.workunit.client.0.smithi094.stdout:1/425: creat d4/d9/db/f95 x:0 0 0 2024-03-24T10:45:57.512 INFO:tasks.workunit.client.0.smithi094.stdout:1/426: fdatasync d4/d9/de/d6f/d5f/f68 0 2024-03-24T10:45:57.513 INFO:tasks.workunit.client.0.smithi094.stdout:7/513: unlink d1/d9/d35/c99 0 2024-03-24T10:45:57.513 INFO:tasks.workunit.client.0.smithi094.stdout:7/514: dread - d1/d9/d16/d21/d51/d83/d39/f86 zero size 2024-03-24T10:45:57.513 INFO:tasks.workunit.client.0.smithi094.stdout:7/515: dread - d1/d9/d16/d21/d49/fa4 zero size 2024-03-24T10:45:57.516 INFO:tasks.workunit.client.0.smithi094.stdout:3/451: creat d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d86/fa3 x:0 0 0 2024-03-24T10:45:57.516 INFO:tasks.workunit.client.0.smithi094.stdout:3/452: fdatasync d0/db/dc/d28/d3c/d55/f70 0 2024-03-24T10:45:57.516 INFO:tasks.workunit.client.0.smithi094.stdout:3/453: write d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d4f/d74/f9f [448400,36686] 0 2024-03-24T10:45:57.517 INFO:tasks.workunit.client.0.smithi094.stdout:9/550: symlink df/d2d/d34/d2c/d74/la8 0 2024-03-24T10:45:57.518 INFO:tasks.workunit.client.0.smithi094.stdout:2/454: creat d18/d48/d56/f87 x:0 0 0 2024-03-24T10:45:57.518 INFO:tasks.workunit.client.0.smithi094.stdout:2/455: chown d18/d1e/d34/f35 10832 1 2024-03-24T10:45:57.519 INFO:tasks.workunit.client.0.smithi094.stdout:8/551: truncate df/d25/d28/d31/f79 622822 0 2024-03-24T10:45:57.521 INFO:tasks.workunit.client.0.smithi094.stdout:4/558: sync 2024-03-24T10:45:57.521 INFO:tasks.workunit.client.0.smithi094.stdout:4/559: chown d0/c18 750015702 1 2024-03-24T10:45:57.524 INFO:tasks.workunit.client.0.smithi094.stdout:1/427: symlink d4/d9/db/d31/d3c/l96 0 2024-03-24T10:45:57.524 INFO:tasks.workunit.client.0.smithi094.stdout:7/516: mkdir d1/d9/d14/da5 0 2024-03-24T10:45:57.526 INFO:tasks.workunit.client.0.smithi094.stdout:4/560: mkdir d0/d22/de/d28/d2b/da2 0 2024-03-24T10:45:57.526 INFO:tasks.workunit.client.0.smithi094.stdout:4/561: fdatasync d0/d5/d7d/d97/f67 0 2024-03-24T10:45:57.527 INFO:tasks.workunit.client.0.smithi094.stdout:4/562: creat d0/d34/d73/d93/fa3 x:0 0 0 2024-03-24T10:45:57.530 INFO:tasks.workunit.client.0.smithi094.stdout:9/551: dread df/d2d/f15 [0,4194304] 0 2024-03-24T10:45:57.535 INFO:tasks.workunit.client.0.smithi094.stdout:8/552: dread df/d12/d5b/d63/d75/f7b [0,4194304] 0 2024-03-24T10:45:57.537 INFO:tasks.workunit.client.0.smithi094.stdout:8/553: mkdir df/d25/d28/d31/d47/dbd 0 2024-03-24T10:45:57.537 INFO:tasks.workunit.client.0.smithi094.stdout:8/554: chown df/d25/d28/d31/d95/lbb 126995 1 2024-03-24T10:45:57.540 INFO:tasks.workunit.client.0.smithi094.stdout:2/456: dread d18/f47 [0,4194304] 0 2024-03-24T10:45:57.540 INFO:tasks.workunit.client.0.smithi094.stdout:9/552: read df/d2d/f1e [8676640,25487] 0 2024-03-24T10:45:57.541 INFO:tasks.workunit.client.0.smithi094.stdout:2/457: write d18/f47 [3681188,83435] 0 2024-03-24T10:45:57.544 INFO:tasks.workunit.client.0.smithi094.stdout:4/563: write d0/d5/d7d/f88 [3445905,115889] 0 2024-03-24T10:45:57.544 INFO:tasks.workunit.client.0.smithi094.stdout:4/564: getdents d0/d22/de/d28/d2b/da2 0 2024-03-24T10:45:57.544 INFO:tasks.workunit.client.0.smithi094.stdout:3/454: dwrite d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/f5b [0,4194304] 0 2024-03-24T10:45:57.545 INFO:tasks.workunit.client.0.smithi094.stdout:3/455: chown d0/db/dc/c22 9 1 2024-03-24T10:45:57.545 INFO:tasks.workunit.client.0.smithi094.stdout:3/456: readlink d0/db/dc/d28/d50/l51 0 2024-03-24T10:45:57.545 INFO:tasks.workunit.client.0.smithi094.stdout:3/457: readlink d0/db/dc/d28/d50/l51 0 2024-03-24T10:45:57.545 INFO:tasks.workunit.client.0.smithi094.stdout:3/458: write d0/db/dc/f12 [2899546,44960] 0 2024-03-24T10:45:57.545 INFO:tasks.workunit.client.0.smithi094.stdout:9/553: mknod df/d2d/d1f/ca9 0 2024-03-24T10:45:57.547 INFO:tasks.workunit.client.0.smithi094.stdout:2/458: mkdir d18/d7f/d88 0 2024-03-24T10:45:57.547 INFO:tasks.workunit.client.0.smithi094.stdout:7/517: dwrite d1/d9/f3c [0,4194304] 0 2024-03-24T10:45:57.547 INFO:tasks.workunit.client.0.smithi094.stdout:7/518: chown d1/d9/f2c 6800541 1 2024-03-24T10:45:57.548 INFO:tasks.workunit.client.0.smithi094.stdout:7/519: truncate d1/d9/d35/f9e 32711 0 2024-03-24T10:45:57.550 INFO:tasks.workunit.client.0.smithi094.stdout:4/565: rename d0/d5/d1b/d4f/d70/d6d to d0/d22/de/d28/d2b/d33/da4 0 2024-03-24T10:45:57.551 INFO:tasks.workunit.client.0.smithi094.stdout:4/566: write d0/d5/d7d/f99 [801170,40262] 0 2024-03-24T10:45:57.552 INFO:tasks.workunit.client.0.smithi094.stdout:8/555: dread df/d25/d36/d3f/f59 [0,4194304] 0 2024-03-24T10:45:57.554 INFO:tasks.workunit.client.0.smithi094.stdout:3/459: link d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/c49 d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/d73/ca4 0 2024-03-24T10:45:57.556 INFO:tasks.workunit.client.0.smithi094.stdout:9/554: mknod df/caa 0 2024-03-24T10:45:57.557 INFO:tasks.workunit.client.0.smithi094.stdout:1/428: getdents d4/d9/db/d31/d3c 0 2024-03-24T10:45:57.557 INFO:tasks.workunit.client.0.smithi094.stdout:1/429: chown d4/d9/db/d31 3248 1 2024-03-24T10:45:57.559 INFO:tasks.workunit.client.0.smithi094.stdout:2/459: rmdir d18/d77 39 2024-03-24T10:45:57.560 INFO:tasks.workunit.client.0.smithi094.stdout:7/520: creat d1/d9/d14/d8f/fa6 x:0 0 0 2024-03-24T10:45:57.561 INFO:tasks.workunit.client.0.smithi094.stdout:4/567: mkdir d0/d22/de/d28/d2b/d33/da4/da5 0 2024-03-24T10:45:57.563 INFO:tasks.workunit.client.0.smithi094.stdout:8/556: symlink df/d12/d5b/d63/d75/lbe 0 2024-03-24T10:45:57.563 INFO:tasks.workunit.client.0.smithi094.stdout:8/557: write df/d25/d28/d31/d47/f7e [786547,118545] 0 2024-03-24T10:45:57.563 INFO:tasks.workunit.client.0.smithi094.stdout:8/558: write df/d8c/d8f/fb9 [1317893,35636] 0 2024-03-24T10:45:57.563 INFO:tasks.workunit.client.0.smithi094.stdout:8/559: fdatasync df/d25/daf/fb0 0 2024-03-24T10:45:57.565 INFO:tasks.workunit.client.0.smithi094.stdout:1/430: truncate d4/d9/db/f10 1554084 0 2024-03-24T10:45:57.566 INFO:tasks.workunit.client.0.smithi094.stdout:2/460: symlink d18/d1e/l89 0 2024-03-24T10:45:57.566 INFO:tasks.workunit.client.0.smithi094.stdout:7/521: rmdir d1 39 2024-03-24T10:45:57.568 INFO:tasks.workunit.client.0.smithi094.stdout:2/461: dread d18/d48/f52 [0,4194304] 0 2024-03-24T10:45:57.568 INFO:tasks.workunit.client.0.smithi094.stdout:8/560: mkdir df/d25/d28/d31/d47/d66/da3/db5/dbf 0 2024-03-24T10:45:57.569 INFO:tasks.workunit.client.0.smithi094.stdout:8/561: truncate df/d25/f97 674480 0 2024-03-24T10:45:57.570 INFO:tasks.workunit.client.0.smithi094.stdout:9/555: rename df/d2d/d1f/d11/d66 to df/d2d/d1f/d58/d70/dab 0 2024-03-24T10:45:57.571 INFO:tasks.workunit.client.0.smithi094.stdout:9/556: fdatasync df/d2d/d34/d2c/f40 0 2024-03-24T10:45:57.572 INFO:tasks.workunit.client.0.smithi094.stdout:2/462: mkdir d18/d7f/d88/d8a 0 2024-03-24T10:45:57.572 INFO:tasks.workunit.client.0.smithi094.stdout:8/562: mknod df/d25/d28/cc0 0 2024-03-24T10:45:57.573 INFO:tasks.workunit.client.0.smithi094.stdout:8/563: readlink df/d8c/d8f/lb4 0 2024-03-24T10:45:57.573 INFO:tasks.workunit.client.0.smithi094.stdout:9/557: getdents df/d7b 0 2024-03-24T10:45:57.576 INFO:tasks.workunit.client.0.smithi094.stdout:2/463: symlink d18/d1e/d65/l8b 0 2024-03-24T10:45:57.577 INFO:tasks.workunit.client.0.smithi094.stdout:5/534: sync 2024-03-24T10:45:57.577 INFO:tasks.workunit.client.0.smithi094.stdout:8/564: getdents df/d12/d5b/d63/d75/d96 0 2024-03-24T10:45:57.577 INFO:tasks.workunit.client.0.smithi094.stdout:8/565: dread - df/d12/f6a zero size 2024-03-24T10:45:57.579 INFO:tasks.workunit.client.0.smithi094.stdout:9/558: symlink df/d2d/d1f/d58/d5a/d69/lac 0 2024-03-24T10:45:57.580 INFO:tasks.workunit.client.0.smithi094.stdout:2/464: mknod d18/d48/c8c 0 2024-03-24T10:45:57.582 INFO:tasks.workunit.client.0.smithi094.stdout:8/566: rename df/d25/d36/db2 to df/d8c/d8f/dc1 0 2024-03-24T10:45:57.583 INFO:tasks.workunit.client.0.smithi094.stdout:9/559: symlink df/d2d/d1f/d58/d5a/d61/d87/lad 0 2024-03-24T10:45:57.586 INFO:tasks.workunit.client.0.smithi094.stdout:2/465: creat d18/d7f/f8d x:0 0 0 2024-03-24T10:45:57.586 INFO:tasks.workunit.client.0.smithi094.stdout:4/568: dwrite d0/d34/d73/d93/f96 [0,4194304] 0 2024-03-24T10:45:57.587 INFO:tasks.workunit.client.0.smithi094.stdout:1/431: dwrite d4/d9/d37/f4c [0,4194304] 0 2024-03-24T10:45:57.588 INFO:tasks.workunit.client.0.smithi094.stdout:8/567: rmdir df/d25/d28/d31/d47/d66/da3 39 2024-03-24T10:45:57.588 INFO:tasks.workunit.client.0.smithi094.stdout:8/568: chown df/d25/l6c 16331 1 2024-03-24T10:45:57.588 INFO:tasks.workunit.client.0.smithi094.stdout:8/569: chown df/d12/l53 215476 1 2024-03-24T10:45:57.590 INFO:tasks.workunit.client.0.smithi094.stdout:2/466: link d18/d1e/l2d d18/d48/d56/l8e 0 2024-03-24T10:45:57.595 INFO:tasks.workunit.client.0.smithi094.stdout:7/522: getdents d1/d9/d14/d8f 0 2024-03-24T10:45:57.595 INFO:tasks.workunit.client.0.smithi094.stdout:7/523: stat d1/d9/d96/f9a 0 2024-03-24T10:45:57.596 INFO:tasks.workunit.client.0.smithi094.stdout:5/535: dwrite d7/dc/d12/f1f [0,4194304] 0 2024-03-24T10:45:57.596 INFO:tasks.workunit.client.0.smithi094.stdout:5/536: chown d7/c6c 643674 1 2024-03-24T10:45:57.596 INFO:tasks.workunit.client.0.smithi094.stdout:5/537: dread - d7/dc/d56/f93 zero size 2024-03-24T10:45:57.596 INFO:tasks.workunit.client.0.smithi094.stdout:5/538: creat d7/d26/faa x:0 0 0 2024-03-24T10:45:57.602 INFO:tasks.workunit.client.0.smithi094.stdout:2/467: rename d18/d48/d56/f87 to d18/d48/d56/d73/f8f 0 2024-03-24T10:45:57.602 INFO:tasks.workunit.client.0.smithi094.stdout:2/468: fsync d18/d1e/d34/f84 0 2024-03-24T10:45:57.602 INFO:tasks.workunit.client.0.smithi094.stdout:2/469: write d18/d48/d56/d73/f7a [527912,77717] 0 2024-03-24T10:45:57.604 INFO:tasks.workunit.client.0.smithi094.stdout:2/470: write d18/f3f [151305,82990] 0 2024-03-24T10:45:57.604 INFO:tasks.workunit.client.0.smithi094.stdout:7/524: link d1/d6c/c70 d1/d9/d25/d3d/ca7 0 2024-03-24T10:45:57.604 INFO:tasks.workunit.client.0.smithi094.stdout:7/525: chown d1/d6c/d80/d81 1 1 2024-03-24T10:45:57.604 INFO:tasks.workunit.client.0.smithi094.stdout:7/526: fsync d1/d9/f12 0 2024-03-24T10:45:57.606 INFO:tasks.workunit.client.0.smithi094.stdout:5/539: creat d7/d26/d55/da9/fab x:0 0 0 2024-03-24T10:45:57.607 INFO:tasks.workunit.client.0.smithi094.stdout:5/540: truncate d7/d23/d2b/f4b 1555803 0 2024-03-24T10:45:57.608 INFO:tasks.workunit.client.0.smithi094.stdout:2/471: mkdir d18/d66/d90 0 2024-03-24T10:45:57.610 INFO:tasks.workunit.client.0.smithi094.stdout:7/527: link d1/c11 d1/d9b/ca8 0 2024-03-24T10:45:57.610 INFO:tasks.workunit.client.0.smithi094.stdout:7/528: truncate d1/d9/d35/d65/f89 136054 0 2024-03-24T10:45:57.610 INFO:tasks.workunit.client.0.smithi094.stdout:8/570: dwrite df/d25/d28/d31/d3c/f3e [4194304,4194304] 0 2024-03-24T10:45:57.610 INFO:tasks.workunit.client.0.smithi094.stdout:4/569: dwrite d0/d5/d1b/f2a [0,4194304] 0 2024-03-24T10:45:57.611 INFO:tasks.workunit.client.0.smithi094.stdout:5/541: mknod d7/cac 0 2024-03-24T10:45:57.611 INFO:tasks.workunit.client.0.smithi094.stdout:5/542: chown d7/dc/d12/f39 800698 1 2024-03-24T10:45:57.614 INFO:tasks.workunit.client.0.smithi094.stdout:7/529: mkdir d1/d9/d16/d21/d49/da9 0 2024-03-24T10:45:57.617 INFO:tasks.workunit.client.0.smithi094.stdout:4/570: mknod d0/d5/d7d/ca6 0 2024-03-24T10:45:57.618 INFO:tasks.workunit.client.0.smithi094.stdout:9/560: rmdir df/d2d/d1f/d11 39 2024-03-24T10:45:57.618 INFO:tasks.workunit.client.0.smithi094.stdout:9/561: write df/f36 [2015122,110309] 0 2024-03-24T10:45:57.621 INFO:tasks.workunit.client.0.smithi094.stdout:5/543: symlink d7/dc/d56/lad 0 2024-03-24T10:45:57.623 INFO:tasks.workunit.client.0.smithi094.stdout:5/544: dread d7/d23/d5f/d14/d6f/f91 [0,4194304] 0 2024-03-24T10:45:57.624 INFO:tasks.workunit.client.0.smithi094.stdout:7/530: link d1/d9/f23 d1/d9/d25/d3d/faa 0 2024-03-24T10:45:57.626 INFO:tasks.workunit.client.0.smithi094.stdout:9/562: creat df/d2d/fae x:0 0 0 2024-03-24T10:45:57.628 INFO:tasks.workunit.client.0.smithi094.stdout:5/545: creat d7/d26/d55/fae x:0 0 0 2024-03-24T10:45:57.628 INFO:tasks.workunit.client.0.smithi094.stdout:9/563: stat df/c92 0 2024-03-24T10:45:57.635 INFO:tasks.workunit.client.0.smithi094.stdout:5/546: creat d7/faf x:0 0 0 2024-03-24T10:45:57.635 INFO:tasks.workunit.client.0.smithi094.stdout:9/564: truncate df/d2d/d1f/f4a 1801462 0 2024-03-24T10:45:57.636 INFO:tasks.workunit.client.0.smithi094.stdout:5/547: creat d7/d23/d5f/d14/d36/fb0 x:0 0 0 2024-03-24T10:45:57.637 INFO:tasks.workunit.client.0.smithi094.stdout:9/565: mknod df/d2d/caf 0 2024-03-24T10:45:57.638 INFO:tasks.workunit.client.0.smithi094.stdout:5/548: symlink d7/dc/lb1 0 2024-03-24T10:45:57.639 INFO:tasks.workunit.client.0.smithi094.stdout:9/566: truncate df/f3f 5791355 0 2024-03-24T10:45:57.639 INFO:tasks.workunit.client.0.smithi094.stdout:9/567: readlink df/d2d/d1f/d58/d70/dab/d80/l8e 0 2024-03-24T10:45:57.641 INFO:tasks.workunit.client.0.smithi094.stdout:2/472: dwrite d18/d1e/d34/f44 [0,4194304] 0 2024-03-24T10:45:57.642 INFO:tasks.workunit.client.0.smithi094.stdout:4/571: dwrite d0/d5/d7d/d97/d4a/f61 [8388608,4194304] 0 2024-03-24T10:45:57.643 INFO:tasks.workunit.client.0.smithi094.stdout:5/549: read d7/d23/d5f/d1b/f42 [156723,89547] 0 2024-03-24T10:45:57.645 INFO:tasks.workunit.client.0.smithi094.stdout:5/550: mknod d7/d23/d5f/d9/d48/cb2 0 2024-03-24T10:45:57.645 INFO:tasks.workunit.client.0.smithi094.stdout:5/551: read - d7/d23/d5f/d9/d48/f74 zero size 2024-03-24T10:45:57.645 INFO:tasks.workunit.client.0.smithi094.stdout:5/552: chown d7/dc/d12 247536 1 2024-03-24T10:45:57.647 INFO:tasks.workunit.client.0.smithi094.stdout:5/553: creat d7/d23/d2b/d68/d81/fb3 x:0 0 0 2024-03-24T10:45:57.647 INFO:tasks.workunit.client.0.smithi094.stdout:8/571: dwrite df/d12/d5b/d63/f9c [0,4194304] 0 2024-03-24T10:45:57.647 INFO:tasks.workunit.client.0.smithi094.stdout:7/531: dwrite d1/d9/d96/f9a [0,4194304] 0 2024-03-24T10:45:57.656 INFO:tasks.workunit.client.0.smithi094.stdout:7/532: getdents d1/d9/d25 0 2024-03-24T10:45:57.657 INFO:tasks.workunit.client.0.smithi094.stdout:2/473: dread d18/f19 [0,4194304] 0 2024-03-24T10:45:57.657 INFO:tasks.workunit.client.0.smithi094.stdout:7/533: symlink d1/d9/d96/lab 0 2024-03-24T10:45:57.657 INFO:tasks.workunit.client.0.smithi094.stdout:7/534: write d1/d9/d16/d21/f3f [911179,115243] 0 2024-03-24T10:45:57.659 INFO:tasks.workunit.client.0.smithi094.stdout:4/572: dread d0/d22/de/d28/d2b/f51 [0,4194304] 0 2024-03-24T10:45:57.662 INFO:tasks.workunit.client.0.smithi094.stdout:4/573: creat d0/d5/d1b/d4f/fa7 x:0 0 0 2024-03-24T10:45:57.666 INFO:tasks.workunit.client.0.smithi094.stdout:9/568: dwrite df/f2a [0,4194304] 0 2024-03-24T10:45:57.666 INFO:tasks.workunit.client.0.smithi094.stdout:2/474: dread d18/f47 [0,4194304] 0 2024-03-24T10:45:57.670 INFO:tasks.workunit.client.0.smithi094.stdout:2/475: getdents d18/d48 0 2024-03-24T10:45:57.673 INFO:tasks.workunit.client.0.smithi094.stdout:2/476: symlink d18/d66/d90/l91 0 2024-03-24T10:45:57.673 INFO:tasks.workunit.client.0.smithi094.stdout:2/477: write d18/f47 [2945593,102136] 0 2024-03-24T10:45:57.673 INFO:tasks.workunit.client.0.smithi094.stdout:2/478: readlink d18/d48/d56/l8e 0 2024-03-24T10:45:57.673 INFO:tasks.workunit.client.0.smithi094.stdout:2/479: chown d18/d66/d90 5204431 1 2024-03-24T10:45:57.675 INFO:tasks.workunit.client.0.smithi094.stdout:7/535: dwrite d1/d6c/d80/f6f [0,4194304] 0 2024-03-24T10:45:57.677 INFO:tasks.workunit.client.0.smithi094.stdout:9/569: dread df/d2d/d34/f39 [0,4194304] 0 2024-03-24T10:45:57.678 INFO:tasks.workunit.client.0.smithi094.stdout:2/480: link d18/d48/l62 d18/d7f/d88/l92 0 2024-03-24T10:45:57.683 INFO:tasks.workunit.client.0.smithi094.stdout:7/536: creat d1/d9/fac x:0 0 0 2024-03-24T10:45:57.685 INFO:tasks.workunit.client.0.smithi094.stdout:2/481: symlink d18/d7f/l93 0 2024-03-24T10:45:57.686 INFO:tasks.workunit.client.0.smithi094.stdout:7/537: mknod d1/d6c/d80/cad 0 2024-03-24T10:45:57.686 INFO:tasks.workunit.client.0.smithi094.stdout:7/538: fsync d1/d6c/d80/f6f 0 2024-03-24T10:45:57.687 INFO:tasks.workunit.client.0.smithi094.stdout:7/539: chown d1/d9/l47 162810 1 2024-03-24T10:45:57.687 INFO:tasks.workunit.client.0.smithi094.stdout:7/540: fsync d1/d6c/d80/f69 0 2024-03-24T10:45:57.689 INFO:tasks.workunit.client.0.smithi094.stdout:5/554: fsync d7/d23/d5f/d14/d36/fb0 0 2024-03-24T10:45:57.690 INFO:tasks.workunit.client.0.smithi094.stdout:7/541: mknod d1/d9/d14/d34/cae 0 2024-03-24T10:45:57.690 INFO:tasks.workunit.client.0.smithi094.stdout:7/542: chown d1/d9/d16/c93 1463 1 2024-03-24T10:45:57.690 INFO:tasks.workunit.client.0.smithi094.stdout:7/543: write d1/d9/d25/d3d/f98 [3929224,24058] 0 2024-03-24T10:45:57.691 INFO:tasks.workunit.client.0.smithi094.stdout:2/482: mknod d18/d1e/d34/c94 0 2024-03-24T10:45:57.695 INFO:tasks.workunit.client.0.smithi094.stdout:7/544: creat d1/d9/d35/d65/faf x:0 0 0 2024-03-24T10:45:57.698 INFO:tasks.workunit.client.0.smithi094.stdout:2/483: dread d18/d77/f78 [0,4194304] 0 2024-03-24T10:45:57.698 INFO:tasks.workunit.client.0.smithi094.stdout:2/484: getdents d18/d1e/d34/d71 0 2024-03-24T10:45:57.699 INFO:tasks.workunit.client.0.smithi094.stdout:0/499: sync 2024-03-24T10:45:57.699 INFO:tasks.workunit.client.0.smithi094.stdout:6/546: sync 2024-03-24T10:45:57.699 INFO:tasks.workunit.client.0.smithi094.stdout:6/547: truncate d0/d78/f29 809523 0 2024-03-24T10:45:57.704 INFO:tasks.workunit.client.0.smithi094.stdout:0/500: mknod d14/d22/d32/d70/cb1 0 2024-03-24T10:45:57.707 INFO:tasks.workunit.client.0.smithi094.stdout:6/548: truncate d0/d30/d50/d4d/d6b/d86/f9e 2908799 0 2024-03-24T10:45:57.708 INFO:tasks.workunit.client.0.smithi094.stdout:3/460: sync 2024-03-24T10:45:57.708 INFO:tasks.workunit.client.0.smithi094.stdout:3/461: chown d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/f5b 20 1 2024-03-24T10:45:57.710 INFO:tasks.workunit.client.0.smithi094.stdout:6/549: getdents d0/d30/d50/d4d 0 2024-03-24T10:45:57.715 INFO:tasks.workunit.client.0.smithi094.stdout:9/570: dwrite df/d2d/d1f/d11/f4e [0,4194304] 0 2024-03-24T10:45:57.718 INFO:tasks.workunit.client.0.smithi094.stdout:9/571: mknod df/d2d/d1f/d58/d70/cb0 0 2024-03-24T10:45:57.718 INFO:tasks.workunit.client.0.smithi094.stdout:5/555: dwrite d7/dc/fa2 [0,4194304] 0 2024-03-24T10:45:57.720 INFO:tasks.workunit.client.0.smithi094.stdout:9/572: mkdir df/d2d/d1f/d11/d43/db1 0 2024-03-24T10:45:57.722 INFO:tasks.workunit.client.0.smithi094.stdout:5/556: rmdir d7/d23/d2b/d68/d63 39 2024-03-24T10:45:57.722 INFO:tasks.workunit.client.0.smithi094.stdout:3/462: write d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/d73/f75 [3689205,102566] 0 2024-03-24T10:45:57.724 INFO:tasks.workunit.client.0.smithi094.stdout:5/557: mknod d7/d23/d5f/d14/d36/cb4 0 2024-03-24T10:45:57.724 INFO:tasks.workunit.client.0.smithi094.stdout:2/485: dwrite d18/d1e/f53 [0,4194304] 0 2024-03-24T10:45:57.725 INFO:tasks.workunit.client.0.smithi094.stdout:7/545: dwrite d1/d9/d16/d21/d51/d83/d39/d44/f8a [4194304,4194304] 0 2024-03-24T10:45:57.727 INFO:tasks.workunit.client.0.smithi094.stdout:0/501: dwrite d14/d18/d85/fa3 [0,4194304] 0 2024-03-24T10:45:57.729 INFO:tasks.workunit.client.0.smithi094.stdout:9/573: dread df/d2d/f8d [0,4194304] 0 2024-03-24T10:45:57.729 INFO:tasks.workunit.client.0.smithi094.stdout:9/574: read df/f22 [117509,38165] 0 2024-03-24T10:45:57.730 INFO:tasks.workunit.client.0.smithi094.stdout:6/550: dwrite d0/d78/f5a [0,4194304] 0 2024-03-24T10:45:57.730 INFO:tasks.workunit.client.0.smithi094.stdout:6/551: write d0/d21/f2d [3771656,119868] 0 2024-03-24T10:45:57.731 INFO:tasks.workunit.client.0.smithi094.stdout:0/502: write d14/f49 [41206,103219] 0 2024-03-24T10:45:57.731 INFO:tasks.workunit.client.0.smithi094.stdout:2/486: symlink d18/d1e/d5a/d61/d74/d7e/l95 0 2024-03-24T10:45:57.731 INFO:tasks.workunit.client.0.smithi094.stdout:0/503: write d14/d22/d28/d2b/d41/f7e [822139,64053] 0 2024-03-24T10:45:57.733 INFO:tasks.workunit.client.0.smithi094.stdout:0/504: dread d14/d18/f1f [0,4194304] 0 2024-03-24T10:45:57.735 INFO:tasks.workunit.client.0.smithi094.stdout:7/546: mknod d1/d9/cb0 0 2024-03-24T10:45:57.735 INFO:tasks.workunit.client.0.smithi094.stdout:7/547: fdatasync d1/d9/d14/f4b 0 2024-03-24T10:45:57.740 INFO:tasks.workunit.client.0.smithi094.stdout:9/575: link df/d2d/d1f/d58/d5a/c64 df/d2d/d1f/d11/d43/cb2 0 2024-03-24T10:45:57.740 INFO:tasks.workunit.client.0.smithi094.stdout:9/576: write df/d2d/d34/d2c/f5f [113415,21462] 0 2024-03-24T10:45:57.743 INFO:tasks.workunit.client.0.smithi094.stdout:2/487: dread d18/d1e/d34/d4a/f6c [0,4194304] 0 2024-03-24T10:45:57.743 INFO:tasks.workunit.client.0.smithi094.stdout:2/488: chown d18/d48/l7b 329783 1 2024-03-24T10:45:57.743 INFO:tasks.workunit.client.0.smithi094.stdout:2/489: chown d18/d48/d6b 1507595972 1 2024-03-24T10:45:57.743 INFO:tasks.workunit.client.0.smithi094.stdout:6/552: link d0/d78/da/f2e d0/d21/fa8 0 2024-03-24T10:45:57.746 INFO:tasks.workunit.client.0.smithi094.stdout:2/490: rmdir d18/d48/d6b 0 2024-03-24T10:45:57.746 INFO:tasks.workunit.client.0.smithi094.stdout:3/463: dwrite d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/d73/f80 [0,4194304] 0 2024-03-24T10:45:57.746 INFO:tasks.workunit.client.0.smithi094.stdout:3/464: read - d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/f93 zero size 2024-03-24T10:45:57.746 INFO:tasks.workunit.client.0.smithi094.stdout:5/558: dwrite d7/d23/d5f/d14/f7c [0,4194304] 0 2024-03-24T10:45:57.747 INFO:tasks.workunit.client.0.smithi094.stdout:3/465: truncate d0/db/dc/d28/d3c/d7c/f9d 183273 0 2024-03-24T10:45:57.747 INFO:tasks.workunit.client.0.smithi094.stdout:5/559: truncate d7/d23/d5f/d9/d48/fa7 135896 0 2024-03-24T10:45:57.747 INFO:tasks.workunit.client.0.smithi094.stdout:6/553: mkdir d0/da9 0 2024-03-24T10:45:57.748 INFO:tasks.workunit.client.0.smithi094.stdout:9/577: dread df/d2d/d34/d2c/f40 [0,4194304] 0 2024-03-24T10:45:57.748 INFO:tasks.workunit.client.0.smithi094.stdout:9/578: readlink df/d2d/d34/d2c/d74/l95 0 2024-03-24T10:45:57.748 INFO:tasks.workunit.client.0.smithi094.stdout:9/579: readlink df/d2d/d34/d2c/l48 0 2024-03-24T10:45:57.748 INFO:tasks.workunit.client.0.smithi094.stdout:9/580: chown df/d2d/d1f/d11/d43/l45 4337 1 2024-03-24T10:45:57.748 INFO:tasks.workunit.client.0.smithi094.stdout:9/581: chown df/d2d/d1f/d58/d5a/d50/da5 856560668 1 2024-03-24T10:45:57.751 INFO:tasks.workunit.client.0.smithi094.stdout:0/505: creat d14/d22/d28/da2/fb2 x:0 0 0 2024-03-24T10:45:57.756 INFO:tasks.workunit.client.0.smithi094.stdout:6/554: dread d0/d78/da/f20 [0,4194304] 0 2024-03-24T10:45:57.756 INFO:tasks.workunit.client.0.smithi094.stdout:6/555: creat d0/d21/d2a/d55/faa x:0 0 0 2024-03-24T10:45:57.758 INFO:tasks.workunit.client.0.smithi094.stdout:5/560: unlink d7/dc/d12/l18 0 2024-03-24T10:45:57.759 INFO:tasks.workunit.client.0.smithi094.stdout:5/561: chown d7/d23/d5f/d14/d36/cb4 11940922 1 2024-03-24T10:45:57.759 INFO:tasks.workunit.client.0.smithi094.stdout:1/432: sync 2024-03-24T10:45:57.760 INFO:tasks.workunit.client.0.smithi094.stdout:3/466: mknod d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d87/ca5 0 2024-03-24T10:45:57.760 INFO:tasks.workunit.client.0.smithi094.stdout:3/467: creat d0/db/dc/d28/d38/fa6 x:0 0 0 2024-03-24T10:45:57.760 INFO:tasks.workunit.client.0.smithi094.stdout:2/491: dread d18/d1e/d34/f3d [0,4194304] 0 2024-03-24T10:45:57.761 INFO:tasks.workunit.client.0.smithi094.stdout:7/548: dwrite d1/d6c/f88 [0,4194304] 0 2024-03-24T10:45:57.761 INFO:tasks.workunit.client.0.smithi094.stdout:6/556: mknod d0/d78/da/d23/d7c/cab 0 2024-03-24T10:45:57.761 INFO:tasks.workunit.client.0.smithi094.stdout:6/557: readlink d0/d78/da/d77/l8a 0 2024-03-24T10:45:57.762 INFO:tasks.workunit.client.0.smithi094.stdout:6/558: write d0/d21/d2a/d2f/fa6 [346771,69571] 0 2024-03-24T10:45:57.762 INFO:tasks.workunit.client.0.smithi094.stdout:6/559: creat d0/d30/d50/fac x:0 0 0 2024-03-24T10:45:57.762 INFO:tasks.workunit.client.0.smithi094.stdout:5/562: mknod d7/d23/d5f/d9/d48/cb5 0 2024-03-24T10:45:57.763 INFO:tasks.workunit.client.0.smithi094.stdout:6/560: dread d0/d78/da/d23/d34/d4e/f80 [0,4194304] 0 2024-03-24T10:45:57.763 INFO:tasks.workunit.client.0.smithi094.stdout:6/561: chown d0/d21/d2a/c88 55378 1 2024-03-24T10:45:57.764 INFO:tasks.workunit.client.0.smithi094.stdout:2/492: truncate d18/f1b 2029966 0 2024-03-24T10:45:57.767 INFO:tasks.workunit.client.0.smithi094.stdout:3/468: write d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/d73/f80 [453058,110479] 0 2024-03-24T10:45:57.768 INFO:tasks.workunit.client.0.smithi094.stdout:3/469: chown d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d34 598 1 2024-03-24T10:45:57.768 INFO:tasks.workunit.client.0.smithi094.stdout:3/470: write d0/f42 [1546653,50226] 0 2024-03-24T10:45:57.769 INFO:tasks.workunit.client.0.smithi094.stdout:7/549: creat d1/d6c/d80/fb1 x:0 0 0 2024-03-24T10:45:57.771 INFO:tasks.workunit.client.0.smithi094.stdout:0/506: dread d14/d22/d28/f6c [0,4194304] 0 2024-03-24T10:45:57.771 INFO:tasks.workunit.client.0.smithi094.stdout:0/507: chown d14/f62 346 1 2024-03-24T10:45:57.771 INFO:tasks.workunit.client.0.smithi094.stdout:5/563: symlink d7/d23/d5f/d14/d36/d3a/d57/lb6 0 2024-03-24T10:45:57.771 INFO:tasks.workunit.client.0.smithi094.stdout:5/564: creat d7/fb7 x:0 0 0 2024-03-24T10:45:57.771 INFO:tasks.workunit.client.0.smithi094.stdout:5/565: dread - d7/d23/d5f/d9/d48/f74 zero size 2024-03-24T10:45:57.771 INFO:tasks.workunit.client.0.smithi094.stdout:2/493: symlink d18/d1e/d65/l96 0 2024-03-24T10:45:57.772 INFO:tasks.workunit.client.0.smithi094.stdout:2/494: write d18/d1e/d34/f6f [264923,83563] 0 2024-03-24T10:45:57.772 INFO:tasks.workunit.client.0.smithi094.stdout:8/572: sync 2024-03-24T10:45:57.773 INFO:tasks.workunit.client.0.smithi094.stdout:7/550: symlink d1/d9/d25/d3d/lb2 0 2024-03-24T10:45:57.778 INFO:tasks.workunit.client.0.smithi094.stdout:2/495: link d18/d1e/d34/f35 d18/d66/f97 0 2024-03-24T10:45:57.780 INFO:tasks.workunit.client.0.smithi094.stdout:8/573: truncate df/d25/d28/d31/f32 600142 0 2024-03-24T10:45:57.782 INFO:tasks.workunit.client.0.smithi094.stdout:7/551: mknod d1/d9/d25/cb3 0 2024-03-24T10:45:57.782 INFO:tasks.workunit.client.0.smithi094.stdout:7/552: truncate d1/f5d 800316 0 2024-03-24T10:45:57.782 INFO:tasks.workunit.client.0.smithi094.stdout:7/553: stat d1/d9/d35/d61 0 2024-03-24T10:45:57.783 INFO:tasks.workunit.client.0.smithi094.stdout:0/508: getdents d14/d18 0 2024-03-24T10:45:57.784 INFO:tasks.workunit.client.0.smithi094.stdout:5/566: dread d7/d23/d5f/d9/d48/d51/f52 [0,4194304] 0 2024-03-24T10:45:57.784 INFO:tasks.workunit.client.0.smithi094.stdout:5/567: truncate d7/d23/d2b/d68/d81/fb3 437104 0 2024-03-24T10:45:57.784 INFO:tasks.workunit.client.0.smithi094.stdout:5/568: creat d7/dc/fb8 x:0 0 0 2024-03-24T10:45:57.785 INFO:tasks.workunit.client.0.smithi094.stdout:8/574: truncate df/d25/d28/d31/d47/f5a 87093 0 2024-03-24T10:45:57.787 INFO:tasks.workunit.client.0.smithi094.stdout:4/574: sync 2024-03-24T10:45:57.787 INFO:tasks.workunit.client.0.smithi094.stdout:0/509: chown d14/d22/d28/f31 3 1 2024-03-24T10:45:57.788 INFO:tasks.workunit.client.0.smithi094.stdout:2/496: dread d18/f45 [0,4194304] 0 2024-03-24T10:45:57.788 INFO:tasks.workunit.client.0.smithi094.stdout:2/497: truncate d18/d1e/d34/f6f 768879 0 2024-03-24T10:45:57.789 INFO:tasks.workunit.client.0.smithi094.stdout:5/569: mkdir d7/dc/db9 0 2024-03-24T10:45:57.791 INFO:tasks.workunit.client.0.smithi094.stdout:6/562: dwrite d0/d21/f2d [0,4194304] 0 2024-03-24T10:45:57.793 INFO:tasks.workunit.client.0.smithi094.stdout:3/471: dwrite d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/f93 [0,4194304] 0 2024-03-24T10:45:57.795 INFO:tasks.workunit.client.0.smithi094.stdout:4/575: rename d0/d22/c9d to d0/d22/ca8 0 2024-03-24T10:45:57.796 INFO:tasks.workunit.client.0.smithi094.stdout:0/510: getdents d14/d22/d5b/d86 0 2024-03-24T10:45:57.797 INFO:tasks.workunit.client.0.smithi094.stdout:2/498: creat d18/d48/f98 x:0 0 0 2024-03-24T10:45:57.801 INFO:tasks.workunit.client.0.smithi094.stdout:5/570: write d7/d23/d5f/d14/f27 [3723507,84415] 0 2024-03-24T10:45:57.803 INFO:tasks.workunit.client.0.smithi094.stdout:6/563: mknod d0/d21/d2a/cad 0 2024-03-24T10:45:57.803 INFO:tasks.workunit.client.0.smithi094.stdout:2/499: dread d18/f3f [0,4194304] 0 2024-03-24T10:45:57.804 INFO:tasks.workunit.client.0.smithi094.stdout:8/575: dwrite df/d25/d28/d31/f72 [4194304,4194304] 0 2024-03-24T10:45:57.804 INFO:tasks.workunit.client.0.smithi094.stdout:8/576: readlink df/d25/d36/l6e 0 2024-03-24T10:45:57.804 INFO:tasks.workunit.client.0.smithi094.stdout:8/577: chown df/d25/d28/d31/d47/f7e 125 1 2024-03-24T10:45:57.804 INFO:tasks.workunit.client.0.smithi094.stdout:7/554: rename d1/d9/d35 to d1/db4 0 2024-03-24T10:45:57.806 INFO:tasks.workunit.client.0.smithi094.stdout:2/500: dread d18/d1e/d34/f35 [0,4194304] 0 2024-03-24T10:45:57.809 INFO:tasks.workunit.client.0.smithi094.stdout:4/576: creat d0/d22/de/d28/d2b/d7a/fa9 x:0 0 0 2024-03-24T10:45:57.809 INFO:tasks.workunit.client.0.smithi094.stdout:4/577: truncate d0/d22/de/d28/d2b/d33/da4/f8b 23558 0 2024-03-24T10:45:57.809 INFO:tasks.workunit.client.0.smithi094.stdout:0/511: mknod d14/d22/d28/d48/cb3 0 2024-03-24T10:45:57.811 INFO:tasks.workunit.client.0.smithi094.stdout:5/571: truncate d7/dc/fa2 2202632 0 2024-03-24T10:45:57.814 INFO:tasks.workunit.client.0.smithi094.stdout:6/564: creat d0/fae x:0 0 0 2024-03-24T10:45:57.818 INFO:tasks.workunit.client.0.smithi094.stdout:3/472: getdents d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d4f/d74 0 2024-03-24T10:45:57.820 INFO:tasks.workunit.client.0.smithi094.stdout:4/578: creat d0/d1f/faa x:0 0 0 2024-03-24T10:45:57.822 INFO:tasks.workunit.client.0.smithi094.stdout:0/512: unlink d14/d18/f37 0 2024-03-24T10:45:57.822 INFO:tasks.workunit.client.0.smithi094.stdout:0/513: fsync d14/d22/d28/d48/d7b/f8e 0 2024-03-24T10:45:57.824 INFO:tasks.workunit.client.0.smithi094.stdout:6/565: mknod d0/d30/d50/d4d/d6b/d86/d9c/caf 0 2024-03-24T10:45:57.824 INFO:tasks.workunit.client.0.smithi094.stdout:6/566: dread - d0/d78/da/d23/d34/d4e/f7e zero size 2024-03-24T10:45:57.824 INFO:tasks.workunit.client.0.smithi094.stdout:2/501: dread d18/f45 [0,4194304] 0 2024-03-24T10:45:57.826 INFO:tasks.workunit.client.0.smithi094.stdout:0/514: symlink d14/d18/lb4 0 2024-03-24T10:45:57.826 INFO:tasks.workunit.client.0.smithi094.stdout:2/502: dread d18/d66/f97 [0,4194304] 0 2024-03-24T10:45:57.828 INFO:tasks.workunit.client.0.smithi094.stdout:6/567: getdents d0/d21/d2a 0 2024-03-24T10:45:57.829 INFO:tasks.workunit.client.0.smithi094.stdout:6/568: chown d0/d78/da/d23/d34/l47 4 1 2024-03-24T10:45:57.830 INFO:tasks.workunit.client.0.smithi094.stdout:3/473: mknod d0/db/dc/ca7 0 2024-03-24T10:45:57.832 INFO:tasks.workunit.client.0.smithi094.stdout:2/503: creat d18/d1e/d5a/d61/d74/f99 x:0 0 0 2024-03-24T10:45:57.832 INFO:tasks.workunit.client.0.smithi094.stdout:2/504: chown d18/d7f/l93 1422036 1 2024-03-24T10:45:57.832 INFO:tasks.workunit.client.0.smithi094.stdout:3/474: symlink d0/db/dc/d28/d3c/d55/d7f/d20/d23/d8b/la8 0 2024-03-24T10:45:57.833 INFO:tasks.workunit.client.0.smithi094.stdout:2/505: dread d18/d66/f97 [0,4194304] 0 2024-03-24T10:45:57.834 INFO:tasks.workunit.client.0.smithi094.stdout:7/555: dwrite d1/d9/f2c [4194304,4194304] 0 2024-03-24T10:45:57.834 INFO:tasks.workunit.client.0.smithi094.stdout:2/506: truncate d18/d1e/d34/f46 3673330 0 2024-03-24T10:45:57.835 INFO:tasks.workunit.client.0.smithi094.stdout:7/556: symlink d1/d9/d16/d21/d51/d83/d39/lb5 0 2024-03-24T10:45:57.836 INFO:tasks.workunit.client.0.smithi094.stdout:7/557: creat d1/db4/d65/fb6 x:0 0 0 2024-03-24T10:45:57.836 INFO:tasks.workunit.client.0.smithi094.stdout:7/558: write d1/d9/d16/d21/f3f [1311737,30303] 0 2024-03-24T10:45:57.837 INFO:tasks.workunit.client.0.smithi094.stdout:7/559: truncate d1/d9/d16/d21/d51/d83/d39/f48 1174348 0 2024-03-24T10:45:57.838 INFO:tasks.workunit.client.0.smithi094.stdout:7/560: creat d1/d9/d14/da5/fb7 x:0 0 0 2024-03-24T10:45:57.838 INFO:tasks.workunit.client.0.smithi094.stdout:7/561: read - d1/d9/d16/d21/d51/d83/f90 zero size 2024-03-24T10:45:57.839 INFO:tasks.workunit.client.0.smithi094.stdout:8/578: dwrite df/d25/d28/d31/d47/f7e [0,4194304] 0 2024-03-24T10:45:57.842 INFO:tasks.workunit.client.0.smithi094.stdout:5/572: dwrite d7/d23/d5f/d9/f78 [0,4194304] 0 2024-03-24T10:45:57.842 INFO:tasks.workunit.client.0.smithi094.stdout:8/579: mknod df/d25/d28/d31/d47/d66/da3/db5/cc2 0 2024-03-24T10:45:57.844 INFO:tasks.workunit.client.0.smithi094.stdout:5/573: symlink d7/d26/d55/d64/lba 0 2024-03-24T10:45:57.846 INFO:tasks.workunit.client.0.smithi094.stdout:4/579: dwrite d0/d22/de/d28/f7f [0,4194304] 0 2024-03-24T10:45:57.846 INFO:tasks.workunit.client.0.smithi094.stdout:4/580: creat d0/d5/fab x:0 0 0 2024-03-24T10:45:57.846 INFO:tasks.workunit.client.0.smithi094.stdout:4/581: readlink d0/d34/l3c 0 2024-03-24T10:45:57.847 INFO:tasks.workunit.client.0.smithi094.stdout:3/475: write d0/db/dc/d28/d38/f72 [2905241,74227] 0 2024-03-24T10:45:57.847 INFO:tasks.workunit.client.0.smithi094.stdout:3/476: rename d0 to d0/db/dc/d28/d38/d53/da9 22 2024-03-24T10:45:57.847 INFO:tasks.workunit.client.0.smithi094.stdout:8/580: creat df/d25/d28/d31/d47/d66/da3/db5/fc3 x:0 0 0 2024-03-24T10:45:57.848 INFO:tasks.workunit.client.0.smithi094.stdout:8/581: dread df/d25/d28/d31/d47/d22/f38 [0,4194304] 0 2024-03-24T10:45:57.851 INFO:tasks.workunit.client.0.smithi094.stdout:5/574: read d7/d23/d5f/d1b/f37 [3995804,21944] 0 2024-03-24T10:45:57.851 INFO:tasks.workunit.client.0.smithi094.stdout:7/562: read d1/d9/d16/d21/d51/d83/d39/d44/f8a [5580397,68203] 0 2024-03-24T10:45:57.852 INFO:tasks.workunit.client.0.smithi094.stdout:4/582: dread d0/d22/f9e [0,4194304] 0 2024-03-24T10:45:57.858 INFO:tasks.workunit.client.0.smithi094.stdout:3/477: rmdir d0/db/dc/d28/d50 39 2024-03-24T10:45:57.858 INFO:tasks.workunit.client.0.smithi094.stdout:3/478: stat d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d34 0 2024-03-24T10:45:57.861 INFO:tasks.workunit.client.0.smithi094.stdout:7/563: creat d1/d9/d16/d21/d49/da9/fb8 x:0 0 0 2024-03-24T10:45:57.863 INFO:tasks.workunit.client.0.smithi094.stdout:4/583: unlink d0/d22/c6a 0 2024-03-24T10:45:57.863 INFO:tasks.workunit.client.0.smithi094.stdout:4/584: fdatasync d0/d5/d7d/f99 0 2024-03-24T10:45:57.864 INFO:tasks.workunit.client.0.smithi094.stdout:4/585: chown d0/d22/l15 208 1 2024-03-24T10:45:57.864 INFO:tasks.workunit.client.0.smithi094.stdout:4/586: creat d0/d34/fac x:0 0 0 2024-03-24T10:45:57.864 INFO:tasks.workunit.client.0.smithi094.stdout:7/564: mknod d1/d9/d16/d21/cb9 0 2024-03-24T10:45:57.867 INFO:tasks.workunit.client.0.smithi094.stdout:4/587: mknod d0/d22/de/d28/d2b/cad 0 2024-03-24T10:45:57.877 INFO:tasks.workunit.client.0.smithi094.stdout:5/575: dwrite d7/d23/d5f/d14/f35 [0,4194304] 0 2024-03-24T10:45:57.877 INFO:tasks.workunit.client.0.smithi094.stdout:5/576: rename d7 to d7/dc/db9/dbb 22 2024-03-24T10:45:57.877 INFO:tasks.workunit.client.0.smithi094.stdout:3/479: dwrite d0/db/f2b [0,4194304] 0 2024-03-24T10:45:57.877 INFO:tasks.workunit.client.0.smithi094.stdout:7/565: dread d1/d9/d16/d21/d51/d83/d39/d44/f8a [4194304,4194304] 0 2024-03-24T10:45:57.881 INFO:tasks.workunit.client.0.smithi094.stdout:8/582: dwrite df/d25/d28/d31/d47/f7e [4194304,4194304] 0 2024-03-24T10:45:57.882 INFO:tasks.workunit.client.0.smithi094.stdout:7/566: rename d1/d9/f2c to d1/db4/fba 0 2024-03-24T10:45:57.886 INFO:tasks.workunit.client.0.smithi094.stdout:8/583: creat df/d8c/fc4 x:0 0 0 2024-03-24T10:45:57.888 INFO:tasks.workunit.client.0.smithi094.stdout:7/567: rename d1/d9/d16/d21/d51/f8e to d1/d9/d14/fbb 0 2024-03-24T10:45:57.890 INFO:tasks.workunit.client.0.smithi094.stdout:8/584: link df/d25/d36/d3f/f59 df/d25/d28/d31/d47/d22/fc5 0 2024-03-24T10:45:57.890 INFO:tasks.workunit.client.0.smithi094.stdout:8/585: stat df/d12/d5b/l5e 0 2024-03-24T10:45:57.890 INFO:tasks.workunit.client.0.smithi094.stdout:8/586: chown df/d25/d28/d31/d47/f85 23809 1 2024-03-24T10:45:57.891 INFO:tasks.workunit.client.0.smithi094.stdout:3/480: dread d0/db/dc/d28/d3c/d55/d7f/d20/f24 [0,4194304] 0 2024-03-24T10:45:57.891 INFO:tasks.workunit.client.0.smithi094.stdout:3/481: truncate d0/db/dc/d28/d38/d53/f89 450373 0 2024-03-24T10:45:57.891 INFO:tasks.workunit.client.0.smithi094.stdout:5/577: dwrite d7/d23/d5f/d14/d36/d3a/f7f [0,4194304] 0 2024-03-24T10:45:57.892 INFO:tasks.workunit.client.0.smithi094.stdout:8/587: unlink df/d25/d36/d3f/faa 0 2024-03-24T10:45:57.892 INFO:tasks.workunit.client.0.smithi094.stdout:8/588: readlink df/d25/d28/l4b 0 2024-03-24T10:45:57.897 INFO:tasks.workunit.client.0.smithi094.stdout:3/482: mknod d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d4f/caa 0 2024-03-24T10:45:57.898 INFO:tasks.workunit.client.0.smithi094.stdout:5/578: rename d7/dc/d12/f61 to d7/d23/d5f/d14/d9f/fbc 0 2024-03-24T10:45:57.898 INFO:tasks.workunit.client.0.smithi094.stdout:1/433: sync 2024-03-24T10:45:57.898 INFO:tasks.workunit.client.0.smithi094.stdout:9/582: sync 2024-03-24T10:45:57.899 INFO:tasks.workunit.client.0.smithi094.stdout:9/583: chown df/d2d/d1f/d11/d53 244 1 2024-03-24T10:45:57.899 INFO:tasks.workunit.client.0.smithi094.stdout:9/584: fdatasync df/d2d/d1f/d11/f4e 0 2024-03-24T10:45:57.899 INFO:tasks.workunit.client.0.smithi094.stdout:9/585: fdatasync df/d2d/d1f/d58/d5a/d69/f78 0 2024-03-24T10:45:57.899 INFO:tasks.workunit.client.0.smithi094.stdout:3/483: mknod d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/d73/cab 0 2024-03-24T10:45:57.899 INFO:tasks.workunit.client.0.smithi094.stdout:1/434: read d4/d9/d37/d33/f51 [525329,52930] 0 2024-03-24T10:45:57.899 INFO:tasks.workunit.client.0.smithi094.stdout:3/484: read d0/f1e [5044890,85246] 0 2024-03-24T10:45:57.902 INFO:tasks.workunit.client.0.smithi094.stdout:7/568: dwrite d1/d9/d25/d3d/faa [0,4194304] 0 2024-03-24T10:45:57.903 INFO:tasks.workunit.client.0.smithi094.stdout:8/589: rename df/d12/c19 to df/d25/cc6 0 2024-03-24T10:45:57.904 INFO:tasks.workunit.client.0.smithi094.stdout:5/579: unlink d7/d23/d5f/d1b/f60 0 2024-03-24T10:45:57.907 INFO:tasks.workunit.client.0.smithi094.stdout:1/435: symlink d4/d9/de/d6f/d62/l97 0 2024-03-24T10:45:57.907 INFO:tasks.workunit.client.0.smithi094.stdout:1/436: truncate d4/d9/d37/d33/f5b 640750 0 2024-03-24T10:45:57.908 INFO:tasks.workunit.client.0.smithi094.stdout:7/569: symlink d1/d9/d25/d3d/lbc 0 2024-03-24T10:45:57.908 INFO:tasks.workunit.client.0.smithi094.stdout:8/590: getdents df/d8c/d8f 0 2024-03-24T10:45:57.908 INFO:tasks.workunit.client.0.smithi094.stdout:8/591: creat df/d12/d5b/d63/fc7 x:0 0 0 2024-03-24T10:45:57.909 INFO:tasks.workunit.client.0.smithi094.stdout:8/592: write df/d25/d28/d31/d3c/f84 [961574,127283] 0 2024-03-24T10:45:57.911 INFO:tasks.workunit.client.0.smithi094.stdout:9/586: rmdir df/d2d/d1f/d58/d5a/d69 39 2024-03-24T10:45:57.911 INFO:tasks.workunit.client.0.smithi094.stdout:9/587: write df/d2d/d1f/d58/d5a/f6f [4650738,108310] 0 2024-03-24T10:45:57.913 INFO:tasks.workunit.client.0.smithi094.stdout:1/437: mkdir d4/d9/db/d98 0 2024-03-24T10:45:57.914 INFO:tasks.workunit.client.0.smithi094.stdout:7/570: mknod d1/d9/d14/cbd 0 2024-03-24T10:45:57.919 INFO:tasks.workunit.client.0.smithi094.stdout:9/588: rename df/d7b/ca2 to df/d2d/d34/d2c/cb3 0 2024-03-24T10:45:57.920 INFO:tasks.workunit.client.0.smithi094.stdout:5/580: dwrite d7/d26/f53 [0,4194304] 0 2024-03-24T10:45:57.920 INFO:tasks.workunit.client.0.smithi094.stdout:1/438: rmdir d4/d9/db/d5d 39 2024-03-24T10:45:57.920 INFO:tasks.workunit.client.0.smithi094.stdout:8/593: truncate df/d25/d28/d31/d47/f7e 5523189 0 2024-03-24T10:45:57.920 INFO:tasks.workunit.client.0.smithi094.stdout:8/594: fdatasync fd 0 2024-03-24T10:45:57.921 INFO:tasks.workunit.client.0.smithi094.stdout:9/589: creat df/d7b/fb4 x:0 0 0 2024-03-24T10:45:57.922 INFO:tasks.workunit.client.0.smithi094.stdout:1/439: mkdir d4/d9/de/d6f/d62/d99 0 2024-03-24T10:45:57.925 INFO:tasks.workunit.client.0.smithi094.stdout:8/595: rename df/c34 to df/d25/d28/d31/d47/cc8 0 2024-03-24T10:45:57.925 INFO:tasks.workunit.client.0.smithi094.stdout:8/596: creat df/d25/daf/fc9 x:0 0 0 2024-03-24T10:45:57.925 INFO:tasks.workunit.client.0.smithi094.stdout:8/597: stat f4 0 2024-03-24T10:45:57.929 INFO:tasks.workunit.client.0.smithi094.stdout:9/590: symlink df/d7b/lb5 0 2024-03-24T10:45:57.929 INFO:tasks.workunit.client.0.smithi094.stdout:9/591: chown df/d2d/d34/d2c/l33 1983 1 2024-03-24T10:45:57.933 INFO:tasks.workunit.client.0.smithi094.stdout:5/581: write d7/d23/f44 [3283140,79961] 0 2024-03-24T10:45:57.937 INFO:tasks.workunit.client.0.smithi094.stdout:0/515: sync 2024-03-24T10:45:57.940 INFO:tasks.workunit.client.0.smithi094.stdout:0/516: link d14/d22/d28/d2b/d68/c76 d14/d22/d28/d2b/cb5 0 2024-03-24T10:45:57.940 INFO:tasks.workunit.client.0.smithi094.stdout:9/592: dread df/f3f [4194304,4194304] 0 2024-03-24T10:45:57.941 INFO:tasks.workunit.client.0.smithi094.stdout:0/517: symlink d14/d22/d28/d2b/lb6 0 2024-03-24T10:45:57.941 INFO:tasks.workunit.client.0.smithi094.stdout:0/518: write d14/d22/d7d/fa6 [878063,61567] 0 2024-03-24T10:45:57.941 INFO:tasks.workunit.client.0.smithi094.stdout:0/519: write d14/d18/fa0 [385726,91514] 0 2024-03-24T10:45:57.941 INFO:tasks.workunit.client.0.smithi094.stdout:9/593: getdents df/d2d/d1f/d58/d70/dab/d80 0 2024-03-24T10:45:57.942 INFO:tasks.workunit.client.0.smithi094.stdout:0/520: stat d14/d22/d28/d48/d7b/ca9 0 2024-03-24T10:45:57.942 INFO:tasks.workunit.client.0.smithi094.stdout:9/594: symlink df/d2d/d34/lb6 0 2024-03-24T10:45:57.943 INFO:tasks.workunit.client.0.smithi094.stdout:0/521: rename d14/d22/d28/d2b/f44 to d14/d22/d5b/fb7 0 2024-03-24T10:45:57.944 INFO:tasks.workunit.client.0.smithi094.stdout:9/595: creat df/d2d/d1f/d58/d82/fb7 x:0 0 0 2024-03-24T10:45:57.944 INFO:tasks.workunit.client.0.smithi094.stdout:9/596: write df/d2d/f84 [299621,92028] 0 2024-03-24T10:45:57.945 INFO:tasks.workunit.client.0.smithi094.stdout:9/597: dread - df/d2d/d1f/d7e/f91 zero size 2024-03-24T10:45:57.945 INFO:tasks.workunit.client.0.smithi094.stdout:0/522: rename c12 to d14/d18/d85/cb8 0 2024-03-24T10:45:57.945 INFO:tasks.workunit.client.0.smithi094.stdout:0/523: creat d14/d22/d28/da2/fb9 x:0 0 0 2024-03-24T10:45:57.945 INFO:tasks.workunit.client.0.smithi094.stdout:0/524: creat d14/d22/d28/d2b/d68/fba x:0 0 0 2024-03-24T10:45:57.946 INFO:tasks.workunit.client.0.smithi094.stdout:9/598: mkdir df/d2d/d34/db8 0 2024-03-24T10:45:57.947 INFO:tasks.workunit.client.0.smithi094.stdout:9/599: symlink df/d2d/d34/db8/lb9 0 2024-03-24T10:45:57.948 INFO:tasks.workunit.client.0.smithi094.stdout:9/600: mkdir df/d2d/d34/dba 0 2024-03-24T10:45:57.948 INFO:tasks.workunit.client.0.smithi094.stdout:9/601: fdatasync df/d2d/d1f/d58/d5a/f6f 0 2024-03-24T10:45:57.949 INFO:tasks.workunit.client.0.smithi094.stdout:9/602: fdatasync df/f2a 0 2024-03-24T10:45:57.949 INFO:tasks.workunit.client.0.smithi094.stdout:9/603: dread df/d2d/d1f/d58/d5a/d69/f78 [0,4194304] 0 2024-03-24T10:45:57.949 INFO:tasks.workunit.client.0.smithi094.stdout:9/604: readlink df/d2d/d1f/d11/d43/l51 0 2024-03-24T10:45:57.949 INFO:tasks.workunit.client.0.smithi094.stdout:9/605: chown df/d2d/d1f/f4a 1282637 1 2024-03-24T10:45:57.950 INFO:tasks.workunit.client.0.smithi094.stdout:5/582: dwrite d7/d23/d5f/d14/f6d [0,4194304] 0 2024-03-24T10:45:57.950 INFO:tasks.workunit.client.0.smithi094.stdout:9/606: dread df/f22 [0,4194304] 0 2024-03-24T10:45:57.950 INFO:tasks.workunit.client.0.smithi094.stdout:9/607: creat df/d7b/fbb x:0 0 0 2024-03-24T10:45:57.951 INFO:tasks.workunit.client.0.smithi094.stdout:8/598: dwrite df/d25/d28/d31/f72 [4194304,4194304] 0 2024-03-24T10:45:57.951 INFO:tasks.workunit.client.0.smithi094.stdout:8/599: chown df/d12/d5b/d63/d75/c81 58533942 1 2024-03-24T10:45:57.954 INFO:tasks.workunit.client.0.smithi094.stdout:5/583: rename d7/d23/f77 to d7/d23/d5f/d14/d36/fbd 0 2024-03-24T10:45:57.958 INFO:tasks.workunit.client.0.smithi094.stdout:8/600: unlink df/d25/d28/d31/d47/d66/fb7 0 2024-03-24T10:45:57.958 INFO:tasks.workunit.client.0.smithi094.stdout:1/440: dwrite d4/d36/f8e [0,4194304] 0 2024-03-24T10:45:57.958 INFO:tasks.workunit.client.0.smithi094.stdout:0/525: dread d14/d22/d32/f5d [0,4194304] 0 2024-03-24T10:45:57.962 INFO:tasks.workunit.client.0.smithi094.stdout:8/601: mknod df/d25/d28/d31/d3c/cca 0 2024-03-24T10:45:57.966 INFO:tasks.workunit.client.0.smithi094.stdout:0/526: creat d14/d22/d28/d2b/d41/d53/fbb x:0 0 0 2024-03-24T10:45:57.967 INFO:tasks.workunit.client.0.smithi094.stdout:1/441: mknod d4/d9/de/d6f/d62/d80/c9a 0 2024-03-24T10:45:57.969 INFO:tasks.workunit.client.0.smithi094.stdout:8/602: rmdir df/d25/d36/d98/dae 0 2024-03-24T10:45:57.970 INFO:tasks.workunit.client.0.smithi094.stdout:1/442: creat d4/d9/d37/d33/d76/f9b x:0 0 0 2024-03-24T10:45:57.972 INFO:tasks.workunit.client.0.smithi094.stdout:5/584: dwrite d7/d23/d5f/d9/f49 [4194304,4194304] 0 2024-03-24T10:45:57.972 INFO:tasks.workunit.client.0.smithi094.stdout:1/443: mknod d4/d9/db/d39/c9c 0 2024-03-24T10:45:57.974 INFO:tasks.workunit.client.0.smithi094.stdout:5/585: mkdir d7/d23/d5f/d14/d6f/dbe 0 2024-03-24T10:45:57.974 INFO:tasks.workunit.client.0.smithi094.stdout:8/603: dread df/d25/d28/d31/d47/d22/fc5 [0,4194304] 0 2024-03-24T10:45:57.974 INFO:tasks.workunit.client.0.smithi094.stdout:8/604: fdatasync df/d25/d28/d31/d47/f7c 0 2024-03-24T10:45:57.975 INFO:tasks.workunit.client.0.smithi094.stdout:5/586: read d7/d26/f5c [190408,92035] 0 2024-03-24T10:45:57.975 INFO:tasks.workunit.client.0.smithi094.stdout:5/587: chown d7/dc/d12/l1d 165504206 1 2024-03-24T10:45:57.975 INFO:tasks.workunit.client.0.smithi094.stdout:1/444: symlink d4/d9/de/d3d/l9d 0 2024-03-24T10:45:57.978 INFO:tasks.workunit.client.0.smithi094.stdout:1/445: rename d4/d9/db/f42 to d4/d9/de/d6f/d62/d99/f9e 0 2024-03-24T10:45:57.978 INFO:tasks.workunit.client.0.smithi094.stdout:1/446: getdents d4/d9/de/d6f/d62/d84 0 2024-03-24T10:45:57.979 INFO:tasks.workunit.client.0.smithi094.stdout:0/527: dwrite d14/d22/d28/d48/faa [0,4194304] 0 2024-03-24T10:45:57.980 INFO:tasks.workunit.client.0.smithi094.stdout:1/447: unlink d4/d9/db/f1d 0 2024-03-24T10:45:57.985 INFO:tasks.workunit.client.0.smithi094.stdout:0/528: rename d14/d22/d28/d2b/f3d to d14/d22/d7d/fbc 0 2024-03-24T10:45:57.987 INFO:tasks.workunit.client.0.smithi094.stdout:1/448: mkdir d4/d9/db/d31/d3c/d9f 0 2024-03-24T10:45:57.988 INFO:tasks.workunit.client.0.smithi094.stdout:1/449: chown d4/d36/f8e 664803976 1 2024-03-24T10:45:57.990 INFO:tasks.workunit.client.0.smithi094.stdout:0/529: unlink d14/d22/d28/da2/fb2 0 2024-03-24T10:45:58.016 INFO:tasks.workunit.client.0.smithi094.stdout:8/605: dwrite df/f56 [0,4194304] 0 2024-03-24T10:45:58.016 INFO:tasks.workunit.client.0.smithi094.stdout:5/588: dwrite d7/d23/d5f/d1b/f42 [0,4194304] 0 2024-03-24T10:45:58.016 INFO:tasks.workunit.client.0.smithi094.stdout:1/450: dwrite d4/d9/db/d31/f48 [0,4194304] 0 2024-03-24T10:45:58.016 INFO:tasks.workunit.client.0.smithi094.stdout:0/530: dwrite d14/d22/d7d/fa4 [0,4194304] 0 2024-03-24T10:45:58.018 INFO:tasks.workunit.client.0.smithi094.stdout:8/606: link df/d25/d28/d31/d47/d66/da3/db5/fc3 df/d25/daf/fcb 0 2024-03-24T10:45:58.018 INFO:tasks.workunit.client.0.smithi094.stdout:8/607: chown df/d25/d28/d31/d3c/f84 2429981 1 2024-03-24T10:45:58.019 INFO:tasks.workunit.client.0.smithi094.stdout:5/589: rename d7/d23/d5f/d14/d36/fb0 to d7/d26/d55/d64/fbf 0 2024-03-24T10:45:58.019 INFO:tasks.workunit.client.0.smithi094.stdout:5/590: creat d7/d23/d2b/d68/fc0 x:0 0 0 2024-03-24T10:45:58.021 INFO:tasks.workunit.client.0.smithi094.stdout:8/608: dread df/d8c/f9e [0,4194304] 0 2024-03-24T10:45:58.021 INFO:tasks.workunit.client.0.smithi094.stdout:8/609: creat df/d25/d28/fcc x:0 0 0 2024-03-24T10:45:58.022 INFO:tasks.workunit.client.0.smithi094.stdout:1/451: symlink d4/d9/la0 0 2024-03-24T10:45:58.022 INFO:tasks.workunit.client.0.smithi094.stdout:1/452: write d4/f41 [1336780,73319] 0 2024-03-24T10:45:58.025 INFO:tasks.workunit.client.0.smithi094.stdout:6/569: sync 2024-03-24T10:45:58.026 INFO:tasks.workunit.client.0.smithi094.stdout:2/507: sync 2024-03-24T10:45:58.026 INFO:tasks.workunit.client.0.smithi094.stdout:4/588: sync 2024-03-24T10:45:58.026 INFO:tasks.workunit.client.0.smithi094.stdout:6/570: fdatasync d0/d78/da/d23/d34/d4e/d5d/f7f 0 2024-03-24T10:45:58.027 INFO:tasks.workunit.client.0.smithi094.stdout:3/485: sync 2024-03-24T10:45:58.027 INFO:tasks.workunit.client.0.smithi094.stdout:7/571: sync 2024-03-24T10:45:58.027 INFO:tasks.workunit.client.0.smithi094.stdout:9/608: sync 2024-03-24T10:45:58.027 INFO:tasks.workunit.client.0.smithi094.stdout:7/572: truncate d1/d9/fac 154191 0 2024-03-24T10:45:58.027 INFO:tasks.workunit.client.0.smithi094.stdout:7/573: creat d1/d9/d16/fbe x:0 0 0 2024-03-24T10:45:58.029 INFO:tasks.workunit.client.0.smithi094.stdout:8/610: mkdir df/d12/dcd 0 2024-03-24T10:45:58.030 INFO:tasks.workunit.client.0.smithi094.stdout:5/591: write d7/d23/d5f/d14/d36/fbd [3496981,5181] 0 2024-03-24T10:45:58.030 INFO:tasks.workunit.client.0.smithi094.stdout:5/592: creat d7/dc/fc1 x:0 0 0 2024-03-24T10:45:58.031 INFO:tasks.workunit.client.0.smithi094.stdout:6/571: mkdir d0/d78/da/d23/d7c/db0 0 2024-03-24T10:45:58.031 INFO:tasks.workunit.client.0.smithi094.stdout:6/572: write d0/d1a/f7b [212331,121106] 0 2024-03-24T10:45:58.033 INFO:tasks.workunit.client.0.smithi094.stdout:9/609: creat df/fbc x:0 0 0 2024-03-24T10:45:58.033 INFO:tasks.workunit.client.0.smithi094.stdout:9/610: write df/d2d/d1f/d58/d5a/d69/f98 [851125,938] 0 2024-03-24T10:45:58.033 INFO:tasks.workunit.client.0.smithi094.stdout:9/611: read df/d2d/d1f/d11/d43/f76 [150567,127359] 0 2024-03-24T10:45:58.035 INFO:tasks.workunit.client.0.smithi094.stdout:8/611: mkdir df/d25/dce 0 2024-03-24T10:45:58.037 INFO:tasks.workunit.client.0.smithi094.stdout:5/593: mknod d7/d23/d5f/d1b/cc2 0 2024-03-24T10:45:58.038 INFO:tasks.workunit.client.0.smithi094.stdout:6/573: dread d0/f49 [0,4194304] 0 2024-03-24T10:45:58.038 INFO:tasks.workunit.client.0.smithi094.stdout:2/508: rename d18/d1e/d5a/d61 to d18/d66/d90/d9a 0 2024-03-24T10:45:58.039 INFO:tasks.workunit.client.0.smithi094.stdout:3/486: mknod d0/db/dc/cac 0 2024-03-24T10:45:58.040 INFO:tasks.workunit.client.0.smithi094.stdout:9/612: getdents df/d2d/d1f/d11/d43 0 2024-03-24T10:45:58.040 INFO:tasks.workunit.client.0.smithi094.stdout:8/612: mknod df/d25/d28/d31/d47/d22/ccf 0 2024-03-24T10:45:58.041 INFO:tasks.workunit.client.0.smithi094.stdout:5/594: symlink d7/d23/d5f/d9/d48/d51/lc3 0 2024-03-24T10:45:58.042 INFO:tasks.workunit.client.0.smithi094.stdout:6/574: mknod d0/d78/da/d23/d34/d4e/cb1 0 2024-03-24T10:45:58.042 INFO:tasks.workunit.client.0.smithi094.stdout:2/509: symlink d18/d48/l9b 0 2024-03-24T10:45:58.042 INFO:tasks.workunit.client.0.smithi094.stdout:2/510: chown d18/d1e/d5a/f5e 2 1 2024-03-24T10:45:58.044 INFO:tasks.workunit.client.0.smithi094.stdout:9/613: symlink df/d2d/d1f/d58/d5a/d61/d87/lbd 0 2024-03-24T10:45:58.044 INFO:tasks.workunit.client.0.smithi094.stdout:4/589: dwrite d0/d22/f62 [0,4194304] 0 2024-03-24T10:45:58.044 INFO:tasks.workunit.client.0.smithi094.stdout:8/613: mkdir df/d25/d28/d31/d47/d66/da3/da8/dd0 0 2024-03-24T10:45:58.045 INFO:tasks.workunit.client.0.smithi094.stdout:5/595: creat d7/d23/d5f/d9/fc4 x:0 0 0 2024-03-24T10:45:58.048 INFO:tasks.workunit.client.0.smithi094.stdout:3/487: rmdir d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69 39 2024-03-24T10:45:58.048 INFO:tasks.workunit.client.0.smithi094.stdout:5/596: dread d7/d26/f84 [0,4194304] 0 2024-03-24T10:45:58.051 INFO:tasks.workunit.client.0.smithi094.stdout:4/590: unlink d0/d22/de/d28/d2b/d7a/fa9 0 2024-03-24T10:45:58.051 INFO:tasks.workunit.client.0.smithi094.stdout:4/591: chown d0/d22/de/d28/f7f 2 1 2024-03-24T10:45:58.053 INFO:tasks.workunit.client.0.smithi094.stdout:8/614: unlink df/d25/d28/f6b 0 2024-03-24T10:45:58.053 INFO:tasks.workunit.client.0.smithi094.stdout:8/615: chown df/d25/l74 16603385 1 2024-03-24T10:45:58.054 INFO:tasks.workunit.client.0.smithi094.stdout:6/575: getdents d0/d78/da 0 2024-03-24T10:45:58.056 INFO:tasks.workunit.client.0.smithi094.stdout:8/616: dread df/d25/d28/d31/d47/d22/d55/f6d [0,4194304] 0 2024-03-24T10:45:58.056 INFO:tasks.workunit.client.0.smithi094.stdout:8/617: stat df/d12/d5b/d63/fbc 0 2024-03-24T10:45:58.056 INFO:tasks.workunit.client.0.smithi094.stdout:8/618: truncate df/d25/f58 4633233 0 2024-03-24T10:45:58.056 INFO:tasks.workunit.client.0.smithi094.stdout:3/488: write d0/db/dc/d28/d3c/d55/d7f/d20/d23/f9a [823259,79411] 0 2024-03-24T10:45:58.056 INFO:tasks.workunit.client.0.smithi094.stdout:3/489: chown d0/db/dc/d28/d38/d6a/l84 3693 1 2024-03-24T10:45:58.061 INFO:tasks.workunit.client.0.smithi094.stdout:5/597: dread d7/dc/d12/f1f [0,4194304] 0 2024-03-24T10:45:58.061 INFO:tasks.workunit.client.0.smithi094.stdout:2/511: rmdir d18 39 2024-03-24T10:45:58.061 INFO:tasks.workunit.client.0.smithi094.stdout:2/512: stat d18/f3b 0 2024-03-24T10:45:58.062 INFO:tasks.workunit.client.0.smithi094.stdout:2/513: write d18/d1e/d65/f67 [67578,7873] 0 2024-03-24T10:45:58.065 INFO:tasks.workunit.client.0.smithi094.stdout:9/614: rmdir df/d2d 39 2024-03-24T10:45:58.065 INFO:tasks.workunit.client.0.smithi094.stdout:9/615: write df/f62 [1700779,12590] 0 2024-03-24T10:45:58.066 INFO:tasks.workunit.client.0.smithi094.stdout:6/576: symlink d0/d1a/d35/lb2 0 2024-03-24T10:45:58.066 INFO:tasks.workunit.client.0.smithi094.stdout:6/577: write d0/f1e [2039114,28741] 0 2024-03-24T10:45:58.067 INFO:tasks.workunit.client.0.smithi094.stdout:8/619: truncate df/f2b 2659806 0 2024-03-24T10:45:58.067 INFO:tasks.workunit.client.0.smithi094.stdout:8/620: creat df/fd1 x:0 0 0 2024-03-24T10:45:58.068 INFO:tasks.workunit.client.0.smithi094.stdout:4/592: dwrite d0/d5/d7d/f99 [0,4194304] 0 2024-03-24T10:45:58.070 INFO:tasks.workunit.client.0.smithi094.stdout:3/490: rename d0/db/dc/c8c to d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d4f/cad 0 2024-03-24T10:45:58.072 INFO:tasks.workunit.client.0.smithi094.stdout:5/598: symlink d7/d23/d5f/d14/d6f/lc5 0 2024-03-24T10:45:58.075 INFO:tasks.workunit.client.0.smithi094.stdout:2/514: creat d18/d1e/d65/f9c x:0 0 0 2024-03-24T10:45:58.078 INFO:tasks.workunit.client.0.smithi094.stdout:6/578: unlink d0/d30/d50/d4d/l8e 0 2024-03-24T10:45:58.078 INFO:tasks.workunit.client.0.smithi094.stdout:6/579: chown d0/d30/d50/d4d 1486519 1 2024-03-24T10:45:58.080 INFO:tasks.workunit.client.0.smithi094.stdout:8/621: symlink df/d25/d36/d98/ld2 0 2024-03-24T10:45:58.082 INFO:tasks.workunit.client.0.smithi094.stdout:3/491: dread d0/db/dc/d28/d3c/d55/d7f/d20/f24 [0,4194304] 0 2024-03-24T10:45:58.082 INFO:tasks.workunit.client.0.smithi094.stdout:3/492: chown d0/c21 478773018 1 2024-03-24T10:45:58.082 INFO:tasks.workunit.client.0.smithi094.stdout:3/493: fsync d0/db/dc/d28/d3c/d55/f70 0 2024-03-24T10:45:58.082 INFO:tasks.workunit.client.0.smithi094.stdout:5/599: creat d7/d23/d5f/d14/d6f/dbe/fc6 x:0 0 0 2024-03-24T10:45:58.082 INFO:tasks.workunit.client.0.smithi094.stdout:5/600: stat d7/d26 0 2024-03-24T10:45:58.083 INFO:tasks.workunit.client.0.smithi094.stdout:2/515: symlink d18/d66/l9d 0 2024-03-24T10:45:58.086 INFO:tasks.workunit.client.0.smithi094.stdout:8/622: rename df/d25/d28/d31/d47/fa2 to df/d8c/d8f/dc1/fd3 0 2024-03-24T10:45:58.086 INFO:tasks.workunit.client.0.smithi094.stdout:6/580: dread d0/d21/d2a/d2f/f36 [0,4194304] 0 2024-03-24T10:45:58.088 INFO:tasks.workunit.client.0.smithi094.stdout:2/516: creat d18/d7f/d88/f9e x:0 0 0 2024-03-24T10:45:58.088 INFO:tasks.workunit.client.0.smithi094.stdout:5/601: getdents d7/d23/d2b/d68 0 2024-03-24T10:45:58.089 INFO:tasks.workunit.client.0.smithi094.stdout:6/581: getdents d0/d78/da/d23 0 2024-03-24T10:45:58.095 INFO:tasks.workunit.client.0.smithi094.stdout:3/494: dread d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/f5b [0,4194304] 0 2024-03-24T10:45:58.096 INFO:tasks.workunit.client.0.smithi094.stdout:4/593: dwrite d0/d22/de/d28/f3f [0,4194304] 0 2024-03-24T10:45:58.097 INFO:tasks.workunit.client.0.smithi094.stdout:3/495: dread d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d4f/d74/f9f [0,4194304] 0 2024-03-24T10:45:58.100 INFO:tasks.workunit.client.0.smithi094.stdout:9/616: dwrite df/f3f [4194304,4194304] 0 2024-03-24T10:45:58.100 INFO:tasks.workunit.client.0.smithi094.stdout:9/617: write df/d2d/d1f/f49 [3163673,28609] 0 2024-03-24T10:45:58.101 INFO:tasks.workunit.client.0.smithi094.stdout:2/517: getdents d18/d1e/d65 0 2024-03-24T10:45:58.101 INFO:tasks.workunit.client.0.smithi094.stdout:2/518: readlink d18/d48/d56/l8e 0 2024-03-24T10:45:58.101 INFO:tasks.workunit.client.0.smithi094.stdout:2/519: stat d18/f23 0 2024-03-24T10:45:58.101 INFO:tasks.workunit.client.0.smithi094.stdout:2/520: stat d18/l4c 0 2024-03-24T10:45:58.102 INFO:tasks.workunit.client.0.smithi094.stdout:8/623: dread df/d12/d5b/d63/f9c [0,4194304] 0 2024-03-24T10:45:58.106 INFO:tasks.workunit.client.0.smithi094.stdout:5/602: dwrite d7/d23/d5f/f83 [0,4194304] 0 2024-03-24T10:45:58.106 INFO:tasks.workunit.client.0.smithi094.stdout:5/603: stat d7/d23/d5f/d1b/cc2 0 2024-03-24T10:45:58.107 INFO:tasks.workunit.client.0.smithi094.stdout:4/594: unlink d0/d22/f62 0 2024-03-24T10:45:58.114 INFO:tasks.workunit.client.0.smithi094.stdout:9/618: creat df/d2d/d1f/d58/d5a/d69/fbe x:0 0 0 2024-03-24T10:45:58.114 INFO:tasks.workunit.client.0.smithi094.stdout:9/619: fsync df/d2d/d1f/f49 0 2024-03-24T10:45:58.117 INFO:tasks.workunit.client.0.smithi094.stdout:4/595: mknod d0/cae 0 2024-03-24T10:45:58.117 INFO:tasks.workunit.client.0.smithi094.stdout:4/596: write d0/d22/de/d28/f3f [4537197,87901] 0 2024-03-24T10:45:58.117 INFO:tasks.workunit.client.0.smithi094.stdout:4/597: creat d0/d5/d1b/faf x:0 0 0 2024-03-24T10:45:58.117 INFO:tasks.workunit.client.0.smithi094.stdout:9/620: creat df/d2d/d1f/d11/fbf x:0 0 0 2024-03-24T10:45:58.120 INFO:tasks.workunit.client.0.smithi094.stdout:5/604: read d7/d23/d5f/d1b/fa4 [3003745,113796] 0 2024-03-24T10:45:58.120 INFO:tasks.workunit.client.0.smithi094.stdout:5/605: creat d7/d23/d5f/d9/d48/d51/fc7 x:0 0 0 2024-03-24T10:45:58.120 INFO:tasks.workunit.client.0.smithi094.stdout:5/606: stat d7/d23/d2b/f4b 0 2024-03-24T10:45:58.120 INFO:tasks.workunit.client.0.smithi094.stdout:5/607: creat d7/d26/d55/da9/fc8 x:0 0 0 2024-03-24T10:45:58.120 INFO:tasks.workunit.client.0.smithi094.stdout:4/598: rename d0/d5/d7d/d97/d4a/c6f to d0/d22/de/d28/d2b/d33/cb0 0 2024-03-24T10:45:58.120 INFO:tasks.workunit.client.0.smithi094.stdout:4/599: readlink d0/d1f/l9a 0 2024-03-24T10:45:58.122 INFO:tasks.workunit.client.0.smithi094.stdout:5/608: mknod d7/d26/d55/da9/cc9 0 2024-03-24T10:45:58.123 INFO:tasks.workunit.client.0.smithi094.stdout:4/600: unlink d0/d5/d7d/f88 0 2024-03-24T10:45:58.124 INFO:tasks.workunit.client.0.smithi094.stdout:5/609: dread d7/d23/d5f/d9/d48/d51/f6a [0,4194304] 0 2024-03-24T10:45:58.124 INFO:tasks.workunit.client.0.smithi094.stdout:5/610: creat d7/fca x:0 0 0 2024-03-24T10:45:58.124 INFO:tasks.workunit.client.0.smithi094.stdout:5/611: write d7/d26/f5c [1358851,23478] 0 2024-03-24T10:45:58.125 INFO:tasks.workunit.client.0.smithi094.stdout:5/612: truncate d7/d23/d5f/d9/d48/d51/fc7 817355 0 2024-03-24T10:45:58.125 INFO:tasks.workunit.client.0.smithi094.stdout:6/582: dwrite d0/d78/f29 [0,4194304] 0 2024-03-24T10:45:58.125 INFO:tasks.workunit.client.0.smithi094.stdout:8/624: dwrite df/d25/d28/d31/f40 [0,4194304] 0 2024-03-24T10:45:58.126 INFO:tasks.workunit.client.0.smithi094.stdout:8/625: write df/d25/d28/fa7 [625744,115958] 0 2024-03-24T10:45:58.126 INFO:tasks.workunit.client.0.smithi094.stdout:8/626: fdatasync df/d12/d5b/d63/f9c 0 2024-03-24T10:45:58.126 INFO:tasks.workunit.client.0.smithi094.stdout:5/613: write d7/d23/d2b/d68/d81/fb3 [154851,24164] 0 2024-03-24T10:45:58.126 INFO:tasks.workunit.client.0.smithi094.stdout:5/614: write d7/d26/faa [782440,92137] 0 2024-03-24T10:45:58.126 INFO:tasks.workunit.client.0.smithi094.stdout:5/615: chown d7/dc/fa2 2 1 2024-03-24T10:45:58.126 INFO:tasks.workunit.client.0.smithi094.stdout:4/601: getdents d0/d34/d73 0 2024-03-24T10:45:58.132 INFO:tasks.workunit.client.0.smithi094.stdout:9/621: dwrite df/d2d/d1f/d11/f4f [0,4194304] 0 2024-03-24T10:45:58.132 INFO:tasks.workunit.client.0.smithi094.stdout:9/622: chown df/d2d/f83 31 1 2024-03-24T10:45:58.132 INFO:tasks.workunit.client.0.smithi094.stdout:8/627: creat df/d8c/d8f/dc1/fd4 x:0 0 0 2024-03-24T10:45:58.134 INFO:tasks.workunit.client.0.smithi094.stdout:4/602: creat d0/d22/de/d28/d2b/d33/fb1 x:0 0 0 2024-03-24T10:45:58.136 INFO:tasks.workunit.client.0.smithi094.stdout:6/583: rename d0/d30/d50/d4d/d6b/d86 to d0/d30/d50/d4d/db3 0 2024-03-24T10:45:58.139 INFO:tasks.workunit.client.0.smithi094.stdout:9/623: creat df/d2d/d1f/d58/d5a/d61/fc0 x:0 0 0 2024-03-24T10:45:58.139 INFO:tasks.workunit.client.0.smithi094.stdout:9/624: write df/d2d/d1f/d58/d5a/d69/fbe [5928,51754] 0 2024-03-24T10:45:58.140 INFO:tasks.workunit.client.0.smithi094.stdout:8/628: symlink df/d25/d28/d31/d47/d22/d55/ld5 0 2024-03-24T10:45:58.140 INFO:tasks.workunit.client.0.smithi094.stdout:8/629: chown df/d25/d36/d98 20220 1 2024-03-24T10:45:58.141 INFO:tasks.workunit.client.0.smithi094.stdout:5/616: getdents d7/d23/d5f/d1b 0 2024-03-24T10:45:58.143 INFO:tasks.workunit.client.0.smithi094.stdout:4/603: write d0/d22/de/d28/f3a [2909083,15164] 0 2024-03-24T10:45:58.143 INFO:tasks.workunit.client.0.smithi094.stdout:6/584: truncate d0/d78/da/d23/f26 1409806 0 2024-03-24T10:45:58.143 INFO:tasks.workunit.client.0.smithi094.stdout:6/585: write d0/d78/f44 [1515880,113672] 0 2024-03-24T10:45:58.143 INFO:tasks.workunit.client.0.smithi094.stdout:6/586: write d0/d78/d66/d7a/f8b [89816,104354] 0 2024-03-24T10:45:58.144 INFO:tasks.workunit.client.0.smithi094.stdout:8/630: unlink df/d25/d28/d31/f72 0 2024-03-24T10:45:58.144 INFO:tasks.workunit.client.0.smithi094.stdout:8/631: creat df/d25/d28/d31/fd6 x:0 0 0 2024-03-24T10:45:58.144 INFO:tasks.workunit.client.0.smithi094.stdout:8/632: truncate df/d12/d5b/d63/fc7 589959 0 2024-03-24T10:45:58.145 INFO:tasks.workunit.client.0.smithi094.stdout:5/617: creat d7/d23/d2b/fcb x:0 0 0 2024-03-24T10:45:58.147 INFO:tasks.workunit.client.0.smithi094.stdout:9/625: dread df/d2d/d1f/d58/d82/fa0 [0,4194304] 0 2024-03-24T10:45:58.149 INFO:tasks.workunit.client.0.smithi094.stdout:5/618: symlink d7/d23/d2b/d68/d81/lcc 0 2024-03-24T10:45:58.153 INFO:tasks.workunit.client.0.smithi094.stdout:9/626: creat df/d2d/d1f/d7e/fc1 x:0 0 0 2024-03-24T10:45:58.154 INFO:tasks.workunit.client.0.smithi094.stdout:4/604: rename d0/d1f to d0/d34/db2 0 2024-03-24T10:45:58.155 INFO:tasks.workunit.client.0.smithi094.stdout:4/605: fdatasync d0/d34/d73/d93/fa3 0 2024-03-24T10:45:58.155 INFO:tasks.workunit.client.0.smithi094.stdout:4/606: stat d0/d34/f48 0 2024-03-24T10:45:58.158 INFO:tasks.workunit.client.0.smithi094.stdout:8/633: rename df/d25/d28/d31/d3c/cca to df/d8c/d8f/dc1/cd7 0 2024-03-24T10:45:58.158 INFO:tasks.workunit.client.0.smithi094.stdout:8/634: chown df/d25/d28/d31/d47/f69 985410 1 2024-03-24T10:45:58.162 INFO:tasks.workunit.client.0.smithi094.stdout:9/627: rename df/d2d/d1f/d11/d43/f68 to df/d2d/d1f/d58/d5a/d61/fc2 0 2024-03-24T10:45:58.162 INFO:tasks.workunit.client.0.smithi094.stdout:9/628: write df/d2d/d1f/d58/d5a/d69/f78 [669642,61505] 0 2024-03-24T10:45:58.163 INFO:tasks.workunit.client.0.smithi094.stdout:6/587: dwrite d0/d78/da/d23/f9b [0,4194304] 0 2024-03-24T10:45:58.164 INFO:tasks.workunit.client.0.smithi094.stdout:6/588: read d0/d78/f19 [1174483,113849] 0 2024-03-24T10:45:58.164 INFO:tasks.workunit.client.0.smithi094.stdout:6/589: chown d0/d78/da/f2e 2158 1 2024-03-24T10:45:58.165 INFO:tasks.workunit.client.0.smithi094.stdout:6/590: truncate d0/d30/d50/f81 608230 0 2024-03-24T10:45:58.168 INFO:tasks.workunit.client.0.smithi094.stdout:5/619: dwrite d7/dc/f19 [0,4194304] 0 2024-03-24T10:45:58.168 INFO:tasks.workunit.client.0.smithi094.stdout:5/620: truncate d7/d23/d5f/d14/f9a 442884 0 2024-03-24T10:45:58.169 INFO:tasks.workunit.client.0.smithi094.stdout:9/629: creat df/d2d/d1f/d58/d5a/d61/d87/fc3 x:0 0 0 2024-03-24T10:45:58.173 INFO:tasks.workunit.client.0.smithi094.stdout:6/591: dread d0/d21/f13 [0,4194304] 0 2024-03-24T10:45:58.174 INFO:tasks.workunit.client.0.smithi094.stdout:5/621: truncate d7/d23/d5f/d14/d9f/fbc 24606 0 2024-03-24T10:45:58.174 INFO:tasks.workunit.client.0.smithi094.stdout:5/622: creat d7/d26/d55/fcd x:0 0 0 2024-03-24T10:45:58.175 INFO:tasks.workunit.client.0.smithi094.stdout:5/623: chown d7/dc/fa2 1031574 1 2024-03-24T10:45:58.175 INFO:tasks.workunit.client.0.smithi094.stdout:5/624: chown d7/dc/lb1 265657 1 2024-03-24T10:45:58.175 INFO:tasks.workunit.client.0.smithi094.stdout:0/531: sync 2024-03-24T10:45:58.180 INFO:tasks.workunit.client.0.smithi094.stdout:5/625: mkdir d7/d26/dce 0 2024-03-24T10:45:58.182 INFO:tasks.workunit.client.0.smithi094.stdout:6/592: getdents d0/d78/da/d23/d34 0 2024-03-24T10:45:58.182 INFO:tasks.workunit.client.0.smithi094.stdout:6/593: creat d0/d30/d50/fb4 x:0 0 0 2024-03-24T10:45:58.182 INFO:tasks.workunit.client.0.smithi094.stdout:6/594: read d0/d78/da/d23/f26 [1402736,57268] 0 2024-03-24T10:45:58.183 INFO:tasks.workunit.client.0.smithi094.stdout:5/626: symlink d7/d23/d5f/lcf 0 2024-03-24T10:45:58.184 INFO:tasks.workunit.client.0.smithi094.stdout:0/532: rmdir d14/d22/d28/d2b/d68 39 2024-03-24T10:45:58.187 INFO:tasks.workunit.client.0.smithi094.stdout:6/595: unlink d0/d78/c15 0 2024-03-24T10:45:58.187 INFO:tasks.workunit.client.0.smithi094.stdout:4/607: dwrite d0/d5/d1b/d4f/fa7 [0,4194304] 0 2024-03-24T10:45:58.187 INFO:tasks.workunit.client.0.smithi094.stdout:8/635: dwrite df/d8c/d8f/dc1/fd4 [0,4194304] 0 2024-03-24T10:45:58.187 INFO:tasks.workunit.client.0.smithi094.stdout:9/630: dwrite df/d2d/f8d [0,4194304] 0 2024-03-24T10:45:58.188 INFO:tasks.workunit.client.0.smithi094.stdout:5/627: mknod d7/d23/d5f/d14/cd0 0 2024-03-24T10:45:58.190 INFO:tasks.workunit.client.0.smithi094.stdout:0/533: unlink d14/d18/d85/caf 0 2024-03-24T10:45:58.199 INFO:tasks.workunit.client.0.smithi094.stdout:8/636: dread df/d25/d28/d31/d47/f85 [0,4194304] 0 2024-03-24T10:45:58.199 INFO:tasks.workunit.client.0.smithi094.stdout:8/637: truncate df/f1b 3671842 0 2024-03-24T10:45:58.199 INFO:tasks.workunit.client.0.smithi094.stdout:9/631: creat df/d2d/d34/d2c/d74/fc4 x:0 0 0 2024-03-24T10:45:58.201 INFO:tasks.workunit.client.0.smithi094.stdout:5/628: rename d7/d23/d5f/d14/d36/fbd to d7/d23/d5f/d14/d6f/dbe/fd1 0 2024-03-24T10:45:58.202 INFO:tasks.workunit.client.0.smithi094.stdout:8/638: link df/d25/d36/d3f/f92 df/d25/d28/d31/d47/d66/da3/da8/fd8 0 2024-03-24T10:45:58.202 INFO:tasks.workunit.client.0.smithi094.stdout:8/639: dread - df/d25/daf/fc9 zero size 2024-03-24T10:45:58.204 INFO:tasks.workunit.client.0.smithi094.stdout:4/608: dwrite d0/d22/de/d28/f59 [0,4194304] 0 2024-03-24T10:45:58.204 INFO:tasks.workunit.client.0.smithi094.stdout:9/632: mkdir df/d2d/d34/d2c/d74/dc5 0 2024-03-24T10:45:58.206 INFO:tasks.workunit.client.0.smithi094.stdout:8/640: mkdir df/d12/d5b/d63/d75/dd9 0 2024-03-24T10:45:58.206 INFO:tasks.workunit.client.0.smithi094.stdout:8/641: creat df/d25/daf/fda x:0 0 0 2024-03-24T10:45:58.206 INFO:tasks.workunit.client.0.smithi094.stdout:8/642: chown df/d12/d5b/d63/fc7 55 1 2024-03-24T10:45:58.207 INFO:tasks.workunit.client.0.smithi094.stdout:4/609: link d0/d34/db2/faa d0/d22/de/d28/d2b/d7a/fb3 0 2024-03-24T10:45:58.211 INFO:tasks.workunit.client.0.smithi094.stdout:8/643: dread df/d25/d28/d31/f32 [0,4194304] 0 2024-03-24T10:45:58.217 INFO:tasks.workunit.client.0.smithi094.stdout:0/534: dwrite d14/d18/f40 [0,4194304] 0 2024-03-24T10:45:58.221 INFO:tasks.workunit.client.0.smithi094.stdout:0/535: creat d14/d22/d5b/fbd x:0 0 0 2024-03-24T10:45:58.221 INFO:tasks.workunit.client.0.smithi094.stdout:1/453: sync 2024-03-24T10:45:58.221 INFO:tasks.workunit.client.0.smithi094.stdout:1/454: chown d4/d36/f70 730 1 2024-03-24T10:45:58.221 INFO:tasks.workunit.client.0.smithi094.stdout:0/536: creat d14/d22/d28/fbe x:0 0 0 2024-03-24T10:45:58.223 INFO:tasks.workunit.client.0.smithi094.stdout:0/537: rename d14/d22/d28/d2b/l61 to d14/lbf 0 2024-03-24T10:45:58.224 INFO:tasks.workunit.client.0.smithi094.stdout:0/538: mkdir d14/d22/d28/d2b/dc0 0 2024-03-24T10:45:58.230 INFO:tasks.workunit.client.0.smithi094.stdout:8/644: dwrite df/d8c/d8f/fb9 [0,4194304] 0 2024-03-24T10:45:58.230 INFO:tasks.workunit.client.0.smithi094.stdout:4/610: dwrite d0/d22/de/d28/d2b/d7a/fb3 [0,4194304] 0 2024-03-24T10:45:58.230 INFO:tasks.workunit.client.0.smithi094.stdout:8/645: dread - df/d8c/fc4 zero size 2024-03-24T10:45:58.230 INFO:tasks.workunit.client.0.smithi094.stdout:8/646: chown df/d25/d28/d31/d47/f4a 1767 1 2024-03-24T10:45:58.235 INFO:tasks.workunit.client.0.smithi094.stdout:0/539: dread d14/d18/f54 [0,4194304] 0 2024-03-24T10:45:58.236 INFO:tasks.workunit.client.0.smithi094.stdout:0/540: creat d14/d18/fc1 x:0 0 0 2024-03-24T10:45:58.236 INFO:tasks.workunit.client.0.smithi094.stdout:0/541: creat d14/d18/fc2 x:0 0 0 2024-03-24T10:45:58.239 INFO:tasks.workunit.client.0.smithi094.stdout:0/542: rmdir d14/d22/d28/da2 39 2024-03-24T10:45:58.239 INFO:tasks.workunit.client.0.smithi094.stdout:0/543: creat d14/d22/d7d/fc3 x:0 0 0 2024-03-24T10:45:58.251 INFO:tasks.workunit.client.0.smithi094.stdout:0/544: dwrite d14/f75 [0,4194304] 0 2024-03-24T10:45:58.251 INFO:tasks.workunit.client.0.smithi094.stdout:0/545: chown d14/d22/d28/d2b/d41/l6a 3 1 2024-03-24T10:45:58.262 INFO:tasks.workunit.client.0.smithi094.stdout:8/647: getdents df/d12/d5b/d63/d75 0 2024-03-24T10:45:58.262 INFO:tasks.workunit.client.0.smithi094.stdout:8/648: creat df/d25/d36/d3f/fdb x:0 0 0 2024-03-24T10:45:58.262 INFO:tasks.workunit.client.0.smithi094.stdout:8/649: truncate df/d25/d28/d31/f40 4302578 0 2024-03-24T10:45:58.262 INFO:tasks.workunit.client.0.smithi094.stdout:8/650: chown df/d25/daf/fb0 167299434 1 2024-03-24T10:45:58.263 INFO:tasks.workunit.client.0.smithi094.stdout:8/651: symlink df/d12/ldc 0 2024-03-24T10:45:58.264 INFO:tasks.workunit.client.0.smithi094.stdout:8/652: write df/d25/d28/d31/d47/d66/da3/da8/fd8 [368931,29809] 0 2024-03-24T10:45:58.264 INFO:tasks.workunit.client.0.smithi094.stdout:8/653: mknod df/d25/d36/d3f/cdd 0 2024-03-24T10:45:58.265 INFO:tasks.workunit.client.0.smithi094.stdout:1/455: dwrite d4/f2a [0,4194304] 0 2024-03-24T10:45:58.265 INFO:tasks.workunit.client.0.smithi094.stdout:1/456: write d4/d9/db/d39/d75/f86 [783980,27522] 0 2024-03-24T10:45:58.271 INFO:tasks.workunit.client.0.smithi094.stdout:0/546: dwrite d14/d18/f1d [0,4194304] 0 2024-03-24T10:45:58.275 INFO:tasks.workunit.client.0.smithi094.stdout:0/547: rename d14/d22/d28/d48/d7b/f94 to d14/d22/d32/d70/fc4 0 2024-03-24T10:45:58.276 INFO:tasks.workunit.client.0.smithi094.stdout:0/548: creat d14/d22/d28/d2b/d68/fc5 x:0 0 0 2024-03-24T10:45:58.276 INFO:tasks.workunit.client.0.smithi094.stdout:0/549: chown d14/d22/d28/d48/f4a 2449 1 2024-03-24T10:45:58.276 INFO:tasks.workunit.client.0.smithi094.stdout:0/550: fdatasync d14/d22/d7d/fa6 0 2024-03-24T10:45:58.279 INFO:tasks.workunit.client.0.smithi094.stdout:8/654: dread df/d12/d5b/d63/d75/f7b [0,4194304] 0 2024-03-24T10:45:58.281 INFO:tasks.workunit.client.0.smithi094.stdout:1/457: dread d4/d9/f25 [0,4194304] 0 2024-03-24T10:45:58.281 INFO:tasks.workunit.client.0.smithi094.stdout:1/458: getdents d4/d9/db/d85 0 2024-03-24T10:45:58.282 INFO:tasks.workunit.client.0.smithi094.stdout:0/551: rmdir d14/d22/d28 39 2024-03-24T10:45:58.288 INFO:tasks.workunit.client.0.smithi094.stdout:0/552: dread fa [0,4194304] 0 2024-03-24T10:45:58.289 INFO:tasks.workunit.client.0.smithi094.stdout:0/553: mkdir d14/d22/d28/da2/dc6 0 2024-03-24T10:45:58.291 INFO:tasks.workunit.client.0.smithi094.stdout:0/554: unlink d14/f75 0 2024-03-24T10:45:58.291 INFO:tasks.workunit.client.0.smithi094.stdout:0/555: dread - d14/d22/d28/d2b/d68/fc5 zero size 2024-03-24T10:45:58.295 INFO:tasks.workunit.client.0.smithi094.stdout:7/574: sync 2024-03-24T10:45:58.295 INFO:tasks.workunit.client.0.smithi094.stdout:3/496: sync 2024-03-24T10:45:58.295 INFO:tasks.workunit.client.0.smithi094.stdout:7/575: write d1/d9/d16/d21/d51/d83/f90 [991640,11183] 0 2024-03-24T10:45:58.296 INFO:tasks.workunit.client.0.smithi094.stdout:3/497: getdents d0/db/dc/d28/d3c/d55/d7f/d20 0 2024-03-24T10:45:58.296 INFO:tasks.workunit.client.0.smithi094.stdout:1/459: dwrite d4/d9/db/d31/d3c/f7c [0,4194304] 0 2024-03-24T10:45:58.297 INFO:tasks.workunit.client.0.smithi094.stdout:2/521: sync 2024-03-24T10:45:58.297 INFO:tasks.workunit.client.0.smithi094.stdout:2/522: write d18/d1e/d5a/f5e [2727964,24699] 0 2024-03-24T10:45:58.297 INFO:tasks.workunit.client.0.smithi094.stdout:2/523: getdents d18/d1e/d65 0 2024-03-24T10:45:58.298 INFO:tasks.workunit.client.0.smithi094.stdout:7/576: mknod d1/db4/cbf 0 2024-03-24T10:45:58.298 INFO:tasks.workunit.client.0.smithi094.stdout:7/577: fsync d1/d9/d16/d21/d51/d83/d39/f86 0 2024-03-24T10:45:58.299 INFO:tasks.workunit.client.0.smithi094.stdout:7/578: chown d1/d9/d14/l4a 1304841235 1 2024-03-24T10:45:58.299 INFO:tasks.workunit.client.0.smithi094.stdout:7/579: chown d1/d9/d14/d34 13557115 1 2024-03-24T10:45:58.307 INFO:tasks.workunit.client.0.smithi094.stdout:2/524: dread d18/d1e/d34/f46 [0,4194304] 0 2024-03-24T10:45:58.307 INFO:tasks.workunit.client.0.smithi094.stdout:7/580: write d1/d6c/f88 [1391724,40974] 0 2024-03-24T10:45:58.308 INFO:tasks.workunit.client.0.smithi094.stdout:1/460: dread d4/d9/de/d6f/d62/d99/f9e [0,4194304] 0 2024-03-24T10:45:58.308 INFO:tasks.workunit.client.0.smithi094.stdout:1/461: chown d4/d9/db/d31/l56 0 1 2024-03-24T10:45:58.308 INFO:tasks.workunit.client.0.smithi094.stdout:2/525: symlink d18/d1e/d5a/l9f 0 2024-03-24T10:45:58.308 INFO:tasks.workunit.client.0.smithi094.stdout:2/526: write d18/f3f [1569387,83047] 0 2024-03-24T10:45:58.311 INFO:tasks.workunit.client.0.smithi094.stdout:1/462: symlink d4/d9/de/d3d/la1 0 2024-03-24T10:45:58.312 INFO:tasks.workunit.client.0.smithi094.stdout:2/527: mkdir d18/d7f/d88/da0 0 2024-03-24T10:45:58.312 INFO:tasks.workunit.client.0.smithi094.stdout:2/528: write d18/d48/d56/f58 [599772,44907] 0 2024-03-24T10:45:58.312 INFO:tasks.workunit.client.0.smithi094.stdout:2/529: stat d18/d7f/d88/da0 0 2024-03-24T10:45:58.315 INFO:tasks.workunit.client.0.smithi094.stdout:0/556: dwrite d14/d18/d85/fa3 [4194304,4194304] 0 2024-03-24T10:45:58.315 INFO:tasks.workunit.client.0.smithi094.stdout:0/557: chown d14/d22/d28/d2b/f3b 3 1 2024-03-24T10:45:58.317 INFO:tasks.workunit.client.0.smithi094.stdout:7/581: write d1/d9/d25/d3d/f98 [6209205,59041] 0 2024-03-24T10:45:58.318 INFO:tasks.workunit.client.0.smithi094.stdout:0/558: mknod d14/d22/d28/d48/d7b/cc7 0 2024-03-24T10:45:58.318 INFO:tasks.workunit.client.0.smithi094.stdout:0/559: write f3 [2745419,296] 0 2024-03-24T10:45:58.318 INFO:tasks.workunit.client.0.smithi094.stdout:0/560: dread - d14/d22/d7d/fc3 zero size 2024-03-24T10:45:58.319 INFO:tasks.workunit.client.0.smithi094.stdout:1/463: dread d4/d9/d37/f4c [0,4194304] 0 2024-03-24T10:45:58.319 INFO:tasks.workunit.client.0.smithi094.stdout:3/498: dwrite d0/db/dc/d28/d3c/d7c/f9d [0,4194304] 0 2024-03-24T10:45:58.321 INFO:tasks.workunit.client.0.smithi094.stdout:7/582: link d1/d6c/c70 d1/d9/d16/d21/d51/d83/d39/cc0 0 2024-03-24T10:45:58.321 INFO:tasks.workunit.client.0.smithi094.stdout:7/583: fsync d1/d9/d25/d3d/f6d 0 2024-03-24T10:45:58.321 INFO:tasks.workunit.client.0.smithi094.stdout:7/584: dread - d1/d9/d14/d8f/fa6 zero size 2024-03-24T10:45:58.321 INFO:tasks.workunit.client.0.smithi094.stdout:7/585: truncate d1/d9/d25/d3d/f55 5351700 0 2024-03-24T10:45:58.321 INFO:tasks.workunit.client.0.smithi094.stdout:7/586: read - d1/d6c/d80/f69 zero size 2024-03-24T10:45:58.321 INFO:tasks.workunit.client.0.smithi094.stdout:7/587: dread - d1/d9/d14/d8f/f91 zero size 2024-03-24T10:45:58.321 INFO:tasks.workunit.client.0.smithi094.stdout:7/588: fsync d1/d9/d25/d3d/f73 0 2024-03-24T10:45:58.321 INFO:tasks.workunit.client.0.smithi094.stdout:7/589: creat d1/d9/d16/fc1 x:0 0 0 2024-03-24T10:45:58.322 INFO:tasks.workunit.client.0.smithi094.stdout:0/561: mknod d14/d22/d32/d70/cc8 0 2024-03-24T10:45:58.322 INFO:tasks.workunit.client.0.smithi094.stdout:0/562: fdatasync d14/d22/d28/d2b/d68/f72 0 2024-03-24T10:45:58.322 INFO:tasks.workunit.client.0.smithi094.stdout:0/563: chown d14/d22/d28/d2b/lb6 5 1 2024-03-24T10:45:58.324 INFO:tasks.workunit.client.0.smithi094.stdout:1/464: symlink d4/d9/db/d39/la2 0 2024-03-24T10:45:58.326 INFO:tasks.workunit.client.0.smithi094.stdout:3/499: creat d0/db/dc/d28/d3c/d55/d7f/d20/fae x:0 0 0 2024-03-24T10:45:58.328 INFO:tasks.workunit.client.0.smithi094.stdout:7/590: creat d1/d9/d14/d8f/fc2 x:0 0 0 2024-03-24T10:45:58.329 INFO:tasks.workunit.client.0.smithi094.stdout:0/564: rmdir d14/d22/d28/d48/d7b 39 2024-03-24T10:45:58.329 INFO:tasks.workunit.client.0.smithi094.stdout:6/596: sync 2024-03-24T10:45:58.330 INFO:tasks.workunit.client.0.smithi094.stdout:2/530: dwrite f15 [0,4194304] 0 2024-03-24T10:45:58.330 INFO:tasks.workunit.client.0.smithi094.stdout:7/591: unlink d1/d9/d16/d21/d51/d83/d39/f86 0 2024-03-24T10:45:58.332 INFO:tasks.workunit.client.0.smithi094.stdout:6/597: write d0/d21/d2a/d2f/fa6 [514442,89953] 0 2024-03-24T10:45:58.334 INFO:tasks.workunit.client.0.smithi094.stdout:3/500: dread d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/f7b [0,4194304] 0 2024-03-24T10:45:58.334 INFO:tasks.workunit.client.0.smithi094.stdout:7/592: mkdir d1/d9/dc3 0 2024-03-24T10:45:58.334 INFO:tasks.workunit.client.0.smithi094.stdout:7/593: write d1/d9/d25/d3d/f5a [247255,91208] 0 2024-03-24T10:45:58.335 INFO:tasks.workunit.client.0.smithi094.stdout:6/598: mkdir d0/da9/db5 0 2024-03-24T10:45:58.336 INFO:tasks.workunit.client.0.smithi094.stdout:2/531: symlink d18/d1e/d34/la1 0 2024-03-24T10:45:58.336 INFO:tasks.workunit.client.0.smithi094.stdout:2/532: write d18/f3f [2494152,43808] 0 2024-03-24T10:45:58.337 INFO:tasks.workunit.client.0.smithi094.stdout:3/501: symlink d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/laf 0 2024-03-24T10:45:58.337 INFO:tasks.workunit.client.0.smithi094.stdout:3/502: creat d0/db/dc/d28/d3c/d7c/fb0 x:0 0 0 2024-03-24T10:45:58.337 INFO:tasks.workunit.client.0.smithi094.stdout:6/599: rmdir d0/d21 39 2024-03-24T10:45:58.339 INFO:tasks.workunit.client.0.smithi094.stdout:2/533: rmdir d18/d66/d90/d9a 39 2024-03-24T10:45:58.339 INFO:tasks.workunit.client.0.smithi094.stdout:2/534: truncate d18/d48/f52 645626 0 2024-03-24T10:45:58.342 INFO:tasks.workunit.client.0.smithi094.stdout:6/600: rename d0/d30/d50/d4d/db3 to d0/d78/da/d77/db6 0 2024-03-24T10:45:58.342 INFO:tasks.workunit.client.0.smithi094.stdout:6/601: chown d0/d78/c64 12877 1 2024-03-24T10:45:58.342 INFO:tasks.workunit.client.0.smithi094.stdout:3/503: write d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/f5b [4153726,5596] 0 2024-03-24T10:45:58.343 INFO:tasks.workunit.client.0.smithi094.stdout:2/535: mkdir d18/d7f/d88/da2 0 2024-03-24T10:45:58.347 INFO:tasks.workunit.client.0.smithi094.stdout:3/504: rename d0/db/dc/d28/d50/d5e/f7e to d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/d73/fb1 0 2024-03-24T10:45:58.347 INFO:tasks.workunit.client.0.smithi094.stdout:3/505: readlink d0/db/l35 0 2024-03-24T10:45:58.348 INFO:tasks.workunit.client.0.smithi094.stdout:6/602: write d0/d1a/f6f [1116899,4840] 0 2024-03-24T10:45:58.348 INFO:tasks.workunit.client.0.smithi094.stdout:6/603: write d0/d30/d50/f81 [860026,21395] 0 2024-03-24T10:45:58.349 INFO:tasks.workunit.client.0.smithi094.stdout:2/536: rename d18/d1e/d34/la1 to d18/d7f/d88/d8a/la3 0 2024-03-24T10:45:58.350 INFO:tasks.workunit.client.0.smithi094.stdout:3/506: creat d0/db/dc/d28/d3c/d55/fb2 x:0 0 0 2024-03-24T10:45:58.352 INFO:tasks.workunit.client.0.smithi094.stdout:0/565: dwrite d14/d22/d7d/fa4 [0,4194304] 0 2024-03-24T10:45:58.354 INFO:tasks.workunit.client.0.smithi094.stdout:3/507: creat d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/fb3 x:0 0 0 2024-03-24T10:45:58.357 INFO:tasks.workunit.client.0.smithi094.stdout:6/604: write d0/d78/f5a [3366239,22229] 0 2024-03-24T10:45:58.357 INFO:tasks.workunit.client.0.smithi094.stdout:3/508: mknod d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/cb4 0 2024-03-24T10:45:58.357 INFO:tasks.workunit.client.0.smithi094.stdout:3/509: dread - d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d86/fa3 zero size 2024-03-24T10:45:58.359 INFO:tasks.workunit.client.0.smithi094.stdout:6/605: creat d0/d78/da/d77/db6/fb7 x:0 0 0 2024-03-24T10:45:58.361 INFO:tasks.workunit.client.0.smithi094.stdout:3/510: unlink d0/db/dc/f12 0 2024-03-24T10:45:58.367 INFO:tasks.workunit.client.0.smithi094.stdout:2/537: dwrite d18/d1e/f25 [0,4194304] 0 2024-03-24T10:45:58.369 INFO:tasks.workunit.client.0.smithi094.stdout:2/538: truncate d18/d1e/f53 3038690 0 2024-03-24T10:45:58.371 INFO:tasks.workunit.client.0.smithi094.stdout:2/539: rename d18/d1e/d5a to d18/d7f/d88/da2/da4 0 2024-03-24T10:45:58.372 INFO:tasks.workunit.client.0.smithi094.stdout:2/540: creat d18/d66/fa5 x:0 0 0 2024-03-24T10:45:58.373 INFO:tasks.workunit.client.0.smithi094.stdout:3/511: write d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/f93 [1395063,110128] 0 2024-03-24T10:45:58.374 INFO:tasks.workunit.client.0.smithi094.stdout:2/541: unlink d18/d48/d56/d73/f8f 0 2024-03-24T10:45:58.381 INFO:tasks.workunit.client.0.smithi094.stdout:6/606: dwrite d0/d30/fa0 [0,4194304] 0 2024-03-24T10:45:58.385 INFO:tasks.workunit.client.0.smithi094.stdout:5/629: sync 2024-03-24T10:45:58.385 INFO:tasks.workunit.client.0.smithi094.stdout:4/611: sync 2024-03-24T10:45:58.385 INFO:tasks.workunit.client.0.smithi094.stdout:9/633: sync 2024-03-24T10:45:58.386 INFO:tasks.workunit.client.0.smithi094.stdout:6/607: dread d0/d78/da/d23/f6c [0,4194304] 0 2024-03-24T10:45:58.387 INFO:tasks.workunit.client.0.smithi094.stdout:6/608: truncate d0/d21/f3a 813130 0 2024-03-24T10:45:58.396 INFO:tasks.workunit.client.0.smithi094.stdout:5/630: dread d7/d23/d5f/f83 [0,4194304] 0 2024-03-24T10:45:58.396 INFO:tasks.workunit.client.0.smithi094.stdout:9/634: dread df/d2d/d1f/d58/d5a/f6f [0,4194304] 0 2024-03-24T10:45:58.397 INFO:tasks.workunit.client.0.smithi094.stdout:5/631: getdents d7/d23/d2b/d68/d63 0 2024-03-24T10:45:58.397 INFO:tasks.workunit.client.0.smithi094.stdout:5/632: creat d7/d26/d55/da9/fd2 x:0 0 0 2024-03-24T10:45:58.397 INFO:tasks.workunit.client.0.smithi094.stdout:5/633: truncate d7/faf 1027414 0 2024-03-24T10:45:58.398 INFO:tasks.workunit.client.0.smithi094.stdout:9/635: creat df/d2d/d1f/d11/d43/db1/fc6 x:0 0 0 2024-03-24T10:45:58.398 INFO:tasks.workunit.client.0.smithi094.stdout:9/636: creat df/d2d/d34/d2c/d74/fc7 x:0 0 0 2024-03-24T10:45:58.398 INFO:tasks.workunit.client.0.smithi094.stdout:9/637: read - df/d2d/d1f/d58/d5a/d61/fc0 zero size 2024-03-24T10:45:58.401 INFO:tasks.workunit.client.0.smithi094.stdout:9/638: unlink df/d2d/d34/d2c/d74/fc7 0 2024-03-24T10:45:58.402 INFO:tasks.workunit.client.0.smithi094.stdout:9/639: mkdir df/d2d/d1f/d58/d5a/d61/d87/dc8 0 2024-03-24T10:45:58.404 INFO:tasks.workunit.client.0.smithi094.stdout:9/640: rename df/d2d/d34/d2c/f8c to df/d2d/d1f/d58/d70/dab/fc9 0 2024-03-24T10:45:58.405 INFO:tasks.workunit.client.0.smithi094.stdout:3/512: dwrite d0/db/dc/d28/d38/d53/f78 [0,4194304] 0 2024-03-24T10:45:58.406 INFO:tasks.workunit.client.0.smithi094.stdout:9/641: mknod df/d2d/d1f/d58/d5a/d61/d87/cca 0 2024-03-24T10:45:58.406 INFO:tasks.workunit.client.0.smithi094.stdout:4/612: dwrite d0/d34/d73/d93/fa3 [0,4194304] 0 2024-03-24T10:45:58.406 INFO:tasks.workunit.client.0.smithi094.stdout:4/613: creat d0/d34/d73/d93/fb4 x:0 0 0 2024-03-24T10:45:58.407 INFO:tasks.workunit.client.0.smithi094.stdout:2/542: dwrite d18/d77/f78 [0,4194304] 0 2024-03-24T10:45:58.410 INFO:tasks.workunit.client.0.smithi094.stdout:2/543: fdatasync d18/d1e/d34/f43 0 2024-03-24T10:45:58.418 INFO:tasks.workunit.client.0.smithi094.stdout:5/634: dwrite d7/d23/d2b/f4b [0,4194304] 0 2024-03-24T10:45:58.418 INFO:tasks.workunit.client.0.smithi094.stdout:5/635: creat d7/d23/d5f/d14/d6f/dbe/fd3 x:0 0 0 2024-03-24T10:45:58.428 INFO:tasks.workunit.client.0.smithi094.stdout:2/544: dread f15 [0,4194304] 0 2024-03-24T10:45:58.428 INFO:tasks.workunit.client.0.smithi094.stdout:2/545: read - d18/d1e/f76 zero size 2024-03-24T10:45:58.429 INFO:tasks.workunit.client.0.smithi094.stdout:4/614: dwrite d0/d34/fac [0,4194304] 0 2024-03-24T10:45:58.432 INFO:tasks.workunit.client.0.smithi094.stdout:9/642: dwrite df/d2d/d34/d2c/f4c [0,4194304] 0 2024-03-24T10:45:58.432 INFO:tasks.workunit.client.0.smithi094.stdout:9/643: creat df/d2d/d1f/d11/fcb x:0 0 0 2024-03-24T10:45:58.432 INFO:tasks.workunit.client.0.smithi094.stdout:9/644: truncate df/d2d/d1f/d58/d5a/d61/fc2 4466025 0 2024-03-24T10:45:58.432 INFO:tasks.workunit.client.0.smithi094.stdout:5/636: dread d7/d26/f53 [4194304,4194304] 0 2024-03-24T10:45:58.433 INFO:tasks.workunit.client.0.smithi094.stdout:4/615: rename d0/d22/de/d28/d2b/d7a/fb3 to d0/d22/de/d28/d2b/d7a/fb5 0 2024-03-24T10:45:58.433 INFO:tasks.workunit.client.0.smithi094.stdout:4/616: read d0/d22/de/d28/d2b/d33/f54 [4104723,88607] 0 2024-03-24T10:45:58.434 INFO:tasks.workunit.client.0.smithi094.stdout:4/617: link d0/d5/d7d/d97/f67 d0/d22/de/d28/d3d/d8c/fb6 0 2024-03-24T10:45:58.454 INFO:tasks.workunit.client.0.smithi094.stdout:9/645: dwrite df/d2d/d34/d2c/f5f [0,4194304] 0 2024-03-24T10:45:58.454 INFO:tasks.workunit.client.0.smithi094.stdout:4/618: dwrite d0/d22/de/d28/d2b/f69 [4194304,4194304] 0 2024-03-24T10:45:58.459 INFO:tasks.workunit.client.0.smithi094.stdout:4/619: creat d0/fb7 x:0 0 0 2024-03-24T10:45:58.459 INFO:tasks.workunit.client.0.smithi094.stdout:4/620: fdatasync d0/d5/d7d/d97/d4a/f61 0 2024-03-24T10:45:58.459 INFO:tasks.workunit.client.0.smithi094.stdout:4/621: fsync d0/d22/de/d28/f6b 0 2024-03-24T10:45:58.462 INFO:tasks.workunit.client.0.smithi094.stdout:4/622: symlink d0/d22/de/d28/d2b/lb8 0 2024-03-24T10:45:58.462 INFO:tasks.workunit.client.0.smithi094.stdout:4/623: chown d0/f60 506882272 1 2024-03-24T10:45:58.462 INFO:tasks.workunit.client.0.smithi094.stdout:4/624: chown d0/d22/c1a 39959 1 2024-03-24T10:45:58.464 INFO:tasks.workunit.client.0.smithi094.stdout:8/655: sync 2024-03-24T10:45:58.464 INFO:tasks.workunit.client.0.smithi094.stdout:1/465: sync 2024-03-24T10:45:58.464 INFO:tasks.workunit.client.0.smithi094.stdout:7/594: sync 2024-03-24T10:45:58.464 INFO:tasks.workunit.client.0.smithi094.stdout:7/595: read - d1/d6c/d80/f69 zero size 2024-03-24T10:45:58.465 INFO:tasks.workunit.client.0.smithi094.stdout:1/466: creat d4/d9/db/d5d/fa3 x:0 0 0 2024-03-24T10:45:58.467 INFO:tasks.workunit.client.0.smithi094.stdout:1/467: dread d4/f41 [0,4194304] 0 2024-03-24T10:45:58.469 INFO:tasks.workunit.client.0.smithi094.stdout:8/656: dread df/d12/d5b/d63/f9c [0,4194304] 0 2024-03-24T10:45:58.469 INFO:tasks.workunit.client.0.smithi094.stdout:8/657: fdatasync df/d25/d28/f87 0 2024-03-24T10:45:58.469 INFO:tasks.workunit.client.0.smithi094.stdout:8/658: symlink df/d25/d28/d31/d47/d66/da3/db5/dbf/lde 0 2024-03-24T10:45:58.472 INFO:tasks.workunit.client.0.smithi094.stdout:9/646: dwrite df/d2d/d1f/d11/f75 [0,4194304] 0 2024-03-24T10:45:58.472 INFO:tasks.workunit.client.0.smithi094.stdout:9/647: truncate df/fbc 476352 0 2024-03-24T10:45:58.474 INFO:tasks.workunit.client.0.smithi094.stdout:9/648: creat df/d2d/d34/d2c/d97/fcc x:0 0 0 2024-03-24T10:45:58.476 INFO:tasks.workunit.client.0.smithi094.stdout:4/625: dread d0/d22/de/d28/d2b/d7a/fb5 [0,4194304] 0 2024-03-24T10:45:58.479 INFO:tasks.workunit.client.0.smithi094.stdout:7/596: write d1/d9/d14/f4b [4032946,46598] 0 2024-03-24T10:45:58.479 INFO:tasks.workunit.client.0.smithi094.stdout:7/597: chown d1/d9/d14/d8f/fa6 189 1 2024-03-24T10:45:58.482 INFO:tasks.workunit.client.0.smithi094.stdout:4/626: dread d0/d22/ff [0,4194304] 0 2024-03-24T10:45:58.483 INFO:tasks.workunit.client.0.smithi094.stdout:4/627: truncate d0/d22/de/d28/f8f 3817594 0 2024-03-24T10:45:58.483 INFO:tasks.workunit.client.0.smithi094.stdout:4/628: getdents d0/d22/de/d28/d3d/d91 0 2024-03-24T10:45:58.484 INFO:tasks.workunit.client.0.smithi094.stdout:4/629: mkdir d0/d22/de/d28/d2b/d33/da4/d95/db9 0 2024-03-24T10:45:58.484 INFO:tasks.workunit.client.0.smithi094.stdout:4/630: write d0/d5/d1b/f4e [1040436,67888] 0 2024-03-24T10:45:58.485 INFO:tasks.workunit.client.0.smithi094.stdout:4/631: unlink d0/d22/l53 0 2024-03-24T10:45:58.485 INFO:tasks.workunit.client.0.smithi094.stdout:4/632: chown d0/d34/db2/l9a 242 1 2024-03-24T10:45:58.486 INFO:tasks.workunit.client.0.smithi094.stdout:4/633: symlink d0/d34/lba 0 2024-03-24T10:45:58.486 INFO:tasks.workunit.client.0.smithi094.stdout:4/634: creat d0/d22/de/d28/fbb x:0 0 0 2024-03-24T10:45:58.486 INFO:tasks.workunit.client.0.smithi094.stdout:7/598: dread d1/d9/d96/f9a [0,4194304] 0 2024-03-24T10:45:58.487 INFO:tasks.workunit.client.0.smithi094.stdout:4/635: mknod d0/d5/d1b/d4f/d70/cbc 0 2024-03-24T10:45:58.487 INFO:tasks.workunit.client.0.smithi094.stdout:4/636: creat d0/d22/de/d28/d2b/d7a/fbd x:0 0 0 2024-03-24T10:45:58.487 INFO:tasks.workunit.client.0.smithi094.stdout:9/649: dwrite df/f56 [4194304,4194304] 0 2024-03-24T10:45:58.488 INFO:tasks.workunit.client.0.smithi094.stdout:9/650: dread - df/d2d/d34/d2c/d74/fc4 zero size 2024-03-24T10:45:58.488 INFO:tasks.workunit.client.0.smithi094.stdout:7/599: mkdir d1/d9/d16/d21/d51/dc4 0 2024-03-24T10:45:58.488 INFO:tasks.workunit.client.0.smithi094.stdout:7/600: chown d1/d9/d16/d21/d51/d83/c57 14206705 1 2024-03-24T10:45:58.488 INFO:tasks.workunit.client.0.smithi094.stdout:7/601: fsync d1/d9/f3c 0 2024-03-24T10:45:58.489 INFO:tasks.workunit.client.0.smithi094.stdout:7/602: getdents d1/d9/dc3 0 2024-03-24T10:45:58.491 INFO:tasks.workunit.client.0.smithi094.stdout:7/603: symlink d1/d9/d16/d21/d51/d83/d9d/lc5 0 2024-03-24T10:45:58.492 INFO:tasks.workunit.client.0.smithi094.stdout:4/637: read d0/d34/db2/f55 [1492495,99739] 0 2024-03-24T10:45:58.493 INFO:tasks.workunit.client.0.smithi094.stdout:4/638: symlink d0/lbe 0 2024-03-24T10:45:58.493 INFO:tasks.workunit.client.0.smithi094.stdout:4/639: chown d0/d22/de/d28/f3a 380 1 2024-03-24T10:45:58.493 INFO:tasks.workunit.client.0.smithi094.stdout:7/604: dread d1/d9/d14/d34/fa3 [0,4194304] 0 2024-03-24T10:45:58.494 INFO:tasks.workunit.client.0.smithi094.stdout:4/640: creat d0/d34/d73/fbf x:0 0 0 2024-03-24T10:45:58.495 INFO:tasks.workunit.client.0.smithi094.stdout:9/651: dread df/f56 [4194304,4194304] 0 2024-03-24T10:45:58.496 INFO:tasks.workunit.client.0.smithi094.stdout:7/605: rmdir d1/db4/d65/da1 0 2024-03-24T10:45:58.498 INFO:tasks.workunit.client.0.smithi094.stdout:4/641: rename d0/d5/l29 to d0/d5/d7d/d97/d4a/lc0 0 2024-03-24T10:45:58.499 INFO:tasks.workunit.client.0.smithi094.stdout:7/606: truncate d1/d9/d16/d21/d51/d83/f72 3587220 0 2024-03-24T10:45:58.507 INFO:tasks.workunit.client.0.smithi094.stdout:4/642: rmdir d0/d22/de 39 2024-03-24T10:45:58.507 INFO:tasks.workunit.client.0.smithi094.stdout:4/643: fsync d0/d5/d1b/f2e 0 2024-03-24T10:45:58.507 INFO:tasks.workunit.client.0.smithi094.stdout:4/644: readlink d0/l65 0 2024-03-24T10:45:58.508 INFO:tasks.workunit.client.0.smithi094.stdout:4/645: rename d0/d5/d1b/c5a to d0/d5/d1b/cc1 0 2024-03-24T10:45:58.509 INFO:tasks.workunit.client.0.smithi094.stdout:4/646: readlink d0/d22/de/d28/d2b/d33/l68 0 2024-03-24T10:45:58.510 INFO:tasks.workunit.client.0.smithi094.stdout:4/647: creat d0/d22/de/d28/d2b/d33/fc2 x:0 0 0 2024-03-24T10:45:58.511 INFO:tasks.workunit.client.0.smithi094.stdout:4/648: link d0/d34/f48 d0/d5/d7d/fc3 0 2024-03-24T10:45:58.513 INFO:tasks.workunit.client.0.smithi094.stdout:4/649: mknod d0/d34/d73/d94/cc4 0 2024-03-24T10:45:58.514 INFO:tasks.workunit.client.0.smithi094.stdout:4/650: read d0/f60 [1228130,12567] 0 2024-03-24T10:45:58.517 INFO:tasks.workunit.client.0.smithi094.stdout:9/652: dwrite df/d2d/d1f/d58/d70/f9f [0,4194304] 0 2024-03-24T10:45:58.522 INFO:tasks.workunit.client.0.smithi094.stdout:9/653: dread df/f36 [0,4194304] 0 2024-03-24T10:45:58.522 INFO:tasks.workunit.client.0.smithi094.stdout:7/607: write d1/d9/d16/d21/d51/d83/f72 [470626,125989] 0 2024-03-24T10:45:58.524 INFO:tasks.workunit.client.0.smithi094.stdout:9/654: symlink df/d2d/d1f/d58/d70/lcd 0 2024-03-24T10:45:58.524 INFO:tasks.workunit.client.0.smithi094.stdout:9/655: read df/d2d/d1f/d58/d5a/f6a [4232465,38216] 0 2024-03-24T10:45:58.526 INFO:tasks.workunit.client.0.smithi094.stdout:9/656: creat df/d2d/d1f/d58/d70/dab/d80/fce x:0 0 0 2024-03-24T10:45:58.530 INFO:tasks.workunit.client.0.smithi094.stdout:9/657: dread df/d2d/d1f/d58/d5a/f4b [0,4194304] 0 2024-03-24T10:45:58.531 INFO:tasks.workunit.client.0.smithi094.stdout:9/658: mknod df/d7b/ccf 0 2024-03-24T10:45:58.532 INFO:tasks.workunit.client.0.smithi094.stdout:9/659: creat df/d2d/d34/d2c/fd0 x:0 0 0 2024-03-24T10:45:58.534 INFO:tasks.workunit.client.0.smithi094.stdout:7/608: dwrite d1/db4/d65/f89 [0,4194304] 0 2024-03-24T10:45:58.534 INFO:tasks.workunit.client.0.smithi094.stdout:9/660: dread df/d2d/f84 [0,4194304] 0 2024-03-24T10:45:58.536 INFO:tasks.workunit.client.0.smithi094.stdout:7/609: symlink d1/d6c/d80/d81/lc6 0 2024-03-24T10:45:58.538 INFO:tasks.workunit.client.0.smithi094.stdout:9/661: dread df/d2d/f15 [0,4194304] 0 2024-03-24T10:45:58.538 INFO:tasks.workunit.client.0.smithi094.stdout:9/662: dread - df/d2d/d1f/d11/d53/fa1 zero size 2024-03-24T10:45:58.539 INFO:tasks.workunit.client.0.smithi094.stdout:7/610: dread d1/d9/fe [0,4194304] 0 2024-03-24T10:45:58.539 INFO:tasks.workunit.client.0.smithi094.stdout:7/611: read d1/d9/f30 [1459001,61112] 0 2024-03-24T10:45:58.539 INFO:tasks.workunit.client.0.smithi094.stdout:7/612: chown d1/d9/d25/d3d/f73 0 1 2024-03-24T10:45:58.539 INFO:tasks.workunit.client.0.smithi094.stdout:9/663: rename df/d2d/d34/d2c/d97/c9a to df/d2d/d34/db8/cd1 0 2024-03-24T10:45:58.540 INFO:tasks.workunit.client.0.smithi094.stdout:9/664: dread - df/d7b/fbb zero size 2024-03-24T10:45:58.540 INFO:tasks.workunit.client.0.smithi094.stdout:9/665: chown df/d2d/d1f/d11/f90 954 1 2024-03-24T10:45:58.541 INFO:tasks.workunit.client.0.smithi094.stdout:7/613: rename d1/d9/d14/da5/fb7 to d1/d9/d16/fc7 0 2024-03-24T10:45:58.557 INFO:tasks.workunit.client.0.smithi094.stdout:7/614: dwrite d1/d9/d14/d34/fa3 [0,4194304] 0 2024-03-24T10:45:58.557 INFO:tasks.workunit.client.0.smithi094.stdout:9/666: dwrite df/d2d/d1f/d11/fbf [0,4194304] 0 2024-03-24T10:45:58.559 INFO:tasks.workunit.client.0.smithi094.stdout:7/615: mkdir d1/d9/d14/d34/dc8 0 2024-03-24T10:45:58.563 INFO:tasks.workunit.client.0.smithi094.stdout:9/667: mknod df/d2d/d1f/d58/d5a/d50/cd2 0 2024-03-24T10:45:58.563 INFO:tasks.workunit.client.0.smithi094.stdout:9/668: creat df/d7b/fd3 x:0 0 0 2024-03-24T10:45:58.563 INFO:tasks.workunit.client.0.smithi094.stdout:9/669: write df/d2d/d1f/d58/d5a/f6a [4764715,103906] 0 2024-03-24T10:45:58.564 INFO:tasks.workunit.client.0.smithi094.stdout:9/670: mkdir df/d2d/d1f/d58/d5a/d50/da5/dd4 0 2024-03-24T10:45:58.565 INFO:tasks.workunit.client.0.smithi094.stdout:7/616: write d1/d9/d25/d3d/f55 [2754195,40322] 0 2024-03-24T10:45:58.569 INFO:tasks.workunit.client.0.smithi094.stdout:9/671: write df/d2d/d1f/d58/d82/fa0 [4142273,29880] 0 2024-03-24T10:45:58.569 INFO:tasks.workunit.client.0.smithi094.stdout:9/672: chown df/d2d/d1f/d58/d70/dab/l85 57003390 1 2024-03-24T10:45:58.570 INFO:tasks.workunit.client.0.smithi094.stdout:7/617: dread d1/f10 [0,4194304] 0 2024-03-24T10:45:58.570 INFO:tasks.workunit.client.0.smithi094.stdout:9/673: mkdir df/d2d/d1f/d11/d53/d88/dd5 0 2024-03-24T10:45:58.570 INFO:tasks.workunit.client.0.smithi094.stdout:9/674: fsync df/d2d/d1f/f49 0 2024-03-24T10:45:58.571 INFO:tasks.workunit.client.0.smithi094.stdout:7/618: mknod d1/d9/d16/cc9 0 2024-03-24T10:45:58.571 INFO:tasks.workunit.client.0.smithi094.stdout:9/675: rmdir df/d2d/d1f/d58/d70/dab 39 2024-03-24T10:45:58.572 INFO:tasks.workunit.client.0.smithi094.stdout:7/619: truncate d1/d9/d16/d21/d51/d83/f90 74245 0 2024-03-24T10:45:58.572 INFO:tasks.workunit.client.0.smithi094.stdout:9/676: dread df/d2d/f84 [0,4194304] 0 2024-03-24T10:45:58.573 INFO:tasks.workunit.client.0.smithi094.stdout:9/677: creat df/d2d/d34/fd6 x:0 0 0 2024-03-24T10:45:58.573 INFO:tasks.workunit.client.0.smithi094.stdout:7/620: rmdir d1/d9/d96 39 2024-03-24T10:45:58.575 INFO:tasks.workunit.client.0.smithi094.stdout:9/678: creat df/d2d/d1f/d11/d96/fd7 x:0 0 0 2024-03-24T10:45:58.575 INFO:tasks.workunit.client.0.smithi094.stdout:9/679: read - df/d2d/d1f/d58/d5a/d69/d7a/f8f zero size 2024-03-24T10:45:58.575 INFO:tasks.workunit.client.0.smithi094.stdout:9/680: chown df/fbc 3522635 1 2024-03-24T10:45:58.575 INFO:tasks.workunit.client.0.smithi094.stdout:9/681: write df/d2d/d1f/d7e/fc1 [936923,104562] 0 2024-03-24T10:45:58.575 INFO:tasks.workunit.client.0.smithi094.stdout:9/682: getdents df/d2d/d1f/d58/d5a/d50/da5/dd4 0 2024-03-24T10:45:58.576 INFO:tasks.workunit.client.0.smithi094.stdout:9/683: rmdir df/d2d/d1f/d58/d5a/d61/d87 39 2024-03-24T10:45:58.578 INFO:tasks.workunit.client.0.smithi094.stdout:9/684: rename df/d2d/d1f/d58/d70/f9f to df/d2d/d1f/d58/fd8 0 2024-03-24T10:45:58.578 INFO:tasks.workunit.client.0.smithi094.stdout:9/685: chown df/d2d/d1f/d58/d5a/f77 2696 1 2024-03-24T10:45:58.578 INFO:tasks.workunit.client.0.smithi094.stdout:9/686: chown df/d2d/f8d 533374 1 2024-03-24T10:45:58.578 INFO:tasks.workunit.client.0.smithi094.stdout:9/687: write df/d2d/d1f/d58/d5a/f4b [263780,120611] 0 2024-03-24T10:45:58.580 INFO:tasks.workunit.client.0.smithi094.stdout:9/688: symlink df/d2d/d1f/d11/d96/ld9 0 2024-03-24T10:45:58.582 INFO:tasks.workunit.client.0.smithi094.stdout:7/621: write d1/d9/d16/d21/d51/d83/d39/d44/f8a [2046111,118981] 0 2024-03-24T10:45:58.582 INFO:tasks.workunit.client.0.smithi094.stdout:7/622: chown d1/d9/d16 43438756 1 2024-03-24T10:45:58.582 INFO:tasks.workunit.client.0.smithi094.stdout:7/623: write d1/d9/f23 [4394623,48804] 0 2024-03-24T10:45:58.584 INFO:tasks.workunit.client.0.smithi094.stdout:7/624: link d1/d9/d16/d21/d51/d83/d39/lb5 d1/d9/d16/d21/d49/da9/lca 0 2024-03-24T10:45:58.584 INFO:tasks.workunit.client.0.smithi094.stdout:7/625: chown d1/d9/d25/d3d/f46 131383938 1 2024-03-24T10:45:58.584 INFO:tasks.workunit.client.0.smithi094.stdout:7/626: stat d1/d9/d14/d34/l92 0 2024-03-24T10:45:58.585 INFO:tasks.workunit.client.0.smithi094.stdout:7/627: creat d1/fcb x:0 0 0 2024-03-24T10:45:58.586 INFO:tasks.workunit.client.0.smithi094.stdout:7/628: mkdir d1/d9/d16/d21/d51/dcc 0 2024-03-24T10:45:58.586 INFO:tasks.workunit.client.0.smithi094.stdout:7/629: fdatasync d1/db4/f6b 0 2024-03-24T10:45:58.587 INFO:tasks.workunit.client.0.smithi094.stdout:7/630: truncate d1/d9/d25/d3d/f46 3258756 0 2024-03-24T10:45:58.588 INFO:tasks.workunit.client.0.smithi094.stdout:9/689: write df/f2a [503191,113734] 0 2024-03-24T10:45:58.588 INFO:tasks.workunit.client.0.smithi094.stdout:7/631: rename d1/d9/d16/d21/d51/d83/d39/d44/f4e to d1/d9/d16/d21/d51/d83/d39/fcd 0 2024-03-24T10:45:58.590 INFO:tasks.workunit.client.0.smithi094.stdout:9/690: mkdir df/dda 0 2024-03-24T10:45:58.593 INFO:tasks.workunit.client.0.smithi094.stdout:7/632: rename d1/d9/d25/d3d/lbc to d1/d6c/d80/d81/da2/lce 0 2024-03-24T10:45:58.593 INFO:tasks.workunit.client.0.smithi094.stdout:7/633: read d1/d9/d16/d21/d51/d83/d39/f48 [459459,12459] 0 2024-03-24T10:45:58.593 INFO:tasks.workunit.client.0.smithi094.stdout:7/634: getdents d1/d9/dc3 0 2024-03-24T10:45:58.594 INFO:tasks.workunit.client.0.smithi094.stdout:9/691: creat df/d2d/d1f/d58/d5a/d61/d87/dc8/fdb x:0 0 0 2024-03-24T10:45:58.597 INFO:tasks.workunit.client.0.smithi094.stdout:0/566: sync 2024-03-24T10:45:58.597 INFO:tasks.workunit.client.0.smithi094.stdout:9/692: symlink df/d2d/d34/d2c/d97/ldc 0 2024-03-24T10:45:58.597 INFO:tasks.workunit.client.0.smithi094.stdout:9/693: fdatasync df/d2d/d1f/d58/fd8 0 2024-03-24T10:45:58.597 INFO:tasks.workunit.client.0.smithi094.stdout:9/694: write df/d2d/d1f/d58/d5a/d61/fc0 [631891,19211] 0 2024-03-24T10:45:58.598 INFO:tasks.workunit.client.0.smithi094.stdout:9/695: chown df/d2d/d1f/d11/d96 11912 1 2024-03-24T10:45:58.598 INFO:tasks.workunit.client.0.smithi094.stdout:0/567: link d14/d22/d32/d70/cb1 d14/d22/d32/d65/cc9 0 2024-03-24T10:45:58.608 INFO:tasks.workunit.client.0.smithi094.stdout:9/696: dread df/d2d/d1f/d58/fd8 [0,4194304] 0 2024-03-24T10:45:58.608 INFO:tasks.workunit.client.0.smithi094.stdout:9/697: chown df/d2d/d34/db8/lb9 21 1 2024-03-24T10:45:58.608 INFO:tasks.workunit.client.0.smithi094.stdout:9/698: truncate df/d2d/d1f/f4a 1376195 0 2024-03-24T10:45:58.609 INFO:tasks.workunit.client.0.smithi094.stdout:9/699: creat df/d2d/d1f/d11/d53/d88/fdd x:0 0 0 2024-03-24T10:45:58.611 INFO:tasks.workunit.client.0.smithi094.stdout:9/700: rmdir df/d2d/d1f/d58/d5a/d61 39 2024-03-24T10:45:58.611 INFO:tasks.workunit.client.0.smithi094.stdout:9/701: stat df/d2d/d1f/d58/d5a/d69/d7a/f8f 0 2024-03-24T10:45:58.611 INFO:tasks.workunit.client.0.smithi094.stdout:7/635: dwrite d1/d9/d16/d21/d51/d83/d39/fcd [0,4194304] 0 2024-03-24T10:45:58.613 INFO:tasks.workunit.client.0.smithi094.stdout:0/568: dwrite d14/d18/f1f [0,4194304] 0 2024-03-24T10:45:58.616 INFO:tasks.workunit.client.0.smithi094.stdout:7/636: rmdir d1/d9/d16/d21/d49 39 2024-03-24T10:45:58.619 INFO:tasks.workunit.client.0.smithi094.stdout:0/569: rmdir d14/d22 39 2024-03-24T10:45:58.619 INFO:tasks.workunit.client.0.smithi094.stdout:0/570: stat d14/d22/f2f 0 2024-03-24T10:45:58.619 INFO:tasks.workunit.client.0.smithi094.stdout:0/571: chown d14/d22/d5b/c84 97244 1 2024-03-24T10:45:58.621 INFO:tasks.workunit.client.0.smithi094.stdout:7/637: symlink d1/d6c/d80/d81/lcf 0 2024-03-24T10:45:58.621 INFO:tasks.workunit.client.0.smithi094.stdout:7/638: chown d1/d9/d16/d21/d51/d83/d39/f48 19202 1 2024-03-24T10:45:58.622 INFO:tasks.workunit.client.0.smithi094.stdout:7/639: getdents d1/d9/d14/d34 0 2024-03-24T10:45:58.623 INFO:tasks.workunit.client.0.smithi094.stdout:9/702: dwrite df/d2d/d1f/d11/d53/d88/f89 [0,4194304] 0 2024-03-24T10:45:58.624 INFO:tasks.workunit.client.0.smithi094.stdout:7/640: rename d1/d9/l47 to d1/d9/d25/ld0 0 2024-03-24T10:45:58.626 INFO:tasks.workunit.client.0.smithi094.stdout:9/703: mkdir df/d2d/d1f/d11/d53/dde 0 2024-03-24T10:45:58.628 INFO:tasks.workunit.client.0.smithi094.stdout:9/704: mkdir df/d2d/d34/d2c/d97/ddf 0 2024-03-24T10:45:58.642 INFO:tasks.workunit.client.0.smithi094.stdout:7/641: dwrite d1/d6c/d80/f6f [0,4194304] 0 2024-03-24T10:45:58.642 INFO:tasks.workunit.client.0.smithi094.stdout:7/642: truncate d1/d9/f13 1219612 0 2024-03-24T10:45:58.647 INFO:tasks.workunit.client.0.smithi094.stdout:0/572: dwrite d14/d22/d7d/f9a [0,4194304] 0 2024-03-24T10:45:58.647 INFO:tasks.workunit.client.0.smithi094.stdout:0/573: chown d14/d18/d85/cb8 32257 1 2024-03-24T10:45:58.648 INFO:tasks.workunit.client.0.smithi094.stdout:9/705: dwrite df/d2d/f8d [0,4194304] 0 2024-03-24T10:45:58.648 INFO:tasks.workunit.client.0.smithi094.stdout:9/706: readlink df/d2d/d1f/d11/d43/l51 0 2024-03-24T10:45:58.652 INFO:tasks.workunit.client.0.smithi094.stdout:7/643: read d1/d9/fd [295593,27200] 0 2024-03-24T10:45:58.653 INFO:tasks.workunit.client.0.smithi094.stdout:9/707: creat df/d2d/d1f/d58/d5a/fe0 x:0 0 0 2024-03-24T10:45:58.653 INFO:tasks.workunit.client.0.smithi094.stdout:0/574: dread d14/f2a [0,4194304] 0 2024-03-24T10:45:58.654 INFO:tasks.workunit.client.0.smithi094.stdout:7/644: rename d1/d9/d16/d21/d49/fa4 to d1/d9/fd1 0 2024-03-24T10:45:58.655 INFO:tasks.workunit.client.0.smithi094.stdout:9/708: symlink df/d2d/d34/d2c/le1 0 2024-03-24T10:45:58.656 INFO:tasks.workunit.client.0.smithi094.stdout:0/575: symlink d14/d22/d5b/d86/lca 0 2024-03-24T10:45:58.657 INFO:tasks.workunit.client.0.smithi094.stdout:0/576: mkdir d14/d22/dcb 0 2024-03-24T10:45:58.660 INFO:tasks.workunit.client.0.smithi094.stdout:2/546: sync 2024-03-24T10:45:58.660 INFO:tasks.workunit.client.0.smithi094.stdout:6/609: sync 2024-03-24T10:45:58.660 INFO:tasks.workunit.client.0.smithi094.stdout:1/468: sync 2024-03-24T10:45:58.660 INFO:tasks.workunit.client.0.smithi094.stdout:5/637: sync 2024-03-24T10:45:58.660 INFO:tasks.workunit.client.0.smithi094.stdout:3/513: sync 2024-03-24T10:45:58.660 INFO:tasks.workunit.client.0.smithi094.stdout:3/514: fsync d0/db/dc/d28/d3c/d55/d7f/d20/d23/f9a 0 2024-03-24T10:45:58.660 INFO:tasks.workunit.client.0.smithi094.stdout:4/651: sync 2024-03-24T10:45:58.661 INFO:tasks.workunit.client.0.smithi094.stdout:8/659: sync 2024-03-24T10:45:58.661 INFO:tasks.workunit.client.0.smithi094.stdout:8/660: fsync df/f1b 0 2024-03-24T10:45:58.662 INFO:tasks.workunit.client.0.smithi094.stdout:8/661: write df/d12/d5b/d63/fc7 [399484,56887] 0 2024-03-24T10:45:58.663 INFO:tasks.workunit.client.0.smithi094.stdout:2/547: getdents d18/d54 0 2024-03-24T10:45:58.664 INFO:tasks.workunit.client.0.smithi094.stdout:6/610: truncate d0/f33 1165839 0 2024-03-24T10:45:58.665 INFO:tasks.workunit.client.0.smithi094.stdout:6/611: write d0/d30/d50/d4d/d6b/f90 [5407,36872] 0 2024-03-24T10:45:58.665 INFO:tasks.workunit.client.0.smithi094.stdout:1/469: creat d4/d9/db/d98/fa4 x:0 0 0 2024-03-24T10:45:58.667 INFO:tasks.workunit.client.0.smithi094.stdout:3/515: unlink d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/f99 0 2024-03-24T10:45:58.667 INFO:tasks.workunit.client.0.smithi094.stdout:4/652: rename d0/d22/c4d to d0/d22/de/d28/d2b/d7a/cc5 0 2024-03-24T10:45:58.668 INFO:tasks.workunit.client.0.smithi094.stdout:4/653: chown d0/d34/db2 1332777 1 2024-03-24T10:45:58.671 INFO:tasks.workunit.client.0.smithi094.stdout:1/470: creat d4/d9/de/d6f/fa5 x:0 0 0 2024-03-24T10:45:58.673 INFO:tasks.workunit.client.0.smithi094.stdout:1/471: chown d4/d9/de/d6f/d62/d99 2155 1 2024-03-24T10:45:58.673 INFO:tasks.workunit.client.0.smithi094.stdout:8/662: mkdir df/d25/d28/d31/d47/ddf 0 2024-03-24T10:45:58.673 INFO:tasks.workunit.client.0.smithi094.stdout:6/612: rename d0/d3b/f69 to d0/d78/d66/d7a/fb8 0 2024-03-24T10:45:58.675 INFO:tasks.workunit.client.0.smithi094.stdout:1/472: truncate d4/d9/de/f2b 1066677 0 2024-03-24T10:45:58.675 INFO:tasks.workunit.client.0.smithi094.stdout:1/473: truncate d4/d36/f8e 4530470 0 2024-03-24T10:45:58.676 INFO:tasks.workunit.client.0.smithi094.stdout:0/577: dread d14/d18/f40 [0,4194304] 0 2024-03-24T10:45:58.676 INFO:tasks.workunit.client.0.smithi094.stdout:7/645: dwrite d1/d6c/d80/f6f [0,4194304] 0 2024-03-24T10:45:58.677 INFO:tasks.workunit.client.0.smithi094.stdout:7/646: dread - d1/d9/d16/fbe zero size 2024-03-24T10:45:58.677 INFO:tasks.workunit.client.0.smithi094.stdout:7/647: stat d1/d9/cb0 0 2024-03-24T10:45:58.677 INFO:tasks.workunit.client.0.smithi094.stdout:7/648: write d1/d9/d16/d21/d51/d83/d39/d44/f8a [6225168,85849] 0 2024-03-24T10:45:58.677 INFO:tasks.workunit.client.0.smithi094.stdout:7/649: creat d1/fd2 x:0 0 0 2024-03-24T10:45:58.677 INFO:tasks.workunit.client.0.smithi094.stdout:8/663: truncate df/d25/d28/f42 182124 0 2024-03-24T10:45:58.679 INFO:tasks.workunit.client.0.smithi094.stdout:6/613: truncate d0/f3c 461007 0 2024-03-24T10:45:58.679 INFO:tasks.workunit.client.0.smithi094.stdout:6/614: fdatasync d0/d21/f2d 0 2024-03-24T10:45:58.690 INFO:tasks.workunit.client.0.smithi094.stdout:1/474: link d4/d9/db/d39/d43/l79 d4/d9/db/d85/la6 0 2024-03-24T10:45:58.690 INFO:tasks.workunit.client.0.smithi094.stdout:1/475: stat d4/d9/de/d3d/f92 0 2024-03-24T10:45:58.693 INFO:tasks.workunit.client.0.smithi094.stdout:0/578: creat d14/d22/d32/d65/fcc x:0 0 0 2024-03-24T10:45:58.693 INFO:tasks.workunit.client.0.smithi094.stdout:0/579: readlink d14/d22/d28/d2b/d41/d53/l58 0 2024-03-24T10:45:58.693 INFO:tasks.workunit.client.0.smithi094.stdout:0/580: creat d14/fcd x:0 0 0 2024-03-24T10:45:58.693 INFO:tasks.workunit.client.0.smithi094.stdout:0/581: chown f0 18491048 1 2024-03-24T10:45:58.693 INFO:tasks.workunit.client.0.smithi094.stdout:0/582: write d14/d18/fb0 [2288787,53804] 0 2024-03-24T10:45:58.696 INFO:tasks.workunit.client.0.smithi094.stdout:7/650: rename d1/d6c/d80/d81/da2/lce to d1/d9/d14/d8f/ld3 0 2024-03-24T10:45:58.697 INFO:tasks.workunit.client.0.smithi094.stdout:2/548: dwrite d18/d1e/d34/f44 [0,4194304] 0 2024-03-24T10:45:58.698 INFO:tasks.workunit.client.0.smithi094.stdout:2/549: write d18/d1e/d34/f84 [721657,20710] 0 2024-03-24T10:45:58.699 INFO:tasks.workunit.client.0.smithi094.stdout:6/615: symlink d0/d78/da/d23/d83/lb9 0 2024-03-24T10:45:58.699 INFO:tasks.workunit.client.0.smithi094.stdout:6/616: read - d0/d78/da/d23/d34/f91 zero size 2024-03-24T10:45:58.705 INFO:tasks.workunit.client.0.smithi094.stdout:0/583: mkdir d14/d18/d85/dce 0 2024-03-24T10:45:58.705 INFO:tasks.workunit.client.0.smithi094.stdout:7/651: mkdir d1/d9/d25/dd4 0 2024-03-24T10:45:58.705 INFO:tasks.workunit.client.0.smithi094.stdout:7/652: stat d1/d9/d25/d3d/f73 0 2024-03-24T10:45:58.706 INFO:tasks.workunit.client.0.smithi094.stdout:2/550: getdents d18/d48/d56 0 2024-03-24T10:45:58.706 INFO:tasks.workunit.client.0.smithi094.stdout:2/551: chown d18/d66/fa5 252104 1 2024-03-24T10:45:58.708 INFO:tasks.workunit.client.0.smithi094.stdout:6/617: write d0/f49 [2576970,71401] 0 2024-03-24T10:45:58.709 INFO:tasks.workunit.client.0.smithi094.stdout:0/584: mknod d14/d18/ccf 0 2024-03-24T10:45:58.709 INFO:tasks.workunit.client.0.smithi094.stdout:7/653: symlink d1/db4/d65/ld5 0 2024-03-24T10:45:58.710 INFO:tasks.workunit.client.0.smithi094.stdout:2/552: rmdir d18/d66/d90/d9a/d74 39 2024-03-24T10:45:58.711 INFO:tasks.workunit.client.0.smithi094.stdout:6/618: creat d0/d1a/d35/fba x:0 0 0 2024-03-24T10:45:58.712 INFO:tasks.workunit.client.0.smithi094.stdout:6/619: dread - d0/d78/da/d23/d34/f91 zero size 2024-03-24T10:45:58.713 INFO:tasks.workunit.client.0.smithi094.stdout:2/553: mkdir d18/d66/da6 0 2024-03-24T10:45:58.714 INFO:tasks.workunit.client.0.smithi094.stdout:6/620: mknod d0/d78/da/d23/d7c/db0/cbb 0 2024-03-24T10:45:58.714 INFO:tasks.workunit.client.0.smithi094.stdout:3/516: dwrite d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/ff [0,4194304] 0 2024-03-24T10:45:58.714 INFO:tasks.workunit.client.0.smithi094.stdout:3/517: write d0/db/dc/d28/d3c/d55/d7f/d20/f24 [1316974,19459] 0 2024-03-24T10:45:58.718 INFO:tasks.workunit.client.0.smithi094.stdout:2/554: unlink d18/d66/l9d 0 2024-03-24T10:45:58.718 INFO:tasks.workunit.client.0.smithi094.stdout:2/555: fsync d18/d1e/d65/f72 0 2024-03-24T10:45:58.720 INFO:tasks.workunit.client.0.smithi094.stdout:2/556: dread d18/f31 [0,4194304] 0 2024-03-24T10:45:58.721 INFO:tasks.workunit.client.0.smithi094.stdout:8/664: dwrite df/d25/d28/d31/d47/d22/fc5 [0,4194304] 0 2024-03-24T10:45:58.721 INFO:tasks.workunit.client.0.smithi094.stdout:8/665: fdatasync df/f56 0 2024-03-24T10:45:58.722 INFO:tasks.workunit.client.0.smithi094.stdout:3/518: mkdir d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/db5 0 2024-03-24T10:45:58.722 INFO:tasks.workunit.client.0.smithi094.stdout:3/519: creat d0/fb6 x:0 0 0 2024-03-24T10:45:58.722 INFO:tasks.workunit.client.0.smithi094.stdout:4/654: dwrite d0/d22/de/d28/d3d/d8c/fb6 [0,4194304] 0 2024-03-24T10:45:58.722 INFO:tasks.workunit.client.0.smithi094.stdout:9/709: dwrite df/d2d/f1e [0,4194304] 0 2024-03-24T10:45:58.723 INFO:tasks.workunit.client.0.smithi094.stdout:8/666: symlink df/d25/d28/d31/d47/d66/da3/da8/le0 0 2024-03-24T10:45:58.726 INFO:tasks.workunit.client.0.smithi094.stdout:3/520: rmdir d0/db/dc/d28/d3c/d55/d7f/d20/d23 39 2024-03-24T10:45:58.727 INFO:tasks.workunit.client.0.smithi094.stdout:6/621: dread d0/d78/da/d23/f9b [0,4194304] 0 2024-03-24T10:45:58.728 INFO:tasks.workunit.client.0.smithi094.stdout:2/557: write d18/d1e/d34/d4a/f6c [2034115,101561] 0 2024-03-24T10:45:58.728 INFO:tasks.workunit.client.0.smithi094.stdout:1/476: dwrite d4/d9/de/d6f/d5f/f68 [0,4194304] 0 2024-03-24T10:45:58.728 INFO:tasks.workunit.client.0.smithi094.stdout:3/521: write d0/db/dc/d28/d3c/d7c/f8e [122577,4730] 0 2024-03-24T10:45:58.729 INFO:tasks.workunit.client.0.smithi094.stdout:7/654: dwrite d1/db4/d65/fb6 [0,4194304] 0 2024-03-24T10:45:58.731 INFO:tasks.workunit.client.0.smithi094.stdout:6/622: dread d0/f2 [0,4194304] 0 2024-03-24T10:45:58.733 INFO:tasks.workunit.client.0.smithi094.stdout:9/710: mkdir df/d2d/d1f/d58/d70/de2 0 2024-03-24T10:45:58.733 INFO:tasks.workunit.client.0.smithi094.stdout:9/711: fdatasync df/d2d/d34/d2c/d97/fcc 0 2024-03-24T10:45:58.739 INFO:tasks.workunit.client.0.smithi094.stdout:6/623: dread d0/d21/d2a/d2f/fa6 [0,4194304] 0 2024-03-24T10:45:58.739 INFO:tasks.workunit.client.0.smithi094.stdout:2/558: creat d18/d54/fa7 x:0 0 0 2024-03-24T10:45:58.743 INFO:tasks.workunit.client.0.smithi094.stdout:6/624: creat d0/da9/fbc x:0 0 0 2024-03-24T10:45:58.745 INFO:tasks.workunit.client.0.smithi094.stdout:2/559: mknod d18/d1e/ca8 0 2024-03-24T10:45:58.745 INFO:tasks.workunit.client.0.smithi094.stdout:2/560: chown d18/d1e/d65/f6a 46 1 2024-03-24T10:45:58.745 INFO:tasks.workunit.client.0.smithi094.stdout:2/561: stat d18/d1e/d65/l96 0 2024-03-24T10:45:58.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:45:58 smithi094 ceph-mon[29446]: pgmap v242: 65 pgs: 65 active+clean; 970 MiB data, 4.2 GiB used, 532 GiB / 536 GiB avail; 92 MiB/s rd, 116 MiB/s wr, 743 op/s 2024-03-24T10:45:58.749 INFO:tasks.workunit.client.0.smithi094.stdout:3/522: dread d0/db/dc/d28/d38/d6a/f79 [0,4194304] 0 2024-03-24T10:45:58.750 INFO:tasks.workunit.client.0.smithi094.stdout:7/655: dread d1/d9/d16/d21/d51/d83/d39/d44/f8a [0,4194304] 0 2024-03-24T10:45:58.750 INFO:tasks.workunit.client.0.smithi094.stdout:0/585: rename d14/d18/d85 to d14/d22/d32/d70/dd0 0 2024-03-24T10:45:58.750 INFO:tasks.workunit.client.0.smithi094.stdout:0/586: fsync d14/d22/f2f 0 2024-03-24T10:45:58.752 INFO:tasks.workunit.client.0.smithi094.stdout:3/523: truncate d0/db/dc/d28/d3c/d55/d7f/d20/d23/f9a 4779387 0 2024-03-24T10:45:58.753 INFO:tasks.workunit.client.0.smithi094.stdout:1/477: dwrite d4/d9/d37/d33/f51 [0,4194304] 0 2024-03-24T10:45:58.753 INFO:tasks.workunit.client.0.smithi094.stdout:1/478: read - d4/d9/de/d6f/fa5 zero size 2024-03-24T10:45:58.754 INFO:tasks.workunit.client.0.smithi094.stdout:2/562: write d18/d48/f5d [3501871,116890] 0 2024-03-24T10:45:58.755 INFO:tasks.workunit.client.0.smithi094.stdout:6/625: dread d0/d78/da/f20 [0,4194304] 0 2024-03-24T10:45:58.755 INFO:tasks.workunit.client.0.smithi094.stdout:8/667: dwrite df/d25/d28/d31/d47/f77 [0,4194304] 0 2024-03-24T10:45:58.756 INFO:tasks.workunit.client.0.smithi094.stdout:4/655: rename d0/d22/de/d28/d2b/d33/da4/c75 to d0/d5/d7d/d9c/cc6 0 2024-03-24T10:45:58.760 INFO:tasks.workunit.client.0.smithi094.stdout:0/587: unlink d14/d18/fa0 0 2024-03-24T10:45:58.760 INFO:tasks.workunit.client.0.smithi094.stdout:0/588: write d14/d22/d7d/fa6 [1353410,42366] 0 2024-03-24T10:45:58.760 INFO:tasks.workunit.client.0.smithi094.stdout:0/589: fsync d14/d22/f33 0 2024-03-24T10:45:58.760 INFO:tasks.workunit.client.0.smithi094.stdout:0/590: fdatasync d14/d22/f35 0 2024-03-24T10:45:58.761 INFO:tasks.workunit.client.0.smithi094.stdout:3/524: write d0/db/dc/d28/d3c/d55/d7f/d20/d23/f5f [3972682,11531] 0 2024-03-24T10:45:58.766 INFO:tasks.workunit.client.0.smithi094.stdout:1/479: link d4/d9/f25 d4/d9/de/fa7 0 2024-03-24T10:45:58.766 INFO:tasks.workunit.client.0.smithi094.stdout:1/480: fsync d4/d36/f89 0 2024-03-24T10:45:58.769 INFO:tasks.workunit.client.0.smithi094.stdout:2/563: symlink d18/d48/la9 0 2024-03-24T10:45:58.769 INFO:tasks.workunit.client.0.smithi094.stdout:2/564: stat d18/d1e 0 2024-03-24T10:45:58.769 INFO:tasks.workunit.client.0.smithi094.stdout:2/565: dread - d18/d7f/f8d zero size 2024-03-24T10:45:58.770 INFO:tasks.workunit.client.0.smithi094.stdout:2/566: fsync d18/f45 0 2024-03-24T10:45:58.770 INFO:tasks.workunit.client.0.smithi094.stdout:4/656: dread d0/d22/de/d28/d3d/d8c/fb6 [0,4194304] 0 2024-03-24T10:45:58.773 INFO:tasks.workunit.client.0.smithi094.stdout:3/525: rmdir d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a 39 2024-03-24T10:45:58.775 INFO:tasks.workunit.client.0.smithi094.stdout:3/526: creat d0/db/dc/d28/d50/d5e/fb7 x:0 0 0 2024-03-24T10:45:58.775 INFO:tasks.workunit.client.0.smithi094.stdout:8/668: dwrite df/d12/d5b/d63/fc7 [0,4194304] 0 2024-03-24T10:45:58.777 INFO:tasks.workunit.client.0.smithi094.stdout:7/656: dwrite d1/d9/d14/d8f/fc2 [0,4194304] 0 2024-03-24T10:45:58.777 INFO:tasks.workunit.client.0.smithi094.stdout:7/657: chown d1/d9/d14/l6a 1556 1 2024-03-24T10:45:58.777 INFO:tasks.workunit.client.0.smithi094.stdout:7/658: read d1/d9/d25/d3d/f46 [1395970,89425] 0 2024-03-24T10:45:58.779 INFO:tasks.workunit.client.0.smithi094.stdout:8/669: rename df/f56 to df/d8c/d8f/dc1/fe1 0 2024-03-24T10:45:58.779 INFO:tasks.workunit.client.0.smithi094.stdout:8/670: readlink df/d25/d28/d31/d47/l24 0 2024-03-24T10:45:58.779 INFO:tasks.workunit.client.0.smithi094.stdout:8/671: fdatasync df/d25/d28/f6f 0 2024-03-24T10:45:58.779 INFO:tasks.workunit.client.0.smithi094.stdout:8/672: creat df/d25/d28/d31/d47/d66/fe2 x:0 0 0 2024-03-24T10:45:58.780 INFO:tasks.workunit.client.0.smithi094.stdout:1/481: write d4/d9/de/f3b [2047214,10892] 0 2024-03-24T10:45:58.782 INFO:tasks.workunit.client.0.smithi094.stdout:4/657: read d0/d5/f16 [1044181,67572] 0 2024-03-24T10:45:58.782 INFO:tasks.workunit.client.0.smithi094.stdout:4/658: readlink d0/d5/l19 0 2024-03-24T10:45:58.783 INFO:tasks.workunit.client.0.smithi094.stdout:8/673: mkdir df/d25/d28/d31/d95/de3 0 2024-03-24T10:45:58.783 INFO:tasks.workunit.client.0.smithi094.stdout:0/591: dwrite d14/d22/d28/d48/d7b/f8e [0,4194304] 0 2024-03-24T10:45:58.783 INFO:tasks.workunit.client.0.smithi094.stdout:6/626: dwrite d0/f61 [0,4194304] 0 2024-03-24T10:45:58.784 INFO:tasks.workunit.client.0.smithi094.stdout:1/482: symlink d4/d9/d7d/la8 0 2024-03-24T10:45:58.784 INFO:tasks.workunit.client.0.smithi094.stdout:1/483: fsync d4/d9/f2c 0 2024-03-24T10:45:58.784 INFO:tasks.workunit.client.0.smithi094.stdout:1/484: fdatasync d4/d9/de/fa7 0 2024-03-24T10:45:58.784 INFO:tasks.workunit.client.0.smithi094.stdout:1/485: dread - d4/d9/d37/d33/d76/f9b zero size 2024-03-24T10:45:58.784 INFO:tasks.workunit.client.0.smithi094.stdout:1/486: chown d4/d9/d37/d33 1161699 1 2024-03-24T10:45:58.786 INFO:tasks.workunit.client.0.smithi094.stdout:4/659: link d0/d22/de/f30 d0/d22/de/d28/d3d/d8c/fc7 0 2024-03-24T10:45:58.788 INFO:tasks.workunit.client.0.smithi094.stdout:8/674: dread df/d25/d28/d31/d47/d66/da3/fa4 [0,4194304] 0 2024-03-24T10:45:58.789 INFO:tasks.workunit.client.0.smithi094.stdout:6/627: dread d0/d30/f5b [0,4194304] 0 2024-03-24T10:45:58.790 INFO:tasks.workunit.client.0.smithi094.stdout:0/592: creat d14/d22/d28/da2/fd1 x:0 0 0 2024-03-24T10:45:58.791 INFO:tasks.workunit.client.0.smithi094.stdout:0/593: fdatasync d14/d22/d28/f31 0 2024-03-24T10:45:58.792 INFO:tasks.workunit.client.0.smithi094.stdout:1/487: getdents d4/d9/db/d85 0 2024-03-24T10:45:58.794 INFO:tasks.workunit.client.0.smithi094.stdout:0/594: mknod d14/d22/d28/d2b/dc0/cd2 0 2024-03-24T10:45:58.795 INFO:tasks.workunit.client.0.smithi094.stdout:4/660: dread d0/d22/f76 [0,4194304] 0 2024-03-24T10:45:58.795 INFO:tasks.workunit.client.0.smithi094.stdout:4/661: dread - d0/d34/d73/d93/fb4 zero size 2024-03-24T10:45:58.795 INFO:tasks.workunit.client.0.smithi094.stdout:1/488: symlink d4/d9/de/d3d/d66/la9 0 2024-03-24T10:45:58.795 INFO:tasks.workunit.client.0.smithi094.stdout:1/489: chown d4/d9/de/l60 222644824 1 2024-03-24T10:45:58.797 INFO:tasks.workunit.client.0.smithi094.stdout:0/595: unlink d14/d22/d32/f5d 0 2024-03-24T10:45:58.800 INFO:tasks.workunit.client.0.smithi094.stdout:4/662: mkdir d0/d22/de/dc8 0 2024-03-24T10:45:58.800 INFO:tasks.workunit.client.0.smithi094.stdout:4/663: write d0/d5/d1b/f2e [2872327,47000] 0 2024-03-24T10:45:58.800 INFO:tasks.workunit.client.0.smithi094.stdout:4/664: write d0/d22/de/d28/d2b/d33/fc2 [342204,73702] 0 2024-03-24T10:45:58.800 INFO:tasks.workunit.client.0.smithi094.stdout:4/665: truncate d0/f60 2296399 0 2024-03-24T10:45:58.800 INFO:tasks.workunit.client.0.smithi094.stdout:4/666: creat d0/d5/d1b/fc9 x:0 0 0 2024-03-24T10:45:58.800 INFO:tasks.workunit.client.0.smithi094.stdout:8/675: write df/d12/d5b/f83 [1663989,43104] 0 2024-03-24T10:45:58.800 INFO:tasks.workunit.client.0.smithi094.stdout:8/676: write df/d25/d28/d31/f40 [4890940,5765] 0 2024-03-24T10:45:58.800 INFO:tasks.workunit.client.0.smithi094.stdout:8/677: dread - df/d25/d28/d31/d47/d66/fe2 zero size 2024-03-24T10:45:58.801 INFO:tasks.workunit.client.0.smithi094.stdout:5/638: sync 2024-03-24T10:45:58.801 INFO:tasks.workunit.client.0.smithi094.stdout:4/667: dread d0/d22/de/d28/d2b/d33/da4/f8b [0,4194304] 0 2024-03-24T10:45:58.802 INFO:tasks.workunit.client.0.smithi094.stdout:3/527: dwrite d0/db/dc/d28/d3c/d55/d7f/d20/d23/f5f [0,4194304] 0 2024-03-24T10:45:58.803 INFO:tasks.workunit.client.0.smithi094.stdout:3/528: fsync d0/db/dc/d28/d3c/f6f 0 2024-03-24T10:45:58.803 INFO:tasks.workunit.client.0.smithi094.stdout:6/628: dread d0/d78/d66/d7a/fb8 [0,4194304] 0 2024-03-24T10:45:58.804 INFO:tasks.workunit.client.0.smithi094.stdout:2/567: dwrite d18/d48/f52 [0,4194304] 0 2024-03-24T10:45:58.804 INFO:tasks.workunit.client.0.smithi094.stdout:2/568: fdatasync d18/d48/f98 0 2024-03-24T10:45:58.804 INFO:tasks.workunit.client.0.smithi094.stdout:2/569: creat d18/d66/faa x:0 0 0 2024-03-24T10:45:58.806 INFO:tasks.workunit.client.0.smithi094.stdout:8/678: rename df/d25/d28/d31/d47/d22/c73 to df/d12/dcd/ce4 0 2024-03-24T10:45:58.806 INFO:tasks.workunit.client.0.smithi094.stdout:7/659: dwrite d1/fd2 [0,4194304] 0 2024-03-24T10:45:58.807 INFO:tasks.workunit.client.0.smithi094.stdout:5/639: symlink d7/d26/d55/d64/ld4 0 2024-03-24T10:45:58.807 INFO:tasks.workunit.client.0.smithi094.stdout:5/640: fsync d7/d23/d2b/f5a 0 2024-03-24T10:45:58.807 INFO:tasks.workunit.client.0.smithi094.stdout:5/641: fdatasync d7/d23/d5f/f58 0 2024-03-24T10:45:58.807 INFO:tasks.workunit.client.0.smithi094.stdout:5/642: truncate d7/d23/d5f/d14/d6f/dbe/fc6 670018 0 2024-03-24T10:45:58.808 INFO:tasks.workunit.client.0.smithi094.stdout:4/668: link d0/d5/d1b/c2f d0/d22/de/d28/d2b/d33/da4/da5/cca 0 2024-03-24T10:45:58.813 INFO:tasks.workunit.client.0.smithi094.stdout:2/570: mknod d18/d66/d90/cab 0 2024-03-24T10:45:58.816 INFO:tasks.workunit.client.0.smithi094.stdout:3/529: rename d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/laf to d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/db5/lb8 0 2024-03-24T10:45:58.816 INFO:tasks.workunit.client.0.smithi094.stdout:3/530: fdatasync d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/d73/f75 0 2024-03-24T10:45:58.817 INFO:tasks.workunit.client.0.smithi094.stdout:8/679: creat df/d25/d28/d31/d47/d66/da3/da8/fe5 x:0 0 0 2024-03-24T10:45:58.817 INFO:tasks.workunit.client.0.smithi094.stdout:8/680: read - df/d25/d36/d3f/fdb zero size 2024-03-24T10:45:58.818 INFO:tasks.workunit.client.0.smithi094.stdout:7/660: rmdir d1/db4/d61 39 2024-03-24T10:45:58.820 INFO:tasks.workunit.client.0.smithi094.stdout:4/669: mkdir d0/d22/de/d28/d2b/d33/da4/dcb 0 2024-03-24T10:45:58.820 INFO:tasks.workunit.client.0.smithi094.stdout:1/490: dwrite d4/d9/d37/d33/d58/f88 [0,4194304] 0 2024-03-24T10:45:58.820 INFO:tasks.workunit.client.0.smithi094.stdout:1/491: readlink d4/d9/d7d/l59 0 2024-03-24T10:45:58.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:45:58 smithi096 ceph-mon[38097]: pgmap v242: 65 pgs: 65 active+clean; 970 MiB data, 4.2 GiB used, 532 GiB / 536 GiB avail; 92 MiB/s rd, 116 MiB/s wr, 743 op/s 2024-03-24T10:45:58.826 INFO:tasks.workunit.client.0.smithi094.stdout:6/629: rename d0/d78/da/f2e to d0/d30/d50/fbd 0 2024-03-24T10:45:58.826 INFO:tasks.workunit.client.0.smithi094.stdout:6/630: creat d0/d3b/fbe x:0 0 0 2024-03-24T10:45:58.827 INFO:tasks.workunit.client.0.smithi094.stdout:6/631: fdatasync d0/d30/d50/fbd 0 2024-03-24T10:45:58.827 INFO:tasks.workunit.client.0.smithi094.stdout:2/571: write d18/f1d [1408630,66244] 0 2024-03-24T10:45:58.828 INFO:tasks.workunit.client.0.smithi094.stdout:3/531: mkdir d0/db/dc/d28/d3c/d55/db9 0 2024-03-24T10:45:58.829 INFO:tasks.workunit.client.0.smithi094.stdout:7/661: creat d1/d9/d25/d3d/fd6 x:0 0 0 2024-03-24T10:45:58.831 INFO:tasks.workunit.client.0.smithi094.stdout:2/572: dread d18/d1e/d34/d4a/f6c [0,4194304] 0 2024-03-24T10:45:58.831 INFO:tasks.workunit.client.0.smithi094.stdout:1/492: symlink d4/d9/de/d3d/d66/laa 0 2024-03-24T10:45:58.832 INFO:tasks.workunit.client.0.smithi094.stdout:3/532: dread d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/f4c [0,4194304] 0 2024-03-24T10:45:58.832 INFO:tasks.workunit.client.0.smithi094.stdout:3/533: stat d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/d73/fb1 0 2024-03-24T10:45:58.835 INFO:tasks.workunit.client.0.smithi094.stdout:8/681: dwrite df/d8c/d8f/dc1/fe1 [4194304,4194304] 0 2024-03-24T10:45:58.835 INFO:tasks.workunit.client.0.smithi094.stdout:8/682: readlink df/d12/d5b/d63/d75/d96/lb8 0 2024-03-24T10:45:58.836 INFO:tasks.workunit.client.0.smithi094.stdout:6/632: creat d0/d1a/d35/d96/fbf x:0 0 0 2024-03-24T10:45:58.837 INFO:tasks.workunit.client.0.smithi094.stdout:7/662: mkdir d1/d9/d16/d21/d51/d83/d9d/dd7 0 2024-03-24T10:45:58.837 INFO:tasks.workunit.client.0.smithi094.stdout:7/663: fdatasync d1/db4/d65/f89 0 2024-03-24T10:45:58.837 INFO:tasks.workunit.client.0.smithi094.stdout:2/573: rmdir d18/d7f/d88/da2 39 2024-03-24T10:45:58.837 INFO:tasks.workunit.client.0.smithi094.stdout:2/574: fdatasync d18/d1e/d34/f3d 0 2024-03-24T10:45:58.840 INFO:tasks.workunit.client.0.smithi094.stdout:3/534: write d0/f2 [4979647,13140] 0 2024-03-24T10:45:58.841 INFO:tasks.workunit.client.0.smithi094.stdout:8/683: symlink df/d8c/d8f/dc1/le6 0 2024-03-24T10:45:58.843 INFO:tasks.workunit.client.0.smithi094.stdout:6/633: symlink d0/d78/da/d23/d34/d4e/lc0 0 2024-03-24T10:45:58.843 INFO:tasks.workunit.client.0.smithi094.stdout:6/634: creat d0/d1a/d35/d96/fc1 x:0 0 0 2024-03-24T10:45:58.843 INFO:tasks.workunit.client.0.smithi094.stdout:6/635: creat d0/fc2 x:0 0 0 2024-03-24T10:45:58.845 INFO:tasks.workunit.client.0.smithi094.stdout:2/575: symlink d18/d7f/lac 0 2024-03-24T10:45:58.845 INFO:tasks.workunit.client.0.smithi094.stdout:2/576: fdatasync d18/f1d 0 2024-03-24T10:45:58.845 INFO:tasks.workunit.client.0.smithi094.stdout:2/577: write d18/d7f/d88/da2/da4/f5e [891470,43529] 0 2024-03-24T10:45:58.845 INFO:tasks.workunit.client.0.smithi094.stdout:2/578: chown d18/d1e/d34/d71 52528 1 2024-03-24T10:45:58.845 INFO:tasks.workunit.client.0.smithi094.stdout:5/643: rename d7/d26/d55 to d7/d23/dd5 0 2024-03-24T10:45:58.845 INFO:tasks.workunit.client.0.smithi094.stdout:5/644: read - d7/d23/dd5/da9/fab zero size 2024-03-24T10:45:58.847 INFO:tasks.workunit.client.0.smithi094.stdout:8/684: creat df/d8c/d8f/dc1/fe7 x:0 0 0 2024-03-24T10:45:58.847 INFO:tasks.workunit.client.0.smithi094.stdout:8/685: chown df/d25/d36 17323484 1 2024-03-24T10:45:58.850 INFO:tasks.workunit.client.0.smithi094.stdout:7/664: getdents d1/d9/d16/d21/d51/d83 0 2024-03-24T10:45:58.853 INFO:tasks.workunit.client.0.smithi094.stdout:2/579: creat d18/d5f/fad x:0 0 0 2024-03-24T10:45:58.855 INFO:tasks.workunit.client.0.smithi094.stdout:3/535: mkdir d0/db/dc/d28/d3c/d55/d7f/d20/d23/d8b/dba 0 2024-03-24T10:45:58.858 INFO:tasks.workunit.client.0.smithi094.stdout:3/536: symlink d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d4f/d74/lbb 0 2024-03-24T10:45:58.859 INFO:tasks.workunit.client.0.smithi094.stdout:5/645: dwrite d7/d23/d5f/d1b/f37 [4194304,4194304] 0 2024-03-24T10:45:58.859 INFO:tasks.workunit.client.0.smithi094.stdout:5/646: chown d7/d23/d5f/d14/d36/cb4 18 1 2024-03-24T10:45:58.861 INFO:tasks.workunit.client.0.smithi094.stdout:3/537: symlink d0/db/dc/d28/d50/d5e/lbc 0 2024-03-24T10:45:58.861 INFO:tasks.workunit.client.0.smithi094.stdout:3/538: fdatasync d0/db/dc/d28/d3c/d7c/fb0 0 2024-03-24T10:45:58.861 INFO:tasks.workunit.client.0.smithi094.stdout:2/580: dread d18/d1e/d34/f44 [0,4194304] 0 2024-03-24T10:45:58.862 INFO:tasks.workunit.client.0.smithi094.stdout:6/636: rename d0/d3b to d0/d78/da/d23/dc3 0 2024-03-24T10:45:58.863 INFO:tasks.workunit.client.0.smithi094.stdout:3/539: creat d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/db5/fbd x:0 0 0 2024-03-24T10:45:58.864 INFO:tasks.workunit.client.0.smithi094.stdout:2/581: symlink d18/d1e/d65/lae 0 2024-03-24T10:45:58.866 INFO:tasks.workunit.client.0.smithi094.stdout:8/686: rename df/d25/d28/d31/f88 to df/d12/d5b/d63/d75/dd9/fe8 0 2024-03-24T10:45:58.867 INFO:tasks.workunit.client.0.smithi094.stdout:2/582: link d18/d1e/ca8 d18/d77/caf 0 2024-03-24T10:45:58.868 INFO:tasks.workunit.client.0.smithi094.stdout:9/712: sync 2024-03-24T10:45:58.869 INFO:tasks.workunit.client.0.smithi094.stdout:9/713: chown df/d2d/l1d 9805 1 2024-03-24T10:45:58.869 INFO:tasks.workunit.client.0.smithi094.stdout:9/714: truncate df/d2d/fae 231369 0 2024-03-24T10:45:58.869 INFO:tasks.workunit.client.0.smithi094.stdout:9/715: getdents df/d2d/d34/dba 0 2024-03-24T10:45:58.869 INFO:tasks.workunit.client.0.smithi094.stdout:8/687: mknod df/d12/d5b/ce9 0 2024-03-24T10:45:58.870 INFO:tasks.workunit.client.0.smithi094.stdout:8/688: chown df/d25/d36/d3f/cad 6 1 2024-03-24T10:45:58.871 INFO:tasks.workunit.client.0.smithi094.stdout:3/540: dread d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/d73/f75 [0,4194304] 0 2024-03-24T10:45:58.871 INFO:tasks.workunit.client.0.smithi094.stdout:3/541: chown d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d87/c8a 556 1 2024-03-24T10:45:58.872 INFO:tasks.workunit.client.0.smithi094.stdout:2/583: creat d18/d48/d56/fb0 x:0 0 0 2024-03-24T10:45:58.872 INFO:tasks.workunit.client.0.smithi094.stdout:2/584: read d18/d1e/d34/f3d [997041,95604] 0 2024-03-24T10:45:58.872 INFO:tasks.workunit.client.0.smithi094.stdout:2/585: stat d18/d48/d56/c64 0 2024-03-24T10:45:58.873 INFO:tasks.workunit.client.0.smithi094.stdout:9/716: mkdir df/d2d/d1f/d11/d53/de3 0 2024-03-24T10:45:58.875 INFO:tasks.workunit.client.0.smithi094.stdout:3/542: dread d0/db/dc/d28/d3c/d55/d7f/d20/d23/f9a [0,4194304] 0 2024-03-24T10:45:58.875 INFO:tasks.workunit.client.0.smithi094.stdout:3/543: fsync d0/db/f2b 0 2024-03-24T10:45:58.875 INFO:tasks.workunit.client.0.smithi094.stdout:3/544: chown d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/cb4 12 1 2024-03-24T10:45:58.875 INFO:tasks.workunit.client.0.smithi094.stdout:3/545: fsync d0/f2 0 2024-03-24T10:45:58.875 INFO:tasks.workunit.client.0.smithi094.stdout:2/586: mknod d18/cb1 0 2024-03-24T10:45:58.876 INFO:tasks.workunit.client.0.smithi094.stdout:3/546: dread d0/db/dc/d28/d3c/f83 [0,4194304] 0 2024-03-24T10:45:58.876 INFO:tasks.workunit.client.0.smithi094.stdout:3/547: creat d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d86/fbe x:0 0 0 2024-03-24T10:45:58.876 INFO:tasks.workunit.client.0.smithi094.stdout:9/717: symlink df/d2d/d1f/d58/d5a/d50/da5/dd4/le4 0 2024-03-24T10:45:58.876 INFO:tasks.workunit.client.0.smithi094.stdout:3/548: write d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d4f/d74/f9f [190159,127698] 0 2024-03-24T10:45:58.877 INFO:tasks.workunit.client.0.smithi094.stdout:8/689: getdents df/d25/d36/d3f 0 2024-03-24T10:45:58.880 INFO:tasks.workunit.client.0.smithi094.stdout:9/718: mknod df/d2d/d1f/d58/ce5 0 2024-03-24T10:45:58.880 INFO:tasks.workunit.client.0.smithi094.stdout:9/719: stat df/d2d/d1f/d58/d82/fa0 0 2024-03-24T10:45:58.883 INFO:tasks.workunit.client.0.smithi094.stdout:8/690: mkdir df/d12/d5b/dea 0 2024-03-24T10:45:58.883 INFO:tasks.workunit.client.0.smithi094.stdout:2/587: dread d18/d1e/d34/f50 [0,4194304] 0 2024-03-24T10:45:58.884 INFO:tasks.workunit.client.0.smithi094.stdout:3/549: truncate d0/db/dc/d28/d3c/d55/d7f/d20/d23/f29 479224 0 2024-03-24T10:45:58.885 INFO:tasks.workunit.client.0.smithi094.stdout:0/596: sync 2024-03-24T10:45:58.887 INFO:tasks.workunit.client.0.smithi094.stdout:8/691: unlink df/d12/d5b/d63/fc7 0 2024-03-24T10:45:58.888 INFO:tasks.workunit.client.0.smithi094.stdout:2/588: mkdir d18/d7f/d88/d8a/db2 0 2024-03-24T10:45:58.889 INFO:tasks.workunit.client.0.smithi094.stdout:0/597: symlink d14/d22/d28/d48/ld3 0 2024-03-24T10:45:58.889 INFO:tasks.workunit.client.0.smithi094.stdout:0/598: write d14/d22/d32/d65/fcc [653550,44342] 0 2024-03-24T10:45:58.889 INFO:tasks.workunit.client.0.smithi094.stdout:0/599: read d14/f62 [573541,91976] 0 2024-03-24T10:45:58.892 INFO:tasks.workunit.client.0.smithi094.stdout:2/589: rmdir d18/d7f/d88/da2 39 2024-03-24T10:45:58.892 INFO:tasks.workunit.client.0.smithi094.stdout:2/590: readlink d18/d7f/l93 0 2024-03-24T10:45:58.892 INFO:tasks.workunit.client.0.smithi094.stdout:2/591: dread - d18/d1e/f76 zero size 2024-03-24T10:45:58.892 INFO:tasks.workunit.client.0.smithi094.stdout:2/592: write d18/d1e/d34/f35 [1549491,55855] 0 2024-03-24T10:45:58.893 INFO:tasks.workunit.client.0.smithi094.stdout:0/600: creat d14/d22/d28/d2b/fd4 x:0 0 0 2024-03-24T10:45:58.895 INFO:tasks.workunit.client.0.smithi094.stdout:2/593: truncate d18/f1d 690011 0 2024-03-24T10:45:58.896 INFO:tasks.workunit.client.0.smithi094.stdout:0/601: creat d14/d22/d7d/fd5 x:0 0 0 2024-03-24T10:45:58.896 INFO:tasks.workunit.client.0.smithi094.stdout:8/692: dread df/d25/d28/d31/d47/f85 [0,4194304] 0 2024-03-24T10:45:58.897 INFO:tasks.workunit.client.0.smithi094.stdout:0/602: creat d14/d22/d28/d48/fd6 x:0 0 0 2024-03-24T10:45:58.898 INFO:tasks.workunit.client.0.smithi094.stdout:8/693: rename df/d12/c17 to df/d8c/d8f/dc1/ceb 0 2024-03-24T10:45:58.899 INFO:tasks.workunit.client.0.smithi094.stdout:0/603: mknod d14/d22/d5b/d86/cd7 0 2024-03-24T10:45:58.899 INFO:tasks.workunit.client.0.smithi094.stdout:0/604: chown d14/d22/d5b/l82 57 1 2024-03-24T10:45:58.900 INFO:tasks.workunit.client.0.smithi094.stdout:0/605: dread - d14/d22/d28/da2/fd1 zero size 2024-03-24T10:45:58.900 INFO:tasks.workunit.client.0.smithi094.stdout:0/606: write d14/d22/f35 [848870,105681] 0 2024-03-24T10:45:58.900 INFO:tasks.workunit.client.0.smithi094.stdout:3/550: dwrite d0/db/dc/d28/d3c/d55/f70 [0,4194304] 0 2024-03-24T10:45:58.901 INFO:tasks.workunit.client.0.smithi094.stdout:8/694: creat df/d8c/d8f/dc1/fec x:0 0 0 2024-03-24T10:45:58.906 INFO:tasks.workunit.client.0.smithi094.stdout:3/551: creat d0/db/dc/fbf x:0 0 0 2024-03-24T10:45:58.909 INFO:tasks.workunit.client.0.smithi094.stdout:8/695: truncate f3 1743387 0 2024-03-24T10:45:58.911 INFO:tasks.workunit.client.0.smithi094.stdout:8/696: link df/d25/d36/d3f/l49 df/d25/d28/d31/d47/d66/da3/db5/dbf/led 0 2024-03-24T10:45:58.915 INFO:tasks.workunit.client.0.smithi094.stdout:3/552: dread d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/d73/f75 [0,4194304] 0 2024-03-24T10:45:58.917 INFO:tasks.workunit.client.0.smithi094.stdout:3/553: link d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/d73/cab d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/db5/cc0 0 2024-03-24T10:45:58.918 INFO:tasks.workunit.client.0.smithi094.stdout:3/554: rmdir d0/db/dc/d28/d3c/d55/db9 0 2024-03-24T10:45:58.922 INFO:tasks.workunit.client.0.smithi094.stdout:8/697: dread df/d25/f29 [0,4194304] 0 2024-03-24T10:45:58.924 INFO:tasks.workunit.client.0.smithi094.stdout:0/607: dwrite d14/d22/d28/f6c [0,4194304] 0 2024-03-24T10:45:58.930 INFO:tasks.workunit.client.0.smithi094.stdout:3/555: write d0/db/dc/d28/d3c/d55/d7f/d20/d23/f3f [3384432,8684] 0 2024-03-24T10:45:58.933 INFO:tasks.workunit.client.0.smithi094.stdout:8/698: rmdir df/d25 39 2024-03-24T10:45:58.933 INFO:tasks.workunit.client.0.smithi094.stdout:1/493: sync 2024-03-24T10:45:58.933 INFO:tasks.workunit.client.0.smithi094.stdout:4/670: sync 2024-03-24T10:45:58.935 INFO:tasks.workunit.client.0.smithi094.stdout:0/608: symlink d14/d22/d28/d2b/d41/ld8 0 2024-03-24T10:45:58.937 INFO:tasks.workunit.client.0.smithi094.stdout:3/556: chown d0/db/dc/d28/d3c/d55/d7f/d20/d23/fa2 141106085 1 2024-03-24T10:45:58.939 INFO:tasks.workunit.client.0.smithi094.stdout:3/557: rename d0/db/c41 to d0/db/cc1 0 2024-03-24T10:45:58.939 INFO:tasks.workunit.client.0.smithi094.stdout:3/558: fsync d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d4f/d74/f9f 0 2024-03-24T10:45:58.940 INFO:tasks.workunit.client.0.smithi094.stdout:4/671: dread d0/d22/de/d28/f3a [0,4194304] 0 2024-03-24T10:45:58.941 INFO:tasks.workunit.client.0.smithi094.stdout:1/494: dread d4/f4b [0,4194304] 0 2024-03-24T10:45:58.943 INFO:tasks.workunit.client.0.smithi094.stdout:4/672: symlink d0/d34/d73/d93/lcc 0 2024-03-24T10:45:58.943 INFO:tasks.workunit.client.0.smithi094.stdout:4/673: stat d0/d5/d1b/faf 0 2024-03-24T10:45:58.943 INFO:tasks.workunit.client.0.smithi094.stdout:4/674: write d0/d22/de/d28/f6b [733439,130918] 0 2024-03-24T10:45:58.944 INFO:tasks.workunit.client.0.smithi094.stdout:4/675: read d0/d22/de/d28/d2b/d33/da4/f8b [4457,107435] 0 2024-03-24T10:45:58.952 INFO:tasks.workunit.client.0.smithi094.stdout:8/699: dwrite fd [0,4194304] 0 2024-03-24T10:45:58.952 INFO:tasks.workunit.client.0.smithi094.stdout:1/495: rename d4/d9/de/d3d/la1 to d4/d9/d37/lab 0 2024-03-24T10:45:58.952 INFO:tasks.workunit.client.0.smithi094.stdout:1/496: chown d4/d36/f70 0 1 2024-03-24T10:45:58.955 INFO:tasks.workunit.client.0.smithi094.stdout:3/559: rmdir d0/db/dc 39 2024-03-24T10:45:58.955 INFO:tasks.workunit.client.0.smithi094.stdout:3/560: chown d0/db/dc/ca7 34432 1 2024-03-24T10:45:58.956 INFO:tasks.workunit.client.0.smithi094.stdout:4/676: mkdir d0/d34/d73/dcd 0 2024-03-24T10:45:58.956 INFO:tasks.workunit.client.0.smithi094.stdout:4/677: dread - d0/d5/fab zero size 2024-03-24T10:45:58.957 INFO:tasks.workunit.client.0.smithi094.stdout:8/700: symlink df/d12/d5b/d63/lee 0 2024-03-24T10:45:58.958 INFO:tasks.workunit.client.0.smithi094.stdout:1/497: symlink d4/d9/db/d5d/d63/lac 0 2024-03-24T10:45:58.958 INFO:tasks.workunit.client.0.smithi094.stdout:1/498: truncate d4/d9/d37/d33/f5b 860895 0 2024-03-24T10:45:58.958 INFO:tasks.workunit.client.0.smithi094.stdout:1/499: dread - d4/d36/f89 zero size 2024-03-24T10:45:58.959 INFO:tasks.workunit.client.0.smithi094.stdout:8/701: creat df/d25/d28/d31/d47/d66/da3/db5/fef x:0 0 0 2024-03-24T10:45:58.959 INFO:tasks.workunit.client.0.smithi094.stdout:8/702: getdents df/d25/d28/d31/d47/ddf 0 2024-03-24T10:45:58.959 INFO:tasks.workunit.client.0.smithi094.stdout:8/703: dread - df/d25/daf/fda zero size 2024-03-24T10:45:58.959 INFO:tasks.workunit.client.0.smithi094.stdout:8/704: rename df/d12/c3b to df/d25/d28/d31/d95/cf0 0 2024-03-24T10:45:58.960 INFO:tasks.workunit.client.0.smithi094.stdout:8/705: mkdir df/d25/df1 0 2024-03-24T10:45:58.962 INFO:tasks.workunit.client.0.smithi094.stdout:4/678: write d0/d22/de/d28/d2b/f83 [225652,69162] 0 2024-03-24T10:45:58.962 INFO:tasks.workunit.client.0.smithi094.stdout:8/706: mkdir df/d8c/d8f/df2 0 2024-03-24T10:45:58.962 INFO:tasks.workunit.client.0.smithi094.stdout:8/707: creat df/ff3 x:0 0 0 2024-03-24T10:45:58.963 INFO:tasks.workunit.client.0.smithi094.stdout:4/679: truncate d0/d34/db2/f55 2530333 0 2024-03-24T10:45:58.964 INFO:tasks.workunit.client.0.smithi094.stdout:1/500: write d4/d9/f2d [990315,82043] 0 2024-03-24T10:45:58.965 INFO:tasks.workunit.client.0.smithi094.stdout:1/501: chown d4/d9/d7d/l59 513872 1 2024-03-24T10:45:58.965 INFO:tasks.workunit.client.0.smithi094.stdout:1/502: fsync d4/d9/d37/d33/d58/f93 0 2024-03-24T10:45:58.965 INFO:tasks.workunit.client.0.smithi094.stdout:4/680: symlink d0/d22/de/d28/d2b/d33/da4/da5/lce 0 2024-03-24T10:45:58.966 INFO:tasks.workunit.client.0.smithi094.stdout:1/503: creat d4/d9/d7d/fad x:0 0 0 2024-03-24T10:45:58.966 INFO:tasks.workunit.client.0.smithi094.stdout:1/504: write d4/d9/de/d6f/d62/d80/f8f [1022813,4803] 0 2024-03-24T10:45:58.968 INFO:tasks.workunit.client.0.smithi094.stdout:4/681: mkdir d0/d5/d7d/dcf 0 2024-03-24T10:45:58.972 INFO:tasks.workunit.client.0.smithi094.stdout:4/682: rename d0/d22/f11 to d0/d22/de/d28/d2b/fd0 0 2024-03-24T10:45:58.972 INFO:tasks.workunit.client.0.smithi094.stdout:4/683: read d0/d22/de/d28/d2b/f83 [3124459,80540] 0 2024-03-24T10:45:58.972 INFO:tasks.workunit.client.0.smithi094.stdout:4/684: truncate d0/d22/de/d28/f59 4423356 0 2024-03-24T10:45:58.972 INFO:tasks.workunit.client.0.smithi094.stdout:4/685: write d0/d22/de/d28/fbb [34686,60211] 0 2024-03-24T10:45:58.974 INFO:tasks.workunit.client.0.smithi094.stdout:8/708: read df/d12/d5b/f83 [5782340,88250] 0 2024-03-24T10:45:58.975 INFO:tasks.workunit.client.0.smithi094.stdout:4/686: rename d0/d5/d7d/c8e to d0/d22/de/d28/d2b/d33/cd1 0 2024-03-24T10:45:58.976 INFO:tasks.workunit.client.0.smithi094.stdout:8/709: symlink df/d8c/d8f/df2/lf4 0 2024-03-24T10:45:58.978 INFO:tasks.workunit.client.0.smithi094.stdout:4/687: truncate d0/d5/d1b/d4f/d70/f7c 545351 0 2024-03-24T10:45:58.980 INFO:tasks.workunit.client.0.smithi094.stdout:8/710: creat df/d8c/d8f/df2/ff5 x:0 0 0 2024-03-24T10:45:58.986 INFO:tasks.workunit.client.0.smithi094.stdout:3/561: dwrite d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/d73/f75 [0,4194304] 0 2024-03-24T10:45:58.993 INFO:tasks.workunit.client.0.smithi094.stdout:3/562: rename d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/d73/f75 to d0/db/dc/d28/d3c/d55/d7f/d20/d23/fc2 0 2024-03-24T10:45:58.995 INFO:tasks.workunit.client.0.smithi094.stdout:3/563: unlink d0/db/dc/d28/d3c/d55/d7f/d20/d23/c98 0 2024-03-24T10:45:58.996 INFO:tasks.workunit.client.0.smithi094.stdout:3/564: mkdir d0/db/dc3 0 2024-03-24T10:45:58.997 INFO:tasks.workunit.client.0.smithi094.stdout:3/565: creat d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d34/d71/fc4 x:0 0 0 2024-03-24T10:45:58.997 INFO:tasks.workunit.client.0.smithi094.stdout:3/566: dread - d0/fb6 zero size 2024-03-24T10:45:58.997 INFO:tasks.workunit.client.0.smithi094.stdout:3/567: mkdir d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d87/dc5 0 2024-03-24T10:45:58.999 INFO:tasks.workunit.client.0.smithi094.stdout:3/568: creat d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/db5/fc6 x:0 0 0 2024-03-24T10:45:58.999 INFO:tasks.workunit.client.0.smithi094.stdout:3/569: write d0/db/dc/d28/d3c/d55/d7f/d20/fae [937289,61842] 0 2024-03-24T10:45:58.999 INFO:tasks.workunit.client.0.smithi094.stdout:4/688: dwrite d0/d34/f82 [0,4194304] 0 2024-03-24T10:45:59.000 INFO:tasks.workunit.client.0.smithi094.stdout:1/505: dwrite d4/d9/f2c [0,4194304] 0 2024-03-24T10:45:59.000 INFO:tasks.workunit.client.0.smithi094.stdout:3/570: dread d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/f64 [0,4194304] 0 2024-03-24T10:45:59.000 INFO:tasks.workunit.client.0.smithi094.stdout:8/711: dwrite df/d25/d28/d31/d47/d22/f38 [0,4194304] 0 2024-03-24T10:45:59.006 INFO:tasks.workunit.client.0.smithi094.stdout:4/689: dread d0/f5e [0,4194304] 0 2024-03-24T10:45:59.008 INFO:tasks.workunit.client.0.smithi094.stdout:1/506: stat d4/d9/l26 0 2024-03-24T10:45:59.009 INFO:tasks.workunit.client.0.smithi094.stdout:3/571: mknod d0/db/dc/d28/cc7 0 2024-03-24T10:45:59.010 INFO:tasks.workunit.client.0.smithi094.stdout:8/712: unlink df/d12/d5b/d63/fbc 0 2024-03-24T10:45:59.011 INFO:tasks.workunit.client.0.smithi094.stdout:4/690: rmdir d0/d22/de/d28/d2b/d7a 39 2024-03-24T10:45:59.012 INFO:tasks.workunit.client.0.smithi094.stdout:1/507: mkdir d4/d9/de/d3d/dae 0 2024-03-24T10:45:59.012 INFO:tasks.workunit.client.0.smithi094.stdout:1/508: creat d4/d9/de/d3d/faf x:0 0 0 2024-03-24T10:45:59.014 INFO:tasks.workunit.client.0.smithi094.stdout:3/572: rmdir d0/db/dc/d28/d3c/d55/d7f/d20/d23/d8b/dba 0 2024-03-24T10:45:59.014 INFO:tasks.workunit.client.0.smithi094.stdout:3/573: write d0/db/dc/d28/d3c/d55/d7f/d20/d23/f3f [181814,58564] 0 2024-03-24T10:45:59.016 INFO:tasks.workunit.client.0.smithi094.stdout:4/691: symlink d0/d22/de/d28/d2b/d33/da4/da5/ld2 0 2024-03-24T10:45:59.016 INFO:tasks.workunit.client.0.smithi094.stdout:4/692: readlink d0/d34/db2/l9a 0 2024-03-24T10:45:59.016 INFO:tasks.workunit.client.0.smithi094.stdout:4/693: truncate d0/d22/de/d28/d2b/f43 1432831 0 2024-03-24T10:45:59.018 INFO:tasks.workunit.client.0.smithi094.stdout:1/509: mknod d4/d9/db/d31/d3c/cb0 0 2024-03-24T10:45:59.020 INFO:tasks.workunit.client.0.smithi094.stdout:3/574: creat d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/fc8 x:0 0 0 2024-03-24T10:45:59.022 INFO:tasks.workunit.client.0.smithi094.stdout:4/694: creat d0/d22/de/d28/d2b/fd3 x:0 0 0 2024-03-24T10:45:59.022 INFO:tasks.workunit.client.0.smithi094.stdout:4/695: readlink d0/d34/lba 0 2024-03-24T10:45:59.022 INFO:tasks.workunit.client.0.smithi094.stdout:4/696: write d0/d34/d73/d93/fa3 [4514940,126735] 0 2024-03-24T10:45:59.023 INFO:tasks.workunit.client.0.smithi094.stdout:1/510: mkdir d4/d9/db1 0 2024-03-24T10:45:59.024 INFO:tasks.workunit.client.0.smithi094.stdout:6/637: sync 2024-03-24T10:45:59.024 INFO:tasks.workunit.client.0.smithi094.stdout:5/647: sync 2024-03-24T10:45:59.024 INFO:tasks.workunit.client.0.smithi094.stdout:7/665: sync 2024-03-24T10:45:59.024 INFO:tasks.workunit.client.0.smithi094.stdout:7/666: truncate d1/d9/fd1 872890 0 2024-03-24T10:45:59.024 INFO:tasks.workunit.client.0.smithi094.stdout:9/720: sync 2024-03-24T10:45:59.024 INFO:tasks.workunit.client.0.smithi094.stdout:7/667: write d1/db4/f6b [2055657,1962] 0 2024-03-24T10:45:59.024 INFO:tasks.workunit.client.0.smithi094.stdout:5/648: write d7/d23/d5f/d9/f49 [9182381,31134] 0 2024-03-24T10:45:59.025 INFO:tasks.workunit.client.0.smithi094.stdout:4/697: mknod d0/d5/d7d/d97/cd4 0 2024-03-24T10:45:59.025 INFO:tasks.workunit.client.0.smithi094.stdout:4/698: chown d0/d5/d7d/d9c/cc6 17 1 2024-03-24T10:45:59.025 INFO:tasks.workunit.client.0.smithi094.stdout:2/594: sync 2024-03-24T10:45:59.025 INFO:tasks.workunit.client.0.smithi094.stdout:6/638: mknod d0/d21/d2a/d55/cc4 0 2024-03-24T10:45:59.028 INFO:tasks.workunit.client.0.smithi094.stdout:7/668: rename d1/d9/d14/fbb to d1/db4/d65/fd8 0 2024-03-24T10:45:59.028 INFO:tasks.workunit.client.0.smithi094.stdout:0/609: sync 2024-03-24T10:45:59.030 INFO:tasks.workunit.client.0.smithi094.stdout:9/721: creat df/d2d/d1f/d58/d5a/d69/d7a/fe6 x:0 0 0 2024-03-24T10:45:59.031 INFO:tasks.workunit.client.0.smithi094.stdout:5/649: write d7/d23/d5f/d14/d36/f47 [271020,59438] 0 2024-03-24T10:45:59.031 INFO:tasks.workunit.client.0.smithi094.stdout:8/713: rmdir df/d12/d5b/d63 39 2024-03-24T10:45:59.031 INFO:tasks.workunit.client.0.smithi094.stdout:8/714: readlink df/d8c/d8f/dc1/le6 0 2024-03-24T10:45:59.031 INFO:tasks.workunit.client.0.smithi094.stdout:3/575: dread d0/db/f48 [0,4194304] 0 2024-03-24T10:45:59.033 INFO:tasks.workunit.client.0.smithi094.stdout:4/699: symlink d0/d22/de/d28/d2b/d7a/ld5 0 2024-03-24T10:45:59.033 INFO:tasks.workunit.client.0.smithi094.stdout:1/511: dread d4/d9/db/d31/d3c/f50 [0,4194304] 0 2024-03-24T10:45:59.033 INFO:tasks.workunit.client.0.smithi094.stdout:1/512: chown d4/d9/de/d3d/d66 2872 1 2024-03-24T10:45:59.034 INFO:tasks.workunit.client.0.smithi094.stdout:2/595: symlink d18/d7f/d88/da2/da4/lb3 0 2024-03-24T10:45:59.034 INFO:tasks.workunit.client.0.smithi094.stdout:5/650: write d7/dc/fa2 [990688,8057] 0 2024-03-24T10:45:59.035 INFO:tasks.workunit.client.0.smithi094.stdout:6/639: symlink d0/d30/d50/lc5 0 2024-03-24T10:45:59.035 INFO:tasks.workunit.client.0.smithi094.stdout:6/640: write d0/d78/da/d23/f68 [4212172,38942] 0 2024-03-24T10:45:59.037 INFO:tasks.workunit.client.0.smithi094.stdout:0/610: mknod d14/d22/d28/da2/cd9 0 2024-03-24T10:45:59.037 INFO:tasks.workunit.client.0.smithi094.stdout:0/611: fdatasync d14/f1b 0 2024-03-24T10:45:59.039 INFO:tasks.workunit.client.0.smithi094.stdout:9/722: unlink df/d2d/d34/f39 0 2024-03-24T10:45:59.041 INFO:tasks.workunit.client.0.smithi094.stdout:8/715: creat df/d25/dce/ff6 x:0 0 0 2024-03-24T10:45:59.041 INFO:tasks.workunit.client.0.smithi094.stdout:8/716: dread - df/d12/d5b/d63/fa9 zero size 2024-03-24T10:45:59.042 INFO:tasks.workunit.client.0.smithi094.stdout:3/576: mkdir d0/db/dc/d28/d38/dc9 0 2024-03-24T10:45:59.042 INFO:tasks.workunit.client.0.smithi094.stdout:3/577: truncate d0/db/dc/d28/d3c/d55/d7f/d20/d23/f3f 4274320 0 2024-03-24T10:45:59.044 INFO:tasks.workunit.client.0.smithi094.stdout:4/700: unlink d0/d22/de/d28/d2b/f43 0 2024-03-24T10:45:59.046 INFO:tasks.workunit.client.0.smithi094.stdout:1/513: rename d4/d9/l26 to d4/d9/de/d6f/d5f/lb2 0 2024-03-24T10:45:59.047 INFO:tasks.workunit.client.0.smithi094.stdout:2/596: mkdir d18/d66/d90/d9a/d74/db4 0 2024-03-24T10:45:59.047 INFO:tasks.workunit.client.0.smithi094.stdout:2/597: fdatasync d18/f31 0 2024-03-24T10:45:59.048 INFO:tasks.workunit.client.0.smithi094.stdout:2/598: truncate d18/d1e/d34/f3d 4578742 0 2024-03-24T10:45:59.048 INFO:tasks.workunit.client.0.smithi094.stdout:2/599: creat d18/d5f/fb5 x:0 0 0 2024-03-24T10:45:59.053 INFO:tasks.workunit.client.0.smithi094.stdout:6/641: creat d0/d78/da/d23/d7c/db0/fc6 x:0 0 0 2024-03-24T10:45:59.054 INFO:tasks.workunit.client.0.smithi094.stdout:0/612: unlink d14/d22/d28/d48/fd6 0 2024-03-24T10:45:59.056 INFO:tasks.workunit.client.0.smithi094.stdout:3/578: mkdir d0/db/dc/d28/d38/d6a/d9b/dca 0 2024-03-24T10:45:59.059 INFO:tasks.workunit.client.0.smithi094.stdout:2/600: creat d18/d1e/d34/d4a/fb6 x:0 0 0 2024-03-24T10:45:59.059 INFO:tasks.workunit.client.0.smithi094.stdout:2/601: creat d18/d54/fb7 x:0 0 0 2024-03-24T10:45:59.059 INFO:tasks.workunit.client.0.smithi094.stdout:2/602: write d18/f1b [2817224,33440] 0 2024-03-24T10:45:59.059 INFO:tasks.workunit.client.0.smithi094.stdout:2/603: truncate d18/d66/faa 426172 0 2024-03-24T10:45:59.060 INFO:tasks.workunit.client.0.smithi094.stdout:5/651: getdents d7/d23/d5f/d9 0 2024-03-24T10:45:59.060 INFO:tasks.workunit.client.0.smithi094.stdout:5/652: fdatasync d7/d23/f44 0 2024-03-24T10:45:59.060 INFO:tasks.workunit.client.0.smithi094.stdout:4/701: dwrite d0/d22/de/d28/d3d/d8c/fc7 [0,4194304] 0 2024-03-24T10:45:59.063 INFO:tasks.workunit.client.0.smithi094.stdout:0/613: unlink d14/d22/d32/d65/fab 0 2024-03-24T10:45:59.063 INFO:tasks.workunit.client.0.smithi094.stdout:0/614: creat d14/d22/d32/d65/fda x:0 0 0 2024-03-24T10:45:59.064 INFO:tasks.workunit.client.0.smithi094.stdout:3/579: mknod d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d34/ccb 0 2024-03-24T10:45:59.064 INFO:tasks.workunit.client.0.smithi094.stdout:3/580: read d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d4f/d74/f9f [283972,81776] 0 2024-03-24T10:45:59.070 INFO:tasks.workunit.client.0.smithi094.stdout:0/615: symlink d14/d22/d28/d2b/ldb 0 2024-03-24T10:45:59.072 INFO:tasks.workunit.client.0.smithi094.stdout:3/581: creat d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/fcc x:0 0 0 2024-03-24T10:45:59.072 INFO:tasks.workunit.client.0.smithi094.stdout:3/582: fsync d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d34/fa0 0 2024-03-24T10:45:59.074 INFO:tasks.workunit.client.0.smithi094.stdout:5/653: rename d7/dc/d12 to d7/d23/dd5/d64/dd6 0 2024-03-24T10:45:59.076 INFO:tasks.workunit.client.0.smithi094.stdout:5/654: rmdir d7/d23/d5f/d14/d6f/dbe 39 2024-03-24T10:45:59.076 INFO:tasks.workunit.client.0.smithi094.stdout:5/655: write d7/d23/d2b/d68/f8f [1036383,114719] 0 2024-03-24T10:45:59.077 INFO:tasks.workunit.client.0.smithi094.stdout:5/656: symlink d7/dc/d56/ld7 0 2024-03-24T10:45:59.081 INFO:tasks.workunit.client.0.smithi094.stdout:1/514: dwrite d4/d36/f73 [0,4194304] 0 2024-03-24T10:45:59.083 INFO:tasks.workunit.client.0.smithi094.stdout:1/515: mknod d4/d9/cb3 0 2024-03-24T10:45:59.083 INFO:tasks.workunit.client.0.smithi094.stdout:1/516: chown d4/d9/d37/d33/d76 0 1 2024-03-24T10:45:59.083 INFO:tasks.workunit.client.0.smithi094.stdout:0/616: dread d14/d22/f33 [0,4194304] 0 2024-03-24T10:45:59.083 INFO:tasks.workunit.client.0.smithi094.stdout:0/617: read - d14/d18/f9d zero size 2024-03-24T10:45:59.083 INFO:tasks.workunit.client.0.smithi094.stdout:0/618: chown d14/d22/d32/d70/dd0/c95 3 1 2024-03-24T10:45:59.083 INFO:tasks.workunit.client.0.smithi094.stdout:1/517: getdents d4/d36 0 2024-03-24T10:45:59.084 INFO:tasks.workunit.client.0.smithi094.stdout:0/619: symlink d14/d22/d5b/ldc 0 2024-03-24T10:45:59.085 INFO:tasks.workunit.client.0.smithi094.stdout:0/620: mkdir d14/d22/d28/dac/ddd 0 2024-03-24T10:45:59.092 INFO:tasks.workunit.client.0.smithi094.stdout:6/642: dwrite d0/d78/da/d77/db6/f9e [0,4194304] 0 2024-03-24T10:45:59.092 INFO:tasks.workunit.client.0.smithi094.stdout:6/643: creat d0/d78/da/d23/d7c/db0/fc7 x:0 0 0 2024-03-24T10:45:59.092 INFO:tasks.workunit.client.0.smithi094.stdout:6/644: chown d0/d78/da/d23/d7c/db0 241 1 2024-03-24T10:45:59.092 INFO:tasks.workunit.client.0.smithi094.stdout:6/645: write d0/d78/da/d23/d34/d4e/fa7 [254864,71812] 0 2024-03-24T10:45:59.093 INFO:tasks.workunit.client.0.smithi094.stdout:2/604: dwrite d18/d54/f5c [4194304,4194304] 0 2024-03-24T10:45:59.094 INFO:tasks.workunit.client.0.smithi094.stdout:6/646: truncate d0/d78/da/d23/f75 798007 0 2024-03-24T10:45:59.096 INFO:tasks.workunit.client.0.smithi094.stdout:2/605: mknod d18/d66/cb8 0 2024-03-24T10:45:59.097 INFO:tasks.workunit.client.0.smithi094.stdout:6/647: symlink d0/d30/lc8 0 2024-03-24T10:45:59.097 INFO:tasks.workunit.client.0.smithi094.stdout:1/518: dread d4/d9/db/d31/d3c/f7c [0,4194304] 0 2024-03-24T10:45:59.098 INFO:tasks.workunit.client.0.smithi094.stdout:6/648: rename d0/da9/db5 to d0/d78/da/dc9 0 2024-03-24T10:45:59.098 INFO:tasks.workunit.client.0.smithi094.stdout:6/649: truncate d0/d30/f5b 895381 0 2024-03-24T10:45:59.099 INFO:tasks.workunit.client.0.smithi094.stdout:1/519: symlink d4/d9/de/d6f/d5f/lb4 0 2024-03-24T10:45:59.100 INFO:tasks.workunit.client.0.smithi094.stdout:6/650: mkdir d0/d1a/dca 0 2024-03-24T10:45:59.100 INFO:tasks.workunit.client.0.smithi094.stdout:6/651: mkdir d0/d21/d2a/dcb 0 2024-03-24T10:45:59.107 INFO:tasks.workunit.client.0.smithi094.stdout:6/652: write d0/d78/da/d77/db6/f9e [3556012,111302] 0 2024-03-24T10:45:59.108 INFO:tasks.workunit.client.0.smithi094.stdout:5/657: dread d7/d23/d2b/d68/f8f [0,4194304] 0 2024-03-24T10:45:59.108 INFO:tasks.workunit.client.0.smithi094.stdout:3/583: dwrite d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/d73/fb1 [0,4194304] 0 2024-03-24T10:45:59.108 INFO:tasks.workunit.client.0.smithi094.stdout:3/584: chown d0/db/dc/c2c 14441151 1 2024-03-24T10:45:59.108 INFO:tasks.workunit.client.0.smithi094.stdout:3/585: readlink d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/l4a 0 2024-03-24T10:45:59.109 INFO:tasks.workunit.client.0.smithi094.stdout:5/658: write d7/d23/d5f/d14/f85 [511484,61279] 0 2024-03-24T10:45:59.111 INFO:tasks.workunit.client.0.smithi094.stdout:0/621: dwrite d14/d22/d28/d2b/d68/fba [0,4194304] 0 2024-03-24T10:45:59.111 INFO:tasks.workunit.client.0.smithi094.stdout:1/520: write d4/d9/f32 [2371229,93774] 0 2024-03-24T10:45:59.111 INFO:tasks.workunit.client.0.smithi094.stdout:0/622: creat d14/d18/fde x:0 0 0 2024-03-24T10:45:59.113 INFO:tasks.workunit.client.0.smithi094.stdout:2/606: dread d18/d1e/d34/f43 [0,4194304] 0 2024-03-24T10:45:59.115 INFO:tasks.workunit.client.0.smithi094.stdout:6/653: unlink d0/d21/f13 0 2024-03-24T10:45:59.119 INFO:tasks.workunit.client.0.smithi094.stdout:5/659: write d7/d23/d5f/d14/d6f/dbe/fd1 [4692487,87074] 0 2024-03-24T10:45:59.124 INFO:tasks.workunit.client.0.smithi094.stdout:1/521: unlink d4/d9/f27 0 2024-03-24T10:45:59.124 INFO:tasks.workunit.client.0.smithi094.stdout:1/522: fsync f1 0 2024-03-24T10:45:59.125 INFO:tasks.workunit.client.0.smithi094.stdout:3/586: dwrite d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d4f/d74/f9f [0,4194304] 0 2024-03-24T10:45:59.126 INFO:tasks.workunit.client.0.smithi094.stdout:0/623: rmdir d14/d22/d5b 39 2024-03-24T10:45:59.126 INFO:tasks.workunit.client.0.smithi094.stdout:0/624: readlink d14/d22/d28/d2b/l6d 0 2024-03-24T10:45:59.126 INFO:tasks.workunit.client.0.smithi094.stdout:0/625: dread - d14/d22/d5b/fb7 zero size 2024-03-24T10:45:59.127 INFO:tasks.workunit.client.0.smithi094.stdout:2/607: mkdir d18/d54/db9 0 2024-03-24T10:45:59.129 INFO:tasks.workunit.client.0.smithi094.stdout:6/654: rename d0/d78/l8f to d0/d30/d50/lcc 0 2024-03-24T10:45:59.132 INFO:tasks.workunit.client.0.smithi094.stdout:5/660: link d7/d23/d5f/d9/d48/fa7 d7/d23/d5f/d9/d48/fd8 0 2024-03-24T10:45:59.134 INFO:tasks.workunit.client.0.smithi094.stdout:1/523: truncate d4/d9/d7d/f4f 670039 0 2024-03-24T10:45:59.134 INFO:tasks.workunit.client.0.smithi094.stdout:1/524: chown d4/d9/d37/d33/d58 2597 1 2024-03-24T10:45:59.138 INFO:tasks.workunit.client.0.smithi094.stdout:2/608: unlink d18/d1e/c41 0 2024-03-24T10:45:59.139 INFO:tasks.workunit.client.0.smithi094.stdout:6/655: unlink d0/d1a/l9f 0 2024-03-24T10:45:59.139 INFO:tasks.workunit.client.0.smithi094.stdout:0/626: write d14/d22/d28/f31 [2075771,119788] 0 2024-03-24T10:45:59.139 INFO:tasks.workunit.client.0.smithi094.stdout:6/656: creat d0/da9/fcd x:0 0 0 2024-03-24T10:45:59.139 INFO:tasks.workunit.client.0.smithi094.stdout:0/627: write d14/d18/fc2 [569771,6451] 0 2024-03-24T10:45:59.140 INFO:tasks.workunit.client.0.smithi094.stdout:5/661: truncate d7/d26/f53 2492336 0 2024-03-24T10:45:59.141 INFO:tasks.workunit.client.0.smithi094.stdout:1/525: mkdir d4/d9/de/d6f/d62/d99/db5 0 2024-03-24T10:45:59.144 INFO:tasks.workunit.client.0.smithi094.stdout:0/628: mkdir d14/d22/d28/dac/ddf 0 2024-03-24T10:45:59.146 INFO:tasks.workunit.client.0.smithi094.stdout:5/662: creat d7/d23/d5f/d14/d6f/dbe/fd9 x:0 0 0 2024-03-24T10:45:59.146 INFO:tasks.workunit.client.0.smithi094.stdout:5/663: chown d7/d23/d5f/d14/f9a 345990885 1 2024-03-24T10:45:59.146 INFO:tasks.workunit.client.0.smithi094.stdout:3/587: dwrite d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/f5b [0,4194304] 0 2024-03-24T10:45:59.147 INFO:tasks.workunit.client.0.smithi094.stdout:3/588: truncate d0/db/dc/d28/d38/fa6 282406 0 2024-03-24T10:45:59.147 INFO:tasks.workunit.client.0.smithi094.stdout:3/589: fdatasync d0/db/dc/d28/d3c/d55/d7f/d20/d23/fc2 0 2024-03-24T10:45:59.149 INFO:tasks.workunit.client.0.smithi094.stdout:7/669: sync 2024-03-24T10:45:59.153 INFO:tasks.workunit.client.0.smithi094.stdout:2/609: dread d18/d1e/d34/d4a/f6d [0,4194304] 0 2024-03-24T10:45:59.154 INFO:tasks.workunit.client.0.smithi094.stdout:2/610: read d18/d1e/d34/f84 [469740,32680] 0 2024-03-24T10:45:59.156 INFO:tasks.workunit.client.0.smithi094.stdout:1/526: dread d4/d9/de/fa7 [0,4194304] 0 2024-03-24T10:45:59.157 INFO:tasks.workunit.client.0.smithi094.stdout:2/611: rmdir d18/d7f/d88/da2 39 2024-03-24T10:45:59.157 INFO:tasks.workunit.client.0.smithi094.stdout:2/612: chown d18/d77/c79 1615800 1 2024-03-24T10:45:59.159 INFO:tasks.workunit.client.0.smithi094.stdout:1/527: mknod d4/d9/d37/d33/cb6 0 2024-03-24T10:45:59.159 INFO:tasks.workunit.client.0.smithi094.stdout:5/664: dwrite d7/d23/d5f/d14/f6d [0,4194304] 0 2024-03-24T10:45:59.159 INFO:tasks.workunit.client.0.smithi094.stdout:5/665: write d7/d23/d5f/d14/d36/f47 [811930,106588] 0 2024-03-24T10:45:59.160 INFO:tasks.workunit.client.0.smithi094.stdout:2/613: creat d18/d54/db9/fba x:0 0 0 2024-03-24T10:45:59.162 INFO:tasks.workunit.client.0.smithi094.stdout:1/528: rename d4/d9/d37/d33/d58/l87 to d4/d9/db/d39/lb7 0 2024-03-24T10:45:59.166 INFO:tasks.workunit.client.0.smithi094.stdout:5/666: symlink d7/d26/dce/lda 0 2024-03-24T10:45:59.166 INFO:tasks.workunit.client.0.smithi094.stdout:5/667: fdatasync d7/d23/d5f/d9/d48/f6b 0 2024-03-24T10:45:59.166 INFO:tasks.workunit.client.0.smithi094.stdout:5/668: dread - d7/d23/dd5/fae zero size 2024-03-24T10:45:59.166 INFO:tasks.workunit.client.0.smithi094.stdout:5/669: creat d7/d23/d5f/d14/d6f/dbe/fdb x:0 0 0 2024-03-24T10:45:59.166 INFO:tasks.workunit.client.0.smithi094.stdout:5/670: write d7/d26/f65 [299189,72122] 0 2024-03-24T10:45:59.167 INFO:tasks.workunit.client.0.smithi094.stdout:7/670: dread d1/d9/d25/d3d/f98 [0,4194304] 0 2024-03-24T10:45:59.168 INFO:tasks.workunit.client.0.smithi094.stdout:7/671: write d1/db4/d65/f7d [167286,85112] 0 2024-03-24T10:45:59.168 INFO:tasks.workunit.client.0.smithi094.stdout:2/614: mknod d18/d48/cbb 0 2024-03-24T10:45:59.171 INFO:tasks.workunit.client.0.smithi094.stdout:3/590: dread d0/db/dc/d28/d3c/d55/f70 [0,4194304] 0 2024-03-24T10:45:59.171 INFO:tasks.workunit.client.0.smithi094.stdout:3/591: stat d0 0 2024-03-24T10:45:59.176 INFO:tasks.workunit.client.0.smithi094.stdout:3/592: mknod d0/db/dc3/ccd 0 2024-03-24T10:45:59.177 INFO:tasks.workunit.client.0.smithi094.stdout:6/657: dwrite d0/d21/f2b [4194304,4194304] 0 2024-03-24T10:45:59.178 INFO:tasks.workunit.client.0.smithi094.stdout:7/672: rename d1/d9/d14 to d1/d9/d16/d21/d51/d83/d39/d44/dd9 0 2024-03-24T10:45:59.179 INFO:tasks.workunit.client.0.smithi094.stdout:7/673: truncate d1/d9/fac 438759 0 2024-03-24T10:45:59.179 INFO:tasks.workunit.client.0.smithi094.stdout:7/674: creat d1/d9/d25/d3d/fda x:0 0 0 2024-03-24T10:45:59.179 INFO:tasks.workunit.client.0.smithi094.stdout:7/675: readlink d1/d9/d16/d21/d51/d83/d39/lb5 0 2024-03-24T10:45:59.179 INFO:tasks.workunit.client.0.smithi094.stdout:9/723: sync 2024-03-24T10:45:59.180 INFO:tasks.workunit.client.0.smithi094.stdout:3/593: mknod d0/db/dc3/cce 0 2024-03-24T10:45:59.181 INFO:tasks.workunit.client.0.smithi094.stdout:6/658: mkdir d0/d21/d2a/d55/dce 0 2024-03-24T10:45:59.184 INFO:tasks.workunit.client.0.smithi094.stdout:3/594: read d0/db/dc/d28/d3c/d55/d7f/d20/d23/f29 [105955,94541] 0 2024-03-24T10:45:59.184 INFO:tasks.workunit.client.0.smithi094.stdout:3/595: write d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d86/fbe [287910,56715] 0 2024-03-24T10:45:59.184 INFO:tasks.workunit.client.0.smithi094.stdout:3/596: chown d0/db/dc/d28/d3c/d55/d7f/d20/d23/d8b/la8 1573099 1 2024-03-24T10:45:59.184 INFO:tasks.workunit.client.0.smithi094.stdout:7/676: unlink d1/l29 0 2024-03-24T10:45:59.185 INFO:tasks.workunit.client.0.smithi094.stdout:0/629: dwrite d14/d18/f69 [0,4194304] 0 2024-03-24T10:45:59.185 INFO:tasks.workunit.client.0.smithi094.stdout:5/671: dwrite d7/d23/d5f/d9/f4f [0,4194304] 0 2024-03-24T10:45:59.185 INFO:tasks.workunit.client.0.smithi094.stdout:5/672: chown d7/d23/d2b/l38 3 1 2024-03-24T10:45:59.186 INFO:tasks.workunit.client.0.smithi094.stdout:2/615: rename d18/d1e/d34 to d18/d66/d90/d9a/dbc 0 2024-03-24T10:45:59.186 INFO:tasks.workunit.client.0.smithi094.stdout:2/616: getdents d18/d66/d90/d9a/dbc/d71 0 2024-03-24T10:45:59.189 INFO:tasks.workunit.client.0.smithi094.stdout:7/677: mknod d1/d9/d16/d21/d51/d83/d39/cdb 0 2024-03-24T10:45:59.189 INFO:tasks.workunit.client.0.smithi094.stdout:1/529: rmdir d4/d9/db/d39 39 2024-03-24T10:45:59.192 INFO:tasks.workunit.client.0.smithi094.stdout:0/630: unlink d14/d22/d28/d48/f4a 0 2024-03-24T10:45:59.196 INFO:tasks.workunit.client.0.smithi094.stdout:5/673: unlink d7/d23/d5f/d14/d6f/dbe/fd3 0 2024-03-24T10:45:59.204 INFO:tasks.workunit.client.0.smithi094.stdout:6/659: rename d0/d78/c60 to d0/d78/d66/ccf 0 2024-03-24T10:45:59.208 INFO:tasks.workunit.client.0.smithi094.stdout:2/617: unlink d18/d48/l4b 0 2024-03-24T10:45:59.210 INFO:tasks.workunit.client.0.smithi094.stdout:3/597: rmdir d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25 39 2024-03-24T10:45:59.217 INFO:tasks.workunit.client.0.smithi094.stdout:5/674: link d7/d23/d5f/lcf d7/dc/d56/ldc 0 2024-03-24T10:45:59.219 INFO:tasks.workunit.client.0.smithi094.stdout:5/675: dread d7/d23/d5f/d14/d6f/f91 [0,4194304] 0 2024-03-24T10:45:59.219 INFO:tasks.workunit.client.0.smithi094.stdout:3/598: rename d0/f2 to d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/fcf 0 2024-03-24T10:45:59.220 INFO:tasks.workunit.client.0.smithi094.stdout:1/530: getdents d4/d9/de/d6f/d62/d80 0 2024-03-24T10:45:59.222 INFO:tasks.workunit.client.0.smithi094.stdout:3/599: creat d0/db/dc/d28/d3c/fd0 x:0 0 0 2024-03-24T10:45:59.222 INFO:tasks.workunit.client.0.smithi094.stdout:3/600: chown d0/fb6 104617 1 2024-03-24T10:45:59.224 INFO:tasks.workunit.client.0.smithi094.stdout:3/601: rmdir d0/db 39 2024-03-24T10:45:59.226 INFO:tasks.workunit.client.0.smithi094.stdout:3/602: creat d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/fd1 x:0 0 0 2024-03-24T10:45:59.229 INFO:tasks.workunit.client.0.smithi094.stdout:3/603: unlink d0/db/dc/d28/d3c/d55/d7f/d20/c65 0 2024-03-24T10:45:59.229 INFO:tasks.workunit.client.0.smithi094.stdout:8/717: sync 2024-03-24T10:45:59.229 INFO:tasks.workunit.client.0.smithi094.stdout:4/702: sync 2024-03-24T10:45:59.230 INFO:tasks.workunit.client.0.smithi094.stdout:3/604: mknod d0/db/dc/d28/d38/d53/cd2 0 2024-03-24T10:45:59.231 INFO:tasks.workunit.client.0.smithi094.stdout:4/703: mkdir d0/d5/d1b/d4f/dd6 0 2024-03-24T10:45:59.231 INFO:tasks.workunit.client.0.smithi094.stdout:4/704: write d0/d22/de/d28/d2b/d7a/fbd [814560,110181] 0 2024-03-24T10:45:59.231 INFO:tasks.workunit.client.0.smithi094.stdout:4/705: chown d0/l65 41497635 1 2024-03-24T10:45:59.232 INFO:tasks.workunit.client.0.smithi094.stdout:3/605: creat d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d34/fd3 x:0 0 0 2024-03-24T10:45:59.232 INFO:tasks.workunit.client.0.smithi094.stdout:3/606: chown d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/c49 8173 1 2024-03-24T10:45:59.233 INFO:tasks.workunit.client.0.smithi094.stdout:3/607: truncate d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d34/fa0 398713 0 2024-03-24T10:45:59.233 INFO:tasks.workunit.client.0.smithi094.stdout:3/608: write d0/db/dc/d28/d3c/d55/d7f/d20/d23/f3f [4724018,89650] 0 2024-03-24T10:45:59.240 INFO:tasks.workunit.client.0.smithi094.stdout:1/531: dread d4/d9/d37/d33/d58/f88 [0,4194304] 0 2024-03-24T10:45:59.240 INFO:tasks.workunit.client.0.smithi094.stdout:4/706: dread d0/d5/d7d/fc3 [4194304,4194304] 0 2024-03-24T10:45:59.241 INFO:tasks.workunit.client.0.smithi094.stdout:1/532: mknod d4/d9/de/d6f/d5f/d8d/cb8 0 2024-03-24T10:45:59.242 INFO:tasks.workunit.client.0.smithi094.stdout:1/533: mkdir d4/d9/db1/db9 0 2024-03-24T10:45:59.242 INFO:tasks.workunit.client.0.smithi094.stdout:1/534: creat d4/d9/d37/d33/fba x:0 0 0 2024-03-24T10:45:59.243 INFO:tasks.workunit.client.0.smithi094.stdout:6/660: dwrite d0/d21/d2a/d55/f93 [0,4194304] 0 2024-03-24T10:45:59.243 INFO:tasks.workunit.client.0.smithi094.stdout:6/661: creat d0/d1a/d35/d96/fd0 x:0 0 0 2024-03-24T10:45:59.244 INFO:tasks.workunit.client.0.smithi094.stdout:1/535: creat d4/d9/de/d6f/d62/fbb x:0 0 0 2024-03-24T10:45:59.247 INFO:tasks.workunit.client.0.smithi094.stdout:1/536: creat d4/d9/db/d5d/d63/fbc x:0 0 0 2024-03-24T10:45:59.248 INFO:tasks.workunit.client.0.smithi094.stdout:1/537: creat d4/d9/de/fbd x:0 0 0 2024-03-24T10:45:59.248 INFO:tasks.workunit.client.0.smithi094.stdout:1/538: fsync d4/d9/de/f2b 0 2024-03-24T10:45:59.249 INFO:tasks.workunit.client.0.smithi094.stdout:3/609: dread d0/db/dc/d28/d3c/d55/d7f/d20/d23/fc2 [0,4194304] 0 2024-03-24T10:45:59.249 INFO:tasks.workunit.client.0.smithi094.stdout:1/539: getdents d4/d9/db/d5d 0 2024-03-24T10:45:59.249 INFO:tasks.workunit.client.0.smithi094.stdout:1/540: dread - d4/d36/f44 zero size 2024-03-24T10:45:59.250 INFO:tasks.workunit.client.0.smithi094.stdout:3/610: getdents d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a 0 2024-03-24T10:45:59.251 INFO:tasks.workunit.client.0.smithi094.stdout:1/541: mkdir d4/d9/de/d3d/d66/dbe 0 2024-03-24T10:45:59.252 INFO:tasks.workunit.client.0.smithi094.stdout:3/611: truncate d0/db/dc/d28/d3c/d7c/f9d 2668475 0 2024-03-24T10:45:59.252 INFO:tasks.workunit.client.0.smithi094.stdout:1/542: mknod d4/d9/db/d5d/cbf 0 2024-03-24T10:45:59.253 INFO:tasks.workunit.client.0.smithi094.stdout:2/618: dwrite d18/d1e/d65/f67 [0,4194304] 0 2024-03-24T10:45:59.254 INFO:tasks.workunit.client.0.smithi094.stdout:2/619: read d18/d66/d90/d9a/dbc/f51 [1200734,65183] 0 2024-03-24T10:45:59.254 INFO:tasks.workunit.client.0.smithi094.stdout:5/676: dwrite d7/d23/d5f/d9/d48/d51/fc7 [0,4194304] 0 2024-03-24T10:45:59.254 INFO:tasks.workunit.client.0.smithi094.stdout:4/707: dread d0/d22/de/d28/d2b/f69 [4194304,4194304] 0 2024-03-24T10:45:59.256 INFO:tasks.workunit.client.0.smithi094.stdout:0/631: dwrite d14/d18/f21 [0,4194304] 0 2024-03-24T10:45:59.256 INFO:tasks.workunit.client.0.smithi094.stdout:8/718: dwrite df/d25/d28/d31/d47/d66/da3/fa4 [0,4194304] 0 2024-03-24T10:45:59.263 INFO:tasks.workunit.client.0.smithi094.stdout:4/708: write d0/d22/de/d28/d3d/f5c [3342030,39725] 0 2024-03-24T10:45:59.264 INFO:tasks.workunit.client.0.smithi094.stdout:5/677: dread d7/dc/f19 [0,4194304] 0 2024-03-24T10:45:59.265 INFO:tasks.workunit.client.0.smithi094.stdout:5/678: creat d7/fdd x:0 0 0 2024-03-24T10:45:59.267 INFO:tasks.workunit.client.0.smithi094.stdout:0/632: rmdir d14/d22/d28/da2/dc6 0 2024-03-24T10:45:59.268 INFO:tasks.workunit.client.0.smithi094.stdout:4/709: dread d0/d34/f48 [4194304,4194304] 0 2024-03-24T10:45:59.268 INFO:tasks.workunit.client.0.smithi094.stdout:4/710: write d0/d5/d1b/d4f/d70/f7c [932251,128551] 0 2024-03-24T10:45:59.269 INFO:tasks.workunit.client.0.smithi094.stdout:8/719: rmdir df/d25/d9d 0 2024-03-24T10:45:59.271 INFO:tasks.workunit.client.0.smithi094.stdout:5/679: mkdir d7/d23/d5f/d9/dde 0 2024-03-24T10:45:59.273 INFO:tasks.workunit.client.0.smithi094.stdout:0/633: mkdir d14/d22/d28/d48/da5/de0 0 2024-03-24T10:45:59.273 INFO:tasks.workunit.client.0.smithi094.stdout:8/720: truncate df/d25/d36/d3f/f4f 5007636 0 2024-03-24T10:45:59.275 INFO:tasks.workunit.client.0.smithi094.stdout:0/634: creat d14/d22/d28/fe1 x:0 0 0 2024-03-24T10:45:59.276 INFO:tasks.workunit.client.0.smithi094.stdout:8/721: symlink df/d8c/lf7 0 2024-03-24T10:45:59.277 INFO:tasks.workunit.client.0.smithi094.stdout:0/635: creat d14/d22/d32/d70/fe2 x:0 0 0 2024-03-24T10:45:59.279 INFO:tasks.workunit.client.0.smithi094.stdout:0/636: creat d14/d22/d28/d2b/d41/fe3 x:0 0 0 2024-03-24T10:45:59.279 INFO:tasks.workunit.client.0.smithi094.stdout:0/637: read - d14/d18/fde zero size 2024-03-24T10:45:59.281 INFO:tasks.workunit.client.0.smithi094.stdout:2/620: dwrite d18/d66/d90/d9a/dbc/d4a/f55 [4194304,4194304] 0 2024-03-24T10:45:59.281 INFO:tasks.workunit.client.0.smithi094.stdout:0/638: symlink d14/d22/d28/dac/ddf/le4 0 2024-03-24T10:45:59.281 INFO:tasks.workunit.client.0.smithi094.stdout:0/639: stat d14/d18/lb4 0 2024-03-24T10:45:59.284 INFO:tasks.workunit.client.0.smithi094.stdout:0/640: unlink d14/d18/c66 0 2024-03-24T10:45:59.284 INFO:tasks.workunit.client.0.smithi094.stdout:0/641: mkdir d14/d22/dcb/de5 0 2024-03-24T10:45:59.289 INFO:tasks.workunit.client.0.smithi094.stdout:4/711: dwrite d0/d5/d1b/d4f/f85 [0,4194304] 0 2024-03-24T10:45:59.291 INFO:tasks.workunit.client.0.smithi094.stdout:4/712: mkdir d0/d5/d1b/d4f/dd6/dd7 0 2024-03-24T10:45:59.291 INFO:tasks.workunit.client.0.smithi094.stdout:4/713: creat d0/d5/d1b/d4f/fd8 x:0 0 0 2024-03-24T10:45:59.293 INFO:tasks.workunit.client.0.smithi094.stdout:4/714: rmdir d0/d5/d1b/d4f/dd6/dd7 0 2024-03-24T10:45:59.295 INFO:tasks.workunit.client.0.smithi094.stdout:4/715: creat d0/d22/de/fd9 x:0 0 0 2024-03-24T10:45:59.295 INFO:tasks.workunit.client.0.smithi094.stdout:4/716: fdatasync d0/d5/d7d/d97/d4a/f61 0 2024-03-24T10:45:59.296 INFO:tasks.workunit.client.0.smithi094.stdout:4/717: readlink d0/d22/de/d28/d2b/d33/l68 0 2024-03-24T10:45:59.307 INFO:tasks.workunit.client.0.smithi094.stdout:0/642: dwrite d14/d22/d28/d2b/d41/d53/f5a [4194304,4194304] 0 2024-03-24T10:45:59.307 INFO:tasks.workunit.client.0.smithi094.stdout:0/643: chown d14/d22/d7d 0 1 2024-03-24T10:45:59.307 INFO:tasks.workunit.client.0.smithi094.stdout:0/644: write d14/f49 [1109776,96290] 0 2024-03-24T10:45:59.307 INFO:tasks.workunit.client.0.smithi094.stdout:0/645: write d14/d22/d28/d2b/f3b [5151812,479] 0 2024-03-24T10:45:59.310 INFO:tasks.workunit.client.0.smithi094.stdout:0/646: truncate d14/d18/f1d 4542570 0 2024-03-24T10:45:59.312 INFO:tasks.workunit.client.0.smithi094.stdout:0/647: creat d14/d22/fe6 x:0 0 0 2024-03-24T10:45:59.312 INFO:tasks.workunit.client.0.smithi094.stdout:0/648: chown d14/d22/d7d/f9c 9370129 1 2024-03-24T10:45:59.314 INFO:tasks.workunit.client.0.smithi094.stdout:0/649: dread d14/f1e [0,4194304] 0 2024-03-24T10:45:59.314 INFO:tasks.workunit.client.0.smithi094.stdout:0/650: truncate d14/d22/d28/d2b/d68/fc5 623365 0 2024-03-24T10:45:59.320 INFO:tasks.workunit.client.0.smithi094.stdout:0/651: dread d14/d18/f1f [0,4194304] 0 2024-03-24T10:45:59.321 INFO:tasks.workunit.client.0.smithi094.stdout:0/652: fsync d14/d22/d7d/fbc 0 2024-03-24T10:45:59.321 INFO:tasks.workunit.client.0.smithi094.stdout:0/653: dread - d14/d22/d28/fe1 zero size 2024-03-24T10:45:59.321 INFO:tasks.workunit.client.0.smithi094.stdout:0/654: mkdir d14/d22/d28/d2b/dc0/de7 0 2024-03-24T10:45:59.334 INFO:tasks.workunit.client.0.smithi094.stdout:3/612: truncate d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d34/fa0 236879 0 2024-03-24T10:45:59.334 INFO:tasks.workunit.client.0.smithi094.stdout:3/613: write d0/db/dc/d28/d3c/f83 [991477,19732] 0 2024-03-24T10:45:59.345 INFO:tasks.workunit.client.0.smithi094.stdout:1/543: dwrite d4/d36/f44 [0,4194304] 0 2024-03-24T10:45:59.348 INFO:tasks.workunit.client.0.smithi094.stdout:1/544: mknod d4/d9/d37/d33/d58/cc0 0 2024-03-24T10:45:59.350 INFO:tasks.workunit.client.0.smithi094.stdout:3/614: dread d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/ff [0,4194304] 0 2024-03-24T10:45:59.352 INFO:tasks.workunit.client.0.smithi094.stdout:3/615: rename d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/f5b to d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d4f/d74/fd4 0 2024-03-24T10:45:59.352 INFO:tasks.workunit.client.0.smithi094.stdout:5/680: dwrite d7/d23/dd5/da9/fab [0,4194304] 0 2024-03-24T10:45:59.353 INFO:tasks.workunit.client.0.smithi094.stdout:4/718: dwrite d0/d22/de/d28/d2b/d7a/f84 [0,4194304] 0 2024-03-24T10:45:59.353 INFO:tasks.workunit.client.0.smithi094.stdout:0/655: dwrite d14/d18/f4f [0,4194304] 0 2024-03-24T10:45:59.353 INFO:tasks.workunit.client.0.smithi094.stdout:3/616: symlink d0/db/dc/d28/d50/d5e/ld5 0 2024-03-24T10:45:59.353 INFO:tasks.workunit.client.0.smithi094.stdout:3/617: rename d0/db/dc/d28/d38/d6a to d0/db/dc/d28/d38/d6a/dd6 22 2024-03-24T10:45:59.354 INFO:tasks.workunit.client.0.smithi094.stdout:0/656: write d14/d22/d32/d65/fcc [335464,98862] 0 2024-03-24T10:45:59.365 INFO:tasks.workunit.client.0.smithi094.stdout:3/618: mknod d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/cd7 0 2024-03-24T10:45:59.365 INFO:tasks.workunit.client.0.smithi094.stdout:5/681: write d7/d23/dd5/da9/fab [3065385,12185] 0 2024-03-24T10:45:59.365 INFO:tasks.workunit.client.0.smithi094.stdout:5/682: write d7/d23/d5f/f58 [122802,15005] 0 2024-03-24T10:45:59.365 INFO:tasks.workunit.client.0.smithi094.stdout:0/657: creat d14/d22/d5b/d86/fe8 x:0 0 0 2024-03-24T10:45:59.366 INFO:tasks.workunit.client.0.smithi094.stdout:5/683: mknod d7/d23/d5f/cdf 0 2024-03-24T10:45:59.368 INFO:tasks.workunit.client.0.smithi094.stdout:1/545: dwrite d4/d9/db/d31/d3c/f50 [4194304,4194304] 0 2024-03-24T10:45:59.368 INFO:tasks.workunit.client.0.smithi094.stdout:0/658: link d14/d22/d28/d2b/d41/f7e d14/d22/d32/d65/fe9 0 2024-03-24T10:45:59.370 INFO:tasks.workunit.client.0.smithi094.stdout:9/724: sync 2024-03-24T10:45:59.373 INFO:tasks.workunit.client.0.smithi094.stdout:9/725: dread df/f22 [0,4194304] 0 2024-03-24T10:45:59.373 INFO:tasks.workunit.client.0.smithi094.stdout:1/546: link d4/d9/de/d6f/d62/d80/f8f d4/d9/de/d6f/d62/d84/fc1 0 2024-03-24T10:45:59.373 INFO:tasks.workunit.client.0.smithi094.stdout:1/547: dread - d4/d9/d7d/fad zero size 2024-03-24T10:45:59.376 INFO:tasks.workunit.client.0.smithi094.stdout:1/548: rename d4/d9/de/d3d/f90 to d4/d9/de/d6f/d5f/d8d/fc2 0 2024-03-24T10:45:59.378 INFO:tasks.workunit.client.0.smithi094.stdout:1/549: dread d4/d9/d37/d33/f5b [0,4194304] 0 2024-03-24T10:45:59.378 INFO:tasks.workunit.client.0.smithi094.stdout:1/550: chown d4/d9/d37/d33/d76 162 1 2024-03-24T10:45:59.379 INFO:tasks.workunit.client.0.smithi094.stdout:4/719: dread d0/d34/db2/faa [0,4194304] 0 2024-03-24T10:45:59.379 INFO:tasks.workunit.client.0.smithi094.stdout:4/720: write d0/d5/d1b/d4f/d70/f7c [382948,50850] 0 2024-03-24T10:45:59.381 INFO:tasks.workunit.client.0.smithi094.stdout:9/726: read df/d2d/d1f/d11/d53/d88/f89 [321644,37016] 0 2024-03-24T10:45:59.381 INFO:tasks.workunit.client.0.smithi094.stdout:9/727: dread - df/d2d/d1f/d58/d70/dab/d80/fce zero size 2024-03-24T10:45:59.381 INFO:tasks.workunit.client.0.smithi094.stdout:1/551: rename d4/d9/db/d5d/d63/fbc to d4/d9/db/d5d/fc3 0 2024-03-24T10:45:59.383 INFO:tasks.workunit.client.0.smithi094.stdout:3/619: dread d0/f3 [0,4194304] 0 2024-03-24T10:45:59.383 INFO:tasks.workunit.client.0.smithi094.stdout:3/620: fdatasync d0/db/dc/d28/d38/d6a/d9b/fa1 0 2024-03-24T10:45:59.383 INFO:tasks.workunit.client.0.smithi094.stdout:3/621: fsync d0/f14 0 2024-03-24T10:45:59.383 INFO:tasks.workunit.client.0.smithi094.stdout:4/721: unlink d0/d22/de/c21 0 2024-03-24T10:45:59.384 INFO:tasks.workunit.client.0.smithi094.stdout:9/728: creat df/d2d/d34/db8/fe7 x:0 0 0 2024-03-24T10:45:59.385 INFO:tasks.workunit.client.0.smithi094.stdout:1/552: mknod d4/d9/d37/d33/cc4 0 2024-03-24T10:45:59.386 INFO:tasks.workunit.client.0.smithi094.stdout:3/622: symlink d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/ld8 0 2024-03-24T10:45:59.386 INFO:tasks.workunit.client.0.smithi094.stdout:7/678: sync 2024-03-24T10:45:59.387 INFO:tasks.workunit.client.0.smithi094.stdout:7/679: fdatasync d1/d9/d25/d3d/fda 0 2024-03-24T10:45:59.387 INFO:tasks.workunit.client.0.smithi094.stdout:7/680: stat d1/d9/d25/d3d/faa 0 2024-03-24T10:45:59.387 INFO:tasks.workunit.client.0.smithi094.stdout:8/722: sync 2024-03-24T10:45:59.387 INFO:tasks.workunit.client.0.smithi094.stdout:6/662: sync 2024-03-24T10:45:59.389 INFO:tasks.workunit.client.0.smithi094.stdout:2/621: sync 2024-03-24T10:45:59.389 INFO:tasks.workunit.client.0.smithi094.stdout:2/622: read - d18/d1e/d65/f9c zero size 2024-03-24T10:45:59.391 INFO:tasks.workunit.client.0.smithi094.stdout:8/723: creat df/d12/d5b/ff8 x:0 0 0 2024-03-24T10:45:59.391 INFO:tasks.workunit.client.0.smithi094.stdout:8/724: fdatasync df/d25/d28/d31/d47/d22/d55/f7a 0 2024-03-24T10:45:59.392 INFO:tasks.workunit.client.0.smithi094.stdout:0/659: write d14/f2a [2772684,10466] 0 2024-03-24T10:45:59.392 INFO:tasks.workunit.client.0.smithi094.stdout:0/660: write d14/d22/d28/d48/f81 [334802,42452] 0 2024-03-24T10:45:59.393 INFO:tasks.workunit.client.0.smithi094.stdout:6/663: truncate d0/f8 886466 0 2024-03-24T10:45:59.393 INFO:tasks.workunit.client.0.smithi094.stdout:5/684: dwrite d7/d23/dd5/d64/fbf [0,4194304] 0 2024-03-24T10:45:59.393 INFO:tasks.workunit.client.0.smithi094.stdout:5/685: dread - d7/d23/dd5/da9/fc8 zero size 2024-03-24T10:45:59.394 INFO:tasks.workunit.client.0.smithi094.stdout:5/686: creat d7/d23/d5f/d14/d36/d3a/fe0 x:0 0 0 2024-03-24T10:45:59.394 INFO:tasks.workunit.client.0.smithi094.stdout:5/687: write d7/d23/d5f/d14/d6f/dbe/fd9 [587542,61017] 0 2024-03-24T10:45:59.394 INFO:tasks.workunit.client.0.smithi094.stdout:2/623: rmdir d18/d7f/d88/da0 0 2024-03-24T10:45:59.397 INFO:tasks.workunit.client.0.smithi094.stdout:6/664: rename d0/d78/c64 to d0/d30/d50/d4d/d6b/cd1 0 2024-03-24T10:45:59.400 INFO:tasks.workunit.client.0.smithi094.stdout:4/722: dread d0/d5/d7d/f99 [0,4194304] 0 2024-03-24T10:45:59.400 INFO:tasks.workunit.client.0.smithi094.stdout:4/723: dread - d0/d5/d1b/fc9 zero size 2024-03-24T10:45:59.400 INFO:tasks.workunit.client.0.smithi094.stdout:7/681: dread d1/d6c/f88 [0,4194304] 0 2024-03-24T10:45:59.401 INFO:tasks.workunit.client.0.smithi094.stdout:7/682: dread d1/d9/f13 [0,4194304] 0 2024-03-24T10:45:59.401 INFO:tasks.workunit.client.0.smithi094.stdout:7/683: write d1/d9/d16/fbe [883032,46769] 0 2024-03-24T10:45:59.403 INFO:tasks.workunit.client.0.smithi094.stdout:3/623: dread d0/db/dc/f2d [0,4194304] 0 2024-03-24T10:45:59.405 INFO:tasks.workunit.client.0.smithi094.stdout:5/688: link d7/d23/d5f/d14/d9f/ca3 d7/d23/d2b/d68/ce1 0 2024-03-24T10:45:59.407 INFO:tasks.workunit.client.0.smithi094.stdout:6/665: mkdir d0/d30/d50/d4d/dd2 0 2024-03-24T10:45:59.407 INFO:tasks.workunit.client.0.smithi094.stdout:6/666: fsync d0/d78/d66/d7a/f8b 0 2024-03-24T10:45:59.410 INFO:tasks.workunit.client.0.smithi094.stdout:6/667: symlink d0/d21/ld3 0 2024-03-24T10:45:59.412 INFO:tasks.workunit.client.0.smithi094.stdout:9/729: dwrite df/d2d/fae [0,4194304] 0 2024-03-24T10:45:59.414 INFO:tasks.workunit.client.0.smithi094.stdout:8/725: write df/d25/d28/d31/d47/d22/fc5 [837984,91521] 0 2024-03-24T10:45:59.414 INFO:tasks.workunit.client.0.smithi094.stdout:2/624: dread d18/f45 [0,4194304] 0 2024-03-24T10:45:59.415 INFO:tasks.workunit.client.0.smithi094.stdout:2/625: dread - d18/d54/db9/fba zero size 2024-03-24T10:45:59.415 INFO:tasks.workunit.client.0.smithi094.stdout:2/626: fsync d18/f4e 0 2024-03-24T10:45:59.417 INFO:tasks.workunit.client.0.smithi094.stdout:4/724: rmdir d0/d5 39 2024-03-24T10:45:59.417 INFO:tasks.workunit.client.0.smithi094.stdout:0/661: dwrite d14/d22/d5b/fb7 [0,4194304] 0 2024-03-24T10:45:59.420 INFO:tasks.workunit.client.0.smithi094.stdout:6/668: rename d0/d21/f2d to d0/d21/d2a/d55/dce/fd4 0 2024-03-24T10:45:59.420 INFO:tasks.workunit.client.0.smithi094.stdout:5/689: dread d7/d23/d5f/d9/f49 [4194304,4194304] 0 2024-03-24T10:45:59.424 INFO:tasks.workunit.client.0.smithi094.stdout:6/669: dread d0/d78/f19 [0,4194304] 0 2024-03-24T10:45:59.425 INFO:tasks.workunit.client.0.smithi094.stdout:6/670: creat d0/da9/fd5 x:0 0 0 2024-03-24T10:45:59.425 INFO:tasks.workunit.client.0.smithi094.stdout:6/671: fsync d0/d78/da/d23/dc3/f6a 0 2024-03-24T10:45:59.426 INFO:tasks.workunit.client.0.smithi094.stdout:1/553: dwrite d4/d9/fd [0,4194304] 0 2024-03-24T10:45:59.427 INFO:tasks.workunit.client.0.smithi094.stdout:8/726: creat df/d25/d28/d31/d95/de3/ff9 x:0 0 0 2024-03-24T10:45:59.431 INFO:tasks.workunit.client.0.smithi094.stdout:3/624: dwrite d0/db/dc/d28/d38/d53/f89 [0,4194304] 0 2024-03-24T10:45:59.431 INFO:tasks.workunit.client.0.smithi094.stdout:3/625: readlink d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/d73/l76 0 2024-03-24T10:45:59.432 INFO:tasks.workunit.client.0.smithi094.stdout:4/725: creat d0/d22/de/d28/d2b/d33/da1/fda x:0 0 0 2024-03-24T10:45:59.440 INFO:tasks.workunit.client.0.smithi094.stdout:4/726: dread d0/d22/de/d28/d3d/d8c/fc7 [0,4194304] 0 2024-03-24T10:45:59.442 INFO:tasks.workunit.client.0.smithi094.stdout:5/690: mknod d7/d23/d5f/d14/d9f/ce2 0 2024-03-24T10:45:59.442 INFO:tasks.workunit.client.0.smithi094.stdout:5/691: readlink d7/dc/d56/ldc 0 2024-03-24T10:45:59.444 INFO:tasks.workunit.client.0.smithi094.stdout:1/554: truncate d4/d9/db/f1f 3211067 0 2024-03-24T10:45:59.446 INFO:tasks.workunit.client.0.smithi094.stdout:8/727: write df/d25/d28/d31/d47/f77 [1112313,32681] 0 2024-03-24T10:45:59.446 INFO:tasks.workunit.client.0.smithi094.stdout:8/728: truncate df/d25/daf/fc9 787090 0 2024-03-24T10:45:59.447 INFO:tasks.workunit.client.0.smithi094.stdout:3/626: truncate d0/db/dc/d28/d3c/f83 778536 0 2024-03-24T10:45:59.447 INFO:tasks.workunit.client.0.smithi094.stdout:8/729: dread df/d25/d28/d31/d47/d22/d55/f6d [0,4194304] 0 2024-03-24T10:45:59.449 INFO:tasks.workunit.client.0.smithi094.stdout:4/727: unlink d0/d22/de/fd9 0 2024-03-24T10:45:59.449 INFO:tasks.workunit.client.0.smithi094.stdout:5/692: mknod d7/d23/d5f/d9/ce3 0 2024-03-24T10:45:59.449 INFO:tasks.workunit.client.0.smithi094.stdout:5/693: dread - d7/d23/dd5/da9/fc8 zero size 2024-03-24T10:45:59.450 INFO:tasks.workunit.client.0.smithi094.stdout:1/555: symlink d4/d9/db/lc5 0 2024-03-24T10:45:59.452 INFO:tasks.workunit.client.0.smithi094.stdout:8/730: symlink df/d12/lfa 0 2024-03-24T10:45:59.452 INFO:tasks.workunit.client.0.smithi094.stdout:6/672: dwrite d0/d30/d50/fb4 [0,4194304] 0 2024-03-24T10:45:59.452 INFO:tasks.workunit.client.0.smithi094.stdout:0/662: dwrite d14/d22/d28/d48/d7b/f8e [0,4194304] 0 2024-03-24T10:45:59.452 INFO:tasks.workunit.client.0.smithi094.stdout:0/663: fsync d14/d18/f21 0 2024-03-24T10:45:59.453 INFO:tasks.workunit.client.0.smithi094.stdout:9/730: rename df/d2d/d1f/d11/d43/db1 to df/d2d/d1f/d58/d5a/d61/d87/de8 0 2024-03-24T10:45:59.454 INFO:tasks.workunit.client.0.smithi094.stdout:9/731: fdatasync df/d2d/d1f/f49 0 2024-03-24T10:45:59.454 INFO:tasks.workunit.client.0.smithi094.stdout:5/694: rmdir d7/d23/d2b/d68/d63 39 2024-03-24T10:45:59.455 INFO:tasks.workunit.client.0.smithi094.stdout:1/556: symlink d4/lc6 0 2024-03-24T10:45:59.459 INFO:tasks.workunit.client.0.smithi094.stdout:6/673: creat d0/d21/d2a/fd6 x:0 0 0 2024-03-24T10:45:59.459 INFO:tasks.workunit.client.0.smithi094.stdout:6/674: stat d0/d30 0 2024-03-24T10:45:59.459 INFO:tasks.workunit.client.0.smithi094.stdout:6/675: creat d0/d1a/fd7 x:0 0 0 2024-03-24T10:45:59.464 INFO:tasks.workunit.client.0.smithi094.stdout:2/627: rename d18/d66/d90/d9a/dbc/d4a/c4f to d18/d7f/d88/da2/da4/cbd 0 2024-03-24T10:45:59.464 INFO:tasks.workunit.client.0.smithi094.stdout:1/557: dread d4/d9/d37/d33/d58/f88 [0,4194304] 0 2024-03-24T10:45:59.464 INFO:tasks.workunit.client.0.smithi094.stdout:1/558: chown d4/d36/f73 3398 1 2024-03-24T10:45:59.466 INFO:tasks.workunit.client.0.smithi094.stdout:5/695: link d7/d23/f44 d7/d23/d5f/d1b/fe4 0 2024-03-24T10:45:59.466 INFO:tasks.workunit.client.0.smithi094.stdout:5/696: read - d7/d23/dd5/fcd zero size 2024-03-24T10:45:59.469 INFO:tasks.workunit.client.0.smithi094.stdout:6/676: symlink d0/d78/d66/ld8 0 2024-03-24T10:45:59.469 INFO:tasks.workunit.client.0.smithi094.stdout:9/732: dread df/d2d/d1f/d58/d82/fa0 [0,4194304] 0 2024-03-24T10:45:59.471 INFO:tasks.workunit.client.0.smithi094.stdout:0/664: mknod d14/d22/d28/cea 0 2024-03-24T10:45:59.472 INFO:tasks.workunit.client.0.smithi094.stdout:1/559: symlink d4/d9/de/d6f/d5f/lc7 0 2024-03-24T10:45:59.472 INFO:tasks.workunit.client.0.smithi094.stdout:0/665: dread d14/d22/d28/d2b/d68/fc5 [0,4194304] 0 2024-03-24T10:45:59.472 INFO:tasks.workunit.client.0.smithi094.stdout:0/666: stat d14/d22/d7d/fbc 0 2024-03-24T10:45:59.475 INFO:tasks.workunit.client.0.smithi094.stdout:6/677: unlink d0/d1a/d35/l79 0 2024-03-24T10:45:59.477 INFO:tasks.workunit.client.0.smithi094.stdout:0/667: link d14/d22/d28/f6c d14/d22/d28/d48/feb 0 2024-03-24T10:45:59.477 INFO:tasks.workunit.client.0.smithi094.stdout:0/668: stat d14/d22/d32/d70/dd0/l89 0 2024-03-24T10:45:59.479 INFO:tasks.workunit.client.0.smithi094.stdout:0/669: link d14/d22/d28/d2b/cb5 d14/d22/d28/d2b/dc0/de7/cec 0 2024-03-24T10:45:59.482 INFO:tasks.workunit.client.0.smithi094.stdout:0/670: creat d14/d22/d28/d2b/d41/d53/fed x:0 0 0 2024-03-24T10:45:59.483 INFO:tasks.workunit.client.0.smithi094.stdout:3/627: dwrite d0/db/dc/d28/d38/fa6 [0,4194304] 0 2024-03-24T10:45:59.483 INFO:tasks.workunit.client.0.smithi094.stdout:3/628: readlink d0/db/dc/d28/d3c/d55/d7f/d20/d23/d8b/la8 0 2024-03-24T10:45:59.484 INFO:tasks.workunit.client.0.smithi094.stdout:0/671: mknod d14/d22/d28/dac/ddf/cee 0 2024-03-24T10:45:59.484 INFO:tasks.workunit.client.0.smithi094.stdout:0/672: chown d14/d22/d28/d48/da5/fa7 58 1 2024-03-24T10:45:59.485 INFO:tasks.workunit.client.0.smithi094.stdout:4/728: dwrite d0/d22/de/d28/fbb [0,4194304] 0 2024-03-24T10:45:59.485 INFO:tasks.workunit.client.0.smithi094.stdout:4/729: chown d0/d5/d1b/d4f 469417431 1 2024-03-24T10:45:59.489 INFO:tasks.workunit.client.0.smithi094.stdout:0/673: dread d14/d22/d28/d2b/d68/f72 [0,4194304] 0 2024-03-24T10:45:59.490 INFO:tasks.workunit.client.0.smithi094.stdout:8/731: dwrite df/d12/d5b/d63/d75/dd9/fe8 [0,4194304] 0 2024-03-24T10:45:59.492 INFO:tasks.workunit.client.0.smithi094.stdout:3/629: mkdir d0/db/dc/dd9 0 2024-03-24T10:45:59.494 INFO:tasks.workunit.client.0.smithi094.stdout:5/697: dwrite d7/d26/f5c [0,4194304] 0 2024-03-24T10:45:59.495 INFO:tasks.workunit.client.0.smithi094.stdout:8/732: mkdir df/d12/d5b/d63/d75/dd9/dfb 0 2024-03-24T10:45:59.495 INFO:tasks.workunit.client.0.smithi094.stdout:8/733: truncate df/d25/d28/d31/fd6 962342 0 2024-03-24T10:45:59.495 INFO:tasks.workunit.client.0.smithi094.stdout:4/730: dread d0/d5/d7d/f99 [0,4194304] 0 2024-03-24T10:45:59.496 INFO:tasks.workunit.client.0.smithi094.stdout:3/630: chown d0/db/dc/d28/d3c/d55/d7f/d20/d23/l68 35925 1 2024-03-24T10:45:59.496 INFO:tasks.workunit.client.0.smithi094.stdout:3/631: stat d0/db/dc/d28/d3c/d55/d7f/d20 0 2024-03-24T10:45:59.496 INFO:tasks.workunit.client.0.smithi094.stdout:3/632: stat d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d34/ccb 0 2024-03-24T10:45:59.496 INFO:tasks.workunit.client.0.smithi094.stdout:3/633: chown d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/l5d 15857 1 2024-03-24T10:45:59.498 INFO:tasks.workunit.client.0.smithi094.stdout:0/674: symlink d14/d22/d28/lef 0 2024-03-24T10:45:59.501 INFO:tasks.workunit.client.0.smithi094.stdout:5/698: mknod d7/d23/d5f/d14/d36/d3a/d57/ce5 0 2024-03-24T10:45:59.502 INFO:tasks.workunit.client.0.smithi094.stdout:3/634: read d0/db/dc/f2d [1297850,114363] 0 2024-03-24T10:45:59.502 INFO:tasks.workunit.client.0.smithi094.stdout:3/635: chown d0/db/dc/d28/d50/l6e 965 1 2024-03-24T10:45:59.502 INFO:tasks.workunit.client.0.smithi094.stdout:4/731: creat d0/d22/de/d28/d3d/d8c/fdb x:0 0 0 2024-03-24T10:45:59.502 INFO:tasks.workunit.client.0.smithi094.stdout:4/732: truncate d0/d22/de/d28/fbb 4748063 0 2024-03-24T10:45:59.503 INFO:tasks.workunit.client.0.smithi094.stdout:0/675: rename d14/d22/d28/d48/d7b/cc7 to d14/d22/d28/d2b/dc0/cf0 0 2024-03-24T10:45:59.504 INFO:tasks.workunit.client.0.smithi094.stdout:5/699: creat d7/d23/d5f/d1b/fe6 x:0 0 0 2024-03-24T10:45:59.505 INFO:tasks.workunit.client.0.smithi094.stdout:3/636: symlink d0/lda 0 2024-03-24T10:45:59.506 INFO:tasks.workunit.client.0.smithi094.stdout:5/700: dread d7/d23/d5f/d9/d48/fa7 [0,4194304] 0 2024-03-24T10:45:59.507 INFO:tasks.workunit.client.0.smithi094.stdout:0/676: mkdir d14/d22/d28/d2b/d41/df1 0 2024-03-24T10:45:59.507 INFO:tasks.workunit.client.0.smithi094.stdout:0/677: fsync d14/d18/f25 0 2024-03-24T10:45:59.507 INFO:tasks.workunit.client.0.smithi094.stdout:0/678: stat d14/d22/d28/d2b/l9f 0 2024-03-24T10:45:59.507 INFO:tasks.workunit.client.0.smithi094.stdout:0/679: chown d14/d22/d32/d65/fda 1457 1 2024-03-24T10:45:59.507 INFO:tasks.workunit.client.0.smithi094.stdout:0/680: fdatasync d14/f2a 0 2024-03-24T10:45:59.509 INFO:tasks.workunit.client.0.smithi094.stdout:3/637: rename d0/db/dc/d28/d38/d6a/l8f to d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d34/ldb 0 2024-03-24T10:45:59.511 INFO:tasks.workunit.client.0.smithi094.stdout:5/701: symlink d7/d23/d5f/d14/le7 0 2024-03-24T10:45:59.511 INFO:tasks.workunit.client.0.smithi094.stdout:4/733: write d0/d22/f76 [2555265,36000] 0 2024-03-24T10:45:59.514 INFO:tasks.workunit.client.0.smithi094.stdout:5/702: creat d7/d23/d2b/d68/fe8 x:0 0 0 2024-03-24T10:45:59.516 INFO:tasks.workunit.client.0.smithi094.stdout:4/734: truncate d0/d22/de/d28/d3d/d8c/fb6 2405166 0 2024-03-24T10:45:59.517 INFO:tasks.workunit.client.0.smithi094.stdout:7/684: sync 2024-03-24T10:45:59.518 INFO:tasks.workunit.client.0.smithi094.stdout:1/560: dwrite d4/d9/d37/d33/f7e [0,4194304] 0 2024-03-24T10:45:59.518 INFO:tasks.workunit.client.0.smithi094.stdout:8/734: write df/d25/d28/d31/d3c/f3e [4258059,17683] 0 2024-03-24T10:45:59.518 INFO:tasks.workunit.client.0.smithi094.stdout:8/735: readlink df/d25/l74 0 2024-03-24T10:45:59.518 INFO:tasks.workunit.client.0.smithi094.stdout:8/736: write df/d25/d28/d31/d3c/f5f [3190478,122966] 0 2024-03-24T10:45:59.519 INFO:tasks.workunit.client.0.smithi094.stdout:4/735: dread d0/d22/de/d28/f6b [0,4194304] 0 2024-03-24T10:45:59.519 INFO:tasks.workunit.client.0.smithi094.stdout:5/703: write d7/dc/f19 [2419105,35056] 0 2024-03-24T10:45:59.522 INFO:tasks.workunit.client.0.smithi094.stdout:9/733: dwrite df/d7b/fbb [0,4194304] 0 2024-03-24T10:45:59.522 INFO:tasks.workunit.client.0.smithi094.stdout:6/678: dwrite d0/d1a/d35/d96/fbf [0,4194304] 0 2024-03-24T10:45:59.524 INFO:tasks.workunit.client.0.smithi094.stdout:1/561: mkdir d4/d9/de/d6f/d62/dc8 0 2024-03-24T10:45:59.525 INFO:tasks.workunit.client.0.smithi094.stdout:5/704: dread d7/d23/d5f/d9/d48/d51/f6a [0,4194304] 0 2024-03-24T10:45:59.525 INFO:tasks.workunit.client.0.smithi094.stdout:5/705: fdatasync d7/d23/dd5/d64/f7d 0 2024-03-24T10:45:59.526 INFO:tasks.workunit.client.0.smithi094.stdout:7/685: write d1/d9/d16/d21/d49/f84 [1297777,32816] 0 2024-03-24T10:45:59.526 INFO:tasks.workunit.client.0.smithi094.stdout:7/686: truncate d1/d9/fe 1334832 0 2024-03-24T10:45:59.526 INFO:tasks.workunit.client.0.smithi094.stdout:7/687: readlink d1/d9/d16/d21/d49/da9/lca 0 2024-03-24T10:45:59.531 INFO:tasks.workunit.client.0.smithi094.stdout:9/734: creat df/d2d/d1f/d58/d5a/d50/da5/fe9 x:0 0 0 2024-03-24T10:45:59.531 INFO:tasks.workunit.client.0.smithi094.stdout:9/735: read df/f52 [3135708,98248] 0 2024-03-24T10:45:59.531 INFO:tasks.workunit.client.0.smithi094.stdout:9/736: chown df/d2d/d1f/d11/d53/de3 1791054640 1 2024-03-24T10:45:59.532 INFO:tasks.workunit.client.0.smithi094.stdout:6/679: creat d0/d78/da/d23/d7c/fd9 x:0 0 0 2024-03-24T10:45:59.543 INFO:tasks.workunit.client.0.smithi094.stdout:9/737: dread df/f52 [0,4194304] 0 2024-03-24T10:45:59.543 INFO:tasks.workunit.client.0.smithi094.stdout:9/738: dread - df/d2d/d1f/d11/fcb zero size 2024-03-24T10:45:59.543 INFO:tasks.workunit.client.0.smithi094.stdout:9/739: stat df/d2d/d1f/d58/d5a/d69/d7a/f8f 0 2024-03-24T10:45:59.543 INFO:tasks.workunit.client.0.smithi094.stdout:9/740: rename df/d2d/d1f to df/d2d/d1f/d58/d5a/d61/d87/dc8/dea 22 2024-03-24T10:45:59.543 INFO:tasks.workunit.client.0.smithi094.stdout:1/562: write d4/d9/db/d31/f48 [2314087,68358] 0 2024-03-24T10:45:59.545 INFO:tasks.workunit.client.0.smithi094.stdout:9/741: link df/d2d/d1f/f49 df/d2d/d1f/d58/d5a/d69/d7a/feb 0 2024-03-24T10:45:59.546 INFO:tasks.workunit.client.0.smithi094.stdout:0/681: dwrite d14/d22/d5b/fbd [0,4194304] 0 2024-03-24T10:45:59.548 INFO:tasks.workunit.client.0.smithi094.stdout:9/742: mkdir df/d2d/d1f/d58/d70/dab/dec 0 2024-03-24T10:45:59.550 INFO:tasks.workunit.client.0.smithi094.stdout:0/682: stat d14/d22/d28/d2b/cb5 0 2024-03-24T10:45:59.550 INFO:tasks.workunit.client.0.smithi094.stdout:0/683: write d14/f49 [444549,101918] 0 2024-03-24T10:45:59.552 INFO:tasks.workunit.client.0.smithi094.stdout:9/743: creat df/d2d/d1f/fed x:0 0 0 2024-03-24T10:45:59.555 INFO:tasks.workunit.client.0.smithi094.stdout:0/684: unlink fa 0 2024-03-24T10:45:59.555 INFO:tasks.workunit.client.0.smithi094.stdout:0/685: creat d14/d22/d28/d2b/ff2 x:0 0 0 2024-03-24T10:45:59.567 INFO:tasks.workunit.client.0.smithi094.stdout:3/638: dwrite d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/fc8 [0,4194304] 0 2024-03-24T10:45:59.567 INFO:tasks.workunit.client.0.smithi094.stdout:4/736: dwrite d0/d34/d73/d93/fb4 [0,4194304] 0 2024-03-24T10:45:59.570 INFO:tasks.workunit.client.0.smithi094.stdout:9/744: write df/d2d/d1f/d11/f75 [510940,14311] 0 2024-03-24T10:45:59.575 INFO:tasks.workunit.client.0.smithi094.stdout:9/745: mknod df/d2d/d1f/d7e/cee 0 2024-03-24T10:45:59.578 INFO:tasks.workunit.client.0.smithi094.stdout:9/746: link df/f36 df/d2d/d1f/d58/d70/dab/fef 0 2024-03-24T10:45:59.580 INFO:tasks.workunit.client.0.smithi094.stdout:9/747: creat df/d2d/d1f/d58/d70/de2/ff0 x:0 0 0 2024-03-24T10:45:59.582 INFO:tasks.workunit.client.0.smithi094.stdout:9/748: mknod df/d2d/d34/d2c/d74/dc5/cf1 0 2024-03-24T10:45:59.582 INFO:tasks.workunit.client.0.smithi094.stdout:3/639: write d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d4f/d74/fd4 [3450589,11996] 0 2024-03-24T10:45:59.582 INFO:tasks.workunit.client.0.smithi094.stdout:9/749: mkdir df/d2d/df2 0 2024-03-24T10:45:59.583 INFO:tasks.workunit.client.0.smithi094.stdout:9/750: mkdir df/d2d/d1f/d11/d96/df3 0 2024-03-24T10:45:59.587 INFO:tasks.workunit.client.0.smithi094.stdout:9/751: rmdir df/d2d/d34/dba 0 2024-03-24T10:45:59.593 INFO:tasks.workunit.client.0.smithi094.stdout:7/688: dwrite d1/db4/d65/f89 [0,4194304] 0 2024-03-24T10:45:59.593 INFO:tasks.workunit.client.0.smithi094.stdout:7/689: chown d1/d9/d16/d21/d49/da9/fb8 91646662 1 2024-03-24T10:45:59.593 INFO:tasks.workunit.client.0.smithi094.stdout:6/680: dwrite d0/d1a/d35/d96/fc1 [0,4194304] 0 2024-03-24T10:45:59.593 INFO:tasks.workunit.client.0.smithi094.stdout:5/706: dwrite d7/d23/d5f/d1b/fa4 [0,4194304] 0 2024-03-24T10:45:59.593 INFO:tasks.workunit.client.0.smithi094.stdout:5/707: write d7/d23/d5f/d14/d9f/fbc [226003,98835] 0 2024-03-24T10:45:59.593 INFO:tasks.workunit.client.0.smithi094.stdout:0/686: dwrite d14/d22/d28/d2b/d41/f7e [0,4194304] 0 2024-03-24T10:45:59.593 INFO:tasks.workunit.client.0.smithi094.stdout:0/687: stat d14/d22/d5b/d86/lca 0 2024-03-24T10:45:59.600 INFO:tasks.workunit.client.0.smithi094.stdout:3/640: dwrite d0/db/dc/d28/d3c/d55/fb2 [0,4194304] 0 2024-03-24T10:45:59.600 INFO:tasks.workunit.client.0.smithi094.stdout:3/641: fdatasync d0/fb6 0 2024-03-24T10:45:59.600 INFO:tasks.workunit.client.0.smithi094.stdout:3/642: fdatasync d0/db/dc/d28/d38/d6a/d9b/fa1 0 2024-03-24T10:45:59.600 INFO:tasks.workunit.client.0.smithi094.stdout:0/688: truncate d14/d22/d28/d48/feb 211056 0 2024-03-24T10:45:59.600 INFO:tasks.workunit.client.0.smithi094.stdout:0/689: readlink d14/d22/d28/d2b/l4d 0 2024-03-24T10:45:59.600 INFO:tasks.workunit.client.0.smithi094.stdout:0/690: creat d14/d22/d32/d65/ff3 x:0 0 0 2024-03-24T10:45:59.600 INFO:tasks.workunit.client.0.smithi094.stdout:0/691: dread - d14/d22/d5b/d86/fe8 zero size 2024-03-24T10:45:59.600 INFO:tasks.workunit.client.0.smithi094.stdout:4/737: dwrite d0/d22/de/d28/d2b/d7a/fb5 [0,4194304] 0 2024-03-24T10:45:59.601 INFO:tasks.workunit.client.0.smithi094.stdout:1/563: dwrite d4/d9/d37/d33/d58/f88 [0,4194304] 0 2024-03-24T10:45:59.606 INFO:tasks.workunit.client.0.smithi094.stdout:7/690: getdents d1 0 2024-03-24T10:45:59.610 INFO:tasks.workunit.client.0.smithi094.stdout:3/643: symlink d0/db/dc/d28/d38/ldc 0 2024-03-24T10:45:59.610 INFO:tasks.workunit.client.0.smithi094.stdout:3/644: truncate d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d86/fa3 354357 0 2024-03-24T10:45:59.612 INFO:tasks.workunit.client.0.smithi094.stdout:0/692: symlink d14/d22/d32/d70/dd0/dce/lf4 0 2024-03-24T10:45:59.614 INFO:tasks.workunit.client.0.smithi094.stdout:1/564: symlink d4/d9/d7d/lc9 0 2024-03-24T10:45:59.617 INFO:tasks.workunit.client.0.smithi094.stdout:7/691: rename d1/d9/d16/d21/d51/d83/d39/d44/dd9/d34/cae to d1/d6c/cdc 0 2024-03-24T10:45:59.618 INFO:tasks.workunit.client.0.smithi094.stdout:3/645: truncate d0/db/dc/d28/d3c/d55/d7f/d20/d23/f3f 1692556 0 2024-03-24T10:45:59.619 INFO:tasks.workunit.client.0.smithi094.stdout:0/693: dread d14/d22/d28/d2b/f45 [0,4194304] 0 2024-03-24T10:45:59.619 INFO:tasks.workunit.client.0.smithi094.stdout:0/694: read - d14/fcd zero size 2024-03-24T10:45:59.619 INFO:tasks.workunit.client.0.smithi094.stdout:7/692: creat d1/db4/fdd x:0 0 0 2024-03-24T10:45:59.619 INFO:tasks.workunit.client.0.smithi094.stdout:7/693: chown d1/d9/d16/d21/d51/d83/d9d 61 1 2024-03-24T10:45:59.620 INFO:tasks.workunit.client.0.smithi094.stdout:3/646: rename d0/c6d to d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d86/cdd 0 2024-03-24T10:45:59.620 INFO:tasks.workunit.client.0.smithi094.stdout:3/647: stat d0/db/dc/fbf 0 2024-03-24T10:45:59.622 INFO:tasks.workunit.client.0.smithi094.stdout:7/694: truncate d1/db4/fba 6777222 0 2024-03-24T10:45:59.622 INFO:tasks.workunit.client.0.smithi094.stdout:6/681: dwrite d0/d78/da/d23/dc3/fbe [0,4194304] 0 2024-03-24T10:45:59.622 INFO:tasks.workunit.client.0.smithi094.stdout:5/708: dwrite d7/d23/d5f/d14/d6f/f94 [0,4194304] 0 2024-03-24T10:45:59.624 INFO:tasks.workunit.client.0.smithi094.stdout:0/695: rename d14/d22/d28/d2b/d68/fc5 to d14/d22/d32/d70/ff5 0 2024-03-24T10:45:59.626 INFO:tasks.workunit.client.0.smithi094.stdout:3/648: creat d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d87/dc5/fde x:0 0 0 2024-03-24T10:45:59.630 INFO:tasks.workunit.client.0.smithi094.stdout:6/682: stat d0/d1a/l31 0 2024-03-24T10:45:59.632 INFO:tasks.workunit.client.0.smithi094.stdout:7/695: write d1/d9/d96/f9a [1981107,513] 0 2024-03-24T10:45:59.638 INFO:tasks.workunit.client.0.smithi094.stdout:5/709: rename d7/d23/d5f/d9/ce3 to d7/d23/d5f/d1b/ce9 0 2024-03-24T10:45:59.638 INFO:tasks.workunit.client.0.smithi094.stdout:5/710: chown d7/d23/d5f/d14/d36/f47 1452145 1 2024-03-24T10:45:59.638 INFO:tasks.workunit.client.0.smithi094.stdout:5/711: readlink d7/d23/d2b/l45 0 2024-03-24T10:45:59.641 INFO:tasks.workunit.client.0.smithi094.stdout:7/696: unlink d1/d9/d16/d21/d51/d83/d39/d44/dd9/f6e 0 2024-03-24T10:45:59.643 INFO:tasks.workunit.client.0.smithi094.stdout:5/712: link d7/d23/d2b/d68/d81/l82 d7/d23/d5f/d14/d9f/lea 0 2024-03-24T10:45:59.644 INFO:tasks.workunit.client.0.smithi094.stdout:7/697: rmdir d1/d9/d16/d21/d49/da9 39 2024-03-24T10:45:59.644 INFO:tasks.workunit.client.0.smithi094.stdout:1/565: dwrite d4/f16 [0,4194304] 0 2024-03-24T10:45:59.644 INFO:tasks.workunit.client.0.smithi094.stdout:6/683: dread d0/d1a/d35/d96/fbf [0,4194304] 0 2024-03-24T10:45:59.644 INFO:tasks.workunit.client.0.smithi094.stdout:1/566: fsync d4/d36/f70 0 2024-03-24T10:45:59.644 INFO:tasks.workunit.client.0.smithi094.stdout:5/713: symlink d7/d23/d5f/d9/dde/leb 0 2024-03-24T10:45:59.645 INFO:tasks.workunit.client.0.smithi094.stdout:7/698: mknod d1/d9/d25/dd4/cde 0 2024-03-24T10:45:59.648 INFO:tasks.workunit.client.0.smithi094.stdout:6/684: symlink d0/d30/lda 0 2024-03-24T10:45:59.648 INFO:tasks.workunit.client.0.smithi094.stdout:6/685: chown d0/d30/d50/fb4 773776278 1 2024-03-24T10:45:59.649 INFO:tasks.workunit.client.0.smithi094.stdout:1/567: rename d4/d9/d37/l52 to d4/d9/d37/lca 0 2024-03-24T10:45:59.650 INFO:tasks.workunit.client.0.smithi094.stdout:7/699: truncate d1/fd2 3007405 0 2024-03-24T10:45:59.652 INFO:tasks.workunit.client.0.smithi094.stdout:6/686: getdents d0/d21/d2a 0 2024-03-24T10:45:59.652 INFO:tasks.workunit.client.0.smithi094.stdout:6/687: fsync d0/d21/d2a/d55/f93 0 2024-03-24T10:45:59.656 INFO:tasks.workunit.client.0.smithi094.stdout:5/714: write d7/d23/d5f/d9/f4f [3782752,74906] 0 2024-03-24T10:45:59.657 INFO:tasks.workunit.client.0.smithi094.stdout:5/715: write d7/d23/f44 [2879343,105252] 0 2024-03-24T10:45:59.657 INFO:tasks.workunit.client.0.smithi094.stdout:5/716: chown d7/d23/d5f/lcf 120974829 1 2024-03-24T10:45:59.657 INFO:tasks.workunit.client.0.smithi094.stdout:5/717: chown d7/d23/d5f/d14/d6f/f91 1989495 1 2024-03-24T10:45:59.662 INFO:tasks.workunit.client.0.smithi094.stdout:0/696: dwrite d14/d22/d28/da2/fd1 [0,4194304] 0 2024-03-24T10:45:59.662 INFO:tasks.workunit.client.0.smithi094.stdout:0/697: stat d14 0 2024-03-24T10:45:59.671 INFO:tasks.workunit.client.0.smithi094.stdout:3/649: dwrite d0/db/dc/d28/d3c/f6f [0,4194304] 0 2024-03-24T10:45:59.671 INFO:tasks.workunit.client.0.smithi094.stdout:1/568: dread d4/d9/de/d6f/d5f/f68 [0,4194304] 0 2024-03-24T10:45:59.674 INFO:tasks.workunit.client.0.smithi094.stdout:7/700: dwrite d1/d9/d25/d3d/f46 [0,4194304] 0 2024-03-24T10:45:59.675 INFO:tasks.workunit.client.0.smithi094.stdout:3/650: symlink d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d34/d71/ldf 0 2024-03-24T10:45:59.675 INFO:tasks.workunit.client.0.smithi094.stdout:1/569: dread d4/d9/db/f1f [0,4194304] 0 2024-03-24T10:45:59.677 INFO:tasks.workunit.client.0.smithi094.stdout:3/651: creat d0/db/dc/d28/d3c/d7c/fe0 x:0 0 0 2024-03-24T10:45:59.678 INFO:tasks.workunit.client.0.smithi094.stdout:1/570: dread d4/d9/db/f10 [0,4194304] 0 2024-03-24T10:45:59.679 INFO:tasks.workunit.client.0.smithi094.stdout:3/652: creat d0/db/dc/d28/d3c/d55/d7f/fe1 x:0 0 0 2024-03-24T10:45:59.682 INFO:tasks.workunit.client.0.smithi094.stdout:6/688: dwrite d0/d21/f56 [0,4194304] 0 2024-03-24T10:45:59.685 INFO:tasks.workunit.client.0.smithi094.stdout:6/689: symlink d0/d21/d2a/ldb 0 2024-03-24T10:45:59.685 INFO:tasks.workunit.client.0.smithi094.stdout:6/690: chown d0/d78/d66/d7a/fb8 4192508 1 2024-03-24T10:45:59.685 INFO:tasks.workunit.client.0.smithi094.stdout:6/691: fsync d0/f33 0 2024-03-24T10:45:59.685 INFO:tasks.workunit.client.0.smithi094.stdout:1/571: write d4/d36/f73 [3516519,56613] 0 2024-03-24T10:45:59.685 INFO:tasks.workunit.client.0.smithi094.stdout:2/628: sync 2024-03-24T10:45:59.688 INFO:tasks.workunit.client.0.smithi094.stdout:1/572: rename d4/l19 to d4/d9/de/d6f/d5f/lcb 0 2024-03-24T10:45:59.688 INFO:tasks.workunit.client.0.smithi094.stdout:1/573: write d4/d9/de/d3d/f92 [52437,36027] 0 2024-03-24T10:45:59.688 INFO:tasks.workunit.client.0.smithi094.stdout:1/574: creat d4/d9/de/d6f/d62/fcc x:0 0 0 2024-03-24T10:45:59.689 INFO:tasks.workunit.client.0.smithi094.stdout:2/629: unlink d18/d7f/d88/l92 0 2024-03-24T10:45:59.690 INFO:tasks.workunit.client.0.smithi094.stdout:7/701: read d1/d9/d16/d21/d51/d83/d39/f94 [216776,107620] 0 2024-03-24T10:45:59.690 INFO:tasks.workunit.client.0.smithi094.stdout:5/718: dwrite d7/d23/d5f/d14/f35 [0,4194304] 0 2024-03-24T10:45:59.690 INFO:tasks.workunit.client.0.smithi094.stdout:5/719: write d7/fb7 [965598,16182] 0 2024-03-24T10:45:59.691 INFO:tasks.workunit.client.0.smithi094.stdout:3/653: read d0/f11 [2550470,114836] 0 2024-03-24T10:45:59.691 INFO:tasks.workunit.client.0.smithi094.stdout:1/575: getdents d4/d9/db/d5d 0 2024-03-24T10:45:59.693 INFO:tasks.workunit.client.0.smithi094.stdout:2/630: mkdir d18/d66/dbe 0 2024-03-24T10:45:59.697 INFO:tasks.workunit.client.0.smithi094.stdout:5/720: write d7/d23/d5f/f83 [1853999,103151] 0 2024-03-24T10:45:59.697 INFO:tasks.workunit.client.0.smithi094.stdout:5/721: write d7/d23/d5f/d14/d6f/dbe/fdb [853193,11228] 0 2024-03-24T10:45:59.697 INFO:tasks.workunit.client.0.smithi094.stdout:5/722: chown d7/d26/dce 62 1 2024-03-24T10:45:59.704 INFO:tasks.workunit.client.0.smithi094.stdout:7/702: symlink d1/db4/d61/ldf 0 2024-03-24T10:45:59.707 INFO:tasks.workunit.client.0.smithi094.stdout:3/654: creat d0/db/dc/d28/d50/fe2 x:0 0 0 2024-03-24T10:45:59.707 INFO:tasks.workunit.client.0.smithi094.stdout:6/692: dwrite d0/da9/fbc [0,4194304] 0 2024-03-24T10:45:59.713 INFO:tasks.workunit.client.0.smithi094.stdout:2/631: rmdir d18/d66/d90/d9a/dbc 39 2024-03-24T10:45:59.713 INFO:tasks.workunit.client.0.smithi094.stdout:6/693: dread d0/da9/fbc [0,4194304] 0 2024-03-24T10:45:59.718 INFO:tasks.workunit.client.0.smithi094.stdout:5/723: mknod d7/d23/d2b/cec 0 2024-03-24T10:45:59.722 INFO:tasks.workunit.client.0.smithi094.stdout:7/703: symlink d1/d9/d16/le0 0 2024-03-24T10:45:59.723 INFO:tasks.workunit.client.0.smithi094.stdout:3/655: mkdir d0/db/dc/d28/d3c/d7c/de3 0 2024-03-24T10:45:59.724 INFO:tasks.workunit.client.0.smithi094.stdout:8/737: sync 2024-03-24T10:45:59.724 INFO:tasks.workunit.client.0.smithi094.stdout:8/738: fdatasync df/d12/d5b/d63/f9c 0 2024-03-24T10:45:59.724 INFO:tasks.workunit.client.0.smithi094.stdout:8/739: write df/d25/d28/d31/d47/d22/d55/f6d [754828,33286] 0 2024-03-24T10:45:59.731 INFO:tasks.workunit.client.0.smithi094.stdout:1/576: getdents d4/d9 0 2024-03-24T10:45:59.732 INFO:tasks.workunit.client.0.smithi094.stdout:1/577: dread - d4/d9/db/f95 zero size 2024-03-24T10:45:59.732 INFO:tasks.workunit.client.0.smithi094.stdout:6/694: rmdir d0/d78/da/d23/d34/d4e/d5d 39 2024-03-24T10:45:59.733 INFO:tasks.workunit.client.0.smithi094.stdout:2/632: dwrite d18/d66/d90/d9a/dbc/f43 [4194304,4194304] 0 2024-03-24T10:45:59.733 INFO:tasks.workunit.client.0.smithi094.stdout:2/633: dread - d18/d7f/f8d zero size 2024-03-24T10:45:59.734 INFO:tasks.workunit.client.0.smithi094.stdout:5/724: dwrite d7/dc/f19 [0,4194304] 0 2024-03-24T10:45:59.734 INFO:tasks.workunit.client.0.smithi094.stdout:7/704: symlink d1/d6c/d80/le1 0 2024-03-24T10:45:59.737 INFO:tasks.workunit.client.0.smithi094.stdout:3/656: mknod d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d87/ce4 0 2024-03-24T10:45:59.741 INFO:tasks.workunit.client.0.smithi094.stdout:8/740: link df/f90 df/d8c/d8f/dc1/ffc 0 2024-03-24T10:45:59.746 INFO:tasks.workunit.client.0.smithi094.stdout:1/578: link d4/f7 d4/fcd 0 2024-03-24T10:45:59.749 INFO:tasks.workunit.client.0.smithi094.stdout:5/725: dread d7/f34 [0,4194304] 0 2024-03-24T10:45:59.749 INFO:tasks.workunit.client.0.smithi094.stdout:5/726: creat d7/dc/fed x:0 0 0 2024-03-24T10:45:59.751 INFO:tasks.workunit.client.0.smithi094.stdout:6/695: mknod d0/d78/da/cdc 0 2024-03-24T10:45:59.751 INFO:tasks.workunit.client.0.smithi094.stdout:6/696: readlink d0/d30/d50/l70 0 2024-03-24T10:45:59.752 INFO:tasks.workunit.client.0.smithi094.stdout:6/697: dread d0/d30/d50/d4d/d6b/f90 [0,4194304] 0 2024-03-24T10:45:59.758 INFO:tasks.workunit.client.0.smithi094.stdout:3/657: dread d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d4f/d74/fd4 [0,4194304] 0 2024-03-24T10:45:59.758 INFO:tasks.workunit.client.0.smithi094.stdout:3/658: creat d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d34/fe5 x:0 0 0 2024-03-24T10:45:59.759 INFO:tasks.workunit.client.0.smithi094.stdout:3/659: creat d0/db/dc/d28/d50/fe6 x:0 0 0 2024-03-24T10:45:59.762 INFO:tasks.workunit.client.0.smithi094.stdout:6/698: link d0/d1a/l43 d0/d1a/dca/ldd 0 2024-03-24T10:45:59.780 INFO:tasks.workunit.client.0.smithi094.stdout:8/741: dwrite df/d25/d36/d3f/fdb [0,4194304] 0 2024-03-24T10:45:59.783 INFO:tasks.workunit.client.0.smithi094.stdout:7/705: dwrite d1/d9/d16/d21/d51/d83/d39/d44/dd9/d8f/fa6 [0,4194304] 0 2024-03-24T10:45:59.783 INFO:tasks.workunit.client.0.smithi094.stdout:7/706: write d1/fcb [87552,127403] 0 2024-03-24T10:45:59.783 INFO:tasks.workunit.client.0.smithi094.stdout:7/707: write d1/d9/d16/fc1 [403481,42940] 0 2024-03-24T10:45:59.783 INFO:tasks.workunit.client.0.smithi094.stdout:7/708: fsync d1/d6c/f88 0 2024-03-24T10:45:59.783 INFO:tasks.workunit.client.0.smithi094.stdout:7/709: chown d1/db4/d65/fb6 3 1 2024-03-24T10:45:59.783 INFO:tasks.workunit.client.0.smithi094.stdout:8/742: mkdir df/d25/d28/d31/d47/ddf/dfd 0 2024-03-24T10:45:59.786 INFO:tasks.workunit.client.0.smithi094.stdout:7/710: mkdir d1/d9/d25/de2 0 2024-03-24T10:45:59.787 INFO:tasks.workunit.client.0.smithi094.stdout:8/743: mkdir df/d25/d28/d31/d47/dfe 0 2024-03-24T10:45:59.789 INFO:tasks.workunit.client.0.smithi094.stdout:8/744: mkdir df/d25/d28/d31/d95/de3/dff 0 2024-03-24T10:45:59.789 INFO:tasks.workunit.client.0.smithi094.stdout:8/745: chown df/d25/d28/d31/d47/ddf/dfd 563585 1 2024-03-24T10:45:59.793 INFO:tasks.workunit.client.0.smithi094.stdout:2/634: dwrite d18/f1d [0,4194304] 0 2024-03-24T10:45:59.794 INFO:tasks.workunit.client.0.smithi094.stdout:5/727: dwrite d7/d23/dd5/d64/dd6/f39 [0,4194304] 0 2024-03-24T10:45:59.794 INFO:tasks.workunit.client.0.smithi094.stdout:5/728: chown d7/d23/dd5/da9/fc8 8 1 2024-03-24T10:45:59.794 INFO:tasks.workunit.client.0.smithi094.stdout:1/579: dwrite d4/d9/d37/d33/d76/f9b [0,4194304] 0 2024-03-24T10:45:59.794 INFO:tasks.workunit.client.0.smithi094.stdout:1/580: getdents d4/d9/de/d6f/d62/d80 0 2024-03-24T10:45:59.794 INFO:tasks.workunit.client.0.smithi094.stdout:1/581: read d4/d9/de/f3b [1730417,115137] 0 2024-03-24T10:45:59.794 INFO:tasks.workunit.client.0.smithi094.stdout:1/582: chown d4/d9/db/d98/fa4 2052 1 2024-03-24T10:45:59.794 INFO:tasks.workunit.client.0.smithi094.stdout:6/699: dwrite d0/d78/da/f20 [0,4194304] 0 2024-03-24T10:45:59.799 INFO:tasks.workunit.client.0.smithi094.stdout:8/746: read df/d25/d28/d31/d47/d22/d55/f6d [480370,64645] 0 2024-03-24T10:45:59.801 INFO:tasks.workunit.client.0.smithi094.stdout:7/711: dread d1/d9/d16/d21/d51/d83/d39/d44/dd9/d8f/fc2 [0,4194304] 0 2024-03-24T10:45:59.806 INFO:tasks.workunit.client.0.smithi094.stdout:2/635: write d18/d66/d90/d9a/dbc/f43 [6931805,7161] 0 2024-03-24T10:45:59.806 INFO:tasks.workunit.client.0.smithi094.stdout:5/729: dwrite d7/dc/fa2 [0,4194304] 0 2024-03-24T10:45:59.816 INFO:tasks.workunit.client.0.smithi094.stdout:1/583: creat d4/d9/de/d6f/d62/d99/fce x:0 0 0 2024-03-24T10:45:59.816 INFO:tasks.workunit.client.0.smithi094.stdout:1/584: dread - d4/d9/de/d6f/fa5 zero size 2024-03-24T10:45:59.818 INFO:tasks.workunit.client.0.smithi094.stdout:5/730: dread d7/d23/d5f/d9/f29 [0,4194304] 0 2024-03-24T10:45:59.819 INFO:tasks.workunit.client.0.smithi094.stdout:8/747: creat df/d12/d5b/d63/d75/f100 x:0 0 0 2024-03-24T10:45:59.819 INFO:tasks.workunit.client.0.smithi094.stdout:8/748: getdents df/d25/d28/d31/d47/dbd 0 2024-03-24T10:45:59.824 INFO:tasks.workunit.client.0.smithi094.stdout:2/636: creat d18/d48/fbf x:0 0 0 2024-03-24T10:45:59.824 INFO:tasks.workunit.client.0.smithi094.stdout:2/637: chown d18/d1e/l3c 28070 1 2024-03-24T10:45:59.829 INFO:tasks.workunit.client.0.smithi094.stdout:8/749: dread df/d25/d28/d31/d47/f7e [0,4194304] 0 2024-03-24T10:45:59.830 INFO:tasks.workunit.client.0.smithi094.stdout:1/585: rename d4/d9/de/d6f/d62/dc8 to d4/d9/d37/dcf 0 2024-03-24T10:45:59.830 INFO:tasks.workunit.client.0.smithi094.stdout:1/586: write d4/d9/de/d6f/d62/fcc [599563,77078] 0 2024-03-24T10:45:59.832 INFO:tasks.workunit.client.0.smithi094.stdout:5/731: link d7/d23/d5f/d9/d48/cb5 d7/d23/d2b/d68/d63/cee 0 2024-03-24T10:45:59.837 INFO:tasks.workunit.client.0.smithi094.stdout:8/750: dread df/d25/d28/d31/d47/d22/fc5 [0,4194304] 0 2024-03-24T10:45:59.837 INFO:tasks.workunit.client.0.smithi094.stdout:8/751: chown df/d12/l64 77223045 1 2024-03-24T10:45:59.837 INFO:tasks.workunit.client.0.smithi094.stdout:1/587: mknod d4/d9/de/d6f/d62/d84/cd0 0 2024-03-24T10:45:59.837 INFO:tasks.workunit.client.0.smithi094.stdout:1/588: readlink d4/d9/de/d3d/d66/la9 0 2024-03-24T10:45:59.839 INFO:tasks.workunit.client.0.smithi094.stdout:5/732: mkdir d7/d23/d2b/def 0 2024-03-24T10:45:59.840 INFO:tasks.workunit.client.0.smithi094.stdout:8/752: mknod df/d25/c101 0 2024-03-24T10:45:59.843 INFO:tasks.workunit.client.0.smithi094.stdout:5/733: symlink d7/d23/lf0 0 2024-03-24T10:45:59.845 INFO:tasks.workunit.client.0.smithi094.stdout:2/638: dwrite d18/d48/d56/f58 [0,4194304] 0 2024-03-24T10:45:59.851 INFO:tasks.workunit.client.0.smithi094.stdout:2/639: truncate d18/f1b 1967631 0 2024-03-24T10:45:59.854 INFO:tasks.workunit.client.0.smithi094.stdout:8/753: dwrite df/d25/d28/d31/d3c/f8d [0,4194304] 0 2024-03-24T10:45:59.854 INFO:tasks.workunit.client.0.smithi094.stdout:8/754: chown df/d25/d28/d31/d47/d66/da3/db5/dbf 76839992 1 2024-03-24T10:45:59.854 INFO:tasks.workunit.client.0.smithi094.stdout:8/755: fdatasync df/d25/d28/d31/d47/d22/d55/f7a 0 2024-03-24T10:45:59.857 INFO:tasks.workunit.client.0.smithi094.stdout:2/640: rename d18/d66/d90/d9a/dbc/d4a to d18/d66/dbe/dc0 0 2024-03-24T10:45:59.857 INFO:tasks.workunit.client.0.smithi094.stdout:2/641: readlink d18/l2c 0 2024-03-24T10:45:59.862 INFO:tasks.workunit.client.0.smithi094.stdout:8/756: creat df/d25/d28/d31/d47/dfe/f102 x:0 0 0 2024-03-24T10:45:59.866 INFO:tasks.workunit.client.0.smithi094.stdout:2/642: symlink d18/d66/dbe/dc0/lc1 0 2024-03-24T10:45:59.869 INFO:tasks.workunit.client.0.smithi094.stdout:2/643: symlink d18/d66/d90/d9a/dbc/d71/lc2 0 2024-03-24T10:45:59.871 INFO:tasks.workunit.client.0.smithi094.stdout:8/757: dread df/d12/d5b/d63/f9c [0,4194304] 0 2024-03-24T10:45:59.871 INFO:tasks.workunit.client.0.smithi094.stdout:8/758: chown df/d8c/d8f/dc1/cd7 710 1 2024-03-24T10:45:59.871 INFO:tasks.workunit.client.0.smithi094.stdout:8/759: write df/d12/d5b/f83 [4189256,109213] 0 2024-03-24T10:45:59.872 INFO:tasks.workunit.client.0.smithi094.stdout:2/644: getdents d18 0 2024-03-24T10:45:59.872 INFO:tasks.workunit.client.0.smithi094.stdout:2/645: fsync d18/d66/d90/d9a/dbc/f43 0 2024-03-24T10:45:59.873 INFO:tasks.workunit.client.0.smithi094.stdout:2/646: write d18/d48/fbf [546713,95438] 0 2024-03-24T10:45:59.873 INFO:tasks.workunit.client.0.smithi094.stdout:2/647: chown d18/d66/faa 1565 1 2024-03-24T10:45:59.874 INFO:tasks.workunit.client.0.smithi094.stdout:8/760: rename df/d12/d5b/d63/d75/d96/lac to df/d25/d28/d31/d47/d22/l103 0 2024-03-24T10:45:59.877 INFO:tasks.workunit.client.0.smithi094.stdout:8/761: unlink df/d8c/d8f/dc1/fd4 0 2024-03-24T10:45:59.879 INFO:tasks.workunit.client.0.smithi094.stdout:2/648: write d18/d1e/f25 [627128,119618] 0 2024-03-24T10:45:59.879 INFO:tasks.workunit.client.0.smithi094.stdout:8/762: rmdir df/d25/daf 39 2024-03-24T10:45:59.882 INFO:tasks.workunit.client.0.smithi094.stdout:8/763: unlink df/d12/l53 0 2024-03-24T10:45:59.887 INFO:tasks.workunit.client.0.smithi094.stdout:8/764: dread df/f1b [0,4194304] 0 2024-03-24T10:45:59.890 INFO:tasks.workunit.client.0.smithi094.stdout:8/765: rename df/d25/d36/cb1 to df/d8c/c104 0 2024-03-24T10:45:59.890 INFO:tasks.workunit.client.0.smithi094.stdout:8/766: readlink df/d25/d28/l46 0 2024-03-24T10:45:59.892 INFO:tasks.workunit.client.0.smithi094.stdout:2/649: dwrite d18/d48/d56/fb0 [0,4194304] 0 2024-03-24T10:45:59.892 INFO:tasks.workunit.client.0.smithi094.stdout:8/767: creat df/d25/d28/f105 x:0 0 0 2024-03-24T10:45:59.893 INFO:tasks.workunit.client.0.smithi094.stdout:9/752: sync 2024-03-24T10:45:59.893 INFO:tasks.workunit.client.0.smithi094.stdout:0/698: sync 2024-03-24T10:45:59.893 INFO:tasks.workunit.client.0.smithi094.stdout:4/738: sync 2024-03-24T10:45:59.893 INFO:tasks.workunit.client.0.smithi094.stdout:9/753: write df/d2d/d1f/d58/d5a/f77 [608092,111393] 0 2024-03-24T10:45:59.894 INFO:tasks.workunit.client.0.smithi094.stdout:1/589: fsync d4/d9/de/d6f/d62/d99/fce 0 2024-03-24T10:45:59.894 INFO:tasks.workunit.client.0.smithi094.stdout:1/590: fsync d4/d9/de/d6f/d62/d84/fc1 0 2024-03-24T10:45:59.895 INFO:tasks.workunit.client.0.smithi094.stdout:0/699: dread d14/d22/d5b/d86/f8f [0,4194304] 0 2024-03-24T10:45:59.895 INFO:tasks.workunit.client.0.smithi094.stdout:0/700: creat d14/d22/d32/d70/ff6 x:0 0 0 2024-03-24T10:45:59.895 INFO:tasks.workunit.client.0.smithi094.stdout:0/701: write d14/d22/fe6 [1013976,6451] 0 2024-03-24T10:45:59.895 INFO:tasks.workunit.client.0.smithi094.stdout:0/702: write d14/d22/d7d/fd5 [857847,58158] 0 2024-03-24T10:45:59.910 INFO:tasks.workunit.client.0.smithi094.stdout:2/650: mkdir d18/d48/dc3 0 2024-03-24T10:45:59.911 INFO:tasks.workunit.client.0.smithi094.stdout:1/591: dwrite d4/d9/f25 [0,4194304] 0 2024-03-24T10:45:59.914 INFO:tasks.workunit.client.0.smithi094.stdout:8/768: unlink df/d12/dcd/ce4 0 2024-03-24T10:45:59.914 INFO:tasks.workunit.client.0.smithi094.stdout:8/769: chown df/d12/f6a 331045780 1 2024-03-24T10:45:59.914 INFO:tasks.workunit.client.0.smithi094.stdout:8/770: write df/d25/d28/d31/d47/f69 [3749732,5228] 0 2024-03-24T10:45:59.916 INFO:tasks.workunit.client.0.smithi094.stdout:2/651: dread d18/d48/d56/f58 [0,4194304] 0 2024-03-24T10:45:59.918 INFO:tasks.workunit.client.0.smithi094.stdout:4/739: creat d0/d5/d7d/fdc x:0 0 0 2024-03-24T10:45:59.918 INFO:tasks.workunit.client.0.smithi094.stdout:4/740: fdatasync d0/f5e 0 2024-03-24T10:45:59.918 INFO:tasks.workunit.client.0.smithi094.stdout:4/741: chown d0/d22/de 2281 1 2024-03-24T10:45:59.921 INFO:tasks.workunit.client.0.smithi094.stdout:9/754: creat df/d2d/d1f/d58/d70/dab/ff4 x:0 0 0 2024-03-24T10:45:59.926 INFO:tasks.workunit.client.0.smithi094.stdout:2/652: write d18/d66/d90/d9a/dbc/f50 [3390293,50563] 0 2024-03-24T10:45:59.926 INFO:tasks.workunit.client.0.smithi094.stdout:2/653: truncate d18/d48/f98 954070 0 2024-03-24T10:45:59.927 INFO:tasks.workunit.client.0.smithi094.stdout:0/703: truncate d14/d18/f21 1103007 0 2024-03-24T10:45:59.940 INFO:tasks.workunit.client.0.smithi094.stdout:1/592: creat d4/d9/db1/db9/fd1 x:0 0 0 2024-03-24T10:45:59.944 INFO:tasks.workunit.client.0.smithi094.stdout:8/771: truncate fa 2172457 0 2024-03-24T10:45:59.947 INFO:tasks.workunit.client.0.smithi094.stdout:4/742: creat d0/d5/d7d/d97/d4a/fdd x:0 0 0 2024-03-24T10:45:59.949 INFO:tasks.workunit.client.0.smithi094.stdout:9/755: mkdir df/d2d/d34/d2c/d97/df5 0 2024-03-24T10:45:59.949 INFO:tasks.workunit.client.0.smithi094.stdout:9/756: chown df/d2d/d34/d2c/c32 33241349 1 2024-03-24T10:45:59.954 INFO:tasks.workunit.client.0.smithi094.stdout:4/743: symlink d0/d5/d7d/dcf/lde 0 2024-03-24T10:45:59.954 INFO:tasks.workunit.client.0.smithi094.stdout:4/744: truncate d0/fb7 933095 0 2024-03-24T10:45:59.957 INFO:tasks.workunit.client.0.smithi094.stdout:9/757: truncate df/d2d/d1f/d58/d5a/d61/fc0 506422 0 2024-03-24T10:45:59.958 INFO:tasks.workunit.client.0.smithi094.stdout:8/772: dread f8 [0,4194304] 0 2024-03-24T10:45:59.960 INFO:tasks.workunit.client.0.smithi094.stdout:4/745: symlink d0/d22/de/dc8/ldf 0 2024-03-24T10:45:59.961 INFO:tasks.workunit.client.0.smithi094.stdout:2/654: dwrite d18/d48/d56/f58 [0,4194304] 0 2024-03-24T10:45:59.963 INFO:tasks.workunit.client.0.smithi094.stdout:9/758: mkdir df/dda/df6 0 2024-03-24T10:45:59.963 INFO:tasks.workunit.client.0.smithi094.stdout:0/704: dwrite d14/d22/d28/d2b/d68/f72 [0,4194304] 0 2024-03-24T10:45:59.964 INFO:tasks.workunit.client.0.smithi094.stdout:8/773: mkdir df/d12/d5b/d63/d106 0 2024-03-24T10:45:59.966 INFO:tasks.workunit.client.0.smithi094.stdout:4/746: creat d0/d22/de/d28/d2b/da2/fe0 x:0 0 0 2024-03-24T10:45:59.966 INFO:tasks.workunit.client.0.smithi094.stdout:4/747: fdatasync d0/d34/d73/d93/fa3 0 2024-03-24T10:45:59.969 INFO:tasks.workunit.client.0.smithi094.stdout:2/655: mknod d18/d7f/d88/da2/da4/cc4 0 2024-03-24T10:45:59.970 INFO:tasks.workunit.client.0.smithi094.stdout:0/705: dread d14/d22/d28/da2/fd1 [0,4194304] 0 2024-03-24T10:45:59.970 INFO:tasks.workunit.client.0.smithi094.stdout:9/759: creat df/d2d/d1f/d11/d43/ff7 x:0 0 0 2024-03-24T10:45:59.972 INFO:tasks.workunit.client.0.smithi094.stdout:2/656: mknod d18/d7f/d88/da2/cc5 0 2024-03-24T10:45:59.975 INFO:tasks.workunit.client.0.smithi094.stdout:0/706: symlink d14/d22/d28/d2b/d68/lf7 0 2024-03-24T10:45:59.976 INFO:tasks.workunit.client.0.smithi094.stdout:8/774: dread df/d25/f58 [0,4194304] 0 2024-03-24T10:45:59.977 INFO:tasks.workunit.client.0.smithi094.stdout:9/760: mkdir df/d2d/d1f/d7e/df8 0 2024-03-24T10:45:59.980 INFO:tasks.workunit.client.0.smithi094.stdout:2/657: unlink d18/d66/d90/d9a/d74/d7e/l95 0 2024-03-24T10:45:59.982 INFO:tasks.workunit.client.0.smithi094.stdout:4/748: dwrite d0/d5/d1b/f2e [0,4194304] 0 2024-03-24T10:45:59.983 INFO:tasks.workunit.client.0.smithi094.stdout:4/749: chown d0/d5/d7d/d97/d4a/fdd 0 1 2024-03-24T10:45:59.984 INFO:tasks.workunit.client.0.smithi094.stdout:8/775: creat df/d25/d28/d31/d47/ddf/dfd/f107 x:0 0 0 2024-03-24T10:45:59.987 INFO:tasks.workunit.client.0.smithi094.stdout:9/761: rename df/d2d/d34/db8/fe7 to df/d2d/d1f/d58/d5a/d61/d87/de8/ff9 0 2024-03-24T10:45:59.987 INFO:tasks.workunit.client.0.smithi094.stdout:9/762: truncate df/d2d/d1f/d58/d5a/d69/d7a/fe6 674299 0 2024-03-24T10:45:59.987 INFO:tasks.workunit.client.0.smithi094.stdout:9/763: fdatasync df/d2d/d1f/d58/d82/fb7 0 2024-03-24T10:45:59.990 INFO:tasks.workunit.client.0.smithi094.stdout:7/712: sync 2024-03-24T10:45:59.990 INFO:tasks.workunit.client.0.smithi094.stdout:8/776: creat df/d25/d28/d31/d3c/f108 x:0 0 0 2024-03-24T10:45:59.992 INFO:tasks.workunit.client.0.smithi094.stdout:6/700: sync 2024-03-24T10:45:59.992 INFO:tasks.workunit.client.0.smithi094.stdout:3/660: sync 2024-03-24T10:45:59.992 INFO:tasks.workunit.client.0.smithi094.stdout:6/701: readlink d0/d78/da/d23/d34/l45 0 2024-03-24T10:45:59.992 INFO:tasks.workunit.client.0.smithi094.stdout:3/661: fsync d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d34/fa0 0 2024-03-24T10:45:59.992 INFO:tasks.workunit.client.0.smithi094.stdout:3/662: fdatasync d0/db/dc/d28/d3c/d55/d7f/d20/d23/f5f 0 2024-03-24T10:45:59.993 INFO:tasks.workunit.client.0.smithi094.stdout:9/764: symlink df/d2d/d34/d2c/d74/lfa 0 2024-03-24T10:45:59.993 INFO:tasks.workunit.client.0.smithi094.stdout:9/765: creat df/d7b/ffb x:0 0 0 2024-03-24T10:45:59.993 INFO:tasks.workunit.client.0.smithi094.stdout:9/766: stat df/dda/df6 0 2024-03-24T10:45:59.993 INFO:tasks.workunit.client.0.smithi094.stdout:9/767: write df/d2d/d1f/d58/d5a/f4b [560696,61888] 0 2024-03-24T10:45:59.993 INFO:tasks.workunit.client.0.smithi094.stdout:9/768: creat df/ffc x:0 0 0 2024-03-24T10:45:59.993 INFO:tasks.workunit.client.0.smithi094.stdout:9/769: creat df/d7b/ffd x:0 0 0 2024-03-24T10:45:59.994 INFO:tasks.workunit.client.0.smithi094.stdout:0/707: dwrite d14/f2c [0,4194304] 0 2024-03-24T10:45:59.997 INFO:tasks.workunit.client.0.smithi094.stdout:7/713: rename d1/d9/d16/d21/d51/d83/l54 to d1/db4/d61/le3 0 2024-03-24T10:45:59.999 INFO:tasks.workunit.client.0.smithi094.stdout:3/663: write d0/f14 [2272587,121921] 0 2024-03-24T10:46:00.000 INFO:tasks.workunit.client.0.smithi094.stdout:3/664: readlink d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/l4a 0 2024-03-24T10:46:00.001 INFO:tasks.workunit.client.0.smithi094.stdout:6/702: mkdir d0/d78/da/d23/dde 0 2024-03-24T10:46:00.007 INFO:tasks.workunit.client.0.smithi094.stdout:2/658: dwrite d18/d1e/f53 [0,4194304] 0 2024-03-24T10:46:00.007 INFO:tasks.workunit.client.0.smithi094.stdout:4/750: dwrite d0/d5/d7d/d97/f67 [0,4194304] 0 2024-03-24T10:46:00.008 INFO:tasks.workunit.client.0.smithi094.stdout:6/703: dread d0/d78/da/d23/f6c [0,4194304] 0 2024-03-24T10:46:00.014 INFO:tasks.workunit.client.0.smithi094.stdout:0/708: truncate d14/d22/d28/d2b/f3b 647905 0 2024-03-24T10:46:00.015 INFO:tasks.workunit.client.0.smithi094.stdout:9/770: dwrite df/d2d/d1f/d58/d5a/f4b [0,4194304] 0 2024-03-24T10:46:00.017 INFO:tasks.workunit.client.0.smithi094.stdout:5/734: sync 2024-03-24T10:46:00.017 INFO:tasks.workunit.client.0.smithi094.stdout:5/735: write d7/d23/d5f/d9/d48/f74 [217535,30605] 0 2024-03-24T10:46:00.017 INFO:tasks.workunit.client.0.smithi094.stdout:5/736: dread - d7/d23/d5f/d9/f92 zero size 2024-03-24T10:46:00.023 INFO:tasks.workunit.client.0.smithi094.stdout:1/593: getdents d4/d9/db1/db9 0 2024-03-24T10:46:00.025 INFO:tasks.workunit.client.0.smithi094.stdout:3/665: rename d0/db/dc/d28/d3c/d55/d7f/d20/d23/fc2 to d0/db/dc/d28/d3c/d55/fe7 0 2024-03-24T10:46:00.028 INFO:tasks.workunit.client.0.smithi094.stdout:9/771: dread df/d2d/d1f/d11/f90 [0,4194304] 0 2024-03-24T10:46:00.028 INFO:tasks.workunit.client.0.smithi094.stdout:9/772: read df/d2d/d1f/d58/d5a/f77 [682031,25920] 0 2024-03-24T10:46:00.028 INFO:tasks.workunit.client.0.smithi094.stdout:6/704: dwrite d0/d78/da/d23/d34/d4e/d5d/f7f [0,4194304] 0 2024-03-24T10:46:00.029 INFO:tasks.workunit.client.0.smithi094.stdout:2/659: mkdir d18/d77/dc6 0 2024-03-24T10:46:00.033 INFO:tasks.workunit.client.0.smithi094.stdout:1/594: dread d4/d9/db/d31/d3c/f7c [0,4194304] 0 2024-03-24T10:46:00.034 INFO:tasks.workunit.client.0.smithi094.stdout:1/595: dread - d4/d9/de/d6f/fa5 zero size 2024-03-24T10:46:00.034 INFO:tasks.workunit.client.0.smithi094.stdout:1/596: fsync d4/d9/db/d31/f48 0 2024-03-24T10:46:00.034 INFO:tasks.workunit.client.0.smithi094.stdout:1/597: chown d4/d36/f73 17 1 2024-03-24T10:46:00.036 INFO:tasks.workunit.client.0.smithi094.stdout:4/751: mknod d0/d22/de/d28/d2b/ce1 0 2024-03-24T10:46:00.037 INFO:tasks.workunit.client.0.smithi094.stdout:0/709: mknod d14/d22/d28/d2b/dc0/cf8 0 2024-03-24T10:46:00.039 INFO:tasks.workunit.client.0.smithi094.stdout:5/737: mknod d7/d23/d5f/cf1 0 2024-03-24T10:46:00.039 INFO:tasks.workunit.client.0.smithi094.stdout:0/710: dread d14/d22/d5b/d86/f8f [0,4194304] 0 2024-03-24T10:46:00.039 INFO:tasks.workunit.client.0.smithi094.stdout:0/711: stat d14/d22/d28/d2b/d41/d53/fed 0 2024-03-24T10:46:00.041 INFO:tasks.workunit.client.0.smithi094.stdout:7/714: rmdir d1/d6c/d80 39 2024-03-24T10:46:00.043 INFO:tasks.workunit.client.0.smithi094.stdout:3/666: mkdir d0/db/dc/d28/d3c/d7c/de8 0 2024-03-24T10:46:00.045 INFO:tasks.workunit.client.0.smithi094.stdout:9/773: truncate df/d2d/f15 1042815 0 2024-03-24T10:46:00.045 INFO:tasks.workunit.client.0.smithi094.stdout:9/774: write df/f22 [711681,32734] 0 2024-03-24T10:46:00.045 INFO:tasks.workunit.client.0.smithi094.stdout:9/775: fsync df/d2d/d1f/d11/f4f 0 2024-03-24T10:46:00.046 INFO:tasks.workunit.client.0.smithi094.stdout:4/752: write d0/d5/d1b/d4f/fa7 [908394,75708] 0 2024-03-24T10:46:00.046 INFO:tasks.workunit.client.0.smithi094.stdout:4/753: dread - d0/d22/de/d28/d2b/d33/da1/fda zero size 2024-03-24T10:46:00.046 INFO:tasks.workunit.client.0.smithi094.stdout:0/712: write d14/d18/f1d [1505006,64332] 0 2024-03-24T10:46:00.046 INFO:tasks.workunit.client.0.smithi094.stdout:0/713: chown d14/d22/d28/d2b/dc0/cd2 6 1 2024-03-24T10:46:00.053 INFO:tasks.workunit.client.0.smithi094.stdout:6/705: symlink d0/d78/da/d23/ldf 0 2024-03-24T10:46:00.056 INFO:tasks.workunit.client.0.smithi094.stdout:4/754: read d0/d5/d7d/f99 [115825,96971] 0 2024-03-24T10:46:00.063 INFO:tasks.workunit.client.0.smithi094.stdout:3/667: dwrite d0/db/dc/d28/d3c/d7c/fb0 [0,4194304] 0 2024-03-24T10:46:00.064 INFO:tasks.workunit.client.0.smithi094.stdout:0/714: dwrite d14/d18/f1d [0,4194304] 0 2024-03-24T10:46:00.069 INFO:tasks.workunit.client.0.smithi094.stdout:5/738: rename d7/d23/d5f/d9/d48/d51/f52 to d7/d23/dd5/da9/ff2 0 2024-03-24T10:46:00.069 INFO:tasks.workunit.client.0.smithi094.stdout:5/739: fsync d7/d23/d5f/d9/f4f 0 2024-03-24T10:46:00.070 INFO:tasks.workunit.client.0.smithi094.stdout:7/715: symlink d1/db4/d61/le4 0 2024-03-24T10:46:00.074 INFO:tasks.workunit.client.0.smithi094.stdout:9/776: unlink df/d2d/d34/d2c/l48 0 2024-03-24T10:46:00.074 INFO:tasks.workunit.client.0.smithi094.stdout:8/777: rmdir df/d25/d28/d31/d3c 39 2024-03-24T10:46:00.081 INFO:tasks.workunit.client.0.smithi094.stdout:9/777: dread df/d7b/fbb [0,4194304] 0 2024-03-24T10:46:00.081 INFO:tasks.workunit.client.0.smithi094.stdout:9/778: creat df/d7b/ffe x:0 0 0 2024-03-24T10:46:00.081 INFO:tasks.workunit.client.0.smithi094.stdout:9/779: stat df/d2d/d1f/d11/d96/df3 0 2024-03-24T10:46:00.082 INFO:tasks.workunit.client.0.smithi094.stdout:4/755: truncate d0/d34/d73/d93/fb4 1102476 0 2024-03-24T10:46:00.083 INFO:tasks.workunit.client.0.smithi094.stdout:5/740: dwrite d7/d23/d5f/d14/d6f/f89 [0,4194304] 0 2024-03-24T10:46:00.083 INFO:tasks.workunit.client.0.smithi094.stdout:5/741: creat d7/d23/d5f/d14/d6f/dbe/ff3 x:0 0 0 2024-03-24T10:46:00.084 INFO:tasks.workunit.client.0.smithi094.stdout:8/778: write df/d12/d5b/d63/f94 [432581,112869] 0 2024-03-24T10:46:00.084 INFO:tasks.workunit.client.0.smithi094.stdout:8/779: write df/d25/d28/d31/d47/d66/da3/da8/fe5 [433153,53087] 0 2024-03-24T10:46:00.084 INFO:tasks.workunit.client.0.smithi094.stdout:8/780: fsync df/d25/d28/fa7 0 2024-03-24T10:46:00.085 INFO:tasks.workunit.client.0.smithi094.stdout:8/781: write df/d25/d28/d31/d47/f5a [327,38576] 0 2024-03-24T10:46:00.085 INFO:tasks.workunit.client.0.smithi094.stdout:8/782: fdatasync df/d25/d28/fa7 0 2024-03-24T10:46:00.085 INFO:tasks.workunit.client.0.smithi094.stdout:8/783: creat df/d25/d28/d31/d95/de3/f109 x:0 0 0 2024-03-24T10:46:00.089 INFO:tasks.workunit.client.0.smithi094.stdout:3/668: mkdir d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/de9 0 2024-03-24T10:46:00.092 INFO:tasks.workunit.client.0.smithi094.stdout:0/715: link d14/d22/d7d/fd5 d14/d22/d32/d70/ff9 0 2024-03-24T10:46:00.094 INFO:tasks.workunit.client.0.smithi094.stdout:3/669: dread d0/db/dc/d28/d3c/d55/d7f/d20/d23/f3f [0,4194304] 0 2024-03-24T10:46:00.096 INFO:tasks.workunit.client.0.smithi094.stdout:7/716: rename d1/db4/fdd to d1/d9/d16/d21/d51/dcc/fe5 0 2024-03-24T10:46:00.098 INFO:tasks.workunit.client.0.smithi094.stdout:8/784: dwrite fa [0,4194304] 0 2024-03-24T10:46:00.099 INFO:tasks.workunit.client.0.smithi094.stdout:6/706: getdents d0/d78/da/d23/d34/d4e 0 2024-03-24T10:46:00.099 INFO:tasks.workunit.client.0.smithi094.stdout:6/707: write d0/d78/da/d23/f26 [1294701,58729] 0 2024-03-24T10:46:00.104 INFO:tasks.workunit.client.0.smithi094.stdout:9/780: symlink df/d2d/d1f/d58/d70/dab/d80/lff 0 2024-03-24T10:46:00.104 INFO:tasks.workunit.client.0.smithi094.stdout:9/781: fsync df/f22 0 2024-03-24T10:46:00.107 INFO:tasks.workunit.client.0.smithi094.stdout:4/756: creat d0/d22/de/d28/d2b/d33/da4/da5/fe2 x:0 0 0 2024-03-24T10:46:00.107 INFO:tasks.workunit.client.0.smithi094.stdout:4/757: truncate d0/d5/d7d/fdc 134429 0 2024-03-24T10:46:00.114 INFO:tasks.workunit.client.0.smithi094.stdout:7/717: dwrite d1/d9/d25/d3d/f5a [0,4194304] 0 2024-03-24T10:46:00.118 INFO:tasks.workunit.client.0.smithi094.stdout:9/782: write df/d2d/d1f/d58/fd8 [1827167,128711] 0 2024-03-24T10:46:00.118 INFO:tasks.workunit.client.0.smithi094.stdout:9/783: dread - df/d2d/d1f/fed zero size 2024-03-24T10:46:00.118 INFO:tasks.workunit.client.0.smithi094.stdout:9/784: chown df/d2d/d1f/d58/d5a/d61/d87/dc8/fdb 58988 1 2024-03-24T10:46:00.122 INFO:tasks.workunit.client.0.smithi094.stdout:5/742: link d7/d23/d5f/d9/d48/d51/lc3 d7/d23/d2b/d68/d63/lf4 0 2024-03-24T10:46:00.128 INFO:tasks.workunit.client.0.smithi094.stdout:3/670: mkdir d0/db/dc3/dea 0 2024-03-24T10:46:00.140 INFO:tasks.workunit.client.0.smithi094.stdout:8/785: creat df/d25/d28/d31/d47/d22/d55/f10a x:0 0 0 2024-03-24T10:46:00.142 INFO:tasks.workunit.client.0.smithi094.stdout:6/708: symlink d0/d78/d66/d7a/le0 0 2024-03-24T10:46:00.144 INFO:tasks.workunit.client.0.smithi094.stdout:7/718: creat d1/d9/d16/d21/d51/d83/fe6 x:0 0 0 2024-03-24T10:46:00.144 INFO:tasks.workunit.client.0.smithi094.stdout:3/671: dwrite d0/db/f2b [4194304,4194304] 0 2024-03-24T10:46:00.146 INFO:tasks.workunit.client.0.smithi094.stdout:9/785: link df/d2d/d1f/d11/c18 df/d2d/d1f/d11/d53/c100 0 2024-03-24T10:46:00.146 INFO:tasks.workunit.client.0.smithi094.stdout:9/786: write df/d2d/d1f/d58/d70/dab/d80/fa3 [226912,2452] 0 2024-03-24T10:46:00.150 INFO:tasks.workunit.client.0.smithi094.stdout:8/786: mkdir df/d25/d36/d10b 0 2024-03-24T10:46:00.156 INFO:tasks.workunit.client.0.smithi094.stdout:7/719: creat d1/d9/d16/d21/d51/d83/d39/d44/dd9/d34/fe7 x:0 0 0 2024-03-24T10:46:00.160 INFO:tasks.workunit.client.0.smithi094.stdout:3/672: truncate d0/db/dc/d28/d3c/d55/fe7 2290238 0 2024-03-24T10:46:00.161 INFO:tasks.workunit.client.0.smithi094.stdout:9/787: dwrite df/d2d/f8d [0,4194304] 0 2024-03-24T10:46:00.165 INFO:tasks.workunit.client.0.smithi094.stdout:8/787: dwrite df/d25/d28/d31/d47/d22/d55/f10a [0,4194304] 0 2024-03-24T10:46:00.166 INFO:tasks.workunit.client.0.smithi094.stdout:6/709: symlink d0/d21/d2a/d2f/le1 0 2024-03-24T10:46:00.174 INFO:tasks.workunit.client.0.smithi094.stdout:8/788: dread df/d25/d28/d31/d47/f85 [0,4194304] 0 2024-03-24T10:46:00.174 INFO:tasks.workunit.client.0.smithi094.stdout:3/673: dwrite d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/fd1 [0,4194304] 0 2024-03-24T10:46:00.174 INFO:tasks.workunit.client.0.smithi094.stdout:3/674: chown d0/f1e 148562801 1 2024-03-24T10:46:00.175 INFO:tasks.workunit.client.0.smithi094.stdout:3/675: chown d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d87 1920 1 2024-03-24T10:46:00.177 INFO:tasks.workunit.client.0.smithi094.stdout:9/788: fsync df/d7b/ffe 0 2024-03-24T10:46:00.184 INFO:tasks.workunit.client.0.smithi094.stdout:6/710: symlink d0/d1a/le2 0 2024-03-24T10:46:00.184 INFO:tasks.workunit.client.0.smithi094.stdout:2/660: sync 2024-03-24T10:46:00.184 INFO:tasks.workunit.client.0.smithi094.stdout:2/661: truncate d18/d5f/fad 362771 0 2024-03-24T10:46:00.184 INFO:tasks.workunit.client.0.smithi094.stdout:6/711: write d0/da9/fd5 [42875,82752] 0 2024-03-24T10:46:00.185 INFO:tasks.workunit.client.0.smithi094.stdout:6/712: write d0/d78/da/d77/fa3 [83851,42134] 0 2024-03-24T10:46:00.187 INFO:tasks.workunit.client.0.smithi094.stdout:8/789: rmdir df/d25/d28/d31/d47/d66/da3/da8 39 2024-03-24T10:46:00.193 INFO:tasks.workunit.client.0.smithi094.stdout:5/743: rmdir d7/d23/d2b/d68 39 2024-03-24T10:46:00.195 INFO:tasks.workunit.client.0.smithi094.stdout:6/713: dread d0/d30/d50/fb4 [0,4194304] 0 2024-03-24T10:46:00.197 INFO:tasks.workunit.client.0.smithi094.stdout:3/676: mknod d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d87/dc5/ceb 0 2024-03-24T10:46:00.200 INFO:tasks.workunit.client.0.smithi094.stdout:9/789: mkdir df/d2d/d1f/d58/d5a/d50/da5/d101 0 2024-03-24T10:46:00.203 INFO:tasks.workunit.client.0.smithi094.stdout:2/662: unlink d18/d66/faa 0 2024-03-24T10:46:00.204 INFO:tasks.workunit.client.0.smithi094.stdout:8/790: dwrite df/d25/d28/d31/d47/d66/fe2 [0,4194304] 0 2024-03-24T10:46:00.206 INFO:tasks.workunit.client.0.smithi094.stdout:5/744: creat d7/d23/d5f/d9/dde/ff5 x:0 0 0 2024-03-24T10:46:00.208 INFO:tasks.workunit.client.0.smithi094.stdout:6/714: mkdir d0/d78/da/d77/db6/de3 0 2024-03-24T10:46:00.208 INFO:tasks.workunit.client.0.smithi094.stdout:1/598: sync 2024-03-24T10:46:00.211 INFO:tasks.workunit.client.0.smithi094.stdout:3/677: creat d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/fec x:0 0 0 2024-03-24T10:46:00.216 INFO:tasks.workunit.client.0.smithi094.stdout:9/790: rename df/d2d/d1f/d7e/cee to df/d2d/d1f/d11/d43/c102 0 2024-03-24T10:46:00.216 INFO:tasks.workunit.client.0.smithi094.stdout:9/791: dread - df/d7b/ffd zero size 2024-03-24T10:46:00.218 INFO:tasks.workunit.client.0.smithi094.stdout:2/663: mknod d18/d66/d90/cc7 0 2024-03-24T10:46:00.219 INFO:tasks.workunit.client.0.smithi094.stdout:2/664: fdatasync d18/d48/d56/d73/f7a 0 2024-03-24T10:46:00.220 INFO:tasks.workunit.client.0.smithi094.stdout:6/715: read d0/d21/f56 [1307002,51389] 0 2024-03-24T10:46:00.229 INFO:tasks.workunit.client.0.smithi094.stdout:8/791: unlink df/d12/d5b/d63/fa9 0 2024-03-24T10:46:00.233 INFO:tasks.workunit.client.0.smithi094.stdout:6/716: rename d0/d21/c25 to d0/d78/da/d23/d7c/ce4 0 2024-03-24T10:46:00.233 INFO:tasks.workunit.client.0.smithi094.stdout:6/717: fsync d0/d78/da/d23/d7c/fd9 0 2024-03-24T10:46:00.233 INFO:tasks.workunit.client.0.smithi094.stdout:6/718: chown d0/d78/da/d77/db6/fb7 4 1 2024-03-24T10:46:00.237 INFO:tasks.workunit.client.0.smithi094.stdout:6/719: truncate d0/f3c 86818 0 2024-03-24T10:46:00.240 INFO:tasks.workunit.client.0.smithi094.stdout:5/745: dwrite d7/d23/d5f/d14/f85 [0,4194304] 0 2024-03-24T10:46:00.247 INFO:tasks.workunit.client.0.smithi094.stdout:1/599: dwrite d4/d9/de/f2b [0,4194304] 0 2024-03-24T10:46:00.247 INFO:tasks.workunit.client.0.smithi094.stdout:1/600: chown d4/d9/de/d6f/d5f/d8d/fc2 21 1 2024-03-24T10:46:00.247 INFO:tasks.workunit.client.0.smithi094.stdout:1/601: truncate d4/d9/d37/d33/f51 4644376 0 2024-03-24T10:46:00.248 INFO:tasks.workunit.client.0.smithi094.stdout:9/792: dwrite df/d2d/d1f/d11/f4e [0,4194304] 0 2024-03-24T10:46:00.248 INFO:tasks.workunit.client.0.smithi094.stdout:9/793: creat df/d2d/d1f/d58/d5a/f103 x:0 0 0 2024-03-24T10:46:00.248 INFO:tasks.workunit.client.0.smithi094.stdout:3/678: dwrite d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/d73/f80 [0,4194304] 0 2024-03-24T10:46:00.248 INFO:tasks.workunit.client.0.smithi094.stdout:2/665: dwrite d18/d66/dbe/dc0/f55 [4194304,4194304] 0 2024-03-24T10:46:00.251 INFO:tasks.workunit.client.0.smithi094.stdout:1/602: symlink d4/d9/d37/dcf/ld2 0 2024-03-24T10:46:00.251 INFO:tasks.workunit.client.0.smithi094.stdout:1/603: write d4/d9/de/d6f/fa5 [271662,46125] 0 2024-03-24T10:46:00.251 INFO:tasks.workunit.client.0.smithi094.stdout:1/604: write d4/d9/f32 [1708361,10166] 0 2024-03-24T10:46:00.255 INFO:tasks.workunit.client.0.smithi094.stdout:9/794: mkdir df/d2d/d1f/d58/d70/d104 0 2024-03-24T10:46:00.255 INFO:tasks.workunit.client.0.smithi094.stdout:6/720: dwrite d0/d78/d66/d7a/f8b [0,4194304] 0 2024-03-24T10:46:00.258 INFO:tasks.workunit.client.0.smithi094.stdout:1/605: dread d4/d9/d37/f4c [0,4194304] 0 2024-03-24T10:46:00.270 INFO:tasks.workunit.client.0.smithi094.stdout:0/716: sync 2024-03-24T10:46:00.273 INFO:tasks.workunit.client.0.smithi094.stdout:3/679: link d0/db/dc/d28/d50/fe6 d0/db/dc/d28/d38/fed 0 2024-03-24T10:46:00.276 INFO:tasks.workunit.client.0.smithi094.stdout:2/666: mkdir d18/d66/dbe/dc8 0 2024-03-24T10:46:00.279 INFO:tasks.workunit.client.0.smithi094.stdout:8/792: dwrite df/d25/d28/d31/d47/d22/d55/f10a [4194304,4194304] 0 2024-03-24T10:46:00.279 INFO:tasks.workunit.client.0.smithi094.stdout:8/793: write df/d25/d28/d31/d47/d22/fc5 [1777274,123479] 0 2024-03-24T10:46:00.279 INFO:tasks.workunit.client.0.smithi094.stdout:8/794: write df/d25/d28/d31/d3c/f108 [235167,88583] 0 2024-03-24T10:46:00.289 INFO:tasks.workunit.client.0.smithi094.stdout:6/721: rename d0/d30/d50/l70 to d0/le5 0 2024-03-24T10:46:00.289 INFO:tasks.workunit.client.0.smithi094.stdout:6/722: fsync d0/d1a/f6f 0 2024-03-24T10:46:00.290 INFO:tasks.workunit.client.0.smithi094.stdout:0/717: mkdir d14/d22/d28/d48/d7b/dfa 0 2024-03-24T10:46:00.293 INFO:tasks.workunit.client.0.smithi094.stdout:3/680: unlink d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/l5a 0 2024-03-24T10:46:00.293 INFO:tasks.workunit.client.0.smithi094.stdout:3/681: creat d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d87/dc5/fee x:0 0 0 2024-03-24T10:46:00.293 INFO:tasks.workunit.client.0.smithi094.stdout:3/682: creat d0/db/dc/d28/d3c/d7c/fef x:0 0 0 2024-03-24T10:46:00.294 INFO:tasks.workunit.client.0.smithi094.stdout:2/667: creat d18/d66/d90/d9a/dbc/d71/fc9 x:0 0 0 2024-03-24T10:46:00.295 INFO:tasks.workunit.client.0.smithi094.stdout:2/668: write d18/d48/f98 [419383,23270] 0 2024-03-24T10:46:00.298 INFO:tasks.workunit.client.0.smithi094.stdout:2/669: dread d18/d66/d90/d9a/dbc/f35 [0,4194304] 0 2024-03-24T10:46:00.298 INFO:tasks.workunit.client.0.smithi094.stdout:2/670: readlink d18/d48/l9b 0 2024-03-24T10:46:00.298 INFO:tasks.workunit.client.0.smithi094.stdout:2/671: stat d18/d48/d56/l7c 0 2024-03-24T10:46:00.298 INFO:tasks.workunit.client.0.smithi094.stdout:8/795: creat df/d25/d36/f10c x:0 0 0 2024-03-24T10:46:00.299 INFO:tasks.workunit.client.0.smithi094.stdout:8/796: creat df/d25/dce/f10d x:0 0 0 2024-03-24T10:46:00.300 INFO:tasks.workunit.client.0.smithi094.stdout:9/795: rmdir df/d7b 39 2024-03-24T10:46:00.301 INFO:tasks.workunit.client.0.smithi094.stdout:9/796: creat df/d2d/d34/d2c/d97/f105 x:0 0 0 2024-03-24T10:46:00.301 INFO:tasks.workunit.client.0.smithi094.stdout:9/797: dread - df/d2d/d1f/d58/d70/de2/ff0 zero size 2024-03-24T10:46:00.301 INFO:tasks.workunit.client.0.smithi094.stdout:9/798: dread - df/d2d/d1f/d58/d70/dab/d80/fce zero size 2024-03-24T10:46:00.301 INFO:tasks.workunit.client.0.smithi094.stdout:9/799: write df/d2d/d1f/d58/d5a/f103 [721199,69573] 0 2024-03-24T10:46:00.301 INFO:tasks.workunit.client.0.smithi094.stdout:9/800: truncate df/d7b/ffd 114555 0 2024-03-24T10:46:00.303 INFO:tasks.workunit.client.0.smithi094.stdout:6/723: rename d0/d78/da/d77/db6 to d0/d30/de6 0 2024-03-24T10:46:00.304 INFO:tasks.workunit.client.0.smithi094.stdout:6/724: dread - d0/da9/fcd zero size 2024-03-24T10:46:00.304 INFO:tasks.workunit.client.0.smithi094.stdout:6/725: fdatasync d0/d1a/f6f 0 2024-03-24T10:46:00.304 INFO:tasks.workunit.client.0.smithi094.stdout:6/726: chown d0/d78/d66/ld8 57349229 1 2024-03-24T10:46:00.309 INFO:tasks.workunit.client.0.smithi094.stdout:2/672: dread d18/f19 [0,4194304] 0 2024-03-24T10:46:00.309 INFO:tasks.workunit.client.0.smithi094.stdout:2/673: write f15 [7553809,97856] 0 2024-03-24T10:46:00.310 INFO:tasks.workunit.client.0.smithi094.stdout:3/683: mknod d0/db/dc/d28/d38/d6a/cf0 0 2024-03-24T10:46:00.310 INFO:tasks.workunit.client.0.smithi094.stdout:3/684: chown d0/lda 20682751 1 2024-03-24T10:46:00.314 INFO:tasks.workunit.client.0.smithi094.stdout:9/801: truncate df/d2d/d1f/d58/d5a/d61/f8a 549564 0 2024-03-24T10:46:00.314 INFO:tasks.workunit.client.0.smithi094.stdout:9/802: readlink df/d2d/d1f/d58/d5a/d61/d87/lad 0 2024-03-24T10:46:00.314 INFO:tasks.workunit.client.0.smithi094.stdout:9/803: stat df/d2d/d1f/d58/d5a/d61/f8a 0 2024-03-24T10:46:00.320 INFO:tasks.workunit.client.0.smithi094.stdout:6/727: link d0/d21/d2a/d55/la1 d0/d78/da/d23/dc3/le7 0 2024-03-24T10:46:00.320 INFO:tasks.workunit.client.0.smithi094.stdout:6/728: creat d0/d1a/fe8 x:0 0 0 2024-03-24T10:46:00.320 INFO:tasks.workunit.client.0.smithi094.stdout:6/729: fdatasync d0/d21/d2a/fd6 0 2024-03-24T10:46:00.321 INFO:tasks.workunit.client.0.smithi094.stdout:0/718: symlink d14/d22/d28/lfb 0 2024-03-24T10:46:00.322 INFO:tasks.workunit.client.0.smithi094.stdout:4/758: sync 2024-03-24T10:46:00.327 INFO:tasks.workunit.client.0.smithi094.stdout:2/674: mkdir d18/d66/d90/d9a/d74/d7e/dca 0 2024-03-24T10:46:00.327 INFO:tasks.workunit.client.0.smithi094.stdout:2/675: write d18/f31 [52219,10628] 0 2024-03-24T10:46:00.329 INFO:tasks.workunit.client.0.smithi094.stdout:3/685: creat d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/ff1 x:0 0 0 2024-03-24T10:46:00.329 INFO:tasks.workunit.client.0.smithi094.stdout:3/686: write d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d87/dc5/fde [600759,6618] 0 2024-03-24T10:46:00.331 INFO:tasks.workunit.client.0.smithi094.stdout:9/804: link df/d7b/ffe df/d2d/d1f/d11/d96/df3/f106 0 2024-03-24T10:46:00.331 INFO:tasks.workunit.client.0.smithi094.stdout:7/720: sync 2024-03-24T10:46:00.331 INFO:tasks.workunit.client.0.smithi094.stdout:7/721: write d1/d9/f30 [2725576,80489] 0 2024-03-24T10:46:00.337 INFO:tasks.workunit.client.0.smithi094.stdout:0/719: rename d14/d22/d32/c3f to d14/d22/d28/d48/cfc 0 2024-03-24T10:46:00.339 INFO:tasks.workunit.client.0.smithi094.stdout:4/759: rmdir d0/d22/de/d28/d3d/d91 39 2024-03-24T10:46:00.341 INFO:tasks.workunit.client.0.smithi094.stdout:7/722: dread d1/d9/d16/d21/d51/d83/d39/d44/dd9/f4b [0,4194304] 0 2024-03-24T10:46:00.341 INFO:tasks.workunit.client.0.smithi094.stdout:2/676: creat d18/fcb x:0 0 0 2024-03-24T10:46:00.341 INFO:tasks.workunit.client.0.smithi094.stdout:2/677: write d18/d66/d90/d9a/dbc/f6f [6409,77188] 0 2024-03-24T10:46:00.343 INFO:tasks.workunit.client.0.smithi094.stdout:6/730: dwrite d0/d1a/f7b [4194304,4194304] 0 2024-03-24T10:46:00.344 INFO:tasks.workunit.client.0.smithi094.stdout:6/731: write d0/d78/f44 [1287421,74805] 0 2024-03-24T10:46:00.344 INFO:tasks.workunit.client.0.smithi094.stdout:6/732: write d0/d1a/d35/f72 [1018928,101001] 0 2024-03-24T10:46:00.344 INFO:tasks.workunit.client.0.smithi094.stdout:3/687: unlink d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d87/ca5 0 2024-03-24T10:46:00.344 INFO:tasks.workunit.client.0.smithi094.stdout:3/688: write d0/db/f48 [4868376,16121] 0 2024-03-24T10:46:00.344 INFO:tasks.workunit.client.0.smithi094.stdout:3/689: readlink d0/db/l8 0 2024-03-24T10:46:00.347 INFO:tasks.workunit.client.0.smithi094.stdout:9/805: rmdir df/d2d 39 2024-03-24T10:46:00.350 INFO:tasks.workunit.client.0.smithi094.stdout:0/720: rmdir d14/d22/d32/d70/dd0/dce 39 2024-03-24T10:46:00.350 INFO:tasks.workunit.client.0.smithi094.stdout:0/721: readlink d14/d18/lb4 0 2024-03-24T10:46:00.350 INFO:tasks.workunit.client.0.smithi094.stdout:0/722: fdatasync d14/d22/d28/d2b/d41/fe3 0 2024-03-24T10:46:00.353 INFO:tasks.workunit.client.0.smithi094.stdout:7/723: link d1/db4/l62 d1/db4/le8 0 2024-03-24T10:46:00.356 INFO:tasks.workunit.client.0.smithi094.stdout:0/723: dread d14/d22/fe6 [0,4194304] 0 2024-03-24T10:46:00.356 INFO:tasks.workunit.client.0.smithi094.stdout:0/724: chown d14/d22/d28/fe1 1659 1 2024-03-24T10:46:00.360 INFO:tasks.workunit.client.0.smithi094.stdout:6/733: link d0/d21/f56 d0/d78/da/d23/d83/fe9 0 2024-03-24T10:46:00.360 INFO:tasks.workunit.client.0.smithi094.stdout:4/760: dwrite d0/d22/ff [0,4194304] 0 2024-03-24T10:46:00.362 INFO:tasks.workunit.client.0.smithi094.stdout:9/806: rmdir df/d2d/d1f/d58/d5a/d61/d87/de8 39 2024-03-24T10:46:00.365 INFO:tasks.workunit.client.0.smithi094.stdout:9/807: write df/f36 [4620187,24670] 0 2024-03-24T10:46:00.366 INFO:tasks.workunit.client.0.smithi094.stdout:3/690: dwrite d0/f11 [0,4194304] 0 2024-03-24T10:46:00.366 INFO:tasks.workunit.client.0.smithi094.stdout:7/724: mknod d1/d9/d16/d21/d51/d83/d39/d44/dd9/ce9 0 2024-03-24T10:46:00.372 INFO:tasks.workunit.client.0.smithi094.stdout:6/734: unlink d0/d30/d50/d4d/d6b/f90 0 2024-03-24T10:46:00.375 INFO:tasks.workunit.client.0.smithi094.stdout:4/761: symlink d0/d22/de/le3 0 2024-03-24T10:46:00.375 INFO:tasks.workunit.client.0.smithi094.stdout:4/762: chown d0/d22/ff 6635833 1 2024-03-24T10:46:00.376 INFO:tasks.workunit.client.0.smithi094.stdout:0/725: dwrite d14/d22/d32/f9b [0,4194304] 0 2024-03-24T10:46:00.376 INFO:tasks.workunit.client.0.smithi094.stdout:0/726: dread - d14/d22/d28/d48/d7b/fa8 zero size 2024-03-24T10:46:00.381 INFO:tasks.workunit.client.0.smithi094.stdout:9/808: read df/d2d/d1f/d11/d43/f6d [3525664,119099] 0 2024-03-24T10:46:00.381 INFO:tasks.workunit.client.0.smithi094.stdout:9/809: creat df/d2d/d1f/d58/d70/dab/d80/f107 x:0 0 0 2024-03-24T10:46:00.382 INFO:tasks.workunit.client.0.smithi094.stdout:9/810: chown df/d2d/d1f/d11/d43/f76 2 1 2024-03-24T10:46:00.382 INFO:tasks.workunit.client.0.smithi094.stdout:9/811: write df/d2d/d1f/d11/d53/f81 [819773,27626] 0 2024-03-24T10:46:00.382 INFO:tasks.workunit.client.0.smithi094.stdout:9/812: write df/d2d/d1f/d58/d5a/d69/d7a/fe6 [907310,97895] 0 2024-03-24T10:46:00.382 INFO:tasks.workunit.client.0.smithi094.stdout:9/813: fdatasync df/d2d/d1f/d11/fcb 0 2024-03-24T10:46:00.398 INFO:tasks.workunit.client.0.smithi094.stdout:3/691: rename d0/db/dc3/ccd to d0/db/dc/d28/d3c/d7c/de3/cf2 0 2024-03-24T10:46:00.400 INFO:tasks.workunit.client.0.smithi094.stdout:7/725: link d1/d9/d16/d21/d51/d83/d39/d44/dd9/d34/c4d d1/d6c/d80/d81/da2/cea 0 2024-03-24T10:46:00.400 INFO:tasks.workunit.client.0.smithi094.stdout:7/726: fdatasync d1/d9/fe 0 2024-03-24T10:46:00.400 INFO:tasks.workunit.client.0.smithi094.stdout:7/727: write d1/d9/fd [198404,112058] 0 2024-03-24T10:46:00.402 INFO:tasks.workunit.client.0.smithi094.stdout:2/678: rename d18/d48/d56/d73 to d18/d48/d56/dcc 0 2024-03-24T10:46:00.402 INFO:tasks.workunit.client.0.smithi094.stdout:2/679: write d18/d1e/f25 [3665884,127283] 0 2024-03-24T10:46:00.403 INFO:tasks.workunit.client.0.smithi094.stdout:6/735: symlink d0/d30/d50/d4d/lea 0 2024-03-24T10:46:00.403 INFO:tasks.workunit.client.0.smithi094.stdout:6/736: read d0/f49 [777199,79590] 0 2024-03-24T10:46:00.405 INFO:tasks.workunit.client.0.smithi094.stdout:4/763: mkdir d0/d22/de4 0 2024-03-24T10:46:00.411 INFO:tasks.workunit.client.0.smithi094.stdout:0/727: mkdir d14/d22/d32/dfd 0 2024-03-24T10:46:00.414 INFO:tasks.workunit.client.0.smithi094.stdout:9/814: creat df/d2d/d1f/d58/d5a/d50/da5/d101/f108 x:0 0 0 2024-03-24T10:46:00.419 INFO:tasks.workunit.client.0.smithi094.stdout:7/728: dwrite d1/d9/d25/d3d/fd6 [0,4194304] 0 2024-03-24T10:46:00.419 INFO:tasks.workunit.client.0.smithi094.stdout:7/729: truncate d1/d9/d16/d21/d49/da9/fb8 979222 0 2024-03-24T10:46:00.426 INFO:tasks.workunit.client.0.smithi094.stdout:9/815: write df/f3f [4905454,7913] 0 2024-03-24T10:46:00.426 INFO:tasks.workunit.client.0.smithi094.stdout:3/692: symlink d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d34/d71/lf3 0 2024-03-24T10:46:00.433 INFO:tasks.workunit.client.0.smithi094.stdout:6/737: creat d0/d30/d50/d4d/dd2/feb x:0 0 0 2024-03-24T10:46:00.433 INFO:tasks.workunit.client.0.smithi094.stdout:7/730: dwrite d1/d9/fd1 [0,4194304] 0 2024-03-24T10:46:00.434 INFO:tasks.workunit.client.0.smithi094.stdout:7/731: write d1/f5d [1802415,125274] 0 2024-03-24T10:46:00.434 INFO:tasks.workunit.client.0.smithi094.stdout:7/732: read - d1/d9/d16/d21/d51/d83/d39/d44/dd9/d8f/f91 zero size 2024-03-24T10:46:00.435 INFO:tasks.workunit.client.0.smithi094.stdout:4/764: mknod d0/d5/ce5 0 2024-03-24T10:46:00.436 INFO:tasks.workunit.client.0.smithi094.stdout:0/728: symlink d14/d22/d28/d48/da5/lfe 0 2024-03-24T10:46:00.452 INFO:tasks.workunit.client.0.smithi094.stdout:5/746: sync 2024-03-24T10:46:00.465 INFO:tasks.workunit.client.0.smithi094.stdout:2/680: rmdir d18 39 2024-03-24T10:46:00.465 INFO:tasks.workunit.client.0.smithi094.stdout:2/681: chown d18/d66/d90/d9a/dbc/c42 15401979 1 2024-03-24T10:46:00.465 INFO:tasks.workunit.client.0.smithi094.stdout:2/682: write d18/f4e [717972,109309] 0 2024-03-24T10:46:00.467 INFO:tasks.workunit.client.0.smithi094.stdout:6/738: mknod d0/d30/d50/d4d/dd2/cec 0 2024-03-24T10:46:00.468 INFO:tasks.workunit.client.0.smithi094.stdout:8/797: sync 2024-03-24T10:46:00.468 INFO:tasks.workunit.client.0.smithi094.stdout:1/606: sync 2024-03-24T10:46:00.468 INFO:tasks.workunit.client.0.smithi094.stdout:1/607: creat d4/d9/db/d5d/d63/fd3 x:0 0 0 2024-03-24T10:46:00.469 INFO:tasks.workunit.client.0.smithi094.stdout:1/608: fdatasync d4/d9/db/d31/d3c/f50 0 2024-03-24T10:46:00.469 INFO:tasks.workunit.client.0.smithi094.stdout:1/609: stat d4/d36/f44 0 2024-03-24T10:46:00.469 INFO:tasks.workunit.client.0.smithi094.stdout:1/610: write d4/f1e [1457527,44578] 0 2024-03-24T10:46:00.469 INFO:tasks.workunit.client.0.smithi094.stdout:1/611: chown d4/d9/db/l34 371 1 2024-03-24T10:46:00.469 INFO:tasks.workunit.client.0.smithi094.stdout:1/612: write d4/d9/de/d6f/d62/fcc [725042,100202] 0 2024-03-24T10:46:00.475 INFO:tasks.workunit.client.0.smithi094.stdout:7/733: unlink d1/d9/d16/c95 0 2024-03-24T10:46:00.476 INFO:tasks.workunit.client.0.smithi094.stdout:2/683: dwrite d18/d66/d90/d9a/d74/f99 [0,4194304] 0 2024-03-24T10:46:00.477 INFO:tasks.workunit.client.0.smithi094.stdout:2/684: chown d18/d54/fa7 2014 1 2024-03-24T10:46:00.477 INFO:tasks.workunit.client.0.smithi094.stdout:2/685: fdatasync d18/d66/dbe/dc0/f6c 0 2024-03-24T10:46:00.481 INFO:tasks.workunit.client.0.smithi094.stdout:4/765: mkdir d0/d22/de/dc8/de6 0 2024-03-24T10:46:00.482 INFO:tasks.workunit.client.0.smithi094.stdout:0/729: symlink d14/d22/d28/d2b/d68/lff 0 2024-03-24T10:46:00.484 INFO:tasks.workunit.client.0.smithi094.stdout:8/798: dwrite df/d12/d5b/d63/d75/dd9/fe8 [0,4194304] 0 2024-03-24T10:46:00.484 INFO:tasks.workunit.client.0.smithi094.stdout:6/739: mknod d0/d1a/d35/d96/ced 0 2024-03-24T10:46:00.488 INFO:tasks.workunit.client.0.smithi094.stdout:1/613: rename d4/d9/db/d5d/fa3 to d4/d9/d37/d33/d58/fd4 0 2024-03-24T10:46:00.488 INFO:tasks.workunit.client.0.smithi094.stdout:1/614: write d4/d36/f70 [2179234,78745] 0 2024-03-24T10:46:00.491 INFO:tasks.workunit.client.0.smithi094.stdout:7/734: creat d1/feb x:0 0 0 2024-03-24T10:46:00.491 INFO:tasks.workunit.client.0.smithi094.stdout:7/735: creat d1/fec x:0 0 0 2024-03-24T10:46:00.493 INFO:tasks.workunit.client.0.smithi094.stdout:4/766: dwrite d0/d5/d1b/f2a [0,4194304] 0 2024-03-24T10:46:00.498 INFO:tasks.workunit.client.0.smithi094.stdout:2/686: dread d18/d66/d90/d9a/dbc/f49 [0,4194304] 0 2024-03-24T10:46:00.498 INFO:tasks.workunit.client.0.smithi094.stdout:2/687: write d18/d54/fa7 [401564,65004] 0 2024-03-24T10:46:00.498 INFO:tasks.workunit.client.0.smithi094.stdout:2/688: write d18/d5f/fad [1155787,6776] 0 2024-03-24T10:46:00.499 INFO:tasks.workunit.client.0.smithi094.stdout:0/730: write d14/d18/f4f [2778222,50172] 0 2024-03-24T10:46:00.499 INFO:tasks.workunit.client.0.smithi094.stdout:0/731: chown d14/d22/d28/d48/da5/lfe 3835189 1 2024-03-24T10:46:00.500 INFO:tasks.workunit.client.0.smithi094.stdout:6/740: write d0/d78/da/d23/dc3/fbe [779345,17620] 0 2024-03-24T10:46:00.500 INFO:tasks.workunit.client.0.smithi094.stdout:8/799: mkdir df/d12/d5b/d63/d75/d10e 0 2024-03-24T10:46:00.503 INFO:tasks.workunit.client.0.smithi094.stdout:1/615: symlink d4/d9/d37/d33/d58/ld5 0 2024-03-24T10:46:00.506 INFO:tasks.workunit.client.0.smithi094.stdout:7/736: creat d1/d9/dc3/fed x:0 0 0 2024-03-24T10:46:00.512 INFO:tasks.workunit.client.0.smithi094.stdout:2/689: mknod d18/d66/d90/d9a/d74/d7e/dca/ccd 0 2024-03-24T10:46:00.514 INFO:tasks.workunit.client.0.smithi094.stdout:0/732: dwrite d14/d22/d28/da2/fb9 [0,4194304] 0 2024-03-24T10:46:00.515 INFO:tasks.workunit.client.0.smithi094.stdout:0/733: write d14/d22/d28/d2b/f45 [1651495,88660] 0 2024-03-24T10:46:00.515 INFO:tasks.workunit.client.0.smithi094.stdout:0/734: write fd [68738,10583] 0 2024-03-24T10:46:00.518 INFO:tasks.workunit.client.0.smithi094.stdout:6/741: creat d0/d78/d66/fee x:0 0 0 2024-03-24T10:46:00.518 INFO:tasks.workunit.client.0.smithi094.stdout:6/742: chown d0/d78/da/d23/d7c/db0/fc6 1 1 2024-03-24T10:46:00.518 INFO:tasks.workunit.client.0.smithi094.stdout:6/743: chown d0/d21/d2a/cad 2722414 1 2024-03-24T10:46:00.518 INFO:tasks.workunit.client.0.smithi094.stdout:6/744: fsync d0/d78/da/d23/f6c 0 2024-03-24T10:46:00.519 INFO:tasks.workunit.client.0.smithi094.stdout:2/690: dread d18/d48/d56/fb0 [0,4194304] 0 2024-03-24T10:46:00.519 INFO:tasks.workunit.client.0.smithi094.stdout:8/800: mknod df/d12/d5b/dea/c10f 0 2024-03-24T10:46:00.522 INFO:tasks.workunit.client.0.smithi094.stdout:1/616: unlink d4/d9/de/l23 0 2024-03-24T10:46:00.525 INFO:tasks.workunit.client.0.smithi094.stdout:7/737: link d1/d9/d16/fc1 d1/d6c/fee 0 2024-03-24T10:46:00.528 INFO:tasks.workunit.client.0.smithi094.stdout:4/767: link d0/d34/db2/c32 d0/d34/d73/d94/ce7 0 2024-03-24T10:46:00.528 INFO:tasks.workunit.client.0.smithi094.stdout:4/768: write d0/d22/de/d28/d3d/f5c [4012953,130639] 0 2024-03-24T10:46:00.534 INFO:tasks.workunit.client.0.smithi094.stdout:7/738: write d1/d9/d25/f50 [2750696,23597] 0 2024-03-24T10:46:00.535 INFO:tasks.workunit.client.0.smithi094.stdout:8/801: dwrite df/d12/f2f [0,4194304] 0 2024-03-24T10:46:00.536 INFO:tasks.workunit.client.0.smithi094.stdout:1/617: read d4/d9/f2c [661479,24011] 0 2024-03-24T10:46:00.536 INFO:tasks.workunit.client.0.smithi094.stdout:0/735: dwrite d14/d22/d28/fe1 [0,4194304] 0 2024-03-24T10:46:00.536 INFO:tasks.workunit.client.0.smithi094.stdout:1/618: read d4/d9/f2c [522145,27309] 0 2024-03-24T10:46:00.536 INFO:tasks.workunit.client.0.smithi094.stdout:0/736: truncate d14/d22/d5b/d86/fe8 932256 0 2024-03-24T10:46:00.536 INFO:tasks.workunit.client.0.smithi094.stdout:0/737: readlink d14/d22/d28/d2b/l6d 0 2024-03-24T10:46:00.536 INFO:tasks.workunit.client.0.smithi094.stdout:0/738: chown d14/d22/f2f 101005 1 2024-03-24T10:46:00.547 INFO:tasks.workunit.client.0.smithi094.stdout:6/745: creat d0/d21/d2a/dcb/fef x:0 0 0 2024-03-24T10:46:00.549 INFO:tasks.workunit.client.0.smithi094.stdout:4/769: dwrite d0/d22/de/f89 [0,4194304] 0 2024-03-24T10:46:00.550 INFO:tasks.workunit.client.0.smithi094.stdout:2/691: truncate d18/d54/f5c 6114478 0 2024-03-24T10:46:00.550 INFO:tasks.workunit.client.0.smithi094.stdout:2/692: dread - d18/d54/db9/fba zero size 2024-03-24T10:46:00.550 INFO:tasks.workunit.client.0.smithi094.stdout:2/693: write d18/d66/dbe/dc0/fb6 [494153,66824] 0 2024-03-24T10:46:00.550 INFO:tasks.workunit.client.0.smithi094.stdout:2/694: write d18/d1e/f76 [857026,59464] 0 2024-03-24T10:46:00.550 INFO:tasks.workunit.client.0.smithi094.stdout:2/695: write d18/d48/d56/dcc/f7a [646856,114366] 0 2024-03-24T10:46:00.551 INFO:tasks.workunit.client.0.smithi094.stdout:2/696: stat d18/d66/d90/d9a/d74/d7e 0 2024-03-24T10:46:00.552 INFO:tasks.workunit.client.0.smithi094.stdout:7/739: symlink d1/d9/d16/d21/d51/d83/d39/d44/lef 0 2024-03-24T10:46:00.558 INFO:tasks.workunit.client.0.smithi094.stdout:1/619: dwrite d4/d9/db/d39/f49 [4194304,4194304] 0 2024-03-24T10:46:00.558 INFO:tasks.workunit.client.0.smithi094.stdout:8/802: dwrite df/d25/f61 [0,4194304] 0 2024-03-24T10:46:00.566 INFO:tasks.workunit.client.0.smithi094.stdout:2/697: dwrite d18/d66/d90/d9a/dbc/f46 [0,4194304] 0 2024-03-24T10:46:00.572 INFO:tasks.workunit.client.0.smithi094.stdout:2/698: dread d18/d66/dbe/dc0/f6c [0,4194304] 0 2024-03-24T10:46:00.572 INFO:tasks.workunit.client.0.smithi094.stdout:2/699: chown d18/d66/d90 122151 1 2024-03-24T10:46:00.572 INFO:tasks.workunit.client.0.smithi094.stdout:0/739: symlink d14/l100 0 2024-03-24T10:46:00.574 INFO:tasks.workunit.client.0.smithi094.stdout:6/746: truncate d0/d78/f44 667576 0 2024-03-24T10:46:00.581 INFO:tasks.workunit.client.0.smithi094.stdout:4/770: mkdir d0/d5/d1b/de8 0 2024-03-24T10:46:00.583 INFO:tasks.workunit.client.0.smithi094.stdout:7/740: creat d1/d9/d25/dd4/ff0 x:0 0 0 2024-03-24T10:46:00.586 INFO:tasks.workunit.client.0.smithi094.stdout:1/620: rmdir d4/d9/de/d6f/d62/d99/db5 0 2024-03-24T10:46:00.589 INFO:tasks.workunit.client.0.smithi094.stdout:6/747: dwrite d0/d78/da/f20 [4194304,4194304] 0 2024-03-24T10:46:00.589 INFO:tasks.workunit.client.0.smithi094.stdout:0/740: dwrite d14/f62 [0,4194304] 0 2024-03-24T10:46:00.599 INFO:tasks.workunit.client.0.smithi094.stdout:8/803: mknod df/d25/c110 0 2024-03-24T10:46:00.602 INFO:tasks.workunit.client.0.smithi094.stdout:9/816: sync 2024-03-24T10:46:00.602 INFO:tasks.workunit.client.0.smithi094.stdout:3/693: sync 2024-03-24T10:46:00.602 INFO:tasks.workunit.client.0.smithi094.stdout:3/694: write d0/db/dc/d28/d3c/d55/d7f/fe1 [856602,30090] 0 2024-03-24T10:46:00.603 INFO:tasks.workunit.client.0.smithi094.stdout:2/700: mkdir d18/d66/d90/d9a/d74/d7e/dce 0 2024-03-24T10:46:00.605 INFO:tasks.workunit.client.0.smithi094.stdout:4/771: mknod d0/ce9 0 2024-03-24T10:46:00.607 INFO:tasks.workunit.client.0.smithi094.stdout:7/741: link d1/d6c/c70 d1/d9/d25/de2/cf1 0 2024-03-24T10:46:00.609 INFO:tasks.workunit.client.0.smithi094.stdout:8/804: dread f4 [0,4194304] 0 2024-03-24T10:46:00.610 INFO:tasks.workunit.client.0.smithi094.stdout:1/621: rename d4/d9/cb3 to d4/d9/de/d3d/dae/cd6 0 2024-03-24T10:46:00.610 INFO:tasks.workunit.client.0.smithi094.stdout:4/772: dread d0/d5/d1b/d4f/d70/f7c [0,4194304] 0 2024-03-24T10:46:00.610 INFO:tasks.workunit.client.0.smithi094.stdout:4/773: dread - d0/d22/de/d28/d2b/d33/da1/fda zero size 2024-03-24T10:46:00.610 INFO:tasks.workunit.client.0.smithi094.stdout:4/774: dread - d0/d5/d1b/faf zero size 2024-03-24T10:46:00.610 INFO:tasks.workunit.client.0.smithi094.stdout:4/775: chown d0/d22/l15 935 1 2024-03-24T10:46:00.614 INFO:tasks.workunit.client.0.smithi094.stdout:6/748: mkdir d0/d78/da/d23/d34/d4e/d5d/df0 0 2024-03-24T10:46:00.614 INFO:tasks.workunit.client.0.smithi094.stdout:6/749: dread - d0/d1a/fe8 zero size 2024-03-24T10:46:00.614 INFO:tasks.workunit.client.0.smithi094.stdout:6/750: write d0/d78/da/d77/fa3 [756170,139] 0 2024-03-24T10:46:00.619 INFO:tasks.workunit.client.0.smithi094.stdout:9/817: symlink df/d2d/d1f/d11/d43/l109 0 2024-03-24T10:46:00.620 INFO:tasks.workunit.client.0.smithi094.stdout:9/818: dread - df/d2d/d1f/d11/d53/d88/fdd zero size 2024-03-24T10:46:00.621 INFO:tasks.workunit.client.0.smithi094.stdout:3/695: creat d0/db/dc3/ff4 x:0 0 0 2024-03-24T10:46:00.624 INFO:tasks.workunit.client.0.smithi094.stdout:2/701: getdents d18/d7f/d88 0 2024-03-24T10:46:00.627 INFO:tasks.workunit.client.0.smithi094.stdout:8/805: dwrite df/d25/d36/d3f/f92 [0,4194304] 0 2024-03-24T10:46:00.627 INFO:tasks.workunit.client.0.smithi094.stdout:8/806: fsync df/d25/f44 0 2024-03-24T10:46:00.629 INFO:tasks.workunit.client.0.smithi094.stdout:7/742: creat d1/d9/d16/d21/d51/dc4/ff2 x:0 0 0 2024-03-24T10:46:00.630 INFO:tasks.workunit.client.0.smithi094.stdout:6/751: dwrite d0/d78/f19 [0,4194304] 0 2024-03-24T10:46:00.633 INFO:tasks.workunit.client.0.smithi094.stdout:0/741: rename d14/d22/d5b/d86/lca to d14/d22/d28/d2b/dc0/l101 0 2024-03-24T10:46:00.633 INFO:tasks.workunit.client.0.smithi094.stdout:0/742: readlink d14/d22/d28/d48/l71 0 2024-03-24T10:46:00.633 INFO:tasks.workunit.client.0.smithi094.stdout:9/819: dwrite df/d7b/ffb [0,4194304] 0 2024-03-24T10:46:00.634 INFO:tasks.workunit.client.0.smithi094.stdout:5/747: sync 2024-03-24T10:46:00.635 INFO:tasks.workunit.client.0.smithi094.stdout:5/748: getdents d7/dc/db9 0 2024-03-24T10:46:00.636 INFO:tasks.workunit.client.0.smithi094.stdout:6/752: dread d0/d21/d2a/d2f/fa6 [0,4194304] 0 2024-03-24T10:46:00.636 INFO:tasks.workunit.client.0.smithi094.stdout:6/753: fdatasync d0/d78/da/d23/f68 0 2024-03-24T10:46:00.636 INFO:tasks.workunit.client.0.smithi094.stdout:6/754: readlink d0/d1a/l43 0 2024-03-24T10:46:00.636 INFO:tasks.workunit.client.0.smithi094.stdout:6/755: chown d0/d21/d2a/d55/dce/fd4 0 1 2024-03-24T10:46:00.636 INFO:tasks.workunit.client.0.smithi094.stdout:1/622: link d4/d9/d37/d33/f51 d4/d9/db/d31/d3c/fd7 0 2024-03-24T10:46:00.639 INFO:tasks.workunit.client.0.smithi094.stdout:4/776: unlink d0/d22/ca8 0 2024-03-24T10:46:00.642 INFO:tasks.workunit.client.0.smithi094.stdout:9/820: write df/d2d/f8d [4029757,62477] 0 2024-03-24T10:46:00.642 INFO:tasks.workunit.client.0.smithi094.stdout:9/821: truncate df/d2d/d1f/d58/d70/dab/fc9 108018 0 2024-03-24T10:46:00.642 INFO:tasks.workunit.client.0.smithi094.stdout:9/822: chown df/d2d/d34/d2c/d74 164 1 2024-03-24T10:46:00.642 INFO:tasks.workunit.client.0.smithi094.stdout:2/702: mkdir d18/d77/dcf 0 2024-03-24T10:46:00.642 INFO:tasks.workunit.client.0.smithi094.stdout:3/696: dwrite d0/db/dc/d28/d50/d5e/fb7 [0,4194304] 0 2024-03-24T10:46:00.642 INFO:tasks.workunit.client.0.smithi094.stdout:8/807: read df/d25/d28/d31/d3c/f3e [2378153,121171] 0 2024-03-24T10:46:00.652 INFO:tasks.workunit.client.0.smithi094.stdout:7/743: mknod d1/d9/d16/d21/d51/d83/d39/d44/dd9/d8f/cf3 0 2024-03-24T10:46:00.652 INFO:tasks.workunit.client.0.smithi094.stdout:7/744: rename d1/d9/d16 to d1/d9/d16/d21/d51/d83/d9d/dd7/df4 22 2024-03-24T10:46:00.658 INFO:tasks.workunit.client.0.smithi094.stdout:4/777: read d0/d22/ff [1251276,6579] 0 2024-03-24T10:46:00.658 INFO:tasks.workunit.client.0.smithi094.stdout:4/778: creat d0/d22/de/d28/d2b/d33/da1/fea x:0 0 0 2024-03-24T10:46:00.659 INFO:tasks.workunit.client.0.smithi094.stdout:0/743: mkdir d14/d22/d28/d48/d7b/dfa/d102 0 2024-03-24T10:46:00.659 INFO:tasks.workunit.client.0.smithi094.stdout:0/744: chown d14/d22/d5b/ldc 24011632 1 2024-03-24T10:46:00.660 INFO:tasks.workunit.client.0.smithi094.stdout:5/749: dwrite d7/d23/dd5/d64/dd6/f39 [0,4194304] 0 2024-03-24T10:46:00.663 INFO:tasks.workunit.client.0.smithi094.stdout:2/703: dwrite d18/d66/d90/d9a/dbc/f50 [0,4194304] 0 2024-03-24T10:46:00.665 INFO:tasks.workunit.client.0.smithi094.stdout:6/756: link d0/d78/da/d77/fa3 d0/d78/da/d23/d7c/db0/ff1 0 2024-03-24T10:46:00.665 INFO:tasks.workunit.client.0.smithi094.stdout:6/757: dread - d0/d1a/fd7 zero size 2024-03-24T10:46:00.666 INFO:tasks.workunit.client.0.smithi094.stdout:6/758: truncate d0/d30/d50/fac 84836 0 2024-03-24T10:46:00.667 INFO:tasks.workunit.client.0.smithi094.stdout:1/623: symlink d4/d9/de/d6f/d62/ld8 0 2024-03-24T10:46:00.672 INFO:tasks.workunit.client.0.smithi094.stdout:3/697: rename d0/db/cc1 to d0/db/cf5 0 2024-03-24T10:46:00.674 INFO:tasks.workunit.client.0.smithi094.stdout:8/808: mkdir df/d8c/d8f/dc1/d111 0 2024-03-24T10:46:00.674 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:00 smithi096 ceph-mon[38097]: pgmap v243: 65 pgs: 65 active+clean; 1.0 GiB data, 4.8 GiB used, 532 GiB / 536 GiB avail; 102 MiB/s rd, 126 MiB/s wr, 600 op/s 2024-03-24T10:46:00.678 INFO:tasks.workunit.client.0.smithi094.stdout:7/745: mknod d1/d9/d16/d21/d51/d83/d39/d44/dd9/d34/cf5 0 2024-03-24T10:46:00.678 INFO:tasks.workunit.client.0.smithi094.stdout:7/746: write d1/d9/d16/d21/d51/d83/d39/d44/dd9/f4c [869607,56421] 0 2024-03-24T10:46:00.687 INFO:tasks.workunit.client.0.smithi094.stdout:4/779: truncate d0/d22/f76 4568926 0 2024-03-24T10:46:00.690 INFO:tasks.workunit.client.0.smithi094.stdout:0/745: stat d14/d22/d28/f47 0 2024-03-24T10:46:00.691 INFO:tasks.workunit.client.0.smithi094.stdout:7/747: dwrite d1/d6c/d80/f69 [4194304,4194304] 0 2024-03-24T10:46:00.697 INFO:tasks.workunit.client.0.smithi094.stdout:5/750: creat d7/d23/d5f/d9/d48/d73/ff6 x:0 0 0 2024-03-24T10:46:00.703 INFO:tasks.workunit.client.0.smithi094.stdout:4/780: dwrite d0/d5/fab [0,4194304] 0 2024-03-24T10:46:00.703 INFO:tasks.workunit.client.0.smithi094.stdout:4/781: fdatasync d0/d22/de/d28/d3d/d8c/fc7 0 2024-03-24T10:46:00.703 INFO:tasks.workunit.client.0.smithi094.stdout:4/782: fsync d0/d5/d7d/d97/d4a/f61 0 2024-03-24T10:46:00.705 INFO:tasks.workunit.client.0.smithi094.stdout:2/704: creat d18/d66/dbe/fd0 x:0 0 0 2024-03-24T10:46:00.707 INFO:tasks.workunit.client.0.smithi094.stdout:6/759: creat d0/d30/de6/ff2 x:0 0 0 2024-03-24T10:46:00.709 INFO:tasks.workunit.client.0.smithi094.stdout:1/624: mknod d4/d9/cd9 0 2024-03-24T10:46:00.714 INFO:tasks.workunit.client.0.smithi094.stdout:3/698: mknod d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/d73/cf6 0 2024-03-24T10:46:00.715 INFO:tasks.workunit.client.0.smithi094.stdout:3/699: fsync d0/fb6 0 2024-03-24T10:46:00.715 INFO:tasks.workunit.client.0.smithi094.stdout:3/700: chown d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/db5/lb8 2068308 1 2024-03-24T10:46:00.717 INFO:tasks.workunit.client.0.smithi094.stdout:8/809: symlink df/d12/d5b/d63/d75/l112 0 2024-03-24T10:46:00.717 INFO:tasks.workunit.client.0.smithi094.stdout:8/810: fsync f4 0 2024-03-24T10:46:00.717 INFO:tasks.workunit.client.0.smithi094.stdout:8/811: chown df/d25/d28/d31/d3c/f3e 13 1 2024-03-24T10:46:00.719 INFO:tasks.workunit.client.0.smithi094.stdout:0/746: mkdir d14/d22/d103 0 2024-03-24T10:46:00.723 INFO:tasks.workunit.client.0.smithi094.stdout:8/812: read df/d25/d28/d31/d47/f85 [2648682,113202] 0 2024-03-24T10:46:00.723 INFO:tasks.workunit.client.0.smithi094.stdout:2/705: dwrite d18/d66/fa5 [0,4194304] 0 2024-03-24T10:46:00.723 INFO:tasks.workunit.client.0.smithi094.stdout:2/706: readlink d18/d48/d56/l80 0 2024-03-24T10:46:00.724 INFO:tasks.workunit.client.0.smithi094.stdout:7/748: mknod d1/d9b/cf6 0 2024-03-24T10:46:00.726 INFO:tasks.workunit.client.0.smithi094.stdout:5/751: getdents d7/d23/dd5 0 2024-03-24T10:46:00.729 INFO:tasks.workunit.client.0.smithi094.stdout:3/701: dwrite d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/d73/f80 [0,4194304] 0 2024-03-24T10:46:00.730 INFO:tasks.workunit.client.0.smithi094.stdout:8/813: dread df/d25/d28/d31/d47/d22/fc5 [0,4194304] 0 2024-03-24T10:46:00.730 INFO:tasks.workunit.client.0.smithi094.stdout:8/814: chown df/d25/d28/cc0 379984 1 2024-03-24T10:46:00.731 INFO:tasks.workunit.client.0.smithi094.stdout:2/707: dread d18/d1e/f53 [0,4194304] 0 2024-03-24T10:46:00.732 INFO:tasks.workunit.client.0.smithi094.stdout:4/783: rename d0/d22/c1a to d0/d34/d73/d94/ceb 0 2024-03-24T10:46:00.735 INFO:tasks.workunit.client.0.smithi094.stdout:6/760: mkdir d0/d21/d2a/df3 0 2024-03-24T10:46:00.737 INFO:tasks.workunit.client.0.smithi094.stdout:1/625: creat d4/d9/db/d31/d3c/fda x:0 0 0 2024-03-24T10:46:00.737 INFO:tasks.workunit.client.0.smithi094.stdout:1/626: truncate d4/d9/db/d5d/d63/fd3 609786 0 2024-03-24T10:46:00.738 INFO:tasks.workunit.client.0.smithi094.stdout:1/627: fsync d4/f7 0 2024-03-24T10:46:00.741 INFO:tasks.workunit.client.0.smithi094.stdout:4/784: read d0/d34/f82 [4167782,1672] 0 2024-03-24T10:46:00.744 INFO:tasks.workunit.client.0.smithi094.stdout:0/747: symlink d14/d22/d28/dac/ddf/l104 0 2024-03-24T10:46:00.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:00 smithi094 ceph-mon[29446]: pgmap v243: 65 pgs: 65 active+clean; 1.0 GiB data, 4.8 GiB used, 532 GiB / 536 GiB avail; 102 MiB/s rd, 126 MiB/s wr, 600 op/s 2024-03-24T10:46:00.750 INFO:tasks.workunit.client.0.smithi094.stdout:7/749: symlink d1/d9/d16/d21/d49/da9/lf7 0 2024-03-24T10:46:00.751 INFO:tasks.workunit.client.0.smithi094.stdout:5/752: mkdir d7/d23/d5f/d9/dde/df7 0 2024-03-24T10:46:00.755 INFO:tasks.workunit.client.0.smithi094.stdout:3/702: mknod d0/db/dc/d28/d3c/d7c/de8/cf7 0 2024-03-24T10:46:00.756 INFO:tasks.workunit.client.0.smithi094.stdout:2/708: rmdir d18/d48/d56/dcc 39 2024-03-24T10:46:00.760 INFO:tasks.workunit.client.0.smithi094.stdout:6/761: truncate d0/d1a/d35/f72 850905 0 2024-03-24T10:46:00.764 INFO:tasks.workunit.client.0.smithi094.stdout:1/628: getdents d4/d9/db/d39/d75 0 2024-03-24T10:46:00.764 INFO:tasks.workunit.client.0.smithi094.stdout:1/629: getdents d4/d9/de/d6f/d62/d80 0 2024-03-24T10:46:00.766 INFO:tasks.workunit.client.0.smithi094.stdout:2/709: dread d18/d54/f5c [0,4194304] 0 2024-03-24T10:46:00.766 INFO:tasks.workunit.client.0.smithi094.stdout:2/710: read d18/f23 [3663554,102170] 0 2024-03-24T10:46:00.767 INFO:tasks.workunit.client.0.smithi094.stdout:4/785: mkdir d0/d22/de/d28/d3d/d8c/dec 0 2024-03-24T10:46:00.770 INFO:tasks.workunit.client.0.smithi094.stdout:0/748: mkdir d14/d22/d28/d2b/d41/d53/d105 0 2024-03-24T10:46:00.775 INFO:tasks.workunit.client.0.smithi094.stdout:7/750: unlink d1/db4/d61/l97 0 2024-03-24T10:46:00.775 INFO:tasks.workunit.client.0.smithi094.stdout:7/751: fdatasync f0 0 2024-03-24T10:46:00.776 INFO:tasks.workunit.client.0.smithi094.stdout:7/752: write d1/feb [280054,128123] 0 2024-03-24T10:46:00.777 INFO:tasks.workunit.client.0.smithi094.stdout:6/762: dwrite d0/d30/d50/fbd [0,4194304] 0 2024-03-24T10:46:00.779 INFO:tasks.workunit.client.0.smithi094.stdout:2/711: dread d18/f45 [0,4194304] 0 2024-03-24T10:46:00.779 INFO:tasks.workunit.client.0.smithi094.stdout:2/712: fdatasync d18/d54/f5c 0 2024-03-24T10:46:00.779 INFO:tasks.workunit.client.0.smithi094.stdout:5/753: rename d7/d23/d5f/d14/d9f/lea to d7/d23/d5f/d14/d36/d3a/lf8 0 2024-03-24T10:46:00.781 INFO:tasks.workunit.client.0.smithi094.stdout:6/763: dread d0/d78/da/d23/dc3/f6a [0,4194304] 0 2024-03-24T10:46:00.782 INFO:tasks.workunit.client.0.smithi094.stdout:3/703: creat d0/db/dc/d28/d38/d6a/ff8 x:0 0 0 2024-03-24T10:46:00.782 INFO:tasks.workunit.client.0.smithi094.stdout:3/704: write d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/ff1 [845753,28636] 0 2024-03-24T10:46:00.785 INFO:tasks.workunit.client.0.smithi094.stdout:4/786: getdents d0/d34/db2 0 2024-03-24T10:46:00.788 INFO:tasks.workunit.client.0.smithi094.stdout:0/749: creat d14/d22/d28/dac/ddd/f106 x:0 0 0 2024-03-24T10:46:00.788 INFO:tasks.workunit.client.0.smithi094.stdout:0/750: write d14/d18/fc2 [763562,31137] 0 2024-03-24T10:46:00.798 INFO:tasks.workunit.client.0.smithi094.stdout:7/753: rename f0 to d1/d6c/d80/ff8 0 2024-03-24T10:46:00.799 INFO:tasks.workunit.client.0.smithi094.stdout:6/764: read d0/d21/f2b [6658884,45320] 0 2024-03-24T10:46:00.800 INFO:tasks.workunit.client.0.smithi094.stdout:6/765: write d0/f2 [1419458,12593] 0 2024-03-24T10:46:00.800 INFO:tasks.workunit.client.0.smithi094.stdout:6/766: creat d0/d78/da/d23/d7c/db0/ff4 x:0 0 0 2024-03-24T10:46:00.800 INFO:tasks.workunit.client.0.smithi094.stdout:6/767: chown d0/d21/d2a/c88 165 1 2024-03-24T10:46:00.802 INFO:tasks.workunit.client.0.smithi094.stdout:3/705: unlink d0/db/dc/d28/d38/d53/cd2 0 2024-03-24T10:46:00.802 INFO:tasks.workunit.client.0.smithi094.stdout:3/706: write d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/fcf [7501823,27191] 0 2024-03-24T10:46:00.802 INFO:tasks.workunit.client.0.smithi094.stdout:3/707: creat d0/db/dc3/ff9 x:0 0 0 2024-03-24T10:46:00.803 INFO:tasks.workunit.client.0.smithi094.stdout:1/630: dwrite d4/d9/f2d [4194304,4194304] 0 2024-03-24T10:46:00.804 INFO:tasks.workunit.client.0.smithi094.stdout:4/787: mknod d0/d22/de/dc8/de6/ced 0 2024-03-24T10:46:00.806 INFO:tasks.workunit.client.0.smithi094.stdout:2/713: dwrite d18/d66/d90/d9a/dbc/f50 [0,4194304] 0 2024-03-24T10:46:00.806 INFO:tasks.workunit.client.0.smithi094.stdout:5/754: dwrite d7/d23/d5f/d14/d9f/fbc [0,4194304] 0 2024-03-24T10:46:00.807 INFO:tasks.workunit.client.0.smithi094.stdout:0/751: mkdir d14/d22/d28/dac/ddf/d107 0 2024-03-24T10:46:00.807 INFO:tasks.workunit.client.0.smithi094.stdout:0/752: read d14/d22/d32/d70/ff5 [373591,44820] 0 2024-03-24T10:46:00.809 INFO:tasks.workunit.client.0.smithi094.stdout:7/754: mknod d1/d9/d25/de2/cf9 0 2024-03-24T10:46:00.810 INFO:tasks.workunit.client.0.smithi094.stdout:7/755: chown d1/d9/d16/d21/d51/d83/d39/d44/dd9/d8f/f91 87 1 2024-03-24T10:46:00.811 INFO:tasks.workunit.client.0.smithi094.stdout:3/708: dread d0/db/dc/d28/d3c/d55/fe7 [0,4194304] 0 2024-03-24T10:46:00.814 INFO:tasks.workunit.client.0.smithi094.stdout:6/768: symlink d0/d78/da/d23/d34/lf5 0 2024-03-24T10:46:00.816 INFO:tasks.workunit.client.0.smithi094.stdout:0/753: write d14/d22/d32/d70/fc4 [1670493,34764] 0 2024-03-24T10:46:00.818 INFO:tasks.workunit.client.0.smithi094.stdout:1/631: truncate d4/d9/db/d31/d3c/fd7 2645940 0 2024-03-24T10:46:00.818 INFO:tasks.workunit.client.0.smithi094.stdout:1/632: chown d4/d9/db/d5d 870166300 1 2024-03-24T10:46:00.819 INFO:tasks.workunit.client.0.smithi094.stdout:5/755: dwrite d7/d23/d5f/d14/f7c [4194304,4194304] 0 2024-03-24T10:46:00.819 INFO:tasks.workunit.client.0.smithi094.stdout:5/756: fdatasync d7/d23/d5f/d9/f49 0 2024-03-24T10:46:00.820 INFO:tasks.workunit.client.0.smithi094.stdout:4/788: symlink d0/d5/d1b/d4f/lee 0 2024-03-24T10:46:00.820 INFO:tasks.workunit.client.0.smithi094.stdout:4/789: write d0/f47 [1334471,35371] 0 2024-03-24T10:46:00.821 INFO:tasks.workunit.client.0.smithi094.stdout:4/790: chown d0/d5/d1b/d4f/dd6 29130 1 2024-03-24T10:46:00.829 INFO:tasks.workunit.client.0.smithi094.stdout:2/714: creat d18/d66/d90/d9a/d74/fd1 x:0 0 0 2024-03-24T10:46:00.847 INFO:tasks.workunit.client.0.smithi094.stdout:7/756: mkdir d1/d9/d16/d21/d49/dfa 0 2024-03-24T10:46:00.847 INFO:tasks.workunit.client.0.smithi094.stdout:7/757: creat d1/d9/d16/d21/d51/d83/ffb x:0 0 0 2024-03-24T10:46:00.853 INFO:tasks.workunit.client.0.smithi094.stdout:1/633: creat d4/d9/db/d39/d43/fdb x:0 0 0 2024-03-24T10:46:00.856 INFO:tasks.workunit.client.0.smithi094.stdout:5/757: rmdir d7/d23/d5f/d14/d6f 39 2024-03-24T10:46:00.859 INFO:tasks.workunit.client.0.smithi094.stdout:2/715: symlink d18/d66/d90/d9a/ld2 0 2024-03-24T10:46:00.866 INFO:tasks.workunit.client.0.smithi094.stdout:7/758: truncate d1/d6c/d80/f7e 318504 0 2024-03-24T10:46:00.868 INFO:tasks.workunit.client.0.smithi094.stdout:2/716: dread d18/f23 [0,4194304] 0 2024-03-24T10:46:00.868 INFO:tasks.workunit.client.0.smithi094.stdout:2/717: write d18/f3b [922074,21914] 0 2024-03-24T10:46:00.868 INFO:tasks.workunit.client.0.smithi094.stdout:2/718: dread - d18/d66/dbe/fd0 zero size 2024-03-24T10:46:00.868 INFO:tasks.workunit.client.0.smithi094.stdout:0/754: rename d14/d22/d28/c36 to d14/d22/d28/d2b/c108 0 2024-03-24T10:46:00.870 INFO:tasks.workunit.client.0.smithi094.stdout:1/634: symlink d4/d9/d7d/ldc 0 2024-03-24T10:46:00.872 INFO:tasks.workunit.client.0.smithi094.stdout:7/759: symlink d1/d9/d16/d21/d51/dc4/lfc 0 2024-03-24T10:46:00.875 INFO:tasks.workunit.client.0.smithi094.stdout:2/719: creat d18/d66/d90/d9a/d74/d7e/fd3 x:0 0 0 2024-03-24T10:46:00.878 INFO:tasks.workunit.client.0.smithi094.stdout:0/755: creat d14/d22/d28/d48/da5/f109 x:0 0 0 2024-03-24T10:46:00.881 INFO:tasks.workunit.client.0.smithi094.stdout:1/635: getdents d4/d9/db1 0 2024-03-24T10:46:00.887 INFO:tasks.workunit.client.0.smithi094.stdout:2/720: mkdir d18/d66/d90/d9a/d74/db4/dd4 0 2024-03-24T10:46:00.889 INFO:tasks.workunit.client.0.smithi094.stdout:0/756: mkdir d14/d22/d28/da2/d10a 0 2024-03-24T10:46:00.892 INFO:tasks.workunit.client.0.smithi094.stdout:2/721: creat d18/d1e/d65/fd5 x:0 0 0 2024-03-24T10:46:00.892 INFO:tasks.workunit.client.0.smithi094.stdout:7/760: dwrite d1/d9/f13 [0,4194304] 0 2024-03-24T10:46:00.894 INFO:tasks.workunit.client.0.smithi094.stdout:0/757: mknod d14/d22/c10b 0 2024-03-24T10:46:00.894 INFO:tasks.workunit.client.0.smithi094.stdout:0/758: write d14/d22/d32/d70/fe2 [333767,87365] 0 2024-03-24T10:46:00.895 INFO:tasks.workunit.client.0.smithi094.stdout:9/823: sync 2024-03-24T10:46:00.895 INFO:tasks.workunit.client.0.smithi094.stdout:9/824: readlink df/d2d/d1f/d7e/l9d 0 2024-03-24T10:46:00.895 INFO:tasks.workunit.client.0.smithi094.stdout:0/759: dread d14/d22/d32/d70/ff5 [0,4194304] 0 2024-03-24T10:46:00.895 INFO:tasks.workunit.client.0.smithi094.stdout:0/760: write d14/f1b [1860300,101229] 0 2024-03-24T10:46:00.896 INFO:tasks.workunit.client.0.smithi094.stdout:2/722: link d18/d66/dbe/dc0/f55 d18/d1e/d65/fd6 0 2024-03-24T10:46:00.899 INFO:tasks.workunit.client.0.smithi094.stdout:1/636: dwrite d4/d9/d37/d33/d76/f9b [0,4194304] 0 2024-03-24T10:46:00.901 INFO:tasks.workunit.client.0.smithi094.stdout:9/825: dread df/d2d/d1f/d11/f4f [0,4194304] 0 2024-03-24T10:46:00.901 INFO:tasks.workunit.client.0.smithi094.stdout:9/826: truncate df/d2d/d34/d2c/d97/fcc 989486 0 2024-03-24T10:46:00.905 INFO:tasks.workunit.client.0.smithi094.stdout:7/761: mknod d1/d6c/d80/d81/da2/cfd 0 2024-03-24T10:46:00.911 INFO:tasks.workunit.client.0.smithi094.stdout:0/761: creat d14/d22/d32/d70/dd0/f10c x:0 0 0 2024-03-24T10:46:00.912 INFO:tasks.workunit.client.0.smithi094.stdout:2/723: truncate d18/f19 2695894 0 2024-03-24T10:46:00.916 INFO:tasks.workunit.client.0.smithi094.stdout:9/827: dwrite df/d2d/d1f/d58/d5a/d61/fc0 [0,4194304] 0 2024-03-24T10:46:00.916 INFO:tasks.workunit.client.0.smithi094.stdout:9/828: dread - df/d7b/fb4 zero size 2024-03-24T10:46:00.917 INFO:tasks.workunit.client.0.smithi094.stdout:1/637: rename d4/d9/cd9 to d4/cdd 0 2024-03-24T10:46:00.919 INFO:tasks.workunit.client.0.smithi094.stdout:0/762: symlink d14/d22/d28/dac/l10d 0 2024-03-24T10:46:00.919 INFO:tasks.workunit.client.0.smithi094.stdout:0/763: write d14/d22/d28/d48/f78 [2161974,115129] 0 2024-03-24T10:46:00.921 INFO:tasks.workunit.client.0.smithi094.stdout:8/815: sync 2024-03-24T10:46:00.921 INFO:tasks.workunit.client.0.smithi094.stdout:8/816: chown df/d25/d28/d31/d47/d22/d55/f6d 150350604 1 2024-03-24T10:46:00.921 INFO:tasks.workunit.client.0.smithi094.stdout:8/817: write df/f90 [279055,3783] 0 2024-03-24T10:46:00.921 INFO:tasks.workunit.client.0.smithi094.stdout:2/724: creat d18/d5f/fd7 x:0 0 0 2024-03-24T10:46:00.921 INFO:tasks.workunit.client.0.smithi094.stdout:2/725: write d18/d7f/d88/da2/da4/f5e [3497837,68464] 0 2024-03-24T10:46:00.923 INFO:tasks.workunit.client.0.smithi094.stdout:8/818: dread df/d25/d28/d31/f79 [0,4194304] 0 2024-03-24T10:46:00.923 INFO:tasks.workunit.client.0.smithi094.stdout:8/819: dread - df/d8c/fc4 zero size 2024-03-24T10:46:00.925 INFO:tasks.workunit.client.0.smithi094.stdout:2/726: dread d18/d1e/f33 [0,4194304] 0 2024-03-24T10:46:00.934 INFO:tasks.workunit.client.0.smithi094.stdout:9/829: mkdir df/d2d/d1f/d58/d5a/d50/da5/dd4/d10a 0 2024-03-24T10:46:00.934 INFO:tasks.workunit.client.0.smithi094.stdout:9/830: fsync df/d2d/d1f/d58/d5a/d69/d7a/fe6 0 2024-03-24T10:46:00.934 INFO:tasks.workunit.client.0.smithi094.stdout:9/831: fsync df/d2d/d1f/d58/d5a/d69/d7a/f7d 0 2024-03-24T10:46:00.934 INFO:tasks.workunit.client.0.smithi094.stdout:9/832: fdatasync df/d2d/d1f/d58/d5a/fe0 0 2024-03-24T10:46:00.937 INFO:tasks.workunit.client.0.smithi094.stdout:7/762: rename d1/d9/d16/d21/d51/d83/d39/lb5 to d1/d9/d16/d21/d51/d83/d9d/lfe 0 2024-03-24T10:46:00.938 INFO:tasks.workunit.client.0.smithi094.stdout:2/727: rename d18/d54/db9 to d18/d54/db9/dd8 22 2024-03-24T10:46:00.938 INFO:tasks.workunit.client.0.smithi094.stdout:2/728: write d18/d1e/d65/f9c [565828,39251] 0 2024-03-24T10:46:00.940 INFO:tasks.workunit.client.0.smithi094.stdout:1/638: unlink d4/d9/de/d6f/d62/d80/c9a 0 2024-03-24T10:46:00.944 INFO:tasks.workunit.client.0.smithi094.stdout:8/820: mknod df/d25/d36/d10b/c113 0 2024-03-24T10:46:00.949 INFO:tasks.workunit.client.0.smithi094.stdout:9/833: rename df/d2d/d1f/d58/d5a/f103 to df/d2d/d1f/d58/d5a/d61/f10b 0 2024-03-24T10:46:00.952 INFO:tasks.workunit.client.0.smithi094.stdout:7/763: symlink d1/d9/d16/d21/d51/d83/d39/d44/dd9/da5/lff 0 2024-03-24T10:46:00.953 INFO:tasks.workunit.client.0.smithi094.stdout:9/834: dread df/d2d/f84 [0,4194304] 0 2024-03-24T10:46:00.953 INFO:tasks.workunit.client.0.smithi094.stdout:9/835: fsync df/d2d/d1f/d58/d5a/f63 0 2024-03-24T10:46:00.954 INFO:tasks.workunit.client.0.smithi094.stdout:9/836: stat df/d2d/d1f/d58/d70/dab/dec 0 2024-03-24T10:46:00.955 INFO:tasks.workunit.client.0.smithi094.stdout:2/729: mknod d18/d48/cd9 0 2024-03-24T10:46:00.955 INFO:tasks.workunit.client.0.smithi094.stdout:2/730: readlink l3 0 2024-03-24T10:46:00.957 INFO:tasks.workunit.client.0.smithi094.stdout:2/731: write d18/d1e/f33 [1595616,72521] 0 2024-03-24T10:46:00.959 INFO:tasks.workunit.client.0.smithi094.stdout:1/639: link f1 d4/d9/de/d3d/d66/dbe/fde 0 2024-03-24T10:46:00.959 INFO:tasks.workunit.client.0.smithi094.stdout:1/640: truncate d4/d9/de/d3d/faf 694802 0 2024-03-24T10:46:00.963 INFO:tasks.workunit.client.0.smithi094.stdout:0/764: rmdir d14/d22/d28/d2b/d41 39 2024-03-24T10:46:00.964 INFO:tasks.workunit.client.0.smithi094.stdout:8/821: dwrite df/d25/d28/d31/d47/d22/d55/fa5 [0,4194304] 0 2024-03-24T10:46:00.964 INFO:tasks.workunit.client.0.smithi094.stdout:8/822: chown df/d25/d36 49243 1 2024-03-24T10:46:00.964 INFO:tasks.workunit.client.0.smithi094.stdout:8/823: write df/d25/f29 [1374591,109562] 0 2024-03-24T10:46:00.966 INFO:tasks.workunit.client.0.smithi094.stdout:7/764: mknod d1/d9b/c100 0 2024-03-24T10:46:00.969 INFO:tasks.workunit.client.0.smithi094.stdout:9/837: unlink df/d2d/d1f/d11/fa6 0 2024-03-24T10:46:00.969 INFO:tasks.workunit.client.0.smithi094.stdout:9/838: dread - df/d2d/d1f/d58/d5a/d69/d7a/f8f zero size 2024-03-24T10:46:00.969 INFO:tasks.workunit.client.0.smithi094.stdout:9/839: fdatasync df/d2d/d34/d2c/d74/fc4 0 2024-03-24T10:46:00.969 INFO:tasks.workunit.client.0.smithi094.stdout:9/840: stat df/d2d/d1f/d11/f4e 0 2024-03-24T10:46:00.970 INFO:tasks.workunit.client.0.smithi094.stdout:2/732: write d18/f1d [2680309,29716] 0 2024-03-24T10:46:00.970 INFO:tasks.workunit.client.0.smithi094.stdout:3/709: sync 2024-03-24T10:46:00.970 INFO:tasks.workunit.client.0.smithi094.stdout:3/710: creat d0/db/dc/d28/d3c/d55/d7f/ffa x:0 0 0 2024-03-24T10:46:00.970 INFO:tasks.workunit.client.0.smithi094.stdout:3/711: truncate d0/db/dc/d28/d3c/d7c/f8e 1280373 0 2024-03-24T10:46:00.974 INFO:tasks.workunit.client.0.smithi094.stdout:0/765: symlink d14/d22/d5b/l10e 0 2024-03-24T10:46:00.975 INFO:tasks.workunit.client.0.smithi094.stdout:0/766: fsync fd 0 2024-03-24T10:46:00.979 INFO:tasks.workunit.client.0.smithi094.stdout:2/733: dread d18/d66/d90/d9a/dbc/f46 [0,4194304] 0 2024-03-24T10:46:00.998 INFO:tasks.workunit.client.0.smithi094.stdout:8/824: symlink df/d25/d28/d31/d47/ddf/dfd/l114 0 2024-03-24T10:46:01.001 INFO:tasks.workunit.client.0.smithi094.stdout:7/765: rename d1/d9/d16/fc1 to d1/db4/f101 0 2024-03-24T10:46:01.004 INFO:tasks.workunit.client.0.smithi094.stdout:9/841: creat df/d2d/d1f/d58/d70/d104/f10c x:0 0 0 2024-03-24T10:46:01.013 INFO:tasks.workunit.client.0.smithi094.stdout:3/712: unlink d0/lda 0 2024-03-24T10:46:01.019 INFO:tasks.workunit.client.0.smithi094.stdout:1/641: rmdir d4/d9/de/d3d 39 2024-03-24T10:46:01.019 INFO:tasks.workunit.client.0.smithi094.stdout:1/642: fdatasync d4/fcd 0 2024-03-24T10:46:01.019 INFO:tasks.workunit.client.0.smithi094.stdout:1/643: write d4/d9/d37/d33/f5b [24488,49812] 0 2024-03-24T10:46:01.019 INFO:tasks.workunit.client.0.smithi094.stdout:1/644: write d4/d9/f28 [704578,28444] 0 2024-03-24T10:46:01.020 INFO:tasks.workunit.client.0.smithi094.stdout:7/766: dwrite d1/d9/d16/f2f [0,4194304] 0 2024-03-24T10:46:01.024 INFO:tasks.workunit.client.0.smithi094.stdout:0/767: creat d14/d22/d5b/f10f x:0 0 0 2024-03-24T10:46:01.024 INFO:tasks.workunit.client.0.smithi094.stdout:0/768: getdents d14/d22/d28/d48/d7b/dfa/d102 0 2024-03-24T10:46:01.024 INFO:tasks.workunit.client.0.smithi094.stdout:0/769: write d14/d22/d28/d2b/fd4 [413086,126673] 0 2024-03-24T10:46:01.027 INFO:tasks.workunit.client.0.smithi094.stdout:2/734: unlink d18/d1e/ca8 0 2024-03-24T10:46:01.030 INFO:tasks.workunit.client.0.smithi094.stdout:6/769: sync 2024-03-24T10:46:01.030 INFO:tasks.workunit.client.0.smithi094.stdout:4/791: sync 2024-03-24T10:46:01.030 INFO:tasks.workunit.client.0.smithi094.stdout:5/758: sync 2024-03-24T10:46:01.033 INFO:tasks.workunit.client.0.smithi094.stdout:9/842: creat df/d2d/d1f/d58/d5a/d61/d87/dc8/f10d x:0 0 0 2024-03-24T10:46:01.038 INFO:tasks.workunit.client.0.smithi094.stdout:4/792: write d0/d34/d73/d93/f96 [2687084,39686] 0 2024-03-24T10:46:01.039 INFO:tasks.workunit.client.0.smithi094.stdout:1/645: dwrite d4/d9/de/d3d/d66/dbe/fde [4194304,4194304] 0 2024-03-24T10:46:01.043 INFO:tasks.workunit.client.0.smithi094.stdout:7/767: symlink d1/d9/d16/d21/d51/d83/d39/l102 0 2024-03-24T10:46:01.045 INFO:tasks.workunit.client.0.smithi094.stdout:5/759: dread d7/d23/d5f/d9/f29 [0,4194304] 0 2024-03-24T10:46:01.047 INFO:tasks.workunit.client.0.smithi094.stdout:2/735: creat d18/d66/d90/fda x:0 0 0 2024-03-24T10:46:01.049 INFO:tasks.workunit.client.0.smithi094.stdout:6/770: symlink d0/d78/da/d77/lf6 0 2024-03-24T10:46:01.054 INFO:tasks.workunit.client.0.smithi094.stdout:8/825: getdents df/d25/d36 0 2024-03-24T10:46:01.059 INFO:tasks.workunit.client.0.smithi094.stdout:9/843: mkdir df/d2d/d1f/d58/d5a/d61/d87/d10e 0 2024-03-24T10:46:01.060 INFO:tasks.workunit.client.0.smithi094.stdout:5/760: write d7/d23/d5f/d1b/fa4 [1457497,88911] 0 2024-03-24T10:46:01.060 INFO:tasks.workunit.client.0.smithi094.stdout:4/793: dwrite d0/d22/ff [0,4194304] 0 2024-03-24T10:46:01.066 INFO:tasks.workunit.client.0.smithi094.stdout:1/646: creat d4/d9/db/d85/fdf x:0 0 0 2024-03-24T10:46:01.070 INFO:tasks.workunit.client.0.smithi094.stdout:0/770: getdents d14/d22/d32/d70/dd0 0 2024-03-24T10:46:01.070 INFO:tasks.workunit.client.0.smithi094.stdout:0/771: write d14/d22/d32/d65/fcc [398889,127858] 0 2024-03-24T10:46:01.073 INFO:tasks.workunit.client.0.smithi094.stdout:2/736: dwrite d18/d66/d90/d9a/dbc/f6f [0,4194304] 0 2024-03-24T10:46:01.073 INFO:tasks.workunit.client.0.smithi094.stdout:8/826: symlink df/d25/d36/l115 0 2024-03-24T10:46:01.073 INFO:tasks.workunit.client.0.smithi094.stdout:8/827: chown df/d25/d28/d31/d47/f20 151979021 1 2024-03-24T10:46:01.073 INFO:tasks.workunit.client.0.smithi094.stdout:8/828: chown df/d12/l1a 193766 1 2024-03-24T10:46:01.075 INFO:tasks.workunit.client.0.smithi094.stdout:9/844: symlink df/d2d/d34/l10f 0 2024-03-24T10:46:01.078 INFO:tasks.workunit.client.0.smithi094.stdout:5/761: unlink d7/dc/fb8 0 2024-03-24T10:46:01.079 INFO:tasks.workunit.client.0.smithi094.stdout:6/771: dwrite d0/d1a/fe8 [0,4194304] 0 2024-03-24T10:46:01.080 INFO:tasks.workunit.client.0.smithi094.stdout:6/772: stat d0/d21/d2a/df3 0 2024-03-24T10:46:01.080 INFO:tasks.workunit.client.0.smithi094.stdout:1/647: link d4/cf d4/d9/db/d31/d3c/ce0 0 2024-03-24T10:46:01.081 INFO:tasks.workunit.client.0.smithi094.stdout:4/794: dwrite d0/d5/d1b/f2e [0,4194304] 0 2024-03-24T10:46:01.086 INFO:tasks.workunit.client.0.smithi094.stdout:7/768: rmdir d1/d9/d16/d21/d51/d83/d39/d44 39 2024-03-24T10:46:01.087 INFO:tasks.workunit.client.0.smithi094.stdout:8/829: write df/d25/d28/d31/f40 [4163121,114670] 0 2024-03-24T10:46:01.092 INFO:tasks.workunit.client.0.smithi094.stdout:1/648: write d4/d9/de/d3d/d66/dbe/fde [6770600,17980] 0 2024-03-24T10:46:01.096 INFO:tasks.workunit.client.0.smithi094.stdout:2/737: mkdir d18/d7f/d88/ddb 0 2024-03-24T10:46:01.104 INFO:tasks.workunit.client.0.smithi094.stdout:9/845: creat df/d2d/d1f/d58/d82/f110 x:0 0 0 2024-03-24T10:46:01.104 INFO:tasks.workunit.client.0.smithi094.stdout:9/846: write df/d7b/ffd [871833,21441] 0 2024-03-24T10:46:01.105 INFO:tasks.workunit.client.0.smithi094.stdout:5/762: mkdir d7/d23/d5f/d9/d48/df9 0 2024-03-24T10:46:01.105 INFO:tasks.workunit.client.0.smithi094.stdout:5/763: fsync d7/d23/dd5/da9/fc8 0 2024-03-24T10:46:01.108 INFO:tasks.workunit.client.0.smithi094.stdout:7/769: dwrite d1/d9/f13 [0,4194304] 0 2024-03-24T10:46:01.109 INFO:tasks.workunit.client.0.smithi094.stdout:7/770: chown d1/d9/d16/d21/d51/d83/d9d/lfe 80 1 2024-03-24T10:46:01.109 INFO:tasks.workunit.client.0.smithi094.stdout:7/771: readlink d1/d9/d16/d21/d51/d83/d39/d44/lef 0 2024-03-24T10:46:01.110 INFO:tasks.workunit.client.0.smithi094.stdout:6/773: rename d0/d78/da/d23/d7c/db0/cbb to d0/d30/de6/de3/cf7 0 2024-03-24T10:46:01.125 INFO:tasks.workunit.client.0.smithi094.stdout:4/795: mkdir d0/d22/de/dc8/de6/def 0 2024-03-24T10:46:01.130 INFO:tasks.workunit.client.0.smithi094.stdout:8/830: link df/d25/d36/d3f/l49 df/d25/d36/d3f/l116 0 2024-03-24T10:46:01.132 INFO:tasks.workunit.client.0.smithi094.stdout:4/796: dread d0/d22/de/d28/f8f [0,4194304] 0 2024-03-24T10:46:01.132 INFO:tasks.workunit.client.0.smithi094.stdout:4/797: write d0/d5/d1b/faf [287723,46685] 0 2024-03-24T10:46:01.132 INFO:tasks.workunit.client.0.smithi094.stdout:0/772: truncate d14/d22/d28/f46 3722403 0 2024-03-24T10:46:01.132 INFO:tasks.workunit.client.0.smithi094.stdout:0/773: dread - d14/d22/d7d/fc3 zero size 2024-03-24T10:46:01.138 INFO:tasks.workunit.client.0.smithi094.stdout:1/649: symlink d4/d9/d37/le1 0 2024-03-24T10:46:01.140 INFO:tasks.workunit.client.0.smithi094.stdout:7/772: dwrite d1/d9/d16/fc7 [0,4194304] 0 2024-03-24T10:46:01.141 INFO:tasks.workunit.client.0.smithi094.stdout:9/847: mkdir df/d2d/d1f/d111 0 2024-03-24T10:46:01.141 INFO:tasks.workunit.client.0.smithi094.stdout:9/848: readlink df/d2d/d34/d2c/l33 0 2024-03-24T10:46:01.143 INFO:tasks.workunit.client.0.smithi094.stdout:6/774: dwrite d0/da9/fcd [0,4194304] 0 2024-03-24T10:46:01.149 INFO:tasks.workunit.client.0.smithi094.stdout:2/738: rename d18/d7f/d88/da2/da4/l85 to d18/d66/d90/d9a/d74/db4/ldc 0 2024-03-24T10:46:01.149 INFO:tasks.workunit.client.0.smithi094.stdout:2/739: chown l13 9776 1 2024-03-24T10:46:01.149 INFO:tasks.workunit.client.0.smithi094.stdout:2/740: write d18/d66/d90/d9a/dbc/f84 [1228156,88423] 0 2024-03-24T10:46:01.151 INFO:tasks.workunit.client.0.smithi094.stdout:1/650: creat d4/d9/de/d6f/d62/d80/fe2 x:0 0 0 2024-03-24T10:46:01.154 INFO:tasks.workunit.client.0.smithi094.stdout:8/831: dwrite df/d25/d28/d31/d47/f20 [0,4194304] 0 2024-03-24T10:46:01.156 INFO:tasks.workunit.client.0.smithi094.stdout:3/713: sync 2024-03-24T10:46:01.159 INFO:tasks.workunit.client.0.smithi094.stdout:7/773: mknod d1/d9/d16/d21/d49/da9/c103 0 2024-03-24T10:46:01.159 INFO:tasks.workunit.client.0.smithi094.stdout:7/774: write d1/d9/d25/d3d/f98 [1952519,47052] 0 2024-03-24T10:46:01.162 INFO:tasks.workunit.client.0.smithi094.stdout:9/849: mkdir df/d2d/d1f/d11/d53/d112 0 2024-03-24T10:46:01.162 INFO:tasks.workunit.client.0.smithi094.stdout:9/850: readlink df/d2d/d1f/d58/d5a/d61/d87/lbd 0 2024-03-24T10:46:01.173 INFO:tasks.workunit.client.0.smithi094.stdout:5/764: rename d7/d23/d5f/d14/d6f/c7b to d7/d23/d2b/d68/d81/cfa 0 2024-03-24T10:46:01.173 INFO:tasks.workunit.client.0.smithi094.stdout:5/765: creat d7/d23/d5f/d9/d48/ffb x:0 0 0 2024-03-24T10:46:01.175 INFO:tasks.workunit.client.0.smithi094.stdout:1/651: symlink d4/d9/db/d5d/le3 0 2024-03-24T10:46:01.176 INFO:tasks.workunit.client.0.smithi094.stdout:2/741: dwrite d18/d1e/d65/f67 [4194304,4194304] 0 2024-03-24T10:46:01.178 INFO:tasks.workunit.client.0.smithi094.stdout:8/832: dwrite df/d25/d28/d31/d47/f5a [0,4194304] 0 2024-03-24T10:46:01.178 INFO:tasks.workunit.client.0.smithi094.stdout:8/833: chown df/d25/d28/d31/d47/d66/da3/db5/cc2 189875 1 2024-03-24T10:46:01.178 INFO:tasks.workunit.client.0.smithi094.stdout:8/834: fdatasync df/d12/d5b/d63/d75/fa6 0 2024-03-24T10:46:01.183 INFO:tasks.workunit.client.0.smithi094.stdout:3/714: mkdir d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/dfb 0 2024-03-24T10:46:01.185 INFO:tasks.workunit.client.0.smithi094.stdout:3/715: dread d0/db/dc/d28/d3c/d55/d7f/fe1 [0,4194304] 0 2024-03-24T10:46:01.185 INFO:tasks.workunit.client.0.smithi094.stdout:3/716: read - d0/db/dc/d28/d38/d6a/d9b/fa1 zero size 2024-03-24T10:46:01.191 INFO:tasks.workunit.client.0.smithi094.stdout:9/851: link df/d2d/d1f/d58/d5a/c9c df/d2d/d1f/d11/c113 0 2024-03-24T10:46:01.193 INFO:tasks.workunit.client.0.smithi094.stdout:6/775: getdents d0/d30/d50/d4d 0 2024-03-24T10:46:01.195 INFO:tasks.workunit.client.0.smithi094.stdout:7/775: rename d1/d9/d16/l19 to d1/db4/l104 0 2024-03-24T10:46:01.196 INFO:tasks.workunit.client.0.smithi094.stdout:7/776: creat d1/d9/d16/f105 x:0 0 0 2024-03-24T10:46:01.198 INFO:tasks.workunit.client.0.smithi094.stdout:1/652: unlink d4/d9/db/f67 0 2024-03-24T10:46:01.208 INFO:tasks.workunit.client.0.smithi094.stdout:3/717: mknod d0/db/dc/d28/cfc 0 2024-03-24T10:46:01.210 INFO:tasks.workunit.client.0.smithi094.stdout:9/852: unlink df/d2d/d34/d2c/d97/f105 0 2024-03-24T10:46:01.213 INFO:tasks.workunit.client.0.smithi094.stdout:1/653: dwrite d4/d9/d37/d33/fba [0,4194304] 0 2024-03-24T10:46:01.213 INFO:tasks.workunit.client.0.smithi094.stdout:1/654: readlink d4/d9/d7d/ldc 0 2024-03-24T10:46:01.213 INFO:tasks.workunit.client.0.smithi094.stdout:2/742: rename d18/d7f/f8d to d18/d54/db9/fdd 0 2024-03-24T10:46:01.214 INFO:tasks.workunit.client.0.smithi094.stdout:1/655: dread d4/d36/f3f [0,4194304] 0 2024-03-24T10:46:01.217 INFO:tasks.workunit.client.0.smithi094.stdout:7/777: link d1/db4/fba d1/db4/d65/f106 0 2024-03-24T10:46:01.217 INFO:tasks.workunit.client.0.smithi094.stdout:2/743: dread d18/d66/d90/d9a/dbc/f44 [0,4194304] 0 2024-03-24T10:46:01.223 INFO:tasks.workunit.client.0.smithi094.stdout:2/744: dread d18/d66/d90/d9a/dbc/f50 [0,4194304] 0 2024-03-24T10:46:01.223 INFO:tasks.workunit.client.0.smithi094.stdout:6/776: rename d0/d30/d50/d4d/dd2/cec to d0/d21/d2a/d55/dce/cf8 0 2024-03-24T10:46:01.225 INFO:tasks.workunit.client.0.smithi094.stdout:1/656: creat d4/d9/d37/d33/d58/d6b/fe4 x:0 0 0 2024-03-24T10:46:01.226 INFO:tasks.workunit.client.0.smithi094.stdout:3/718: dwrite d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/d73/f80 [0,4194304] 0 2024-03-24T10:46:01.226 INFO:tasks.workunit.client.0.smithi094.stdout:3/719: stat d0/db/dc/d28/d38/d6a/f82 0 2024-03-24T10:46:01.226 INFO:tasks.workunit.client.0.smithi094.stdout:3/720: chown d0/db/l8 427 1 2024-03-24T10:46:01.226 INFO:tasks.workunit.client.0.smithi094.stdout:3/721: chown d0/db/dc/d28/d3c/d55/d7f/ffa 0 1 2024-03-24T10:46:01.227 INFO:tasks.workunit.client.0.smithi094.stdout:3/722: fsync d0/db/dc/d28/d3c/d55/d7f/d20/d23/f3f 0 2024-03-24T10:46:01.227 INFO:tasks.workunit.client.0.smithi094.stdout:2/745: read d18/f19 [1803340,37982] 0 2024-03-24T10:46:01.227 INFO:tasks.workunit.client.0.smithi094.stdout:7/778: symlink d1/d9/dc3/l107 0 2024-03-24T10:46:01.229 INFO:tasks.workunit.client.0.smithi094.stdout:9/853: rename df/d7b/ffe to df/d2d/d1f/d58/d5a/d69/d7a/f114 0 2024-03-24T10:46:01.231 INFO:tasks.workunit.client.0.smithi094.stdout:1/657: creat d4/d9/db/d39/d75/fe5 x:0 0 0 2024-03-24T10:46:01.235 INFO:tasks.workunit.client.0.smithi094.stdout:3/723: mkdir d0/db/dfd 0 2024-03-24T10:46:01.235 INFO:tasks.workunit.client.0.smithi094.stdout:3/724: write d0/db/dc/d28/d3c/d55/d7f/d20/d23/f3f [706381,29810] 0 2024-03-24T10:46:01.238 INFO:tasks.workunit.client.0.smithi094.stdout:1/658: dread f1 [0,4194304] 0 2024-03-24T10:46:01.239 INFO:tasks.workunit.client.0.smithi094.stdout:7/779: truncate d1/d6c/f88 757045 0 2024-03-24T10:46:01.239 INFO:tasks.workunit.client.0.smithi094.stdout:7/780: stat d1/d9/d16/d21/d51/d83 0 2024-03-24T10:46:01.244 INFO:tasks.workunit.client.0.smithi094.stdout:6/777: rename d0/d78/f5a to d0/d21/d2a/df3/ff9 0 2024-03-24T10:46:01.252 INFO:tasks.workunit.client.0.smithi094.stdout:6/778: creat d0/d30/d50/d4d/ffa x:0 0 0 2024-03-24T10:46:01.252 INFO:tasks.workunit.client.0.smithi094.stdout:6/779: chown d0/d78/c5e 4125 1 2024-03-24T10:46:01.254 INFO:tasks.workunit.client.0.smithi094.stdout:1/659: write d4/d9/de/d6f/d5f/f68 [1953215,127224] 0 2024-03-24T10:46:01.254 INFO:tasks.workunit.client.0.smithi094.stdout:1/660: chown d4/d9/de/d6f/d62/l97 7258 1 2024-03-24T10:46:01.257 INFO:tasks.workunit.client.0.smithi094.stdout:6/780: link d0/d30/c71 d0/d78/da/d23/d7c/cfb 0 2024-03-24T10:46:01.257 INFO:tasks.workunit.client.0.smithi094.stdout:6/781: chown d0/d1a/d35/d96/ced 6051672 1 2024-03-24T10:46:01.260 INFO:tasks.workunit.client.0.smithi094.stdout:9/854: dwrite df/d2d/d1f/d11/d53/d88/fdd [0,4194304] 0 2024-03-24T10:46:01.262 INFO:tasks.workunit.client.0.smithi094.stdout:1/661: rename d4/d9/db/d39/d75/f86 to d4/d9/de/d6f/d62/d84/fe6 0 2024-03-24T10:46:01.262 INFO:tasks.workunit.client.0.smithi094.stdout:1/662: creat d4/d9/de/d6f/d62/fe7 x:0 0 0 2024-03-24T10:46:01.264 INFO:tasks.workunit.client.0.smithi094.stdout:6/782: truncate d0/d21/f5c 1645724 0 2024-03-24T10:46:01.264 INFO:tasks.workunit.client.0.smithi094.stdout:6/783: truncate d0/d78/da/d23/d83/f85 1046177 0 2024-03-24T10:46:01.266 INFO:tasks.workunit.client.0.smithi094.stdout:6/784: dread d0/f33 [0,4194304] 0 2024-03-24T10:46:01.267 INFO:tasks.workunit.client.0.smithi094.stdout:3/725: dwrite d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/fc8 [0,4194304] 0 2024-03-24T10:46:01.267 INFO:tasks.workunit.client.0.smithi094.stdout:7/781: dwrite d1/d9/d16/d21/d51/d83/d39/d44/dd9/f36 [0,4194304] 0 2024-03-24T10:46:01.267 INFO:tasks.workunit.client.0.smithi094.stdout:7/782: chown d1/d9/d16/d21/d51/d83/d39/d44/dd9/d34/cf5 3677225 1 2024-03-24T10:46:01.273 INFO:tasks.workunit.client.0.smithi094.stdout:1/663: write d4/d9/de/d6f/d62/d84/fe6 [617654,100968] 0 2024-03-24T10:46:01.274 INFO:tasks.workunit.client.0.smithi094.stdout:6/785: truncate d0/f33 565632 0 2024-03-24T10:46:01.280 INFO:tasks.workunit.client.0.smithi094.stdout:3/726: rename d0/db/l27 to d0/db/dc/d28/d3c/d55/d7f/lfe 0 2024-03-24T10:46:01.281 INFO:tasks.workunit.client.0.smithi094.stdout:9/855: dwrite df/d2d/d1f/d11/d43/f6d [0,4194304] 0 2024-03-24T10:46:01.282 INFO:tasks.workunit.client.0.smithi094.stdout:7/783: truncate d1/d6c/d80/ff8 148167 0 2024-03-24T10:46:01.284 INFO:tasks.workunit.client.0.smithi094.stdout:1/664: mknod d4/d9/db1/ce8 0 2024-03-24T10:46:01.285 INFO:tasks.workunit.client.0.smithi094.stdout:6/786: creat d0/d78/d66/d7a/ffc x:0 0 0 2024-03-24T10:46:01.287 INFO:tasks.workunit.client.0.smithi094.stdout:3/727: symlink d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/dfb/lff 0 2024-03-24T10:46:01.292 INFO:tasks.workunit.client.0.smithi094.stdout:1/665: read d4/d9/d37/d33/f7e [1895125,9552] 0 2024-03-24T10:46:01.298 INFO:tasks.workunit.client.0.smithi094.stdout:9/856: creat df/d2d/d1f/d11/d53/d112/f115 x:0 0 0 2024-03-24T10:46:01.299 INFO:tasks.workunit.client.0.smithi094.stdout:9/857: write df/d2d/d1f/d11/d53/f81 [1518436,107205] 0 2024-03-24T10:46:01.299 INFO:tasks.workunit.client.0.smithi094.stdout:9/858: dread - df/d2d/d1f/d58/d5a/d61/d87/de8/fc6 zero size 2024-03-24T10:46:01.299 INFO:tasks.workunit.client.0.smithi094.stdout:9/859: readlink df/d7b/lb5 0 2024-03-24T10:46:01.303 INFO:tasks.workunit.client.0.smithi094.stdout:7/784: mknod d1/db4/d65/c108 0 2024-03-24T10:46:01.303 INFO:tasks.workunit.client.0.smithi094.stdout:7/785: creat d1/d9/d16/f109 x:0 0 0 2024-03-24T10:46:01.304 INFO:tasks.workunit.client.0.smithi094.stdout:1/666: dread d4/d36/f70 [0,4194304] 0 2024-03-24T10:46:01.304 INFO:tasks.workunit.client.0.smithi094.stdout:1/667: write d4/d9/de/d6f/f4e [1014055,50373] 0 2024-03-24T10:46:01.304 INFO:tasks.workunit.client.0.smithi094.stdout:1/668: chown d4/d9/db/d31 26321182 1 2024-03-24T10:46:01.304 INFO:tasks.workunit.client.0.smithi094.stdout:1/669: fsync d4/d9/f28 0 2024-03-24T10:46:01.305 INFO:tasks.workunit.client.0.smithi094.stdout:6/787: rename d0/f49 to d0/d30/de6/d9c/ffd 0 2024-03-24T10:46:01.309 INFO:tasks.workunit.client.0.smithi094.stdout:9/860: rmdir df/d2d/d1f/d58/d5a/d61/d87 39 2024-03-24T10:46:01.315 INFO:tasks.workunit.client.0.smithi094.stdout:1/670: mkdir d4/d9/de/d6f/d62/d84/de9 0 2024-03-24T10:46:01.315 INFO:tasks.workunit.client.0.smithi094.stdout:1/671: fdatasync d4/d9/de/d6f/fa5 0 2024-03-24T10:46:01.317 INFO:tasks.workunit.client.0.smithi094.stdout:7/786: rename d1/d9/d16/d21/d51/d83/d9d/lc5 to d1/d9/d16/d21/d51/d83/d39/d44/dd9/d8f/l10a 0 2024-03-24T10:46:01.319 INFO:tasks.workunit.client.0.smithi094.stdout:6/788: unlink d0/d30/f5b 0 2024-03-24T10:46:01.323 INFO:tasks.workunit.client.0.smithi094.stdout:7/787: unlink d1/db4/d61/ldf 0 2024-03-24T10:46:01.324 INFO:tasks.workunit.client.0.smithi094.stdout:0/774: sync 2024-03-24T10:46:01.324 INFO:tasks.workunit.client.0.smithi094.stdout:4/798: sync 2024-03-24T10:46:01.325 INFO:tasks.workunit.client.0.smithi094.stdout:1/672: rename d4/d9/db/d31/d3c/fda to d4/d9/db1/db9/fea 0 2024-03-24T10:46:01.326 INFO:tasks.workunit.client.0.smithi094.stdout:6/789: mkdir d0/d78/d66/d7a/dfe 0 2024-03-24T10:46:01.328 INFO:tasks.workunit.client.0.smithi094.stdout:7/788: getdents d1/d6c/d80/d81/da2 0 2024-03-24T10:46:01.331 INFO:tasks.workunit.client.0.smithi094.stdout:0/775: mkdir d14/d22/d103/d110 0 2024-03-24T10:46:01.332 INFO:tasks.workunit.client.0.smithi094.stdout:4/799: mknod d0/d5/d7d/d97/cf0 0 2024-03-24T10:46:01.332 INFO:tasks.workunit.client.0.smithi094.stdout:4/800: write d0/d34/d73/d93/f96 [3780192,96305] 0 2024-03-24T10:46:01.334 INFO:tasks.workunit.client.0.smithi094.stdout:1/673: creat d4/d9/db/d5d/feb x:0 0 0 2024-03-24T10:46:01.334 INFO:tasks.workunit.client.0.smithi094.stdout:1/674: truncate d4/d9/db/d85/fdf 1008876 0 2024-03-24T10:46:01.337 INFO:tasks.workunit.client.0.smithi094.stdout:6/790: truncate d0/d78/da/f20 35360 0 2024-03-24T10:46:01.337 INFO:tasks.workunit.client.0.smithi094.stdout:6/791: readlink d0/d78/da/d77/la5 0 2024-03-24T10:46:01.337 INFO:tasks.workunit.client.0.smithi094.stdout:6/792: stat d0/d78/da/d23/d34 0 2024-03-24T10:46:01.337 INFO:tasks.workunit.client.0.smithi094.stdout:6/793: truncate d0/d78/f44 1342452 0 2024-03-24T10:46:01.341 INFO:tasks.workunit.client.0.smithi094.stdout:7/789: symlink d1/d9/d16/d21/d51/d83/d9d/dd7/l10b 0 2024-03-24T10:46:01.343 INFO:tasks.workunit.client.0.smithi094.stdout:0/776: creat d14/d22/dcb/de5/f111 x:0 0 0 2024-03-24T10:46:01.345 INFO:tasks.workunit.client.0.smithi094.stdout:1/675: truncate d4/d9/f77 3422899 0 2024-03-24T10:46:01.345 INFO:tasks.workunit.client.0.smithi094.stdout:1/676: dread - d4/d9/db/f95 zero size 2024-03-24T10:46:01.345 INFO:tasks.workunit.client.0.smithi094.stdout:1/677: write d4/fcd [402884,59491] 0 2024-03-24T10:46:01.352 INFO:tasks.workunit.client.0.smithi094.stdout:4/801: dwrite d0/d5/d1b/fc9 [0,4194304] 0 2024-03-24T10:46:01.352 INFO:tasks.workunit.client.0.smithi094.stdout:4/802: chown d0/d5/d1b/f39 5874839 1 2024-03-24T10:46:01.365 INFO:tasks.workunit.client.0.smithi094.stdout:4/803: rename d0/d22/de/d28/d2b/fd0 to d0/d22/de/d28/d2b/da2/ff1 0 2024-03-24T10:46:01.376 INFO:tasks.workunit.client.0.smithi094.stdout:7/790: dwrite d1/f5d [0,4194304] 0 2024-03-24T10:46:01.377 INFO:tasks.workunit.client.0.smithi094.stdout:1/678: dwrite d4/d9/db/f30 [0,4194304] 0 2024-03-24T10:46:01.377 INFO:tasks.workunit.client.0.smithi094.stdout:4/804: read d0/d22/de/d28/f7f [4027347,59374] 0 2024-03-24T10:46:01.377 INFO:tasks.workunit.client.0.smithi094.stdout:4/805: write d0/d5/d7d/d97/d4a/fdd [700795,108413] 0 2024-03-24T10:46:01.377 INFO:tasks.workunit.client.0.smithi094.stdout:4/806: write d0/d22/de/f30 [4362299,25260] 0 2024-03-24T10:46:01.378 INFO:tasks.workunit.client.0.smithi094.stdout:0/777: dwrite d14/f2a [0,4194304] 0 2024-03-24T10:46:01.378 INFO:tasks.workunit.client.0.smithi094.stdout:0/778: dread - d14/d22/d28/d48/da5/fa7 zero size 2024-03-24T10:46:01.378 INFO:tasks.workunit.client.0.smithi094.stdout:0/779: fdatasync d14/d22/d7d/fc3 0 2024-03-24T10:46:01.378 INFO:tasks.workunit.client.0.smithi094.stdout:0/780: chown d14/d18/c7a 7140 1 2024-03-24T10:46:01.386 INFO:tasks.workunit.client.0.smithi094.stdout:7/791: creat d1/d9/d25/f10c x:0 0 0 2024-03-24T10:46:01.389 INFO:tasks.workunit.client.0.smithi094.stdout:0/781: dread d14/d22/d5b/fbd [0,4194304] 0 2024-03-24T10:46:01.391 INFO:tasks.workunit.client.0.smithi094.stdout:1/679: dwrite d4/d9/d37/f4c [4194304,4194304] 0 2024-03-24T10:46:01.399 INFO:tasks.workunit.client.0.smithi094.stdout:1/680: dread d4/d9/de/d6f/d62/fcc [0,4194304] 0 2024-03-24T10:46:01.399 INFO:tasks.workunit.client.0.smithi094.stdout:1/681: creat d4/d9/d37/d33/d58/fec x:0 0 0 2024-03-24T10:46:01.404 INFO:tasks.workunit.client.0.smithi094.stdout:1/682: dread d4/d9/db/f18 [0,4194304] 0 2024-03-24T10:46:01.404 INFO:tasks.workunit.client.0.smithi094.stdout:1/683: creat d4/d9/db/d5d/fed x:0 0 0 2024-03-24T10:46:01.406 INFO:tasks.workunit.client.0.smithi094.stdout:0/782: dwrite d14/f1b [0,4194304] 0 2024-03-24T10:46:01.406 INFO:tasks.workunit.client.0.smithi094.stdout:0/783: readlink d14/d22/d28/d2b/d41/l6a 0 2024-03-24T10:46:01.407 INFO:tasks.workunit.client.0.smithi094.stdout:7/792: rename d1/d9/d16/d21/d51/d83/d39/l102 to d1/d9/d25/dd4/l10d 0 2024-03-24T10:46:01.408 INFO:tasks.workunit.client.0.smithi094.stdout:1/684: dread f1 [0,4194304] 0 2024-03-24T10:46:01.408 INFO:tasks.workunit.client.0.smithi094.stdout:7/793: read d1/d9/d16/d21/d51/d83/d39/d44/dd9/f4c [404337,35953] 0 2024-03-24T10:46:01.417 INFO:tasks.workunit.client.0.smithi094.stdout:4/807: getdents d0/d22/de/d28/d2b/d33 0 2024-03-24T10:46:01.417 INFO:tasks.workunit.client.0.smithi094.stdout:0/784: dread d14/d18/fb0 [0,4194304] 0 2024-03-24T10:46:01.430 INFO:tasks.workunit.client.0.smithi094.stdout:1/685: rename d4/d9/d37/f4c to d4/d9/db/d31/d3c/d9f/fee 0 2024-03-24T10:46:01.431 INFO:tasks.workunit.client.0.smithi094.stdout:1/686: creat d4/d9/de/d6f/d62/fef x:0 0 0 2024-03-24T10:46:01.432 INFO:tasks.workunit.client.0.smithi094.stdout:4/808: symlink d0/d5/lf2 0 2024-03-24T10:46:01.434 INFO:tasks.workunit.client.0.smithi094.stdout:5/766: sync 2024-03-24T10:46:01.434 INFO:tasks.workunit.client.0.smithi094.stdout:5/767: creat d7/d23/d2b/ffc x:0 0 0 2024-03-24T10:46:01.435 INFO:tasks.workunit.client.0.smithi094.stdout:7/794: rename d1/d9/d16/d21/d51/d83/d39/d44/dd9/d8f/fa6 to d1/d9/dc3/f10e 0 2024-03-24T10:46:01.435 INFO:tasks.workunit.client.0.smithi094.stdout:7/795: dread - d1/d9/d16/d21/d51/dc4/ff2 zero size 2024-03-24T10:46:01.438 INFO:tasks.workunit.client.0.smithi094.stdout:2/746: sync 2024-03-24T10:46:01.438 INFO:tasks.workunit.client.0.smithi094.stdout:3/728: sync 2024-03-24T10:46:01.439 INFO:tasks.workunit.client.0.smithi094.stdout:8/835: sync 2024-03-24T10:46:01.439 INFO:tasks.workunit.client.0.smithi094.stdout:2/747: fdatasync d18/d54/fb7 0 2024-03-24T10:46:01.439 INFO:tasks.workunit.client.0.smithi094.stdout:3/729: fdatasync d0/db/dc3/ff4 0 2024-03-24T10:46:01.439 INFO:tasks.workunit.client.0.smithi094.stdout:2/748: chown d18/d1e/d65/f67 333127443 1 2024-03-24T10:46:01.439 INFO:tasks.workunit.client.0.smithi094.stdout:2/749: fsync d18/d1e/f25 0 2024-03-24T10:46:01.440 INFO:tasks.workunit.client.0.smithi094.stdout:3/730: dread d0/db/dc/d28/d3c/d55/d7f/fe1 [0,4194304] 0 2024-03-24T10:46:01.440 INFO:tasks.workunit.client.0.smithi094.stdout:0/785: dwrite d14/d22/d32/d70/ff6 [0,4194304] 0 2024-03-24T10:46:01.441 INFO:tasks.workunit.client.0.smithi094.stdout:1/687: symlink d4/d9/db/d85/lf0 0 2024-03-24T10:46:01.442 INFO:tasks.workunit.client.0.smithi094.stdout:2/750: dread d18/d1e/f76 [0,4194304] 0 2024-03-24T10:46:01.442 INFO:tasks.workunit.client.0.smithi094.stdout:2/751: write d18/d48/fbf [1489046,13656] 0 2024-03-24T10:46:01.445 INFO:tasks.workunit.client.0.smithi094.stdout:4/809: rmdir d0/d22/de/d28/d2b/d33/da4/d95/db9 0 2024-03-24T10:46:01.455 INFO:tasks.workunit.client.0.smithi094.stdout:6/794: sync 2024-03-24T10:46:01.455 INFO:tasks.workunit.client.0.smithi094.stdout:9/861: sync 2024-03-24T10:46:01.457 INFO:tasks.workunit.client.0.smithi094.stdout:1/688: mkdir d4/d9/de/d6f/df1 0 2024-03-24T10:46:01.458 INFO:tasks.workunit.client.0.smithi094.stdout:1/689: fdatasync d4/d9/de/f2b 0 2024-03-24T10:46:01.458 INFO:tasks.workunit.client.0.smithi094.stdout:2/752: rmdir d18 39 2024-03-24T10:46:01.461 INFO:tasks.workunit.client.0.smithi094.stdout:4/810: unlink d0/d22/c36 0 2024-03-24T10:46:01.462 INFO:tasks.workunit.client.0.smithi094.stdout:8/836: rename df/d8c/d8f to df/d25/d28/d31/d47/d66/da3/da8/d117 0 2024-03-24T10:46:01.464 INFO:tasks.workunit.client.0.smithi094.stdout:5/768: dwrite d7/f34 [4194304,4194304] 0 2024-03-24T10:46:01.465 INFO:tasks.workunit.client.0.smithi094.stdout:6/795: unlink d0/d30/de6/fb7 0 2024-03-24T10:46:01.465 INFO:tasks.workunit.client.0.smithi094.stdout:8/837: dread df/d25/d28/d31/d47/d22/d55/f8a [0,4194304] 0 2024-03-24T10:46:01.467 INFO:tasks.workunit.client.0.smithi094.stdout:4/811: mkdir d0/d22/de/d28/d2b/d7a/df3 0 2024-03-24T10:46:01.469 INFO:tasks.workunit.client.0.smithi094.stdout:3/731: dwrite d0/f1e [0,4194304] 0 2024-03-24T10:46:01.469 INFO:tasks.workunit.client.0.smithi094.stdout:9/862: dwrite df/d2d/d1f/d58/d5a/d69/fbe [0,4194304] 0 2024-03-24T10:46:01.470 INFO:tasks.workunit.client.0.smithi094.stdout:0/786: rename d14/d22/d32/d65/cad to d14/d22/dcb/c112 0 2024-03-24T10:46:01.470 INFO:tasks.workunit.client.0.smithi094.stdout:0/787: chown d14/d22/d28/d2b/d68/c76 279284746 1 2024-03-24T10:46:01.471 INFO:tasks.workunit.client.0.smithi094.stdout:1/690: dread d4/d9/f25 [0,4194304] 0 2024-03-24T10:46:01.472 INFO:tasks.workunit.client.0.smithi094.stdout:9/863: read df/d2d/f83 [220957,3154] 0 2024-03-24T10:46:01.472 INFO:tasks.workunit.client.0.smithi094.stdout:9/864: write df/d2d/d1f/d11/f75 [575356,89445] 0 2024-03-24T10:46:01.474 INFO:tasks.workunit.client.0.smithi094.stdout:9/865: write df/d2d/d1f/d58/d5a/f77 [470527,40861] 0 2024-03-24T10:46:01.474 INFO:tasks.workunit.client.0.smithi094.stdout:5/769: symlink d7/d23/d5f/d14/d9f/lfd 0 2024-03-24T10:46:01.478 INFO:tasks.workunit.client.0.smithi094.stdout:2/753: dwrite d18/d48/fbf [0,4194304] 0 2024-03-24T10:46:01.480 INFO:tasks.workunit.client.0.smithi094.stdout:8/838: mkdir df/d25/d28/d31/d95/d118 0 2024-03-24T10:46:01.482 INFO:tasks.workunit.client.0.smithi094.stdout:4/812: unlink d0/d22/de/d28/d2b/d7a/fbd 0 2024-03-24T10:46:01.484 INFO:tasks.workunit.client.0.smithi094.stdout:3/732: creat d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d34/d71/f100 x:0 0 0 2024-03-24T10:46:01.487 INFO:tasks.workunit.client.0.smithi094.stdout:1/691: dwrite d4/d9/db/d31/f48 [0,4194304] 0 2024-03-24T10:46:01.490 INFO:tasks.workunit.client.0.smithi094.stdout:0/788: rename d14/d22/d32/d70/dd0/f10c to d14/d22/d28/d2b/d41/d53/d105/f113 0 2024-03-24T10:46:01.490 INFO:tasks.workunit.client.0.smithi094.stdout:0/789: readlink d14/d22/d28/dac/l10d 0 2024-03-24T10:46:01.490 INFO:tasks.workunit.client.0.smithi094.stdout:0/790: creat d14/d22/d7d/f114 x:0 0 0 2024-03-24T10:46:01.492 INFO:tasks.workunit.client.0.smithi094.stdout:9/866: rmdir df/d2d/d34/d2c/d97/ddf 0 2024-03-24T10:46:01.495 INFO:tasks.workunit.client.0.smithi094.stdout:6/796: rmdir d0/d78/da/d23/d7c/db0 39 2024-03-24T10:46:01.495 INFO:tasks.workunit.client.0.smithi094.stdout:6/797: read d0/d30/f76 [659163,37733] 0 2024-03-24T10:46:01.500 INFO:tasks.workunit.client.0.smithi094.stdout:2/754: truncate d18/f45 498103 0 2024-03-24T10:46:01.500 INFO:tasks.workunit.client.0.smithi094.stdout:2/755: chown d18 32400283 1 2024-03-24T10:46:01.503 INFO:tasks.workunit.client.0.smithi094.stdout:1/692: dwrite f1 [0,4194304] 0 2024-03-24T10:46:01.503 INFO:tasks.workunit.client.0.smithi094.stdout:4/813: mkdir d0/d22/de/d28/d2b/d33/da4/df4 0 2024-03-24T10:46:01.503 INFO:tasks.workunit.client.0.smithi094.stdout:4/814: fsync d0/d5/d1b/f2e 0 2024-03-24T10:46:01.510 INFO:tasks.workunit.client.0.smithi094.stdout:8/839: rename f6 to df/d12/d5b/d63/f119 0 2024-03-24T10:46:01.513 INFO:tasks.workunit.client.0.smithi094.stdout:0/791: link d14/d22/d28/d2b/d68/f72 d14/d22/d28/d2b/dc0/f115 0 2024-03-24T10:46:01.513 INFO:tasks.workunit.client.0.smithi094.stdout:0/792: write d14/d22/d28/d2b/d41/d53/d105/f113 [509091,80303] 0 2024-03-24T10:46:01.514 INFO:tasks.workunit.client.0.smithi094.stdout:9/867: dwrite df/d2d/d1f/d58/d82/fb7 [0,4194304] 0 2024-03-24T10:46:01.515 INFO:tasks.workunit.client.0.smithi094.stdout:5/770: getdents d7/d23/d5f/d9/dde 0 2024-03-24T10:46:01.519 INFO:tasks.workunit.client.0.smithi094.stdout:6/798: dwrite d0/d78/da/d23/dc3/f6e [0,4194304] 0 2024-03-24T10:46:01.519 INFO:tasks.workunit.client.0.smithi094.stdout:6/799: read - d0/d78/d66/fee zero size 2024-03-24T10:46:01.519 INFO:tasks.workunit.client.0.smithi094.stdout:1/693: creat d4/d9/db/d39/ff2 x:0 0 0 2024-03-24T10:46:01.520 INFO:tasks.workunit.client.0.smithi094.stdout:1/694: chown d4/d9/db/d5d/fc3 1018 1 2024-03-24T10:46:01.521 INFO:tasks.workunit.client.0.smithi094.stdout:9/868: dread df/d2d/d1f/d11/d53/f81 [0,4194304] 0 2024-03-24T10:46:01.521 INFO:tasks.workunit.client.0.smithi094.stdout:9/869: creat df/d2d/d1f/d58/d5a/d50/da5/d101/f116 x:0 0 0 2024-03-24T10:46:01.522 INFO:tasks.workunit.client.0.smithi094.stdout:4/815: symlink d0/d5/d1b/d4f/dd6/lf5 0 2024-03-24T10:46:01.522 INFO:tasks.workunit.client.0.smithi094.stdout:5/771: write d7/d23/d5f/d9/f78 [4153559,97202] 0 2024-03-24T10:46:01.523 INFO:tasks.workunit.client.0.smithi094.stdout:9/870: write df/d7b/ffd [77741,66486] 0 2024-03-24T10:46:01.525 INFO:tasks.workunit.client.0.smithi094.stdout:0/793: read d14/d22/d32/d70/ff6 [1432310,121502] 0 2024-03-24T10:46:01.526 INFO:tasks.workunit.client.0.smithi094.stdout:0/794: write d14/d18/fc2 [1071933,15322] 0 2024-03-24T10:46:01.526 INFO:tasks.workunit.client.0.smithi094.stdout:0/795: stat d14/d22/d28/d2b/d68/lff 0 2024-03-24T10:46:01.526 INFO:tasks.workunit.client.0.smithi094.stdout:3/733: rename d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d4f/d74 to d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d87/dc5/d101 0 2024-03-24T10:46:01.529 INFO:tasks.workunit.client.0.smithi094.stdout:8/840: symlink df/d25/d36/l11a 0 2024-03-24T10:46:01.535 INFO:tasks.workunit.client.0.smithi094.stdout:2/756: truncate d18/f45 565476 0 2024-03-24T10:46:01.536 INFO:tasks.workunit.client.0.smithi094.stdout:0/796: read d14/d22/d32/d70/dd0/fa3 [7288533,17852] 0 2024-03-24T10:46:01.548 INFO:tasks.workunit.client.0.smithi094.stdout:6/800: creat d0/d78/da/d23/d7c/db0/fff x:0 0 0 2024-03-24T10:46:01.548 INFO:tasks.workunit.client.0.smithi094.stdout:6/801: fdatasync d0/d78/da/d23/d7c/db0/fff 0 2024-03-24T10:46:01.548 INFO:tasks.workunit.client.0.smithi094.stdout:6/802: chown d0/d78/d66/d7a/f8b 1769448 1 2024-03-24T10:46:01.549 INFO:tasks.workunit.client.0.smithi094.stdout:6/803: creat d0/d1a/d35/f100 x:0 0 0 2024-03-24T10:46:01.556 INFO:tasks.workunit.client.0.smithi094.stdout:4/816: creat d0/d5/d1b/d4f/dd6/ff6 x:0 0 0 2024-03-24T10:46:01.556 INFO:tasks.workunit.client.0.smithi094.stdout:4/817: truncate d0/d22/de/d28/d2b/d33/da1/fea 128500 0 2024-03-24T10:46:01.559 INFO:tasks.workunit.client.0.smithi094.stdout:9/871: symlink df/d2d/d1f/d7e/df8/l117 0 2024-03-24T10:46:01.561 INFO:tasks.workunit.client.0.smithi094.stdout:3/734: mkdir d0/db/dc/dd9/d102 0 2024-03-24T10:46:01.563 INFO:tasks.workunit.client.0.smithi094.stdout:8/841: unlink df/d25/d28/d31/d47/d22/d55/fa5 0 2024-03-24T10:46:01.570 INFO:tasks.workunit.client.0.smithi094.stdout:0/797: link d14/d22/d32/d70/dd0/f87 d14/d22/dcb/f116 0 2024-03-24T10:46:01.572 INFO:tasks.workunit.client.0.smithi094.stdout:7/796: sync 2024-03-24T10:46:01.573 INFO:tasks.workunit.client.0.smithi094.stdout:6/804: symlink d0/d78/da/d77/l101 0 2024-03-24T10:46:01.578 INFO:tasks.workunit.client.0.smithi094.stdout:1/695: getdents d4/d9/de/d6f/d62 0 2024-03-24T10:46:01.578 INFO:tasks.workunit.client.0.smithi094.stdout:4/818: mknod d0/d22/de/d28/d3d/d91/cf7 0 2024-03-24T10:46:01.578 INFO:tasks.workunit.client.0.smithi094.stdout:4/819: readlink d0/d34/db2/l9f 0 2024-03-24T10:46:01.578 INFO:tasks.workunit.client.0.smithi094.stdout:9/872: symlink df/d2d/d34/d2c/d97/df5/l118 0 2024-03-24T10:46:01.578 INFO:tasks.workunit.client.0.smithi094.stdout:9/873: write df/d2d/f83 [456743,7585] 0 2024-03-24T10:46:01.578 INFO:tasks.workunit.client.0.smithi094.stdout:9/874: fsync df/d2d/d1f/d58/d82/fb7 0 2024-03-24T10:46:01.578 INFO:tasks.workunit.client.0.smithi094.stdout:9/875: fsync df/d2d/f15 0 2024-03-24T10:46:01.579 INFO:tasks.workunit.client.0.smithi094.stdout:9/876: fdatasync df/f22 0 2024-03-24T10:46:01.579 INFO:tasks.workunit.client.0.smithi094.stdout:3/735: link d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/c31 d0/db/dc3/dea/c103 0 2024-03-24T10:46:01.581 INFO:tasks.workunit.client.0.smithi094.stdout:8/842: getdents df/d25/d36/d3f 0 2024-03-24T10:46:01.583 INFO:tasks.workunit.client.0.smithi094.stdout:2/757: rmdir d18/d1e 39 2024-03-24T10:46:01.587 INFO:tasks.workunit.client.0.smithi094.stdout:0/798: rename d14/d22/d28/d48/ld3 to d14/l117 0 2024-03-24T10:46:01.587 INFO:tasks.workunit.client.0.smithi094.stdout:0/799: truncate d14/d22/d28/f6c 1092956 0 2024-03-24T10:46:01.588 INFO:tasks.workunit.client.0.smithi094.stdout:0/800: dread d14/d22/d5b/d86/f8f [0,4194304] 0 2024-03-24T10:46:01.588 INFO:tasks.workunit.client.0.smithi094.stdout:0/801: creat d14/d22/d32/d65/f118 x:0 0 0 2024-03-24T10:46:01.589 INFO:tasks.workunit.client.0.smithi094.stdout:7/797: creat d1/d9/d16/d21/d51/d83/d39/d44/dd9/da5/f10f x:0 0 0 2024-03-24T10:46:01.589 INFO:tasks.workunit.client.0.smithi094.stdout:6/805: mknod d0/d30/de6/c102 0 2024-03-24T10:46:01.591 INFO:tasks.workunit.client.0.smithi094.stdout:1/696: symlink d4/d9/db/d98/lf3 0 2024-03-24T10:46:01.592 INFO:tasks.workunit.client.0.smithi094.stdout:4/820: mknod d0/d22/de/d28/d2b/d33/da4/df4/cf8 0 2024-03-24T10:46:01.593 INFO:tasks.workunit.client.0.smithi094.stdout:9/877: link df/d2d/d1f/d11/d43/cb2 df/d2d/d1f/d11/d53/d88/dd5/c119 0 2024-03-24T10:46:01.596 INFO:tasks.workunit.client.0.smithi094.stdout:3/736: creat d0/db/dc/d28/d38/d53/f104 x:0 0 0 2024-03-24T10:46:01.597 INFO:tasks.workunit.client.0.smithi094.stdout:8/843: mknod df/d25/d28/c11b 0 2024-03-24T10:46:01.597 INFO:tasks.workunit.client.0.smithi094.stdout:8/844: fdatasync df/d12/d5b/d63/f119 0 2024-03-24T10:46:01.598 INFO:tasks.workunit.client.0.smithi094.stdout:8/845: chown df/d25/d36/f9b 3897751 1 2024-03-24T10:46:01.598 INFO:tasks.workunit.client.0.smithi094.stdout:2/758: truncate d18/d1e/f33 1594797 0 2024-03-24T10:46:01.606 INFO:tasks.workunit.client.0.smithi094.stdout:7/798: getdents d1/d9/d16/d21/d51/d83/d39/d44/dd9 0 2024-03-24T10:46:01.607 INFO:tasks.workunit.client.0.smithi094.stdout:6/806: truncate d0/d1a/d35/f72 294662 0 2024-03-24T10:46:01.609 INFO:tasks.workunit.client.0.smithi094.stdout:1/697: truncate d4/d9/f25 3045446 0 2024-03-24T10:46:01.612 INFO:tasks.workunit.client.0.smithi094.stdout:4/821: creat d0/d22/de/dc8/de6/ff9 x:0 0 0 2024-03-24T10:46:01.612 INFO:tasks.workunit.client.0.smithi094.stdout:9/878: dwrite df/d7b/fbb [4194304,4194304] 0 2024-03-24T10:46:01.612 INFO:tasks.workunit.client.0.smithi094.stdout:0/802: dwrite d14/d18/fb0 [0,4194304] 0 2024-03-24T10:46:01.616 INFO:tasks.workunit.client.0.smithi094.stdout:3/737: unlink d0/db/l35 0 2024-03-24T10:46:01.631 INFO:tasks.workunit.client.0.smithi094.stdout:2/759: dwrite d18/d5f/fb5 [0,4194304] 0 2024-03-24T10:46:01.631 INFO:tasks.workunit.client.0.smithi094.stdout:2/760: stat d18/d5f 0 2024-03-24T10:46:01.632 INFO:tasks.workunit.client.0.smithi094.stdout:2/761: read d18/d66/d90/d9a/dbc/f50 [4107001,78339] 0 2024-03-24T10:46:01.632 INFO:tasks.workunit.client.0.smithi094.stdout:8/846: dwrite f4 [0,4194304] 0 2024-03-24T10:46:01.632 INFO:tasks.workunit.client.0.smithi094.stdout:9/879: dwrite df/d2d/d1f/d58/d70/dab/fc9 [0,4194304] 0 2024-03-24T10:46:01.632 INFO:tasks.workunit.client.0.smithi094.stdout:9/880: dread - df/d2d/d1f/d58/d70/dab/d80/fce zero size 2024-03-24T10:46:01.636 INFO:tasks.workunit.client.0.smithi094.stdout:9/881: write df/d2d/d1f/d11/f4f [3452971,126832] 0 2024-03-24T10:46:01.636 INFO:tasks.workunit.client.0.smithi094.stdout:9/882: chown df/d2d/d1f/d58/d5a/d69/d7a 992537 1 2024-03-24T10:46:01.658 INFO:tasks.workunit.client.0.smithi094.stdout:6/807: unlink d0/d1a/dca/ldd 0 2024-03-24T10:46:01.658 INFO:tasks.workunit.client.0.smithi094.stdout:6/808: dread - d0/d21/d2a/dcb/fef zero size 2024-03-24T10:46:01.660 INFO:tasks.workunit.client.0.smithi094.stdout:1/698: mknod d4/d9/de/d6f/d5f/d8d/cf4 0 2024-03-24T10:46:01.663 INFO:tasks.workunit.client.0.smithi094.stdout:4/822: mkdir d0/d22/de4/dfa 0 2024-03-24T10:46:01.663 INFO:tasks.workunit.client.0.smithi094.stdout:4/823: stat d0/d5/d1b/d4f/lee 0 2024-03-24T10:46:01.663 INFO:tasks.workunit.client.0.smithi094.stdout:4/824: dread - d0/d34/d73/fbf zero size 2024-03-24T10:46:01.667 INFO:tasks.workunit.client.0.smithi094.stdout:1/699: dread d4/f1e [0,4194304] 0 2024-03-24T10:46:01.670 INFO:tasks.workunit.client.0.smithi094.stdout:3/738: link d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d34/d71/lf3 d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/d73/l105 0 2024-03-24T10:46:01.672 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:01 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:46:01.677 INFO:tasks.workunit.client.0.smithi094.stdout:3/739: write d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/d73/f80 [2637086,118291] 0 2024-03-24T10:46:01.689 INFO:tasks.workunit.client.0.smithi094.stdout:5/772: sync 2024-03-24T10:46:01.689 INFO:tasks.workunit.client.0.smithi094.stdout:5/773: fdatasync d7/d23/d5f/d9/fc4 0 2024-03-24T10:46:01.689 INFO:tasks.workunit.client.0.smithi094.stdout:5/774: fsync d7/d23/d2b/d68/fc0 0 2024-03-24T10:46:01.689 INFO:tasks.workunit.client.0.smithi094.stdout:5/775: truncate d7/d23/d5f/f28 1337400 0 2024-03-24T10:46:01.689 INFO:tasks.workunit.client.0.smithi094.stdout:5/776: write d7/d23/d5f/d14/d6f/dbe/fd9 [214182,42413] 0 2024-03-24T10:46:01.690 INFO:tasks.workunit.client.0.smithi094.stdout:5/777: read - d7/dc/d56/f93 zero size 2024-03-24T10:46:01.702 INFO:tasks.workunit.client.0.smithi094.stdout:5/778: dwrite d7/d23/d5f/d14/d6f/dbe/ff3 [0,4194304] 0 2024-03-24T10:46:01.703 INFO:tasks.workunit.client.0.smithi094.stdout:5/779: write d7/d23/d5f/d14/d6f/dbe/fc6 [337524,94975] 0 2024-03-24T10:46:01.715 INFO:tasks.workunit.client.0.smithi094.stdout:5/780: dread d7/d23/dd5/d64/dd6/f3e [0,4194304] 0 2024-03-24T10:46:01.721 INFO:tasks.workunit.client.0.smithi094.stdout:5/781: write d7/d23/d5f/d14/d36/f4d [9889100,40876] 0 2024-03-24T10:46:01.734 INFO:tasks.workunit.client.0.smithi094.stdout:2/762: rmdir d18/d7f/d88/d8a 39 2024-03-24T10:46:01.736 INFO:tasks.workunit.client.0.smithi094.stdout:2/763: dread d18/d48/f98 [0,4194304] 0 2024-03-24T10:46:01.737 INFO:tasks.workunit.client.0.smithi094.stdout:2/764: write d18/f3f [1077517,65203] 0 2024-03-24T10:46:01.746 INFO:tasks.workunit.client.0.smithi094.stdout:8/847: creat df/d25/d28/d31/d47/dbd/f11c x:0 0 0 2024-03-24T10:46:01.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:01 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:46:01.895 INFO:tasks.workunit.client.0.smithi094.stdout:4/825: link d0/d5/d1b/f2e d0/d34/d73/d94/ffb 0 2024-03-24T10:46:01.897 INFO:tasks.workunit.client.0.smithi094.stdout:1/700: mkdir d4/d9/db/d85/df5 0 2024-03-24T10:46:01.898 INFO:tasks.workunit.client.0.smithi094.stdout:1/701: dread d4/d9/db/d5d/d63/fd3 [0,4194304] 0 2024-03-24T10:46:01.898 INFO:tasks.workunit.client.0.smithi094.stdout:1/702: creat d4/d9/db/d31/d3c/ff6 x:0 0 0 2024-03-24T10:46:01.906 INFO:tasks.workunit.client.0.smithi094.stdout:5/782: mknod d7/d23/d5f/d9/d48/df9/cfe 0 2024-03-24T10:46:01.910 INFO:tasks.workunit.client.0.smithi094.stdout:2/765: mknod d18/d1e/cde 0 2024-03-24T10:46:01.910 INFO:tasks.workunit.client.0.smithi094.stdout:2/766: stat d18/d7f/d88/da2/da4/cc4 0 2024-03-24T10:46:01.912 INFO:tasks.workunit.client.0.smithi094.stdout:8/848: creat df/d25/df1/f11d x:0 0 0 2024-03-24T10:46:01.914 INFO:tasks.workunit.client.0.smithi094.stdout:1/703: dwrite d4/d9/db/d5d/fc3 [0,4194304] 0 2024-03-24T10:46:01.914 INFO:tasks.workunit.client.0.smithi094.stdout:1/704: read d4/d9/db/d5d/f64 [8339,42100] 0 2024-03-24T10:46:01.920 INFO:tasks.workunit.client.0.smithi094.stdout:5/783: dread d7/d26/f80 [0,4194304] 0 2024-03-24T10:46:01.920 INFO:tasks.workunit.client.0.smithi094.stdout:5/784: stat d7/d23/d2b/d68/d81 0 2024-03-24T10:46:01.923 INFO:tasks.workunit.client.0.smithi094.stdout:4/826: mknod d0/d5/d7d/d97/cfc 0 2024-03-24T10:46:01.927 INFO:tasks.workunit.client.0.smithi094.stdout:2/767: dwrite d18/d48/f5d [0,4194304] 0 2024-03-24T10:46:01.927 INFO:tasks.workunit.client.0.smithi094.stdout:2/768: fsync d18/d7f/d88/da2/da4/f5e 0 2024-03-24T10:46:01.928 INFO:tasks.workunit.client.0.smithi094.stdout:8/849: mkdir df/d25/d28/d31/d47/d66/da3/da8/d117/df2/d11e 0 2024-03-24T10:46:01.930 INFO:tasks.workunit.client.0.smithi094.stdout:8/850: write df/d25/d28/d31/d47/f7c [912278,53720] 0 2024-03-24T10:46:01.935 INFO:tasks.workunit.client.0.smithi094.stdout:1/705: mknod d4/d9/de/d6f/d62/cf7 0 2024-03-24T10:46:01.935 INFO:tasks.workunit.client.0.smithi094.stdout:1/706: write d4/d9/db/d31/d3c/f7c [5008555,67232] 0 2024-03-24T10:46:01.935 INFO:tasks.workunit.client.0.smithi094.stdout:1/707: truncate d4/d9/de/d6f/d62/d80/fe2 238148 0 2024-03-24T10:46:01.935 INFO:tasks.workunit.client.0.smithi094.stdout:4/827: dwrite d0/d5/d1b/d4f/dd6/ff6 [0,4194304] 0 2024-03-24T10:46:01.937 INFO:tasks.workunit.client.0.smithi094.stdout:5/785: truncate d7/d23/dd5/d64/fbf 2709383 0 2024-03-24T10:46:01.937 INFO:tasks.workunit.client.0.smithi094.stdout:5/786: chown d7/d23/dd5/d64/lba 12 1 2024-03-24T10:46:01.937 INFO:tasks.workunit.client.0.smithi094.stdout:5/787: write d7/d26/f65 [74108,126471] 0 2024-03-24T10:46:01.941 INFO:tasks.workunit.client.0.smithi094.stdout:2/769: mkdir d18/d66/dbe/ddf 0 2024-03-24T10:46:01.943 INFO:tasks.workunit.client.0.smithi094.stdout:8/851: symlink df/d25/d28/d31/d47/l11f 0 2024-03-24T10:46:01.949 INFO:tasks.workunit.client.0.smithi094.stdout:1/708: truncate d4/d9/de/d6f/d62/d84/fc1 236158 0 2024-03-24T10:46:01.958 INFO:tasks.workunit.client.0.smithi094.stdout:4/828: truncate d0/d34/f48 1486073 0 2024-03-24T10:46:01.961 INFO:tasks.workunit.client.0.smithi094.stdout:5/788: mkdir d7/d23/dd5/d64/dff 0 2024-03-24T10:46:01.961 INFO:tasks.workunit.client.0.smithi094.stdout:5/789: write d7/d23/dd5/da9/fd2 [30870,81161] 0 2024-03-24T10:46:01.961 INFO:tasks.workunit.client.0.smithi094.stdout:5/790: chown d7/d23/d5f/d14/d6f/c98 41582093 1 2024-03-24T10:46:01.961 INFO:tasks.workunit.client.0.smithi094.stdout:5/791: dread - d7/dc/fc1 zero size 2024-03-24T10:46:01.965 INFO:tasks.workunit.client.0.smithi094.stdout:5/792: mkdir d7/dc/d100 0 2024-03-24T10:46:01.967 INFO:tasks.workunit.client.0.smithi094.stdout:5/793: mkdir d7/d26/d101 0 2024-03-24T10:46:01.969 INFO:tasks.workunit.client.0.smithi094.stdout:8/852: dwrite df/d25/d28/d31/d47/ddf/dfd/f107 [0,4194304] 0 2024-03-24T10:46:01.969 INFO:tasks.workunit.client.0.smithi094.stdout:8/853: truncate df/d25/d28/d31/d47/d66/da3/da8/fe5 514860 0 2024-03-24T10:46:01.969 INFO:tasks.workunit.client.0.smithi094.stdout:2/770: dwrite d18/f1d [0,4194304] 0 2024-03-24T10:46:01.970 INFO:tasks.workunit.client.0.smithi094.stdout:5/794: mkdir d7/dc/d102 0 2024-03-24T10:46:01.970 INFO:tasks.workunit.client.0.smithi094.stdout:5/795: truncate d7/fca 917447 0 2024-03-24T10:46:01.972 INFO:tasks.workunit.client.0.smithi094.stdout:5/796: dread d7/d23/d5f/f58 [0,4194304] 0 2024-03-24T10:46:01.972 INFO:tasks.workunit.client.0.smithi094.stdout:5/797: write d7/d23/d2b/d68/f8f [1369866,87643] 0 2024-03-24T10:46:01.979 INFO:tasks.workunit.client.0.smithi094.stdout:7/799: unlink d1/d9/fd 0 2024-03-24T10:46:01.983 INFO:tasks.workunit.client.0.smithi094.stdout:2/771: unlink d18/d48/l7b 0 2024-03-24T10:46:01.983 INFO:tasks.workunit.client.0.smithi094.stdout:2/772: read f15 [6239852,100783] 0 2024-03-24T10:46:01.983 INFO:tasks.workunit.client.0.smithi094.stdout:2/773: readlink l6 0 2024-03-24T10:46:01.983 INFO:tasks.workunit.client.0.smithi094.stdout:2/774: read d18/f31 [352021,419] 0 2024-03-24T10:46:01.985 INFO:tasks.workunit.client.0.smithi094.stdout:7/800: creat d1/d6c/d80/d81/f110 x:0 0 0 2024-03-24T10:46:01.998 INFO:tasks.workunit.client.0.smithi094.stdout:8/854: dwrite df/d25/d28/d31/d47/d66/da3/da8/d117/df2/ff5 [0,4194304] 0 2024-03-24T10:46:02.003 INFO:tasks.workunit.client.0.smithi094.stdout:8/855: truncate df/f1b 3860271 0 2024-03-24T10:46:02.009 INFO:tasks.workunit.client.0.smithi094.stdout:8/856: getdents df/d12/d5b/d63 0 2024-03-24T10:46:02.010 INFO:tasks.workunit.client.0.smithi094.stdout:5/798: dwrite d7/d23/d5f/d14/d36/f4d [0,4194304] 0 2024-03-24T10:46:02.010 INFO:tasks.workunit.client.0.smithi094.stdout:5/799: readlink d7/d23/d5f/d9/dde/leb 0 2024-03-24T10:46:02.011 INFO:tasks.workunit.client.0.smithi094.stdout:2/775: dwrite d18/d48/d56/fb0 [4194304,4194304] 0 2024-03-24T10:46:02.011 INFO:tasks.workunit.client.0.smithi094.stdout:2/776: creat d18/d66/d90/d9a/d74/fe0 x:0 0 0 2024-03-24T10:46:02.011 INFO:tasks.workunit.client.0.smithi094.stdout:2/777: write d18/d66/d90/d9a/d74/fd1 [146467,46897] 0 2024-03-24T10:46:02.013 INFO:tasks.workunit.client.0.smithi094.stdout:5/800: truncate d7/d23/d5f/d9/d48/f6b 639851 0 2024-03-24T10:46:02.019 INFO:tasks.workunit.client.0.smithi094.stdout:0/803: rename d14/d22/d5b/lae to d14/d22/d28/d48/da5/l119 0 2024-03-24T10:46:02.019 INFO:tasks.workunit.client.0.smithi094.stdout:2/778: write d18/d1e/d65/f67 [2323300,4665] 0 2024-03-24T10:46:02.021 INFO:tasks.workunit.client.0.smithi094.stdout:2/779: dread d18/f45 [0,4194304] 0 2024-03-24T10:46:02.023 INFO:tasks.workunit.client.0.smithi094.stdout:5/801: symlink d7/d23/d5f/d9/d48/df9/l103 0 2024-03-24T10:46:02.028 INFO:tasks.workunit.client.0.smithi094.stdout:9/883: rename df/d2d/d1f/d58/d5a/d61/d87/de8 to df/d2d/d1f/d58/d5a/d50/da5/dd4/d10a/d11a 0 2024-03-24T10:46:02.028 INFO:tasks.workunit.client.0.smithi094.stdout:9/884: chown df/d2d/d1f/d58/d5a/d69/d7a/feb 720 1 2024-03-24T10:46:02.040 INFO:tasks.workunit.client.0.smithi094.stdout:3/740: rename d0/db/dc/d28/d38/d6a to d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/db5/d106 0 2024-03-24T10:46:02.042 INFO:tasks.workunit.client.0.smithi094.stdout:1/709: rename d4/d9/db/d39/f49 to d4/d9/d37/d33/d58/d6b/ff8 0 2024-03-24T10:46:02.044 INFO:tasks.workunit.client.0.smithi094.stdout:9/885: dread df/d2d/d1f/d11/f75 [0,4194304] 0 2024-03-24T10:46:02.044 INFO:tasks.workunit.client.0.smithi094.stdout:3/741: creat d0/db/dc/d28/d38/dc9/f107 x:0 0 0 2024-03-24T10:46:02.046 INFO:tasks.workunit.client.0.smithi094.stdout:1/710: unlink d4/d9/la0 0 2024-03-24T10:46:02.047 INFO:tasks.workunit.client.0.smithi094.stdout:9/886: truncate df/d2d/d1f/d58/d5a/f4b 2671085 0 2024-03-24T10:46:02.048 INFO:tasks.workunit.client.0.smithi094.stdout:0/804: dwrite d14/d22/d7d/fc3 [0,4194304] 0 2024-03-24T10:46:02.048 INFO:tasks.workunit.client.0.smithi094.stdout:0/805: fsync d14/d22/d28/f31 0 2024-03-24T10:46:02.048 INFO:tasks.workunit.client.0.smithi094.stdout:2/780: dwrite d18/d66/d90/d9a/dbc/d71/fc9 [0,4194304] 0 2024-03-24T10:46:02.048 INFO:tasks.workunit.client.0.smithi094.stdout:2/781: write d18/d66/d90/d9a/dbc/f84 [1566421,129737] 0 2024-03-24T10:46:02.050 INFO:tasks.workunit.client.0.smithi094.stdout:3/742: mkdir d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d4f/d108 0 2024-03-24T10:46:02.050 INFO:tasks.workunit.client.0.smithi094.stdout:3/743: chown d0/db/dc/cac 16840 1 2024-03-24T10:46:02.052 INFO:tasks.workunit.client.0.smithi094.stdout:1/711: creat d4/d9/de/d6f/d62/ff9 x:0 0 0 2024-03-24T10:46:02.055 INFO:tasks.workunit.client.0.smithi094.stdout:9/887: mknod df/d2d/d1f/d58/d70/c11b 0 2024-03-24T10:46:02.060 INFO:tasks.workunit.client.0.smithi094.stdout:0/806: getdents d14/d22/d103 0 2024-03-24T10:46:02.065 INFO:tasks.workunit.client.0.smithi094.stdout:2/782: creat d18/d48/dc3/fe1 x:0 0 0 2024-03-24T10:46:02.069 INFO:tasks.workunit.client.0.smithi094.stdout:3/744: unlink d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/db5/fbd 0 2024-03-24T10:46:02.069 INFO:tasks.workunit.client.0.smithi094.stdout:3/745: fsync d0/db/dc/d28/d3c/d55/d7f/d20/d23/f3f 0 2024-03-24T10:46:02.080 INFO:tasks.workunit.client.0.smithi094.stdout:1/712: mkdir d4/d9/db/d85/df5/dfa 0 2024-03-24T10:46:02.080 INFO:tasks.workunit.client.0.smithi094.stdout:1/713: readlink d4/d9/l35 0 2024-03-24T10:46:02.080 INFO:tasks.workunit.client.0.smithi094.stdout:1/714: chown d4/d9/de/d6f 59 1 2024-03-24T10:46:02.084 INFO:tasks.workunit.client.0.smithi094.stdout:0/807: getdents d14/d22/d28/d48 0 2024-03-24T10:46:02.084 INFO:tasks.workunit.client.0.smithi094.stdout:0/808: write d14/d18/fc2 [1606298,129184] 0 2024-03-24T10:46:02.086 INFO:tasks.workunit.client.0.smithi094.stdout:2/783: rename d18/d1e/d65/fd6 to d18/d7f/d88/d8a/db2/fe2 0 2024-03-24T10:46:02.088 INFO:tasks.workunit.client.0.smithi094.stdout:0/809: getdents d14 0 2024-03-24T10:46:02.090 INFO:tasks.workunit.client.0.smithi094.stdout:9/888: symlink df/d2d/d1f/d11/l11c 0 2024-03-24T10:46:02.091 INFO:tasks.workunit.client.0.smithi094.stdout:9/889: creat df/d2d/d1f/d58/d70/dab/d80/f11d x:0 0 0 2024-03-24T10:46:02.091 INFO:tasks.workunit.client.0.smithi094.stdout:9/890: write df/d2d/d34/d2c/f5f [4369028,50423] 0 2024-03-24T10:46:02.092 INFO:tasks.workunit.client.0.smithi094.stdout:2/784: creat d18/d1e/fe3 x:0 0 0 2024-03-24T10:46:02.093 INFO:tasks.workunit.client.0.smithi094.stdout:3/746: dwrite d0/db/dc/d28/d3c/d55/fe7 [0,4194304] 0 2024-03-24T10:46:02.093 INFO:tasks.workunit.client.0.smithi094.stdout:3/747: fsync d0/f1e 0 2024-03-24T10:46:02.093 INFO:tasks.workunit.client.0.smithi094.stdout:0/810: symlink d14/d22/d28/dac/l11a 0 2024-03-24T10:46:02.094 INFO:tasks.workunit.client.0.smithi094.stdout:1/715: dread d4/d9/db/d31/f48 [0,4194304] 0 2024-03-24T10:46:02.098 INFO:tasks.workunit.client.0.smithi094.stdout:9/891: symlink df/d2d/d1f/d58/d5a/d69/l11e 0 2024-03-24T10:46:02.100 INFO:tasks.workunit.client.0.smithi094.stdout:2/785: creat d18/fe4 x:0 0 0 2024-03-24T10:46:02.102 INFO:tasks.workunit.client.0.smithi094.stdout:3/748: stat d0/db/cf5 0 2024-03-24T10:46:02.102 INFO:tasks.workunit.client.0.smithi094.stdout:3/749: write d0/f14 [2793156,8607] 0 2024-03-24T10:46:02.105 INFO:tasks.workunit.client.0.smithi094.stdout:0/811: unlink d14/d22/d28/d2b/lb6 0 2024-03-24T10:46:02.105 INFO:tasks.workunit.client.0.smithi094.stdout:0/812: stat d14/d22/d28/d2b/d41/d53/d105 0 2024-03-24T10:46:02.107 INFO:tasks.workunit.client.0.smithi094.stdout:9/892: creat df/d2d/d1f/d58/d5a/d50/da5/dd4/d10a/f11f x:0 0 0 2024-03-24T10:46:02.107 INFO:tasks.workunit.client.0.smithi094.stdout:9/893: fsync df/d2d/d1f/d58/d70/dab/d80/fce 0 2024-03-24T10:46:02.108 INFO:tasks.workunit.client.0.smithi094.stdout:1/716: dwrite d4/d9/de/f3b [0,4194304] 0 2024-03-24T10:46:02.108 INFO:tasks.workunit.client.0.smithi094.stdout:1/717: write d4/d9/db/f18 [1856214,37739] 0 2024-03-24T10:46:02.109 INFO:tasks.workunit.client.0.smithi094.stdout:2/786: link d18/d66/dbe/dc0/c63 d18/d7f/d88/d8a/db2/ce5 0 2024-03-24T10:46:02.112 INFO:tasks.workunit.client.0.smithi094.stdout:3/750: creat d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d4f/f109 x:0 0 0 2024-03-24T10:46:02.112 INFO:tasks.workunit.client.0.smithi094.stdout:3/751: chown d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/de9 267075889 1 2024-03-24T10:46:02.114 INFO:tasks.workunit.client.0.smithi094.stdout:0/813: mkdir d14/d22/d7d/d11b 0 2024-03-24T10:46:02.116 INFO:tasks.workunit.client.0.smithi094.stdout:9/894: symlink df/d2d/d1f/d58/d5a/d50/l120 0 2024-03-24T10:46:02.116 INFO:tasks.workunit.client.0.smithi094.stdout:9/895: stat df/d7b/fbb 0 2024-03-24T10:46:02.116 INFO:tasks.workunit.client.0.smithi094.stdout:9/896: fsync df/d2d/d1f/d11/d43/f76 0 2024-03-24T10:46:02.117 INFO:tasks.workunit.client.0.smithi094.stdout:2/787: mkdir d18/d66/dbe/dc8/de6 0 2024-03-24T10:46:02.117 INFO:tasks.workunit.client.0.smithi094.stdout:2/788: chown d18/d66/d90/cab 0 1 2024-03-24T10:46:02.118 INFO:tasks.workunit.client.0.smithi094.stdout:1/718: dread d4/d9/db/d31/d3c/f50 [0,4194304] 0 2024-03-24T10:46:02.118 INFO:tasks.workunit.client.0.smithi094.stdout:3/752: link d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/d73/cab d0/db/dc/d28/d50/c10a 0 2024-03-24T10:46:02.120 INFO:tasks.workunit.client.0.smithi094.stdout:9/897: rename df/d2d/d1f/d7e/df8/l117 to df/d2d/d1f/d58/d5a/d50/da5/d101/l121 0 2024-03-24T10:46:02.121 INFO:tasks.workunit.client.0.smithi094.stdout:2/789: getdents d18/d77/dcf 0 2024-03-24T10:46:02.121 INFO:tasks.workunit.client.0.smithi094.stdout:2/790: creat d18/d66/dbe/fe7 x:0 0 0 2024-03-24T10:46:02.123 INFO:tasks.workunit.client.0.smithi094.stdout:3/753: link d0/db/dc/d28/d3c/d55/f70 d0/f10b 0 2024-03-24T10:46:02.123 INFO:tasks.workunit.client.0.smithi094.stdout:3/754: chown d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d87/dc5/d101 613065355 1 2024-03-24T10:46:02.123 INFO:tasks.workunit.client.0.smithi094.stdout:0/814: getdents d14/d22/dcb/de5 0 2024-03-24T10:46:02.124 INFO:tasks.workunit.client.0.smithi094.stdout:9/898: mknod df/d2d/d1f/d58/d70/dab/c122 0 2024-03-24T10:46:02.125 INFO:tasks.workunit.client.0.smithi094.stdout:9/899: stat df/d2d/d34 0 2024-03-24T10:46:02.126 INFO:tasks.workunit.client.0.smithi094.stdout:2/791: mknod d18/d48/dc3/ce8 0 2024-03-24T10:46:02.127 INFO:tasks.workunit.client.0.smithi094.stdout:3/755: truncate d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/ff 2035171 0 2024-03-24T10:46:02.129 INFO:tasks.workunit.client.0.smithi094.stdout:0/815: mkdir d14/d22/d28/d2b/d41/d53/d11c 0 2024-03-24T10:46:02.130 INFO:tasks.workunit.client.0.smithi094.stdout:9/900: rename df/d2d/d1f/d58/d70/cb0 to df/d2d/d1f/d11/d53/d112/c123 0 2024-03-24T10:46:02.131 INFO:tasks.workunit.client.0.smithi094.stdout:2/792: mknod d18/d66/d90/d9a/dbc/d71/ce9 0 2024-03-24T10:46:02.131 INFO:tasks.workunit.client.0.smithi094.stdout:2/793: read - d18/d5f/fd7 zero size 2024-03-24T10:46:02.133 INFO:tasks.workunit.client.0.smithi094.stdout:0/816: link d14/d22/d5b/ldc d14/d22/d7d/d11b/l11d 0 2024-03-24T10:46:02.134 INFO:tasks.workunit.client.0.smithi094.stdout:9/901: symlink df/d2d/d1f/d11/d96/df3/l124 0 2024-03-24T10:46:02.134 INFO:tasks.workunit.client.0.smithi094.stdout:1/719: dwrite d4/d9/db/f10 [0,4194304] 0 2024-03-24T10:46:02.134 INFO:tasks.workunit.client.0.smithi094.stdout:2/794: read d18/d48/d56/fb0 [1160296,44375] 0 2024-03-24T10:46:02.136 INFO:tasks.workunit.client.0.smithi094.stdout:0/817: creat d14/d22/d28/da2/d10a/f11e x:0 0 0 2024-03-24T10:46:02.137 INFO:tasks.workunit.client.0.smithi094.stdout:9/902: symlink df/d2d/d1f/d11/d96/df3/l125 0 2024-03-24T10:46:02.139 INFO:tasks.workunit.client.0.smithi094.stdout:2/795: creat d18/d77/dcf/fea x:0 0 0 2024-03-24T10:46:02.140 INFO:tasks.workunit.client.0.smithi094.stdout:3/756: write d0/db/dc/d28/d3c/f6f [4025633,29759] 0 2024-03-24T10:46:02.140 INFO:tasks.workunit.client.0.smithi094.stdout:3/757: chown d0/db/dc/d28/d38/ldc 166 1 2024-03-24T10:46:02.140 INFO:tasks.workunit.client.0.smithi094.stdout:4/829: mkdir d0/d34/dfd 0 2024-03-24T10:46:02.142 INFO:tasks.workunit.client.0.smithi094.stdout:0/818: creat d14/d22/d28/dac/ddf/d107/f11f x:0 0 0 2024-03-24T10:46:02.142 INFO:tasks.workunit.client.0.smithi094.stdout:0/819: chown d14/d22/d5b/c84 21 1 2024-03-24T10:46:02.144 INFO:tasks.workunit.client.0.smithi094.stdout:9/903: mkdir df/d2d/d1f/d58/d70/d104/d126 0 2024-03-24T10:46:02.144 INFO:tasks.workunit.client.0.smithi094.stdout:9/904: fsync df/d2d/d1f/d58/d82/f110 0 2024-03-24T10:46:02.146 INFO:tasks.workunit.client.0.smithi094.stdout:2/796: creat d18/d66/d90/d9a/d74/d7e/dce/feb x:0 0 0 2024-03-24T10:46:02.146 INFO:tasks.workunit.client.0.smithi094.stdout:2/797: fsync d18/d1e/f53 0 2024-03-24T10:46:02.146 INFO:tasks.workunit.client.0.smithi094.stdout:2/798: stat d18/d66/da6 0 2024-03-24T10:46:02.148 INFO:tasks.workunit.client.0.smithi094.stdout:3/758: symlink d0/l10c 0 2024-03-24T10:46:02.150 INFO:tasks.workunit.client.0.smithi094.stdout:4/830: creat d0/d22/de/d28/d2b/d33/da4/df4/ffe x:0 0 0 2024-03-24T10:46:02.150 INFO:tasks.workunit.client.0.smithi094.stdout:1/720: dwrite d4/d9/db/d5d/fed [0,4194304] 0 2024-03-24T10:46:02.151 INFO:tasks.workunit.client.0.smithi094.stdout:1/721: chown d4/d9/de/d3d/d66/dbe/fde 1640 1 2024-03-24T10:46:02.151 INFO:tasks.workunit.client.0.smithi094.stdout:0/820: mkdir d14/d22/d28/d2b/d68/d120 0 2024-03-24T10:46:02.152 INFO:tasks.workunit.client.0.smithi094.stdout:9/905: getdents df/d2d/d1f/d11/d53 0 2024-03-24T10:46:02.154 INFO:tasks.workunit.client.0.smithi094.stdout:2/799: truncate d18/f3b 63646 0 2024-03-24T10:46:02.156 INFO:tasks.workunit.client.0.smithi094.stdout:0/821: dread d14/d22/d28/f31 [0,4194304] 0 2024-03-24T10:46:02.156 INFO:tasks.workunit.client.0.smithi094.stdout:4/831: truncate d0/d22/de/d28/fbb 1853098 0 2024-03-24T10:46:02.160 INFO:tasks.workunit.client.0.smithi094.stdout:1/722: truncate d4/d36/f8e 3883318 0 2024-03-24T10:46:02.162 INFO:tasks.workunit.client.0.smithi094.stdout:9/906: truncate df/d2d/d1f/d11/fbf 2170988 0 2024-03-24T10:46:02.169 INFO:tasks.workunit.client.0.smithi094.stdout:4/832: write d0/d22/de/f30 [1425735,52655] 0 2024-03-24T10:46:02.169 INFO:tasks.workunit.client.0.smithi094.stdout:4/833: read d0/d22/de/d28/d2b/d33/da4/f8b [20563,110161] 0 2024-03-24T10:46:02.170 INFO:tasks.workunit.client.0.smithi094.stdout:4/834: creat d0/d34/d73/fff x:0 0 0 2024-03-24T10:46:02.171 INFO:tasks.workunit.client.0.smithi094.stdout:4/835: dread d0/d5/d1b/f90 [0,4194304] 0 2024-03-24T10:46:02.171 INFO:tasks.workunit.client.0.smithi094.stdout:4/836: chown d0/fb7 6498371 1 2024-03-24T10:46:02.171 INFO:tasks.workunit.client.0.smithi094.stdout:3/759: rename d0/db/dc/d28/d38/d53 to d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/db5/d106/d9b/d10d 0 2024-03-24T10:46:02.171 INFO:tasks.workunit.client.0.smithi094.stdout:3/760: fsync d0/f11 0 2024-03-24T10:46:02.173 INFO:tasks.workunit.client.0.smithi094.stdout:4/837: symlink d0/d22/de4/l100 0 2024-03-24T10:46:02.174 INFO:tasks.workunit.client.0.smithi094.stdout:2/800: rename d18/d48/f98 to d18/d66/d90/d9a/d74/d7e/dca/fec 0 2024-03-24T10:46:02.177 INFO:tasks.workunit.client.0.smithi094.stdout:9/907: rename df/d2d/d1f/d11/d53/fa1 to df/d2d/d34/f127 0 2024-03-24T10:46:02.178 INFO:tasks.workunit.client.0.smithi094.stdout:2/801: mkdir d18/d7f/d88/ded 0 2024-03-24T10:46:02.178 INFO:tasks.workunit.client.0.smithi094.stdout:2/802: chown d18/d66/d90/d9a/dbc/f6f 51676127 1 2024-03-24T10:46:02.178 INFO:tasks.workunit.client.0.smithi094.stdout:4/838: dread d0/d5/d1b/f2e [0,4194304] 0 2024-03-24T10:46:02.179 INFO:tasks.workunit.client.0.smithi094.stdout:9/908: symlink df/d2d/d1f/d58/d5a/d61/l128 0 2024-03-24T10:46:02.180 INFO:tasks.workunit.client.0.smithi094.stdout:1/723: dwrite d4/d36/f44 [4194304,4194304] 0 2024-03-24T10:46:02.181 INFO:tasks.workunit.client.0.smithi094.stdout:9/909: symlink df/d2d/d1f/d11/d53/d88/l129 0 2024-03-24T10:46:02.181 INFO:tasks.workunit.client.0.smithi094.stdout:9/910: rename df/d2d to df/d2d/d1f/d58/d70/d12a 22 2024-03-24T10:46:02.181 INFO:tasks.workunit.client.0.smithi094.stdout:9/911: chown df/d2d/d1f/d11/d43/c102 2233392 1 2024-03-24T10:46:02.185 INFO:tasks.workunit.client.0.smithi094.stdout:4/839: dread d0/d5/d1b/d4f/f85 [0,4194304] 0 2024-03-24T10:46:02.186 INFO:tasks.workunit.client.0.smithi094.stdout:5/802: write d7/d23/d5f/d9/d48/f6b [944560,51463] 0 2024-03-24T10:46:02.190 INFO:tasks.workunit.client.0.smithi094.stdout:5/803: creat d7/dc/db9/f104 x:0 0 0 2024-03-24T10:46:02.190 INFO:tasks.workunit.client.0.smithi094.stdout:5/804: chown d7/d23/d5f/d1b/fe4 2 1 2024-03-24T10:46:02.190 INFO:tasks.workunit.client.0.smithi094.stdout:5/805: read - d7/f4c zero size 2024-03-24T10:46:02.193 INFO:tasks.workunit.client.0.smithi094.stdout:5/806: rename d7/d23/d5f/d1b/fe4 to d7/dc/d56/f105 0 2024-03-24T10:46:02.193 INFO:tasks.workunit.client.0.smithi094.stdout:6/809: sync 2024-03-24T10:46:02.194 INFO:tasks.workunit.client.0.smithi094.stdout:6/810: stat d0/d21/d2a/d55/cc4 0 2024-03-24T10:46:02.198 INFO:tasks.workunit.client.0.smithi094.stdout:3/761: mknod d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/c10e 0 2024-03-24T10:46:02.215 INFO:tasks.workunit.client.0.smithi094.stdout:5/807: dwrite d7/d23/d5f/d14/d36/f47 [0,4194304] 0 2024-03-24T10:46:02.215 INFO:tasks.workunit.client.0.smithi094.stdout:5/808: readlink d7/d23/d5f/lcf 0 2024-03-24T10:46:02.217 INFO:tasks.workunit.client.0.smithi094.stdout:5/809: symlink d7/d23/d2b/d68/d81/l106 0 2024-03-24T10:46:02.217 INFO:tasks.workunit.client.0.smithi094.stdout:5/810: truncate d7/d23/dd5/da9/ff2 4408779 0 2024-03-24T10:46:02.218 INFO:tasks.workunit.client.0.smithi094.stdout:5/811: mknod d7/d23/d5f/d9/d48/d73/c107 0 2024-03-24T10:46:02.218 INFO:tasks.workunit.client.0.smithi094.stdout:5/812: fdatasync d7/d23/f44 0 2024-03-24T10:46:02.218 INFO:tasks.workunit.client.0.smithi094.stdout:5/813: write d7/d23/d5f/d9/d48/d51/f6a [1028259,110149] 0 2024-03-24T10:46:02.219 INFO:tasks.workunit.client.0.smithi094.stdout:5/814: mknod d7/d23/d5f/d14/d9f/c108 0 2024-03-24T10:46:02.220 INFO:tasks.workunit.client.0.smithi094.stdout:5/815: mkdir d7/dc/d100/d109 0 2024-03-24T10:46:02.220 INFO:tasks.workunit.client.0.smithi094.stdout:5/816: mknod d7/d23/d5f/d9/c10a 0 2024-03-24T10:46:02.221 INFO:tasks.workunit.client.0.smithi094.stdout:5/817: creat d7/d26/f10b x:0 0 0 2024-03-24T10:46:02.230 INFO:tasks.workunit.client.0.smithi094.stdout:5/818: write d7/d23/d5f/d14/f85 [3574285,53874] 0 2024-03-24T10:46:02.230 INFO:tasks.workunit.client.0.smithi094.stdout:5/819: dread - d7/fdd zero size 2024-03-24T10:46:02.231 INFO:tasks.workunit.client.0.smithi094.stdout:2/803: dread d18/d48/d56/f58 [0,4194304] 0 2024-03-24T10:46:02.231 INFO:tasks.workunit.client.0.smithi094.stdout:2/804: creat d18/d66/dbe/fee x:0 0 0 2024-03-24T10:46:02.234 INFO:tasks.workunit.client.0.smithi094.stdout:4/840: dwrite d0/d22/de/d28/d3d/d8c/fb6 [4194304,4194304] 0 2024-03-24T10:46:02.236 INFO:tasks.workunit.client.0.smithi094.stdout:9/912: dread df/d2d/d1f/d58/d5a/f4b [0,4194304] 0 2024-03-24T10:46:02.236 INFO:tasks.workunit.client.0.smithi094.stdout:6/811: write d0/da9/fcd [757046,1385] 0 2024-03-24T10:46:02.236 INFO:tasks.workunit.client.0.smithi094.stdout:6/812: creat d0/d30/d50/d4d/d6b/f103 x:0 0 0 2024-03-24T10:46:02.241 INFO:tasks.workunit.client.0.smithi094.stdout:2/805: unlink d18/d7f/lac 0 2024-03-24T10:46:02.241 INFO:tasks.workunit.client.0.smithi094.stdout:2/806: chown d18/d66/dbe/ddf 107097279 1 2024-03-24T10:46:02.241 INFO:tasks.workunit.client.0.smithi094.stdout:2/807: write d18/d66/d90/d9a/d74/d7e/fd3 [143850,121363] 0 2024-03-24T10:46:02.241 INFO:tasks.workunit.client.0.smithi094.stdout:2/808: write d18/d66/dbe/dc0/f6c [3711021,16663] 0 2024-03-24T10:46:02.245 INFO:tasks.workunit.client.1.smithi096.stdout:4/47: sync 2024-03-24T10:46:02.245 INFO:tasks.workunit.client.1.smithi096.stdout:3/68: sync 2024-03-24T10:46:02.245 INFO:tasks.workunit.client.1.smithi096.stdout:1/23: sync 2024-03-24T10:46:02.245 INFO:tasks.workunit.client.1.smithi096.stdout:0/9: sync 2024-03-24T10:46:02.245 INFO:tasks.workunit.client.1.smithi096.stdout:4/48: fdatasync d1/f5 0 2024-03-24T10:46:02.246 INFO:tasks.workunit.client.1.smithi096.stdout:9/21: sync 2024-03-24T10:46:02.246 INFO:tasks.workunit.client.1.smithi096.stdout:6/23: sync 2024-03-24T10:46:02.246 INFO:tasks.workunit.client.1.smithi096.stdout:1/24: creat d3/f8 x:0 0 0 2024-03-24T10:46:02.246 INFO:tasks.workunit.client.1.smithi096.stdout:6/24: fdatasync f2 0 2024-03-24T10:46:02.246 INFO:tasks.workunit.client.1.smithi096.stdout:6/25: dread - f1 zero size 2024-03-24T10:46:02.246 INFO:tasks.workunit.client.1.smithi096.stdout:6/26: chown f2 88 1 2024-03-24T10:46:02.246 INFO:tasks.workunit.client.1.smithi096.stdout:8/35: sync 2024-03-24T10:46:02.246 INFO:tasks.workunit.client.1.smithi096.stdout:6/27: truncate d8/fa 468280 0 2024-03-24T10:46:02.246 INFO:tasks.workunit.client.1.smithi096.stdout:6/28: dread - d8/f9 zero size 2024-03-24T10:46:02.246 INFO:tasks.workunit.client.1.smithi096.stdout:7/17: sync 2024-03-24T10:46:02.246 INFO:tasks.workunit.client.1.smithi096.stdout:2/28: sync 2024-03-24T10:46:02.246 INFO:tasks.workunit.client.1.smithi096.stdout:5/12: sync 2024-03-24T10:46:02.247 INFO:tasks.workunit.client.0.smithi094.stdout:9/913: symlink df/d2d/d1f/d58/d5a/l12b 0 2024-03-24T10:46:02.247 INFO:tasks.workunit.client.1.smithi096.stdout:9/22: write f3 [747497,31710] 0 2024-03-24T10:46:02.249 INFO:tasks.workunit.client.1.smithi096.stdout:3/69: getdents d0/d9 0 2024-03-24T10:46:02.250 INFO:tasks.workunit.client.0.smithi094.stdout:6/813: mknod d0/d30/d50/c104 0 2024-03-24T10:46:02.250 INFO:tasks.workunit.client.0.smithi094.stdout:6/814: fdatasync d0/d78/f19 0 2024-03-24T10:46:02.252 INFO:tasks.workunit.client.0.smithi094.stdout:4/841: dwrite d0/d22/de/d28/d2b/d33/da4/df4/ffe [0,4194304] 0 2024-03-24T10:46:02.255 INFO:tasks.workunit.client.1.smithi096.stdout:0/10: creat d0/d2/f3 x:0 0 0 2024-03-24T10:46:02.258 INFO:tasks.workunit.client.0.smithi094.stdout:9/914: creat df/d2d/d1f/d58/d5a/d61/d87/f12c x:0 0 0 2024-03-24T10:46:02.260 INFO:tasks.workunit.client.0.smithi094.stdout:6/815: mknod d0/d1a/dca/c105 0 2024-03-24T10:46:02.262 INFO:tasks.workunit.client.1.smithi096.stdout:4/49: creat d1/fd x:0 0 0 2024-03-24T10:46:02.267 INFO:tasks.workunit.client.1.smithi096.stdout:6/29: mkdir d8/dc 0 2024-03-24T10:46:02.269 INFO:tasks.workunit.client.0.smithi094.stdout:4/842: link d0/d22/de/d28/d2b/d33/f54 d0/d22/de/d28/d2b/d33/f101 0 2024-03-24T10:46:02.271 INFO:tasks.workunit.client.1.smithi096.stdout:8/36: rename c3 to d8/cb 0 2024-03-24T10:46:02.273 INFO:tasks.workunit.client.0.smithi094.stdout:4/843: dread d0/d22/de/d28/f8f [0,4194304] 0 2024-03-24T10:46:02.273 INFO:tasks.workunit.client.0.smithi094.stdout:9/915: mkdir df/d2d/d1f/d11/d53/de3/d12d 0 2024-03-24T10:46:02.273 INFO:tasks.workunit.client.1.smithi096.stdout:2/29: creat d5/fb x:0 0 0 2024-03-24T10:46:02.274 INFO:tasks.workunit.client.0.smithi094.stdout:6/816: symlink d0/d30/de6/l106 0 2024-03-24T10:46:02.275 INFO:tasks.workunit.client.1.smithi096.stdout:9/23: mknod c6 0 2024-03-24T10:46:02.277 INFO:tasks.workunit.client.1.smithi096.stdout:3/70: mknod d0/c16 0 2024-03-24T10:46:02.278 INFO:tasks.workunit.client.0.smithi094.stdout:4/844: rmdir d0/d5/d7d/d9c 39 2024-03-24T10:46:02.279 INFO:tasks.workunit.client.1.smithi096.stdout:0/11: rmdir d0 39 2024-03-24T10:46:02.281 INFO:tasks.workunit.client.0.smithi094.stdout:9/916: unlink df/d2d/d1f/d11/d53/d88/f89 0 2024-03-24T10:46:02.282 INFO:tasks.workunit.client.0.smithi094.stdout:9/917: chown df/d2d/d1f/d58/d70/dab/d80/fce 539521 1 2024-03-24T10:46:02.282 INFO:tasks.workunit.client.0.smithi094.stdout:9/918: chown df/f52 71436511 1 2024-03-24T10:46:02.282 INFO:tasks.workunit.client.0.smithi094.stdout:9/919: truncate df/d7b/ffd 1646635 0 2024-03-24T10:46:02.282 INFO:tasks.workunit.client.1.smithi096.stdout:4/50: creat d1/fe x:0 0 0 2024-03-24T10:46:02.285 INFO:tasks.workunit.client.0.smithi094.stdout:4/845: mknod d0/d5/d7d/d97/c102 0 2024-03-24T10:46:02.285 INFO:tasks.workunit.client.0.smithi094.stdout:4/846: write d0/d34/f82 [2727864,58745] 0 2024-03-24T10:46:02.286 INFO:tasks.workunit.client.1.smithi096.stdout:8/37: creat d8/fc x:0 0 0 2024-03-24T10:46:02.289 INFO:tasks.workunit.client.0.smithi094.stdout:9/920: link df/l21 df/dda/df6/l12e 0 2024-03-24T10:46:02.291 INFO:tasks.workunit.client.1.smithi096.stdout:8/38: dread f5 [0,4194304] 0 2024-03-24T10:46:02.292 INFO:tasks.workunit.client.1.smithi096.stdout:8/39: fsync f5 0 2024-03-24T10:46:02.292 INFO:tasks.workunit.client.1.smithi096.stdout:7/18: rename d0/l2 to d0/d1/l6 0 2024-03-24T10:46:02.293 INFO:tasks.workunit.client.0.smithi094.stdout:6/817: rmdir d0/d1a/d35/d96 39 2024-03-24T10:46:02.296 INFO:tasks.workunit.client.1.smithi096.stdout:6/30: write f7 [307029,71749] 0 2024-03-24T10:46:02.297 INFO:tasks.workunit.client.0.smithi094.stdout:4/847: truncate d0/d22/de/d28/d2b/f51 1071229 0 2024-03-24T10:46:02.297 INFO:tasks.workunit.client.0.smithi094.stdout:4/848: write d0/f60 [622824,75792] 0 2024-03-24T10:46:02.300 INFO:tasks.workunit.client.1.smithi096.stdout:2/30: mkdir d5/dc 0 2024-03-24T10:46:02.300 INFO:tasks.workunit.client.0.smithi094.stdout:2/809: dwrite d18/d66/d90/d9a/dbc/f46 [0,4194304] 0 2024-03-24T10:46:02.304 INFO:tasks.workunit.client.1.smithi096.stdout:9/24: rename c4 to c7 0 2024-03-24T10:46:02.305 INFO:tasks.workunit.client.0.smithi094.stdout:5/820: getdents d7/d23/d2b/d68/d81 0 2024-03-24T10:46:02.305 INFO:tasks.workunit.client.0.smithi094.stdout:5/821: stat d7/d23/d5f/d9/c10a 0 2024-03-24T10:46:02.305 INFO:tasks.workunit.client.0.smithi094.stdout:5/822: truncate d7/d23/d2b/d68/f9e 682839 0 2024-03-24T10:46:02.307 INFO:tasks.workunit.client.0.smithi094.stdout:9/921: unlink df/d2d/d1f/d58/d5a/d69/d7a/f7d 0 2024-03-24T10:46:02.308 INFO:tasks.workunit.client.1.smithi096.stdout:3/71: creat d0/d13/f17 x:0 0 0 2024-03-24T10:46:02.310 INFO:tasks.workunit.client.1.smithi096.stdout:0/12: mknod d0/d2/c4 0 2024-03-24T10:46:02.312 INFO:tasks.workunit.client.0.smithi094.stdout:4/849: rename d0/d22/de/d28/d2b/ce1 to d0/d22/de/d28/d2b/d33/da1/c103 0 2024-03-24T10:46:02.312 INFO:tasks.workunit.client.0.smithi094.stdout:4/850: stat d0/d34/d73/d93/fb4 0 2024-03-24T10:46:02.313 INFO:tasks.workunit.client.1.smithi096.stdout:4/51: dwrite d1/f8 [0,4194304] 0 2024-03-24T10:46:02.316 INFO:tasks.workunit.client.1.smithi096.stdout:6/31: creat d8/dc/fd x:0 0 0 2024-03-24T10:46:02.317 INFO:tasks.workunit.client.0.smithi094.stdout:6/818: dwrite d0/d78/da/d23/dc3/fbe [0,4194304] 0 2024-03-24T10:46:02.318 INFO:tasks.workunit.client.1.smithi096.stdout:2/31: mknod d5/cd 0 2024-03-24T10:46:02.321 INFO:tasks.workunit.client.0.smithi094.stdout:7/801: sync 2024-03-24T10:46:02.321 INFO:tasks.workunit.client.0.smithi094.stdout:8/857: sync 2024-03-24T10:46:02.323 INFO:tasks.workunit.client.1.smithi096.stdout:3/72: truncate d0/f7 131686 0 2024-03-24T10:46:02.326 INFO:tasks.workunit.client.1.smithi096.stdout:0/13: creat d0/f5 x:0 0 0 2024-03-24T10:46:02.328 INFO:tasks.workunit.client.1.smithi096.stdout:4/52: creat d1/ff x:0 0 0 2024-03-24T10:46:02.330 INFO:tasks.workunit.client.1.smithi096.stdout:2/32: mkdir d5/de 0 2024-03-24T10:46:02.331 INFO:tasks.workunit.client.1.smithi096.stdout:6/32: dwrite d8/fb [0,4194304] 0 2024-03-24T10:46:02.331 INFO:tasks.workunit.client.0.smithi094.stdout:2/810: getdents d18/d66/d90/d9a 0 2024-03-24T10:46:02.332 INFO:tasks.workunit.client.1.smithi096.stdout:8/40: dread f1 [0,4194304] 0 2024-03-24T10:46:02.333 INFO:tasks.workunit.client.1.smithi096.stdout:1/25: getdents d3 0 2024-03-24T10:46:02.335 INFO:tasks.workunit.client.0.smithi094.stdout:5/823: creat d7/dc/d100/d109/f10c x:0 0 0 2024-03-24T10:46:02.339 INFO:tasks.workunit.client.1.smithi096.stdout:3/73: unlink d0/l15 0 2024-03-24T10:46:02.340 INFO:tasks.workunit.client.0.smithi094.stdout:9/922: unlink df/d2d/d1f/d58/d5a/l12b 0 2024-03-24T10:46:02.340 INFO:tasks.workunit.client.0.smithi094.stdout:9/923: write df/d7b/fd3 [589769,94036] 0 2024-03-24T10:46:02.340 INFO:tasks.workunit.client.0.smithi094.stdout:9/924: readlink df/d2d/d1f/d11/d53/d88/l129 0 2024-03-24T10:46:02.345 INFO:tasks.workunit.client.1.smithi096.stdout:2/33: creat d5/dc/ff x:0 0 0 2024-03-24T10:46:02.347 INFO:tasks.workunit.client.0.smithi094.stdout:6/819: getdents d0/d78/da 0 2024-03-24T10:46:02.347 INFO:tasks.workunit.client.0.smithi094.stdout:6/820: creat d0/d1a/d35/f107 x:0 0 0 2024-03-24T10:46:02.347 INFO:tasks.workunit.client.0.smithi094.stdout:6/821: write d0/d1a/fd7 [958778,56687] 0 2024-03-24T10:46:02.347 INFO:tasks.workunit.client.0.smithi094.stdout:6/822: write d0/d1a/d35/fba [731218,60521] 0 2024-03-24T10:46:02.347 INFO:tasks.workunit.client.0.smithi094.stdout:6/823: write d0/d30/de6/d9c/fa4 [375587,90939] 0 2024-03-24T10:46:02.348 INFO:tasks.workunit.client.0.smithi094.stdout:6/824: write d0/d78/da/d23/dc3/f6a [43106,84489] 0 2024-03-24T10:46:02.348 INFO:tasks.workunit.client.0.smithi094.stdout:6/825: write d0/da9/fcd [4964849,60211] 0 2024-03-24T10:46:02.349 INFO:tasks.workunit.client.0.smithi094.stdout:7/802: mknod d1/d9/d16/d21/d49/dfa/c111 0 2024-03-24T10:46:02.351 INFO:tasks.workunit.client.0.smithi094.stdout:7/803: dread d1/d9/d25/d3d/f73 [0,4194304] 0 2024-03-24T10:46:02.351 INFO:tasks.workunit.client.0.smithi094.stdout:8/858: mknod df/d25/d28/d31/d47/d66/da3/da8/d117/dc1/d111/c120 0 2024-03-24T10:46:02.353 INFO:tasks.workunit.client.1.smithi096.stdout:6/33: symlink d8/le 0 2024-03-24T10:46:02.354 INFO:tasks.workunit.client.1.smithi096.stdout:6/34: dread d8/fa [0,4194304] 0 2024-03-24T10:46:02.354 INFO:tasks.workunit.client.0.smithi094.stdout:2/811: mkdir d18/d66/d90/d9a/def 0 2024-03-24T10:46:02.355 INFO:tasks.workunit.client.1.smithi096.stdout:8/41: symlink d8/ld 0 2024-03-24T10:46:02.357 INFO:tasks.workunit.client.1.smithi096.stdout:1/26: creat d3/f9 x:0 0 0 2024-03-24T10:46:02.358 INFO:tasks.workunit.client.1.smithi096.stdout:1/27: truncate d3/f9 649376 0 2024-03-24T10:46:02.358 INFO:tasks.workunit.client.1.smithi096.stdout:1/28: write d3/f9 [233569,94326] 0 2024-03-24T10:46:02.358 INFO:tasks.workunit.client.1.smithi096.stdout:1/29: chown d3 6663738 1 2024-03-24T10:46:02.358 INFO:tasks.workunit.client.1.smithi096.stdout:1/30: write d3/f9 [1688041,74302] 0 2024-03-24T10:46:02.359 INFO:tasks.workunit.client.0.smithi094.stdout:9/925: mknod df/d2d/d1f/d58/d5a/d61/d87/d10e/c12f 0 2024-03-24T10:46:02.361 INFO:tasks.workunit.client.1.smithi096.stdout:3/74: mknod d0/d13/c18 0 2024-03-24T10:46:02.361 INFO:tasks.workunit.client.1.smithi096.stdout:3/75: stat d0/c10 0 2024-03-24T10:46:02.362 INFO:tasks.workunit.client.0.smithi094.stdout:4/851: symlink d0/d22/de/d28/d2b/d33/da4/da5/l104 0 2024-03-24T10:46:02.362 INFO:tasks.workunit.client.0.smithi094.stdout:4/852: write d0/d22/de/d28/d2b/d33/da1/fda [924532,76370] 0 2024-03-24T10:46:02.377 INFO:tasks.workunit.client.1.smithi096.stdout:4/53: rename d1/f5 to d1/f10 0 2024-03-24T10:46:02.377 INFO:tasks.workunit.client.1.smithi096.stdout:4/54: write d1/fe [844576,28820] 0 2024-03-24T10:46:02.377 INFO:tasks.workunit.client.1.smithi096.stdout:4/55: write d1/fe [1097071,72824] 0 2024-03-24T10:46:02.380 INFO:tasks.workunit.client.0.smithi094.stdout:6/826: unlink d0/d78/da/d23/d34/l47 0 2024-03-24T10:46:02.382 INFO:tasks.workunit.client.1.smithi096.stdout:2/34: symlink d5/l10 0 2024-03-24T10:46:02.382 INFO:tasks.workunit.client.1.smithi096.stdout:7/19: getdents d0 0 2024-03-24T10:46:02.383 INFO:tasks.workunit.client.0.smithi094.stdout:9/926: dwrite df/d2d/d1f/d58/d5a/d50/da5/d101/f108 [0,4194304] 0 2024-03-24T10:46:02.383 INFO:tasks.workunit.client.1.smithi096.stdout:3/76: dwrite d0/f1 [0,4194304] 0 2024-03-24T10:46:02.384 INFO:tasks.workunit.client.0.smithi094.stdout:7/804: mkdir d1/d9/d25/d3d/d112 0 2024-03-24T10:46:02.387 INFO:tasks.workunit.client.0.smithi094.stdout:8/859: symlink df/d25/d28/d31/d47/ddf/dfd/l121 0 2024-03-24T10:46:02.387 INFO:tasks.workunit.client.0.smithi094.stdout:8/860: stat df/d25/d28/d31/d47/d66/da3/da8/d117/dc1/fec 0 2024-03-24T10:46:02.391 INFO:tasks.workunit.client.1.smithi096.stdout:9/25: rename c7 to c8 0 2024-03-24T10:46:02.395 INFO:tasks.workunit.client.0.smithi094.stdout:4/853: rename d0/d34/d73/d94/ceb to d0/d22/de/d28/d2b/d33/da4/df4/c105 0 2024-03-24T10:46:02.395 INFO:tasks.workunit.client.1.smithi096.stdout:0/14: rmdir d0/d2 39 2024-03-24T10:46:02.396 INFO:tasks.workunit.client.1.smithi096.stdout:7/20: dwrite d0/d1/f5 [0,4194304] 0 2024-03-24T10:46:02.397 INFO:tasks.workunit.client.1.smithi096.stdout:1/31: dread d3/f6 [0,4194304] 0 2024-03-24T10:46:02.397 INFO:tasks.workunit.client.1.smithi096.stdout:1/32: write d3/f6 [1399405,115220] 0 2024-03-24T10:46:02.398 INFO:tasks.workunit.client.0.smithi094.stdout:6/827: mknod d0/d78/da/d23/dc3/c108 0 2024-03-24T10:46:02.400 INFO:tasks.workunit.client.1.smithi096.stdout:2/35: mknod d5/c11 0 2024-03-24T10:46:02.400 INFO:tasks.workunit.client.1.smithi096.stdout:2/36: write d5/dc/ff [938981,21939] 0 2024-03-24T10:46:02.401 INFO:tasks.workunit.client.0.smithi094.stdout:4/854: write d0/d22/de/d28/d2b/d33/da4/df4/ffe [1172803,115191] 0 2024-03-24T10:46:02.401 INFO:tasks.workunit.client.0.smithi094.stdout:4/855: chown d0/d22/de/d28/d2b/d33/da4/df4 11 1 2024-03-24T10:46:02.403 INFO:tasks.workunit.client.1.smithi096.stdout:3/77: write d0/f7 [1139057,63252] 0 2024-03-24T10:46:02.403 INFO:tasks.workunit.client.1.smithi096.stdout:3/78: chown d0/d9/le 303412 1 2024-03-24T10:46:02.403 INFO:tasks.workunit.client.0.smithi094.stdout:3/762: sync 2024-03-24T10:46:02.403 INFO:tasks.workunit.client.0.smithi094.stdout:0/822: sync 2024-03-24T10:46:02.403 INFO:tasks.workunit.client.0.smithi094.stdout:1/724: sync 2024-03-24T10:46:02.403 INFO:tasks.workunit.client.0.smithi094.stdout:1/725: chown d4/d9/d7d/ldc 3 1 2024-03-24T10:46:02.404 INFO:tasks.workunit.client.0.smithi094.stdout:1/726: write d4/d9/db/d31/d3c/f50 [1875232,1267] 0 2024-03-24T10:46:02.410 INFO:tasks.workunit.client.1.smithi096.stdout:0/15: getdents d0/d2 0 2024-03-24T10:46:02.410 INFO:tasks.workunit.client.1.smithi096.stdout:0/16: read - d0/d2/f3 zero size 2024-03-24T10:46:02.410 INFO:tasks.workunit.client.1.smithi096.stdout:0/17: write d0/f5 [777187,89005] 0 2024-03-24T10:46:02.419 INFO:tasks.workunit.client.0.smithi094.stdout:9/927: symlink df/d7b/l130 0 2024-03-24T10:46:02.420 INFO:tasks.workunit.client.1.smithi096.stdout:6/35: truncate d8/fb 3452809 0 2024-03-24T10:46:02.423 INFO:tasks.workunit.client.0.smithi094.stdout:7/805: link d1/d9/d16/d21/d51/d83/fe6 d1/d9/dc3/f113 0 2024-03-24T10:46:02.424 INFO:tasks.workunit.client.1.smithi096.stdout:4/56: mkdir d1/d11 0 2024-03-24T10:46:02.426 INFO:tasks.workunit.client.0.smithi094.stdout:8/861: mknod df/c122 0 2024-03-24T10:46:02.426 INFO:tasks.workunit.client.0.smithi094.stdout:8/862: creat df/d25/d28/d31/d47/dfe/f123 x:0 0 0 2024-03-24T10:46:02.426 INFO:tasks.workunit.client.0.smithi094.stdout:8/863: write df/d25/d28/d31/d47/d66/da3/da8/d117/dc1/fec [150218,114600] 0 2024-03-24T10:46:02.426 INFO:tasks.workunit.client.0.smithi094.stdout:8/864: creat df/d25/d28/d31/d47/dbd/f124 x:0 0 0 2024-03-24T10:46:02.427 INFO:tasks.workunit.client.0.smithi094.stdout:1/727: dwrite d4/d9/db1/db9/fd1 [0,4194304] 0 2024-03-24T10:46:02.427 INFO:tasks.workunit.client.0.smithi094.stdout:1/728: fsync d4/d9/db/d31/d3c/d9f/fee 0 2024-03-24T10:46:02.428 INFO:tasks.workunit.client.1.smithi096.stdout:2/37: mkdir d5/dc/d12 0 2024-03-24T10:46:02.429 INFO:tasks.workunit.client.1.smithi096.stdout:8/42: rmdir d8 39 2024-03-24T10:46:02.431 INFO:tasks.workunit.client.1.smithi096.stdout:3/79: mknod d0/d9/c19 0 2024-03-24T10:46:02.432 INFO:tasks.workunit.client.0.smithi094.stdout:9/928: dwrite df/d2d/d1f/d58/d5a/d50/da5/d101/f116 [0,4194304] 0 2024-03-24T10:46:02.435 INFO:tasks.workunit.client.0.smithi094.stdout:4/856: getdents d0/d5 0 2024-03-24T10:46:02.438 INFO:tasks.workunit.client.1.smithi096.stdout:6/36: symlink d8/lf 0 2024-03-24T10:46:02.439 INFO:tasks.workunit.client.0.smithi094.stdout:3/763: mknod d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/c10f 0 2024-03-24T10:46:02.443 INFO:tasks.workunit.client.0.smithi094.stdout:0/823: creat d14/d22/d28/d2b/d41/df1/f121 x:0 0 0 2024-03-24T10:46:02.444 INFO:tasks.workunit.client.1.smithi096.stdout:4/57: creat d1/d11/f12 x:0 0 0 2024-03-24T10:46:02.444 INFO:tasks.workunit.client.1.smithi096.stdout:4/58: fdatasync d1/fd 0 2024-03-24T10:46:02.444 INFO:tasks.workunit.client.1.smithi096.stdout:0/18: dwrite d0/f5 [0,4194304] 0 2024-03-24T10:46:02.444 INFO:tasks.workunit.client.1.smithi096.stdout:0/19: dread - d0/d2/f3 zero size 2024-03-24T10:46:02.450 INFO:tasks.workunit.client.0.smithi094.stdout:4/857: dread d0/d5/d1b/fc9 [0,4194304] 0 2024-03-24T10:46:02.451 INFO:tasks.workunit.client.1.smithi096.stdout:8/43: creat d8/fe x:0 0 0 2024-03-24T10:46:02.453 INFO:tasks.workunit.client.0.smithi094.stdout:8/865: dwrite df/d25/d28/d31/d47/d66/da3/da8/d117/dc1/fe7 [0,4194304] 0 2024-03-24T10:46:02.453 INFO:tasks.workunit.client.0.smithi094.stdout:1/729: dwrite d4/d9/de/d6f/d62/d84/fe6 [0,4194304] 0 2024-03-24T10:46:02.454 INFO:tasks.workunit.client.1.smithi096.stdout:3/80: write d0/ff [3048901,126148] 0 2024-03-24T10:46:02.454 INFO:tasks.workunit.client.1.smithi096.stdout:9/26: unlink c8 0 2024-03-24T10:46:02.457 INFO:tasks.workunit.client.1.smithi096.stdout:1/33: rename d3/l7 to d3/la 0 2024-03-24T10:46:02.459 INFO:tasks.workunit.client.1.smithi096.stdout:2/38: dwrite f2 [0,4194304] 0 2024-03-24T10:46:02.462 INFO:tasks.workunit.client.0.smithi094.stdout:2/812: rename d18/fcb to d18/d66/d90/d9a/dbc/ff0 0 2024-03-24T10:46:02.464 INFO:tasks.workunit.client.0.smithi094.stdout:1/730: dread d4/d9/db/f1f [0,4194304] 0 2024-03-24T10:46:02.464 INFO:tasks.workunit.client.1.smithi096.stdout:4/59: link d1/d11/f12 d1/d11/f13 0 2024-03-24T10:46:02.464 INFO:tasks.workunit.client.1.smithi096.stdout:1/34: dread d3/f9 [0,4194304] 0 2024-03-24T10:46:02.468 INFO:tasks.workunit.client.1.smithi096.stdout:8/44: unlink f5 0 2024-03-24T10:46:02.468 INFO:tasks.workunit.client.1.smithi096.stdout:8/45: truncate d8/fe 54219 0 2024-03-24T10:46:02.468 INFO:tasks.workunit.client.1.smithi096.stdout:8/46: creat d8/ff x:0 0 0 2024-03-24T10:46:02.468 INFO:tasks.workunit.client.1.smithi096.stdout:8/47: write d8/fc [620337,48066] 0 2024-03-24T10:46:02.469 INFO:tasks.workunit.client.0.smithi094.stdout:9/929: getdents df/d2d/d1f/d7e/df8 0 2024-03-24T10:46:02.469 INFO:tasks.workunit.client.1.smithi096.stdout:3/81: creat d0/d9/f1a x:0 0 0 2024-03-24T10:46:02.469 INFO:tasks.workunit.client.1.smithi096.stdout:3/82: chown d0 1284164565 1 2024-03-24T10:46:02.470 INFO:tasks.workunit.client.0.smithi094.stdout:3/764: mkdir d0/db/dc/d28/d110 0 2024-03-24T10:46:02.471 INFO:tasks.workunit.client.0.smithi094.stdout:3/765: fdatasync d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/d73/fb1 0 2024-03-24T10:46:02.471 INFO:tasks.workunit.client.0.smithi094.stdout:5/824: sync 2024-03-24T10:46:02.471 INFO:tasks.workunit.client.1.smithi096.stdout:9/27: mkdir d9 0 2024-03-24T10:46:02.472 INFO:tasks.workunit.client.1.smithi096.stdout:7/21: truncate d0/d1/f5 1552803 0 2024-03-24T10:46:02.475 INFO:tasks.workunit.client.0.smithi094.stdout:4/858: truncate d0/d5/d1b/f2a 2767503 0 2024-03-24T10:46:02.482 INFO:tasks.workunit.client.0.smithi094.stdout:8/866: creat df/d25/d28/d31/d47/d66/da3/da8/d117/dc1/f125 x:0 0 0 2024-03-24T10:46:02.482 INFO:tasks.workunit.client.0.smithi094.stdout:8/867: chown df/d25/d28/fa7 1988 1 2024-03-24T10:46:02.482 INFO:tasks.workunit.client.0.smithi094.stdout:8/868: dread - df/d25/d28/d31/d47/d66/da3/da8/d117/dc1/fd3 zero size 2024-03-24T10:46:02.482 INFO:tasks.workunit.client.0.smithi094.stdout:8/869: stat df/d25/d28/d31/d47/ddf/dfd/l121 0 2024-03-24T10:46:02.483 INFO:tasks.workunit.client.0.smithi094.stdout:9/930: dwrite df/f56 [4194304,4194304] 0 2024-03-24T10:46:02.483 INFO:tasks.workunit.client.0.smithi094.stdout:9/931: dread - df/d2d/d34/fd6 zero size 2024-03-24T10:46:02.489 INFO:tasks.workunit.client.0.smithi094.stdout:6/828: rename d0/d30/de6/ff2 to d0/d1a/f109 0 2024-03-24T10:46:02.490 INFO:tasks.workunit.client.0.smithi094.stdout:6/829: chown d0/d78/d66 11944097 1 2024-03-24T10:46:02.490 INFO:tasks.workunit.client.0.smithi094.stdout:9/932: write df/d2d/d34/d2c/f4c [96449,113260] 0 2024-03-24T10:46:02.490 INFO:tasks.workunit.client.0.smithi094.stdout:6/830: dread d0/d30/d50/fac [0,4194304] 0 2024-03-24T10:46:02.491 INFO:tasks.workunit.client.0.smithi094.stdout:6/831: creat d0/da9/f10a x:0 0 0 2024-03-24T10:46:02.491 INFO:tasks.workunit.client.0.smithi094.stdout:6/832: truncate d0/d30/d50/f81 1460824 0 2024-03-24T10:46:02.491 INFO:tasks.workunit.client.0.smithi094.stdout:2/813: truncate d18/d66/d90/d9a/dbc/f46 4093916 0 2024-03-24T10:46:02.494 INFO:tasks.workunit.client.0.smithi094.stdout:1/731: mkdir d4/d9/de/d3d/dae/dfb 0 2024-03-24T10:46:02.494 INFO:tasks.workunit.client.0.smithi094.stdout:1/732: write d4/d9/de/f3b [4255445,122483] 0 2024-03-24T10:46:02.494 INFO:tasks.workunit.client.0.smithi094.stdout:1/733: chown d4/d9/f32 0 1 2024-03-24T10:46:02.495 INFO:tasks.workunit.client.1.smithi096.stdout:3/83: rename d0/d13/c14 to d0/d13/c1b 0 2024-03-24T10:46:02.497 INFO:tasks.workunit.client.0.smithi094.stdout:9/933: write df/d2d/fae [3070883,20929] 0 2024-03-24T10:46:02.497 INFO:tasks.workunit.client.0.smithi094.stdout:9/934: readlink df/d7b/lb5 0 2024-03-24T10:46:02.498 INFO:tasks.workunit.client.0.smithi094.stdout:3/766: unlink d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/fec 0 2024-03-24T10:46:02.499 INFO:tasks.workunit.client.1.smithi096.stdout:9/28: mknod d9/ca 0 2024-03-24T10:46:02.501 INFO:tasks.workunit.client.0.smithi094.stdout:5/825: symlink d7/d23/d2b/d68/d81/l10d 0 2024-03-24T10:46:02.502 INFO:tasks.workunit.client.1.smithi096.stdout:7/22: mkdir d0/d1/d7 0 2024-03-24T10:46:02.505 INFO:tasks.workunit.client.1.smithi096.stdout:8/48: dread f0 [0,4194304] 0 2024-03-24T10:46:02.509 INFO:tasks.workunit.client.1.smithi096.stdout:6/37: rename d8/dc to d8/d10 0 2024-03-24T10:46:02.510 INFO:tasks.workunit.client.1.smithi096.stdout:1/35: dwrite d3/f8 [0,4194304] 0 2024-03-24T10:46:02.510 INFO:tasks.workunit.client.1.smithi096.stdout:1/36: chown d3/f8 7 1 2024-03-24T10:46:02.510 INFO:tasks.workunit.client.0.smithi094.stdout:0/824: rmdir d14/d22/d28/d2b 39 2024-03-24T10:46:02.512 INFO:tasks.workunit.client.0.smithi094.stdout:1/734: write d4/f2a [1367615,19804] 0 2024-03-24T10:46:02.512 INFO:tasks.workunit.client.0.smithi094.stdout:2/814: dwrite d18/d66/d90/fda [0,4194304] 0 2024-03-24T10:46:02.513 INFO:tasks.workunit.client.0.smithi094.stdout:8/870: link df/d25/d36/d3f/cdd df/d12/d5b/dea/c126 0 2024-03-24T10:46:02.514 INFO:tasks.workunit.client.1.smithi096.stdout:3/84: mknod d0/d9/c1c 0 2024-03-24T10:46:02.514 INFO:tasks.workunit.client.0.smithi094.stdout:8/871: dread df/d25/d28/d31/fd6 [0,4194304] 0 2024-03-24T10:46:02.514 INFO:tasks.workunit.client.0.smithi094.stdout:8/872: read df/f89 [510099,68925] 0 2024-03-24T10:46:02.515 INFO:tasks.workunit.client.0.smithi094.stdout:8/873: chown df/d25/daf/fb0 22993683 1 2024-03-24T10:46:02.515 INFO:tasks.workunit.client.1.smithi096.stdout:9/29: mkdir d9/db 0 2024-03-24T10:46:02.516 INFO:tasks.workunit.client.1.smithi096.stdout:9/30: stat d9/db 0 2024-03-24T10:46:02.517 INFO:tasks.workunit.client.0.smithi094.stdout:3/767: dwrite d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/f4c [0,4194304] 0 2024-03-24T10:46:02.519 INFO:tasks.workunit.client.0.smithi094.stdout:7/806: rename d1/d9/d25/d3d/f98 to d1/d9/d16/d21/d51/d83/d39/d44/dd9/f114 0 2024-03-24T10:46:02.522 INFO:tasks.workunit.client.1.smithi096.stdout:9/31: dread f3 [0,4194304] 0 2024-03-24T10:46:02.525 INFO:tasks.workunit.client.1.smithi096.stdout:6/38: rename d8/fa to d8/d10/f11 0 2024-03-24T10:46:02.526 INFO:tasks.workunit.client.0.smithi094.stdout:6/833: creat d0/d30/d50/d4d/f10b x:0 0 0 2024-03-24T10:46:02.526 INFO:tasks.workunit.client.0.smithi094.stdout:6/834: getdents d0/d78/da/dc9 0 2024-03-24T10:46:02.531 INFO:tasks.workunit.client.1.smithi096.stdout:1/37: creat d3/fb x:0 0 0 2024-03-24T10:46:02.568 INFO:tasks.workunit.client.1.smithi096.stdout:4/60: rename d1/l6 to d1/l14 0 2024-03-24T10:46:02.568 INFO:tasks.workunit.client.1.smithi096.stdout:4/61: chown d1 0 1 2024-03-24T10:46:02.568 INFO:tasks.workunit.client.1.smithi096.stdout:4/62: write d1/d11/f12 [124877,91664] 0 2024-03-24T10:46:02.568 INFO:tasks.workunit.client.1.smithi096.stdout:7/23: dwrite d0/d1/f5 [0,4194304] 0 2024-03-24T10:46:02.568 INFO:tasks.workunit.client.1.smithi096.stdout:0/20: truncate d0/f5 3730824 0 2024-03-24T10:46:02.568 INFO:tasks.workunit.client.1.smithi096.stdout:0/21: readlink - no filename 2024-03-24T10:46:02.568 INFO:tasks.workunit.client.1.smithi096.stdout:3/85: link d0/d9/c19 d0/d13/c1d 0 2024-03-24T10:46:02.568 INFO:tasks.workunit.client.1.smithi096.stdout:3/86: fdatasync d0/f7 0 2024-03-24T10:46:02.568 INFO:tasks.workunit.client.1.smithi096.stdout:3/87: chown d0/d13/c1b 83 1 2024-03-24T10:46:02.568 INFO:tasks.workunit.client.1.smithi096.stdout:3/88: mkdir d0/d13/d1e 0 2024-03-24T10:46:02.568 INFO:tasks.workunit.client.1.smithi096.stdout:3/89: write d0/d9/f1a [805837,126607] 0 2024-03-24T10:46:02.569 INFO:tasks.workunit.client.1.smithi096.stdout:3/90: readlink d0/l5 0 2024-03-24T10:46:02.569 INFO:tasks.workunit.client.1.smithi096.stdout:7/24: read d0/d1/f5 [1734909,83261] 0 2024-03-24T10:46:02.569 INFO:tasks.workunit.client.1.smithi096.stdout:7/25: write d0/d1/f5 [4524482,106924] 0 2024-03-24T10:46:02.570 INFO:tasks.workunit.client.0.smithi094.stdout:0/825: mkdir d14/d22/d28/d2b/d68/d122 0 2024-03-24T10:46:02.570 INFO:tasks.workunit.client.0.smithi094.stdout:1/735: link d4/d9/de/d3d/c4d d4/d9/db/d85/df5/cfc 0 2024-03-24T10:46:02.570 INFO:tasks.workunit.client.0.smithi094.stdout:1/736: read - d4/d9/de/d6f/d62/d99/fce zero size 2024-03-24T10:46:02.570 INFO:tasks.workunit.client.0.smithi094.stdout:1/737: chown d4/c72 1077843 1 2024-03-24T10:46:02.570 INFO:tasks.workunit.client.0.smithi094.stdout:2/815: creat d18/d48/d56/ff1 x:0 0 0 2024-03-24T10:46:02.570 INFO:tasks.workunit.client.0.smithi094.stdout:4/859: rename d0/d22/de/d28/d2b/d33/da4/dcb to d0/d22/de4/dfa/d106 0 2024-03-24T10:46:02.570 INFO:tasks.workunit.client.0.smithi094.stdout:4/860: read d0/d5/d7d/fdc [39649,58928] 0 2024-03-24T10:46:02.570 INFO:tasks.workunit.client.0.smithi094.stdout:4/861: fsync d0/d22/de/d28/d2b/d33/f54 0 2024-03-24T10:46:02.570 INFO:tasks.workunit.client.0.smithi094.stdout:4/862: chown d0/d5/d1b/d4f/dd6/lf5 45811 1 2024-03-24T10:46:02.570 INFO:tasks.workunit.client.0.smithi094.stdout:7/807: unlink d1/d9/d16/d21/d51/d83/d39/d44/dd9/d8f/ld3 0 2024-03-24T10:46:02.570 INFO:tasks.workunit.client.0.smithi094.stdout:7/808: chown d1/d9/fd1 24460906 1 2024-03-24T10:46:02.571 INFO:tasks.workunit.client.0.smithi094.stdout:6/835: unlink d0/d21/d2a/fd6 0 2024-03-24T10:46:02.571 INFO:tasks.workunit.client.0.smithi094.stdout:9/935: unlink df/d2d/d1f/d58/d5a/d50/da5/d101/f116 0 2024-03-24T10:46:02.571 INFO:tasks.workunit.client.0.smithi094.stdout:9/936: stat df/d2d/d1f/d58/d5a/f6a 0 2024-03-24T10:46:02.571 INFO:tasks.workunit.client.0.smithi094.stdout:1/738: link d4/d9/d37/d33/cc4 d4/d9/de/d6f/d5f/d8d/cfd 0 2024-03-24T10:46:02.571 INFO:tasks.workunit.client.0.smithi094.stdout:1/739: stat d4/d9/db/d5d/d63/c7f 0 2024-03-24T10:46:02.571 INFO:tasks.workunit.client.0.smithi094.stdout:1/740: chown d4/d9/de/d3d/faf 45 1 2024-03-24T10:46:02.571 INFO:tasks.workunit.client.0.smithi094.stdout:2/816: read d18/d66/d90/d9a/d74/fd1 [124632,88] 0 2024-03-24T10:46:02.571 INFO:tasks.workunit.client.0.smithi094.stdout:2/817: creat d18/d66/d90/ff2 x:0 0 0 2024-03-24T10:46:02.571 INFO:tasks.workunit.client.0.smithi094.stdout:2/818: readlink l13 0 2024-03-24T10:46:02.571 INFO:tasks.workunit.client.0.smithi094.stdout:8/874: dwrite df/d25/d36/d3f/fdb [4194304,4194304] 0 2024-03-24T10:46:02.572 INFO:tasks.workunit.client.1.smithi096.stdout:9/32: dwrite f5 [0,4194304] 0 2024-03-24T10:46:02.572 INFO:tasks.workunit.client.1.smithi096.stdout:9/33: chown d9 5 1 2024-03-24T10:46:02.572 INFO:tasks.workunit.client.1.smithi096.stdout:3/91: unlink d0/cc 0 2024-03-24T10:46:02.572 INFO:tasks.workunit.client.1.smithi096.stdout:3/92: fdatasync d0/f1 0 2024-03-24T10:46:02.572 INFO:tasks.workunit.client.1.smithi096.stdout:3/93: chown d0/d13/c18 202235435 1 2024-03-24T10:46:02.572 INFO:tasks.workunit.client.1.smithi096.stdout:3/94: creat d0/d13/f1f x:0 0 0 2024-03-24T10:46:02.573 INFO:tasks.workunit.client.0.smithi094.stdout:5/826: rename d7/d23/d5f/d9/f78 to d7/d23/d5f/d9/d48/f10e 0 2024-03-24T10:46:02.579 INFO:tasks.workunit.client.0.smithi094.stdout:6/836: dwrite d0/d21/d2a/dcb/fef [0,4194304] 0 2024-03-24T10:46:02.580 INFO:tasks.workunit.client.1.smithi096.stdout:2/39: dwrite f2 [4194304,4194304] 0 2024-03-24T10:46:02.580 INFO:tasks.workunit.client.1.smithi096.stdout:7/26: fsync d0/d1/f5 0 2024-03-24T10:46:02.581 INFO:tasks.workunit.client.0.smithi094.stdout:0/826: dwrite d14/d22/d28/d2b/f3b [0,4194304] 0 2024-03-24T10:46:02.581 INFO:tasks.workunit.client.0.smithi094.stdout:0/827: chown d14/d22/d32 42 1 2024-03-24T10:46:02.581 INFO:tasks.workunit.client.0.smithi094.stdout:7/809: symlink d1/d9/d16/d21/d51/d83/d39/d44/dd9/da5/l115 0 2024-03-24T10:46:02.583 INFO:tasks.workunit.client.0.smithi094.stdout:1/741: dread d4/f16 [0,4194304] 0 2024-03-24T10:46:02.583 INFO:tasks.workunit.client.1.smithi096.stdout:9/34: write f5 [2939670,31985] 0 2024-03-24T10:46:02.583 INFO:tasks.workunit.client.1.smithi096.stdout:9/35: truncate f3 2038082 0 2024-03-24T10:46:02.584 INFO:tasks.workunit.client.0.smithi094.stdout:9/937: mkdir df/d2d/d1f/d58/d5a/d61/d131 0 2024-03-24T10:46:02.584 INFO:tasks.workunit.client.0.smithi094.stdout:9/938: creat df/d2d/d1f/d58/d70/de2/f132 x:0 0 0 2024-03-24T10:46:02.584 INFO:tasks.workunit.client.0.smithi094.stdout:9/939: stat df/d2d/d1f/d58/d5a/d61 0 2024-03-24T10:46:02.584 INFO:tasks.workunit.client.0.smithi094.stdout:9/940: write df/d2d/d1f/d11/f75 [2435743,43375] 0 2024-03-24T10:46:02.585 INFO:tasks.workunit.client.0.smithi094.stdout:9/941: write df/d2d/d34/d2c/fd0 [805259,60362] 0 2024-03-24T10:46:02.585 INFO:tasks.workunit.client.0.smithi094.stdout:3/768: dwrite d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d87/f9c [0,4194304] 0 2024-03-24T10:46:02.585 INFO:tasks.workunit.client.1.smithi096.stdout:6/39: dwrite f7 [0,4194304] 0 2024-03-24T10:46:02.587 INFO:tasks.workunit.client.1.smithi096.stdout:2/40: creat d5/de/f13 x:0 0 0 2024-03-24T10:46:02.587 INFO:tasks.workunit.client.1.smithi096.stdout:2/41: read - d5/de/f13 zero size 2024-03-24T10:46:02.587 INFO:tasks.workunit.client.1.smithi096.stdout:2/42: write d5/fb [376757,39327] 0 2024-03-24T10:46:02.588 INFO:tasks.workunit.client.0.smithi094.stdout:9/942: dread df/d2d/d1f/d58/d5a/d69/f98 [0,4194304] 0 2024-03-24T10:46:02.588 INFO:tasks.workunit.client.0.smithi094.stdout:9/943: creat df/d2d/d1f/d58/d70/d104/f133 x:0 0 0 2024-03-24T10:46:02.588 INFO:tasks.workunit.client.0.smithi094.stdout:9/944: dread - df/d2d/d1f/d58/d70/de2/f132 zero size 2024-03-24T10:46:02.588 INFO:tasks.workunit.client.0.smithi094.stdout:2/819: symlink d18/d7f/d88/lf3 0 2024-03-24T10:46:02.588 INFO:tasks.workunit.client.0.smithi094.stdout:2/820: write d18/f3f [572051,42189] 0 2024-03-24T10:46:02.591 INFO:tasks.workunit.client.0.smithi094.stdout:1/742: read d4/d9/de/f3b [1468427,92051] 0 2024-03-24T10:46:02.592 INFO:tasks.workunit.client.0.smithi094.stdout:8/875: dwrite df/d25/d28/d31/d47/d66/da3/fa4 [0,4194304] 0 2024-03-24T10:46:02.592 INFO:tasks.workunit.client.0.smithi094.stdout:8/876: stat df/d12/d5b/l5e 0 2024-03-24T10:46:02.592 INFO:tasks.workunit.client.1.smithi096.stdout:4/63: rename d1/l14 to d1/l15 0 2024-03-24T10:46:02.595 INFO:tasks.workunit.client.0.smithi094.stdout:2/821: dread d18/d48/d56/f58 [0,4194304] 0 2024-03-24T10:46:02.596 INFO:tasks.workunit.client.1.smithi096.stdout:7/27: symlink d0/d1/l8 0 2024-03-24T10:46:02.596 INFO:tasks.workunit.client.1.smithi096.stdout:7/28: creat d0/d1/f9 x:0 0 0 2024-03-24T10:46:02.596 INFO:tasks.workunit.client.0.smithi094.stdout:8/877: write df/d25/d28/d31/d47/d22/d55/f8a [533728,112047] 0 2024-03-24T10:46:02.601 INFO:tasks.workunit.client.0.smithi094.stdout:5/827: creat d7/d23/d2b/d68/d81/f10f x:0 0 0 2024-03-24T10:46:02.601 INFO:tasks.workunit.client.0.smithi094.stdout:5/828: chown d7/dc/d56/ld7 15422 1 2024-03-24T10:46:02.601 INFO:tasks.workunit.client.0.smithi094.stdout:5/829: write d7/fdd [990799,25418] 0 2024-03-24T10:46:02.601 INFO:tasks.workunit.client.0.smithi094.stdout:4/863: rmdir d0/d22/de/dc8/de6 39 2024-03-24T10:46:02.603 INFO:tasks.workunit.client.0.smithi094.stdout:2/822: dread d18/d66/d90/d9a/d74/d7e/dca/fec [0,4194304] 0 2024-03-24T10:46:02.603 INFO:tasks.workunit.client.0.smithi094.stdout:2/823: readlink d18/d66/d90/l91 0 2024-03-24T10:46:02.607 INFO:tasks.workunit.client.1.smithi096.stdout:9/36: rename f3 to d9/fc 0 2024-03-24T10:46:02.611 INFO:tasks.workunit.client.0.smithi094.stdout:0/828: creat d14/d22/dcb/de5/f123 x:0 0 0 2024-03-24T10:46:02.612 INFO:tasks.workunit.client.1.smithi096.stdout:6/40: mkdir d8/d12 0 2024-03-24T10:46:02.613 INFO:tasks.workunit.client.1.smithi096.stdout:6/41: dread d8/d10/f11 [0,4194304] 0 2024-03-24T10:46:02.613 INFO:tasks.workunit.client.1.smithi096.stdout:6/42: dread - f1 zero size 2024-03-24T10:46:02.613 INFO:tasks.workunit.client.1.smithi096.stdout:6/43: write d8/d10/f11 [695062,2391] 0 2024-03-24T10:46:02.614 INFO:tasks.workunit.client.0.smithi094.stdout:7/810: creat d1/d9/d16/f116 x:0 0 0 2024-03-24T10:46:02.615 INFO:tasks.workunit.client.0.smithi094.stdout:7/811: dread - d1/d9/d16/d21/d51/d83/d39/d44/dd9/d8f/f91 zero size 2024-03-24T10:46:02.615 INFO:tasks.workunit.client.0.smithi094.stdout:6/837: dwrite d0/d30/d50/fbd [0,4194304] 0 2024-03-24T10:46:02.615 INFO:tasks.workunit.client.0.smithi094.stdout:6/838: chown d0/d1a/f109 27149 1 2024-03-24T10:46:02.617 INFO:tasks.workunit.client.1.smithi096.stdout:4/64: dwrite d1/f8 [0,4194304] 0 2024-03-24T10:46:02.618 INFO:tasks.workunit.client.0.smithi094.stdout:3/769: dwrite d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/f7b [4194304,4194304] 0 2024-03-24T10:46:02.618 INFO:tasks.workunit.client.0.smithi094.stdout:3/770: fdatasync d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/f64 0 2024-03-24T10:46:02.618 INFO:tasks.workunit.client.0.smithi094.stdout:3/771: stat d0/db/dc/d28/l3e 0 2024-03-24T10:46:02.618 INFO:tasks.workunit.client.0.smithi094.stdout:3/772: write d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/d73/f80 [1933123,105215] 0 2024-03-24T10:46:02.620 INFO:tasks.workunit.client.1.smithi096.stdout:7/29: mknod d0/d1/ca 0 2024-03-24T10:46:02.622 INFO:tasks.workunit.client.1.smithi096.stdout:2/43: dread d5/fb [0,4194304] 0 2024-03-24T10:46:02.627 INFO:tasks.workunit.client.1.smithi096.stdout:0/22: read d0/f5 [648877,95829] 0 2024-03-24T10:46:02.627 INFO:tasks.workunit.client.1.smithi096.stdout:0/23: dread - d0/d2/f3 zero size 2024-03-24T10:46:02.628 INFO:tasks.workunit.client.0.smithi094.stdout:4/864: dwrite d0/d34/d73/d93/f96 [0,4194304] 0 2024-03-24T10:46:02.628 INFO:tasks.workunit.client.1.smithi096.stdout:6/44: write f7 [3644604,89198] 0 2024-03-24T10:46:02.628 INFO:tasks.workunit.client.1.smithi096.stdout:6/45: truncate f6 1047674 0 2024-03-24T10:46:02.628 INFO:tasks.workunit.client.1.smithi096.stdout:6/46: stat d8/fb 0 2024-03-24T10:46:02.628 INFO:tasks.workunit.client.1.smithi096.stdout:6/47: truncate f2 572900 0 2024-03-24T10:46:02.628 INFO:tasks.workunit.client.1.smithi096.stdout:6/48: fsync f3 0 2024-03-24T10:46:02.637 INFO:tasks.workunit.client.0.smithi094.stdout:9/945: rename df/d2d/d34/d2c/l33 to df/d2d/df2/l134 0 2024-03-24T10:46:02.638 INFO:tasks.workunit.client.1.smithi096.stdout:7/30: rename d0/d1/d7 to d0/db 0 2024-03-24T10:46:02.639 INFO:tasks.workunit.client.1.smithi096.stdout:3/95: rmdir d0 39 2024-03-24T10:46:02.641 INFO:tasks.workunit.client.1.smithi096.stdout:2/44: symlink d5/dc/d12/l14 0 2024-03-24T10:46:02.641 INFO:tasks.workunit.client.1.smithi096.stdout:2/45: dread - d5/de/f13 zero size 2024-03-24T10:46:02.641 INFO:tasks.workunit.client.1.smithi096.stdout:2/46: creat d5/de/f15 x:0 0 0 2024-03-24T10:46:02.641 INFO:tasks.workunit.client.0.smithi094.stdout:4/865: dwrite d0/d34/fac [0,4194304] 0 2024-03-24T10:46:02.643 INFO:tasks.workunit.client.1.smithi096.stdout:0/24: unlink d0/d2/f3 0 2024-03-24T10:46:02.644 INFO:tasks.workunit.client.0.smithi094.stdout:8/878: rmdir df/d25/d28/d31/d47/d66/da3/da8/d117/dc1/d111 39 2024-03-24T10:46:02.646 INFO:tasks.workunit.client.0.smithi094.stdout:2/824: creat d18/d7f/d88/ff4 x:0 0 0 2024-03-24T10:46:02.646 INFO:tasks.workunit.client.0.smithi094.stdout:2/825: chown c11 12849 1 2024-03-24T10:46:02.646 INFO:tasks.workunit.client.1.smithi096.stdout:6/49: mkdir d8/d10/d13 0 2024-03-24T10:46:02.646 INFO:tasks.workunit.client.1.smithi096.stdout:6/50: fsync f3 0 2024-03-24T10:46:02.647 INFO:tasks.workunit.client.0.smithi094.stdout:0/829: rmdir d14/d22/d28/d2b/d68 39 2024-03-24T10:46:02.650 INFO:tasks.workunit.client.0.smithi094.stdout:7/812: mknod d1/d9/d16/d21/d51/d83/d39/d44/c117 0 2024-03-24T10:46:02.651 INFO:tasks.workunit.client.0.smithi094.stdout:4/866: write d0/d22/de/d28/f3f [3286537,108229] 0 2024-03-24T10:46:02.652 INFO:tasks.workunit.client.0.smithi094.stdout:6/839: creat d0/d30/de6/d9c/f10c x:0 0 0 2024-03-24T10:46:02.652 INFO:tasks.workunit.client.0.smithi094.stdout:6/840: stat d0/d30/fa0 0 2024-03-24T10:46:02.658 INFO:tasks.workunit.client.1.smithi096.stdout:3/96: symlink d0/d9/l20 0 2024-03-24T10:46:02.658 INFO:tasks.workunit.client.1.smithi096.stdout:7/31: fsync d0/d1/f9 0 2024-03-24T10:46:02.659 INFO:tasks.workunit.client.1.smithi096.stdout:7/32: chown d0 6806 1 2024-03-24T10:46:02.659 INFO:tasks.workunit.client.1.smithi096.stdout:7/33: readlink d0/d1/l6 0 2024-03-24T10:46:02.659 INFO:tasks.workunit.client.0.smithi094.stdout:4/867: dread d0/f5e [0,4194304] 0 2024-03-24T10:46:02.660 INFO:tasks.workunit.client.1.smithi096.stdout:2/47: rename d5/dc/ff to d5/dc/d12/f16 0 2024-03-24T10:46:02.660 INFO:tasks.workunit.client.1.smithi096.stdout:2/48: readlink d5/l10 0 2024-03-24T10:46:02.661 INFO:tasks.workunit.client.1.smithi096.stdout:0/25: link d0/d2/c4 d0/c6 0 2024-03-24T10:46:02.663 INFO:tasks.workunit.client.1.smithi096.stdout:6/51: rename d8/fb to d8/f14 0 2024-03-24T10:46:02.663 INFO:tasks.workunit.client.1.smithi096.stdout:6/52: write d8/d10/f11 [1080604,55091] 0 2024-03-24T10:46:02.666 INFO:tasks.workunit.client.1.smithi096.stdout:3/97: link d0/f4 d0/d13/f21 0 2024-03-24T10:46:02.669 INFO:tasks.workunit.client.1.smithi096.stdout:7/34: rename d0/d1/l8 to d0/d1/lc 0 2024-03-24T10:46:02.669 INFO:tasks.workunit.client.1.smithi096.stdout:7/35: chown d0/d1/l4 1355141 1 2024-03-24T10:46:02.669 INFO:tasks.workunit.client.1.smithi096.stdout:7/36: dread - d0/d1/f9 zero size 2024-03-24T10:46:02.671 INFO:tasks.workunit.client.1.smithi096.stdout:2/49: mknod d5/c17 0 2024-03-24T10:46:02.674 INFO:tasks.workunit.client.1.smithi096.stdout:6/53: symlink d8/d10/d13/l15 0 2024-03-24T10:46:02.680 INFO:tasks.workunit.client.1.smithi096.stdout:3/98: mkdir d0/d9/d22 0 2024-03-24T10:46:02.680 INFO:tasks.workunit.client.1.smithi096.stdout:3/99: chown d0/d9 25 1 2024-03-24T10:46:02.682 INFO:tasks.workunit.client.1.smithi096.stdout:6/54: dread f7 [0,4194304] 0 2024-03-24T10:46:02.682 INFO:tasks.workunit.client.1.smithi096.stdout:6/55: truncate d8/d10/f11 1262005 0 2024-03-24T10:46:02.683 INFO:tasks.workunit.client.1.smithi096.stdout:6/56: creat d8/f16 x:0 0 0 2024-03-24T10:46:02.684 INFO:tasks.workunit.client.1.smithi096.stdout:2/50: rename d5/dc/d12/l14 to d5/de/l18 0 2024-03-24T10:46:02.687 INFO:tasks.workunit.client.1.smithi096.stdout:0/26: dwrite d0/f5 [0,4194304] 0 2024-03-24T10:46:02.687 INFO:tasks.workunit.client.1.smithi096.stdout:3/100: symlink d0/d9/l23 0 2024-03-24T10:46:02.690 INFO:tasks.workunit.client.0.smithi094.stdout:1/743: rename d4/d9/de/d3d/d66/l78 to d4/d9/db/d85/df5/lfe 0 2024-03-24T10:46:02.690 INFO:tasks.workunit.client.0.smithi094.stdout:1/744: truncate d4/d9/d37/d33/d58/f88 4694690 0 2024-03-24T10:46:02.691 INFO:tasks.workunit.client.1.smithi096.stdout:2/51: mknod d5/de/c19 0 2024-03-24T10:46:02.691 INFO:tasks.workunit.client.1.smithi096.stdout:2/52: truncate d5/fa 332875 0 2024-03-24T10:46:02.692 INFO:tasks.workunit.client.0.smithi094.stdout:9/946: getdents df/d2d/d34/d2c/d97/df5 0 2024-03-24T10:46:02.692 INFO:tasks.workunit.client.0.smithi094.stdout:9/947: truncate df/d2d/d1f/d58/d5a/d61/fc0 5001190 0 2024-03-24T10:46:02.692 INFO:tasks.workunit.client.1.smithi096.stdout:0/27: mknod d0/d2/c7 0 2024-03-24T10:46:02.694 INFO:tasks.workunit.client.1.smithi096.stdout:3/101: mknod d0/d9/d22/c24 0 2024-03-24T10:46:02.695 INFO:tasks.workunit.client.0.smithi094.stdout:2/826: symlink d18/d66/d90/d9a/def/lf5 0 2024-03-24T10:46:02.698 INFO:tasks.workunit.client.1.smithi096.stdout:6/57: dwrite f7 [0,4194304] 0 2024-03-24T10:46:02.699 INFO:tasks.workunit.client.0.smithi094.stdout:0/830: creat d14/d22/f124 x:0 0 0 2024-03-24T10:46:02.702 INFO:tasks.workunit.client.0.smithi094.stdout:7/813: symlink d1/d9/d16/d21/d51/d83/l118 0 2024-03-24T10:46:02.702 INFO:tasks.workunit.client.0.smithi094.stdout:7/814: write d1/d9/d25/dd4/ff0 [713097,95923] 0 2024-03-24T10:46:02.703 INFO:tasks.workunit.client.1.smithi096.stdout:3/102: creat d0/d9/d22/f25 x:0 0 0 2024-03-24T10:46:02.705 INFO:tasks.workunit.client.0.smithi094.stdout:1/745: dwrite d4/d9/d37/d33/d58/f88 [4194304,4194304] 0 2024-03-24T10:46:02.705 INFO:tasks.workunit.client.0.smithi094.stdout:6/841: mknod d0/d78/da/d23/c10d 0 2024-03-24T10:46:02.706 INFO:tasks.workunit.client.1.smithi096.stdout:6/58: dread d8/d10/f11 [0,4194304] 0 2024-03-24T10:46:02.709 INFO:tasks.workunit.client.0.smithi094.stdout:4/868: creat d0/d22/de/d28/d2b/d33/da4/da5/f107 x:0 0 0 2024-03-24T10:46:02.710 INFO:tasks.workunit.client.0.smithi094.stdout:2/827: dwrite d18/d48/dc3/fe1 [0,4194304] 0 2024-03-24T10:46:02.712 INFO:tasks.workunit.client.1.smithi096.stdout:7/37: link d0/d1/lc d0/d1/ld 0 2024-03-24T10:46:02.716 INFO:tasks.workunit.client.0.smithi094.stdout:5/830: rename d7/d23/d5f/d14/d36/d3a/f7f to d7/d26/d101/f110 0 2024-03-24T10:46:02.716 INFO:tasks.workunit.client.0.smithi094.stdout:5/831: chown d7/d23/d5f/d14/f27 9357 1 2024-03-24T10:46:02.719 INFO:tasks.workunit.client.0.smithi094.stdout:9/948: link df/d2d/d1f/d11/d53/f7f df/d2d/d1f/d11/d53/dde/f135 0 2024-03-24T10:46:02.720 INFO:tasks.workunit.client.1.smithi096.stdout:6/59: rename d8/f16 to d8/d10/d13/f17 0 2024-03-24T10:46:02.722 INFO:tasks.workunit.client.0.smithi094.stdout:0/831: creat d14/d22/d28/d2b/dc0/de7/f125 x:0 0 0 2024-03-24T10:46:02.722 INFO:tasks.workunit.client.0.smithi094.stdout:0/832: chown d14/d22/dcb/de5 6 1 2024-03-24T10:46:02.723 INFO:tasks.workunit.client.0.smithi094.stdout:7/815: dwrite d1/d9/d25/f10c [0,4194304] 0 2024-03-24T10:46:02.723 INFO:tasks.workunit.client.0.smithi094.stdout:7/816: chown d1/d6c/c70 177627 1 2024-03-24T10:46:02.723 INFO:tasks.workunit.client.0.smithi094.stdout:1/746: dwrite d4/d9/de/d6f/d62/fbb [0,4194304] 0 2024-03-24T10:46:02.723 INFO:tasks.workunit.client.1.smithi096.stdout:7/38: mkdir d0/db/de 0 2024-03-24T10:46:02.723 INFO:tasks.workunit.client.1.smithi096.stdout:7/39: write d0/d1/f5 [640260,91290] 0 2024-03-24T10:46:02.724 INFO:tasks.workunit.client.0.smithi094.stdout:6/842: mkdir d0/d30/d50/d4d/d10e 0 2024-03-24T10:46:02.724 INFO:tasks.workunit.client.0.smithi094.stdout:6/843: fdatasync d0/d21/f56 0 2024-03-24T10:46:02.725 INFO:tasks.workunit.client.0.smithi094.stdout:4/869: mkdir d0/d34/db2/d108 0 2024-03-24T10:46:02.728 INFO:tasks.workunit.client.1.smithi096.stdout:2/53: rmdir d5 39 2024-03-24T10:46:02.729 INFO:tasks.workunit.client.1.smithi096.stdout:0/28: getdents d0/d2 0 2024-03-24T10:46:02.731 INFO:tasks.workunit.client.1.smithi096.stdout:7/40: rmdir d0/db/de 0 2024-03-24T10:46:02.732 INFO:tasks.workunit.client.0.smithi094.stdout:2/828: truncate d18/d48/d56/fb0 7056872 0 2024-03-24T10:46:02.733 INFO:tasks.workunit.client.1.smithi096.stdout:2/54: rmdir d5/dc/d12 39 2024-03-24T10:46:02.733 INFO:tasks.workunit.client.1.smithi096.stdout:2/55: truncate d5/de/f15 1034513 0 2024-03-24T10:46:02.734 INFO:tasks.workunit.client.1.smithi096.stdout:2/56: write d5/de/f13 [172669,41708] 0 2024-03-24T10:46:02.734 INFO:tasks.workunit.client.1.smithi096.stdout:2/57: stat d5/de/f13 0 2024-03-24T10:46:02.734 INFO:tasks.workunit.client.0.smithi094.stdout:7/817: dread d1/d9/d16/d21/d51/d83/d39/d44/f56 [0,4194304] 0 2024-03-24T10:46:02.734 INFO:tasks.workunit.client.0.smithi094.stdout:7/818: chown d1/d9/d16/d21/d51/d83/d39/d44/dd9/d8f 375693 1 2024-03-24T10:46:02.738 INFO:tasks.workunit.client.0.smithi094.stdout:8/879: rename df/d25/d36/d98/ld2 to df/d12/d5b/d63/d75/dd9/l127 0 2024-03-24T10:46:02.739 INFO:tasks.workunit.client.1.smithi096.stdout:3/103: rmdir d0/d9/d22 39 2024-03-24T10:46:02.739 INFO:tasks.workunit.client.1.smithi096.stdout:3/104: chown d0/f1 1 1 2024-03-24T10:46:02.740 INFO:tasks.workunit.client.1.smithi096.stdout:2/58: getdents d5/de 0 2024-03-24T10:46:02.743 INFO:tasks.workunit.client.0.smithi094.stdout:3/773: sync 2024-03-24T10:46:02.743 INFO:tasks.workunit.client.0.smithi094.stdout:3/774: fdatasync d0/db/dc/d28/d3c/fd0 0 2024-03-24T10:46:02.746 INFO:tasks.workunit.client.0.smithi094.stdout:8/880: dread df/d25/d28/d31/d3c/f84 [0,4194304] 0 2024-03-24T10:46:02.746 INFO:tasks.workunit.client.0.smithi094.stdout:8/881: truncate df/d12/f2f 5214223 0 2024-03-24T10:46:02.746 INFO:tasks.workunit.client.1.smithi096.stdout:3/105: symlink d0/l26 0 2024-03-24T10:46:02.747 INFO:tasks.workunit.client.1.smithi096.stdout:3/106: stat d0/d9/f1a 0 2024-03-24T10:46:02.747 INFO:tasks.workunit.client.1.smithi096.stdout:3/107: chown d0/f7 750 1 2024-03-24T10:46:02.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:02 smithi094 ceph-mon[29446]: pgmap v244: 65 pgs: 65 active+clean; 1.5 GiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 167 MiB/s rd, 206 MiB/s wr, 660 op/s 2024-03-24T10:46:02.748 INFO:tasks.workunit.client.0.smithi094.stdout:9/949: creat df/d2d/d1f/d11/d53/f136 x:0 0 0 2024-03-24T10:46:02.751 INFO:tasks.workunit.client.1.smithi096.stdout:2/59: creat d5/de/f1a x:0 0 0 2024-03-24T10:46:02.752 INFO:tasks.workunit.client.0.smithi094.stdout:0/833: rmdir d14/d22/d28/d48/d7b/dfa 39 2024-03-24T10:46:02.755 INFO:tasks.workunit.client.1.smithi096.stdout:3/108: rename d0/d9/c19 to d0/c27 0 2024-03-24T10:46:02.755 INFO:tasks.workunit.client.1.smithi096.stdout:3/109: dread - d0/d13/f17 zero size 2024-03-24T10:46:02.755 INFO:tasks.workunit.client.1.smithi096.stdout:3/110: readlink d0/l5 0 2024-03-24T10:46:02.758 INFO:tasks.workunit.client.0.smithi094.stdout:2/829: dwrite d18/f4e [0,4194304] 0 2024-03-24T10:46:02.758 INFO:tasks.workunit.client.1.smithi096.stdout:7/41: dwrite d0/d1/f5 [0,4194304] 0 2024-03-24T10:46:02.758 INFO:tasks.workunit.client.1.smithi096.stdout:7/42: dread - d0/d1/f9 zero size 2024-03-24T10:46:02.760 INFO:tasks.workunit.client.1.smithi096.stdout:6/60: link d8/d10/d13/f17 d8/f18 0 2024-03-24T10:46:02.760 INFO:tasks.workunit.client.1.smithi096.stdout:6/61: read - d8/d10/fd zero size 2024-03-24T10:46:02.763 INFO:tasks.workunit.client.1.smithi096.stdout:2/60: unlink d5/c11 0 2024-03-24T10:46:02.765 INFO:tasks.workunit.client.1.smithi096.stdout:3/111: symlink d0/d9/d22/l28 0 2024-03-24T10:46:02.769 INFO:tasks.workunit.client.1.smithi096.stdout:7/43: creat d0/db/ff x:0 0 0 2024-03-24T10:46:02.769 INFO:tasks.workunit.client.1.smithi096.stdout:0/29: truncate d0/f5 1542865 0 2024-03-24T10:46:02.771 INFO:tasks.workunit.client.0.smithi094.stdout:1/747: link d4/d36/f3f d4/d9/de/d6f/d5f/d8d/fff 0 2024-03-24T10:46:02.771 INFO:tasks.workunit.client.1.smithi096.stdout:6/62: mkdir d8/d10/d19 0 2024-03-24T10:46:02.775 INFO:tasks.workunit.client.0.smithi094.stdout:8/882: dwrite df/d25/d28/d31/f40 [0,4194304] 0 2024-03-24T10:46:02.775 INFO:tasks.workunit.client.0.smithi094.stdout:8/883: fsync df/d25/d28/d31/d47/f20 0 2024-03-24T10:46:02.776 INFO:tasks.workunit.client.1.smithi096.stdout:6/63: mknod d8/d10/c1a 0 2024-03-24T10:46:02.778 INFO:tasks.workunit.client.0.smithi094.stdout:9/950: dwrite df/d2d/d1f/f3d [0,4194304] 0 2024-03-24T10:46:02.778 INFO:tasks.workunit.client.0.smithi094.stdout:9/951: creat df/d2d/d1f/d58/d5a/d50/da5/d101/f137 x:0 0 0 2024-03-24T10:46:02.779 INFO:tasks.workunit.client.0.smithi094.stdout:6/844: unlink d0/d78/da/d23/d34/d4e/fa7 0 2024-03-24T10:46:02.779 INFO:tasks.workunit.client.0.smithi094.stdout:6/845: stat d0/d21/d2a/d55/dce/fd4 0 2024-03-24T10:46:02.783 INFO:tasks.workunit.client.0.smithi094.stdout:4/870: creat d0/d22/de/d28/d2b/d33/da4/da5/f109 x:0 0 0 2024-03-24T10:46:02.783 INFO:tasks.workunit.client.0.smithi094.stdout:4/871: write d0/d22/f9e [1924139,64434] 0 2024-03-24T10:46:02.784 INFO:tasks.workunit.client.1.smithi096.stdout:6/64: creat d8/d10/d19/f1b x:0 0 0 2024-03-24T10:46:02.785 INFO:tasks.workunit.client.0.smithi094.stdout:7/819: symlink d1/d9/d25/l119 0 2024-03-24T10:46:02.790 INFO:tasks.workunit.client.0.smithi094.stdout:3/775: mkdir d0/db/dc/d28/d110/d111 0 2024-03-24T10:46:02.794 INFO:tasks.workunit.client.0.smithi094.stdout:0/834: unlink d14/d22/d28/d2b/d41/d53/l58 0 2024-03-24T10:46:02.794 INFO:tasks.workunit.client.0.smithi094.stdout:8/884: dwrite df/d12/d5b/ff8 [0,4194304] 0 2024-03-24T10:46:02.795 INFO:tasks.workunit.client.0.smithi094.stdout:0/835: read d14/d22/d7d/fa6 [506411,3791] 0 2024-03-24T10:46:02.795 INFO:tasks.workunit.client.0.smithi094.stdout:0/836: dread - d14/d18/fc1 zero size 2024-03-24T10:46:02.798 INFO:tasks.workunit.client.0.smithi094.stdout:2/830: rmdir d18/d54/db9 39 2024-03-24T10:46:02.799 INFO:tasks.workunit.client.0.smithi094.stdout:2/831: truncate d18/f31 646669 0 2024-03-24T10:46:02.799 INFO:tasks.workunit.client.1.smithi096.stdout:7/44: rmdir d0/db 39 2024-03-24T10:46:02.799 INFO:tasks.workunit.client.0.smithi094.stdout:2/832: read d18/f3b [19919,35197] 0 2024-03-24T10:46:02.801 INFO:tasks.workunit.client.0.smithi094.stdout:1/748: creat d4/d9/db/d39/f100 x:0 0 0 2024-03-24T10:46:02.801 INFO:tasks.workunit.client.0.smithi094.stdout:1/749: chown d4/d9/f2c 462178664 1 2024-03-24T10:46:02.801 INFO:tasks.workunit.client.0.smithi094.stdout:1/750: truncate d4/d9/de/d6f/d62/fcc 1100372 0 2024-03-24T10:46:02.803 INFO:tasks.workunit.client.1.smithi096.stdout:7/45: link d0/d1/ld d0/l10 0 2024-03-24T10:46:02.803 INFO:tasks.workunit.client.1.smithi096.stdout:7/46: rename d0/d1 to d0/d1/d11 22 2024-03-24T10:46:02.805 INFO:tasks.workunit.client.1.smithi096.stdout:7/47: mknod d0/d1/c12 0 2024-03-24T10:46:02.805 INFO:tasks.workunit.client.0.smithi094.stdout:8/885: dread df/d25/f29 [0,4194304] 0 2024-03-24T10:46:02.806 INFO:tasks.workunit.client.0.smithi094.stdout:8/886: truncate df/d25/d28/d31/d47/d66/da3/da8/d117/dc1/ffc 1133550 0 2024-03-24T10:46:02.808 INFO:tasks.workunit.client.1.smithi096.stdout:0/30: write d0/f5 [1229664,32857] 0 2024-03-24T10:46:02.808 INFO:tasks.workunit.client.1.smithi096.stdout:0/31: chown d0 3886962 1 2024-03-24T10:46:02.808 INFO:tasks.workunit.client.1.smithi096.stdout:6/65: dwrite d8/d10/d13/f17 [0,4194304] 0 2024-03-24T10:46:02.808 INFO:tasks.workunit.client.1.smithi096.stdout:6/66: creat d8/f1c x:0 0 0 2024-03-24T10:46:02.809 INFO:tasks.workunit.client.0.smithi094.stdout:9/952: rename df/d2d/d34/d2c/d74/fc4 to df/d2d/d1f/d58/d70/dab/d80/f138 0 2024-03-24T10:46:02.812 INFO:tasks.workunit.client.0.smithi094.stdout:7/820: dwrite d1/db4/fba [4194304,4194304] 0 2024-03-24T10:46:02.814 INFO:tasks.workunit.client.0.smithi094.stdout:6/846: unlink d0/fc2 0 2024-03-24T10:46:02.814 INFO:tasks.workunit.client.0.smithi094.stdout:6/847: chown d0/d30/d50/fac 1 1 2024-03-24T10:46:02.814 INFO:tasks.workunit.client.1.smithi096.stdout:3/112: truncate d0/f1 3912626 0 2024-03-24T10:46:02.815 INFO:tasks.workunit.client.1.smithi096.stdout:3/113: fsync d0/d13/f21 0 2024-03-24T10:46:02.816 INFO:tasks.workunit.client.0.smithi094.stdout:7/821: dread d1/d9/fe [0,4194304] 0 2024-03-24T10:46:02.816 INFO:tasks.workunit.client.0.smithi094.stdout:9/953: dread df/d2d/d1f/d11/fbf [0,4194304] 0 2024-03-24T10:46:02.817 INFO:tasks.workunit.client.1.smithi096.stdout:0/32: mknod d0/c8 0 2024-03-24T10:46:02.818 INFO:tasks.workunit.client.0.smithi094.stdout:3/776: creat d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/de9/f112 x:0 0 0 2024-03-24T10:46:02.821 INFO:tasks.workunit.client.0.smithi094.stdout:6/848: dread d0/d30/de6/d9c/ffd [0,4194304] 0 2024-03-24T10:46:02.824 INFO:tasks.workunit.client.0.smithi094.stdout:0/837: link d14/d22/d28/fe1 d14/d22/d28/d48/d7b/f126 0 2024-03-24T10:46:02.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:02 smithi096 ceph-mon[38097]: pgmap v244: 65 pgs: 65 active+clean; 1.5 GiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 167 MiB/s rd, 206 MiB/s wr, 660 op/s 2024-03-24T10:46:02.825 INFO:tasks.workunit.client.1.smithi096.stdout:6/67: dwrite f1 [0,4194304] 0 2024-03-24T10:46:02.827 INFO:tasks.workunit.client.0.smithi094.stdout:2/833: mknod d18/d66/d90/d9a/d74/db4/cf6 0 2024-03-24T10:46:02.831 INFO:tasks.workunit.client.1.smithi096.stdout:3/114: creat d0/d9/d22/f29 x:0 0 0 2024-03-24T10:46:02.832 INFO:tasks.workunit.client.0.smithi094.stdout:9/954: dread df/f2a [0,4194304] 0 2024-03-24T10:46:02.832 INFO:tasks.workunit.client.0.smithi094.stdout:4/872: rename d0/d22/de4/l100 to d0/d5/d1b/d4f/d70/l10a 0 2024-03-24T10:46:02.832 INFO:tasks.workunit.client.0.smithi094.stdout:9/955: write df/f22 [1487701,82339] 0 2024-03-24T10:46:02.832 INFO:tasks.workunit.client.0.smithi094.stdout:9/956: creat df/d2d/d1f/d58/d5a/d50/da5/d101/f139 x:0 0 0 2024-03-24T10:46:02.833 INFO:tasks.workunit.client.0.smithi094.stdout:9/957: write df/d2d/d34/f127 [749674,85458] 0 2024-03-24T10:46:02.833 INFO:tasks.workunit.client.0.smithi094.stdout:9/958: dread - df/d2d/d1f/d58/d70/d104/f133 zero size 2024-03-24T10:46:02.833 INFO:tasks.workunit.client.0.smithi094.stdout:9/959: creat df/d2d/d1f/d11/d96/f13a x:0 0 0 2024-03-24T10:46:02.833 INFO:tasks.workunit.client.0.smithi094.stdout:7/822: dread d1/d9/d25/d3d/f5a [0,4194304] 0 2024-03-24T10:46:02.838 INFO:tasks.workunit.client.1.smithi096.stdout:6/68: mknod d8/d12/c1d 0 2024-03-24T10:46:02.841 INFO:tasks.workunit.client.1.smithi096.stdout:7/48: rename d0/d1 to d0/d13 0 2024-03-24T10:46:02.841 INFO:tasks.workunit.client.1.smithi096.stdout:7/49: write d0/db/ff [297653,111389] 0 2024-03-24T10:46:02.843 INFO:tasks.workunit.client.0.smithi094.stdout:3/777: mknod d0/db/dc3/dea/c113 0 2024-03-24T10:46:02.843 INFO:tasks.workunit.client.0.smithi094.stdout:3/778: chown d0/db/dc/d28/cfc 3265 1 2024-03-24T10:46:02.843 INFO:tasks.workunit.client.0.smithi094.stdout:3/779: dread - d0/db/dc/d28/d38/dc9/f107 zero size 2024-03-24T10:46:02.843 INFO:tasks.workunit.client.1.smithi096.stdout:0/33: truncate d0/f5 561941 0 2024-03-24T10:46:02.844 INFO:tasks.workunit.client.0.smithi094.stdout:6/849: mknod d0/d78/da/d23/d34/d4e/d5d/df0/c10f 0 2024-03-24T10:46:02.846 INFO:tasks.workunit.client.1.smithi096.stdout:6/69: unlink f1 0 2024-03-24T10:46:02.846 INFO:tasks.workunit.client.1.smithi096.stdout:6/70: write d8/f18 [4641045,80384] 0 2024-03-24T10:46:02.847 INFO:tasks.workunit.client.0.smithi094.stdout:2/834: symlink d18/d66/d90/d9a/dbc/d71/lf7 0 2024-03-24T10:46:02.847 INFO:tasks.workunit.client.0.smithi094.stdout:2/835: creat d18/d66/d90/d9a/d74/ff8 x:0 0 0 2024-03-24T10:46:02.850 INFO:tasks.workunit.client.1.smithi096.stdout:3/115: truncate d0/f1 2314867 0 2024-03-24T10:46:02.851 INFO:tasks.workunit.client.1.smithi096.stdout:3/116: dread - d0/d13/f1f zero size 2024-03-24T10:46:02.851 INFO:tasks.workunit.client.1.smithi096.stdout:3/117: readlink d0/l5 0 2024-03-24T10:46:02.851 INFO:tasks.workunit.client.1.smithi096.stdout:3/118: stat d0/d9 0 2024-03-24T10:46:02.853 INFO:tasks.workunit.client.0.smithi094.stdout:1/751: rename d4/d9/de/d3d/dae/dfb to d4/d9/de/d3d/d66/dbe/d101 0 2024-03-24T10:46:02.854 INFO:tasks.workunit.client.1.smithi096.stdout:0/34: symlink d0/d2/l9 0 2024-03-24T10:46:02.855 INFO:tasks.workunit.client.1.smithi096.stdout:0/35: chown d0/c1 383 1 2024-03-24T10:46:02.855 INFO:tasks.workunit.client.1.smithi096.stdout:0/36: stat d0/d2 0 2024-03-24T10:46:02.855 INFO:tasks.workunit.client.1.smithi096.stdout:0/37: readlink d0/d2/l9 0 2024-03-24T10:46:02.857 INFO:tasks.workunit.client.0.smithi094.stdout:2/836: write d18/d66/d90/d9a/dbc/d71/fc9 [503115,96773] 0 2024-03-24T10:46:02.857 INFO:tasks.workunit.client.0.smithi094.stdout:2/837: write d18/d66/d90/d9a/dbc/f44 [203910,7766] 0 2024-03-24T10:46:02.857 INFO:tasks.workunit.client.0.smithi094.stdout:2/838: write d18/d77/dcf/fea [272746,104415] 0 2024-03-24T10:46:02.857 INFO:tasks.workunit.client.0.smithi094.stdout:2/839: write d18/d54/db9/fdd [326954,70394] 0 2024-03-24T10:46:02.858 INFO:tasks.workunit.client.1.smithi096.stdout:6/71: mkdir d8/d10/d19/d1e 0 2024-03-24T10:46:02.858 INFO:tasks.workunit.client.1.smithi096.stdout:6/72: readlink d8/lf 0 2024-03-24T10:46:02.859 INFO:tasks.workunit.client.0.smithi094.stdout:4/873: creat d0/d5/d1b/f10b x:0 0 0 2024-03-24T10:46:02.859 INFO:tasks.workunit.client.0.smithi094.stdout:4/874: chown d0/d22/de/d28/d2b/d7a/f7b 505668259 1 2024-03-24T10:46:02.860 INFO:tasks.workunit.client.1.smithi096.stdout:3/119: creat d0/d13/d1e/f2a x:0 0 0 2024-03-24T10:46:02.861 INFO:tasks.workunit.client.0.smithi094.stdout:4/875: dread d0/d22/de/d28/d2b/da2/ff1 [0,4194304] 0 2024-03-24T10:46:02.861 INFO:tasks.workunit.client.0.smithi094.stdout:4/876: chown d0/d22/de/d28/d2b/d33/fc2 3847121 1 2024-03-24T10:46:02.861 INFO:tasks.workunit.client.0.smithi094.stdout:4/877: write d0/d22/de/d28/d2b/d33/fc2 [370495,30120] 0 2024-03-24T10:46:02.862 INFO:tasks.workunit.client.1.smithi096.stdout:7/50: getdents d0 0 2024-03-24T10:46:02.862 INFO:tasks.workunit.client.1.smithi096.stdout:7/51: write d0/d13/f9 [842073,59220] 0 2024-03-24T10:46:02.866 INFO:tasks.workunit.client.0.smithi094.stdout:2/840: dread d18/d1e/d65/f6a [0,4194304] 0 2024-03-24T10:46:02.866 INFO:tasks.workunit.client.1.smithi096.stdout:0/38: mknod d0/ca 0 2024-03-24T10:46:02.867 INFO:tasks.workunit.client.1.smithi096.stdout:0/39: chown d0 523 1 2024-03-24T10:46:02.867 INFO:tasks.workunit.client.1.smithi096.stdout:7/52: dread d0/d13/f9 [0,4194304] 0 2024-03-24T10:46:02.870 INFO:tasks.workunit.client.1.smithi096.stdout:6/73: link d8/d10/d13/f17 d8/d12/f1f 0 2024-03-24T10:46:02.874 INFO:tasks.workunit.client.0.smithi094.stdout:7/823: mkdir d1/d9/d25/dd4/d11a 0 2024-03-24T10:46:02.879 INFO:tasks.workunit.client.1.smithi096.stdout:0/40: creat d0/d2/fb x:0 0 0 2024-03-24T10:46:02.880 INFO:tasks.workunit.client.0.smithi094.stdout:3/780: truncate d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/db5/d106/d9b/d10d/f89 3498242 0 2024-03-24T10:46:02.880 INFO:tasks.workunit.client.0.smithi094.stdout:3/781: truncate d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d4f/f109 334578 0 2024-03-24T10:46:02.880 INFO:tasks.workunit.client.0.smithi094.stdout:3/782: truncate d0/db/dc/d28/d3c/d55/fb2 4789686 0 2024-03-24T10:46:02.882 INFO:tasks.workunit.client.0.smithi094.stdout:6/850: mkdir d0/d21/d2a/dcb/d110 0 2024-03-24T10:46:02.882 INFO:tasks.workunit.client.0.smithi094.stdout:6/851: truncate d0/f3c 770435 0 2024-03-24T10:46:02.887 INFO:tasks.workunit.client.0.smithi094.stdout:0/838: rename d14/d22/d28/d48/cb3 to d14/d22/d32/d70/c127 0 2024-03-24T10:46:02.887 INFO:tasks.workunit.client.0.smithi094.stdout:0/839: truncate d14/d22/d28/f47 1040531 0 2024-03-24T10:46:02.887 INFO:tasks.workunit.client.0.smithi094.stdout:0/840: chown d14/d22/d5b/fbd 18245 1 2024-03-24T10:46:02.888 INFO:tasks.workunit.client.0.smithi094.stdout:3/783: dread d0/db/dc/d28/d3c/f6f [0,4194304] 0 2024-03-24T10:46:02.889 INFO:tasks.workunit.client.0.smithi094.stdout:3/784: truncate d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/de9/f112 531262 0 2024-03-24T10:46:02.889 INFO:tasks.workunit.client.0.smithi094.stdout:2/841: truncate d18/f31 113480 0 2024-03-24T10:46:02.891 INFO:tasks.workunit.client.0.smithi094.stdout:6/852: mknod d0/d30/d50/c111 0 2024-03-24T10:46:02.892 INFO:tasks.workunit.client.0.smithi094.stdout:8/887: rmdir df/d12/dcd 0 2024-03-24T10:46:02.894 INFO:tasks.workunit.client.1.smithi096.stdout:7/53: dwrite d0/d13/f5 [0,4194304] 0 2024-03-24T10:46:02.894 INFO:tasks.workunit.client.1.smithi096.stdout:6/74: dwrite f6 [0,4194304] 0 2024-03-24T10:46:02.894 INFO:tasks.workunit.client.0.smithi094.stdout:9/960: rename df/d2d/d1f/d58/d82/f110 to df/d2d/d1f/d58/d5a/d50/da5/d101/f13b 0 2024-03-24T10:46:02.896 INFO:tasks.workunit.client.0.smithi094.stdout:2/842: dread d18/d48/d56/f58 [0,4194304] 0 2024-03-24T10:46:02.896 INFO:tasks.workunit.client.0.smithi094.stdout:7/824: dwrite d1/d9/d16/d21/d51/d83/d39/d44/dd9/d34/fa3 [0,4194304] 0 2024-03-24T10:46:02.896 INFO:tasks.workunit.client.0.smithi094.stdout:7/825: read - d1/db4/d65/fd8 zero size 2024-03-24T10:46:02.896 INFO:tasks.workunit.client.0.smithi094.stdout:0/841: creat d14/d22/d28/d2b/d41/f128 x:0 0 0 2024-03-24T10:46:02.899 INFO:tasks.workunit.client.0.smithi094.stdout:6/853: symlink d0/d1a/dca/l112 0 2024-03-24T10:46:02.902 INFO:tasks.workunit.client.0.smithi094.stdout:8/888: creat df/d25/d36/d10b/f128 x:0 0 0 2024-03-24T10:46:02.902 INFO:tasks.workunit.client.0.smithi094.stdout:2/843: dread f7 [0,4194304] 0 2024-03-24T10:46:02.904 INFO:tasks.workunit.client.0.smithi094.stdout:1/752: rename d4/d9/de/d3d/d66/dbe/d101 to d4/d9/d102 0 2024-03-24T10:46:02.907 INFO:tasks.workunit.client.0.smithi094.stdout:9/961: symlink df/d2d/d34/d2c/d74/dc5/l13c 0 2024-03-24T10:46:02.907 INFO:tasks.workunit.client.0.smithi094.stdout:9/962: chown df/d2d/d1f/d58/d5a/d61/d87/f12c 3616 1 2024-03-24T10:46:02.907 INFO:tasks.workunit.client.0.smithi094.stdout:9/963: truncate df/d2d/d34/fd6 479773 0 2024-03-24T10:46:02.910 INFO:tasks.workunit.client.0.smithi094.stdout:2/844: dread d18/d66/d90/d9a/dbc/f51 [0,4194304] 0 2024-03-24T10:46:02.910 INFO:tasks.workunit.client.1.smithi096.stdout:7/54: dwrite d0/d13/f9 [0,4194304] 0 2024-03-24T10:46:02.911 INFO:tasks.workunit.client.1.smithi096.stdout:6/75: write d8/d10/f11 [376448,23373] 0 2024-03-24T10:46:02.912 INFO:tasks.workunit.client.0.smithi094.stdout:7/826: mkdir d1/d9/d25/dd4/d11b 0 2024-03-24T10:46:02.912 INFO:tasks.workunit.client.0.smithi094.stdout:7/827: chown d1/d9/d25/d3d/f46 266204 1 2024-03-24T10:46:02.912 INFO:tasks.workunit.client.0.smithi094.stdout:6/854: write d0/d30/fa0 [478602,103753] 0 2024-03-24T10:46:02.913 INFO:tasks.workunit.client.0.smithi094.stdout:0/842: mkdir d14/d22/d5b/d129 0 2024-03-24T10:46:02.917 INFO:tasks.workunit.client.1.smithi096.stdout:7/55: creat d0/f14 x:0 0 0 2024-03-24T10:46:02.921 INFO:tasks.workunit.client.0.smithi094.stdout:1/753: unlink d4/d9/d7d/ldc 0 2024-03-24T10:46:02.924 INFO:tasks.workunit.client.0.smithi094.stdout:9/964: mkdir df/d2d/d1f/d11/d53/d88/dd5/d13d 0 2024-03-24T10:46:02.924 INFO:tasks.workunit.client.0.smithi094.stdout:9/965: truncate df/d2d/d1f/d58/d70/de2/ff0 411351 0 2024-03-24T10:46:02.927 INFO:tasks.workunit.client.0.smithi094.stdout:1/754: write d4/d9/db/d31/d3c/fd7 [2476681,19253] 0 2024-03-24T10:46:02.927 INFO:tasks.workunit.client.0.smithi094.stdout:2/845: rmdir d18/d7f/d88/ddb 0 2024-03-24T10:46:02.928 INFO:tasks.workunit.client.1.smithi096.stdout:7/56: creat d0/d13/f15 x:0 0 0 2024-03-24T10:46:02.928 INFO:tasks.workunit.client.1.smithi096.stdout:7/57: chown d0/d13 971060 1 2024-03-24T10:46:02.928 INFO:tasks.workunit.client.1.smithi096.stdout:7/58: truncate d0/f14 267438 0 2024-03-24T10:46:02.931 INFO:tasks.workunit.client.0.smithi094.stdout:7/828: mknod d1/d9/d16/d21/d51/d83/c11c 0 2024-03-24T10:46:02.934 INFO:tasks.workunit.client.1.smithi096.stdout:7/59: mknod d0/db/c16 0 2024-03-24T10:46:02.935 INFO:tasks.workunit.client.0.smithi094.stdout:6/855: creat d0/d78/d66/f113 x:0 0 0 2024-03-24T10:46:02.937 INFO:tasks.workunit.client.0.smithi094.stdout:0/843: rename d14/d22/d28/fbe to d14/d22/d28/da2/d10a/f12a 0 2024-03-24T10:46:02.938 INFO:tasks.workunit.client.0.smithi094.stdout:0/844: truncate d14/d22/d28/d2b/d41/d53/d105/f113 1045923 0 2024-03-24T10:46:02.938 INFO:tasks.workunit.client.0.smithi094.stdout:0/845: dread - d14/d18/f9d zero size 2024-03-24T10:46:02.938 INFO:tasks.workunit.client.0.smithi094.stdout:0/846: readlink d14/d22/d28/d2b/ldb 0 2024-03-24T10:46:02.939 INFO:tasks.workunit.client.0.smithi094.stdout:5/832: sync 2024-03-24T10:46:02.941 INFO:tasks.workunit.client.0.smithi094.stdout:9/966: symlink df/d2d/d1f/d11/d96/df3/l13e 0 2024-03-24T10:46:02.942 INFO:tasks.workunit.client.1.smithi096.stdout:6/76: dwrite d8/f18 [0,4194304] 0 2024-03-24T10:46:02.943 INFO:tasks.workunit.client.0.smithi094.stdout:1/755: unlink d4/d9/de/d6f/d62/fe7 0 2024-03-24T10:46:02.943 INFO:tasks.workunit.client.0.smithi094.stdout:1/756: fsync d4/d9/de/d6f/fa5 0 2024-03-24T10:46:02.944 INFO:tasks.workunit.client.1.smithi096.stdout:3/120: truncate d0/f1 2195809 0 2024-03-24T10:46:02.946 INFO:tasks.workunit.client.0.smithi094.stdout:5/833: write d7/d26/f5c [1920400,117524] 0 2024-03-24T10:46:02.946 INFO:tasks.workunit.client.0.smithi094.stdout:7/829: creat d1/d9/d16/d21/d51/d83/d39/d44/dd9/f11d x:0 0 0 2024-03-24T10:46:02.946 INFO:tasks.workunit.client.0.smithi094.stdout:7/830: fsync d1/d9/dc3/fed 0 2024-03-24T10:46:02.946 INFO:tasks.workunit.client.0.smithi094.stdout:2/846: dwrite d18/d7f/d88/da2/da4/f5e [0,4194304] 0 2024-03-24T10:46:02.946 INFO:tasks.workunit.client.0.smithi094.stdout:2/847: stat d18/d7f/d88/da2 0 2024-03-24T10:46:02.948 INFO:tasks.workunit.client.0.smithi094.stdout:0/847: rmdir d14 39 2024-03-24T10:46:02.948 INFO:tasks.workunit.client.0.smithi094.stdout:0/848: write d14/d22/d28/d48/da5/fa7 [843745,110699] 0 2024-03-24T10:46:02.949 INFO:tasks.workunit.client.1.smithi096.stdout:3/121: stat d0/c11 0 2024-03-24T10:46:02.949 INFO:tasks.workunit.client.1.smithi096.stdout:3/122: chown d0/ff 927467 1 2024-03-24T10:46:02.951 INFO:tasks.workunit.client.0.smithi094.stdout:8/889: rmdir df/d25/d36/d98 0 2024-03-24T10:46:02.955 INFO:tasks.workunit.client.0.smithi094.stdout:9/967: unlink df/d2d/d1f/d58/d5a/d69/f78 0 2024-03-24T10:46:02.957 INFO:tasks.workunit.client.0.smithi094.stdout:1/757: mknod d4/d9/de/d6f/d62/d84/c103 0 2024-03-24T10:46:02.957 INFO:tasks.workunit.client.0.smithi094.stdout:1/758: fsync d4/d9/db/d31/d3c/fd7 0 2024-03-24T10:46:02.960 INFO:tasks.workunit.client.0.smithi094.stdout:5/834: creat d7/d23/dd5/f111 x:0 0 0 2024-03-24T10:46:02.961 INFO:tasks.workunit.client.0.smithi094.stdout:7/831: rmdir d1/d6c 39 2024-03-24T10:46:02.963 INFO:tasks.workunit.client.0.smithi094.stdout:2/848: creat d18/d1e/d65/ff9 x:0 0 0 2024-03-24T10:46:02.965 INFO:tasks.workunit.client.0.smithi094.stdout:0/849: mkdir d14/d22/d28/d2b/d41/d53/d12b 0 2024-03-24T10:46:02.966 INFO:tasks.workunit.client.0.smithi094.stdout:8/890: creat df/d25/f129 x:0 0 0 2024-03-24T10:46:02.968 INFO:tasks.workunit.client.0.smithi094.stdout:1/759: link d4/d36/f73 d4/f104 0 2024-03-24T10:46:02.970 INFO:tasks.workunit.client.0.smithi094.stdout:5/835: symlink d7/d23/d5f/d9/d48/d51/l112 0 2024-03-24T10:46:02.970 INFO:tasks.workunit.client.1.smithi096.stdout:7/60: fsync d0/d13/f15 0 2024-03-24T10:46:02.971 INFO:tasks.workunit.client.1.smithi096.stdout:7/61: write d0/f14 [606726,48635] 0 2024-03-24T10:46:02.972 INFO:tasks.workunit.client.0.smithi094.stdout:8/891: link df/d25/d28/d31/d47/d66/da3/da8/le0 df/d12/d5b/d63/d75/d96/l12a 0 2024-03-24T10:46:02.975 INFO:tasks.workunit.client.0.smithi094.stdout:1/760: rmdir d4/d9/de/d6f/df1 0 2024-03-24T10:46:02.976 INFO:tasks.workunit.client.0.smithi094.stdout:5/836: dread d7/d23/d5f/d14/f85 [0,4194304] 0 2024-03-24T10:46:02.977 INFO:tasks.workunit.client.0.smithi094.stdout:8/892: symlink df/d25/d28/d31/d47/dfe/l12b 0 2024-03-24T10:46:02.978 INFO:tasks.workunit.client.1.smithi096.stdout:7/62: getdents d0/db 0 2024-03-24T10:46:02.978 INFO:tasks.workunit.client.1.smithi096.stdout:7/63: truncate d0/d13/f9 4984934 0 2024-03-24T10:46:02.978 INFO:tasks.workunit.client.1.smithi096.stdout:7/64: chown d0/d13/ca 13 1 2024-03-24T10:46:02.980 INFO:tasks.workunit.client.0.smithi094.stdout:1/761: symlink d4/d9/db/d31/d3c/l105 0 2024-03-24T10:46:02.981 INFO:tasks.workunit.client.0.smithi094.stdout:5/837: mkdir d7/dc/d100/d109/d113 0 2024-03-24T10:46:02.983 INFO:tasks.workunit.client.1.smithi096.stdout:7/65: mknod d0/db/c17 0 2024-03-24T10:46:02.987 INFO:tasks.workunit.client.0.smithi094.stdout:1/762: symlink d4/d9/db/d5d/l106 0 2024-03-24T10:46:02.987 INFO:tasks.workunit.client.0.smithi094.stdout:1/763: fsync d4/d9/db/d31/d3c/d9f/fee 0 2024-03-24T10:46:02.987 INFO:tasks.workunit.client.0.smithi094.stdout:1/764: truncate d4/d9/d7d/fad 178416 0 2024-03-24T10:46:02.990 INFO:tasks.workunit.client.0.smithi094.stdout:1/765: mkdir d4/d9/db/d39/d75/d107 0 2024-03-24T10:46:02.991 INFO:tasks.workunit.client.1.smithi096.stdout:3/123: dwrite d0/d13/d1e/f2a [0,4194304] 0 2024-03-24T10:46:02.991 INFO:tasks.workunit.client.1.smithi096.stdout:3/124: chown d0/d9/d22/l28 46677 1 2024-03-24T10:46:02.994 INFO:tasks.workunit.client.1.smithi096.stdout:3/125: symlink d0/d13/d1e/l2b 0 2024-03-24T10:46:02.995 INFO:tasks.workunit.client.1.smithi096.stdout:6/77: dwrite f3 [0,4194304] 0 2024-03-24T10:46:03.004 INFO:tasks.workunit.client.0.smithi094.stdout:0/850: dwrite d14/d22/d7d/fd5 [0,4194304] 0 2024-03-24T10:46:03.007 INFO:tasks.workunit.client.0.smithi094.stdout:2/849: dwrite d18/f4e [0,4194304] 0 2024-03-24T10:46:03.007 INFO:tasks.workunit.client.0.smithi094.stdout:9/968: dwrite df/d2d/d1f/d58/d70/de2/ff0 [0,4194304] 0 2024-03-24T10:46:03.007 INFO:tasks.workunit.client.0.smithi094.stdout:9/969: write df/d2d/d34/fd6 [1482176,5741] 0 2024-03-24T10:46:03.010 INFO:tasks.workunit.client.0.smithi094.stdout:5/838: dwrite d7/fb7 [0,4194304] 0 2024-03-24T10:46:03.010 INFO:tasks.workunit.client.0.smithi094.stdout:5/839: chown d7/d23/d5f/d9/d48/f10e 473078136 1 2024-03-24T10:46:03.010 INFO:tasks.workunit.client.0.smithi094.stdout:8/893: dwrite df/d25/d28/d31/d47/d66/fe2 [0,4194304] 0 2024-03-24T10:46:03.010 INFO:tasks.workunit.client.0.smithi094.stdout:8/894: fdatasync df/d25/d28/d31/d47/f20 0 2024-03-24T10:46:03.010 INFO:tasks.workunit.client.0.smithi094.stdout:5/840: write d7/d23/d5f/f58 [794571,82112] 0 2024-03-24T10:46:03.010 INFO:tasks.workunit.client.0.smithi094.stdout:8/895: write df/d25/daf/fcb [68258,60264] 0 2024-03-24T10:46:03.010 INFO:tasks.workunit.client.0.smithi094.stdout:5/841: chown d7/d23/d5f/d9/f92 1 1 2024-03-24T10:46:03.011 INFO:tasks.workunit.client.0.smithi094.stdout:0/851: write d14/d22/d28/da2/fd1 [161744,75265] 0 2024-03-24T10:46:03.011 INFO:tasks.workunit.client.0.smithi094.stdout:0/852: chown d14/d22/d32/d70/cb1 7347888 1 2024-03-24T10:46:03.011 INFO:tasks.workunit.client.0.smithi094.stdout:0/853: write d14/d18/fde [905544,1387] 0 2024-03-24T10:46:03.017 INFO:tasks.workunit.client.0.smithi094.stdout:5/842: dread d7/d23/d5f/f83 [0,4194304] 0 2024-03-24T10:46:03.019 INFO:tasks.workunit.client.0.smithi094.stdout:2/850: unlink d18/d66/dbe/dc0/f6d 0 2024-03-24T10:46:03.022 INFO:tasks.workunit.client.0.smithi094.stdout:9/970: mkdir df/d2d/d1f/d58/d5a/d61/d87/d10e/d13f 0 2024-03-24T10:46:03.032 INFO:tasks.workunit.client.0.smithi094.stdout:0/854: link d14/d22/d28/d2b/l8b d14/d22/d28/d2b/dc0/de7/l12c 0 2024-03-24T10:46:03.032 INFO:tasks.workunit.client.0.smithi094.stdout:0/855: write d14/d22/d28/d2b/f38 [4114545,58234] 0 2024-03-24T10:46:03.035 INFO:tasks.workunit.client.0.smithi094.stdout:5/843: getdents d7/dc 0 2024-03-24T10:46:03.035 INFO:tasks.workunit.client.0.smithi094.stdout:5/844: chown d7/d23/d5f/d9/d48/fd8 236 1 2024-03-24T10:46:03.038 INFO:tasks.workunit.client.0.smithi094.stdout:9/971: rename df/d2d/d1f/d58/d5a/d61/fc0 to df/d2d/d34/d2c/f140 0 2024-03-24T10:46:03.040 INFO:tasks.workunit.client.0.smithi094.stdout:5/845: link d7/d23/d5f/d1b/f37 d7/d23/d5f/d14/d6f/dbe/f114 0 2024-03-24T10:46:03.040 INFO:tasks.workunit.client.0.smithi094.stdout:5/846: write d7/d23/d5f/f97 [1059952,80825] 0 2024-03-24T10:46:03.041 INFO:tasks.workunit.client.0.smithi094.stdout:5/847: read d7/d23/d5f/d14/f9a [17372,114299] 0 2024-03-24T10:46:03.041 INFO:tasks.workunit.client.0.smithi094.stdout:2/851: getdents d18/d66/dbe 0 2024-03-24T10:46:03.041 INFO:tasks.workunit.client.0.smithi094.stdout:0/856: read d14/d18/f21 [3428566,90852] 0 2024-03-24T10:46:03.042 INFO:tasks.workunit.client.0.smithi094.stdout:9/972: mkdir df/d2d/d1f/d11/d96/df3/d141 0 2024-03-24T10:46:03.044 INFO:tasks.workunit.client.0.smithi094.stdout:5/848: truncate d7/d23/d5f/d9/d48/d51/f6a 911941 0 2024-03-24T10:46:03.044 INFO:tasks.workunit.client.0.smithi094.stdout:5/849: dread - d7/d23/d2b/d68/fe8 zero size 2024-03-24T10:46:03.045 INFO:tasks.workunit.client.0.smithi094.stdout:0/857: symlink d14/d22/d32/d70/dd0/l12d 0 2024-03-24T10:46:03.047 INFO:tasks.workunit.client.0.smithi094.stdout:9/973: mkdir df/d2d/d1f/d58/d70/dab/dec/d142 0 2024-03-24T10:46:03.049 INFO:tasks.workunit.client.0.smithi094.stdout:5/850: creat d7/d23/dd5/d64/dd6/f115 x:0 0 0 2024-03-24T10:46:03.049 INFO:tasks.workunit.client.0.smithi094.stdout:5/851: chown d7/d23/d5f/d9/l4a 19869243 1 2024-03-24T10:46:03.049 INFO:tasks.workunit.client.0.smithi094.stdout:5/852: write d7/d23/d5f/d14/d36/d3a/fe0 [474007,106312] 0 2024-03-24T10:46:03.049 INFO:tasks.workunit.client.0.smithi094.stdout:0/858: mknod d14/d22/d28/d2b/d41/c12e 0 2024-03-24T10:46:03.050 INFO:tasks.workunit.client.0.smithi094.stdout:2/852: dread d18/d66/fa5 [0,4194304] 0 2024-03-24T10:46:03.050 INFO:tasks.workunit.client.0.smithi094.stdout:9/974: read df/f62 [9762,28725] 0 2024-03-24T10:46:03.050 INFO:tasks.workunit.client.0.smithi094.stdout:9/975: dread - df/d2d/d1f/d58/d70/dab/ff4 zero size 2024-03-24T10:46:03.053 INFO:tasks.workunit.client.0.smithi094.stdout:2/853: dread d18/d1e/d65/f6a [0,4194304] 0 2024-03-24T10:46:03.053 INFO:tasks.workunit.client.0.smithi094.stdout:5/853: rename d7/d23/d5f/d14/le7 to d7/d23/dd5/da9/l116 0 2024-03-24T10:46:03.054 INFO:tasks.workunit.client.0.smithi094.stdout:5/854: write d7/d23/d5f/d9/f4f [2496166,44016] 0 2024-03-24T10:46:03.054 INFO:tasks.workunit.client.0.smithi094.stdout:5/855: write d7/d23/d5f/d14/d36/d3a/f79 [109788,101730] 0 2024-03-24T10:46:03.054 INFO:tasks.workunit.client.0.smithi094.stdout:9/976: creat df/d2d/d1f/d11/f143 x:0 0 0 2024-03-24T10:46:03.055 INFO:tasks.workunit.client.0.smithi094.stdout:2/854: dread d18/d48/d56/dcc/f7a [0,4194304] 0 2024-03-24T10:46:03.055 INFO:tasks.workunit.client.0.smithi094.stdout:2/855: write d18/d66/d90/d9a/dbc/f84 [1741957,106339] 0 2024-03-24T10:46:03.057 INFO:tasks.workunit.client.0.smithi094.stdout:0/859: rename d14/d22/d7d/d11b/l11d to d14/d22/d28/da2/l12f 0 2024-03-24T10:46:03.057 INFO:tasks.workunit.client.0.smithi094.stdout:0/860: fsync d14/d22/d28/d2b/f38 0 2024-03-24T10:46:03.061 INFO:tasks.workunit.client.0.smithi094.stdout:9/977: creat df/d2d/d1f/d11/d96/df3/f144 x:0 0 0 2024-03-24T10:46:03.061 INFO:tasks.workunit.client.0.smithi094.stdout:9/978: write df/d2d/d1f/d58/d5a/d69/d7a/f8f [792383,110007] 0 2024-03-24T10:46:03.061 INFO:tasks.workunit.client.0.smithi094.stdout:9/979: chown df/d2d/d34/d2c/f40 3181624 1 2024-03-24T10:46:03.064 INFO:tasks.workunit.client.0.smithi094.stdout:2/856: link d18/d48/fbf d18/d66/d90/ffa 0 2024-03-24T10:46:03.066 INFO:tasks.workunit.client.0.smithi094.stdout:0/861: mkdir d14/d22/d28/d2b/dc0/de7/d130 0 2024-03-24T10:46:03.069 INFO:tasks.workunit.client.0.smithi094.stdout:2/857: creat d18/d66/dbe/ddf/ffb x:0 0 0 2024-03-24T10:46:03.071 INFO:tasks.workunit.client.0.smithi094.stdout:4/878: sync 2024-03-24T10:46:03.072 INFO:tasks.workunit.client.0.smithi094.stdout:4/879: fdatasync d0/d5/d1b/d4f/fa7 0 2024-03-24T10:46:03.077 INFO:tasks.workunit.client.0.smithi094.stdout:5/856: dwrite d7/d23/d5f/d9/d48/d51/fc7 [0,4194304] 0 2024-03-24T10:46:03.077 INFO:tasks.workunit.client.0.smithi094.stdout:2/858: read d18/d48/f52 [3302231,49544] 0 2024-03-24T10:46:03.077 INFO:tasks.workunit.client.0.smithi094.stdout:2/859: truncate d18/f31 436593 0 2024-03-24T10:46:03.081 INFO:tasks.workunit.client.0.smithi094.stdout:4/880: dread d0/d5/d7d/fc3 [0,4194304] 0 2024-03-24T10:46:03.081 INFO:tasks.workunit.client.0.smithi094.stdout:9/980: dwrite df/d2d/d1f/d58/d5a/d50/da5/d101/f137 [0,4194304] 0 2024-03-24T10:46:03.081 INFO:tasks.workunit.client.0.smithi094.stdout:9/981: chown df/d2d/d1f/d58/d5a/d50/da5/dd4/d10a 4868 1 2024-03-24T10:46:03.081 INFO:tasks.workunit.client.0.smithi094.stdout:9/982: fdatasync df/d2d/d1f/d11/d53/d112/f115 0 2024-03-24T10:46:03.082 INFO:tasks.workunit.client.0.smithi094.stdout:4/881: creat d0/d22/de/d28/d2b/da2/f10c x:0 0 0 2024-03-24T10:46:03.082 INFO:tasks.workunit.client.0.smithi094.stdout:9/983: write df/d2d/d1f/d58/d5a/d50/da5/dd4/d10a/d11a/ff9 [573830,233] 0 2024-03-24T10:46:03.082 INFO:tasks.workunit.client.0.smithi094.stdout:9/984: dread - df/d2d/d1f/d58/d70/dab/d80/fce zero size 2024-03-24T10:46:03.088 INFO:tasks.workunit.client.0.smithi094.stdout:5/857: write d7/d23/d2b/f4b [470340,31812] 0 2024-03-24T10:46:03.088 INFO:tasks.workunit.client.0.smithi094.stdout:5/858: creat d7/d26/f117 x:0 0 0 2024-03-24T10:46:03.089 INFO:tasks.workunit.client.0.smithi094.stdout:2/860: mknod d18/d66/dbe/cfc 0 2024-03-24T10:46:03.099 INFO:tasks.workunit.client.0.smithi094.stdout:4/882: mkdir d0/d22/de/d28/d3d/d10d 0 2024-03-24T10:46:03.099 INFO:tasks.workunit.client.0.smithi094.stdout:0/862: rmdir d14/d22 39 2024-03-24T10:46:03.103 INFO:tasks.workunit.client.0.smithi094.stdout:9/985: rename df/d2d/d34/d2c/f73 to df/f145 0 2024-03-24T10:46:03.103 INFO:tasks.workunit.client.0.smithi094.stdout:9/986: fsync df/d2d/d1f/f3d 0 2024-03-24T10:46:03.107 INFO:tasks.workunit.client.0.smithi094.stdout:5/859: truncate d7/d23/d5f/d14/f27 3015249 0 2024-03-24T10:46:03.108 INFO:tasks.workunit.client.0.smithi094.stdout:2/861: mkdir d18/d1e/dfd 0 2024-03-24T10:46:03.108 INFO:tasks.workunit.client.0.smithi094.stdout:2/862: stat d18/d7f/d88/da2/da4/f5e 0 2024-03-24T10:46:03.108 INFO:tasks.workunit.client.0.smithi094.stdout:2/863: stat d18/d66/dbe/dc8/de6 0 2024-03-24T10:46:03.113 INFO:tasks.workunit.client.0.smithi094.stdout:9/987: dread df/d2d/d34/d2c/f140 [0,4194304] 0 2024-03-24T10:46:03.113 INFO:tasks.workunit.client.0.smithi094.stdout:9/988: truncate df/d2d/d1f/d7e/f91 81570 0 2024-03-24T10:46:03.113 INFO:tasks.workunit.client.0.smithi094.stdout:9/989: fdatasync df/d2d/d1f/d58/d5a/d50/da5/d101/f13b 0 2024-03-24T10:46:03.113 INFO:tasks.workunit.client.0.smithi094.stdout:3/785: sync 2024-03-24T10:46:03.115 INFO:tasks.workunit.client.0.smithi094.stdout:4/883: creat d0/d22/de/d28/d2b/d33/da4/d95/f10e x:0 0 0 2024-03-24T10:46:03.115 INFO:tasks.workunit.client.0.smithi094.stdout:4/884: write d0/d22/f76 [5492461,36483] 0 2024-03-24T10:46:03.119 INFO:tasks.workunit.client.0.smithi094.stdout:2/864: symlink d18/d66/dbe/ddf/lfe 0 2024-03-24T10:46:03.119 INFO:tasks.workunit.client.0.smithi094.stdout:2/865: chown d18/d48/d56/dcc/c75 124503049 1 2024-03-24T10:46:03.120 INFO:tasks.workunit.client.0.smithi094.stdout:9/990: truncate df/d2d/d1f/d58/d70/dab/fef 1629294 0 2024-03-24T10:46:03.121 INFO:tasks.workunit.client.0.smithi094.stdout:9/991: read df/f2a [2413731,49320] 0 2024-03-24T10:46:03.122 INFO:tasks.workunit.client.0.smithi094.stdout:3/786: creat d0/db/dc3/dea/f114 x:0 0 0 2024-03-24T10:46:03.124 INFO:tasks.workunit.client.0.smithi094.stdout:5/860: getdents d7/d23/dd5/d64/dd6 0 2024-03-24T10:46:03.124 INFO:tasks.workunit.client.0.smithi094.stdout:5/861: getdents d7/d23/d5f/d9/dde/df7 0 2024-03-24T10:46:03.124 INFO:tasks.workunit.client.0.smithi094.stdout:5/862: fdatasync d7/d23/dd5/d64/dd6/f3e 0 2024-03-24T10:46:03.124 INFO:tasks.workunit.client.0.smithi094.stdout:5/863: chown d7/d23/d5f/d9/d48/f10e 7 1 2024-03-24T10:46:03.124 INFO:tasks.workunit.client.0.smithi094.stdout:9/992: dread df/d2d/d1f/f4a [0,4194304] 0 2024-03-24T10:46:03.125 INFO:tasks.workunit.client.0.smithi094.stdout:7/832: sync 2024-03-24T10:46:03.125 INFO:tasks.workunit.client.0.smithi094.stdout:6/856: sync 2024-03-24T10:46:03.125 INFO:tasks.workunit.client.0.smithi094.stdout:4/885: dread d0/d5/d1b/d4f/fa7 [0,4194304] 0 2024-03-24T10:46:03.128 INFO:tasks.workunit.client.0.smithi094.stdout:5/864: dread d7/d23/d5f/d14/d36/f4d [8388608,4194304] 0 2024-03-24T10:46:03.130 INFO:tasks.workunit.client.0.smithi094.stdout:2/866: write d18/d66/dbe/dc0/f55 [6086180,89431] 0 2024-03-24T10:46:03.130 INFO:tasks.workunit.client.0.smithi094.stdout:6/857: mkdir d0/d30/d114 0 2024-03-24T10:46:03.130 INFO:tasks.workunit.client.0.smithi094.stdout:9/993: mknod df/d2d/d1f/d58/c146 0 2024-03-24T10:46:03.130 INFO:tasks.workunit.client.0.smithi094.stdout:2/867: stat d18/d66/d90/d9a/dbc/f6f 0 2024-03-24T10:46:03.130 INFO:tasks.workunit.client.0.smithi094.stdout:2/868: write d18/d66/d90/ff2 [363557,85004] 0 2024-03-24T10:46:03.130 INFO:tasks.workunit.client.0.smithi094.stdout:7/833: symlink d1/d9/d16/d21/d51/d83/d39/d44/dd9/d8f/l11e 0 2024-03-24T10:46:03.131 INFO:tasks.workunit.client.0.smithi094.stdout:2/869: fdatasync d18/f3f 0 2024-03-24T10:46:03.131 INFO:tasks.workunit.client.0.smithi094.stdout:4/886: mknod d0/d22/de/d28/d2b/d33/c10f 0 2024-03-24T10:46:03.131 INFO:tasks.workunit.client.0.smithi094.stdout:4/887: chown d0/d22/de/d28/d2b/d33/c10f 0 1 2024-03-24T10:46:03.131 INFO:tasks.workunit.client.0.smithi094.stdout:4/888: rename d0/d22/de/d28/d2b to d0/d22/de/d28/d2b/d7a/d110 22 2024-03-24T10:46:03.131 INFO:tasks.workunit.client.0.smithi094.stdout:4/889: fsync d0/d34/fac 0 2024-03-24T10:46:03.131 INFO:tasks.workunit.client.0.smithi094.stdout:5/865: creat d7/d23/d2b/d68/f118 x:0 0 0 2024-03-24T10:46:03.132 INFO:tasks.workunit.client.0.smithi094.stdout:9/994: symlink df/d2d/d1f/d11/d53/dde/l147 0 2024-03-24T10:46:03.133 INFO:tasks.workunit.client.0.smithi094.stdout:7/834: mkdir d1/d11f 0 2024-03-24T10:46:03.133 INFO:tasks.workunit.client.0.smithi094.stdout:7/835: write d1/d9/f23 [3893988,89063] 0 2024-03-24T10:46:03.133 INFO:tasks.workunit.client.0.smithi094.stdout:7/836: readlink d1/d9/d16/d21/d51/d83/d39/d44/dd9/da5/l115 0 2024-03-24T10:46:03.134 INFO:tasks.workunit.client.0.smithi094.stdout:5/866: mknod d7/d23/d5f/d14/d36/d3a/c119 0 2024-03-24T10:46:03.134 INFO:tasks.workunit.client.0.smithi094.stdout:2/870: rmdir d18/d48/dc3 39 2024-03-24T10:46:03.136 INFO:tasks.workunit.client.0.smithi094.stdout:9/995: rmdir df/d2d/d1f/d58/d5a/d61/d131 0 2024-03-24T10:46:03.136 INFO:tasks.workunit.client.0.smithi094.stdout:9/996: truncate df/d2d/d1f/d58/d5a/d61/d87/dc8/f10d 440529 0 2024-03-24T10:46:03.137 INFO:tasks.workunit.client.0.smithi094.stdout:5/867: symlink d7/dc/l11a 0 2024-03-24T10:46:03.138 INFO:tasks.workunit.client.0.smithi094.stdout:5/868: creat d7/dc/d100/d109/f11b x:0 0 0 2024-03-24T10:46:03.139 INFO:tasks.workunit.client.0.smithi094.stdout:5/869: mkdir d7/d23/d11c 0 2024-03-24T10:46:03.140 INFO:tasks.workunit.client.0.smithi094.stdout:5/870: chown d7/d23/d2b/def 56 1 2024-03-24T10:46:03.140 INFO:tasks.workunit.client.0.smithi094.stdout:5/871: creat d7/d23/d2b/d68/d81/f11d x:0 0 0 2024-03-24T10:46:03.142 INFO:tasks.workunit.client.0.smithi094.stdout:5/872: rename d7/d23/d5f/d14/d36/cb4 to d7/d23/d5f/d9/d48/df9/c11e 0 2024-03-24T10:46:03.142 INFO:tasks.workunit.client.0.smithi094.stdout:3/787: dwrite d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d4f/f109 [0,4194304] 0 2024-03-24T10:46:03.142 INFO:tasks.workunit.client.0.smithi094.stdout:3/788: write d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/db5/d106/d9b/fa1 [834512,38805] 0 2024-03-24T10:46:03.143 INFO:tasks.workunit.client.0.smithi094.stdout:5/873: symlink d7/d23/d5f/d14/d9f/l11f 0 2024-03-24T10:46:03.171 INFO:tasks.workunit.client.0.smithi094.stdout:6/858: dwrite d0/d21/f56 [0,4194304] 0 2024-03-24T10:46:03.176 INFO:tasks.workunit.client.0.smithi094.stdout:6/859: rename d0/d21/d2a/d2f/c32 to d0/d21/d2a/c115 0 2024-03-24T10:46:03.178 INFO:tasks.workunit.client.0.smithi094.stdout:6/860: mknod d0/d30/c116 0 2024-03-24T10:46:03.178 INFO:tasks.workunit.client.0.smithi094.stdout:6/861: write d0/d78/da/f20 [1029646,37270] 0 2024-03-24T10:46:03.180 INFO:tasks.workunit.client.0.smithi094.stdout:2/871: dread d18/d48/fbf [0,4194304] 0 2024-03-24T10:46:03.180 INFO:tasks.workunit.client.0.smithi094.stdout:2/872: stat d18/d1e/d65/ff9 0 2024-03-24T10:46:03.180 INFO:tasks.workunit.client.0.smithi094.stdout:6/862: symlink d0/da9/l117 0 2024-03-24T10:46:03.184 INFO:tasks.workunit.client.0.smithi094.stdout:4/890: dwrite d0/f5e [0,4194304] 0 2024-03-24T10:46:03.184 INFO:tasks.workunit.client.0.smithi094.stdout:4/891: read - d0/d22/de/d28/d2b/d33/da4/da5/f107 zero size 2024-03-24T10:46:03.184 INFO:tasks.workunit.client.0.smithi094.stdout:4/892: dread - d0/d22/de/dc8/de6/ff9 zero size 2024-03-24T10:46:03.184 INFO:tasks.workunit.client.0.smithi094.stdout:9/997: dwrite df/d2d/d1f/d7e/fc1 [0,4194304] 0 2024-03-24T10:46:03.187 INFO:tasks.workunit.client.0.smithi094.stdout:6/863: mknod d0/d21/d2a/d55/dce/c118 0 2024-03-24T10:46:03.194 INFO:tasks.workunit.client.0.smithi094.stdout:6/864: mkdir d0/d78/da/d23/d34/d119 0 2024-03-24T10:46:03.197 INFO:tasks.workunit.client.0.smithi094.stdout:6/865: rename d0/d78/da/d23/d34/f91 to d0/d21/d2a/dcb/f11a 0 2024-03-24T10:46:03.198 INFO:tasks.workunit.client.0.smithi094.stdout:6/866: mkdir d0/d78/d66/d7a/dfe/d11b 0 2024-03-24T10:46:03.199 INFO:tasks.workunit.client.0.smithi094.stdout:6/867: dread d0/d21/d2a/d55/dce/fd4 [4194304,4194304] 0 2024-03-24T10:46:03.199 INFO:tasks.workunit.client.0.smithi094.stdout:6/868: chown d0/d78/da/d23/d34 953774823 1 2024-03-24T10:46:03.200 INFO:tasks.workunit.client.0.smithi094.stdout:7/837: dwrite d1/d9/d16/d21/d51/d83/f72 [0,4194304] 0 2024-03-24T10:46:03.204 INFO:tasks.workunit.client.0.smithi094.stdout:2/873: dwrite d18/f23 [4194304,4194304] 0 2024-03-24T10:46:03.204 INFO:tasks.workunit.client.0.smithi094.stdout:2/874: chown d18/d5f 1077845482 1 2024-03-24T10:46:03.204 INFO:tasks.workunit.client.0.smithi094.stdout:2/875: chown d18/d7f/d88/d8a/db2/fe2 2150531 1 2024-03-24T10:46:03.207 INFO:tasks.workunit.client.0.smithi094.stdout:2/876: dread d18/d54/fa7 [0,4194304] 0 2024-03-24T10:46:03.207 INFO:tasks.workunit.client.0.smithi094.stdout:2/877: write d18/d48/d56/dcc/f7a [1333968,11426] 0 2024-03-24T10:46:03.214 INFO:tasks.workunit.client.0.smithi094.stdout:2/878: dread d18/d1e/d65/f6a [0,4194304] 0 2024-03-24T10:46:03.214 INFO:tasks.workunit.client.0.smithi094.stdout:2/879: write d18/d66/dbe/dc0/fb6 [861320,123157] 0 2024-03-24T10:46:03.214 INFO:tasks.workunit.client.0.smithi094.stdout:2/880: readlink l5 0 2024-03-24T10:46:03.214 INFO:tasks.workunit.client.0.smithi094.stdout:2/881: readlink d18/d1e/l2d 0 2024-03-24T10:46:03.214 INFO:tasks.workunit.client.0.smithi094.stdout:2/882: symlink d18/d1e/dfd/lff 0 2024-03-24T10:46:03.214 INFO:tasks.workunit.client.0.smithi094.stdout:2/883: symlink d18/d5f/l100 0 2024-03-24T10:46:03.214 INFO:tasks.workunit.client.0.smithi094.stdout:2/884: creat d18/d7f/d88/da2/f101 x:0 0 0 2024-03-24T10:46:03.215 INFO:tasks.workunit.client.0.smithi094.stdout:2/885: mkdir d18/d54/d102 0 2024-03-24T10:46:03.215 INFO:tasks.workunit.client.0.smithi094.stdout:4/893: dwrite d0/d34/fac [0,4194304] 0 2024-03-24T10:46:03.215 INFO:tasks.workunit.client.0.smithi094.stdout:4/894: readlink d0/d5/d7d/d97/l4c 0 2024-03-24T10:46:03.216 INFO:tasks.workunit.client.0.smithi094.stdout:4/895: creat d0/d22/de/d28/d2b/da2/f111 x:0 0 0 2024-03-24T10:46:03.216 INFO:tasks.workunit.client.0.smithi094.stdout:7/838: dwrite d1/d6c/f71 [0,4194304] 0 2024-03-24T10:46:03.219 INFO:tasks.workunit.client.0.smithi094.stdout:7/839: creat d1/f120 x:0 0 0 2024-03-24T10:46:03.222 INFO:tasks.workunit.client.0.smithi094.stdout:6/869: dwrite d0/d1a/f6f [0,4194304] 0 2024-03-24T10:46:03.223 INFO:tasks.workunit.client.0.smithi094.stdout:9/998: dwrite df/d2d/d1f/d58/d70/dab/d80/f138 [0,4194304] 0 2024-03-24T10:46:03.226 INFO:tasks.workunit.client.0.smithi094.stdout:8/896: sync 2024-03-24T10:46:03.226 INFO:tasks.workunit.client.0.smithi094.stdout:1/766: sync 2024-03-24T10:46:03.226 INFO:tasks.workunit.client.0.smithi094.stdout:8/897: write df/d25/d36/d3f/f8e [3268515,86332] 0 2024-03-24T10:46:03.226 INFO:tasks.workunit.client.0.smithi094.stdout:8/898: fsync df/d25/d28/d31/d47/d66/da3/da8/d117/dc1/fe1 0 2024-03-24T10:46:03.226 INFO:tasks.workunit.client.0.smithi094.stdout:8/899: dread - df/d25/daf/fda zero size 2024-03-24T10:46:03.226 INFO:tasks.workunit.client.0.smithi094.stdout:6/870: unlink d0/d21/d2a/f40 0 2024-03-24T10:46:03.229 INFO:tasks.workunit.client.0.smithi094.stdout:9/999: creat df/d2d/d1f/d58/d5a/d61/f148 x:0 0 0 2024-03-24T10:46:03.230 INFO:tasks.workunit.client.0.smithi094.stdout:1/767: mknod d4/d9/db/d31/d3c/d9f/c108 0 2024-03-24T10:46:03.230 INFO:tasks.workunit.client.0.smithi094.stdout:1/768: write d4/d9/de/d6f/f4e [1659649,106941] 0 2024-03-24T10:46:03.231 INFO:tasks.workunit.client.0.smithi094.stdout:8/900: rename df/d25/d28/d31/d47/d66/da3/db5/cc2 to df/d25/d28/d31/d95/de3/c12c 0 2024-03-24T10:46:03.232 INFO:tasks.workunit.client.0.smithi094.stdout:6/871: unlink d0/d78/da/d23/d34/d4e/lc0 0 2024-03-24T10:46:03.233 INFO:tasks.workunit.client.0.smithi094.stdout:8/901: rmdir df/d25/d28/d31/d47/d66/da3 39 2024-03-24T10:46:03.233 INFO:tasks.workunit.client.0.smithi094.stdout:8/902: fdatasync df/d25/d28/d31/d47/d66/da3/db5/fc3 0 2024-03-24T10:46:03.234 INFO:tasks.workunit.client.0.smithi094.stdout:6/872: unlink d0/d21/l4a 0 2024-03-24T10:46:03.234 INFO:tasks.workunit.client.0.smithi094.stdout:8/903: symlink df/d25/d36/l12d 0 2024-03-24T10:46:03.235 INFO:tasks.workunit.client.0.smithi094.stdout:8/904: truncate df/fd1 1030559 0 2024-03-24T10:46:03.235 INFO:tasks.workunit.client.0.smithi094.stdout:6/873: truncate d0/d78/da/d23/f9b 1509271 0 2024-03-24T10:46:03.236 INFO:tasks.workunit.client.0.smithi094.stdout:8/905: symlink df/d25/d28/d31/d47/ddf/l12e 0 2024-03-24T10:46:03.236 INFO:tasks.workunit.client.0.smithi094.stdout:8/906: write df/d25/d28/d31/d47/d22/f38 [5116461,33469] 0 2024-03-24T10:46:03.237 INFO:tasks.workunit.client.0.smithi094.stdout:2/886: dwrite d18/d5f/fd7 [0,4194304] 0 2024-03-24T10:46:03.238 INFO:tasks.workunit.client.0.smithi094.stdout:1/769: write d4/d9/db/d5d/d63/f6a [825839,108925] 0 2024-03-24T10:46:03.238 INFO:tasks.workunit.client.0.smithi094.stdout:1/770: write d4/d9/de/f3b [3543549,56037] 0 2024-03-24T10:46:03.238 INFO:tasks.workunit.client.0.smithi094.stdout:6/874: rename d0/d21/d2a/d2f to d0/d78/da/d23/dde/d11c 0 2024-03-24T10:46:03.239 INFO:tasks.workunit.client.0.smithi094.stdout:4/896: dwrite d0/d5/d1b/f10b [0,4194304] 0 2024-03-24T10:46:03.241 INFO:tasks.workunit.client.0.smithi094.stdout:2/887: symlink d18/d7f/d88/d8a/l103 0 2024-03-24T10:46:03.241 INFO:tasks.workunit.client.0.smithi094.stdout:2/888: write d18/d7f/d88/d8a/db2/fe2 [7354205,39503] 0 2024-03-24T10:46:03.246 INFO:tasks.workunit.client.0.smithi094.stdout:1/771: rename d4/d9/de/d6f/d62/ld8 to d4/d9/db/d85/df5/l109 0 2024-03-24T10:46:03.246 INFO:tasks.workunit.client.0.smithi094.stdout:1/772: chown d4/l40 1817 1 2024-03-24T10:46:03.247 INFO:tasks.workunit.client.0.smithi094.stdout:1/773: dread d4/d9/f28 [0,4194304] 0 2024-03-24T10:46:03.250 INFO:tasks.workunit.client.0.smithi094.stdout:4/897: mkdir d0/d5/d7d/d9c/d112 0 2024-03-24T10:46:03.250 INFO:tasks.workunit.client.0.smithi094.stdout:4/898: creat d0/d34/d73/d93/f113 x:0 0 0 2024-03-24T10:46:03.250 INFO:tasks.workunit.client.0.smithi094.stdout:4/899: creat d0/d5/d7d/d97/d4a/f114 x:0 0 0 2024-03-24T10:46:03.251 INFO:tasks.workunit.client.0.smithi094.stdout:4/900: write d0/d22/de/d28/d2b/d33/da4/da5/fe2 [680836,86907] 0 2024-03-24T10:46:03.253 INFO:tasks.workunit.client.0.smithi094.stdout:1/774: truncate d4/d9/f2d 2906756 0 2024-03-24T10:46:03.253 INFO:tasks.workunit.client.0.smithi094.stdout:1/775: creat d4/d9/de/d6f/d62/d99/f10a x:0 0 0 2024-03-24T10:46:03.253 INFO:tasks.workunit.client.0.smithi094.stdout:1/776: readlink d4/d9/d37/dcf/ld2 0 2024-03-24T10:46:03.257 INFO:tasks.workunit.client.0.smithi094.stdout:4/901: truncate d0/d34/fac 3215046 0 2024-03-24T10:46:03.260 INFO:tasks.workunit.client.0.smithi094.stdout:1/777: rename d4/d9/db/l61 to d4/d9/db/d85/l10b 0 2024-03-24T10:46:03.260 INFO:tasks.workunit.client.0.smithi094.stdout:1/778: chown d4/d9/db/d85 189 1 2024-03-24T10:46:03.261 INFO:tasks.workunit.client.0.smithi094.stdout:4/902: mknod d0/d22/de4/dfa/c115 0 2024-03-24T10:46:03.263 INFO:tasks.workunit.client.0.smithi094.stdout:4/903: symlink d0/d5/d7d/d97/l116 0 2024-03-24T10:46:03.265 INFO:tasks.workunit.client.0.smithi094.stdout:1/779: getdents d4/d9/d37/d33/d76 0 2024-03-24T10:46:03.265 INFO:tasks.workunit.client.0.smithi094.stdout:1/780: chown d4/d9/db1/db9 36 1 2024-03-24T10:46:03.266 INFO:tasks.workunit.client.0.smithi094.stdout:4/904: creat d0/d5/d1b/de8/f117 x:0 0 0 2024-03-24T10:46:03.267 INFO:tasks.workunit.client.0.smithi094.stdout:1/781: rename d4/d9/d37/l4a to d4/d9/de/d3d/d66/dbe/l10c 0 2024-03-24T10:46:03.267 INFO:tasks.workunit.client.0.smithi094.stdout:1/782: fdatasync d4/d9/f32 0 2024-03-24T10:46:03.267 INFO:tasks.workunit.client.0.smithi094.stdout:1/783: dread - d4/d9/db/d39/d75/fe5 zero size 2024-03-24T10:46:03.268 INFO:tasks.workunit.client.0.smithi094.stdout:1/784: chown d4/d9/d7d/l6e 8638368 1 2024-03-24T10:46:03.270 INFO:tasks.workunit.client.0.smithi094.stdout:8/907: dwrite df/d25/d28/f105 [0,4194304] 0 2024-03-24T10:46:03.272 INFO:tasks.workunit.client.0.smithi094.stdout:8/908: rmdir df/d25/d28/d31 39 2024-03-24T10:46:03.272 INFO:tasks.workunit.client.0.smithi094.stdout:8/909: write df/d25/d28/d31/d3c/f41 [6135589,48974] 0 2024-03-24T10:46:03.274 INFO:tasks.workunit.client.0.smithi094.stdout:6/875: dwrite d0/d78/da/d23/d7c/db0/fc6 [0,4194304] 0 2024-03-24T10:46:03.274 INFO:tasks.workunit.client.0.smithi094.stdout:6/876: readlink d0/d78/da/d23/d34/l45 0 2024-03-24T10:46:03.274 INFO:tasks.workunit.client.0.smithi094.stdout:6/877: fsync d0/d30/de6/f9e 0 2024-03-24T10:46:03.274 INFO:tasks.workunit.client.0.smithi094.stdout:6/878: readlink d0/da9/l117 0 2024-03-24T10:46:03.274 INFO:tasks.workunit.client.0.smithi094.stdout:2/889: dwrite d18/d7f/d88/da2/f101 [0,4194304] 0 2024-03-24T10:46:03.274 INFO:tasks.workunit.client.0.smithi094.stdout:2/890: dread - d18/d54/db9/fba zero size 2024-03-24T10:46:03.274 INFO:tasks.workunit.client.0.smithi094.stdout:8/910: rename df/d12/d5b/c60 to df/d25/d28/c12f 0 2024-03-24T10:46:03.279 INFO:tasks.workunit.client.0.smithi094.stdout:6/879: write d0/d78/f29 [2237173,122606] 0 2024-03-24T10:46:03.281 INFO:tasks.workunit.client.0.smithi094.stdout:2/891: truncate d18/f4e 1521580 0 2024-03-24T10:46:03.281 INFO:tasks.workunit.client.0.smithi094.stdout:2/892: dread - d18/d66/dbe/fe7 zero size 2024-03-24T10:46:03.281 INFO:tasks.workunit.client.0.smithi094.stdout:2/893: fsync d18/d48/d56/ff1 0 2024-03-24T10:46:03.281 INFO:tasks.workunit.client.0.smithi094.stdout:2/894: truncate d18/d1e/d65/fd5 821959 0 2024-03-24T10:46:03.285 INFO:tasks.workunit.client.0.smithi094.stdout:8/911: mkdir df/d12/d5b/d63/d130 0 2024-03-24T10:46:03.287 INFO:tasks.workunit.client.0.smithi094.stdout:4/905: rmdir d0/d34/d73 39 2024-03-24T10:46:03.290 INFO:tasks.workunit.client.0.smithi094.stdout:4/906: mkdir d0/d5/d1b/d4f/dd6/d118 0 2024-03-24T10:46:03.291 INFO:tasks.workunit.client.0.smithi094.stdout:4/907: creat d0/d22/de/d28/d3d/d10d/f119 x:0 0 0 2024-03-24T10:46:03.293 INFO:tasks.workunit.client.0.smithi094.stdout:4/908: creat d0/d5/d1b/d4f/f11a x:0 0 0 2024-03-24T10:46:03.298 INFO:tasks.workunit.client.0.smithi094.stdout:8/912: dwrite df/d25/d28/d31/d47/d66/da3/da8/fd8 [0,4194304] 0 2024-03-24T10:46:03.298 INFO:tasks.workunit.client.0.smithi094.stdout:8/913: readlink df/d25/d28/d31/lb6 0 2024-03-24T10:46:03.298 INFO:tasks.workunit.client.0.smithi094.stdout:2/895: dwrite d18/d48/f5d [0,4194304] 0 2024-03-24T10:46:03.310 INFO:tasks.workunit.client.0.smithi094.stdout:8/914: mknod df/d25/d36/d3f/c131 0 2024-03-24T10:46:03.311 INFO:tasks.workunit.client.0.smithi094.stdout:8/915: dread - df/d25/daf/fda zero size 2024-03-24T10:46:03.311 INFO:tasks.workunit.client.0.smithi094.stdout:8/916: fdatasync df/d25/d28/d31/d47/d66/fe2 0 2024-03-24T10:46:03.311 INFO:tasks.workunit.client.0.smithi094.stdout:8/917: stat df/f1b 0 2024-03-24T10:46:03.312 INFO:tasks.workunit.client.0.smithi094.stdout:4/909: dwrite d0/d5/d7d/f99 [0,4194304] 0 2024-03-24T10:46:03.312 INFO:tasks.workunit.client.0.smithi094.stdout:4/910: chown d0/d22/de/dc8 3330 1 2024-03-24T10:46:03.312 INFO:tasks.workunit.client.0.smithi094.stdout:4/911: write d0/d22/de/d28/d3d/d8c/fdb [203991,15816] 0 2024-03-24T10:46:03.312 INFO:tasks.workunit.client.0.smithi094.stdout:4/912: stat d0/d22/de/d28/d2b/fd3 0 2024-03-24T10:46:03.312 INFO:tasks.workunit.client.0.smithi094.stdout:4/913: fdatasync d0/d22/de/d28/d2b/f69 0 2024-03-24T10:46:03.312 INFO:tasks.workunit.client.0.smithi094.stdout:8/918: mknod df/d25/daf/c132 0 2024-03-24T10:46:03.314 INFO:tasks.workunit.client.0.smithi094.stdout:4/914: rmdir d0/d5/d7d/d9c/d112 0 2024-03-24T10:46:03.315 INFO:tasks.workunit.client.0.smithi094.stdout:8/919: creat df/d25/d28/d31/d47/d66/da3/db5/f133 x:0 0 0 2024-03-24T10:46:03.315 INFO:tasks.workunit.client.0.smithi094.stdout:8/920: fsync df/d25/d28/d31/d47/d22/d55/f6d 0 2024-03-24T10:46:03.317 INFO:tasks.workunit.client.0.smithi094.stdout:4/915: rename d0/d5/d7d/d97/d4a/fdd to d0/d22/de/f11b 0 2024-03-24T10:46:03.318 INFO:tasks.workunit.client.0.smithi094.stdout:8/921: symlink df/d25/d28/d31/d47/dbd/l134 0 2024-03-24T10:46:03.318 INFO:tasks.workunit.client.0.smithi094.stdout:4/916: creat d0/d5/d1b/d4f/dd6/d118/f11c x:0 0 0 2024-03-24T10:46:03.319 INFO:tasks.workunit.client.0.smithi094.stdout:4/917: chown d0/d22/de/d28/d2b/d33/f54 55257102 1 2024-03-24T10:46:03.319 INFO:tasks.workunit.client.0.smithi094.stdout:2/896: dwrite d18/d66/dbe/ddf/ffb [0,4194304] 0 2024-03-24T10:46:03.319 INFO:tasks.workunit.client.0.smithi094.stdout:2/897: fdatasync d18/d66/dbe/fee 0 2024-03-24T10:46:03.319 INFO:tasks.workunit.client.0.smithi094.stdout:8/922: creat df/d25/daf/f135 x:0 0 0 2024-03-24T10:46:03.320 INFO:tasks.workunit.client.0.smithi094.stdout:4/918: symlink d0/l11d 0 2024-03-24T10:46:03.323 INFO:tasks.workunit.client.0.smithi094.stdout:2/898: truncate d18/d66/d90/d9a/dbc/f44 657934 0 2024-03-24T10:46:03.324 INFO:tasks.workunit.client.0.smithi094.stdout:8/923: mkdir df/d25/d28/d31/d47/dfe/d136 0 2024-03-24T10:46:03.325 INFO:tasks.workunit.client.0.smithi094.stdout:4/919: creat d0/d22/de/d28/d3d/d91/f11e x:0 0 0 2024-03-24T10:46:03.325 INFO:tasks.workunit.client.0.smithi094.stdout:3/789: sync 2024-03-24T10:46:03.325 INFO:tasks.workunit.client.0.smithi094.stdout:5/874: sync 2024-03-24T10:46:03.325 INFO:tasks.workunit.client.0.smithi094.stdout:0/863: sync 2024-03-24T10:46:03.325 INFO:tasks.workunit.client.0.smithi094.stdout:0/864: fsync d14/f1e 0 2024-03-24T10:46:03.326 INFO:tasks.workunit.client.0.smithi094.stdout:0/865: readlink d14/lbf 0 2024-03-24T10:46:03.326 INFO:tasks.workunit.client.0.smithi094.stdout:0/866: fsync d14/d22/d32/d70/ff5 0 2024-03-24T10:46:03.326 INFO:tasks.workunit.client.0.smithi094.stdout:8/924: dread df/d25/d28/d31/d47/d66/da3/da8/fe5 [0,4194304] 0 2024-03-24T10:46:03.326 INFO:tasks.workunit.client.0.smithi094.stdout:8/925: readlink df/d12/l4d 0 2024-03-24T10:46:03.327 INFO:tasks.workunit.client.0.smithi094.stdout:8/926: dread - df/d25/f129 zero size 2024-03-24T10:46:03.327 INFO:tasks.workunit.client.0.smithi094.stdout:8/927: read df/d25/d28/fa7 [519357,114713] 0 2024-03-24T10:46:03.327 INFO:tasks.workunit.client.0.smithi094.stdout:4/920: mkdir d0/d5/d11f 0 2024-03-24T10:46:03.327 INFO:tasks.workunit.client.0.smithi094.stdout:5/875: dread d7/d26/f65 [0,4194304] 0 2024-03-24T10:46:03.329 INFO:tasks.workunit.client.0.smithi094.stdout:3/790: getdents d0/db/dc/d28 0 2024-03-24T10:46:03.329 INFO:tasks.workunit.client.0.smithi094.stdout:2/899: getdents d18/d66/dbe/ddf 0 2024-03-24T10:46:03.329 INFO:tasks.workunit.client.0.smithi094.stdout:3/791: fdatasync d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/db5/d106/d9b/d10d/f89 0 2024-03-24T10:46:03.330 INFO:tasks.workunit.client.0.smithi094.stdout:4/921: creat d0/d22/de4/f120 x:0 0 0 2024-03-24T10:46:03.332 INFO:tasks.workunit.client.0.smithi094.stdout:5/876: link d7/d23/d5f/d14/d6f/dbe/fd9 d7/d23/d5f/d9/f120 0 2024-03-24T10:46:03.332 INFO:tasks.workunit.client.0.smithi094.stdout:5/877: chown d7/d26/dce/lda 232246 1 2024-03-24T10:46:03.333 INFO:tasks.workunit.client.0.smithi094.stdout:2/900: chown d18/d77/dc6 174 1 2024-03-24T10:46:03.333 INFO:tasks.workunit.client.0.smithi094.stdout:2/901: creat d18/d66/d90/f104 x:0 0 0 2024-03-24T10:46:03.333 INFO:tasks.workunit.client.0.smithi094.stdout:2/902: fsync d18/f27 0 2024-03-24T10:46:03.334 INFO:tasks.workunit.client.0.smithi094.stdout:4/922: write d0/d5/d1b/faf [73934,9325] 0 2024-03-24T10:46:03.336 INFO:tasks.workunit.client.0.smithi094.stdout:3/792: creat d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/dfb/f115 x:0 0 0 2024-03-24T10:46:03.340 INFO:tasks.workunit.client.0.smithi094.stdout:2/903: unlink d18/d7f/d88/d8a/la3 0 2024-03-24T10:46:03.341 INFO:tasks.workunit.client.0.smithi094.stdout:4/923: symlink d0/d22/de/d28/d2b/l121 0 2024-03-24T10:46:03.341 INFO:tasks.workunit.client.0.smithi094.stdout:5/878: write d7/d26/f80 [2462904,46221] 0 2024-03-24T10:46:03.341 INFO:tasks.workunit.client.0.smithi094.stdout:5/879: chown d7/l33 1284 1 2024-03-24T10:46:03.342 INFO:tasks.workunit.client.0.smithi094.stdout:2/904: creat d18/d66/d90/d9a/d74/d7e/dca/f105 x:0 0 0 2024-03-24T10:46:03.343 INFO:tasks.workunit.client.0.smithi094.stdout:4/924: rmdir d0/d22/de/d28/d3d 39 2024-03-24T10:46:03.343 INFO:tasks.workunit.client.0.smithi094.stdout:4/925: truncate d0/d22/de/d28/d2b/fd3 667824 0 2024-03-24T10:46:03.344 INFO:tasks.workunit.client.0.smithi094.stdout:2/905: creat d18/d66/dbe/dc8/de6/f106 x:0 0 0 2024-03-24T10:46:03.344 INFO:tasks.workunit.client.0.smithi094.stdout:2/906: creat d18/d7f/d88/da2/f107 x:0 0 0 2024-03-24T10:46:03.346 INFO:tasks.workunit.client.0.smithi094.stdout:4/926: dread d0/d5/d7d/fc3 [0,4194304] 0 2024-03-24T10:46:03.346 INFO:tasks.workunit.client.0.smithi094.stdout:4/927: write d0/d22/de4/f120 [10114,120800] 0 2024-03-24T10:46:03.347 INFO:tasks.workunit.client.0.smithi094.stdout:4/928: write d0/f5f [2566745,15870] 0 2024-03-24T10:46:03.347 INFO:tasks.workunit.client.0.smithi094.stdout:2/907: creat d18/d66/d90/d9a/d74/db4/dd4/f108 x:0 0 0 2024-03-24T10:46:03.349 INFO:tasks.workunit.client.0.smithi094.stdout:3/793: write d0/db/dc/d28/d3c/d7c/fb0 [3729805,58151] 0 2024-03-24T10:46:03.349 INFO:tasks.workunit.client.0.smithi094.stdout:4/929: mkdir d0/d22/de/d28/d122 0 2024-03-24T10:46:03.351 INFO:tasks.workunit.client.0.smithi094.stdout:8/928: dwrite df/d12/f6a [0,4194304] 0 2024-03-24T10:46:03.351 INFO:tasks.workunit.client.0.smithi094.stdout:2/908: mknod d18/d77/c109 0 2024-03-24T10:46:03.355 INFO:tasks.workunit.client.0.smithi094.stdout:0/867: dwrite d14/d22/d28/d2b/d41/d53/fbb [0,4194304] 0 2024-03-24T10:46:03.357 INFO:tasks.workunit.client.0.smithi094.stdout:5/880: dread d7/f10 [0,4194304] 0 2024-03-24T10:46:03.359 INFO:tasks.workunit.client.0.smithi094.stdout:2/909: write d18/d1e/f53 [3469830,53655] 0 2024-03-24T10:46:03.359 INFO:tasks.workunit.client.0.smithi094.stdout:2/910: stat d18/d66/dbe/ddf/lfe 0 2024-03-24T10:46:03.366 INFO:tasks.workunit.client.0.smithi094.stdout:0/868: creat d14/d22/d28/d2b/d68/f131 x:0 0 0 2024-03-24T10:46:03.366 INFO:tasks.workunit.client.0.smithi094.stdout:4/930: dread d0/f5f [0,4194304] 0 2024-03-24T10:46:03.368 INFO:tasks.workunit.client.0.smithi094.stdout:5/881: creat d7/d23/f121 x:0 0 0 2024-03-24T10:46:03.368 INFO:tasks.workunit.client.0.smithi094.stdout:5/882: write d7/d23/d2b/d68/fc0 [986659,125802] 0 2024-03-24T10:46:03.368 INFO:tasks.workunit.client.0.smithi094.stdout:5/883: write d7/d23/d5f/f99 [1086630,50738] 0 2024-03-24T10:46:03.370 INFO:tasks.workunit.client.0.smithi094.stdout:2/911: rename d18/d48/c5b to d18/d7f/d88/ded/c10a 0 2024-03-24T10:46:03.371 INFO:tasks.workunit.client.0.smithi094.stdout:0/869: mkdir d14/d22/d28/d2b/d41/d53/d132 0 2024-03-24T10:46:03.372 INFO:tasks.workunit.client.0.smithi094.stdout:3/794: dwrite d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/db5/d106/d9b/d10d/f78 [0,4194304] 0 2024-03-24T10:46:03.372 INFO:tasks.workunit.client.0.smithi094.stdout:4/931: creat d0/d34/d73/d93/f123 x:0 0 0 2024-03-24T10:46:03.376 INFO:tasks.workunit.client.0.smithi094.stdout:5/884: mknod d7/d23/d5f/d14/d6f/c122 0 2024-03-24T10:46:03.376 INFO:tasks.workunit.client.0.smithi094.stdout:5/885: dread - d7/d23/dd5/da9/fc8 zero size 2024-03-24T10:46:03.377 INFO:tasks.workunit.client.0.smithi094.stdout:2/912: mknod d18/d66/da6/c10b 0 2024-03-24T10:46:03.379 INFO:tasks.workunit.client.0.smithi094.stdout:0/870: creat d14/d22/d28/d2b/dc0/de7/f133 x:0 0 0 2024-03-24T10:46:03.379 INFO:tasks.workunit.client.0.smithi094.stdout:0/871: stat d14/d18/f21 0 2024-03-24T10:46:03.382 INFO:tasks.workunit.client.0.smithi094.stdout:3/795: creat d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/f116 x:0 0 0 2024-03-24T10:46:03.383 INFO:tasks.workunit.client.0.smithi094.stdout:5/886: mkdir d7/d23/d123 0 2024-03-24T10:46:03.384 INFO:tasks.workunit.client.0.smithi094.stdout:7/840: sync 2024-03-24T10:46:03.384 INFO:tasks.workunit.client.0.smithi094.stdout:1/785: sync 2024-03-24T10:46:03.384 INFO:tasks.workunit.client.0.smithi094.stdout:1/786: read - d4/d9/db/d31/f91 zero size 2024-03-24T10:46:03.385 INFO:tasks.workunit.client.0.smithi094.stdout:2/913: link d18/d66/d90/d9a/dbc/f51 d18/d54/d102/f10c 0 2024-03-24T10:46:03.387 INFO:tasks.workunit.client.0.smithi094.stdout:5/887: mknod d7/d23/dd5/d64/dff/c124 0 2024-03-24T10:46:03.389 INFO:tasks.workunit.client.0.smithi094.stdout:7/841: truncate d1/d9/f30 3695014 0 2024-03-24T10:46:03.389 INFO:tasks.workunit.client.0.smithi094.stdout:7/842: chown d1/d9/d16/d21/d49/da9/lca 6524 1 2024-03-24T10:46:03.390 INFO:tasks.workunit.client.0.smithi094.stdout:7/843: truncate d1/db4/d65/faf 738743 0 2024-03-24T10:46:03.390 INFO:tasks.workunit.client.0.smithi094.stdout:6/880: sync 2024-03-24T10:46:03.391 INFO:tasks.workunit.client.0.smithi094.stdout:4/932: dwrite d0/f60 [0,4194304] 0 2024-03-24T10:46:03.391 INFO:tasks.workunit.client.0.smithi094.stdout:4/933: chown d0/d22/de/le3 3 1 2024-03-24T10:46:03.391 INFO:tasks.workunit.client.0.smithi094.stdout:4/934: getdents d0/d22/de/dc8/de6/def 0 2024-03-24T10:46:03.391 INFO:tasks.workunit.client.0.smithi094.stdout:1/787: link d4/d9/db/d31/f91 d4/d9/de/d6f/d5f/f10d 0 2024-03-24T10:46:03.394 INFO:tasks.workunit.client.0.smithi094.stdout:6/881: creat d0/d78/da/d23/d34/d119/f11d x:0 0 0 2024-03-24T10:46:03.399 INFO:tasks.workunit.client.0.smithi094.stdout:4/935: rename d0/ce9 to d0/d22/de4/c124 0 2024-03-24T10:46:03.399 INFO:tasks.workunit.client.0.smithi094.stdout:2/914: dread d18/f1d [0,4194304] 0 2024-03-24T10:46:03.399 INFO:tasks.workunit.client.0.smithi094.stdout:1/788: creat d4/d9/f10e x:0 0 0 2024-03-24T10:46:03.399 INFO:tasks.workunit.client.0.smithi094.stdout:1/789: stat d4/d9/de/d6f/d62/ff9 0 2024-03-24T10:46:03.399 INFO:tasks.workunit.client.0.smithi094.stdout:6/882: mknod d0/d78/da/d23/d34/d4e/d5d/c11e 0 2024-03-24T10:46:03.400 INFO:tasks.workunit.client.0.smithi094.stdout:6/883: dread - d0/d78/da/d23/d34/d4e/f7e zero size 2024-03-24T10:46:03.400 INFO:tasks.workunit.client.0.smithi094.stdout:6/884: creat d0/d30/de6/d9c/f11f x:0 0 0 2024-03-24T10:46:03.400 INFO:tasks.workunit.client.0.smithi094.stdout:2/915: mknod d18/d66/d90/d9a/d74/db4/dd4/c10d 0 2024-03-24T10:46:03.400 INFO:tasks.workunit.client.0.smithi094.stdout:1/790: creat d4/d9/de/d3d/dae/f10f x:0 0 0 2024-03-24T10:46:03.401 INFO:tasks.workunit.client.0.smithi094.stdout:6/885: mknod d0/d21/c120 0 2024-03-24T10:46:03.402 INFO:tasks.workunit.client.0.smithi094.stdout:5/888: dread d7/d23/f44 [0,4194304] 0 2024-03-24T10:46:03.403 INFO:tasks.workunit.client.0.smithi094.stdout:2/916: rename d18/d66/d90/d9a/d74/f99 to d18/d66/dbe/dc8/f10e 0 2024-03-24T10:46:03.404 INFO:tasks.workunit.client.0.smithi094.stdout:5/889: mknod d7/dc/d100/c125 0 2024-03-24T10:46:03.405 INFO:tasks.workunit.client.0.smithi094.stdout:2/917: symlink d18/d48/l10f 0 2024-03-24T10:46:03.405 INFO:tasks.workunit.client.0.smithi094.stdout:2/918: chown d18 298526616 1 2024-03-24T10:46:03.406 INFO:tasks.workunit.client.0.smithi094.stdout:5/890: truncate d7/d23/d5f/d14/d6f/f89 1950318 0 2024-03-24T10:46:03.406 INFO:tasks.workunit.client.0.smithi094.stdout:5/891: chown d7/d23 1670 1 2024-03-24T10:46:03.407 INFO:tasks.workunit.client.0.smithi094.stdout:2/919: mknod d18/d54/d102/c110 0 2024-03-24T10:46:03.407 INFO:tasks.workunit.client.0.smithi094.stdout:2/920: stat d18/d54/f5c 0 2024-03-24T10:46:03.407 INFO:tasks.workunit.client.0.smithi094.stdout:0/872: dwrite d14/d22/d28/da2/d10a/f11e [0,4194304] 0 2024-03-24T10:46:03.407 INFO:tasks.workunit.client.0.smithi094.stdout:0/873: chown d14/d22/d32/d70/dd0/l12d 19425909 1 2024-03-24T10:46:03.408 INFO:tasks.workunit.client.0.smithi094.stdout:3/796: dwrite d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d87/f9c [0,4194304] 0 2024-03-24T10:46:03.408 INFO:tasks.workunit.client.0.smithi094.stdout:5/892: symlink d7/d23/d2b/l126 0 2024-03-24T10:46:03.409 INFO:tasks.workunit.client.0.smithi094.stdout:2/921: mkdir d18/d66/da6/d111 0 2024-03-24T10:46:03.413 INFO:tasks.workunit.client.0.smithi094.stdout:0/874: creat d14/d18/f134 x:0 0 0 2024-03-24T10:46:03.416 INFO:tasks.workunit.client.0.smithi094.stdout:4/936: dwrite d0/d22/de/d28/d2b/d7a/f84 [0,4194304] 0 2024-03-24T10:46:03.417 INFO:tasks.workunit.client.0.smithi094.stdout:5/893: symlink d7/d23/dd5/d64/dff/l127 0 2024-03-24T10:46:03.419 INFO:tasks.workunit.client.0.smithi094.stdout:2/922: unlink d18/d66/dbe/dc0/fb6 0 2024-03-24T10:46:03.421 INFO:tasks.workunit.client.0.smithi094.stdout:0/875: truncate d14/d22/d32/d70/ff5 122480 0 2024-03-24T10:46:03.422 INFO:tasks.workunit.client.0.smithi094.stdout:4/937: symlink d0/d34/d73/d93/l125 0 2024-03-24T10:46:03.424 INFO:tasks.workunit.client.0.smithi094.stdout:2/923: write d18/d66/d90/d9a/dbc/f50 [1916683,47657] 0 2024-03-24T10:46:03.425 INFO:tasks.workunit.client.0.smithi094.stdout:4/938: dread d0/d22/de/d28/d3d/d8c/fdb [0,4194304] 0 2024-03-24T10:46:03.426 INFO:tasks.workunit.client.0.smithi094.stdout:3/797: dwrite d0/f11 [0,4194304] 0 2024-03-24T10:46:03.426 INFO:tasks.workunit.client.0.smithi094.stdout:3/798: write d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/dfb/f115 [181013,88758] 0 2024-03-24T10:46:03.427 INFO:tasks.workunit.client.0.smithi094.stdout:3/799: dread - d0/db/dc/d28/d3c/d7c/fe0 zero size 2024-03-24T10:46:03.427 INFO:tasks.workunit.client.0.smithi094.stdout:3/800: dread - d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/db5/fc6 zero size 2024-03-24T10:46:03.427 INFO:tasks.workunit.client.0.smithi094.stdout:2/924: symlink d18/d66/d90/d9a/d74/d7e/dca/l112 0 2024-03-24T10:46:03.432 INFO:tasks.workunit.client.0.smithi094.stdout:5/894: dread d7/f34 [4194304,4194304] 0 2024-03-24T10:46:03.432 INFO:tasks.workunit.client.0.smithi094.stdout:3/801: truncate d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/d73/f80 449472 0 2024-03-24T10:46:03.432 INFO:tasks.workunit.client.0.smithi094.stdout:3/802: fsync d0/db/dc/d28/d38/dc9/f107 0 2024-03-24T10:46:03.434 INFO:tasks.workunit.client.0.smithi094.stdout:2/925: creat d18/d54/db9/f113 x:0 0 0 2024-03-24T10:46:03.434 INFO:tasks.workunit.client.0.smithi094.stdout:2/926: creat d18/d66/d90/d9a/d74/d7e/dca/f114 x:0 0 0 2024-03-24T10:46:03.434 INFO:tasks.workunit.client.0.smithi094.stdout:4/939: dread d0/d22/de/d28/f72 [0,4194304] 0 2024-03-24T10:46:03.436 INFO:tasks.workunit.client.0.smithi094.stdout:3/803: creat d0/db/dc/d28/f117 x:0 0 0 2024-03-24T10:46:03.436 INFO:tasks.workunit.client.0.smithi094.stdout:3/804: fdatasync d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/f93 0 2024-03-24T10:46:03.439 INFO:tasks.workunit.client.0.smithi094.stdout:2/927: mknod d18/d77/dcf/c115 0 2024-03-24T10:46:03.439 INFO:tasks.workunit.client.0.smithi094.stdout:5/895: rmdir d7/d23/d2b/d68/d81 39 2024-03-24T10:46:03.441 INFO:tasks.workunit.client.0.smithi094.stdout:4/940: write d0/d34/fac [2681321,119246] 0 2024-03-24T10:46:03.442 INFO:tasks.workunit.client.0.smithi094.stdout:2/928: creat d18/d48/d56/dcc/f116 x:0 0 0 2024-03-24T10:46:03.443 INFO:tasks.workunit.client.0.smithi094.stdout:0/876: dwrite d14/d22/d28/d48/feb [0,4194304] 0 2024-03-24T10:46:03.443 INFO:tasks.workunit.client.0.smithi094.stdout:0/877: stat d14/d22/d28/d2b/d68/d122 0 2024-03-24T10:46:03.445 INFO:tasks.workunit.client.0.smithi094.stdout:5/896: unlink d7/d23/d2b/d68/fe8 0 2024-03-24T10:46:03.445 INFO:tasks.workunit.client.0.smithi094.stdout:5/897: fsync d7/d23/d5f/d9/d48/f74 0 2024-03-24T10:46:03.447 INFO:tasks.workunit.client.0.smithi094.stdout:5/898: read d7/d23/d2b/d68/f8f [94724,42061] 0 2024-03-24T10:46:03.447 INFO:tasks.workunit.client.0.smithi094.stdout:5/899: dread - d7/dc/d100/d109/f11b zero size 2024-03-24T10:46:03.450 INFO:tasks.workunit.client.0.smithi094.stdout:5/900: getdents d7/dc/d100/d109 0 2024-03-24T10:46:03.454 INFO:tasks.workunit.client.0.smithi094.stdout:5/901: rename d7/d23/d5f/f8b to d7/d23/dd5/d64/dff/f128 0 2024-03-24T10:46:03.454 INFO:tasks.workunit.client.0.smithi094.stdout:5/902: write d7/d23/d5f/d14/f6d [5047393,126394] 0 2024-03-24T10:46:03.458 INFO:tasks.workunit.client.0.smithi094.stdout:5/903: rename d7/d23/d2b/d68/f118 to d7/d23/d5f/d9/dde/df7/f129 0 2024-03-24T10:46:03.459 INFO:tasks.workunit.client.0.smithi094.stdout:5/904: read - d7/d23/d5f/d9/d48/d73/ff6 zero size 2024-03-24T10:46:03.460 INFO:tasks.workunit.client.0.smithi094.stdout:4/941: dread d0/d22/de/d28/f3a [0,4194304] 0 2024-03-24T10:46:03.460 INFO:tasks.workunit.client.0.smithi094.stdout:4/942: readlink d0/d34/d73/d93/l125 0 2024-03-24T10:46:03.460 INFO:tasks.workunit.client.0.smithi094.stdout:5/905: link d7/c90 d7/d26/dce/c12a 0 2024-03-24T10:46:03.462 INFO:tasks.workunit.client.0.smithi094.stdout:4/943: link d0/cae d0/d34/d73/d93/c126 0 2024-03-24T10:46:03.464 INFO:tasks.workunit.client.0.smithi094.stdout:0/878: write d14/d22/d5b/fbd [471555,100570] 0 2024-03-24T10:46:03.464 INFO:tasks.workunit.client.0.smithi094.stdout:3/805: dwrite d0/db/dc/d28/d50/d5e/fb7 [0,4194304] 0 2024-03-24T10:46:03.464 INFO:tasks.workunit.client.0.smithi094.stdout:5/906: creat d7/d23/dd5/da9/f12b x:0 0 0 2024-03-24T10:46:03.466 INFO:tasks.workunit.client.0.smithi094.stdout:4/944: mknod d0/d22/de/d28/d3d/d10d/c127 0 2024-03-24T10:46:03.468 INFO:tasks.workunit.client.0.smithi094.stdout:2/929: dwrite d18/d5f/fd7 [4194304,4194304] 0 2024-03-24T10:46:03.468 INFO:tasks.workunit.client.0.smithi094.stdout:2/930: chown d18/d48/la9 164 1 2024-03-24T10:46:03.469 INFO:tasks.workunit.client.0.smithi094.stdout:0/879: mkdir d14/d22/dcb/de5/d135 0 2024-03-24T10:46:03.470 INFO:tasks.workunit.client.0.smithi094.stdout:0/880: write d14/d22/d28/d48/d7b/fa8 [792576,25556] 0 2024-03-24T10:46:03.470 INFO:tasks.workunit.client.0.smithi094.stdout:3/806: mknod d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/db5/d106/d9b/dca/c118 0 2024-03-24T10:46:03.471 INFO:tasks.workunit.client.0.smithi094.stdout:4/945: mkdir d0/d22/de/d28/d2b/d33/d128 0 2024-03-24T10:46:03.476 INFO:tasks.workunit.client.0.smithi094.stdout:2/931: rmdir d18/d7f/d88/ded 39 2024-03-24T10:46:03.476 INFO:tasks.workunit.client.0.smithi094.stdout:2/932: chown d18/d48/d56/f58 724426342 1 2024-03-24T10:46:03.480 INFO:tasks.workunit.client.0.smithi094.stdout:2/933: creat d18/d66/da6/f117 x:0 0 0 2024-03-24T10:46:03.482 INFO:tasks.workunit.client.0.smithi094.stdout:4/946: dread d0/d22/f9e [0,4194304] 0 2024-03-24T10:46:03.484 INFO:tasks.workunit.client.0.smithi094.stdout:4/947: unlink d0/d5/d1b/d4f/dd6/ff6 0 2024-03-24T10:46:03.485 INFO:tasks.workunit.client.0.smithi094.stdout:0/881: dwrite d14/f62 [4194304,4194304] 0 2024-03-24T10:46:03.485 INFO:tasks.workunit.client.0.smithi094.stdout:7/844: truncate d1/d9/f30 3069818 0 2024-03-24T10:46:03.485 INFO:tasks.workunit.client.0.smithi094.stdout:7/845: creat d1/d9/d16/f121 x:0 0 0 2024-03-24T10:46:03.488 INFO:tasks.workunit.client.0.smithi094.stdout:4/948: unlink d0/d34/d73/d93/l125 0 2024-03-24T10:46:03.488 INFO:tasks.workunit.client.0.smithi094.stdout:4/949: chown d0/d22/de/d28/d2b/d33/da4/ca0 69747013 1 2024-03-24T10:46:03.492 INFO:tasks.workunit.client.0.smithi094.stdout:0/882: rename d14/d22/d5b/fb7 to d14/d22/dcb/de5/f136 0 2024-03-24T10:46:03.492 INFO:tasks.workunit.client.0.smithi094.stdout:0/883: dread - d14/d22/d7d/f9c zero size 2024-03-24T10:46:03.493 INFO:tasks.workunit.client.0.smithi094.stdout:2/934: dwrite d18/d66/dbe/fe7 [0,4194304] 0 2024-03-24T10:46:03.495 INFO:tasks.workunit.client.0.smithi094.stdout:7/846: unlink d1/d6c/d80/f6f 0 2024-03-24T10:46:03.496 INFO:tasks.workunit.client.0.smithi094.stdout:4/950: link d0/d34/d73/d93/f96 d0/d22/de/d28/d2b/d7a/f129 0 2024-03-24T10:46:03.496 INFO:tasks.workunit.client.0.smithi094.stdout:0/884: mknod d14/d22/d28/d2b/d68/c137 0 2024-03-24T10:46:03.496 INFO:tasks.workunit.client.0.smithi094.stdout:0/885: stat d14 0 2024-03-24T10:46:03.497 INFO:tasks.workunit.client.0.smithi094.stdout:4/951: write d0/d5/d1b/f4e [732564,15276] 0 2024-03-24T10:46:03.497 INFO:tasks.workunit.client.0.smithi094.stdout:7/847: dread d1/d9/d16/fbe [0,4194304] 0 2024-03-24T10:46:03.497 INFO:tasks.workunit.client.0.smithi094.stdout:7/848: truncate d1/d9/d16/f121 591643 0 2024-03-24T10:46:03.498 INFO:tasks.workunit.client.0.smithi094.stdout:2/935: creat d18/d48/dc3/f118 x:0 0 0 2024-03-24T10:46:03.499 INFO:tasks.workunit.client.0.smithi094.stdout:2/936: fdatasync d18/d66/dbe/dc0/f6c 0 2024-03-24T10:46:03.499 INFO:tasks.workunit.client.0.smithi094.stdout:2/937: write d18/d1e/fe3 [667634,83524] 0 2024-03-24T10:46:03.499 INFO:tasks.workunit.client.0.smithi094.stdout:2/938: write d18/f3f [3199448,31749] 0 2024-03-24T10:46:03.501 INFO:tasks.workunit.client.0.smithi094.stdout:4/952: mkdir d0/d22/de/d28/d3d/d8c/dec/d12a 0 2024-03-24T10:46:03.501 INFO:tasks.workunit.client.0.smithi094.stdout:7/849: symlink d1/d9/d16/d21/d51/d83/d39/d44/dd9/d8f/l122 0 2024-03-24T10:46:03.502 INFO:tasks.workunit.client.0.smithi094.stdout:2/939: mkdir d18/d48/d56/d119 0 2024-03-24T10:46:03.503 INFO:tasks.workunit.client.0.smithi094.stdout:2/940: chown d18/d1e/d65/lae 1 1 2024-03-24T10:46:03.503 INFO:tasks.workunit.client.0.smithi094.stdout:2/941: chown d18/d66/d90/d9a/d74 12746859 1 2024-03-24T10:46:03.504 INFO:tasks.workunit.client.0.smithi094.stdout:0/886: mkdir d14/d22/d28/d138 0 2024-03-24T10:46:03.504 INFO:tasks.workunit.client.0.smithi094.stdout:4/953: symlink d0/d5/d7d/d97/d4a/l12b 0 2024-03-24T10:46:03.504 INFO:tasks.workunit.client.0.smithi094.stdout:4/954: write d0/d22/de/d28/d2b/da2/f111 [453165,3283] 0 2024-03-24T10:46:03.506 INFO:tasks.workunit.client.0.smithi094.stdout:7/850: rmdir d1/d9/d25/dd4/d11b 0 2024-03-24T10:46:03.506 INFO:tasks.workunit.client.0.smithi094.stdout:7/851: fsync d1/d9/d16/fc7 0 2024-03-24T10:46:03.507 INFO:tasks.workunit.client.0.smithi094.stdout:0/887: symlink d14/d22/d28/da2/d10a/l139 0 2024-03-24T10:46:03.509 INFO:tasks.workunit.client.0.smithi094.stdout:4/955: rename d0/d22/de/d28/d2b/d7a/f129 to d0/d5/d7d/d97/d4a/f12c 0 2024-03-24T10:46:03.509 INFO:tasks.workunit.client.0.smithi094.stdout:8/929: sync 2024-03-24T10:46:03.510 INFO:tasks.workunit.client.0.smithi094.stdout:1/791: sync 2024-03-24T10:46:03.510 INFO:tasks.workunit.client.0.smithi094.stdout:6/886: sync 2024-03-24T10:46:03.510 INFO:tasks.workunit.client.0.smithi094.stdout:6/887: creat d0/d1a/d35/f121 x:0 0 0 2024-03-24T10:46:03.510 INFO:tasks.workunit.client.0.smithi094.stdout:1/792: read d4/d9/d37/d33/f7e [3876840,129069] 0 2024-03-24T10:46:03.510 INFO:tasks.workunit.client.0.smithi094.stdout:0/888: rename d14/d22/d28/d2b/dc0/cf0 to d14/d22/d28/d2b/d41/d53/c13a 0 2024-03-24T10:46:03.513 INFO:tasks.workunit.client.0.smithi094.stdout:4/956: symlink d0/d22/de/d28/d2b/d33/da4/d95/l12d 0 2024-03-24T10:46:03.517 INFO:tasks.workunit.client.0.smithi094.stdout:1/793: link d4/f7 d4/d9/d37/d33/f110 0 2024-03-24T10:46:03.518 INFO:tasks.workunit.client.0.smithi094.stdout:1/794: read d4/f7 [93375,81251] 0 2024-03-24T10:46:03.521 INFO:tasks.workunit.client.0.smithi094.stdout:0/889: mkdir d14/d22/d7d/d13b 0 2024-03-24T10:46:03.521 INFO:tasks.workunit.client.0.smithi094.stdout:0/890: stat d14/d22/d7d/f9c 0 2024-03-24T10:46:03.522 INFO:tasks.workunit.client.0.smithi094.stdout:2/942: dwrite d18/d1e/d65/f6a [0,4194304] 0 2024-03-24T10:46:03.524 INFO:tasks.workunit.client.0.smithi094.stdout:4/957: symlink d0/d22/de/dc8/de6/def/l12e 0 2024-03-24T10:46:03.526 INFO:tasks.workunit.client.0.smithi094.stdout:8/930: getdents df/d25 0 2024-03-24T10:46:03.529 INFO:tasks.workunit.client.0.smithi094.stdout:4/958: symlink d0/d5/l12f 0 2024-03-24T10:46:03.529 INFO:tasks.workunit.client.0.smithi094.stdout:4/959: fsync d0/d5/d7d/f99 0 2024-03-24T10:46:03.530 INFO:tasks.workunit.client.0.smithi094.stdout:8/931: truncate df/d25/f58 1538886 0 2024-03-24T10:46:03.530 INFO:tasks.workunit.client.0.smithi094.stdout:8/932: write df/ff3 [967572,40858] 0 2024-03-24T10:46:03.530 INFO:tasks.workunit.client.0.smithi094.stdout:2/943: read d18/d48/dc3/fe1 [1588553,123120] 0 2024-03-24T10:46:03.531 INFO:tasks.workunit.client.0.smithi094.stdout:4/960: unlink d0/d34/d73/fbf 0 2024-03-24T10:46:03.532 INFO:tasks.workunit.client.0.smithi094.stdout:8/933: rename df/d25/d28/d31/d47/d22/d55/f10a to df/d12/d5b/d63/d130/f137 0 2024-03-24T10:46:03.532 INFO:tasks.workunit.client.0.smithi094.stdout:8/934: stat df/d12/d5b/l5e 0 2024-03-24T10:46:03.532 INFO:tasks.workunit.client.0.smithi094.stdout:8/935: chown df/d25/d28/d31/d47/f77 371370741 1 2024-03-24T10:46:03.533 INFO:tasks.workunit.client.0.smithi094.stdout:4/961: mknod d0/d22/c130 0 2024-03-24T10:46:03.534 INFO:tasks.workunit.client.0.smithi094.stdout:2/944: rename d18/d66/d90/d9a/dbc/f84 to d18/d66/d90/d9a/d74/db4/dd4/f11a 0 2024-03-24T10:46:03.536 INFO:tasks.workunit.client.0.smithi094.stdout:4/962: rmdir d0/d5/d1b/d4f/dd6 39 2024-03-24T10:46:03.537 INFO:tasks.workunit.client.0.smithi094.stdout:4/963: truncate d0/d34/db2/faa 207916 0 2024-03-24T10:46:03.537 INFO:tasks.workunit.client.0.smithi094.stdout:8/936: dread df/d12/f21 [0,4194304] 0 2024-03-24T10:46:03.538 INFO:tasks.workunit.client.0.smithi094.stdout:4/964: write d0/d5/d1b/f4e [811893,79502] 0 2024-03-24T10:46:03.538 INFO:tasks.workunit.client.0.smithi094.stdout:4/965: fsync d0/d22/de/d28/f72 0 2024-03-24T10:46:03.538 INFO:tasks.workunit.client.0.smithi094.stdout:4/966: getdents d0/d34/db2 0 2024-03-24T10:46:03.539 INFO:tasks.workunit.client.0.smithi094.stdout:4/967: readlink d0/d22/de/le3 0 2024-03-24T10:46:03.540 INFO:tasks.workunit.client.0.smithi094.stdout:4/968: unlink d0/d34/l3c 0 2024-03-24T10:46:03.544 INFO:tasks.workunit.client.0.smithi094.stdout:7/852: dwrite d1/feb [0,4194304] 0 2024-03-24T10:46:03.549 INFO:tasks.workunit.client.0.smithi094.stdout:6/888: dwrite d0/d1a/d35/d96/fd0 [0,4194304] 0 2024-03-24T10:46:03.549 INFO:tasks.workunit.client.0.smithi094.stdout:1/795: dwrite d4/d9/db/d98/fa4 [0,4194304] 0 2024-03-24T10:46:03.549 INFO:tasks.workunit.client.0.smithi094.stdout:2/945: dread d18/d5f/fd7 [0,4194304] 0 2024-03-24T10:46:03.549 INFO:tasks.workunit.client.0.smithi094.stdout:1/796: fsync d4/d9/f2d 0 2024-03-24T10:46:03.552 INFO:tasks.workunit.client.0.smithi094.stdout:8/937: write df/d25/d28/d31/d47/d66/da3/da8/d117/dc1/fe1 [2881118,103203] 0 2024-03-24T10:46:03.552 INFO:tasks.workunit.client.0.smithi094.stdout:8/938: fsync df/d25/df1/f11d 0 2024-03-24T10:46:03.552 INFO:tasks.workunit.client.0.smithi094.stdout:8/939: fsync df/d25/daf/fcb 0 2024-03-24T10:46:03.564 INFO:tasks.workunit.client.0.smithi094.stdout:7/853: rmdir d1/d9/d25/dd4 39 2024-03-24T10:46:03.564 INFO:tasks.workunit.client.0.smithi094.stdout:7/854: chown d1/d9/dc3/f10e 37270 1 2024-03-24T10:46:03.566 INFO:tasks.workunit.client.0.smithi094.stdout:6/889: dwrite d0/d1a/fd7 [0,4194304] 0 2024-03-24T10:46:03.566 INFO:tasks.workunit.client.0.smithi094.stdout:6/890: stat d0/da9/f10a 0 2024-03-24T10:46:03.567 INFO:tasks.workunit.client.0.smithi094.stdout:6/891: creat d0/d78/da/d23/d34/d119/f122 x:0 0 0 2024-03-24T10:46:03.569 INFO:tasks.workunit.client.0.smithi094.stdout:4/969: rename d0/d22/de/d28/d2b/d33 to d0/d22/de/d28/d3d/d8c/dec/d131 0 2024-03-24T10:46:03.570 INFO:tasks.workunit.client.0.smithi094.stdout:1/797: mknod d4/d9/d7d/c111 0 2024-03-24T10:46:03.571 INFO:tasks.workunit.client.0.smithi094.stdout:1/798: chown d4/d9/d7d/c111 225760 1 2024-03-24T10:46:03.573 INFO:tasks.workunit.client.0.smithi094.stdout:7/855: mkdir d1/d9/d16/d21/d51/d83/d9d/dd7/d123 0 2024-03-24T10:46:03.579 INFO:tasks.workunit.client.0.smithi094.stdout:6/892: unlink d0/d78/da/d23/c10d 0 2024-03-24T10:46:03.581 INFO:tasks.workunit.client.0.smithi094.stdout:8/940: rename fa to df/d25/d28/d31/d47/d22/d55/f138 0 2024-03-24T10:46:03.582 INFO:tasks.workunit.client.0.smithi094.stdout:1/799: dread d4/d9/d37/d33/d58/f88 [0,4194304] 0 2024-03-24T10:46:03.583 INFO:tasks.workunit.client.0.smithi094.stdout:4/970: creat d0/d22/de/dc8/f132 x:0 0 0 2024-03-24T10:46:03.586 INFO:tasks.workunit.client.0.smithi094.stdout:7/856: truncate d1/d9/d16/d21/d51/d83/d39/d44/dd9/f4b 2694066 0 2024-03-24T10:46:03.586 INFO:tasks.workunit.client.0.smithi094.stdout:7/857: fdatasync d1/d9/dc3/f10e 0 2024-03-24T10:46:03.586 INFO:tasks.workunit.client.0.smithi094.stdout:7/858: read d1/d9/d25/d3d/f6d [374944,10268] 0 2024-03-24T10:46:03.586 INFO:tasks.workunit.client.0.smithi094.stdout:7/859: readlink d1/db4/l62 0 2024-03-24T10:46:03.589 INFO:tasks.workunit.client.0.smithi094.stdout:8/941: rename df/d25/d28/f6f to df/d25/d28/d31/d95/d118/f139 0 2024-03-24T10:46:03.591 INFO:tasks.workunit.client.0.smithi094.stdout:0/891: dwrite d14/d22/d28/f47 [0,4194304] 0 2024-03-24T10:46:03.592 INFO:tasks.workunit.client.0.smithi094.stdout:4/971: rmdir d0/d22/de/dc8/de6/def 39 2024-03-24T10:46:03.594 INFO:tasks.workunit.client.0.smithi094.stdout:7/860: mknod d1/d11f/c124 0 2024-03-24T10:46:03.594 INFO:tasks.workunit.client.0.smithi094.stdout:7/861: fdatasync d1/d9/d16/fc7 0 2024-03-24T10:46:03.594 INFO:tasks.workunit.client.0.smithi094.stdout:7/862: chown d1/d9/dc3/l107 3068923 1 2024-03-24T10:46:03.594 INFO:tasks.workunit.client.0.smithi094.stdout:7/863: dread - d1/f120 zero size 2024-03-24T10:46:03.595 INFO:tasks.workunit.client.0.smithi094.stdout:8/942: symlink df/d12/l13a 0 2024-03-24T10:46:03.596 INFO:tasks.workunit.client.0.smithi094.stdout:4/972: dread d0/d22/de/d28/d3d/d8c/fc7 [0,4194304] 0 2024-03-24T10:46:03.596 INFO:tasks.workunit.client.0.smithi094.stdout:6/893: write d0/d1a/d35/d96/fd0 [4766283,23957] 0 2024-03-24T10:46:03.596 INFO:tasks.workunit.client.0.smithi094.stdout:6/894: write d0/da9/fcd [2456167,106849] 0 2024-03-24T10:46:03.597 INFO:tasks.workunit.client.0.smithi094.stdout:1/800: rmdir d4/d9/db/d39 39 2024-03-24T10:46:03.597 INFO:tasks.workunit.client.0.smithi094.stdout:1/801: stat d4/d9/d37/d33/d58/d6b 0 2024-03-24T10:46:03.601 INFO:tasks.workunit.client.0.smithi094.stdout:0/892: mkdir d14/d22/d28/d48/da5/de0/d13c 0 2024-03-24T10:46:03.602 INFO:tasks.workunit.client.0.smithi094.stdout:7/864: mknod d1/d6c/d80/c125 0 2024-03-24T10:46:03.605 INFO:tasks.workunit.client.0.smithi094.stdout:8/943: unlink df/d12/d5b/d63/d75/l112 0 2024-03-24T10:46:03.607 INFO:tasks.workunit.client.0.smithi094.stdout:6/895: symlink d0/d1a/dca/l123 0 2024-03-24T10:46:03.608 INFO:tasks.workunit.client.0.smithi094.stdout:1/802: truncate d4/d9/d37/d33/d76/f9b 3061124 0 2024-03-24T10:46:03.608 INFO:tasks.workunit.client.0.smithi094.stdout:1/803: dread - d4/d9/de/fbd zero size 2024-03-24T10:46:03.608 INFO:tasks.workunit.client.0.smithi094.stdout:1/804: chown d4/d9/db/f18 233646 1 2024-03-24T10:46:03.612 INFO:tasks.workunit.client.0.smithi094.stdout:6/896: unlink d0/d78/d66/ccf 0 2024-03-24T10:46:03.614 INFO:tasks.workunit.client.0.smithi094.stdout:7/865: getdents d1/d9/d16/d21/d51/d83/d39/d44/dd9 0 2024-03-24T10:46:03.617 INFO:tasks.workunit.client.0.smithi094.stdout:6/897: link d0/d21/d2a/c115 d0/d78/da/d23/d34/d119/c124 0 2024-03-24T10:46:03.618 INFO:tasks.workunit.client.0.smithi094.stdout:6/898: dread d0/d30/d50/fac [0,4194304] 0 2024-03-24T10:46:03.618 INFO:tasks.workunit.client.0.smithi094.stdout:6/899: stat d0/d30 0 2024-03-24T10:46:03.620 INFO:tasks.workunit.client.0.smithi094.stdout:7/866: truncate d1/d6c/fee 298810 0 2024-03-24T10:46:03.620 INFO:tasks.workunit.client.0.smithi094.stdout:4/973: dwrite d0/d5/d1b/d4f/fa7 [0,4194304] 0 2024-03-24T10:46:03.621 INFO:tasks.workunit.client.0.smithi094.stdout:3/807: sync 2024-03-24T10:46:03.621 INFO:tasks.workunit.client.0.smithi094.stdout:5/907: sync 2024-03-24T10:46:03.621 INFO:tasks.workunit.client.0.smithi094.stdout:8/944: read df/d25/d28/f105 [1081238,58040] 0 2024-03-24T10:46:03.621 INFO:tasks.workunit.client.0.smithi094.stdout:8/945: fsync df/d25/f61 0 2024-03-24T10:46:03.623 INFO:tasks.workunit.client.0.smithi094.stdout:5/908: creat d7/d23/d5f/f12c x:0 0 0 2024-03-24T10:46:03.623 INFO:tasks.workunit.client.0.smithi094.stdout:5/909: chown d7/d23/d5f/d9/d48/d51/fc7 706948 1 2024-03-24T10:46:03.625 INFO:tasks.workunit.client.0.smithi094.stdout:8/946: rename df/d25/f129 to df/d25/d36/d3f/f13b 0 2024-03-24T10:46:03.625 INFO:tasks.workunit.client.0.smithi094.stdout:8/947: readlink df/l82 0 2024-03-24T10:46:03.625 INFO:tasks.workunit.client.0.smithi094.stdout:8/948: readlink df/d12/l13a 0 2024-03-24T10:46:03.625 INFO:tasks.workunit.client.0.smithi094.stdout:8/949: write df/d25/d28/d31/d47/f77 [2151321,111376] 0 2024-03-24T10:46:03.625 INFO:tasks.workunit.client.0.smithi094.stdout:8/950: chown df/d25/d28/d31/d47/f7e 173315 1 2024-03-24T10:46:03.628 INFO:tasks.workunit.client.0.smithi094.stdout:1/805: dwrite d4/d9/de/f2b [4194304,4194304] 0 2024-03-24T10:46:03.629 INFO:tasks.workunit.client.0.smithi094.stdout:8/951: link df/d25/d36/d10b/f128 df/d25/d28/d31/d47/d66/da3/db5/dbf/f13c 0 2024-03-24T10:46:03.630 INFO:tasks.workunit.client.0.smithi094.stdout:1/806: dread d4/d9/d37/d33/f110 [0,4194304] 0 2024-03-24T10:46:03.630 INFO:tasks.workunit.client.0.smithi094.stdout:1/807: chown d4/d9/de/d6f/d62/d84/c103 17 1 2024-03-24T10:46:03.631 INFO:tasks.workunit.client.0.smithi094.stdout:8/952: dread df/d25/d36/f9b [0,4194304] 0 2024-03-24T10:46:03.631 INFO:tasks.workunit.client.0.smithi094.stdout:8/953: chown df/d25/d28/d31/d47/d66/da3/db5/f133 12501 1 2024-03-24T10:46:03.632 INFO:tasks.workunit.client.0.smithi094.stdout:4/974: write d0/d5/d7d/f99 [3119165,33459] 0 2024-03-24T10:46:03.632 INFO:tasks.workunit.client.0.smithi094.stdout:4/975: write d0/d34/d73/d93/f113 [798898,33313] 0 2024-03-24T10:46:03.632 INFO:tasks.workunit.client.0.smithi094.stdout:4/976: truncate d0/d34/db2/faa 477323 0 2024-03-24T10:46:03.633 INFO:tasks.workunit.client.0.smithi094.stdout:6/900: dwrite d0/d21/f56 [0,4194304] 0 2024-03-24T10:46:03.633 INFO:tasks.workunit.client.0.smithi094.stdout:6/901: readlink d0/l9a 0 2024-03-24T10:46:03.634 INFO:tasks.workunit.client.0.smithi094.stdout:1/808: unlink d4/d9/db/d31/d3c/c8c 0 2024-03-24T10:46:03.635 INFO:tasks.workunit.client.0.smithi094.stdout:6/902: dread d0/f33 [0,4194304] 0 2024-03-24T10:46:03.636 INFO:tasks.workunit.client.0.smithi094.stdout:8/954: creat df/d25/d28/d31/d47/ddf/f13d x:0 0 0 2024-03-24T10:46:03.636 INFO:tasks.workunit.client.0.smithi094.stdout:3/808: write d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d87/f9c [3578096,66627] 0 2024-03-24T10:46:03.636 INFO:tasks.workunit.client.0.smithi094.stdout:3/809: chown d0/db/dc/d28/d3c/d7c/de8 1990392 1 2024-03-24T10:46:03.644 INFO:tasks.workunit.client.0.smithi094.stdout:6/903: symlink d0/d21/d2a/d55/dce/l125 0 2024-03-24T10:46:03.649 INFO:tasks.workunit.client.0.smithi094.stdout:8/955: rename df/d25/f97 to df/d12/d5b/f13e 0 2024-03-24T10:46:03.649 INFO:tasks.workunit.client.0.smithi094.stdout:8/956: read df/d25/d28/d31/d3c/f5f [4121157,46853] 0 2024-03-24T10:46:03.650 INFO:tasks.workunit.client.0.smithi094.stdout:3/810: rmdir d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/d73 39 2024-03-24T10:46:03.651 INFO:tasks.workunit.client.0.smithi094.stdout:6/904: mkdir d0/d21/d2a/d55/dce/d126 0 2024-03-24T10:46:03.651 INFO:tasks.workunit.client.0.smithi094.stdout:6/905: fsync d0/d78/da/d23/d34/d4e/f7e 0 2024-03-24T10:46:03.652 INFO:tasks.workunit.client.0.smithi094.stdout:6/906: readlink d0/d21/d2a/d55/dce/l125 0 2024-03-24T10:46:03.655 INFO:tasks.workunit.client.0.smithi094.stdout:6/907: creat d0/d30/f127 x:0 0 0 2024-03-24T10:46:03.656 INFO:tasks.workunit.client.0.smithi094.stdout:4/977: dwrite d0/d34/d73/d93/f96 [4194304,4194304] 0 2024-03-24T10:46:03.662 INFO:tasks.workunit.client.0.smithi094.stdout:4/978: mkdir d0/d22/de/d28/d133 0 2024-03-24T10:46:03.662 INFO:tasks.workunit.client.0.smithi094.stdout:6/908: getdents d0/d21/d2a/d55 0 2024-03-24T10:46:03.664 INFO:tasks.workunit.client.0.smithi094.stdout:3/811: dread d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/fc8 [0,4194304] 0 2024-03-24T10:46:03.664 INFO:tasks.workunit.client.0.smithi094.stdout:1/809: dwrite d4/d9/f32 [0,4194304] 0 2024-03-24T10:46:03.664 INFO:tasks.workunit.client.0.smithi094.stdout:4/979: mkdir d0/d22/de/d28/d3d/d10d/d134 0 2024-03-24T10:46:03.665 INFO:tasks.workunit.client.0.smithi094.stdout:6/909: mkdir d0/d30/d114/d128 0 2024-03-24T10:46:03.667 INFO:tasks.workunit.client.0.smithi094.stdout:3/812: mknod d0/c119 0 2024-03-24T10:46:03.667 INFO:tasks.workunit.client.0.smithi094.stdout:3/813: creat d0/db/dc/d28/d38/dc9/f11a x:0 0 0 2024-03-24T10:46:03.669 INFO:tasks.workunit.client.0.smithi094.stdout:6/910: creat d0/d21/d2a/dcb/d110/f129 x:0 0 0 2024-03-24T10:46:03.670 INFO:tasks.workunit.client.0.smithi094.stdout:7/867: dwrite d1/d9/d25/d3d/faa [0,4194304] 0 2024-03-24T10:46:03.673 INFO:tasks.workunit.client.0.smithi094.stdout:7/868: getdents d1/d9/d25/de2 0 2024-03-24T10:46:03.686 INFO:tasks.workunit.client.0.smithi094.stdout:7/869: dread d1/db4/fba [4194304,4194304] 0 2024-03-24T10:46:03.687 INFO:tasks.workunit.client.0.smithi094.stdout:7/870: chown d1/db4/d61 439090381 1 2024-03-24T10:46:03.691 INFO:tasks.workunit.client.0.smithi094.stdout:5/910: write d7/d23/d5f/f28 [312741,102654] 0 2024-03-24T10:46:03.691 INFO:tasks.workunit.client.0.smithi094.stdout:5/911: chown d7/d23/dd5/da9/fab 0 1 2024-03-24T10:46:03.691 INFO:tasks.workunit.client.0.smithi094.stdout:5/912: truncate d7/dc/fed 681867 0 2024-03-24T10:46:03.691 INFO:tasks.workunit.client.0.smithi094.stdout:5/913: fdatasync d7/d23/dd5/d64/dd6/f1f 0 2024-03-24T10:46:03.692 INFO:tasks.workunit.client.0.smithi094.stdout:6/911: write d0/d78/da/d23/dc3/fbe [2622720,126694] 0 2024-03-24T10:46:03.692 INFO:tasks.workunit.client.0.smithi094.stdout:6/912: readlink d0/d30/lda 0 2024-03-24T10:46:03.692 INFO:tasks.workunit.client.0.smithi094.stdout:6/913: truncate d0/d78/da/d23/d83/f85 1694877 0 2024-03-24T10:46:03.692 INFO:tasks.workunit.client.0.smithi094.stdout:3/814: dwrite d0/db/dc3/ff9 [0,4194304] 0 2024-03-24T10:46:03.692 INFO:tasks.workunit.client.0.smithi094.stdout:3/815: stat d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/fd1 0 2024-03-24T10:46:03.693 INFO:tasks.workunit.client.0.smithi094.stdout:7/871: write d1/d6c/f71 [906979,69578] 0 2024-03-24T10:46:03.693 INFO:tasks.workunit.client.0.smithi094.stdout:5/914: symlink d7/d23/d5f/d9/dde/df7/l12d 0 2024-03-24T10:46:03.695 INFO:tasks.workunit.client.0.smithi094.stdout:4/980: dwrite d0/d22/de/d28/d3d/d8c/fc7 [0,4194304] 0 2024-03-24T10:46:03.700 INFO:tasks.workunit.client.0.smithi094.stdout:7/872: mkdir d1/db4/d65/d126 0 2024-03-24T10:46:03.702 INFO:tasks.workunit.client.0.smithi094.stdout:4/981: mknod d0/d22/de/c135 0 2024-03-24T10:46:03.704 INFO:tasks.workunit.client.0.smithi094.stdout:5/915: dread d7/d23/d5f/d14/d6f/dbe/ff3 [0,4194304] 0 2024-03-24T10:46:03.704 INFO:tasks.workunit.client.0.smithi094.stdout:5/916: creat d7/d23/dd5/f12e x:0 0 0 2024-03-24T10:46:03.706 INFO:tasks.workunit.client.0.smithi094.stdout:2/946: sync 2024-03-24T10:46:03.706 INFO:tasks.workunit.client.0.smithi094.stdout:2/947: creat d18/d66/d90/d9a/d74/db4/dd4/f11b x:0 0 0 2024-03-24T10:46:03.706 INFO:tasks.workunit.client.0.smithi094.stdout:2/948: creat d18/d1e/d65/f11c x:0 0 0 2024-03-24T10:46:03.706 INFO:tasks.workunit.client.0.smithi094.stdout:2/949: chown d18/d7f/d88/da2/da4/l9f 80883124 1 2024-03-24T10:46:03.708 INFO:tasks.workunit.client.0.smithi094.stdout:5/917: mkdir d7/d23/d5f/d12f 0 2024-03-24T10:46:03.709 INFO:tasks.workunit.client.0.smithi094.stdout:7/873: write d1/db4/fba [814703,80963] 0 2024-03-24T10:46:03.709 INFO:tasks.workunit.client.0.smithi094.stdout:7/874: chown d1/d6c/d80/d81/lcf 87456 1 2024-03-24T10:46:03.710 INFO:tasks.workunit.client.0.smithi094.stdout:2/950: mknod d18/d54/c11d 0 2024-03-24T10:46:03.713 INFO:tasks.workunit.client.0.smithi094.stdout:2/951: mkdir d18/d7f/d88/d11e 0 2024-03-24T10:46:03.714 INFO:tasks.workunit.client.0.smithi094.stdout:6/914: dwrite d0/d78/f44 [0,4194304] 0 2024-03-24T10:46:03.714 INFO:tasks.workunit.client.0.smithi094.stdout:6/915: chown d0/d21/d2a/d55/faa 97280901 1 2024-03-24T10:46:03.714 INFO:tasks.workunit.client.0.smithi094.stdout:6/916: chown d0/d21/d2a/d55/dce/fd4 214647 1 2024-03-24T10:46:03.715 INFO:tasks.workunit.client.0.smithi094.stdout:3/816: dwrite d0/db/dc/d28/d3c/d7c/fb0 [0,4194304] 0 2024-03-24T10:46:03.718 INFO:tasks.workunit.client.0.smithi094.stdout:7/875: write d1/d9/d16/d21/d49/f84 [7900483,74090] 0 2024-03-24T10:46:03.719 INFO:tasks.workunit.client.0.smithi094.stdout:7/876: chown d1/d9/d16/d21/d51/d83/d39/d44/dd9/d8f/cf3 572735795 1 2024-03-24T10:46:03.719 INFO:tasks.workunit.client.0.smithi094.stdout:8/957: rename df/d12 to df/d25/daf/d13f 0 2024-03-24T10:46:03.719 INFO:tasks.workunit.client.0.smithi094.stdout:8/958: creat df/d25/d28/d31/d95/de3/f140 x:0 0 0 2024-03-24T10:46:03.719 INFO:tasks.workunit.client.0.smithi094.stdout:8/959: stat df/d25/d28/d31 0 2024-03-24T10:46:03.722 INFO:tasks.workunit.client.0.smithi094.stdout:6/917: unlink d0/d21/fa8 0 2024-03-24T10:46:03.725 INFO:tasks.workunit.client.0.smithi094.stdout:3/817: mkdir d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d87/dc5/d101/d11b 0 2024-03-24T10:46:03.731 INFO:tasks.workunit.client.0.smithi094.stdout:2/952: dread d18/d54/f5c [4194304,4194304] 0 2024-03-24T10:46:03.731 INFO:tasks.workunit.client.0.smithi094.stdout:2/953: creat d18/d1e/f11f x:0 0 0 2024-03-24T10:46:03.736 INFO:tasks.workunit.client.0.smithi094.stdout:2/954: symlink d18/d54/d102/l120 0 2024-03-24T10:46:03.738 INFO:tasks.workunit.client.0.smithi094.stdout:2/955: rename d18/d77/f78 to d18/d1e/d65/f121 0 2024-03-24T10:46:03.739 INFO:tasks.workunit.client.0.smithi094.stdout:1/810: truncate d4/d9/f32 176011 0 2024-03-24T10:46:03.739 INFO:tasks.workunit.client.0.smithi094.stdout:1/811: readlink d4/d9/d37/le1 0 2024-03-24T10:46:03.740 INFO:tasks.workunit.client.0.smithi094.stdout:2/956: rmdir d18/d66 39 2024-03-24T10:46:03.742 INFO:tasks.workunit.client.0.smithi094.stdout:1/812: creat d4/d9/de/d6f/d62/d84/de9/f112 x:0 0 0 2024-03-24T10:46:03.743 INFO:tasks.workunit.client.0.smithi094.stdout:5/918: dwrite d7/d26/d101/f110 [0,4194304] 0 2024-03-24T10:46:03.743 INFO:tasks.workunit.client.0.smithi094.stdout:5/919: readlink d7/d23/d5f/d9/d48/d51/lc3 0 2024-03-24T10:46:03.743 INFO:tasks.workunit.client.0.smithi094.stdout:2/957: mkdir d18/d66/d90/d9a/d74/d7e/d122 0 2024-03-24T10:46:03.744 INFO:tasks.workunit.client.0.smithi094.stdout:1/813: symlink d4/d9/de/d6f/d62/l113 0 2024-03-24T10:46:03.747 INFO:tasks.workunit.client.0.smithi094.stdout:5/920: creat d7/d23/d2b/def/f130 x:0 0 0 2024-03-24T10:46:03.749 INFO:tasks.workunit.client.0.smithi094.stdout:2/958: mkdir d18/d66/d90/d9a/def/d123 0 2024-03-24T10:46:03.751 INFO:tasks.workunit.client.0.smithi094.stdout:1/814: link d4/d9/db/d5d/feb d4/d36/f114 0 2024-03-24T10:46:03.752 INFO:tasks.workunit.client.0.smithi094.stdout:8/960: dwrite df/d25/daf/fb0 [0,4194304] 0 2024-03-24T10:46:03.752 INFO:tasks.workunit.client.0.smithi094.stdout:3/818: dread d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d87/dc5/d101/fd4 [0,4194304] 0 2024-03-24T10:46:03.754 INFO:tasks.workunit.client.0.smithi094.stdout:5/921: rename d7/d23/d5f/d9/l13 to d7/d23/dd5/da9/l131 0 2024-03-24T10:46:03.754 INFO:tasks.workunit.client.0.smithi094.stdout:5/922: chown d7/d23/d5f/d14/d36/d3a/d57/ce5 3371041 1 2024-03-24T10:46:03.756 INFO:tasks.workunit.client.0.smithi094.stdout:2/959: truncate d18/f47 728264 0 2024-03-24T10:46:03.757 INFO:tasks.workunit.client.0.smithi094.stdout:6/918: dwrite d0/d78/d66/fee [0,4194304] 0 2024-03-24T10:46:03.759 INFO:tasks.workunit.client.0.smithi094.stdout:6/919: write d0/d78/da/d77/fa3 [231532,78351] 0 2024-03-24T10:46:03.759 INFO:tasks.workunit.client.0.smithi094.stdout:6/920: creat d0/d78/da/d23/d34/d119/f12a x:0 0 0 2024-03-24T10:46:03.764 INFO:tasks.workunit.client.0.smithi094.stdout:3/819: rename d0/db/dc/d28/d38/fed to d0/db/dc/d28/d3c/d7c/de8/f11c 0 2024-03-24T10:46:03.764 INFO:tasks.workunit.client.0.smithi094.stdout:3/820: read - d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/db5/d106/ff8 zero size 2024-03-24T10:46:03.764 INFO:tasks.workunit.client.0.smithi094.stdout:3/821: chown d0/db/dc/d28/d38/f72 17 1 2024-03-24T10:46:03.765 INFO:tasks.workunit.client.0.smithi094.stdout:5/923: mkdir d7/d23/d132 0 2024-03-24T10:46:03.767 INFO:tasks.workunit.client.0.smithi094.stdout:2/960: mkdir d18/d66/d90/d9a/def/d123/d124 0 2024-03-24T10:46:03.771 INFO:tasks.workunit.client.0.smithi094.stdout:3/822: write d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d4f/f109 [1497571,24519] 0 2024-03-24T10:46:03.773 INFO:tasks.workunit.client.0.smithi094.stdout:6/921: unlink d0/d30/de6/d9c/f10c 0 2024-03-24T10:46:03.774 INFO:tasks.workunit.client.0.smithi094.stdout:5/924: symlink d7/d23/dd5/d64/dff/l133 0 2024-03-24T10:46:03.774 INFO:tasks.workunit.client.0.smithi094.stdout:5/925: dread - d7/d23/d2b/ffc zero size 2024-03-24T10:46:03.775 INFO:tasks.workunit.client.0.smithi094.stdout:1/815: dwrite d4/d9/d7d/f4f [0,4194304] 0 2024-03-24T10:46:03.779 INFO:tasks.workunit.client.0.smithi094.stdout:5/926: rename d7/d23/d5f/d1b/f42 to d7/d23/d5f/d1b/f134 0 2024-03-24T10:46:03.779 INFO:tasks.workunit.client.0.smithi094.stdout:5/927: chown d7/dc/l5b 7 1 2024-03-24T10:46:03.779 INFO:tasks.workunit.client.0.smithi094.stdout:5/928: chown d7/d23/d5f/d14/d6f/dbe/fdb 83 1 2024-03-24T10:46:03.781 INFO:tasks.workunit.client.0.smithi094.stdout:3/823: write d0/db/dc/d28/f88 [3757127,12380] 0 2024-03-24T10:46:03.880 INFO:tasks.workunit.client.0.smithi094.stdout:3/824: write d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/ff1 [234978,111517] 0 2024-03-24T10:46:03.881 INFO:tasks.workunit.client.0.smithi094.stdout:2/961: rename d18/d66/da6 to d18/d77/dcf/d125 0 2024-03-24T10:46:03.881 INFO:tasks.workunit.client.0.smithi094.stdout:3/825: chown d0/db/dc/c2c 131288079 1 2024-03-24T10:46:03.881 INFO:tasks.workunit.client.0.smithi094.stdout:3/826: dread - d0/db/dc/fbf zero size 2024-03-24T10:46:03.882 INFO:tasks.workunit.client.0.smithi094.stdout:8/961: dwrite df/d25/d28/d31/d95/de3/f109 [0,4194304] 0 2024-03-24T10:46:03.882 INFO:tasks.workunit.client.0.smithi094.stdout:8/962: fsync df/d25/d28/d31/f32 0 2024-03-24T10:46:03.888 INFO:tasks.workunit.client.0.smithi094.stdout:1/816: write d4/d9/db/d98/fa4 [487238,37784] 0 2024-03-24T10:46:03.891 INFO:tasks.workunit.client.0.smithi094.stdout:6/922: dwrite d0/d21/f56 [4194304,4194304] 0 2024-03-24T10:46:03.894 INFO:tasks.workunit.client.0.smithi094.stdout:2/962: dwrite d18/d48/d56/ff1 [0,4194304] 0 2024-03-24T10:46:03.894 INFO:tasks.workunit.client.0.smithi094.stdout:2/963: chown d18/d66/d90/d9a/d74/db4/dd4/c10d 65000592 1 2024-03-24T10:46:03.915 INFO:tasks.workunit.client.0.smithi094.stdout:8/963: truncate df/d25/d28/d31/d47/d22/d55/f138 2324959 0 2024-03-24T10:46:03.916 INFO:tasks.workunit.client.0.smithi094.stdout:3/827: getdents d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/de9 0 2024-03-24T10:46:03.921 INFO:tasks.workunit.client.0.smithi094.stdout:8/964: dread df/d25/d36/d3f/fdb [4194304,4194304] 0 2024-03-24T10:46:03.926 INFO:tasks.workunit.client.0.smithi094.stdout:2/964: rename d18/d77/dcf/d125/f117 to d18/f126 0 2024-03-24T10:46:03.926 INFO:tasks.workunit.client.0.smithi094.stdout:2/965: chown d18/d66/dbe/dc0/f55 64865224 1 2024-03-24T10:46:03.927 INFO:tasks.workunit.client.0.smithi094.stdout:3/828: creat d0/db/dc/d28/d3c/d55/d7f/d20/f11d x:0 0 0 2024-03-24T10:46:03.927 INFO:tasks.workunit.client.0.smithi094.stdout:3/829: dread - d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d34/fe5 zero size 2024-03-24T10:46:03.930 INFO:tasks.workunit.client.0.smithi094.stdout:8/965: symlink df/d25/d36/d3f/l141 0 2024-03-24T10:46:03.930 INFO:tasks.workunit.client.0.smithi094.stdout:1/817: dwrite d4/d9/de/d6f/d62/d80/fe2 [0,4194304] 0 2024-03-24T10:46:03.930 INFO:tasks.workunit.client.0.smithi094.stdout:8/966: fsync f4 0 2024-03-24T10:46:03.931 INFO:tasks.workunit.client.0.smithi094.stdout:3/830: symlink d0/db/dc/d28/d110/d111/l11e 0 2024-03-24T10:46:03.931 INFO:tasks.workunit.client.0.smithi094.stdout:3/831: chown d0/db/dc/d28/d38/ldc 64930 1 2024-03-24T10:46:03.935 INFO:tasks.workunit.client.0.smithi094.stdout:8/967: rename df/d25/d28/d31/d95/d118/f139 to df/d25/d28/d31/d47/d66/da3/db5/f142 0 2024-03-24T10:46:03.936 INFO:tasks.workunit.client.0.smithi094.stdout:8/968: fdatasync df/d25/d28/d31/d47/f7c 0 2024-03-24T10:46:03.936 INFO:tasks.workunit.client.0.smithi094.stdout:8/969: chown df/d25/d28/d31/d47/d22/fc5 3872 1 2024-03-24T10:46:03.938 INFO:tasks.workunit.client.0.smithi094.stdout:3/832: link d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/fc8 d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/d73/f11f 0 2024-03-24T10:46:03.939 INFO:tasks.workunit.client.0.smithi094.stdout:2/966: dwrite d18/d7f/d88/da2/f107 [0,4194304] 0 2024-03-24T10:46:03.939 INFO:tasks.workunit.client.0.smithi094.stdout:2/967: readlink d18/d7f/d88/da2/da4/l9f 0 2024-03-24T10:46:03.940 INFO:tasks.workunit.client.0.smithi094.stdout:8/970: creat df/d25/d28/d31/d47/d66/da3/db5/dbf/f143 x:0 0 0 2024-03-24T10:46:03.940 INFO:tasks.workunit.client.0.smithi094.stdout:8/971: fsync df/d25/d28/d31/d47/d66/da3/da8/d117/dc1/fec 0 2024-03-24T10:46:03.940 INFO:tasks.workunit.client.0.smithi094.stdout:8/972: write df/d25/d28/d31/d47/d22/d55/f8a [123783,103555] 0 2024-03-24T10:46:03.941 INFO:tasks.workunit.client.0.smithi094.stdout:3/833: symlink d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d87/dc5/l120 0 2024-03-24T10:46:03.941 INFO:tasks.workunit.client.0.smithi094.stdout:3/834: readlink d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/ld8 0 2024-03-24T10:46:03.945 INFO:tasks.workunit.client.0.smithi094.stdout:2/968: rename d18/d77/dc6 to d18/d66/d90/d9a/d74/d7e/dce/d127 0 2024-03-24T10:46:03.946 INFO:tasks.workunit.client.0.smithi094.stdout:8/973: creat df/d25/d28/d31/d47/d66/da3/db5/f144 x:0 0 0 2024-03-24T10:46:03.946 INFO:tasks.workunit.client.0.smithi094.stdout:8/974: chown df/d25/d28/d31/d95/lbb 481974176 1 2024-03-24T10:46:03.950 INFO:tasks.workunit.client.0.smithi094.stdout:0/893: sync 2024-03-24T10:46:03.950 INFO:tasks.workunit.client.0.smithi094.stdout:1/818: dwrite d4/d9/db/d5d/fed [0,4194304] 0 2024-03-24T10:46:03.951 INFO:tasks.workunit.client.0.smithi094.stdout:1/819: truncate d4/d9/de/d6f/d62/fef 143396 0 2024-03-24T10:46:03.951 INFO:tasks.workunit.client.0.smithi094.stdout:1/820: dread - d4/d9/db/d31/d3c/ff6 zero size 2024-03-24T10:46:03.953 INFO:tasks.workunit.client.0.smithi094.stdout:1/821: mknod d4/d9/db/d39/d75/c115 0 2024-03-24T10:46:03.955 INFO:tasks.workunit.client.0.smithi094.stdout:2/969: dread d18/d66/d90/d9a/dbc/f3d [0,4194304] 0 2024-03-24T10:46:03.958 INFO:tasks.workunit.client.0.smithi094.stdout:3/835: dwrite d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d86/fbe [0,4194304] 0 2024-03-24T10:46:03.958 INFO:tasks.workunit.client.0.smithi094.stdout:3/836: fsync d0/db/dc/f2d 0 2024-03-24T10:46:03.958 INFO:tasks.workunit.client.0.smithi094.stdout:3/837: readlink d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/l5d 0 2024-03-24T10:46:03.958 INFO:tasks.workunit.client.0.smithi094.stdout:3/838: creat d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d34/d71/f121 x:0 0 0 2024-03-24T10:46:03.958 INFO:tasks.workunit.client.0.smithi094.stdout:3/839: chown d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/d73/l105 102 1 2024-03-24T10:46:03.958 INFO:tasks.workunit.client.0.smithi094.stdout:0/894: dread d14/d22/d28/d2b/d41/d53/f5a [4194304,4194304] 0 2024-03-24T10:46:03.958 INFO:tasks.workunit.client.0.smithi094.stdout:2/970: read d18/d48/d56/dcc/f7a [37346,53625] 0 2024-03-24T10:46:03.958 INFO:tasks.workunit.client.0.smithi094.stdout:8/975: truncate df/d25/daf/fb0 397936 0 2024-03-24T10:46:03.959 INFO:tasks.workunit.client.0.smithi094.stdout:8/976: fdatasync df/d25/d28/d31/d47/d66/da3/da8/d117/dc1/f125 0 2024-03-24T10:46:03.962 INFO:tasks.workunit.client.0.smithi094.stdout:1/822: dread d4/d9/de/d6f/d62/fbb [0,4194304] 0 2024-03-24T10:46:03.963 INFO:tasks.workunit.client.0.smithi094.stdout:3/840: unlink d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d87/dc5/d101/lbb 0 2024-03-24T10:46:03.966 INFO:tasks.workunit.client.0.smithi094.stdout:0/895: creat d14/d22/d28/d2b/d41/df1/f13d x:0 0 0 2024-03-24T10:46:03.968 INFO:tasks.workunit.client.0.smithi094.stdout:1/823: unlink d4/d9/de/d6f/d62/d84/fc1 0 2024-03-24T10:46:03.977 INFO:tasks.workunit.client.0.smithi094.stdout:5/929: getdents d7/d23/d5f/d1b 0 2024-03-24T10:46:03.978 INFO:tasks.workunit.client.0.smithi094.stdout:0/896: dread d14/d22/d28/f31 [0,4194304] 0 2024-03-24T10:46:03.980 INFO:tasks.workunit.client.0.smithi094.stdout:0/897: mkdir d14/d13e 0 2024-03-24T10:46:03.980 INFO:tasks.workunit.client.0.smithi094.stdout:0/898: readlink d14/d22/d5b/l82 0 2024-03-24T10:46:03.983 INFO:tasks.workunit.client.0.smithi094.stdout:7/877: sync 2024-03-24T10:46:03.983 INFO:tasks.workunit.client.0.smithi094.stdout:4/982: sync 2024-03-24T10:46:03.984 INFO:tasks.workunit.client.0.smithi094.stdout:5/930: read d7/d23/d5f/d14/d36/f47 [3970013,95457] 0 2024-03-24T10:46:03.984 INFO:tasks.workunit.client.0.smithi094.stdout:7/878: creat d1/d9/d16/d21/d51/dcc/f127 x:0 0 0 2024-03-24T10:46:03.984 INFO:tasks.workunit.client.0.smithi094.stdout:7/879: write d1/d9/d16/f116 [951838,97215] 0 2024-03-24T10:46:03.985 INFO:tasks.workunit.client.0.smithi094.stdout:5/931: unlink d7/d23/d5f/d1b/cc2 0 2024-03-24T10:46:03.987 INFO:tasks.workunit.client.0.smithi094.stdout:7/880: creat d1/d9/d16/d21/d51/d83/d39/d44/dd9/f128 x:0 0 0 2024-03-24T10:46:03.988 INFO:tasks.workunit.client.0.smithi094.stdout:2/971: dwrite d18/d66/dbe/fd0 [0,4194304] 0 2024-03-24T10:46:03.989 INFO:tasks.workunit.client.0.smithi094.stdout:5/932: mkdir d7/d23/dd5/d135 0 2024-03-24T10:46:03.991 INFO:tasks.workunit.client.0.smithi094.stdout:2/972: truncate d18/f23 6538673 0 2024-03-24T10:46:03.993 INFO:tasks.workunit.client.0.smithi094.stdout:2/973: rename d18/d77/dcf/c115 to d18/d66/d90/d9a/dbc/d71/c128 0 2024-03-24T10:46:03.994 INFO:tasks.workunit.client.0.smithi094.stdout:8/977: dwrite df/d25/daf/d13f/d5b/ff8 [0,4194304] 0 2024-03-24T10:46:03.994 INFO:tasks.workunit.client.0.smithi094.stdout:7/881: write d1/d9/d16/d21/d51/d83/d39/d44/f56 [725182,38539] 0 2024-03-24T10:46:03.995 INFO:tasks.workunit.client.0.smithi094.stdout:7/882: chown d1/d9b/c100 61026 1 2024-03-24T10:46:03.995 INFO:tasks.workunit.client.0.smithi094.stdout:7/883: write d1/db4/d65/f89 [4744510,25026] 0 2024-03-24T10:46:03.995 INFO:tasks.workunit.client.0.smithi094.stdout:1/824: dwrite d4/d9/db/f18 [0,4194304] 0 2024-03-24T10:46:03.996 INFO:tasks.workunit.client.0.smithi094.stdout:3/841: rmdir d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56 39 2024-03-24T10:46:03.997 INFO:tasks.workunit.client.0.smithi094.stdout:3/842: fdatasync d0/db/dc/d28/d50/d5e/fb7 0 2024-03-24T10:46:03.997 INFO:tasks.workunit.client.0.smithi094.stdout:0/899: write d14/d22/d28/d2b/dc0/f115 [2331347,67591] 0 2024-03-24T10:46:03.997 INFO:tasks.workunit.client.0.smithi094.stdout:0/900: fdatasync d14/d22/d32/d70/ff5 0 2024-03-24T10:46:03.997 INFO:tasks.workunit.client.0.smithi094.stdout:0/901: write fd [1317573,126699] 0 2024-03-24T10:46:03.997 INFO:tasks.workunit.client.0.smithi094.stdout:4/983: read d0/d34/d73/d93/f113 [793702,45150] 0 2024-03-24T10:46:03.997 INFO:tasks.workunit.client.0.smithi094.stdout:8/978: rename df/d25/d28/d31/d47/c99 to df/d25/d28/d31/d47/d66/da3/da8/d117/df2/d11e/c145 0 2024-03-24T10:46:04.000 INFO:tasks.workunit.client.0.smithi094.stdout:7/884: creat d1/d9/d16/d21/d49/dfa/f129 x:0 0 0 2024-03-24T10:46:04.007 INFO:tasks.workunit.client.0.smithi094.stdout:5/933: dwrite d7/d23/d5f/d14/d6f/f89 [0,4194304] 0 2024-03-24T10:46:04.007 INFO:tasks.workunit.client.0.smithi094.stdout:5/934: fdatasync d7/f4c 0 2024-03-24T10:46:04.009 INFO:tasks.workunit.client.0.smithi094.stdout:1/825: unlink d4/d9/db/d85/df5/lfe 0 2024-03-24T10:46:04.010 INFO:tasks.workunit.client.0.smithi094.stdout:2/974: mknod d18/d66/d90/d9a/dbc/c129 0 2024-03-24T10:46:04.011 INFO:tasks.workunit.client.0.smithi094.stdout:4/984: mknod d0/d34/d73/c136 0 2024-03-24T10:46:04.011 INFO:tasks.workunit.client.0.smithi094.stdout:4/985: chown d0 9 1 2024-03-24T10:46:04.011 INFO:tasks.workunit.client.0.smithi094.stdout:2/975: dread d18/d54/db9/fdd [0,4194304] 0 2024-03-24T10:46:04.012 INFO:tasks.workunit.client.0.smithi094.stdout:2/976: chown d18/f45 6 1 2024-03-24T10:46:04.012 INFO:tasks.workunit.client.0.smithi094.stdout:2/977: readlink d18/l2c 0 2024-03-24T10:46:04.012 INFO:tasks.workunit.client.0.smithi094.stdout:3/843: dwrite d0/f42 [0,4194304] 0 2024-03-24T10:46:04.012 INFO:tasks.workunit.client.0.smithi094.stdout:2/978: read - d18/d54/db9/f113 zero size 2024-03-24T10:46:04.013 INFO:tasks.workunit.client.0.smithi094.stdout:8/979: creat df/d25/d28/d31/d47/dfe/d136/f146 x:0 0 0 2024-03-24T10:46:04.013 INFO:tasks.workunit.client.0.smithi094.stdout:8/980: dread - df/d25/d36/d10b/f128 zero size 2024-03-24T10:46:04.013 INFO:tasks.workunit.client.0.smithi094.stdout:8/981: truncate df/d25/daf/d13f/d5b/d63/d75/f100 690023 0 2024-03-24T10:46:04.014 INFO:tasks.workunit.client.0.smithi094.stdout:7/885: link d1/d9/d16/d21/d51/d83/d39/d44/dd9/d8f/l122 d1/d9/d16/d21/d51/d83/d39/d44/dd9/d34/dc8/l12a 0 2024-03-24T10:46:04.019 INFO:tasks.workunit.client.0.smithi094.stdout:5/935: rename d7/d23/d5f/d14/d36/d3a/c3f to d7/d23/d132/c136 0 2024-03-24T10:46:04.020 INFO:tasks.workunit.client.0.smithi094.stdout:3/844: dread d0/db/dc/d28/d3c/d55/d7f/d20/d23/f9a [0,4194304] 0 2024-03-24T10:46:04.021 INFO:tasks.workunit.client.0.smithi094.stdout:3/845: fdatasync d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d34/fe5 0 2024-03-24T10:46:04.021 INFO:tasks.workunit.client.0.smithi094.stdout:1/826: truncate d4/d9/db1/db9/fd1 1730117 0 2024-03-24T10:46:04.021 INFO:tasks.workunit.client.0.smithi094.stdout:1/827: truncate d4/f2a 4214041 0 2024-03-24T10:46:04.021 INFO:tasks.workunit.client.0.smithi094.stdout:1/828: fsync d4/d9/db/f95 0 2024-03-24T10:46:04.022 INFO:tasks.workunit.client.0.smithi094.stdout:4/986: mkdir d0/d22/de/d28/d137 0 2024-03-24T10:46:04.022 INFO:tasks.workunit.client.0.smithi094.stdout:4/987: chown d0/d5/d1b/d4f/d70/f7c 249200101 1 2024-03-24T10:46:04.023 INFO:tasks.workunit.client.0.smithi094.stdout:2/979: write d18/d48/f5d [3714944,39798] 0 2024-03-24T10:46:04.024 INFO:tasks.workunit.client.0.smithi094.stdout:4/988: dread d0/d22/de/d28/d3d/d8c/dec/d131/da4/da5/fe2 [0,4194304] 0 2024-03-24T10:46:04.024 INFO:tasks.workunit.client.0.smithi094.stdout:8/982: rmdir df/d25/d36 39 2024-03-24T10:46:04.026 INFO:tasks.workunit.client.0.smithi094.stdout:7/886: mkdir d1/d6c/d12b 0 2024-03-24T10:46:04.030 INFO:tasks.workunit.client.0.smithi094.stdout:5/936: unlink d7/d23/d5f/d14/d6f/f91 0 2024-03-24T10:46:04.030 INFO:tasks.workunit.client.0.smithi094.stdout:5/937: fsync d7/d23/dd5/f12e 0 2024-03-24T10:46:04.030 INFO:tasks.workunit.client.0.smithi094.stdout:5/938: chown d7/d23/d5f/d14/d9f 526044 1 2024-03-24T10:46:04.032 INFO:tasks.workunit.client.0.smithi094.stdout:3/846: creat d0/db/dc/d28/d3c/d55/d7f/f122 x:0 0 0 2024-03-24T10:46:04.033 INFO:tasks.workunit.client.0.smithi094.stdout:3/847: write d0/db/dc/d28/d3c/d7c/f8e [851894,10541] 0 2024-03-24T10:46:04.033 INFO:tasks.workunit.client.0.smithi094.stdout:3/848: chown d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/c49 245490 1 2024-03-24T10:46:04.035 INFO:tasks.workunit.client.0.smithi094.stdout:1/829: rename d4/d9/d7d/l6e to d4/l116 0 2024-03-24T10:46:04.035 INFO:tasks.workunit.client.0.smithi094.stdout:1/830: write d4/d9/d37/d33/d58/d6b/fe4 [1031180,95460] 0 2024-03-24T10:46:04.035 INFO:tasks.workunit.client.0.smithi094.stdout:1/831: getdents d4/d9/d37/d33/d76 0 2024-03-24T10:46:04.035 INFO:tasks.workunit.client.0.smithi094.stdout:2/980: truncate d18/d48/f52 3376736 0 2024-03-24T10:46:04.037 INFO:tasks.workunit.client.0.smithi094.stdout:8/983: symlink df/d25/daf/d13f/d5b/l147 0 2024-03-24T10:46:04.041 INFO:tasks.workunit.client.0.smithi094.stdout:1/832: mknod d4/c117 0 2024-03-24T10:46:04.041 INFO:tasks.workunit.client.0.smithi094.stdout:4/989: dwrite d0/d5/d7d/d97/f67 [0,4194304] 0 2024-03-24T10:46:04.043 INFO:tasks.workunit.client.0.smithi094.stdout:8/984: mkdir df/d25/d28/d31/d47/d66/da3/da8/d117/df2/d148 0 2024-03-24T10:46:04.043 INFO:tasks.workunit.client.0.smithi094.stdout:8/985: chown df/d25/d28/d31/d47/d66/da3/fa4 19707379 1 2024-03-24T10:46:04.043 INFO:tasks.workunit.client.0.smithi094.stdout:6/923: sync 2024-03-24T10:46:04.044 INFO:tasks.workunit.client.0.smithi094.stdout:6/924: write d0/d30/d50/d4d/dd2/feb [140328,41051] 0 2024-03-24T10:46:04.044 INFO:tasks.workunit.client.0.smithi094.stdout:7/887: rmdir d1/d9/d16/d21/d49/dfa 39 2024-03-24T10:46:04.044 INFO:tasks.workunit.client.0.smithi094.stdout:7/888: write d1/f120 [161903,107843] 0 2024-03-24T10:46:04.044 INFO:tasks.workunit.client.0.smithi094.stdout:7/889: stat d1/f120 0 2024-03-24T10:46:04.046 INFO:tasks.workunit.client.0.smithi094.stdout:5/939: rmdir d7/d23/dd5 39 2024-03-24T10:46:04.046 INFO:tasks.workunit.client.0.smithi094.stdout:5/940: creat d7/d23/d5f/d9/d48/d73/f137 x:0 0 0 2024-03-24T10:46:04.047 INFO:tasks.workunit.client.0.smithi094.stdout:3/849: rename d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d86/c91 to d0/db/dc/d28/d3c/d55/d7f/d20/d23/c123 0 2024-03-24T10:46:04.047 INFO:tasks.workunit.client.0.smithi094.stdout:3/850: chown d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/db5/d106/d9b/d10d/f104 1855 1 2024-03-24T10:46:04.048 INFO:tasks.workunit.client.0.smithi094.stdout:1/833: link d4/d9/de/d6f/d62/d84/de9/f112 d4/d9/de/d6f/d62/d99/f118 0 2024-03-24T10:46:04.048 INFO:tasks.workunit.client.0.smithi094.stdout:1/834: dread - d4/d9/db/d31/f91 zero size 2024-03-24T10:46:04.052 INFO:tasks.workunit.client.0.smithi094.stdout:2/981: rmdir d18/d66/d90/d9a/d74 39 2024-03-24T10:46:04.053 INFO:tasks.workunit.client.0.smithi094.stdout:1/835: dread d4/d9/de/d6f/d62/fbb [0,4194304] 0 2024-03-24T10:46:04.053 INFO:tasks.workunit.client.0.smithi094.stdout:1/836: creat d4/d9/db/d5d/d63/f119 x:0 0 0 2024-03-24T10:46:04.054 INFO:tasks.workunit.client.0.smithi094.stdout:8/986: link df/d25/d28/d31/d47/f77 df/d25/d28/d31/d47/ddf/f149 0 2024-03-24T10:46:04.054 INFO:tasks.workunit.client.0.smithi094.stdout:8/987: creat df/d25/daf/f14a x:0 0 0 2024-03-24T10:46:04.054 INFO:tasks.workunit.client.0.smithi094.stdout:8/988: readlink df/d25/d36/d3f/l141 0 2024-03-24T10:46:04.055 INFO:tasks.workunit.client.0.smithi094.stdout:6/925: stat d0/d21/d2a/l98 0 2024-03-24T10:46:04.057 INFO:tasks.workunit.client.0.smithi094.stdout:5/941: truncate d7/d23/d5f/d14/d6f/dbe/fdb 801192 0 2024-03-24T10:46:04.058 INFO:tasks.workunit.client.0.smithi094.stdout:7/890: dwrite d1/db4/f9e [0,4194304] 0 2024-03-24T10:46:04.060 INFO:tasks.workunit.client.0.smithi094.stdout:4/990: rename d0/d5/d1b/f4e to d0/d22/de/d28/d137/f138 0 2024-03-24T10:46:04.061 INFO:tasks.workunit.client.0.smithi094.stdout:3/851: creat d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/f124 x:0 0 0 2024-03-24T10:46:04.061 INFO:tasks.workunit.client.0.smithi094.stdout:3/852: readlink d0/l10c 0 2024-03-24T10:46:04.064 INFO:tasks.workunit.client.0.smithi094.stdout:1/837: creat d4/d9/db/d39/d43/f11a x:0 0 0 2024-03-24T10:46:04.067 INFO:tasks.workunit.client.0.smithi094.stdout:4/991: read d0/d22/de/d28/d3d/d8c/fc7 [945141,13517] 0 2024-03-24T10:46:04.067 INFO:tasks.workunit.client.0.smithi094.stdout:4/992: write d0/d22/de/d28/f3a [5123811,115831] 0 2024-03-24T10:46:04.067 INFO:tasks.workunit.client.0.smithi094.stdout:4/993: dread - d0/d22/de/d28/d3d/d8c/dec/d131/da4/da5/f109 zero size 2024-03-24T10:46:04.067 INFO:tasks.workunit.client.0.smithi094.stdout:4/994: write d0/d5/fab [5121405,15607] 0 2024-03-24T10:46:04.067 INFO:tasks.workunit.client.0.smithi094.stdout:4/995: write d0/d22/de/d28/d2b/da2/fe0 [125857,102888] 0 2024-03-24T10:46:04.067 INFO:tasks.workunit.client.0.smithi094.stdout:4/996: rename d0/d5/d7d/d97/d4a to d0/d5/d7d/d97/d4a/d139 22 2024-03-24T10:46:04.068 INFO:tasks.workunit.client.0.smithi094.stdout:8/989: mknod df/d25/d28/d31/d47/d66/da3/da8/d117/c14b 0 2024-03-24T10:46:04.069 INFO:tasks.workunit.client.0.smithi094.stdout:6/926: getdents d0/d78/da/d23/dde/d11c 0 2024-03-24T10:46:04.069 INFO:tasks.workunit.client.0.smithi094.stdout:6/927: stat d0/d1a/d35 0 2024-03-24T10:46:04.071 INFO:tasks.workunit.client.0.smithi094.stdout:5/942: creat d7/d23/dd5/d135/f138 x:0 0 0 2024-03-24T10:46:04.081 INFO:tasks.workunit.client.0.smithi094.stdout:7/891: creat d1/d9/d16/d21/d51/d83/f12c x:0 0 0 2024-03-24T10:46:04.085 INFO:tasks.workunit.client.0.smithi094.stdout:3/853: link d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/db5/cc0 d0/db/dc/dd9/c125 0 2024-03-24T10:46:04.085 INFO:tasks.workunit.client.0.smithi094.stdout:3/854: write d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/dfb/f115 [746475,67087] 0 2024-03-24T10:46:04.085 INFO:tasks.workunit.client.0.smithi094.stdout:3/855: fsync d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d86/fa3 0 2024-03-24T10:46:04.085 INFO:tasks.workunit.client.0.smithi094.stdout:3/856: creat d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d4f/f126 x:0 0 0 2024-03-24T10:46:04.085 INFO:tasks.workunit.client.0.smithi094.stdout:3/857: fdatasync d0/db/dc/d28/d3c/d55/d7f/f122 0 2024-03-24T10:46:04.085 INFO:tasks.workunit.client.0.smithi094.stdout:8/990: dwrite df/d25/d36/d3f/f92 [0,4194304] 0 2024-03-24T10:46:04.085 INFO:tasks.workunit.client.0.smithi094.stdout:1/838: dwrite d4/d9/d37/d33/d58/fd4 [0,4194304] 0 2024-03-24T10:46:04.088 INFO:tasks.workunit.client.0.smithi094.stdout:4/997: symlink d0/d5/d1b/l13a 0 2024-03-24T10:46:04.089 INFO:tasks.workunit.client.0.smithi094.stdout:4/998: dread - d0/d5/d1b/de8/f117 zero size 2024-03-24T10:46:04.091 INFO:tasks.workunit.client.0.smithi094.stdout:5/943: rename d7/d23/d5f/d9/d48/d73/ff6 to d7/d23/d132/f139 0 2024-03-24T10:46:04.093 INFO:tasks.workunit.client.0.smithi094.stdout:5/944: write d7/fdd [882484,65130] 0 2024-03-24T10:46:04.093 INFO:tasks.workunit.client.0.smithi094.stdout:5/945: stat d7/d23/d5f/d9/d48/d51/f6a 0 2024-03-24T10:46:04.093 INFO:tasks.workunit.client.0.smithi094.stdout:7/892: mknod d1/db4/c12d 0 2024-03-24T10:46:04.093 INFO:tasks.workunit.client.0.smithi094.stdout:7/893: stat d1/d6c/fee 0 2024-03-24T10:46:04.097 INFO:tasks.workunit.client.0.smithi094.stdout:8/991: symlink df/d25/d28/d31/d47/ddf/l14c 0 2024-03-24T10:46:04.097 INFO:tasks.workunit.client.0.smithi094.stdout:1/839: truncate d4/d9/db/d31/f48 3469063 0 2024-03-24T10:46:04.102 INFO:tasks.workunit.client.0.smithi094.stdout:1/840: dread d4/d9/f32 [0,4194304] 0 2024-03-24T10:46:04.103 INFO:tasks.workunit.client.0.smithi094.stdout:5/946: dread d7/d23/dd5/da9/ff2 [0,4194304] 0 2024-03-24T10:46:04.103 INFO:tasks.workunit.client.0.smithi094.stdout:5/947: getdents d7/d23/d5f/d9/dde 0 2024-03-24T10:46:04.103 INFO:tasks.workunit.client.0.smithi094.stdout:5/948: dread - d7/d23/d5f/d9/d48/ffb zero size 2024-03-24T10:46:04.103 INFO:tasks.workunit.client.0.smithi094.stdout:4/999: mkdir d0/d22/de/d28/d3d/d10d/d13b 0 2024-03-24T10:46:04.103 INFO:tasks.workunit.client.0.smithi094.stdout:6/928: getdents d0/d21/d2a 0 2024-03-24T10:46:04.104 INFO:tasks.workunit.client.0.smithi094.stdout:5/949: dread d7/d26/f84 [0,4194304] 0 2024-03-24T10:46:04.106 INFO:tasks.workunit.client.0.smithi094.stdout:3/858: rename d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d4f/f109 to d0/db/dfd/f127 0 2024-03-24T10:46:04.106 INFO:tasks.workunit.client.0.smithi094.stdout:7/894: getdents d1/d9/d16/d21/d51 0 2024-03-24T10:46:04.107 INFO:tasks.workunit.client.0.smithi094.stdout:7/895: read d1/d9/d16/d21/d49/f84 [7997961,8310] 0 2024-03-24T10:46:04.108 INFO:tasks.workunit.client.0.smithi094.stdout:8/992: creat df/d25/d36/d3f/f14d x:0 0 0 2024-03-24T10:46:04.108 INFO:tasks.workunit.client.0.smithi094.stdout:8/993: fdatasync df/d25/daf/d13f/f21 0 2024-03-24T10:46:04.110 INFO:tasks.workunit.client.0.smithi094.stdout:1/841: unlink d4/d36/f3f 0 2024-03-24T10:46:04.112 INFO:tasks.workunit.client.0.smithi094.stdout:5/950: write d7/dc/f19 [3737324,124344] 0 2024-03-24T10:46:04.112 INFO:tasks.workunit.client.0.smithi094.stdout:6/929: rename d0/d78/da/d23/dde/d11c/fa6 to d0/d78/d66/d7a/dfe/f12b 0 2024-03-24T10:46:04.115 INFO:tasks.workunit.client.0.smithi094.stdout:1/842: mkdir d4/d9/d11b 0 2024-03-24T10:46:04.116 INFO:tasks.workunit.client.0.smithi094.stdout:8/994: dread df/d25/daf/d13f/f39 [0,4194304] 0 2024-03-24T10:46:04.116 INFO:tasks.workunit.client.0.smithi094.stdout:8/995: stat df/d25/d36/f9b 0 2024-03-24T10:46:04.117 INFO:tasks.workunit.client.0.smithi094.stdout:5/951: truncate d7/d23/d5f/d9/d48/d51/fc7 3971238 0 2024-03-24T10:46:04.118 INFO:tasks.workunit.client.0.smithi094.stdout:6/930: symlink d0/d30/d50/l12c 0 2024-03-24T10:46:04.120 INFO:tasks.workunit.client.0.smithi094.stdout:3/859: mkdir d0/db/dc/d28/d3c/d55/d7f/d20/d23/d128 0 2024-03-24T10:46:04.121 INFO:tasks.workunit.client.0.smithi094.stdout:1/843: mknod d4/d9/de/c11c 0 2024-03-24T10:46:04.123 INFO:tasks.workunit.client.0.smithi094.stdout:6/931: truncate d0/d78/f58 262328 0 2024-03-24T10:46:04.124 INFO:tasks.workunit.client.0.smithi094.stdout:3/860: rename d0/db/dc/ca7 to d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d4f/c129 0 2024-03-24T10:46:04.124 INFO:tasks.workunit.client.0.smithi094.stdout:3/861: chown d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/db5/d106/d9b/d10d/f104 6689 1 2024-03-24T10:46:04.132 INFO:tasks.workunit.client.0.smithi094.stdout:6/932: write d0/d78/da/d23/f9b [742350,105788] 0 2024-03-24T10:46:04.135 INFO:tasks.workunit.client.0.smithi094.stdout:8/996: dwrite df/d25/d28/d31/d47/d22/d55/f7a [0,4194304] 0 2024-03-24T10:46:04.137 INFO:tasks.workunit.client.0.smithi094.stdout:8/997: truncate df/d25/d28/d31/fd6 378478 0 2024-03-24T10:46:04.137 INFO:tasks.workunit.client.0.smithi094.stdout:8/998: dread - df/d25/d28/d31/d47/dfe/d136/f146 zero size 2024-03-24T10:46:04.137 INFO:tasks.workunit.client.0.smithi094.stdout:8/999: stat df/d25/daf/d13f/l4d 0 2024-03-24T10:46:04.147 INFO:tasks.workunit.client.0.smithi094.stdout:7/896: dwrite d1/db4/d65/fb6 [0,4194304] 0 2024-03-24T10:46:04.151 INFO:tasks.workunit.client.0.smithi094.stdout:7/897: read d1/d9/d16/fbe [84565,130133] 0 2024-03-24T10:46:04.151 INFO:tasks.workunit.client.0.smithi094.stdout:7/898: truncate d1/d9/d25/d3d/fda 575116 0 2024-03-24T10:46:04.153 INFO:tasks.workunit.client.0.smithi094.stdout:7/899: rename d1/db4/fba to d1/d9/d16/f12e 0 2024-03-24T10:46:04.154 INFO:tasks.workunit.client.0.smithi094.stdout:7/900: creat d1/d9/d16/d21/d51/d83/d39/d44/dd9/d8f/f12f x:0 0 0 2024-03-24T10:46:04.159 INFO:tasks.workunit.client.0.smithi094.stdout:7/901: dread d1/d9/f30 [0,4194304] 0 2024-03-24T10:46:04.166 INFO:tasks.workunit.client.0.smithi094.stdout:1/844: dwrite d4/d36/f114 [0,4194304] 0 2024-03-24T10:46:04.166 INFO:tasks.workunit.client.0.smithi094.stdout:7/902: dread d1/db4/d65/f89 [0,4194304] 0 2024-03-24T10:46:04.166 INFO:tasks.workunit.client.0.smithi094.stdout:5/952: dwrite d7/dc/d100/d109/f10c [0,4194304] 0 2024-03-24T10:46:04.166 INFO:tasks.workunit.client.0.smithi094.stdout:3/862: dwrite d0/fb6 [0,4194304] 0 2024-03-24T10:46:04.166 INFO:tasks.workunit.client.0.smithi094.stdout:5/953: write d7/d23/dd5/d135/f138 [664144,114823] 0 2024-03-24T10:46:04.166 INFO:tasks.workunit.client.0.smithi094.stdout:6/933: dwrite d0/d78/da/d23/d83/fe9 [8388608,4194304] 0 2024-03-24T10:46:04.171 INFO:tasks.workunit.client.0.smithi094.stdout:1/845: symlink d4/d9/db/d39/d75/l11d 0 2024-03-24T10:46:04.173 INFO:tasks.workunit.client.0.smithi094.stdout:3/863: write d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d87/dc5/d101/f9f [2609489,121370] 0 2024-03-24T10:46:04.173 INFO:tasks.workunit.client.0.smithi094.stdout:7/903: symlink d1/d9/d25/dd4/l130 0 2024-03-24T10:46:04.173 INFO:tasks.workunit.client.0.smithi094.stdout:7/904: fdatasync d1/fec 0 2024-03-24T10:46:04.173 INFO:tasks.workunit.client.0.smithi094.stdout:7/905: write d1/d9/d96/f9a [4823588,30882] 0 2024-03-24T10:46:04.180 INFO:tasks.workunit.client.0.smithi094.stdout:1/846: symlink d4/d9/db/d85/df5/l11e 0 2024-03-24T10:46:04.182 INFO:tasks.workunit.client.0.smithi094.stdout:3/864: rename d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d87/dc5/fee to d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/f12a 0 2024-03-24T10:46:04.182 INFO:tasks.workunit.client.0.smithi094.stdout:3/865: creat d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/dfb/f12b x:0 0 0 2024-03-24T10:46:04.183 INFO:tasks.workunit.client.0.smithi094.stdout:7/906: mknod d1/c131 0 2024-03-24T10:46:04.185 INFO:tasks.workunit.client.0.smithi094.stdout:1/847: symlink d4/d9/de/d3d/dae/l11f 0 2024-03-24T10:46:04.185 INFO:tasks.workunit.client.0.smithi094.stdout:1/848: dread - d4/d9/d37/d33/d58/fec zero size 2024-03-24T10:46:04.185 INFO:tasks.workunit.client.0.smithi094.stdout:0/902: sync 2024-03-24T10:46:04.186 INFO:tasks.workunit.client.0.smithi094.stdout:2/982: sync 2024-03-24T10:46:04.187 INFO:tasks.workunit.client.0.smithi094.stdout:2/983: creat d18/d1e/dfd/f12a x:0 0 0 2024-03-24T10:46:04.188 INFO:tasks.workunit.client.0.smithi094.stdout:2/984: dread d18/d66/d90/d9a/dbc/f49 [0,4194304] 0 2024-03-24T10:46:04.190 INFO:tasks.workunit.client.0.smithi094.stdout:3/866: dread d0/db/dc/d28/d38/fa6 [0,4194304] 0 2024-03-24T10:46:04.190 INFO:tasks.workunit.client.0.smithi094.stdout:0/903: rename d14/d22/d28/d2b/d41/d53/f5a to d14/d22/d103/d110/f13f 0 2024-03-24T10:46:04.190 INFO:tasks.workunit.client.0.smithi094.stdout:0/904: chown d14/d22/d28/d2b/d68/lf7 1 1 2024-03-24T10:46:04.190 INFO:tasks.workunit.client.0.smithi094.stdout:0/905: truncate d14/d22/d32/d70/dd0/f8c 858066 0 2024-03-24T10:46:04.190 INFO:tasks.workunit.client.0.smithi094.stdout:0/906: readlink d14/d22/d7d/l91 0 2024-03-24T10:46:04.190 INFO:tasks.workunit.client.0.smithi094.stdout:0/907: chown d14/d22/d28/d48/da5/de0/d13c 157561070 1 2024-03-24T10:46:04.190 INFO:tasks.workunit.client.0.smithi094.stdout:0/908: readlink d14/d22/d28/d2b/l9f 0 2024-03-24T10:46:04.190 INFO:tasks.workunit.client.0.smithi094.stdout:0/909: fdatasync d14/d22/d7d/fa6 0 2024-03-24T10:46:04.191 INFO:tasks.workunit.client.0.smithi094.stdout:7/907: dread d1/d9/d16/d21/d51/d83/d39/d44/f8a [0,4194304] 0 2024-03-24T10:46:04.192 INFO:tasks.workunit.client.0.smithi094.stdout:2/985: unlink d18/d1e/l89 0 2024-03-24T10:46:04.193 INFO:tasks.workunit.client.0.smithi094.stdout:3/867: mknod d0/db/dc/d28/d3c/d55/c12c 0 2024-03-24T10:46:04.193 INFO:tasks.workunit.client.0.smithi094.stdout:3/868: chown d0/db/dc/d28/d3c/f83 3005 1 2024-03-24T10:46:04.193 INFO:tasks.workunit.client.0.smithi094.stdout:2/986: dread d18/d54/db9/fdd [0,4194304] 0 2024-03-24T10:46:04.194 INFO:tasks.workunit.client.0.smithi094.stdout:2/987: write d18/d1e/d65/f11c [85047,105947] 0 2024-03-24T10:46:04.194 INFO:tasks.workunit.client.0.smithi094.stdout:2/988: write d18/d66/d90/d9a/d74/d7e/fd3 [409819,129896] 0 2024-03-24T10:46:04.195 INFO:tasks.workunit.client.0.smithi094.stdout:7/908: mknod d1/d9/d16/d21/d51/d83/d39/d44/c132 0 2024-03-24T10:46:04.195 INFO:tasks.workunit.client.0.smithi094.stdout:1/849: write d4/d36/f44 [4405914,27440] 0 2024-03-24T10:46:04.195 INFO:tasks.workunit.client.0.smithi094.stdout:1/850: fsync d4/d9/db/d31/d3c/f7c 0 2024-03-24T10:46:04.196 INFO:tasks.workunit.client.0.smithi094.stdout:3/869: rename d0/db/dc/l40 to d0/l12d 0 2024-03-24T10:46:04.198 INFO:tasks.workunit.client.0.smithi094.stdout:2/989: creat d18/d66/d90/d9a/def/f12b x:0 0 0 2024-03-24T10:46:04.200 INFO:tasks.workunit.client.0.smithi094.stdout:0/910: getdents d14/d18 0 2024-03-24T10:46:04.200 INFO:tasks.workunit.client.0.smithi094.stdout:0/911: write d14/d22/d28/d2b/dc0/de7/f133 [906835,39258] 0 2024-03-24T10:46:04.200 INFO:tasks.workunit.client.0.smithi094.stdout:0/912: dread - d14/d22/d28/d48/da5/f109 zero size 2024-03-24T10:46:04.203 INFO:tasks.workunit.client.0.smithi094.stdout:1/851: mkdir d4/d9/db/d5d/d120 0 2024-03-24T10:46:04.205 INFO:tasks.workunit.client.0.smithi094.stdout:2/990: truncate d18/d66/dbe/ddf/ffb 3884578 0 2024-03-24T10:46:04.205 INFO:tasks.workunit.client.0.smithi094.stdout:2/991: write d18/d66/d90/fda [4479613,96738] 0 2024-03-24T10:46:04.207 INFO:tasks.workunit.client.0.smithi094.stdout:0/913: unlink d14/d22/d7d/f9a 0 2024-03-24T10:46:04.208 INFO:tasks.workunit.client.0.smithi094.stdout:2/992: mknod d18/d66/d90/d9a/d74/d7e/dce/c12c 0 2024-03-24T10:46:04.213 INFO:tasks.workunit.client.0.smithi094.stdout:2/993: dread d18/d1e/f53 [0,4194304] 0 2024-03-24T10:46:04.213 INFO:tasks.workunit.client.0.smithi094.stdout:2/994: chown d18/d7f/d88/da2/f101 480 1 2024-03-24T10:46:04.213 INFO:tasks.workunit.client.0.smithi094.stdout:2/995: chown d18/d7f/d88/d8a/l103 1345 1 2024-03-24T10:46:04.213 INFO:tasks.workunit.client.0.smithi094.stdout:2/996: creat d18/d48/dc3/f12d x:0 0 0 2024-03-24T10:46:04.214 INFO:tasks.workunit.client.0.smithi094.stdout:2/997: stat d18/d7f/d88/d11e 0 2024-03-24T10:46:04.214 INFO:tasks.workunit.client.0.smithi094.stdout:1/852: dread d4/f4b [0,4194304] 0 2024-03-24T10:46:04.216 INFO:tasks.workunit.client.0.smithi094.stdout:1/853: link d4/d9/l35 d4/d9/d37/dcf/l121 0 2024-03-24T10:46:04.218 INFO:tasks.workunit.client.0.smithi094.stdout:7/909: dwrite d1/d9/fac [0,4194304] 0 2024-03-24T10:46:04.218 INFO:tasks.workunit.client.0.smithi094.stdout:3/870: dwrite d0/db/dc/d28/d3c/d55/d7f/d20/d23/f5f [0,4194304] 0 2024-03-24T10:46:04.221 INFO:tasks.workunit.client.0.smithi094.stdout:0/914: dread d14/d22/f2f [0,4194304] 0 2024-03-24T10:46:04.224 INFO:tasks.workunit.client.0.smithi094.stdout:7/910: mkdir d1/d9/d25/d3d/d112/d133 0 2024-03-24T10:46:04.224 INFO:tasks.workunit.client.0.smithi094.stdout:7/911: fdatasync d1/db4/d65/f66 0 2024-03-24T10:46:04.226 INFO:tasks.workunit.client.0.smithi094.stdout:3/871: mkdir d0/db/dc/d28/d38/d12e 0 2024-03-24T10:46:04.226 INFO:tasks.workunit.client.0.smithi094.stdout:3/872: dread - d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/dfb/f12b zero size 2024-03-24T10:46:04.226 INFO:tasks.workunit.client.0.smithi094.stdout:3/873: fdatasync d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d87/f9c 0 2024-03-24T10:46:04.227 INFO:tasks.workunit.client.0.smithi094.stdout:0/915: creat d14/d22/d28/d2b/f140 x:0 0 0 2024-03-24T10:46:04.229 INFO:tasks.workunit.client.0.smithi094.stdout:3/874: mknod d0/db/dc/d28/d3c/d7c/de8/c12f 0 2024-03-24T10:46:04.229 INFO:tasks.workunit.client.0.smithi094.stdout:3/875: readlink d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d4f/l96 0 2024-03-24T10:46:04.230 INFO:tasks.workunit.client.0.smithi094.stdout:3/876: mknod d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/db5/d106/c130 0 2024-03-24T10:46:04.234 INFO:tasks.workunit.client.0.smithi094.stdout:2/998: dwrite d18/d54/f5c [4194304,4194304] 0 2024-03-24T10:46:04.234 INFO:tasks.workunit.client.0.smithi094.stdout:2/999: stat d18/fe4 0 2024-03-24T10:46:04.238 INFO:tasks.workunit.client.0.smithi094.stdout:0/916: dread d14/d22/d28/f31 [0,4194304] 0 2024-03-24T10:46:04.239 INFO:tasks.workunit.client.0.smithi094.stdout:1/854: dwrite d4/d9/db/f1f [0,4194304] 0 2024-03-24T10:46:04.240 INFO:tasks.workunit.client.0.smithi094.stdout:1/855: write d4/d9/db1/db9/fea [216135,70663] 0 2024-03-24T10:46:04.240 INFO:tasks.workunit.client.0.smithi094.stdout:0/917: creat d14/d22/d28/d2b/d41/d53/d105/f141 x:0 0 0 2024-03-24T10:46:04.240 INFO:tasks.workunit.client.0.smithi094.stdout:0/918: chown d14/d22/d103 184901 1 2024-03-24T10:46:04.241 INFO:tasks.workunit.client.0.smithi094.stdout:0/919: mkdir d14/d22/d28/d2b/d68/d142 0 2024-03-24T10:46:04.241 INFO:tasks.workunit.client.0.smithi094.stdout:0/920: fdatasync d14/fcd 0 2024-03-24T10:46:04.243 INFO:tasks.workunit.client.0.smithi094.stdout:0/921: mkdir d14/d22/d32/d70/dd0/dce/d143 0 2024-03-24T10:46:04.243 INFO:tasks.workunit.client.0.smithi094.stdout:0/922: truncate d14/d22/d7d/f9c 809436 0 2024-03-24T10:46:04.245 INFO:tasks.workunit.client.0.smithi094.stdout:0/923: rename d14/d22/dcb/de5/f136 to d14/f144 0 2024-03-24T10:46:04.245 INFO:tasks.workunit.client.0.smithi094.stdout:0/924: readlink d14/l100 0 2024-03-24T10:46:04.252 INFO:tasks.workunit.client.0.smithi094.stdout:3/877: dwrite d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/f7b [4194304,4194304] 0 2024-03-24T10:46:04.252 INFO:tasks.workunit.client.0.smithi094.stdout:3/878: write d0/db/dc3/ff4 [256268,72122] 0 2024-03-24T10:46:04.252 INFO:tasks.workunit.client.0.smithi094.stdout:3/879: chown d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/db5/d106/d9b 141284 1 2024-03-24T10:46:04.252 INFO:tasks.workunit.client.0.smithi094.stdout:1/856: dwrite d4/d9/f2c [4194304,4194304] 0 2024-03-24T10:46:04.265 INFO:tasks.workunit.client.0.smithi094.stdout:3/880: unlink d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/c1f 0 2024-03-24T10:46:04.268 INFO:tasks.workunit.client.0.smithi094.stdout:1/857: mkdir d4/d9/de/d3d/d122 0 2024-03-24T10:46:04.268 INFO:tasks.workunit.client.0.smithi094.stdout:0/925: dwrite d14/d22/d28/d2b/f38 [0,4194304] 0 2024-03-24T10:46:04.275 INFO:tasks.workunit.client.0.smithi094.stdout:0/926: dread d14/d22/d32/d65/fe9 [0,4194304] 0 2024-03-24T10:46:04.279 INFO:tasks.workunit.client.0.smithi094.stdout:3/881: link d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/fb3 d0/db/dc/d28/d3c/d55/d7f/d20/d23/d8b/f131 0 2024-03-24T10:46:04.280 INFO:tasks.workunit.client.0.smithi094.stdout:3/882: stat d0/db/dc/d28/d3c/d55/f70 0 2024-03-24T10:46:04.280 INFO:tasks.workunit.client.0.smithi094.stdout:1/858: mknod d4/d9/d37/d33/d58/d6b/c123 0 2024-03-24T10:46:04.280 INFO:tasks.workunit.client.0.smithi094.stdout:1/859: stat d4/d9/de/d3d/d122 0 2024-03-24T10:46:04.284 INFO:tasks.workunit.client.0.smithi094.stdout:3/883: link d0/db/dc/d28/cfc d0/db/dc/d28/d50/d5e/c132 0 2024-03-24T10:46:04.285 INFO:tasks.workunit.client.0.smithi094.stdout:1/860: rename d4/d9/de/d6f/f4e to d4/d9/db/d85/f124 0 2024-03-24T10:46:04.286 INFO:tasks.workunit.client.0.smithi094.stdout:3/884: mkdir d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/de9/d133 0 2024-03-24T10:46:04.287 INFO:tasks.workunit.client.0.smithi094.stdout:1/861: symlink d4/d9/db1/l125 0 2024-03-24T10:46:04.287 INFO:tasks.workunit.client.0.smithi094.stdout:1/862: fdatasync d4/d9/f28 0 2024-03-24T10:46:04.288 INFO:tasks.workunit.client.0.smithi094.stdout:3/885: creat d0/db/dc/d28/d110/d111/f134 x:0 0 0 2024-03-24T10:46:04.288 INFO:tasks.workunit.client.0.smithi094.stdout:0/927: dwrite d14/d22/d28/da2/fb9 [0,4194304] 0 2024-03-24T10:46:04.289 INFO:tasks.workunit.client.0.smithi094.stdout:0/928: creat d14/d22/d28/d2b/dc0/de7/f145 x:0 0 0 2024-03-24T10:46:04.291 INFO:tasks.workunit.client.0.smithi094.stdout:0/929: dread d14/d22/d32/d65/fcc [0,4194304] 0 2024-03-24T10:46:04.292 INFO:tasks.workunit.client.0.smithi094.stdout:0/930: mkdir d14/d22/d28/d2b/d68/d142/d146 0 2024-03-24T10:46:04.292 INFO:tasks.workunit.client.0.smithi094.stdout:1/863: dread d4/d9/de/d6f/d62/d99/f9e [0,4194304] 0 2024-03-24T10:46:04.293 INFO:tasks.workunit.client.0.smithi094.stdout:0/931: mkdir d14/d22/d28/d2b/d41/d53/d147 0 2024-03-24T10:46:04.294 INFO:tasks.workunit.client.0.smithi094.stdout:1/864: creat d4/d9/de/d3d/d66/f126 x:0 0 0 2024-03-24T10:46:04.294 INFO:tasks.workunit.client.0.smithi094.stdout:1/865: write d4/d9/db/d39/f100 [376438,106498] 0 2024-03-24T10:46:04.294 INFO:tasks.workunit.client.0.smithi094.stdout:6/934: sync 2024-03-24T10:46:04.294 INFO:tasks.workunit.client.0.smithi094.stdout:5/954: sync 2024-03-24T10:46:04.294 INFO:tasks.workunit.client.0.smithi094.stdout:5/955: chown d7/d23/d5f/d14/d6f/l88 30 1 2024-03-24T10:46:04.295 INFO:tasks.workunit.client.0.smithi094.stdout:6/935: symlink d0/d21/d2a/df3/l12d 0 2024-03-24T10:46:04.295 INFO:tasks.workunit.client.0.smithi094.stdout:5/956: creat d7/d23/d5f/d14/d9f/f13a x:0 0 0 2024-03-24T10:46:04.296 INFO:tasks.workunit.client.0.smithi094.stdout:5/957: rmdir d7/d23/d5f/d14/d36/d3a/d57 39 2024-03-24T10:46:04.298 INFO:tasks.workunit.client.0.smithi094.stdout:5/958: unlink d7/d23/d5f/d14/d9f/ca3 0 2024-03-24T10:46:04.304 INFO:tasks.workunit.client.0.smithi094.stdout:6/936: write d0/d21/d2a/d55/f93 [2398878,75148] 0 2024-03-24T10:46:04.306 INFO:tasks.workunit.client.0.smithi094.stdout:6/937: mknod d0/d78/da/d23/c12e 0 2024-03-24T10:46:04.306 INFO:tasks.workunit.client.0.smithi094.stdout:6/938: fsync d0/d21/f4f 0 2024-03-24T10:46:04.307 INFO:tasks.workunit.client.0.smithi094.stdout:1/866: read d4/d9/de/d6f/d62/fbb [2545135,7200] 0 2024-03-24T10:46:04.307 INFO:tasks.workunit.client.0.smithi094.stdout:1/867: stat d4/d9/de/d3d/d66/dbe/fde 0 2024-03-24T10:46:04.307 INFO:tasks.workunit.client.0.smithi094.stdout:1/868: readlink d4/d9/db1/l125 0 2024-03-24T10:46:04.307 INFO:tasks.workunit.client.0.smithi094.stdout:6/939: mkdir d0/d21/d2a/dcb/d12f 0 2024-03-24T10:46:04.307 INFO:tasks.workunit.client.0.smithi094.stdout:0/932: write d14/d18/f1d [23071,20949] 0 2024-03-24T10:46:04.307 INFO:tasks.workunit.client.0.smithi094.stdout:3/886: dwrite d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/f12a [0,4194304] 0 2024-03-24T10:46:04.312 INFO:tasks.workunit.client.0.smithi094.stdout:3/887: mkdir d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/db5/d135 0 2024-03-24T10:46:04.317 INFO:tasks.workunit.client.0.smithi094.stdout:3/888: creat d0/db/dc/dd9/f136 x:0 0 0 2024-03-24T10:46:04.317 INFO:tasks.workunit.client.0.smithi094.stdout:3/889: truncate d0/db/dc/d28/d50/fe6 1010837 0 2024-03-24T10:46:04.317 INFO:tasks.workunit.client.0.smithi094.stdout:3/890: chown d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/ff 5242 1 2024-03-24T10:46:04.317 INFO:tasks.workunit.client.0.smithi094.stdout:3/891: write d0/db/dc/d28/d38/dc9/f107 [176282,92923] 0 2024-03-24T10:46:04.317 INFO:tasks.workunit.client.0.smithi094.stdout:3/892: dread - d0/db/dc/d28/d3c/d7c/fef zero size 2024-03-24T10:46:04.319 INFO:tasks.workunit.client.0.smithi094.stdout:1/869: dread d4/d9/db/f18 [0,4194304] 0 2024-03-24T10:46:04.320 INFO:tasks.workunit.client.0.smithi094.stdout:3/893: rename d0/db/dc/d28/d3c/d7c/de8/cf7 to d0/db/dc/d28/d3c/d55/d7f/d20/c137 0 2024-03-24T10:46:04.323 INFO:tasks.workunit.client.0.smithi094.stdout:3/894: unlink d0/db/dc/d28/d3c/d55/d7f/d20/d23/d8b/f131 0 2024-03-24T10:46:04.324 INFO:tasks.workunit.client.0.smithi094.stdout:3/895: mknod d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/c138 0 2024-03-24T10:46:04.330 INFO:tasks.workunit.client.0.smithi094.stdout:1/870: dread d4/d9/de/d6f/d62/d84/fe6 [0,4194304] 0 2024-03-24T10:46:04.333 INFO:tasks.workunit.client.0.smithi094.stdout:7/912: sync 2024-03-24T10:46:04.333 INFO:tasks.workunit.client.0.smithi094.stdout:7/913: write d1/d9/dc3/fed [790751,8027] 0 2024-03-24T10:46:04.333 INFO:tasks.workunit.client.0.smithi094.stdout:7/914: read - d1/d9/d16/d21/d49/dfa/f129 zero size 2024-03-24T10:46:04.335 INFO:tasks.workunit.client.0.smithi094.stdout:7/915: rename d1/d9/d16/d21/d51/d83/d39/d44/dd9/l17 to d1/d6c/d80/d81/l134 0 2024-03-24T10:46:04.335 INFO:tasks.workunit.client.0.smithi094.stdout:7/916: chown d1/d6c/d80/cad 527128508 1 2024-03-24T10:46:04.335 INFO:tasks.workunit.client.0.smithi094.stdout:7/917: truncate d1/db4/d65/f106 6512356 0 2024-03-24T10:46:04.335 INFO:tasks.workunit.client.0.smithi094.stdout:7/918: fdatasync d1/d6c/f88 0 2024-03-24T10:46:04.336 INFO:tasks.workunit.client.0.smithi094.stdout:7/919: creat d1/d9/d16/d21/d49/f135 x:0 0 0 2024-03-24T10:46:04.337 INFO:tasks.workunit.client.0.smithi094.stdout:1/871: write d4/d9/db/d5d/fc3 [3177174,44218] 0 2024-03-24T10:46:04.337 INFO:tasks.workunit.client.0.smithi094.stdout:1/872: chown d4/d9/de/d6f/d62/d80 1 1 2024-03-24T10:46:04.337 INFO:tasks.workunit.client.0.smithi094.stdout:6/940: dwrite d0/f8 [0,4194304] 0 2024-03-24T10:46:04.338 INFO:tasks.workunit.client.0.smithi094.stdout:5/959: dwrite d7/dc/fed [0,4194304] 0 2024-03-24T10:46:04.338 INFO:tasks.workunit.client.0.smithi094.stdout:0/933: dwrite d14/d22/d28/d2b/f45 [0,4194304] 0 2024-03-24T10:46:04.341 INFO:tasks.workunit.client.0.smithi094.stdout:6/941: mknod d0/d21/d2a/df3/c130 0 2024-03-24T10:46:04.341 INFO:tasks.workunit.client.0.smithi094.stdout:6/942: write d0/d21/d2a/dcb/f11a [392181,24379] 0 2024-03-24T10:46:04.341 INFO:tasks.workunit.client.0.smithi094.stdout:6/943: getdents d0/d30/d114/d128 0 2024-03-24T10:46:04.347 INFO:tasks.workunit.client.0.smithi094.stdout:5/960: rename d7/d23/dd5/da9/f12b to d7/dc/d100/d109/d113/f13b 0 2024-03-24T10:46:04.350 INFO:tasks.workunit.client.0.smithi094.stdout:1/873: write d4/d9/fd [901154,42207] 0 2024-03-24T10:46:04.352 INFO:tasks.workunit.client.0.smithi094.stdout:7/920: write d1/db4/d65/f106 [3198052,96338] 0 2024-03-24T10:46:04.356 INFO:tasks.workunit.client.0.smithi094.stdout:1/874: creat d4/d9/d37/dcf/f127 x:0 0 0 2024-03-24T10:46:04.357 INFO:tasks.workunit.client.0.smithi094.stdout:7/921: creat d1/d9/d96/f136 x:0 0 0 2024-03-24T10:46:04.359 INFO:tasks.workunit.client.0.smithi094.stdout:6/944: dwrite d0/da9/fd5 [0,4194304] 0 2024-03-24T10:46:04.359 INFO:tasks.workunit.client.0.smithi094.stdout:6/945: creat d0/d78/da/d23/d7c/db0/f131 x:0 0 0 2024-03-24T10:46:04.359 INFO:tasks.workunit.client.0.smithi094.stdout:6/946: stat d0/d78/da/d23/d34/d4e/cb1 0 2024-03-24T10:46:04.359 INFO:tasks.workunit.client.0.smithi094.stdout:7/922: rename d1/d9/d16/d21/d51/d83/d9d/dd7/d123 to d1/d9/d16/d21/d51/d137 0 2024-03-24T10:46:04.365 INFO:tasks.workunit.client.0.smithi094.stdout:6/947: creat d0/d30/d50/f132 x:0 0 0 2024-03-24T10:46:04.367 INFO:tasks.workunit.client.0.smithi094.stdout:7/923: rename d1/d9/d16/d21/d51/d83/ffb to d1/d9/d16/d21/d51/d83/d9d/dd7/f138 0 2024-03-24T10:46:04.369 INFO:tasks.workunit.client.0.smithi094.stdout:6/948: mknod d0/d78/da/d23/dc3/c133 0 2024-03-24T10:46:04.369 INFO:tasks.workunit.client.0.smithi094.stdout:6/949: truncate d0/d30/f76 1245139 0 2024-03-24T10:46:04.369 INFO:tasks.workunit.client.0.smithi094.stdout:6/950: dread - d0/d30/d50/d4d/ffa zero size 2024-03-24T10:46:04.369 INFO:tasks.workunit.client.0.smithi094.stdout:6/951: truncate d0/d78/da/d23/d7c/db0/fff 804493 0 2024-03-24T10:46:04.369 INFO:tasks.workunit.client.0.smithi094.stdout:6/952: readlink d0/d30/d50/d4d/lea 0 2024-03-24T10:46:04.369 INFO:tasks.workunit.client.0.smithi094.stdout:1/875: write d4/d9/de/f2b [289878,9917] 0 2024-03-24T10:46:04.369 INFO:tasks.workunit.client.0.smithi094.stdout:1/876: truncate d4/d9/f2c 9141809 0 2024-03-24T10:46:04.370 INFO:tasks.workunit.client.0.smithi094.stdout:1/877: chown d4/d9/db/d31/d3c/c74 5 1 2024-03-24T10:46:04.394 INFO:tasks.workunit.client.0.smithi094.stdout:7/924: dwrite d1/d9/d25/d3d/f46 [4194304,4194304] 0 2024-03-24T10:46:04.394 INFO:tasks.workunit.client.0.smithi094.stdout:6/953: dwrite d0/d78/da/d23/d7c/fd9 [0,4194304] 0 2024-03-24T10:46:04.413 INFO:tasks.workunit.client.0.smithi094.stdout:7/925: getdents d1/db4/d61 0 2024-03-24T10:46:04.414 INFO:tasks.workunit.client.0.smithi094.stdout:7/926: symlink d1/d6c/l139 0 2024-03-24T10:46:04.414 INFO:tasks.workunit.client.0.smithi094.stdout:7/927: readlink d1/d9/d16/d21/d51/d83/d39/d44/lef 0 2024-03-24T10:46:04.415 INFO:tasks.workunit.client.0.smithi094.stdout:6/954: read d0/d21/f5c [1240924,81750] 0 2024-03-24T10:46:04.415 INFO:tasks.workunit.client.0.smithi094.stdout:6/955: fsync d0/d30/d50/f81 0 2024-03-24T10:46:04.415 INFO:tasks.workunit.client.0.smithi094.stdout:6/956: truncate d0/d1a/f109 977962 0 2024-03-24T10:46:04.442 INFO:tasks.workunit.client.0.smithi094.stdout:7/928: dwrite d1/d9/d16/d21/d51/d83/d39/d44/f56 [0,4194304] 0 2024-03-24T10:46:04.443 INFO:tasks.workunit.client.0.smithi094.stdout:6/957: dwrite d0/d21/d2a/dcb/d110/f129 [0,4194304] 0 2024-03-24T10:46:04.444 INFO:tasks.workunit.client.0.smithi094.stdout:7/929: mkdir d1/d9/d16/d21/d51/d83/d39/d44/d13a 0 2024-03-24T10:46:04.447 INFO:tasks.workunit.client.0.smithi094.stdout:7/930: unlink d1/d6c/d80/d81/l8d 0 2024-03-24T10:46:04.447 INFO:tasks.workunit.client.0.smithi094.stdout:7/931: getdents d1/d9/d25/d3d/d112/d133 0 2024-03-24T10:46:04.449 INFO:tasks.workunit.client.0.smithi094.stdout:7/932: dread d1/d6c/fee [0,4194304] 0 2024-03-24T10:46:04.451 INFO:tasks.workunit.client.0.smithi094.stdout:5/961: sync 2024-03-24T10:46:04.451 INFO:tasks.workunit.client.0.smithi094.stdout:3/896: sync 2024-03-24T10:46:04.451 INFO:tasks.workunit.client.0.smithi094.stdout:0/934: sync 2024-03-24T10:46:04.451 INFO:tasks.workunit.client.0.smithi094.stdout:1/878: sync 2024-03-24T10:46:04.451 INFO:tasks.workunit.client.0.smithi094.stdout:1/879: fsync d4/d9/db/d5d/f64 0 2024-03-24T10:46:04.451 INFO:tasks.workunit.client.0.smithi094.stdout:3/897: write d0/db/dc/d28/d38/dc9/f107 [1023370,32016] 0 2024-03-24T10:46:04.451 INFO:tasks.workunit.client.0.smithi094.stdout:3/898: write d0/db/dc/d28/d3c/d55/d7f/d20/d23/f3f [733526,15495] 0 2024-03-24T10:46:04.451 INFO:tasks.workunit.client.0.smithi094.stdout:5/962: truncate d7/d23/f44 3208739 0 2024-03-24T10:46:04.452 INFO:tasks.workunit.client.0.smithi094.stdout:1/880: link d4/d9/db1/l125 d4/d9/d11b/l128 0 2024-03-24T10:46:04.453 INFO:tasks.workunit.client.0.smithi094.stdout:6/958: dwrite d0/d1a/d35/f100 [0,4194304] 0 2024-03-24T10:46:04.453 INFO:tasks.workunit.client.0.smithi094.stdout:3/899: mkdir d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/db5/d135/d139 0 2024-03-24T10:46:04.455 INFO:tasks.workunit.client.0.smithi094.stdout:5/963: rename d7/d23/d5f/d14/d36/d3a to d7/d23/dd5/da9/d13c 0 2024-03-24T10:46:04.457 INFO:tasks.workunit.client.0.smithi094.stdout:1/881: link d4/d9/db/d39/la2 d4/d9/d37/d33/d76/l129 0 2024-03-24T10:46:04.457 INFO:tasks.workunit.client.0.smithi094.stdout:1/882: read d4/d9/de/d6f/d62/fbb [1816627,39873] 0 2024-03-24T10:46:04.457 INFO:tasks.workunit.client.0.smithi094.stdout:0/935: dread d14/f62 [4194304,4194304] 0 2024-03-24T10:46:04.460 INFO:tasks.workunit.client.0.smithi094.stdout:6/959: mkdir d0/d78/d66/d7a/d134 0 2024-03-24T10:46:04.461 INFO:tasks.workunit.client.0.smithi094.stdout:3/900: dread d0/db/dc/d28/d3c/d55/d7f/d20/d23/f3f [0,4194304] 0 2024-03-24T10:46:04.465 INFO:tasks.workunit.client.0.smithi094.stdout:1/883: creat d4/f12a x:0 0 0 2024-03-24T10:46:04.466 INFO:tasks.workunit.client.0.smithi094.stdout:0/936: symlink d14/d22/d28/d2b/d68/d120/l148 0 2024-03-24T10:46:04.466 INFO:tasks.workunit.client.0.smithi094.stdout:0/937: readlink d14/l20 0 2024-03-24T10:46:04.467 INFO:tasks.workunit.client.0.smithi094.stdout:6/960: symlink d0/d1a/d35/l135 0 2024-03-24T10:46:04.467 INFO:tasks.workunit.client.0.smithi094.stdout:6/961: chown d0/da9/fcd 93505777 1 2024-03-24T10:46:04.468 INFO:tasks.workunit.client.0.smithi094.stdout:3/901: creat d0/db/dc/d28/d50/f13a x:0 0 0 2024-03-24T10:46:04.469 INFO:tasks.workunit.client.0.smithi094.stdout:5/964: truncate d7/dc/d56/f105 3141101 0 2024-03-24T10:46:04.470 INFO:tasks.workunit.client.0.smithi094.stdout:1/884: creat d4/d9/d11b/f12b x:0 0 0 2024-03-24T10:46:04.474 INFO:tasks.workunit.client.0.smithi094.stdout:0/938: truncate d14/d18/f1f 3723176 0 2024-03-24T10:46:04.474 INFO:tasks.workunit.client.0.smithi094.stdout:3/902: unlink d0/db/dc/d28/l3e 0 2024-03-24T10:46:04.474 INFO:tasks.workunit.client.0.smithi094.stdout:5/965: rmdir d7/dc/d100/d109 39 2024-03-24T10:46:04.475 INFO:tasks.workunit.client.0.smithi094.stdout:5/966: truncate d7/d23/d5f/d9/dde/ff5 676403 0 2024-03-24T10:46:04.475 INFO:tasks.workunit.client.0.smithi094.stdout:5/967: chown d7/c6c 6 1 2024-03-24T10:46:04.475 INFO:tasks.workunit.client.0.smithi094.stdout:1/885: creat d4/d9/db/d5d/d120/f12c x:0 0 0 2024-03-24T10:46:04.476 INFO:tasks.workunit.client.0.smithi094.stdout:6/962: dread d0/d78/da/d23/dc3/f6e [0,4194304] 0 2024-03-24T10:46:04.476 INFO:tasks.workunit.client.0.smithi094.stdout:6/963: chown d0/d30/f76 2055547 1 2024-03-24T10:46:04.477 INFO:tasks.workunit.client.0.smithi094.stdout:3/903: creat d0/db/dc/d28/d38/d12e/f13b x:0 0 0 2024-03-24T10:46:04.478 INFO:tasks.workunit.client.0.smithi094.stdout:3/904: write d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/f93 [4150404,131010] 0 2024-03-24T10:46:04.478 INFO:tasks.workunit.client.0.smithi094.stdout:3/905: read - d0/db/dc/d28/d3c/d55/d7f/ffa zero size 2024-03-24T10:46:04.481 INFO:tasks.workunit.client.0.smithi094.stdout:6/964: mkdir d0/d78/da/d23/dde/d136 0 2024-03-24T10:46:04.481 INFO:tasks.workunit.client.0.smithi094.stdout:3/906: creat d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/f13c x:0 0 0 2024-03-24T10:46:04.481 INFO:tasks.workunit.client.0.smithi094.stdout:3/907: fsync d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/f124 0 2024-03-24T10:46:04.482 INFO:tasks.workunit.client.0.smithi094.stdout:3/908: readlink d0/db/dc/d28/d3c/d55/d7f/d20/d23/l68 0 2024-03-24T10:46:04.482 INFO:tasks.workunit.client.0.smithi094.stdout:3/909: write d0/db/dc/d28/f117 [1004295,90921] 0 2024-03-24T10:46:04.482 INFO:tasks.workunit.client.0.smithi094.stdout:3/910: fdatasync d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d87/f9c 0 2024-03-24T10:46:04.492 INFO:tasks.workunit.client.0.smithi094.stdout:0/939: dwrite d14/d22/d32/d65/fcc [0,4194304] 0 2024-03-24T10:46:04.498 INFO:tasks.workunit.client.0.smithi094.stdout:0/940: mknod d14/d22/d28/d48/da5/c149 0 2024-03-24T10:46:04.500 INFO:tasks.workunit.client.0.smithi094.stdout:0/941: creat d14/d22/d28/d2b/dc0/f14a x:0 0 0 2024-03-24T10:46:04.503 INFO:tasks.workunit.client.0.smithi094.stdout:1/886: dwrite d4/d9/db/d31/d3c/fd7 [0,4194304] 0 2024-03-24T10:46:04.510 INFO:tasks.workunit.client.0.smithi094.stdout:5/968: dwrite d7/d23/d2b/d68/d81/fb3 [0,4194304] 0 2024-03-24T10:46:04.511 INFO:tasks.workunit.client.0.smithi094.stdout:3/911: dwrite d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d34/fe5 [0,4194304] 0 2024-03-24T10:46:04.511 INFO:tasks.workunit.client.0.smithi094.stdout:6/965: dwrite d0/d78/da/d23/d83/fe9 [8388608,4194304] 0 2024-03-24T10:46:04.513 INFO:tasks.workunit.client.0.smithi094.stdout:5/969: symlink d7/d23/d132/l13d 0 2024-03-24T10:46:04.514 INFO:tasks.workunit.client.0.smithi094.stdout:1/887: write d4/d36/f114 [111125,16085] 0 2024-03-24T10:46:04.514 INFO:tasks.workunit.client.0.smithi094.stdout:1/888: chown d4/d9/db/d85/df5/cfc 15847 1 2024-03-24T10:46:04.517 INFO:tasks.workunit.client.0.smithi094.stdout:3/912: symlink d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/de9/l13d 0 2024-03-24T10:46:04.518 INFO:tasks.workunit.client.0.smithi094.stdout:6/966: link d0/d78/da/d23/dc3/le7 d0/d78/da/d23/dc3/l137 0 2024-03-24T10:46:04.523 INFO:tasks.workunit.client.0.smithi094.stdout:3/913: truncate d0/db/dc/d28/d3c/d7c/fb0 3839822 0 2024-03-24T10:46:04.525 INFO:tasks.workunit.client.0.smithi094.stdout:3/914: symlink d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d34/d71/l13e 0 2024-03-24T10:46:04.525 INFO:tasks.workunit.client.0.smithi094.stdout:3/915: write d0/db/dc/d28/d3c/d55/d7f/d20/d23/f3f [959343,129051] 0 2024-03-24T10:46:04.525 INFO:tasks.workunit.client.0.smithi094.stdout:5/970: dread d7/d23/dd5/d64/f7d [0,4194304] 0 2024-03-24T10:46:04.527 INFO:tasks.workunit.client.0.smithi094.stdout:3/916: truncate d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/fd1 2046685 0 2024-03-24T10:46:04.529 INFO:tasks.workunit.client.0.smithi094.stdout:3/917: rmdir d0/db/dc/d28/d3c/d55/d7f/d20/d23/d128 0 2024-03-24T10:46:04.529 INFO:tasks.workunit.client.0.smithi094.stdout:3/918: stat d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/c4d 0 2024-03-24T10:46:04.536 INFO:tasks.workunit.client.1.smithi096.stdout:5/13: sync 2024-03-24T10:46:04.536 INFO:tasks.workunit.client.1.smithi096.stdout:5/14: rename d2 to d2/d4 22 2024-03-24T10:46:04.537 INFO:tasks.workunit.client.1.smithi096.stdout:5/15: unlink c0 0 2024-03-24T10:46:04.541 INFO:tasks.workunit.client.0.smithi094.stdout:3/919: dread d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/f12a [0,4194304] 0 2024-03-24T10:46:04.542 INFO:tasks.workunit.client.0.smithi094.stdout:3/920: truncate d0/db/dc/d28/d3c/d55/d7f/f122 227088 0 2024-03-24T10:46:04.542 INFO:tasks.workunit.client.0.smithi094.stdout:3/921: mknod d0/db/dc3/dea/c13f 0 2024-03-24T10:46:04.543 INFO:tasks.workunit.client.0.smithi094.stdout:3/922: creat d0/db/dc/dd9/d102/f140 x:0 0 0 2024-03-24T10:46:04.544 INFO:tasks.workunit.client.0.smithi094.stdout:3/923: truncate d0/db/dc/d28/d3c/d7c/fef 276755 0 2024-03-24T10:46:04.548 INFO:tasks.workunit.client.0.smithi094.stdout:1/889: dwrite d4/d9/de/f2b [0,4194304] 0 2024-03-24T10:46:04.548 INFO:tasks.workunit.client.0.smithi094.stdout:6/967: dwrite d0/f8 [0,4194304] 0 2024-03-24T10:46:04.548 INFO:tasks.workunit.client.0.smithi094.stdout:5/971: dwrite d7/dc/fed [4194304,4194304] 0 2024-03-24T10:46:04.550 INFO:tasks.workunit.client.0.smithi094.stdout:6/968: mknod d0/d30/d114/c138 0 2024-03-24T10:46:04.552 INFO:tasks.workunit.client.0.smithi094.stdout:6/969: mkdir d0/d78/d66/d139 0 2024-03-24T10:46:04.552 INFO:tasks.workunit.client.0.smithi094.stdout:6/970: chown d0/f3c 1 1 2024-03-24T10:46:04.555 INFO:tasks.workunit.client.0.smithi094.stdout:6/971: unlink d0/d78/da/d23/d34/l45 0 2024-03-24T10:46:04.555 INFO:tasks.workunit.client.0.smithi094.stdout:1/890: write d4/d9/d37/d33/d76/f9b [372524,90220] 0 2024-03-24T10:46:04.555 INFO:tasks.workunit.client.0.smithi094.stdout:1/891: chown d4/d9/de/d6f/d5f/lb2 2054576 1 2024-03-24T10:46:04.555 INFO:tasks.workunit.client.0.smithi094.stdout:1/892: dread - d4/d9/db/d39/d43/f11a zero size 2024-03-24T10:46:04.556 INFO:tasks.workunit.client.1.smithi096.stdout:5/16: read f1 [4007,32210] 0 2024-03-24T10:46:04.556 INFO:tasks.workunit.client.0.smithi094.stdout:6/972: symlink d0/d30/d114/l13a 0 2024-03-24T10:46:04.556 INFO:tasks.workunit.client.0.smithi094.stdout:6/973: fsync d0/f8 0 2024-03-24T10:46:04.559 INFO:tasks.workunit.client.0.smithi094.stdout:6/974: link d0/d21/d2a/ldb d0/d30/d50/d4d/l13b 0 2024-03-24T10:46:04.560 INFO:tasks.workunit.client.0.smithi094.stdout:1/893: rmdir d4/d9/db/d5d/d63 39 2024-03-24T10:46:04.560 INFO:tasks.workunit.client.1.smithi096.stdout:5/17: rmdir d2 39 2024-03-24T10:46:04.561 INFO:tasks.workunit.client.0.smithi094.stdout:6/975: mkdir d0/d30/d13c 0 2024-03-24T10:46:04.562 INFO:tasks.workunit.client.0.smithi094.stdout:6/976: fdatasync d0/d1a/f6f 0 2024-03-24T10:46:04.563 INFO:tasks.workunit.client.1.smithi096.stdout:5/18: link f1 d2/f5 0 2024-03-24T10:46:04.563 INFO:tasks.workunit.client.1.smithi096.stdout:5/19: write f1 [1734518,36556] 0 2024-03-24T10:46:04.565 INFO:tasks.workunit.client.0.smithi094.stdout:6/977: unlink d0/da9/l117 0 2024-03-24T10:46:04.567 INFO:tasks.workunit.client.0.smithi094.stdout:5/972: read d7/d23/d5f/d14/d6f/f94 [3960477,117096] 0 2024-03-24T10:46:04.567 INFO:tasks.workunit.client.0.smithi094.stdout:5/973: dread - d7/d23/f121 zero size 2024-03-24T10:46:04.567 INFO:tasks.workunit.client.0.smithi094.stdout:3/924: dwrite d0/db/dc/d28/d3c/d55/d7f/d20/d23/f29 [0,4194304] 0 2024-03-24T10:46:04.567 INFO:tasks.workunit.client.0.smithi094.stdout:5/974: write d7/d23/d5f/d9/f49 [2439938,59300] 0 2024-03-24T10:46:04.567 INFO:tasks.workunit.client.0.smithi094.stdout:5/975: creat d7/d23/d5f/d14/d6f/dbe/f13e x:0 0 0 2024-03-24T10:46:04.571 INFO:tasks.workunit.client.1.smithi096.stdout:5/20: rename f1 to d2/f6 0 2024-03-24T10:46:04.576 INFO:tasks.workunit.client.0.smithi094.stdout:3/925: rmdir d0/db/dc/d28/d110 39 2024-03-24T10:46:04.577 INFO:tasks.workunit.client.0.smithi094.stdout:6/978: dread d0/d1a/fd7 [0,4194304] 0 2024-03-24T10:46:04.577 INFO:tasks.workunit.client.0.smithi094.stdout:6/979: creat d0/d30/d50/d4d/d6b/f13d x:0 0 0 2024-03-24T10:46:04.577 INFO:tasks.workunit.client.0.smithi094.stdout:6/980: fsync d0/da9/fcd 0 2024-03-24T10:46:04.580 INFO:tasks.workunit.client.0.smithi094.stdout:5/976: mkdir d7/d23/d5f/d9/dde/d13f 0 2024-03-24T10:46:04.582 INFO:tasks.workunit.client.1.smithi096.stdout:5/21: getdents d2 0 2024-03-24T10:46:04.584 INFO:tasks.workunit.client.0.smithi094.stdout:3/926: write d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/db5/d106/d9b/d10d/f89 [3282421,15479] 0 2024-03-24T10:46:04.585 INFO:tasks.workunit.client.0.smithi094.stdout:1/894: dwrite d4/d9/d37/d33/d58/d6b/fe4 [0,4194304] 0 2024-03-24T10:46:04.585 INFO:tasks.workunit.client.0.smithi094.stdout:1/895: fdatasync d4/d9/de/d6f/d62/d84/de9/f112 0 2024-03-24T10:46:04.585 INFO:tasks.workunit.client.1.smithi096.stdout:5/22: mkdir d2/d7 0 2024-03-24T10:46:04.585 INFO:tasks.workunit.client.0.smithi094.stdout:1/896: dread d4/d9/db1/db9/fea [0,4194304] 0 2024-03-24T10:46:04.587 INFO:tasks.workunit.client.0.smithi094.stdout:5/977: link d7/d23/d2b/l45 d7/d23/dd5/da9/d13c/l140 0 2024-03-24T10:46:04.587 INFO:tasks.workunit.client.0.smithi094.stdout:5/978: stat d7/d23/d5f/d9/d48/d51/l9c 0 2024-03-24T10:46:04.587 INFO:tasks.workunit.client.0.smithi094.stdout:5/979: write d7/d23/d5f/d9/d48/fa7 [819892,6765] 0 2024-03-24T10:46:04.587 INFO:tasks.workunit.client.0.smithi094.stdout:0/942: sync 2024-03-24T10:46:04.588 INFO:tasks.workunit.client.0.smithi094.stdout:7/933: sync 2024-03-24T10:46:04.588 INFO:tasks.workunit.client.0.smithi094.stdout:7/934: readlink d1/d9/d16/d21/d51/d83/d39/d44/dd9/l4a 0 2024-03-24T10:46:04.588 INFO:tasks.workunit.client.1.smithi096.stdout:5/23: mkdir d2/d7/d8 0 2024-03-24T10:46:04.592 INFO:tasks.workunit.client.1.smithi096.stdout:8/49: sync 2024-03-24T10:46:04.593 INFO:tasks.workunit.client.0.smithi094.stdout:3/927: truncate d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/d73/fb1 1360032 0 2024-03-24T10:46:04.593 INFO:tasks.workunit.client.0.smithi094.stdout:3/928: write d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/db5/fc6 [554031,5200] 0 2024-03-24T10:46:04.596 INFO:tasks.workunit.client.0.smithi094.stdout:1/897: unlink d4/d9/db/lc5 0 2024-03-24T10:46:04.596 INFO:tasks.workunit.client.0.smithi094.stdout:1/898: readlink d4/d9/db/d85/df5/l11e 0 2024-03-24T10:46:04.597 INFO:tasks.workunit.client.0.smithi094.stdout:5/980: mknod d7/d23/dd5/da9/d13c/c141 0 2024-03-24T10:46:04.600 INFO:tasks.workunit.client.0.smithi094.stdout:0/943: mknod d14/d22/d28/d48/d7b/dfa/d102/c14b 0 2024-03-24T10:46:04.601 INFO:tasks.workunit.client.0.smithi094.stdout:6/981: dwrite d0/d30/f76 [0,4194304] 0 2024-03-24T10:46:04.603 INFO:tasks.workunit.client.0.smithi094.stdout:3/929: rename d0/f14 to d0/db/dc/d28/d38/f141 0 2024-03-24T10:46:04.604 INFO:tasks.workunit.client.0.smithi094.stdout:1/899: link d4/d9/l17 d4/d9/db/d39/d75/l12d 0 2024-03-24T10:46:04.605 INFO:tasks.workunit.client.0.smithi094.stdout:7/935: dwrite d1/d9/d16/f116 [0,4194304] 0 2024-03-24T10:46:04.608 INFO:tasks.workunit.client.0.smithi094.stdout:0/944: mknod d14/d22/d28/dac/ddd/c14c 0 2024-03-24T10:46:04.608 INFO:tasks.workunit.client.0.smithi094.stdout:0/945: readlink d14/d22/d32/l5f 0 2024-03-24T10:46:04.609 INFO:tasks.workunit.client.0.smithi094.stdout:6/982: dread d0/d78/da/d23/f9b [0,4194304] 0 2024-03-24T10:46:04.609 INFO:tasks.workunit.client.0.smithi094.stdout:3/930: getdents d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/db5/d106/d9b/d10d 0 2024-03-24T10:46:04.611 INFO:tasks.workunit.client.0.smithi094.stdout:1/900: dread d4/d9/de/d6f/d62/d84/fe6 [0,4194304] 0 2024-03-24T10:46:04.611 INFO:tasks.workunit.client.0.smithi094.stdout:1/901: creat d4/d9/de/d6f/d62/d99/f12e x:0 0 0 2024-03-24T10:46:04.619 INFO:tasks.workunit.client.0.smithi094.stdout:3/931: unlink d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d4f/cad 0 2024-03-24T10:46:04.621 INFO:tasks.workunit.client.0.smithi094.stdout:1/902: mkdir d4/d9/d12f 0 2024-03-24T10:46:04.621 INFO:tasks.workunit.client.0.smithi094.stdout:6/983: write d0/d1a/d35/d96/fd0 [1248983,104430] 0 2024-03-24T10:46:04.622 INFO:tasks.workunit.client.0.smithi094.stdout:3/932: mkdir d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d34/d71/d142 0 2024-03-24T10:46:04.623 INFO:tasks.workunit.client.0.smithi094.stdout:3/933: creat d0/db/dc/d28/d38/dc9/f143 x:0 0 0 2024-03-24T10:46:04.623 INFO:tasks.workunit.client.0.smithi094.stdout:3/934: truncate d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/f124 925207 0 2024-03-24T10:46:04.627 INFO:tasks.workunit.client.0.smithi094.stdout:3/935: link d0/f11 d0/db/f144 0 2024-03-24T10:46:04.627 INFO:tasks.workunit.client.0.smithi094.stdout:3/936: write d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/d73/f80 [614799,46266] 0 2024-03-24T10:46:04.627 INFO:tasks.workunit.client.0.smithi094.stdout:3/937: fdatasync d0/db/dc/d28/d3c/f83 0 2024-03-24T10:46:04.627 INFO:tasks.workunit.client.0.smithi094.stdout:3/938: fsync d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/db5/d106/f82 0 2024-03-24T10:46:04.627 INFO:tasks.workunit.client.0.smithi094.stdout:3/939: write d0/db/dc/d28/d50/fe6 [1724697,54845] 0 2024-03-24T10:46:04.627 INFO:tasks.workunit.client.0.smithi094.stdout:3/940: dread - d0/db/dc/d28/d3c/d55/d7f/d20/f11d zero size 2024-03-24T10:46:04.627 INFO:tasks.workunit.client.0.smithi094.stdout:3/941: readlink d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/l5d 0 2024-03-24T10:46:04.627 INFO:tasks.workunit.client.0.smithi094.stdout:1/903: dread d4/d9/db/d85/f124 [0,4194304] 0 2024-03-24T10:46:04.629 INFO:tasks.workunit.client.0.smithi094.stdout:3/942: dread d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/db5/d106/d9b/fa1 [0,4194304] 0 2024-03-24T10:46:04.630 INFO:tasks.workunit.client.0.smithi094.stdout:6/984: rename d0/d30/d50 to d0/d78/da/d23/d7c/d13e 0 2024-03-24T10:46:04.631 INFO:tasks.workunit.client.0.smithi094.stdout:3/943: mkdir d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d145 0 2024-03-24T10:46:04.631 INFO:tasks.workunit.client.0.smithi094.stdout:6/985: symlink d0/da9/l13f 0 2024-03-24T10:46:04.632 INFO:tasks.workunit.client.0.smithi094.stdout:6/986: chown d0/d78/da/d23/d34/d4e 20712857 1 2024-03-24T10:46:04.632 INFO:tasks.workunit.client.0.smithi094.stdout:6/987: chown d0/d78/d66/ld8 0 1 2024-03-24T10:46:04.632 INFO:tasks.workunit.client.0.smithi094.stdout:6/988: write d0/d1a/f109 [1826185,89968] 0 2024-03-24T10:46:04.634 INFO:tasks.workunit.client.0.smithi094.stdout:5/981: dwrite d7/d23/d2b/d68/d81/f11d [0,4194304] 0 2024-03-24T10:46:04.634 INFO:tasks.workunit.client.0.smithi094.stdout:1/904: write d4/d9/db/f10 [79468,45193] 0 2024-03-24T10:46:04.638 INFO:tasks.workunit.client.0.smithi094.stdout:1/905: creat d4/d9/de/d3d/d122/f130 x:0 0 0 2024-03-24T10:46:04.639 INFO:tasks.workunit.client.1.smithi096.stdout:5/24: dwrite d2/f5 [0,4194304] 0 2024-03-24T10:46:04.639 INFO:tasks.workunit.client.1.smithi096.stdout:5/25: stat d2/d7 0 2024-03-24T10:46:04.639 INFO:tasks.workunit.client.1.smithi096.stdout:8/50: dwrite f0 [0,4194304] 0 2024-03-24T10:46:04.639 INFO:tasks.workunit.client.1.smithi096.stdout:8/51: chown d8/ld 1 1 2024-03-24T10:46:04.640 INFO:tasks.workunit.client.0.smithi094.stdout:1/906: symlink d4/d9/db/d31/l131 0 2024-03-24T10:46:04.640 INFO:tasks.workunit.client.0.smithi094.stdout:1/907: readlink d4/d9/db/d39/d43/l79 0 2024-03-24T10:46:04.640 INFO:tasks.workunit.client.0.smithi094.stdout:1/908: creat d4/d9/db/d85/f132 x:0 0 0 2024-03-24T10:46:04.640 INFO:tasks.workunit.client.0.smithi094.stdout:0/946: dwrite d14/d22/d28/d2b/dc0/f14a [0,4194304] 0 2024-03-24T10:46:04.641 INFO:tasks.workunit.client.0.smithi094.stdout:7/936: dwrite d1/d9/d25/f50 [0,4194304] 0 2024-03-24T10:46:04.642 INFO:tasks.workunit.client.0.smithi094.stdout:0/947: dread d14/d22/d7d/fa6 [0,4194304] 0 2024-03-24T10:46:04.642 INFO:tasks.workunit.client.0.smithi094.stdout:0/948: write d14/d18/f4f [4668491,90287] 0 2024-03-24T10:46:04.643 INFO:tasks.workunit.client.0.smithi094.stdout:0/949: readlink d14/d22/d28/d2b/l6d 0 2024-03-24T10:46:04.652 INFO:tasks.workunit.client.1.smithi096.stdout:5/26: symlink d2/l9 0 2024-03-24T10:46:04.652 INFO:tasks.workunit.client.1.smithi096.stdout:5/27: readlink d2/l9 0 2024-03-24T10:46:04.656 INFO:tasks.workunit.client.0.smithi094.stdout:5/982: dwrite d7/d23/dd5/d64/dd6/f3e [0,4194304] 0 2024-03-24T10:46:04.657 INFO:tasks.workunit.client.0.smithi094.stdout:1/909: dwrite d4/d9/de/d6f/d62/d84/de9/f112 [0,4194304] 0 2024-03-24T10:46:04.658 INFO:tasks.workunit.client.1.smithi096.stdout:5/28: link d2/f6 d2/fa 0 2024-03-24T10:46:04.658 INFO:tasks.workunit.client.1.smithi096.stdout:5/29: truncate d2/f6 4509678 0 2024-03-24T10:46:04.659 INFO:tasks.workunit.client.0.smithi094.stdout:7/937: getdents d1/d9/d25/d3d/d112 0 2024-03-24T10:46:04.659 INFO:tasks.workunit.client.0.smithi094.stdout:7/938: chown d1/d9/f3c 417434 1 2024-03-24T10:46:04.660 INFO:tasks.workunit.client.1.smithi096.stdout:5/30: symlink d2/d7/d8/lb 0 2024-03-24T10:46:04.664 INFO:tasks.workunit.client.0.smithi094.stdout:0/950: creat d14/d22/d28/da2/f14d x:0 0 0 2024-03-24T10:46:04.664 INFO:tasks.workunit.client.0.smithi094.stdout:0/951: chown d14/d22/d28/d2b/f140 17 1 2024-03-24T10:46:04.665 INFO:tasks.workunit.client.0.smithi094.stdout:5/983: mknod d7/d23/dd5/d135/c142 0 2024-03-24T10:46:04.669 INFO:tasks.workunit.client.0.smithi094.stdout:1/910: symlink d4/d36/l133 0 2024-03-24T10:46:04.671 INFO:tasks.workunit.client.1.smithi096.stdout:4/65: sync 2024-03-24T10:46:04.671 INFO:tasks.workunit.client.1.smithi096.stdout:3/126: sync 2024-03-24T10:46:04.671 INFO:tasks.workunit.client.1.smithi096.stdout:2/61: sync 2024-03-24T10:46:04.671 INFO:tasks.workunit.client.1.smithi096.stdout:7/66: sync 2024-03-24T10:46:04.671 INFO:tasks.workunit.client.1.smithi096.stdout:9/37: sync 2024-03-24T10:46:04.671 INFO:tasks.workunit.client.1.smithi096.stdout:0/41: sync 2024-03-24T10:46:04.671 INFO:tasks.workunit.client.1.smithi096.stdout:1/38: sync 2024-03-24T10:46:04.671 INFO:tasks.workunit.client.1.smithi096.stdout:6/78: sync 2024-03-24T10:46:04.671 INFO:tasks.workunit.client.0.smithi094.stdout:7/939: rename d1/d9/d25/f50 to d1/d6c/d80/d81/f13b 0 2024-03-24T10:46:04.672 INFO:tasks.workunit.client.0.smithi094.stdout:0/952: creat d14/d22/d28/f14e x:0 0 0 2024-03-24T10:46:04.672 INFO:tasks.workunit.client.0.smithi094.stdout:0/953: dread - d14/d18/f134 zero size 2024-03-24T10:46:04.675 INFO:tasks.workunit.client.1.smithi096.stdout:8/52: dwrite f1 [4194304,4194304] 0 2024-03-24T10:46:04.676 INFO:tasks.workunit.client.1.smithi096.stdout:3/127: rename d0/ff to d0/d13/d1e/f2c 0 2024-03-24T10:46:04.676 INFO:tasks.workunit.client.1.smithi096.stdout:3/128: creat d0/d13/d1e/f2d x:0 0 0 2024-03-24T10:46:04.676 INFO:tasks.workunit.client.1.smithi096.stdout:3/129: readlink d0/d9/le 0 2024-03-24T10:46:04.682 INFO:tasks.workunit.client.1.smithi096.stdout:9/38: rename d9/ca to d9/cd 0 2024-03-24T10:46:04.682 INFO:tasks.workunit.client.1.smithi096.stdout:5/31: getdents d2/d7/d8 0 2024-03-24T10:46:04.685 INFO:tasks.workunit.client.1.smithi096.stdout:4/66: dread d1/f3 [0,4194304] 0 2024-03-24T10:46:04.685 INFO:tasks.workunit.client.1.smithi096.stdout:7/67: dread d0/d13/f9 [0,4194304] 0 2024-03-24T10:46:04.685 INFO:tasks.workunit.client.1.smithi096.stdout:7/68: creat d0/d13/f18 x:0 0 0 2024-03-24T10:46:04.686 INFO:tasks.workunit.client.0.smithi094.stdout:1/911: rename d4/d9/db/d39/f100 to d4/d9/d37/f134 0 2024-03-24T10:46:04.687 INFO:tasks.workunit.client.1.smithi096.stdout:1/39: truncate d3/f9 167819 0 2024-03-24T10:46:04.689 INFO:tasks.workunit.client.0.smithi094.stdout:7/940: truncate d1/d9/d16/d21/d51/d83/d39/d44/dd9/d34/fa3 2705749 0 2024-03-24T10:46:04.691 INFO:tasks.workunit.client.0.smithi094.stdout:0/954: unlink d14/d22/d28/da2/l12f 0 2024-03-24T10:46:04.692 INFO:tasks.workunit.client.1.smithi096.stdout:2/62: write d5/f6 [1927624,95683] 0 2024-03-24T10:46:04.694 INFO:tasks.workunit.client.1.smithi096.stdout:9/39: unlink f5 0 2024-03-24T10:46:04.694 INFO:tasks.workunit.client.1.smithi096.stdout:9/40: chown d9 59731751 1 2024-03-24T10:46:04.696 INFO:tasks.workunit.client.1.smithi096.stdout:5/32: link d2/d7/d8/lb d2/d7/d8/lc 0 2024-03-24T10:46:04.696 INFO:tasks.workunit.client.1.smithi096.stdout:4/67: readlink d1/l7 0 2024-03-24T10:46:04.698 INFO:tasks.workunit.client.1.smithi096.stdout:7/69: creat d0/db/f19 x:0 0 0 2024-03-24T10:46:04.701 INFO:tasks.workunit.client.1.smithi096.stdout:2/63: symlink d5/dc/d12/l1b 0 2024-03-24T10:46:04.702 INFO:tasks.workunit.client.1.smithi096.stdout:6/79: dread d8/f14 [0,4194304] 0 2024-03-24T10:46:04.703 INFO:tasks.workunit.client.1.smithi096.stdout:9/41: link d9/fc d9/db/fe 0 2024-03-24T10:46:04.703 INFO:tasks.workunit.client.1.smithi096.stdout:9/42: rename d9/db to d9/db/df 22 2024-03-24T10:46:04.703 INFO:tasks.workunit.client.1.smithi096.stdout:9/43: truncate d9/db/fe 2348641 0 2024-03-24T10:46:04.703 INFO:tasks.workunit.client.1.smithi096.stdout:9/44: write d9/db/fe [42177,101122] 0 2024-03-24T10:46:04.704 INFO:tasks.workunit.client.1.smithi096.stdout:9/45: read d9/db/fe [1595236,27666] 0 2024-03-24T10:46:04.704 INFO:tasks.workunit.client.1.smithi096.stdout:4/68: symlink d1/d11/l16 0 2024-03-24T10:46:04.706 INFO:tasks.workunit.client.1.smithi096.stdout:3/130: dwrite d0/f4 [0,4194304] 0 2024-03-24T10:46:04.706 INFO:tasks.workunit.client.1.smithi096.stdout:3/131: dread - d0/d9/d22/f25 zero size 2024-03-24T10:46:04.706 INFO:tasks.workunit.client.1.smithi096.stdout:3/132: chown d0/d13/f1f 33103121 1 2024-03-24T10:46:04.706 INFO:tasks.workunit.client.1.smithi096.stdout:6/80: truncate d8/f1c 487504 0 2024-03-24T10:46:04.706 INFO:tasks.workunit.client.1.smithi096.stdout:6/81: fdatasync f7 0 2024-03-24T10:46:04.708 INFO:tasks.workunit.client.1.smithi096.stdout:9/46: unlink d9/fc 0 2024-03-24T10:46:04.710 INFO:tasks.workunit.client.1.smithi096.stdout:4/69: unlink d1/l9 0 2024-03-24T10:46:04.712 INFO:tasks.workunit.client.1.smithi096.stdout:3/133: creat d0/d9/f2e x:0 0 0 2024-03-24T10:46:04.712 INFO:tasks.workunit.client.1.smithi096.stdout:3/134: chown d0/f4 14955653 1 2024-03-24T10:46:04.712 INFO:tasks.workunit.client.1.smithi096.stdout:3/135: truncate d0/d9/f2e 701981 0 2024-03-24T10:46:04.712 INFO:tasks.workunit.client.1.smithi096.stdout:3/136: stat d0/f1 0 2024-03-24T10:46:04.712 INFO:tasks.workunit.client.1.smithi096.stdout:5/33: write d2/fa [1193697,67033] 0 2024-03-24T10:46:04.712 INFO:tasks.workunit.client.1.smithi096.stdout:0/42: dwrite d0/d2/fb [0,4194304] 0 2024-03-24T10:46:04.712 INFO:tasks.workunit.client.1.smithi096.stdout:6/82: mknod d8/d10/d19/d1e/c20 0 2024-03-24T10:46:04.713 INFO:tasks.workunit.client.1.smithi096.stdout:7/70: dread d0/db/ff [0,4194304] 0 2024-03-24T10:46:04.713 INFO:tasks.workunit.client.1.smithi096.stdout:8/53: dwrite d8/ff [0,4194304] 0 2024-03-24T10:46:04.715 INFO:tasks.workunit.client.1.smithi096.stdout:8/54: dread d8/fe [0,4194304] 0 2024-03-24T10:46:04.715 INFO:tasks.workunit.client.1.smithi096.stdout:8/55: creat d8/f10 x:0 0 0 2024-03-24T10:46:04.715 INFO:tasks.workunit.client.1.smithi096.stdout:8/56: chown d8/f10 1643399 1 2024-03-24T10:46:04.715 INFO:tasks.workunit.client.1.smithi096.stdout:9/47: read d9/db/fe [1696555,81626] 0 2024-03-24T10:46:04.715 INFO:tasks.workunit.client.1.smithi096.stdout:9/48: chown d9/cd 0 1 2024-03-24T10:46:04.716 INFO:tasks.workunit.client.1.smithi096.stdout:9/49: write d9/db/fe [3385902,5655] 0 2024-03-24T10:46:04.716 INFO:tasks.workunit.client.1.smithi096.stdout:9/50: chown d9/db 3 1 2024-03-24T10:46:04.716 INFO:tasks.workunit.client.1.smithi096.stdout:8/57: write d8/f10 [548517,106213] 0 2024-03-24T10:46:04.716 INFO:tasks.workunit.client.1.smithi096.stdout:8/58: write d8/f10 [1043431,122511] 0 2024-03-24T10:46:04.721 INFO:tasks.workunit.client.1.smithi096.stdout:5/34: dread - d2/f3 zero size 2024-03-24T10:46:04.723 INFO:tasks.workunit.client.1.smithi096.stdout:0/43: mknod d0/cc 0 2024-03-24T10:46:04.724 INFO:tasks.workunit.client.1.smithi096.stdout:2/64: dread d5/f6 [0,4194304] 0 2024-03-24T10:46:04.725 INFO:tasks.workunit.client.1.smithi096.stdout:7/71: rename d0/db/f19 to d0/d13/f1a 0 2024-03-24T10:46:04.726 INFO:tasks.workunit.client.0.smithi094.stdout:7/941: dwrite d1/d9/d16/d21/d51/d83/d39/fcd [0,4194304] 0 2024-03-24T10:46:04.728 INFO:tasks.workunit.client.1.smithi096.stdout:4/70: mknod d1/c17 0 2024-03-24T10:46:04.729 INFO:tasks.workunit.client.0.smithi094.stdout:0/955: dwrite d14/d18/f25 [4194304,4194304] 0 2024-03-24T10:46:04.730 INFO:tasks.workunit.client.1.smithi096.stdout:5/35: creat d2/d7/fd x:0 0 0 2024-03-24T10:46:04.732 INFO:tasks.workunit.client.1.smithi096.stdout:2/65: creat d5/dc/f1c x:0 0 0 2024-03-24T10:46:04.732 INFO:tasks.workunit.client.1.smithi096.stdout:6/83: dwrite d8/f14 [0,4194304] 0 2024-03-24T10:46:04.736 INFO:tasks.workunit.client.1.smithi096.stdout:0/44: read d0/d2/fb [151878,84541] 0 2024-03-24T10:46:04.736 INFO:tasks.workunit.client.1.smithi096.stdout:0/45: fdatasync d0/d2/fb 0 2024-03-24T10:46:04.736 INFO:tasks.workunit.client.1.smithi096.stdout:1/40: dwrite d3/f5 [0,4194304] 0 2024-03-24T10:46:04.738 INFO:tasks.workunit.client.1.smithi096.stdout:7/72: creat d0/db/f1b x:0 0 0 2024-03-24T10:46:04.738 INFO:tasks.workunit.client.1.smithi096.stdout:1/41: dread d3/f9 [0,4194304] 0 2024-03-24T10:46:04.738 INFO:tasks.workunit.client.1.smithi096.stdout:1/42: dread - d3/fb zero size 2024-03-24T10:46:04.738 INFO:tasks.workunit.client.1.smithi096.stdout:1/43: fsync d3/f5 0 2024-03-24T10:46:04.744 INFO:tasks.workunit.client.1.smithi096.stdout:4/71: creat d1/f18 x:0 0 0 2024-03-24T10:46:04.744 INFO:tasks.workunit.client.1.smithi096.stdout:4/72: creat d1/d11/f19 x:0 0 0 2024-03-24T10:46:04.745 INFO:tasks.workunit.client.1.smithi096.stdout:5/36: rmdir d2 39 2024-03-24T10:46:04.745 INFO:tasks.workunit.client.1.smithi096.stdout:5/37: fsync d2/f3 0 2024-03-24T10:46:04.746 INFO:tasks.workunit.client.1.smithi096.stdout:2/66: rename d5/de to d5/de/d1d 22 2024-03-24T10:46:04.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:04 smithi094 ceph-mon[29446]: pgmap v245: 65 pgs: 65 active+clean; 1.7 GiB data, 6.6 GiB used, 530 GiB / 536 GiB avail; 206 MiB/s rd, 243 MiB/s wr, 413 op/s 2024-03-24T10:46:04.748 INFO:tasks.workunit.client.0.smithi094.stdout:0/956: creat d14/f14f x:0 0 0 2024-03-24T10:46:04.749 INFO:tasks.workunit.client.1.smithi096.stdout:6/84: creat d8/d10/d19/d1e/f21 x:0 0 0 2024-03-24T10:46:04.750 INFO:tasks.workunit.client.1.smithi096.stdout:6/85: fdatasync d8/f1c 0 2024-03-24T10:46:04.753 INFO:tasks.workunit.client.0.smithi094.stdout:0/957: mkdir d14/d22/d28/da2/d10a/d150 0 2024-03-24T10:46:04.755 INFO:tasks.workunit.client.1.smithi096.stdout:3/137: rmdir d0/d9 39 2024-03-24T10:46:04.755 INFO:tasks.workunit.client.1.smithi096.stdout:3/138: creat d0/d13/d1e/f2f x:0 0 0 2024-03-24T10:46:04.755 INFO:tasks.workunit.client.1.smithi096.stdout:3/139: dread - d0/d9/d22/f25 zero size 2024-03-24T10:46:04.760 INFO:tasks.workunit.client.1.smithi096.stdout:0/46: link d0/c1 d0/cd 0 2024-03-24T10:46:04.762 INFO:tasks.workunit.client.1.smithi096.stdout:0/47: dread d0/f5 [0,4194304] 0 2024-03-24T10:46:04.762 INFO:tasks.workunit.client.1.smithi096.stdout:1/44: creat d3/fc x:0 0 0 2024-03-24T10:46:04.763 INFO:tasks.workunit.client.1.smithi096.stdout:8/59: truncate d8/ff 1566145 0 2024-03-24T10:46:04.768 INFO:tasks.workunit.client.1.smithi096.stdout:6/86: read f6 [147109,13338] 0 2024-03-24T10:46:04.768 INFO:tasks.workunit.client.1.smithi096.stdout:9/51: dwrite d9/db/fe [0,4194304] 0 2024-03-24T10:46:04.769 INFO:tasks.workunit.client.0.smithi094.stdout:7/942: dwrite d1/db4/f101 [0,4194304] 0 2024-03-24T10:46:04.772 INFO:tasks.workunit.client.0.smithi094.stdout:3/944: sync 2024-03-24T10:46:04.773 INFO:tasks.workunit.client.1.smithi096.stdout:3/140: mkdir d0/d13/d1e/d30 0 2024-03-24T10:46:04.776 INFO:tasks.workunit.client.1.smithi096.stdout:5/38: dwrite d2/d7/fd [0,4194304] 0 2024-03-24T10:46:04.776 INFO:tasks.workunit.client.1.smithi096.stdout:5/39: readlink d2/l9 0 2024-03-24T10:46:04.776 INFO:tasks.workunit.client.1.smithi096.stdout:7/73: mkdir d0/db/d1c 0 2024-03-24T10:46:04.777 INFO:tasks.workunit.client.0.smithi094.stdout:5/984: sync 2024-03-24T10:46:04.777 INFO:tasks.workunit.client.0.smithi094.stdout:6/989: sync 2024-03-24T10:46:04.777 INFO:tasks.workunit.client.0.smithi094.stdout:5/985: stat d7/d23/dd5/da9/d13c/fe0 0 2024-03-24T10:46:04.777 INFO:tasks.workunit.client.0.smithi094.stdout:5/986: readlink d7/d23/d2b/l38 0 2024-03-24T10:46:04.778 INFO:tasks.workunit.client.1.smithi096.stdout:1/45: chown d3/c4 22049856 1 2024-03-24T10:46:04.778 INFO:tasks.workunit.client.1.smithi096.stdout:1/46: write d3/fc [156219,6441] 0 2024-03-24T10:46:04.779 INFO:tasks.workunit.client.0.smithi094.stdout:0/958: dwrite d14/d22/d28/d2b/f45 [0,4194304] 0 2024-03-24T10:46:04.780 INFO:tasks.workunit.client.0.smithi094.stdout:3/945: dread d0/f42 [0,4194304] 0 2024-03-24T10:46:04.781 INFO:tasks.workunit.client.1.smithi096.stdout:2/67: dread f3 [0,4194304] 0 2024-03-24T10:46:04.783 INFO:tasks.workunit.client.0.smithi094.stdout:6/990: rename d0/d1a/c73 to d0/d78/da/dc9/c140 0 2024-03-24T10:46:04.785 INFO:tasks.workunit.client.0.smithi094.stdout:1/912: sync 2024-03-24T10:46:04.787 INFO:tasks.workunit.client.0.smithi094.stdout:1/913: dread d4/d9/de/d6f/fa5 [0,4194304] 0 2024-03-24T10:46:04.788 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:04 smithi096 ceph-mon[38097]: pgmap v245: 65 pgs: 65 active+clean; 1.7 GiB data, 6.6 GiB used, 530 GiB / 536 GiB avail; 206 MiB/s rd, 243 MiB/s wr, 413 op/s 2024-03-24T10:46:04.788 INFO:tasks.workunit.client.0.smithi094.stdout:3/946: stat d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/c39 0 2024-03-24T10:46:04.793 INFO:tasks.workunit.client.0.smithi094.stdout:1/914: mknod d4/d9/db/d31/d3c/c135 0 2024-03-24T10:46:04.797 INFO:tasks.workunit.client.0.smithi094.stdout:3/947: dread d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/de9/f112 [0,4194304] 0 2024-03-24T10:46:04.798 INFO:tasks.workunit.client.1.smithi096.stdout:3/141: mknod d0/c31 0 2024-03-24T10:46:04.798 INFO:tasks.workunit.client.1.smithi096.stdout:3/142: chown d0/d9/c1c 24431072 1 2024-03-24T10:46:04.800 INFO:tasks.workunit.client.0.smithi094.stdout:1/915: unlink d4/d9/db/d5d/d120/f12c 0 2024-03-24T10:46:04.801 INFO:tasks.workunit.client.1.smithi096.stdout:4/73: write d1/f3 [4350316,16610] 0 2024-03-24T10:46:04.805 INFO:tasks.workunit.client.1.smithi096.stdout:7/74: link d0/d13/ld d0/l1d 0 2024-03-24T10:46:04.807 INFO:tasks.workunit.client.0.smithi094.stdout:3/948: rename d0/db/dc/d28/d3c/d7c to d0/db/dc/dd9/d146 0 2024-03-24T10:46:04.809 INFO:tasks.workunit.client.1.smithi096.stdout:2/68: truncate d5/f9 73893 0 2024-03-24T10:46:04.811 INFO:tasks.workunit.client.0.smithi094.stdout:3/949: creat d0/db/f147 x:0 0 0 2024-03-24T10:46:04.812 INFO:tasks.workunit.client.1.smithi096.stdout:8/60: rmdir d8 39 2024-03-24T10:46:04.815 INFO:tasks.workunit.client.0.smithi094.stdout:3/950: creat d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/db5/d106/d9b/dca/f148 x:0 0 0 2024-03-24T10:46:04.815 INFO:tasks.workunit.client.0.smithi094.stdout:3/951: write d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d87/f9c [742189,75737] 0 2024-03-24T10:46:04.815 INFO:tasks.workunit.client.0.smithi094.stdout:3/952: creat d0/db/dc3/dea/f149 x:0 0 0 2024-03-24T10:46:04.815 INFO:tasks.workunit.client.0.smithi094.stdout:3/953: dread - d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/dfb/f12b zero size 2024-03-24T10:46:04.815 INFO:tasks.workunit.client.0.smithi094.stdout:3/954: chown d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/db5/d106 1487 1 2024-03-24T10:46:04.816 INFO:tasks.workunit.client.1.smithi096.stdout:3/143: creat d0/d9/f32 x:0 0 0 2024-03-24T10:46:04.816 INFO:tasks.workunit.client.1.smithi096.stdout:3/144: write d0/d9/f2e [1474108,53263] 0 2024-03-24T10:46:04.816 INFO:tasks.workunit.client.1.smithi096.stdout:3/145: write d0/d13/d1e/f2d [391530,25341] 0 2024-03-24T10:46:04.819 INFO:tasks.workunit.client.1.smithi096.stdout:8/61: write f1 [5896616,111951] 0 2024-03-24T10:46:04.826 INFO:tasks.workunit.client.0.smithi094.stdout:0/959: dwrite d14/d22/d28/d48/d7b/f8e [0,4194304] 0 2024-03-24T10:46:04.826 INFO:tasks.workunit.client.0.smithi094.stdout:0/960: dread - d14/d18/fc1 zero size 2024-03-24T10:46:04.826 INFO:tasks.workunit.client.1.smithi096.stdout:5/40: symlink d2/le 0 2024-03-24T10:46:04.835 INFO:tasks.workunit.client.0.smithi094.stdout:3/955: rename d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/c4d to d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/db5/d106/d9b/d10d/c14a 0 2024-03-24T10:46:04.839 INFO:tasks.workunit.client.0.smithi094.stdout:0/961: creat d14/d22/d28/dac/ddd/f151 x:0 0 0 2024-03-24T10:46:04.847 INFO:tasks.workunit.client.1.smithi096.stdout:0/48: dwrite d0/f5 [0,4194304] 0 2024-03-24T10:46:04.847 INFO:tasks.workunit.client.1.smithi096.stdout:7/75: unlink d0/d13/lc 0 2024-03-24T10:46:04.847 INFO:tasks.workunit.client.1.smithi096.stdout:7/76: getdents d0/db/d1c 0 2024-03-24T10:46:04.848 INFO:tasks.workunit.client.1.smithi096.stdout:1/47: symlink d3/ld 0 2024-03-24T10:46:04.848 INFO:tasks.workunit.client.1.smithi096.stdout:2/69: mknod d5/c1e 0 2024-03-24T10:46:04.848 INFO:tasks.workunit.client.1.smithi096.stdout:6/87: rename d8/d10/d19/d1e to d8/d10/d22 0 2024-03-24T10:46:04.848 INFO:tasks.workunit.client.1.smithi096.stdout:3/146: symlink d0/d9/d22/l33 0 2024-03-24T10:46:04.848 INFO:tasks.workunit.client.1.smithi096.stdout:1/48: truncate d3/f8 1271954 0 2024-03-24T10:46:04.848 INFO:tasks.workunit.client.1.smithi096.stdout:1/49: chown d3/c4 170 1 2024-03-24T10:46:04.848 INFO:tasks.workunit.client.1.smithi096.stdout:4/74: rename d1/cb to d1/d11/c1a 0 2024-03-24T10:46:04.848 INFO:tasks.workunit.client.1.smithi096.stdout:3/147: mkdir d0/d9/d34 0 2024-03-24T10:46:04.848 INFO:tasks.workunit.client.1.smithi096.stdout:3/148: truncate d0/d13/f1f 68198 0 2024-03-24T10:46:04.849 INFO:tasks.workunit.client.0.smithi094.stdout:7/943: dwrite d1/d9/d16/f2f [0,4194304] 0 2024-03-24T10:46:04.849 INFO:tasks.workunit.client.0.smithi094.stdout:7/944: chown d1/d9/f23 29892110 1 2024-03-24T10:46:04.849 INFO:tasks.workunit.client.1.smithi096.stdout:5/41: mkdir d2/d7/d8/df 0 2024-03-24T10:46:04.849 INFO:tasks.workunit.client.1.smithi096.stdout:5/42: fdatasync d2/f6 0 2024-03-24T10:46:04.849 INFO:tasks.workunit.client.1.smithi096.stdout:5/43: write d2/fa [684883,85336] 0 2024-03-24T10:46:04.849 INFO:tasks.workunit.client.1.smithi096.stdout:5/44: creat d2/d7/f10 x:0 0 0 2024-03-24T10:46:04.850 INFO:tasks.workunit.client.1.smithi096.stdout:6/88: dread f6 [0,4194304] 0 2024-03-24T10:46:04.850 INFO:tasks.workunit.client.0.smithi094.stdout:7/945: symlink d1/d9/d16/d21/d51/d83/d39/d44/dd9/d34/dc8/l13c 0 2024-03-24T10:46:04.854 INFO:tasks.workunit.client.1.smithi096.stdout:0/49: dread d0/d2/fb [0,4194304] 0 2024-03-24T10:46:04.855 INFO:tasks.workunit.client.1.smithi096.stdout:7/77: dread d0/db/ff [0,4194304] 0 2024-03-24T10:46:04.858 INFO:tasks.workunit.client.0.smithi094.stdout:7/946: write d1/d9/fd1 [6642479,43735] 0 2024-03-24T10:46:04.858 INFO:tasks.workunit.client.0.smithi094.stdout:7/947: truncate d1/d9/d16/d21/d49/f135 528480 0 2024-03-24T10:46:04.859 INFO:tasks.workunit.client.0.smithi094.stdout:6/991: dwrite d0/d30/f76 [0,4194304] 0 2024-03-24T10:46:04.859 INFO:tasks.workunit.client.1.smithi096.stdout:1/50: link d3/la d3/le 0 2024-03-24T10:46:04.860 INFO:tasks.workunit.client.0.smithi094.stdout:6/992: write d0/d78/da/d23/f68 [3778787,48968] 0 2024-03-24T10:46:04.861 INFO:tasks.workunit.client.1.smithi096.stdout:6/89: rename d8/d10/d13/l15 to d8/d12/l23 0 2024-03-24T10:46:04.863 INFO:tasks.workunit.client.1.smithi096.stdout:1/51: creat d3/ff x:0 0 0 2024-03-24T10:46:04.864 INFO:tasks.workunit.client.0.smithi094.stdout:6/993: rename d0/d30/de6/d9c/f11f to d0/d21/d2a/dcb/d12f/f141 0 2024-03-24T10:46:04.864 INFO:tasks.workunit.client.0.smithi094.stdout:6/994: fsync d0/d78/da/d23/d7c/d13e/d4d/d6b/f13d 0 2024-03-24T10:46:04.865 INFO:tasks.workunit.client.1.smithi096.stdout:0/50: dread d0/d2/fb [0,4194304] 0 2024-03-24T10:46:04.865 INFO:tasks.workunit.client.1.smithi096.stdout:6/90: mknod d8/d10/d13/c24 0 2024-03-24T10:46:04.868 INFO:tasks.workunit.client.0.smithi094.stdout:7/948: write d1/d9/d16/d21/d51/d83/f72 [2762072,71361] 0 2024-03-24T10:46:04.870 INFO:tasks.workunit.client.0.smithi094.stdout:7/949: mknod d1/db4/c13d 0 2024-03-24T10:46:04.877 INFO:tasks.workunit.client.0.smithi094.stdout:0/962: dwrite d14/d22/d32/d65/fda [0,4194304] 0 2024-03-24T10:46:04.877 INFO:tasks.workunit.client.0.smithi094.stdout:0/963: creat d14/d22/d28/d2b/d68/f152 x:0 0 0 2024-03-24T10:46:04.878 INFO:tasks.workunit.client.0.smithi094.stdout:0/964: chown d14/d22/d7d/fa4 5734555 1 2024-03-24T10:46:04.880 INFO:tasks.workunit.client.0.smithi094.stdout:0/965: creat d14/d22/d28/da2/d10a/f153 x:0 0 0 2024-03-24T10:46:04.881 INFO:tasks.workunit.client.0.smithi094.stdout:0/966: rename d14/d22/d28/d2b/dc0/f14a to d14/d22/d32/f154 0 2024-03-24T10:46:04.883 INFO:tasks.workunit.client.0.smithi094.stdout:0/967: unlink d14/d22/d32/d70/dd0/f8c 0 2024-03-24T10:46:04.884 INFO:tasks.workunit.client.0.smithi094.stdout:0/968: mkdir d14/d22/d28/d48/d7b/dfa/d155 0 2024-03-24T10:46:04.884 INFO:tasks.workunit.client.0.smithi094.stdout:0/969: fsync d14/d18/fde 0 2024-03-24T10:46:04.884 INFO:tasks.workunit.client.0.smithi094.stdout:0/970: write d14/d22/d28/d2b/d41/fe3 [816228,101467] 0 2024-03-24T10:46:04.884 INFO:tasks.workunit.client.0.smithi094.stdout:0/971: dread - d14/d22/d28/d2b/f140 zero size 2024-03-24T10:46:04.885 INFO:tasks.workunit.client.0.smithi094.stdout:0/972: mkdir d14/d22/d28/d48/da5/de0/d13c/d156 0 2024-03-24T10:46:04.885 INFO:tasks.workunit.client.0.smithi094.stdout:0/973: write d14/d22/d7d/fa6 [2304560,73168] 0 2024-03-24T10:46:04.885 INFO:tasks.workunit.client.0.smithi094.stdout:0/974: mkdir d14/d22/d28/d138/d157 0 2024-03-24T10:46:04.891 INFO:tasks.workunit.client.1.smithi096.stdout:6/91: dwrite d8/f1c [0,4194304] 0 2024-03-24T10:46:04.901 INFO:tasks.workunit.client.1.smithi096.stdout:6/92: dwrite d8/d10/d19/f1b [0,4194304] 0 2024-03-24T10:46:04.908 INFO:tasks.workunit.client.1.smithi096.stdout:6/93: dread f7 [0,4194304] 0 2024-03-24T10:46:04.909 INFO:tasks.workunit.client.1.smithi096.stdout:6/94: rename d8/d10/d13/c24 to d8/d10/c25 0 2024-03-24T10:46:05.048 INFO:tasks.workunit.client.1.smithi096.stdout:3/149: dread d0/f4 [0,4194304] 0 2024-03-24T10:46:05.054 INFO:tasks.workunit.client.1.smithi096.stdout:3/150: rename d0/c16 to d0/c35 0 2024-03-24T10:46:05.054 INFO:tasks.workunit.client.1.smithi096.stdout:3/151: rmdir d0/d13/d1e/d30 0 2024-03-24T10:46:05.054 INFO:tasks.workunit.client.1.smithi096.stdout:3/152: creat d0/d9/f36 x:0 0 0 2024-03-24T10:46:05.054 INFO:tasks.workunit.client.1.smithi096.stdout:3/153: write d0/d13/f17 [496875,8612] 0 2024-03-24T10:46:05.054 INFO:tasks.workunit.client.1.smithi096.stdout:3/154: stat d0/d13/d1e/f2d 0 2024-03-24T10:46:05.055 INFO:tasks.workunit.client.0.smithi094.stdout:3/956: dwrite d0/db/dfd/f127 [4194304,4194304] 0 2024-03-24T10:46:05.055 INFO:tasks.workunit.client.0.smithi094.stdout:3/957: getdents d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d4f/d108 0 2024-03-24T10:46:05.055 INFO:tasks.workunit.client.0.smithi094.stdout:3/958: stat d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d87/dc5 0 2024-03-24T10:46:05.056 INFO:tasks.workunit.client.0.smithi094.stdout:3/959: dread d0/db/dc/d28/d50/fe6 [0,4194304] 0 2024-03-24T10:46:05.056 INFO:tasks.workunit.client.0.smithi094.stdout:3/960: write d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d34/fd3 [914179,99504] 0 2024-03-24T10:46:05.056 INFO:tasks.workunit.client.1.smithi096.stdout:7/78: getdents d0/d13 0 2024-03-24T10:46:05.060 INFO:tasks.workunit.client.0.smithi094.stdout:3/961: mknod d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/c14b 0 2024-03-24T10:46:05.060 INFO:tasks.workunit.client.1.smithi096.stdout:3/155: creat d0/d9/d34/f37 x:0 0 0 2024-03-24T10:46:05.063 INFO:tasks.workunit.client.0.smithi094.stdout:3/962: getdents d0/db/dc/dd9/d146/de3 0 2024-03-24T10:46:05.066 INFO:tasks.workunit.client.0.smithi094.stdout:3/963: rename d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/db5/lb8 to d0/db/dc/d28/d38/dc9/l14c 0 2024-03-24T10:46:05.068 INFO:tasks.workunit.client.1.smithi096.stdout:3/156: link d0/d13/c18 d0/d9/c38 0 2024-03-24T10:46:05.069 INFO:tasks.workunit.client.0.smithi094.stdout:3/964: creat d0/db/dc/dd9/d146/de3/f14d x:0 0 0 2024-03-24T10:46:05.071 INFO:tasks.workunit.client.0.smithi094.stdout:3/965: mknod d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d4f/c14e 0 2024-03-24T10:46:05.072 INFO:tasks.workunit.client.0.smithi094.stdout:1/916: dwrite d4/d36/f73 [4194304,4194304] 0 2024-03-24T10:46:05.072 INFO:tasks.workunit.client.0.smithi094.stdout:1/917: write d4/d9/fd [4674332,30097] 0 2024-03-24T10:46:05.074 INFO:tasks.workunit.client.1.smithi096.stdout:9/52: dwrite d9/db/fe [0,4194304] 0 2024-03-24T10:46:05.074 INFO:tasks.workunit.client.1.smithi096.stdout:5/45: dwrite d2/d7/fd [4194304,4194304] 0 2024-03-24T10:46:05.074 INFO:tasks.workunit.client.1.smithi096.stdout:5/46: truncate d2/f6 5231508 0 2024-03-24T10:46:05.078 INFO:tasks.workunit.client.1.smithi096.stdout:2/70: getdents d5 0 2024-03-24T10:46:05.079 INFO:tasks.workunit.client.1.smithi096.stdout:6/95: dwrite f6 [0,4194304] 0 2024-03-24T10:46:05.079 INFO:tasks.workunit.client.1.smithi096.stdout:6/96: write d8/d10/d22/f21 [592944,29113] 0 2024-03-24T10:46:05.080 INFO:tasks.workunit.client.1.smithi096.stdout:0/51: dwrite d0/f5 [0,4194304] 0 2024-03-24T10:46:05.080 INFO:tasks.workunit.client.0.smithi094.stdout:3/966: write d0/f11 [3196447,39185] 0 2024-03-24T10:46:05.080 INFO:tasks.workunit.client.0.smithi094.stdout:3/967: chown d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/fd1 8006 1 2024-03-24T10:46:05.081 INFO:tasks.workunit.client.0.smithi094.stdout:6/995: dwrite d0/d78/d66/d7a/dfe/f12b [0,4194304] 0 2024-03-24T10:46:05.081 INFO:tasks.workunit.client.0.smithi094.stdout:3/968: fdatasync d0/db/dc/d28/d38/dc9/f107 0 2024-03-24T10:46:05.081 INFO:tasks.workunit.client.0.smithi094.stdout:6/996: read d0/d78/d66/d7a/fb8 [840487,93614] 0 2024-03-24T10:46:05.081 INFO:tasks.workunit.client.0.smithi094.stdout:7/950: dwrite d1/d9/d25/d3d/f46 [0,4194304] 0 2024-03-24T10:46:05.081 INFO:tasks.workunit.client.0.smithi094.stdout:7/951: chown d1/d9/d16/d21/d51/d83/d39/d44/lef 2175516 1 2024-03-24T10:46:05.087 INFO:tasks.workunit.client.1.smithi096.stdout:0/52: dread d0/d2/fb [0,4194304] 0 2024-03-24T10:46:05.087 INFO:tasks.workunit.client.1.smithi096.stdout:0/53: creat d0/fe x:0 0 0 2024-03-24T10:46:05.087 INFO:tasks.workunit.client.1.smithi096.stdout:5/47: rmdir d2/d7/d8 39 2024-03-24T10:46:05.090 INFO:tasks.workunit.client.1.smithi096.stdout:2/71: creat d5/f1f x:0 0 0 2024-03-24T10:46:05.091 INFO:tasks.workunit.client.1.smithi096.stdout:4/75: getdents d1/d11 0 2024-03-24T10:46:05.093 INFO:tasks.workunit.client.1.smithi096.stdout:7/79: dwrite d0/db/f1b [0,4194304] 0 2024-03-24T10:46:05.093 INFO:tasks.workunit.client.1.smithi096.stdout:6/97: creat d8/f26 x:0 0 0 2024-03-24T10:46:05.095 INFO:tasks.workunit.client.1.smithi096.stdout:0/54: write d0/f5 [2879648,72981] 0 2024-03-24T10:46:05.096 INFO:tasks.workunit.client.1.smithi096.stdout:0/55: write d0/fe [214286,89157] 0 2024-03-24T10:46:05.096 INFO:tasks.workunit.client.1.smithi096.stdout:0/56: readlink d0/d2/l9 0 2024-03-24T10:46:05.096 INFO:tasks.workunit.client.1.smithi096.stdout:0/57: write d0/f5 [4054858,86279] 0 2024-03-24T10:46:05.096 INFO:tasks.workunit.client.1.smithi096.stdout:0/58: chown d0/c8 2837127 1 2024-03-24T10:46:05.101 INFO:tasks.workunit.client.0.smithi094.stdout:3/969: mkdir d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/db5/d106/d9b/dca/d14f 0 2024-03-24T10:46:05.102 INFO:tasks.workunit.client.1.smithi096.stdout:3/157: dwrite d0/f4 [0,4194304] 0 2024-03-24T10:46:05.102 INFO:tasks.workunit.client.1.smithi096.stdout:3/158: read d0/d13/d1e/f2c [2276622,104836] 0 2024-03-24T10:46:05.102 INFO:tasks.workunit.client.1.smithi096.stdout:9/53: dwrite d9/db/fe [0,4194304] 0 2024-03-24T10:46:05.103 INFO:tasks.workunit.client.0.smithi094.stdout:1/918: dwrite d4/fcd [0,4194304] 0 2024-03-24T10:46:05.107 INFO:tasks.workunit.client.0.smithi094.stdout:7/952: creat d1/d9/d25/dd4/f13e x:0 0 0 2024-03-24T10:46:05.108 INFO:tasks.workunit.client.1.smithi096.stdout:1/52: rmdir d3 39 2024-03-24T10:46:05.115 INFO:tasks.workunit.client.1.smithi096.stdout:0/59: dwrite d0/d2/fb [0,4194304] 0 2024-03-24T10:46:05.115 INFO:tasks.workunit.client.1.smithi096.stdout:5/48: getdents d2/d7/d8 0 2024-03-24T10:46:05.116 INFO:tasks.workunit.client.1.smithi096.stdout:2/72: readlink d5/de/l18 0 2024-03-24T10:46:05.118 INFO:tasks.workunit.client.0.smithi094.stdout:1/919: creat d4/d9/db/f136 x:0 0 0 2024-03-24T10:46:05.118 INFO:tasks.workunit.client.0.smithi094.stdout:1/920: chown d4/d9/db/l34 8146013 1 2024-03-24T10:46:05.122 INFO:tasks.workunit.client.0.smithi094.stdout:1/921: symlink d4/d9/db1/db9/l137 0 2024-03-24T10:46:05.123 INFO:tasks.workunit.client.0.smithi094.stdout:7/953: dwrite d1/d9/d25/d3d/fda [0,4194304] 0 2024-03-24T10:46:05.124 INFO:tasks.workunit.client.0.smithi094.stdout:6/997: rename d0/d78/da/d23/d34/d119 to d0/d78/da/d23/d7c/d13e/d4d/d6b/d142 0 2024-03-24T10:46:05.132 INFO:tasks.workunit.client.0.smithi094.stdout:7/954: truncate d1/d9/f23 1233477 0 2024-03-24T10:46:05.136 INFO:tasks.workunit.client.0.smithi094.stdout:3/970: rename d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/db5/d106/l84 to d0/db/dc/d28/d50/l150 0 2024-03-24T10:46:05.137 INFO:tasks.workunit.client.1.smithi096.stdout:0/60: dwrite d0/d2/fb [0,4194304] 0 2024-03-24T10:46:05.138 INFO:tasks.workunit.client.1.smithi096.stdout:0/61: fsync d0/d2/fb 0 2024-03-24T10:46:05.138 INFO:tasks.workunit.client.1.smithi096.stdout:0/62: stat d0/fe 0 2024-03-24T10:46:05.139 INFO:tasks.workunit.client.0.smithi094.stdout:7/955: link d1/c87 d1/d9/d16/d21/d51/d83/d9d/c13f 0 2024-03-24T10:46:05.142 INFO:tasks.workunit.client.0.smithi094.stdout:7/956: unlink d1/d9/f12 0 2024-03-24T10:46:05.142 INFO:tasks.workunit.client.0.smithi094.stdout:7/957: write d1/d9/fe [61523,22901] 0 2024-03-24T10:46:05.148 INFO:tasks.workunit.client.1.smithi096.stdout:4/76: symlink d1/l1b 0 2024-03-24T10:46:05.149 INFO:tasks.workunit.client.0.smithi094.stdout:1/922: dwrite d4/d9/d37/d33/f51 [0,4194304] 0 2024-03-24T10:46:05.150 INFO:tasks.workunit.client.0.smithi094.stdout:7/958: mkdir d1/d9/d25/d3d/d140 0 2024-03-24T10:46:05.150 INFO:tasks.workunit.client.0.smithi094.stdout:3/971: write d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/f4c [3509530,102449] 0 2024-03-24T10:46:05.151 INFO:tasks.workunit.client.0.smithi094.stdout:6/998: dwrite d0/d1a/f7b [0,4194304] 0 2024-03-24T10:46:05.151 INFO:tasks.workunit.client.0.smithi094.stdout:3/972: dread - d0/db/f147 zero size 2024-03-24T10:46:05.151 INFO:tasks.workunit.client.0.smithi094.stdout:6/999: chown d0/d78/da/d23/d34/d4e/f7e 458 1 2024-03-24T10:46:05.151 INFO:tasks.workunit.client.0.smithi094.stdout:3/973: write d0/db/dc/d28/d3c/d55/d7f/ffa [751341,16499] 0 2024-03-24T10:46:05.151 INFO:tasks.workunit.client.0.smithi094.stdout:3/974: creat d0/db/dc3/f151 x:0 0 0 2024-03-24T10:46:05.151 INFO:tasks.workunit.client.1.smithi096.stdout:9/54: link c6 d9/c10 0 2024-03-24T10:46:05.153 INFO:tasks.workunit.client.0.smithi094.stdout:1/923: rename d4/d9/db/d39/d43/l79 to d4/d9/de/d6f/l138 0 2024-03-24T10:46:05.156 INFO:tasks.workunit.client.1.smithi096.stdout:7/80: truncate d0/d13/f9 1719578 0 2024-03-24T10:46:05.157 INFO:tasks.workunit.client.0.smithi094.stdout:1/924: rmdir d4/d9/de/d3d 39 2024-03-24T10:46:05.157 INFO:tasks.workunit.client.0.smithi094.stdout:1/925: creat d4/d9/d37/d33/d58/d6b/f139 x:0 0 0 2024-03-24T10:46:05.158 INFO:tasks.workunit.client.0.smithi094.stdout:5/987: sync 2024-03-24T10:46:05.158 INFO:tasks.workunit.client.0.smithi094.stdout:5/988: truncate d7/dc/fc1 430358 0 2024-03-24T10:46:05.159 INFO:tasks.workunit.client.0.smithi094.stdout:7/959: rename d1/d9b to d1/d9/d16/d21/d51/d83/d39/d44/dd9/da5/d141 0 2024-03-24T10:46:05.160 INFO:tasks.workunit.client.1.smithi096.stdout:5/49: symlink d2/d7/d8/df/l11 0 2024-03-24T10:46:05.164 INFO:tasks.workunit.client.1.smithi096.stdout:0/63: mknod d0/d2/cf 0 2024-03-24T10:46:05.168 INFO:tasks.workunit.client.0.smithi094.stdout:1/926: unlink d4/d9/de/d6f/d62/fef 0 2024-03-24T10:46:05.171 INFO:tasks.workunit.client.0.smithi094.stdout:5/989: creat d7/d23/d5f/d9/d48/df9/f143 x:0 0 0 2024-03-24T10:46:05.171 INFO:tasks.workunit.client.0.smithi094.stdout:5/990: chown d7/dc/d56/f93 3023 1 2024-03-24T10:46:05.173 INFO:tasks.workunit.client.0.smithi094.stdout:7/960: truncate d1/d9/d16/d21/d51/d83/d39/d44/f8a 2833873 0 2024-03-24T10:46:05.173 INFO:tasks.workunit.client.1.smithi096.stdout:0/64: dread d0/d2/fb [0,4194304] 0 2024-03-24T10:46:05.173 INFO:tasks.workunit.client.1.smithi096.stdout:0/65: fdatasync d0/f5 0 2024-03-24T10:46:05.174 INFO:tasks.workunit.client.1.smithi096.stdout:9/55: mkdir d9/d11 0 2024-03-24T10:46:05.175 INFO:tasks.workunit.client.0.smithi094.stdout:7/961: write d1/d9/dc3/fed [656849,35431] 0 2024-03-24T10:46:05.175 INFO:tasks.workunit.client.0.smithi094.stdout:7/962: chown d1/d9/d16/d21/d51/d83/d39/d44/dd9/ce9 38426554 1 2024-03-24T10:46:05.176 INFO:tasks.workunit.client.1.smithi096.stdout:1/53: mknod d3/c10 0 2024-03-24T10:46:05.178 INFO:tasks.workunit.client.0.smithi094.stdout:5/991: read d7/d23/d5f/d1b/f37 [4644282,45966] 0 2024-03-24T10:46:05.178 INFO:tasks.workunit.client.0.smithi094.stdout:0/975: sync 2024-03-24T10:46:05.179 INFO:tasks.workunit.client.1.smithi096.stdout:0/66: write d0/d2/fb [3121865,52113] 0 2024-03-24T10:46:05.180 INFO:tasks.workunit.client.1.smithi096.stdout:5/50: creat d2/d7/d8/f12 x:0 0 0 2024-03-24T10:46:05.180 INFO:tasks.workunit.client.1.smithi096.stdout:0/67: dread d0/fe [0,4194304] 0 2024-03-24T10:46:05.180 INFO:tasks.workunit.client.1.smithi096.stdout:0/68: write d0/f5 [2692289,3020] 0 2024-03-24T10:46:05.182 INFO:tasks.workunit.client.1.smithi096.stdout:2/73: creat d5/dc/d12/f20 x:0 0 0 2024-03-24T10:46:05.182 INFO:tasks.workunit.client.1.smithi096.stdout:2/74: creat d5/dc/f21 x:0 0 0 2024-03-24T10:46:05.183 INFO:tasks.workunit.client.1.smithi096.stdout:2/75: chown d5/de 112557 1 2024-03-24T10:46:05.185 INFO:tasks.workunit.client.1.smithi096.stdout:5/51: dread d2/fa [0,4194304] 0 2024-03-24T10:46:05.186 INFO:tasks.workunit.client.1.smithi096.stdout:4/77: rename d1/f18 to d1/f1c 0 2024-03-24T10:46:05.187 INFO:tasks.workunit.client.1.smithi096.stdout:4/78: read - d1/fd zero size 2024-03-24T10:46:05.187 INFO:tasks.workunit.client.1.smithi096.stdout:4/79: chown d1/la 7 1 2024-03-24T10:46:05.187 INFO:tasks.workunit.client.1.smithi096.stdout:6/98: truncate f6 960313 0 2024-03-24T10:46:05.187 INFO:tasks.workunit.client.1.smithi096.stdout:6/99: dread - d8/f26 zero size 2024-03-24T10:46:05.189 INFO:tasks.workunit.client.1.smithi096.stdout:9/56: stat c6 0 2024-03-24T10:46:05.189 INFO:tasks.workunit.client.0.smithi094.stdout:7/963: dread d1/db4/d65/faf [0,4194304] 0 2024-03-24T10:46:05.191 INFO:tasks.workunit.client.1.smithi096.stdout:1/54: creat d3/f11 x:0 0 0 2024-03-24T10:46:05.192 INFO:tasks.workunit.client.0.smithi094.stdout:5/992: mknod d7/d23/d5f/d14/d36/c144 0 2024-03-24T10:46:05.193 INFO:tasks.workunit.client.1.smithi096.stdout:7/81: symlink d0/l1e 0 2024-03-24T10:46:05.194 INFO:tasks.workunit.client.0.smithi094.stdout:0/976: mknod d14/d22/d28/dac/ddd/c158 0 2024-03-24T10:46:05.197 INFO:tasks.workunit.client.1.smithi096.stdout:9/57: write d9/db/fe [1058634,115290] 0 2024-03-24T10:46:05.198 INFO:tasks.workunit.client.0.smithi094.stdout:1/927: rename d4/d9/db/f18 to d4/d9/de/d6f/d5f/d8d/f13a 0 2024-03-24T10:46:05.201 INFO:tasks.workunit.client.0.smithi094.stdout:7/964: symlink d1/d9/d16/d21/d51/d83/d39/d44/l142 0 2024-03-24T10:46:05.205 INFO:tasks.workunit.client.0.smithi094.stdout:5/993: link d7/dc/d100/d109/f10c d7/d23/d5f/d9/d48/d51/f145 0 2024-03-24T10:46:05.207 INFO:tasks.workunit.client.1.smithi096.stdout:5/52: rmdir d2/d7/d8/df 39 2024-03-24T10:46:05.208 INFO:tasks.workunit.client.0.smithi094.stdout:5/994: dread d7/d23/dd5/d135/f138 [0,4194304] 0 2024-03-24T10:46:05.210 INFO:tasks.workunit.client.0.smithi094.stdout:1/928: write d4/d9/db/d31/d3c/fd7 [3292674,48821] 0 2024-03-24T10:46:05.210 INFO:tasks.workunit.client.0.smithi094.stdout:0/977: truncate d14/d22/d32/d70/dd0/fa3 6722453 0 2024-03-24T10:46:05.210 INFO:tasks.workunit.client.0.smithi094.stdout:1/929: write d4/d9/db/d5d/f64 [906555,112954] 0 2024-03-24T10:46:05.213 INFO:tasks.workunit.client.1.smithi096.stdout:6/100: unlink d8/d10/d13/f17 0 2024-03-24T10:46:05.214 INFO:tasks.workunit.client.0.smithi094.stdout:5/995: symlink d7/dc/db9/l146 0 2024-03-24T10:46:05.217 INFO:tasks.workunit.client.1.smithi096.stdout:7/82: creat d0/f1f x:0 0 0 2024-03-24T10:46:05.221 INFO:tasks.workunit.client.1.smithi096.stdout:0/69: rmdir d0/d2 39 2024-03-24T10:46:05.221 INFO:tasks.workunit.client.1.smithi096.stdout:9/58: dwrite d9/db/fe [0,4194304] 0 2024-03-24T10:46:05.223 INFO:tasks.workunit.client.0.smithi094.stdout:1/930: mknod d4/d9/de/d3d/d66/dbe/c13b 0 2024-03-24T10:46:05.224 INFO:tasks.workunit.client.1.smithi096.stdout:2/76: mknod d5/c22 0 2024-03-24T10:46:05.226 INFO:tasks.workunit.client.1.smithi096.stdout:4/80: link d1/l7 d1/l1d 0 2024-03-24T10:46:05.226 INFO:tasks.workunit.client.1.smithi096.stdout:4/81: chown d1/f10 100 1 2024-03-24T10:46:05.227 INFO:tasks.workunit.client.0.smithi094.stdout:5/996: stat d7/d23/d5f/d9/d48/cb5 0 2024-03-24T10:46:05.227 INFO:tasks.workunit.client.0.smithi094.stdout:5/997: fsync d7/d23/d5f/d14/f7c 0 2024-03-24T10:46:05.227 INFO:tasks.workunit.client.0.smithi094.stdout:5/998: chown d7/d23/dd5/da9/d13c/f79 56 1 2024-03-24T10:46:05.227 INFO:tasks.workunit.client.0.smithi094.stdout:5/999: write d7/dc/fc1 [214141,91254] 0 2024-03-24T10:46:05.228 INFO:tasks.workunit.client.1.smithi096.stdout:4/82: dread d1/d11/f13 [0,4194304] 0 2024-03-24T10:46:05.228 INFO:tasks.workunit.client.1.smithi096.stdout:4/83: truncate d1/d11/f13 869897 0 2024-03-24T10:46:05.228 INFO:tasks.workunit.client.1.smithi096.stdout:6/101: unlink d8/d10/f11 0 2024-03-24T10:46:05.230 INFO:tasks.workunit.client.0.smithi094.stdout:1/931: rename d4/d9/d7d/l59 to d4/d9/de/d3d/dae/l13c 0 2024-03-24T10:46:05.230 INFO:tasks.workunit.client.0.smithi094.stdout:1/932: chown d4/d9/db/d39/d43/fdb 2169015 1 2024-03-24T10:46:05.231 INFO:tasks.workunit.client.1.smithi096.stdout:1/55: rename d3/f8 to d3/f12 0 2024-03-24T10:46:05.232 INFO:tasks.workunit.client.0.smithi094.stdout:7/965: rmdir d1/d9/d25/d3d/d112/d133 0 2024-03-24T10:46:05.232 INFO:tasks.workunit.client.0.smithi094.stdout:7/966: dread - d1/d9/d16/d21/d51/dc4/ff2 zero size 2024-03-24T10:46:05.233 INFO:tasks.workunit.client.0.smithi094.stdout:7/967: fdatasync d1/d6c/d80/d81/f13b 0 2024-03-24T10:46:05.233 INFO:tasks.workunit.client.0.smithi094.stdout:7/968: readlink d1/d9/d16/d21/d51/dc4/lfc 0 2024-03-24T10:46:05.233 INFO:tasks.workunit.client.1.smithi096.stdout:7/83: creat d0/f20 x:0 0 0 2024-03-24T10:46:05.236 INFO:tasks.workunit.client.0.smithi094.stdout:0/978: dwrite d14/d22/d28/d2b/d68/f72 [0,4194304] 0 2024-03-24T10:46:05.237 INFO:tasks.workunit.client.1.smithi096.stdout:5/53: dwrite d2/f6 [4194304,4194304] 0 2024-03-24T10:46:05.237 INFO:tasks.workunit.client.1.smithi096.stdout:5/54: write d2/f6 [8718456,17742] 0 2024-03-24T10:46:05.237 INFO:tasks.workunit.client.1.smithi096.stdout:0/70: symlink d0/l10 0 2024-03-24T10:46:05.238 INFO:tasks.workunit.client.0.smithi094.stdout:1/933: creat d4/d9/db/d5d/f13d x:0 0 0 2024-03-24T10:46:05.239 INFO:tasks.workunit.client.1.smithi096.stdout:9/59: symlink d9/d11/l12 0 2024-03-24T10:46:05.241 INFO:tasks.workunit.client.0.smithi094.stdout:7/969: mknod d1/d9/c143 0 2024-03-24T10:46:05.241 INFO:tasks.workunit.client.0.smithi094.stdout:7/970: readlink d1/d9/d16/d21/d51/d83/d39/d44/dd9/l4a 0 2024-03-24T10:46:05.241 INFO:tasks.workunit.client.0.smithi094.stdout:7/971: fdatasync d1/d9/d25/d3d/fda 0 2024-03-24T10:46:05.241 INFO:tasks.workunit.client.0.smithi094.stdout:7/972: fsync d1/d9/d16/d21/d51/dc4/ff2 0 2024-03-24T10:46:05.242 INFO:tasks.workunit.client.1.smithi096.stdout:2/77: mkdir d5/dc/d12/d23 0 2024-03-24T10:46:05.244 INFO:tasks.workunit.client.0.smithi094.stdout:0/979: creat d14/d22/d32/d70/dd0/dce/f159 x:0 0 0 2024-03-24T10:46:05.244 INFO:tasks.workunit.client.0.smithi094.stdout:0/980: write f3 [4620467,124504] 0 2024-03-24T10:46:05.245 INFO:tasks.workunit.client.1.smithi096.stdout:6/102: creat d8/d12/f27 x:0 0 0 2024-03-24T10:46:05.247 INFO:tasks.workunit.client.1.smithi096.stdout:4/84: dwrite d1/f1c [0,4194304] 0 2024-03-24T10:46:05.247 INFO:tasks.workunit.client.1.smithi096.stdout:4/85: readlink l0 0 2024-03-24T10:46:05.248 INFO:tasks.workunit.client.1.smithi096.stdout:1/56: creat d3/f13 x:0 0 0 2024-03-24T10:46:05.250 INFO:tasks.workunit.client.0.smithi094.stdout:0/981: symlink d14/d22/l15a 0 2024-03-24T10:46:05.250 INFO:tasks.workunit.client.0.smithi094.stdout:0/982: write d14/f14f [838638,68603] 0 2024-03-24T10:46:05.250 INFO:tasks.workunit.client.0.smithi094.stdout:0/983: truncate d14/d22/dcb/f116 1608080 0 2024-03-24T10:46:05.251 INFO:tasks.workunit.client.0.smithi094.stdout:0/984: dread d14/d22/d28/d48/d7b/fa8 [0,4194304] 0 2024-03-24T10:46:05.252 INFO:tasks.workunit.client.0.smithi094.stdout:0/985: truncate d14/d22/d32/d65/ff3 1033902 0 2024-03-24T10:46:05.252 INFO:tasks.workunit.client.1.smithi096.stdout:9/60: dread d9/db/fe [0,4194304] 0 2024-03-24T10:46:05.253 INFO:tasks.workunit.client.0.smithi094.stdout:1/934: getdents d4/d9/de/d6f/d62/d84/de9 0 2024-03-24T10:46:05.255 INFO:tasks.workunit.client.1.smithi096.stdout:5/55: rename d2/fa to d2/d7/f13 0 2024-03-24T10:46:05.255 INFO:tasks.workunit.client.1.smithi096.stdout:9/61: write d9/db/fe [1390663,41860] 0 2024-03-24T10:46:05.256 INFO:tasks.workunit.client.1.smithi096.stdout:9/62: write d9/db/fe [1776893,13967] 0 2024-03-24T10:46:05.260 INFO:tasks.workunit.client.1.smithi096.stdout:0/71: mkdir d0/d11 0 2024-03-24T10:46:05.265 INFO:tasks.workunit.client.1.smithi096.stdout:9/63: dread d9/db/fe [0,4194304] 0 2024-03-24T10:46:05.266 INFO:tasks.workunit.client.0.smithi094.stdout:1/935: symlink d4/d9/d102/l13e 0 2024-03-24T10:46:05.268 INFO:tasks.workunit.client.0.smithi094.stdout:0/986: dwrite d14/d18/fb0 [0,4194304] 0 2024-03-24T10:46:05.268 INFO:tasks.workunit.client.0.smithi094.stdout:0/987: fsync d14/d22/d28/d2b/d68/f131 0 2024-03-24T10:46:05.278 INFO:tasks.workunit.client.1.smithi096.stdout:1/57: creat d3/f14 x:0 0 0 2024-03-24T10:46:05.279 INFO:tasks.workunit.client.1.smithi096.stdout:7/84: mkdir d0/d21 0 2024-03-24T10:46:05.283 INFO:tasks.workunit.client.0.smithi094.stdout:0/988: truncate d14/d22/d28/fe1 3683302 0 2024-03-24T10:46:05.283 INFO:tasks.workunit.client.0.smithi094.stdout:0/989: readlink d14/d22/l24 0 2024-03-24T10:46:05.285 INFO:tasks.workunit.client.1.smithi096.stdout:5/56: mkdir d2/d7/d8/df/d14 0 2024-03-24T10:46:05.289 INFO:tasks.workunit.client.1.smithi096.stdout:0/72: rename d0/c1 to d0/c12 0 2024-03-24T10:46:05.295 INFO:tasks.workunit.client.1.smithi096.stdout:2/78: rmdir d5 39 2024-03-24T10:46:05.297 INFO:tasks.workunit.client.1.smithi096.stdout:9/64: creat d9/f13 x:0 0 0 2024-03-24T10:46:05.297 INFO:tasks.workunit.client.1.smithi096.stdout:9/65: truncate d9/f13 508077 0 2024-03-24T10:46:05.297 INFO:tasks.workunit.client.1.smithi096.stdout:4/86: symlink d1/l1e 0 2024-03-24T10:46:05.299 INFO:tasks.workunit.client.1.smithi096.stdout:7/85: symlink d0/d21/l22 0 2024-03-24T10:46:05.300 INFO:tasks.workunit.client.1.smithi096.stdout:5/57: symlink d2/d7/d8/df/d14/l15 0 2024-03-24T10:46:05.300 INFO:tasks.workunit.client.1.smithi096.stdout:5/58: chown d2/d7/d8/lc 62805 1 2024-03-24T10:46:05.303 INFO:tasks.workunit.client.0.smithi094.stdout:1/936: dwrite d4/d9/de/d6f/d5f/d8d/fc2 [0,4194304] 0 2024-03-24T10:46:05.304 INFO:tasks.workunit.client.0.smithi094.stdout:0/990: dwrite d14/d22/d28/d48/f81 [0,4194304] 0 2024-03-24T10:46:05.306 INFO:tasks.workunit.client.1.smithi096.stdout:9/66: unlink d9/cd 0 2024-03-24T10:46:05.308 INFO:tasks.workunit.client.1.smithi096.stdout:4/87: creat d1/f1f x:0 0 0 2024-03-24T10:46:05.312 INFO:tasks.workunit.client.1.smithi096.stdout:1/58: rename d3/f5 to d3/f15 0 2024-03-24T10:46:05.317 INFO:tasks.workunit.client.0.smithi094.stdout:1/937: dread d4/d9/d37/d33/d58/d6b/fe4 [0,4194304] 0 2024-03-24T10:46:05.317 INFO:tasks.workunit.client.1.smithi096.stdout:5/59: mknod d2/d7/c16 0 2024-03-24T10:46:05.317 INFO:tasks.workunit.client.1.smithi096.stdout:5/60: write d2/d7/fd [8942619,81744] 0 2024-03-24T10:46:05.321 INFO:tasks.workunit.client.1.smithi096.stdout:4/88: dread d1/f8 [0,4194304] 0 2024-03-24T10:46:05.322 INFO:tasks.workunit.client.0.smithi094.stdout:1/938: dread d4/d9/db/f10 [0,4194304] 0 2024-03-24T10:46:05.322 INFO:tasks.workunit.client.1.smithi096.stdout:4/89: write d1/d11/f12 [84536,58503] 0 2024-03-24T10:46:05.322 INFO:tasks.workunit.client.1.smithi096.stdout:4/90: chown d1/fd 4115664 1 2024-03-24T10:46:05.322 INFO:tasks.workunit.client.1.smithi096.stdout:2/79: unlink d5/f6 0 2024-03-24T10:46:05.325 INFO:tasks.workunit.client.0.smithi094.stdout:1/939: creat d4/d36/f13f x:0 0 0 2024-03-24T10:46:05.325 INFO:tasks.workunit.client.0.smithi094.stdout:0/991: dwrite d14/d22/d28/f31 [0,4194304] 0 2024-03-24T10:46:05.326 INFO:tasks.workunit.client.1.smithi096.stdout:9/67: stat c6 0 2024-03-24T10:46:05.326 INFO:tasks.workunit.client.1.smithi096.stdout:9/68: chown c6 1749901826 1 2024-03-24T10:46:05.326 INFO:tasks.workunit.client.0.smithi094.stdout:0/992: read d14/d22/f35 [785716,49377] 0 2024-03-24T10:46:05.327 INFO:tasks.workunit.client.1.smithi096.stdout:0/73: dwrite d0/f5 [0,4194304] 0 2024-03-24T10:46:05.327 INFO:tasks.workunit.client.1.smithi096.stdout:5/61: dread d2/d7/f13 [4194304,4194304] 0 2024-03-24T10:46:05.327 INFO:tasks.workunit.client.1.smithi096.stdout:5/62: chown d2/f5 546432 1 2024-03-24T10:46:05.327 INFO:tasks.workunit.client.1.smithi096.stdout:0/74: chown d0/c12 7 1 2024-03-24T10:46:05.331 INFO:tasks.workunit.client.1.smithi096.stdout:7/86: mkdir d0/d23 0 2024-03-24T10:46:05.331 INFO:tasks.workunit.client.1.smithi096.stdout:7/87: readlink d0/d21/l22 0 2024-03-24T10:46:05.335 INFO:tasks.workunit.client.0.smithi094.stdout:1/940: creat d4/d9/db/d31/f140 x:0 0 0 2024-03-24T10:46:05.335 INFO:tasks.workunit.client.0.smithi094.stdout:1/941: dread - d4/d9/db/d31/f140 zero size 2024-03-24T10:46:05.335 INFO:tasks.workunit.client.0.smithi094.stdout:1/942: fdatasync d4/d9/f77 0 2024-03-24T10:46:05.335 INFO:tasks.workunit.client.1.smithi096.stdout:2/80: mknod d5/dc/c24 0 2024-03-24T10:46:05.338 INFO:tasks.workunit.client.1.smithi096.stdout:9/69: creat d9/db/f14 x:0 0 0 2024-03-24T10:46:05.340 INFO:tasks.workunit.client.1.smithi096.stdout:1/59: symlink d3/l16 0 2024-03-24T10:46:05.341 INFO:tasks.workunit.client.1.smithi096.stdout:1/60: fdatasync d3/f9 0 2024-03-24T10:46:05.341 INFO:tasks.workunit.client.1.smithi096.stdout:5/63: dwrite d2/f3 [0,4194304] 0 2024-03-24T10:46:05.341 INFO:tasks.workunit.client.1.smithi096.stdout:5/64: stat d2/le 0 2024-03-24T10:46:05.342 INFO:tasks.workunit.client.0.smithi094.stdout:3/975: sync 2024-03-24T10:46:05.343 INFO:tasks.workunit.client.0.smithi094.stdout:7/973: sync 2024-03-24T10:46:05.343 INFO:tasks.workunit.client.0.smithi094.stdout:7/974: truncate d1/d9/f23 1941606 0 2024-03-24T10:46:05.343 INFO:tasks.workunit.client.0.smithi094.stdout:7/975: fsync d1/d9/fac 0 2024-03-24T10:46:05.343 INFO:tasks.workunit.client.0.smithi094.stdout:3/976: write d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/dfb/f12b [525993,34980] 0 2024-03-24T10:46:05.344 INFO:tasks.workunit.client.1.smithi096.stdout:7/88: creat d0/f24 x:0 0 0 2024-03-24T10:46:05.345 INFO:tasks.workunit.client.1.smithi096.stdout:4/91: symlink d1/l20 0 2024-03-24T10:46:05.354 INFO:tasks.workunit.client.1.smithi096.stdout:4/92: read d1/f10 [4015941,85236] 0 2024-03-24T10:46:05.356 INFO:tasks.workunit.client.1.smithi096.stdout:4/93: dread d1/fe [0,4194304] 0 2024-03-24T10:46:05.393 INFO:tasks.workunit.client.1.smithi096.stdout:2/81: getdents d5/dc 0 2024-03-24T10:46:05.396 INFO:tasks.workunit.client.1.smithi096.stdout:9/70: rename d9/d11 to d9/d15 0 2024-03-24T10:46:05.396 INFO:tasks.workunit.client.1.smithi096.stdout:9/71: chown c6 24 1 2024-03-24T10:46:05.396 INFO:tasks.workunit.client.1.smithi096.stdout:9/72: chown d9/f13 29076 1 2024-03-24T10:46:05.397 INFO:tasks.workunit.client.0.smithi094.stdout:0/993: rmdir d14/d22/d28 39 2024-03-24T10:46:05.400 INFO:tasks.workunit.client.0.smithi094.stdout:7/976: rename d1/d9/d16/d21/d51/d83/f12c to d1/d9/d16/d21/d51/d83/d39/d44/dd9/d34/f144 0 2024-03-24T10:46:05.400 INFO:tasks.workunit.client.0.smithi094.stdout:7/977: fsync d1/d9/dc3/f10e 0 2024-03-24T10:46:05.400 INFO:tasks.workunit.client.1.smithi096.stdout:5/65: dwrite d2/f6 [4194304,4194304] 0 2024-03-24T10:46:05.401 INFO:tasks.workunit.client.0.smithi094.stdout:1/943: dwrite d4/d9/d37/d33/f110 [0,4194304] 0 2024-03-24T10:46:05.401 INFO:tasks.workunit.client.0.smithi094.stdout:3/977: symlink d0/db/dc/d28/d3c/d55/d7f/l152 0 2024-03-24T10:46:05.401 INFO:tasks.workunit.client.0.smithi094.stdout:3/978: creat d0/db/f153 x:0 0 0 2024-03-24T10:46:05.404 INFO:tasks.workunit.client.0.smithi094.stdout:0/994: mknod d14/d22/d28/d48/d7b/dfa/c15b 0 2024-03-24T10:46:05.409 INFO:tasks.workunit.client.1.smithi096.stdout:4/94: unlink d1/l1d 0 2024-03-24T10:46:05.410 INFO:tasks.workunit.client.1.smithi096.stdout:2/82: write d5/de/f15 [8247,63915] 0 2024-03-24T10:46:05.412 INFO:tasks.workunit.client.1.smithi096.stdout:1/61: mknod d3/c17 0 2024-03-24T10:46:05.413 INFO:tasks.workunit.client.1.smithi096.stdout:7/89: rename d0/db/c17 to d0/c25 0 2024-03-24T10:46:05.414 INFO:tasks.workunit.client.0.smithi094.stdout:1/944: write d4/d9/de/d6f/d62/d80/fe2 [2566361,92311] 0 2024-03-24T10:46:05.418 INFO:tasks.workunit.client.1.smithi096.stdout:2/83: link f2 d5/dc/f25 0 2024-03-24T10:46:05.420 INFO:tasks.workunit.client.1.smithi096.stdout:4/95: symlink d1/l21 0 2024-03-24T10:46:05.421 INFO:tasks.workunit.client.0.smithi094.stdout:1/945: unlink d4/d9/db/d5d/l7b 0 2024-03-24T10:46:05.421 INFO:tasks.workunit.client.0.smithi094.stdout:1/946: read d4/d9/db/d5d/fc3 [1929873,106115] 0 2024-03-24T10:46:05.422 INFO:tasks.workunit.client.0.smithi094.stdout:7/978: dwrite d1/d9/d16/d21/d51/d83/d39/f48 [0,4194304] 0 2024-03-24T10:46:05.422 INFO:tasks.workunit.client.0.smithi094.stdout:7/979: truncate d1/db4/f9e 4578545 0 2024-03-24T10:46:05.422 INFO:tasks.workunit.client.0.smithi094.stdout:3/979: dwrite d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/f64 [0,4194304] 0 2024-03-24T10:46:05.422 INFO:tasks.workunit.client.0.smithi094.stdout:3/980: readlink d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d4f/l96 0 2024-03-24T10:46:05.423 INFO:tasks.workunit.client.1.smithi096.stdout:4/96: symlink d1/d11/l22 0 2024-03-24T10:46:05.423 INFO:tasks.workunit.client.1.smithi096.stdout:4/97: stat d1/l20 0 2024-03-24T10:46:05.423 INFO:tasks.workunit.client.1.smithi096.stdout:4/98: write d1/d11/f19 [751428,90678] 0 2024-03-24T10:46:05.426 INFO:tasks.workunit.client.1.smithi096.stdout:4/99: rename d1/d11/f13 to d1/d11/f23 0 2024-03-24T10:46:05.429 INFO:tasks.workunit.client.0.smithi094.stdout:3/981: truncate d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/db5/d106/d9b/fa1 77948 0 2024-03-24T10:46:05.429 INFO:tasks.workunit.client.0.smithi094.stdout:3/982: stat d0/db/dc/d28/d3c/d55/l62 0 2024-03-24T10:46:05.431 INFO:tasks.workunit.client.0.smithi094.stdout:3/983: rename d0/db/l8 to d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/dfb/l154 0 2024-03-24T10:46:05.434 INFO:tasks.workunit.client.1.smithi096.stdout:1/62: write d3/f15 [2207029,6835] 0 2024-03-24T10:46:05.436 INFO:tasks.workunit.client.1.smithi096.stdout:1/63: unlink d3/f14 0 2024-03-24T10:46:05.436 INFO:tasks.workunit.client.1.smithi096.stdout:1/64: rename d3 to d3/d18 22 2024-03-24T10:46:05.436 INFO:tasks.workunit.client.1.smithi096.stdout:1/65: dread - d3/fb zero size 2024-03-24T10:46:05.446 INFO:tasks.workunit.client.1.smithi096.stdout:8/62: sync 2024-03-24T10:46:05.451 INFO:tasks.workunit.client.1.smithi096.stdout:5/66: dwrite d2/f5 [0,4194304] 0 2024-03-24T10:46:05.452 INFO:tasks.workunit.client.0.smithi094.stdout:7/980: dwrite d1/d9/d16/d21/d51/d83/d39/d44/dd9/d34/f144 [0,4194304] 0 2024-03-24T10:46:05.454 INFO:tasks.workunit.client.1.smithi096.stdout:5/67: rename d2/d7/f13 to d2/f17 0 2024-03-24T10:46:05.459 INFO:tasks.workunit.client.1.smithi096.stdout:7/90: getdents d0 0 2024-03-24T10:46:05.459 INFO:tasks.workunit.client.1.smithi096.stdout:7/91: readlink d0/d21/l22 0 2024-03-24T10:46:05.462 INFO:tasks.workunit.client.1.smithi096.stdout:7/92: getdents d0/db 0 2024-03-24T10:46:05.464 INFO:tasks.workunit.client.0.smithi094.stdout:1/947: dwrite d4/d36/f8e [0,4194304] 0 2024-03-24T10:46:05.466 INFO:tasks.workunit.client.0.smithi094.stdout:1/948: dread d4/d9/de/d3d/faf [0,4194304] 0 2024-03-24T10:46:05.469 INFO:tasks.workunit.client.1.smithi096.stdout:4/100: getdents d1/d11 0 2024-03-24T10:46:05.471 INFO:tasks.workunit.client.1.smithi096.stdout:4/101: creat d1/f24 x:0 0 0 2024-03-24T10:46:05.472 INFO:tasks.workunit.client.1.smithi096.stdout:2/84: write d5/f9 [268030,32284] 0 2024-03-24T10:46:05.473 INFO:tasks.workunit.client.1.smithi096.stdout:4/102: symlink d1/l25 0 2024-03-24T10:46:05.473 INFO:tasks.workunit.client.1.smithi096.stdout:4/103: stat d1/d11/l22 0 2024-03-24T10:46:05.473 INFO:tasks.workunit.client.1.smithi096.stdout:4/104: write d1/f10 [1298212,12554] 0 2024-03-24T10:46:05.473 INFO:tasks.workunit.client.1.smithi096.stdout:2/85: write d5/f7 [4958168,82787] 0 2024-03-24T10:46:05.474 INFO:tasks.workunit.client.1.smithi096.stdout:7/93: read d0/d13/f5 [456426,21039] 0 2024-03-24T10:46:05.475 INFO:tasks.workunit.client.1.smithi096.stdout:2/86: unlink d5/f9 0 2024-03-24T10:46:05.477 INFO:tasks.workunit.client.1.smithi096.stdout:7/94: rename d0/d23 to d0/db/d26 0 2024-03-24T10:46:05.477 INFO:tasks.workunit.client.1.smithi096.stdout:2/87: creat d5/dc/d12/f26 x:0 0 0 2024-03-24T10:46:05.478 INFO:tasks.workunit.client.1.smithi096.stdout:7/95: creat d0/d21/f27 x:0 0 0 2024-03-24T10:46:05.479 INFO:tasks.workunit.client.0.smithi094.stdout:1/949: dread d4/d9/d37/d33/f110 [0,4194304] 0 2024-03-24T10:46:05.479 INFO:tasks.workunit.client.0.smithi094.stdout:1/950: truncate d4/d9/d37/d33/f51 4798492 0 2024-03-24T10:46:05.479 INFO:tasks.workunit.client.1.smithi096.stdout:2/88: chown d5/de/c19 288 1 2024-03-24T10:46:05.479 INFO:tasks.workunit.client.1.smithi096.stdout:2/89: chown d5/dc/c24 877198070 1 2024-03-24T10:46:05.480 INFO:tasks.workunit.client.1.smithi096.stdout:7/96: rename d0/d13/ld to d0/db/l28 0 2024-03-24T10:46:05.481 INFO:tasks.workunit.client.1.smithi096.stdout:7/97: creat d0/db/d1c/f29 x:0 0 0 2024-03-24T10:46:05.481 INFO:tasks.workunit.client.1.smithi096.stdout:7/98: readlink d0/d13/l4 0 2024-03-24T10:46:05.482 INFO:tasks.workunit.client.0.smithi094.stdout:3/984: dwrite d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/d73/f11f [0,4194304] 0 2024-03-24T10:46:05.482 INFO:tasks.workunit.client.0.smithi094.stdout:3/985: chown d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a 121 1 2024-03-24T10:46:05.482 INFO:tasks.workunit.client.0.smithi094.stdout:1/951: symlink d4/d9/de/d6f/d5f/d8d/l141 0 2024-03-24T10:46:05.483 INFO:tasks.workunit.client.1.smithi096.stdout:2/90: unlink f4 0 2024-03-24T10:46:05.484 INFO:tasks.workunit.client.1.smithi096.stdout:4/105: write d1/f8 [3757070,17118] 0 2024-03-24T10:46:05.485 INFO:tasks.workunit.client.1.smithi096.stdout:7/99: symlink d0/d21/l2a 0 2024-03-24T10:46:05.485 INFO:tasks.workunit.client.1.smithi096.stdout:2/91: write d5/de/f13 [899198,10957] 0 2024-03-24T10:46:05.486 INFO:tasks.workunit.client.0.smithi094.stdout:1/952: truncate d4/d9/f2c 8534290 0 2024-03-24T10:46:05.487 INFO:tasks.workunit.client.0.smithi094.stdout:1/953: write d4/d9/db/d31/d3c/f7c [4691051,51402] 0 2024-03-24T10:46:05.488 INFO:tasks.workunit.client.0.smithi094.stdout:1/954: stat d4/d9/db/d39/d75/fe5 0 2024-03-24T10:46:05.488 INFO:tasks.workunit.client.0.smithi094.stdout:1/955: chown d4/d9/de/c14 4247 1 2024-03-24T10:46:05.488 INFO:tasks.workunit.client.1.smithi096.stdout:7/100: unlink d0/d13/f1a 0 2024-03-24T10:46:05.488 INFO:tasks.workunit.client.1.smithi096.stdout:1/66: dwrite d3/fb [0,4194304] 0 2024-03-24T10:46:05.488 INFO:tasks.workunit.client.1.smithi096.stdout:1/67: fsync d3/ff 0 2024-03-24T10:46:05.489 INFO:tasks.workunit.client.0.smithi094.stdout:7/981: dwrite d1/db4/d65/f89 [0,4194304] 0 2024-03-24T10:46:05.489 INFO:tasks.workunit.client.0.smithi094.stdout:7/982: chown d1/d9/dc3/fed 3471086 1 2024-03-24T10:46:05.489 INFO:tasks.workunit.client.1.smithi096.stdout:2/92: creat d5/dc/f27 x:0 0 0 2024-03-24T10:46:05.489 INFO:tasks.workunit.client.1.smithi096.stdout:5/68: dwrite d2/d7/fd [4194304,4194304] 0 2024-03-24T10:46:05.490 INFO:tasks.workunit.client.1.smithi096.stdout:8/63: dwrite d8/ff [0,4194304] 0 2024-03-24T10:46:05.490 INFO:tasks.workunit.client.1.smithi096.stdout:8/64: chown l2 0 1 2024-03-24T10:46:05.490 INFO:tasks.workunit.client.1.smithi096.stdout:8/65: chown d8/f10 7 1 2024-03-24T10:46:05.490 INFO:tasks.workunit.client.1.smithi096.stdout:8/66: write d8/f10 [1687506,48898] 0 2024-03-24T10:46:05.490 INFO:tasks.workunit.client.1.smithi096.stdout:8/67: fsync f0 0 2024-03-24T10:46:05.495 INFO:tasks.workunit.client.0.smithi094.stdout:1/956: read d4/d9/db/d31/d3c/d9f/fee [4758815,75944] 0 2024-03-24T10:46:05.495 INFO:tasks.workunit.client.0.smithi094.stdout:3/986: dread d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d87/dc5/d101/fd4 [0,4194304] 0 2024-03-24T10:46:05.495 INFO:tasks.workunit.client.1.smithi096.stdout:7/101: mknod d0/db/c2b 0 2024-03-24T10:46:05.501 INFO:tasks.workunit.client.1.smithi096.stdout:2/93: symlink d5/dc/l28 0 2024-03-24T10:46:05.504 INFO:tasks.workunit.client.0.smithi094.stdout:1/957: dread d4/d9/db/d31/d3c/f50 [0,4194304] 0 2024-03-24T10:46:05.505 INFO:tasks.workunit.client.0.smithi094.stdout:7/983: mkdir d1/d9/d16/d21/d51/d83/d145 0 2024-03-24T10:46:05.506 INFO:tasks.workunit.client.1.smithi096.stdout:5/69: mkdir d2/d7/d8/d18 0 2024-03-24T10:46:05.509 INFO:tasks.workunit.client.1.smithi096.stdout:4/106: dwrite d1/f10 [0,4194304] 0 2024-03-24T10:46:05.509 INFO:tasks.workunit.client.1.smithi096.stdout:4/107: truncate d1/ff 745455 0 2024-03-24T10:46:05.509 INFO:tasks.workunit.client.0.smithi094.stdout:3/987: getdents d0/db/dc/d28/d3c/d55 0 2024-03-24T10:46:05.509 INFO:tasks.workunit.client.0.smithi094.stdout:3/988: read - d0/db/dc/d28/d50/f13a zero size 2024-03-24T10:46:05.509 INFO:tasks.workunit.client.0.smithi094.stdout:3/989: truncate d0/db/dc/d28/d3c/d55/d7f/ffa 1653941 0 2024-03-24T10:46:05.514 INFO:tasks.workunit.client.0.smithi094.stdout:7/984: rename d1/d9/d16/d21/d51/d83/f90 to d1/d9/d16/d21/d49/f146 0 2024-03-24T10:46:05.515 INFO:tasks.workunit.client.0.smithi094.stdout:7/985: stat d1/d11f 0 2024-03-24T10:46:05.517 INFO:tasks.workunit.client.1.smithi096.stdout:5/70: dread d2/f6 [0,4194304] 0 2024-03-24T10:46:05.518 INFO:tasks.workunit.client.1.smithi096.stdout:5/71: truncate d2/d7/f10 715889 0 2024-03-24T10:46:05.522 INFO:tasks.workunit.client.0.smithi094.stdout:7/986: link d1/d9/d16/d21/d49/dfa/c111 d1/d9/d16/d21/d51/d83/d39/d44/dd9/da5/c147 0 2024-03-24T10:46:05.525 INFO:tasks.workunit.client.0.smithi094.stdout:7/987: getdents d1/d11f 0 2024-03-24T10:46:05.528 INFO:tasks.workunit.client.0.smithi094.stdout:7/988: creat d1/d9/d16/d21/d51/d83/d39/d44/d13a/f148 x:0 0 0 2024-03-24T10:46:05.530 INFO:tasks.workunit.client.1.smithi096.stdout:8/68: getdents d8 0 2024-03-24T10:46:05.530 INFO:tasks.workunit.client.1.smithi096.stdout:8/69: chown d8/la 30 1 2024-03-24T10:46:05.530 INFO:tasks.workunit.client.1.smithi096.stdout:8/70: chown d8/fc 62649236 1 2024-03-24T10:46:05.531 INFO:tasks.workunit.client.0.smithi094.stdout:7/989: truncate d1/d9/d16/f2f 415617 0 2024-03-24T10:46:05.531 INFO:tasks.workunit.client.0.smithi094.stdout:7/990: stat d1/d9/d25/de2/cf9 0 2024-03-24T10:46:05.532 INFO:tasks.workunit.client.1.smithi096.stdout:7/102: creat d0/d13/f2c x:0 0 0 2024-03-24T10:46:05.532 INFO:tasks.workunit.client.1.smithi096.stdout:7/103: chown d0/db/d26 1612584626 1 2024-03-24T10:46:05.533 INFO:tasks.workunit.client.0.smithi094.stdout:0/995: sync 2024-03-24T10:46:05.534 INFO:tasks.workunit.client.1.smithi096.stdout:1/68: mknod d3/c19 0 2024-03-24T10:46:05.535 INFO:tasks.workunit.client.0.smithi094.stdout:7/991: mknod d1/d9/d16/d21/d51/d83/d39/d44/dd9/da5/c149 0 2024-03-24T10:46:05.535 INFO:tasks.workunit.client.0.smithi094.stdout:7/992: fsync d1/d9/d96/f9a 0 2024-03-24T10:46:05.536 INFO:tasks.workunit.client.0.smithi094.stdout:0/996: rmdir d14/d22/d5b/d86 39 2024-03-24T10:46:05.537 INFO:tasks.workunit.client.1.smithi096.stdout:5/72: rename d2/f6 to d2/d7/d8/df/f19 0 2024-03-24T10:46:05.538 INFO:tasks.workunit.client.0.smithi094.stdout:7/993: truncate d1/d6c/f88 440132 0 2024-03-24T10:46:05.538 INFO:tasks.workunit.client.0.smithi094.stdout:7/994: write d1/fcb [1231403,119884] 0 2024-03-24T10:46:05.539 INFO:tasks.workunit.client.1.smithi096.stdout:8/71: mkdir d8/d11 0 2024-03-24T10:46:05.540 INFO:tasks.workunit.client.1.smithi096.stdout:4/108: dwrite d1/f3 [0,4194304] 0 2024-03-24T10:46:05.540 INFO:tasks.workunit.client.0.smithi094.stdout:1/958: dwrite d4/f2a [0,4194304] 0 2024-03-24T10:46:05.540 INFO:tasks.workunit.client.0.smithi094.stdout:1/959: chown d4/d9/db/d5d/d63/lac 289 1 2024-03-24T10:46:05.540 INFO:tasks.workunit.client.0.smithi094.stdout:0/997: creat d14/d22/d28/d2b/f15c x:0 0 0 2024-03-24T10:46:05.542 INFO:tasks.workunit.client.0.smithi094.stdout:0/998: write d14/d22/d28/d2b/d41/fe3 [746228,128869] 0 2024-03-24T10:46:05.542 INFO:tasks.workunit.client.0.smithi094.stdout:0/999: write d14/d22/d28/d2b/f45 [4839762,34662] 0 2024-03-24T10:46:05.542 INFO:tasks.workunit.client.1.smithi096.stdout:2/94: link d5/c8 d5/dc/d12/c29 0 2024-03-24T10:46:05.542 INFO:tasks.workunit.client.1.smithi096.stdout:2/95: write d5/de/f1a [793662,129568] 0 2024-03-24T10:46:05.543 INFO:tasks.workunit.client.0.smithi094.stdout:7/995: unlink d1/d9/f23 0 2024-03-24T10:46:05.544 INFO:tasks.workunit.client.0.smithi094.stdout:7/996: fdatasync d1/d9/d16/d21/d51/dcc/fe5 0 2024-03-24T10:46:05.544 INFO:tasks.workunit.client.0.smithi094.stdout:7/997: chown d1/d6c/f88 1272018 1 2024-03-24T10:46:05.544 INFO:tasks.workunit.client.0.smithi094.stdout:7/998: chown d1/d9/d16/d21/d51/d83/d39/d44/dd9/d34/c4d 13107 1 2024-03-24T10:46:05.544 INFO:tasks.workunit.client.0.smithi094.stdout:7/999: write d1/d9/d16/d21/d49/da9/fb8 [1184925,76839] 0 2024-03-24T10:46:05.544 INFO:tasks.workunit.client.1.smithi096.stdout:5/73: mknod d2/d7/c1a 0 2024-03-24T10:46:05.545 INFO:tasks.workunit.client.0.smithi094.stdout:1/960: dread f1 [0,4194304] 0 2024-03-24T10:46:05.545 INFO:tasks.workunit.client.0.smithi094.stdout:1/961: write d4/d9/db/f136 [710130,46774] 0 2024-03-24T10:46:05.546 INFO:tasks.workunit.client.1.smithi096.stdout:4/109: symlink d1/d11/l26 0 2024-03-24T10:46:05.547 INFO:tasks.workunit.client.1.smithi096.stdout:4/110: write d1/ff [612762,118935] 0 2024-03-24T10:46:05.547 INFO:tasks.workunit.client.1.smithi096.stdout:1/69: symlink d3/l1a 0 2024-03-24T10:46:05.548 INFO:tasks.workunit.client.0.smithi094.stdout:3/990: dwrite d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d25/d34/d71/fc4 [0,4194304] 0 2024-03-24T10:46:05.548 INFO:tasks.workunit.client.1.smithi096.stdout:2/96: rmdir d5/de 39 2024-03-24T10:46:05.548 INFO:tasks.workunit.client.1.smithi096.stdout:2/97: stat d5/dc/d12/l1b 0 2024-03-24T10:46:05.549 INFO:tasks.workunit.client.1.smithi096.stdout:5/74: creat d2/d7/d8/df/f1b x:0 0 0 2024-03-24T10:46:05.551 INFO:tasks.workunit.client.1.smithi096.stdout:8/72: dread f0 [0,4194304] 0 2024-03-24T10:46:05.551 INFO:tasks.workunit.client.1.smithi096.stdout:4/111: rename d1/l25 to d1/d11/l27 0 2024-03-24T10:46:05.556 INFO:tasks.workunit.client.0.smithi094.stdout:1/962: write d4/d9/de/d6f/d5f/d8d/f13a [3783554,32948] 0 2024-03-24T10:46:05.559 INFO:tasks.workunit.client.0.smithi094.stdout:3/991: mkdir d0/db/dc/d28/d3c/d155 0 2024-03-24T10:46:05.563 INFO:tasks.workunit.client.1.smithi096.stdout:8/73: rename d8/f10 to d8/f12 0 2024-03-24T10:46:05.564 INFO:tasks.workunit.client.1.smithi096.stdout:1/70: mknod d3/c1b 0 2024-03-24T10:46:05.566 INFO:tasks.workunit.client.1.smithi096.stdout:2/98: mkdir d5/de/d2a 0 2024-03-24T10:46:05.566 INFO:tasks.workunit.client.1.smithi096.stdout:4/112: dwrite d1/f3 [4194304,4194304] 0 2024-03-24T10:46:05.567 INFO:tasks.workunit.client.1.smithi096.stdout:8/74: rmdir d8 39 2024-03-24T10:46:05.568 INFO:tasks.workunit.client.0.smithi094.stdout:1/963: dread d4/d9/de/d6f/d62/d84/fe6 [0,4194304] 0 2024-03-24T10:46:05.572 INFO:tasks.workunit.client.1.smithi096.stdout:2/99: creat d5/de/d2a/f2b x:0 0 0 2024-03-24T10:46:05.574 INFO:tasks.workunit.client.0.smithi094.stdout:1/964: symlink d4/d9/d37/d33/d76/l142 0 2024-03-24T10:46:05.574 INFO:tasks.workunit.client.0.smithi094.stdout:1/965: dread - d4/d9/db/d5d/d63/f119 zero size 2024-03-24T10:46:05.575 INFO:tasks.workunit.client.0.smithi094.stdout:1/966: write d4/d9/d37/f134 [340576,52937] 0 2024-03-24T10:46:05.578 INFO:tasks.workunit.client.1.smithi096.stdout:1/71: rename d3/f11 to d3/f1c 0 2024-03-24T10:46:05.581 INFO:tasks.workunit.client.1.smithi096.stdout:2/100: link d5/de/f1a d5/dc/d12/d23/f2c 0 2024-03-24T10:46:05.584 INFO:tasks.workunit.client.0.smithi094.stdout:3/992: dwrite d0/db/dc/d28/d38/f141 [0,4194304] 0 2024-03-24T10:46:05.584 INFO:tasks.workunit.client.0.smithi094.stdout:3/993: write d0/db/dc/d28/d3c/d55/d7f/d20/d23/f9a [2739351,116318] 0 2024-03-24T10:46:05.585 INFO:tasks.workunit.client.1.smithi096.stdout:1/72: dread d3/f6 [0,4194304] 0 2024-03-24T10:46:05.587 INFO:tasks.workunit.client.0.smithi094.stdout:1/967: dread d4/d9/de/fa7 [0,4194304] 0 2024-03-24T10:46:05.588 INFO:tasks.workunit.client.0.smithi094.stdout:1/968: chown d4/d9/d37/d33/d58/d6b/ff8 3 1 2024-03-24T10:46:05.589 INFO:tasks.workunit.client.1.smithi096.stdout:8/75: dwrite f1 [8388608,4194304] 0 2024-03-24T10:46:05.592 INFO:tasks.workunit.client.1.smithi096.stdout:3/159: sync 2024-03-24T10:46:05.593 INFO:tasks.workunit.client.1.smithi096.stdout:8/76: dread d8/fc [0,4194304] 0 2024-03-24T10:46:05.595 INFO:tasks.workunit.client.0.smithi094.stdout:1/969: rename d4/d9/db/d85/df5/l109 to d4/d9/de/d6f/d62/d84/de9/l143 0 2024-03-24T10:46:05.595 INFO:tasks.workunit.client.0.smithi094.stdout:1/970: creat d4/d9/d37/d33/d58/f144 x:0 0 0 2024-03-24T10:46:05.595 INFO:tasks.workunit.client.0.smithi094.stdout:1/971: write d4/d9/de/d6f/d5f/d8d/f13a [2622896,38395] 0 2024-03-24T10:46:05.595 INFO:tasks.workunit.client.0.smithi094.stdout:1/972: creat d4/d9/db/d31/f145 x:0 0 0 2024-03-24T10:46:05.595 INFO:tasks.workunit.client.1.smithi096.stdout:1/73: mknod d3/c1d 0 2024-03-24T10:46:05.599 INFO:tasks.workunit.client.1.smithi096.stdout:3/160: mkdir d0/d9/d34/d39 0 2024-03-24T10:46:05.599 INFO:tasks.workunit.client.1.smithi096.stdout:3/161: read - d0/d9/d22/f29 zero size 2024-03-24T10:46:05.600 INFO:tasks.workunit.client.1.smithi096.stdout:3/162: dread - d0/d9/f32 zero size 2024-03-24T10:46:05.600 INFO:tasks.workunit.client.1.smithi096.stdout:3/163: stat d0/d13/f17 0 2024-03-24T10:46:05.600 INFO:tasks.workunit.client.1.smithi096.stdout:3/164: fdatasync d0/d13/f1f 0 2024-03-24T10:46:05.600 INFO:tasks.workunit.client.1.smithi096.stdout:5/75: getdents d2/d7/d8/df 0 2024-03-24T10:46:05.602 INFO:tasks.workunit.client.1.smithi096.stdout:4/113: unlink d1/d11/l27 0 2024-03-24T10:46:05.604 INFO:tasks.workunit.client.0.smithi094.stdout:3/994: dwrite d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/f116 [0,4194304] 0 2024-03-24T10:46:05.605 INFO:tasks.workunit.client.1.smithi096.stdout:8/77: mknod d8/d11/c13 0 2024-03-24T10:46:05.605 INFO:tasks.workunit.client.1.smithi096.stdout:8/78: truncate d8/fc 728984 0 2024-03-24T10:46:05.607 INFO:tasks.workunit.client.1.smithi096.stdout:5/76: dread d2/f17 [0,4194304] 0 2024-03-24T10:46:05.608 INFO:tasks.workunit.client.1.smithi096.stdout:1/74: unlink d3/fb 0 2024-03-24T10:46:05.608 INFO:tasks.workunit.client.1.smithi096.stdout:1/75: write d3/f15 [1610825,85221] 0 2024-03-24T10:46:05.608 INFO:tasks.workunit.client.1.smithi096.stdout:1/76: chown d3 713 1 2024-03-24T10:46:05.608 INFO:tasks.workunit.client.1.smithi096.stdout:1/77: write d3/f9 [964317,42105] 0 2024-03-24T10:46:05.608 INFO:tasks.workunit.client.1.smithi096.stdout:1/78: chown d3/l1a 176236 1 2024-03-24T10:46:05.610 INFO:tasks.workunit.client.1.smithi096.stdout:3/165: mknod d0/d9/d22/c3a 0 2024-03-24T10:46:05.610 INFO:tasks.workunit.client.1.smithi096.stdout:3/166: write d0/d9/f36 [669779,35253] 0 2024-03-24T10:46:05.613 INFO:tasks.workunit.client.1.smithi096.stdout:4/114: mknod d1/d11/c28 0 2024-03-24T10:46:05.613 INFO:tasks.workunit.client.1.smithi096.stdout:1/79: write d3/f9 [281122,118103] 0 2024-03-24T10:46:05.615 INFO:tasks.workunit.client.0.smithi094.stdout:3/995: mknod d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d87/dc5/c156 0 2024-03-24T10:46:05.615 INFO:tasks.workunit.client.0.smithi094.stdout:3/996: fdatasync d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/db5/fc6 0 2024-03-24T10:46:05.617 INFO:tasks.workunit.client.1.smithi096.stdout:8/79: creat d8/f14 x:0 0 0 2024-03-24T10:46:05.620 INFO:tasks.workunit.client.0.smithi094.stdout:3/997: mkdir d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d145/d157 0 2024-03-24T10:46:05.623 INFO:tasks.workunit.client.1.smithi096.stdout:8/80: unlink f0 0 2024-03-24T10:46:05.623 INFO:tasks.workunit.client.1.smithi096.stdout:8/81: creat d8/f15 x:0 0 0 2024-03-24T10:46:05.623 INFO:tasks.workunit.client.1.smithi096.stdout:8/82: dread - d8/f14 zero size 2024-03-24T10:46:05.623 INFO:tasks.workunit.client.1.smithi096.stdout:8/83: chown d8/ld 1904151 1 2024-03-24T10:46:05.623 INFO:tasks.workunit.client.1.smithi096.stdout:8/84: creat d8/f16 x:0 0 0 2024-03-24T10:46:05.625 INFO:tasks.workunit.client.0.smithi094.stdout:3/998: rename d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/dfb/f12b to d0/db/dc/d28/d3c/d55/d7f/d20/d23/d2a/d3a/d56/d61/d69/f158 0 2024-03-24T10:46:05.627 INFO:tasks.workunit.client.0.smithi094.stdout:3/999: symlink d0/db/dc/d28/d3c/d55/d7f/l159 0 2024-03-24T10:46:05.628 INFO:tasks.workunit.client.1.smithi096.stdout:8/85: mkdir d8/d11/d17 0 2024-03-24T10:46:05.629 INFO:tasks.workunit.client.1.smithi096.stdout:8/86: symlink d8/d11/d17/l18 0 2024-03-24T10:46:05.630 INFO:tasks.workunit.client.1.smithi096.stdout:1/80: unlink d3/f1c 0 2024-03-24T10:46:05.631 INFO:tasks.workunit.client.1.smithi096.stdout:8/87: creat d8/d11/f19 x:0 0 0 2024-03-24T10:46:05.633 INFO:tasks.workunit.client.1.smithi096.stdout:1/81: creat d3/f1e x:0 0 0 2024-03-24T10:46:05.633 INFO:tasks.workunit.client.1.smithi096.stdout:1/82: write d3/f12 [2247922,6590] 0 2024-03-24T10:46:05.633 INFO:tasks.workunit.client.1.smithi096.stdout:8/88: dread d8/fc [0,4194304] 0 2024-03-24T10:46:05.633 INFO:tasks.workunit.client.1.smithi096.stdout:8/89: chown d8/ld 0 1 2024-03-24T10:46:05.636 INFO:tasks.workunit.client.1.smithi096.stdout:1/83: dread d3/fc [0,4194304] 0 2024-03-24T10:46:05.637 INFO:tasks.workunit.client.1.smithi096.stdout:3/167: dwrite d0/d9/f36 [0,4194304] 0 2024-03-24T10:46:05.637 INFO:tasks.workunit.client.1.smithi096.stdout:3/168: creat d0/d9/f3b x:0 0 0 2024-03-24T10:46:05.637 INFO:tasks.workunit.client.0.smithi094.stdout:1/973: dwrite d4/d9/db/f136 [0,4194304] 0 2024-03-24T10:46:05.637 INFO:tasks.workunit.client.0.smithi094.stdout:1/974: chown d4/d9/de/d6f/d5f/lc7 11931 1 2024-03-24T10:46:05.643 INFO:tasks.workunit.client.1.smithi096.stdout:5/77: dwrite d2/f17 [4194304,4194304] 0 2024-03-24T10:46:05.646 INFO:tasks.workunit.client.1.smithi096.stdout:3/169: link d0/c12 d0/c3c 0 2024-03-24T10:46:05.651 INFO:tasks.workunit.client.1.smithi096.stdout:8/90: dread d8/ff [0,4194304] 0 2024-03-24T10:46:05.651 INFO:tasks.workunit.client.1.smithi096.stdout:3/170: rename d0/d9/d22/f25 to d0/d9/d22/f3d 0 2024-03-24T10:46:05.655 INFO:tasks.workunit.client.1.smithi096.stdout:5/78: read d2/f3 [3286740,120081] 0 2024-03-24T10:46:05.656 INFO:tasks.workunit.client.1.smithi096.stdout:4/115: getdents d1/d11 0 2024-03-24T10:46:05.656 INFO:tasks.workunit.client.1.smithi096.stdout:4/116: chown d1/f10 470100382 1 2024-03-24T10:46:05.657 INFO:tasks.workunit.client.1.smithi096.stdout:1/84: dread d3/f6 [0,4194304] 0 2024-03-24T10:46:05.660 INFO:tasks.workunit.client.1.smithi096.stdout:4/117: creat d1/d11/f29 x:0 0 0 2024-03-24T10:46:05.660 INFO:tasks.workunit.client.1.smithi096.stdout:4/118: truncate d1/f24 372343 0 2024-03-24T10:46:05.660 INFO:tasks.workunit.client.1.smithi096.stdout:4/119: truncate d1/fd 965781 0 2024-03-24T10:46:05.661 INFO:tasks.workunit.client.1.smithi096.stdout:2/101: dwrite d5/fa [0,4194304] 0 2024-03-24T10:46:05.663 INFO:tasks.workunit.client.1.smithi096.stdout:1/85: creat d3/f1f x:0 0 0 2024-03-24T10:46:05.663 INFO:tasks.workunit.client.1.smithi096.stdout:1/86: chown c1 105 1 2024-03-24T10:46:05.667 INFO:tasks.workunit.client.1.smithi096.stdout:2/102: symlink d5/dc/l2d 0 2024-03-24T10:46:05.667 INFO:tasks.workunit.client.1.smithi096.stdout:2/103: creat d5/dc/f2e x:0 0 0 2024-03-24T10:46:05.668 INFO:tasks.workunit.client.1.smithi096.stdout:1/87: unlink d3/c17 0 2024-03-24T10:46:05.671 INFO:tasks.workunit.client.1.smithi096.stdout:3/171: dwrite d0/d13/f17 [0,4194304] 0 2024-03-24T10:46:05.671 INFO:tasks.workunit.client.1.smithi096.stdout:3/172: fdatasync d0/d9/d22/f3d 0 2024-03-24T10:46:05.674 INFO:tasks.workunit.client.1.smithi096.stdout:2/104: getdents d5/de/d2a 0 2024-03-24T10:46:05.678 INFO:tasks.workunit.client.1.smithi096.stdout:5/79: dwrite d2/d7/d8/f12 [0,4194304] 0 2024-03-24T10:46:05.678 INFO:tasks.workunit.client.1.smithi096.stdout:5/80: creat d2/d7/f1c x:0 0 0 2024-03-24T10:46:05.678 INFO:tasks.workunit.client.1.smithi096.stdout:5/81: stat d2/d7/d8/df 0 2024-03-24T10:46:05.678 INFO:tasks.workunit.client.1.smithi096.stdout:3/173: mknod d0/d9/d22/c3e 0 2024-03-24T10:46:05.678 INFO:tasks.workunit.client.1.smithi096.stdout:3/174: stat d0/d9/d22/c3e 0 2024-03-24T10:46:05.678 INFO:tasks.workunit.client.1.smithi096.stdout:1/88: symlink d3/l20 0 2024-03-24T10:46:05.679 INFO:tasks.workunit.client.1.smithi096.stdout:2/105: stat d5/l10 0 2024-03-24T10:46:05.684 INFO:tasks.workunit.client.1.smithi096.stdout:5/82: mknod d2/d7/d8/d18/c1d 0 2024-03-24T10:46:05.684 INFO:tasks.workunit.client.1.smithi096.stdout:8/91: dwrite d8/ff [0,4194304] 0 2024-03-24T10:46:05.684 INFO:tasks.workunit.client.1.smithi096.stdout:5/83: fdatasync d2/d7/fd 0 2024-03-24T10:46:05.686 INFO:tasks.workunit.client.1.smithi096.stdout:3/175: link d0/d13/c1d d0/c3f 0 2024-03-24T10:46:05.686 INFO:tasks.workunit.client.1.smithi096.stdout:4/120: dwrite d1/f10 [0,4194304] 0 2024-03-24T10:46:05.687 INFO:tasks.workunit.client.1.smithi096.stdout:1/89: unlink d3/f1e 0 2024-03-24T10:46:05.689 INFO:tasks.workunit.client.1.smithi096.stdout:5/84: truncate d2/d7/fd 3062480 0 2024-03-24T10:46:05.693 INFO:tasks.workunit.client.1.smithi096.stdout:1/90: dread d3/f6 [0,4194304] 0 2024-03-24T10:46:05.693 INFO:tasks.workunit.client.1.smithi096.stdout:1/91: chown d3/ff 13482 1 2024-03-24T10:46:05.695 INFO:tasks.workunit.client.1.smithi096.stdout:3/176: write d0/d9/f36 [1692027,75195] 0 2024-03-24T10:46:05.696 INFO:tasks.workunit.client.0.smithi094.stdout:1/975: sync 2024-03-24T10:46:05.707 INFO:tasks.workunit.client.1.smithi096.stdout:4/121: dread d1/f3 [4194304,4194304] 0 2024-03-24T10:46:05.707 INFO:tasks.workunit.client.1.smithi096.stdout:4/122: dread - d1/f1f zero size 2024-03-24T10:46:05.707 INFO:tasks.workunit.client.0.smithi094.stdout:1/976: unlink d4/d9/de/c14 0 2024-03-24T10:46:05.708 INFO:tasks.workunit.client.0.smithi094.stdout:1/977: write d4/d9/f10e [334729,12169] 0 2024-03-24T10:46:05.709 INFO:tasks.workunit.client.1.smithi096.stdout:3/177: mknod d0/d9/d22/c40 0 2024-03-24T10:46:05.710 INFO:tasks.workunit.client.1.smithi096.stdout:8/92: dwrite f1 [4194304,4194304] 0 2024-03-24T10:46:05.713 INFO:tasks.workunit.client.1.smithi096.stdout:3/178: mkdir d0/d13/d41 0 2024-03-24T10:46:05.717 INFO:tasks.workunit.client.1.smithi096.stdout:8/93: truncate d8/f12 1345339 0 2024-03-24T10:46:05.718 INFO:tasks.workunit.client.1.smithi096.stdout:4/123: getdents d1 0 2024-03-24T10:46:05.718 INFO:tasks.workunit.client.1.smithi096.stdout:5/85: dwrite d2/d7/f1c [0,4194304] 0 2024-03-24T10:46:05.719 INFO:tasks.workunit.client.1.smithi096.stdout:8/94: mkdir d8/d11/d17/d1a 0 2024-03-24T10:46:05.725 INFO:tasks.workunit.client.1.smithi096.stdout:5/86: rename d2/l9 to d2/d7/d8/df/d14/l1e 0 2024-03-24T10:46:05.725 INFO:tasks.workunit.client.1.smithi096.stdout:1/92: rmdir d3 39 2024-03-24T10:46:05.726 INFO:tasks.workunit.client.1.smithi096.stdout:4/124: symlink d1/d11/l2a 0 2024-03-24T10:46:05.730 INFO:tasks.workunit.client.1.smithi096.stdout:4/125: mkdir d1/d2b 0 2024-03-24T10:46:05.731 INFO:tasks.workunit.client.1.smithi096.stdout:5/87: read d2/f5 [2584317,108719] 0 2024-03-24T10:46:05.731 INFO:tasks.workunit.client.1.smithi096.stdout:5/88: chown d2/d7/f10 202016081 1 2024-03-24T10:46:05.736 INFO:tasks.workunit.client.1.smithi096.stdout:5/89: truncate d2/d7/d8/df/f19 135655 0 2024-03-24T10:46:05.740 INFO:tasks.workunit.client.0.smithi094.stdout:1/978: dwrite d4/d9/d37/d33/d58/f144 [0,4194304] 0 2024-03-24T10:46:05.740 INFO:tasks.workunit.client.0.smithi094.stdout:1/979: write d4/d9/db/d39/d43/f11a [1017054,119359] 0 2024-03-24T10:46:05.741 INFO:tasks.workunit.client.1.smithi096.stdout:3/179: dwrite d0/d13/d1e/f2f [0,4194304] 0 2024-03-24T10:46:05.741 INFO:tasks.workunit.client.1.smithi096.stdout:1/93: dwrite d3/f12 [0,4194304] 0 2024-03-24T10:46:05.743 INFO:tasks.workunit.client.1.smithi096.stdout:8/95: write d8/f12 [2381942,43743] 0 2024-03-24T10:46:05.743 INFO:tasks.workunit.client.1.smithi096.stdout:8/96: read - d8/d11/f19 zero size 2024-03-24T10:46:05.749 INFO:tasks.workunit.client.0.smithi094.stdout:1/980: rmdir d4/d9/de/d6f/d62/d84/de9 39 2024-03-24T10:46:05.749 INFO:tasks.workunit.client.0.smithi094.stdout:1/981: fdatasync d4/f12a 0 2024-03-24T10:46:05.749 INFO:tasks.workunit.client.0.smithi094.stdout:1/982: fsync d4/d9/db/d5d/fc3 0 2024-03-24T10:46:05.749 INFO:tasks.workunit.client.0.smithi094.stdout:1/983: chown d4/d9/de/d6f/d62/d80/f8f 17292614 1 2024-03-24T10:46:05.749 INFO:tasks.workunit.client.0.smithi094.stdout:1/984: readlink d4/d36/l133 0 2024-03-24T10:46:05.749 INFO:tasks.workunit.client.0.smithi094.stdout:1/985: readlink d4/d9/de/d3d/dae/l11f 0 2024-03-24T10:46:05.754 INFO:tasks.workunit.client.0.smithi094.stdout:1/986: rename d4/d9/de/d6f/d5f/d8d/cf4 to d4/d9/d102/c146 0 2024-03-24T10:46:05.755 INFO:tasks.workunit.client.0.smithi094.stdout:1/987: creat d4/d9/d37/d33/d58/f147 x:0 0 0 2024-03-24T10:46:05.755 INFO:tasks.workunit.client.0.smithi094.stdout:1/988: write d4/d9/d37/d33/d76/f9b [3298396,24826] 0 2024-03-24T10:46:05.755 INFO:tasks.workunit.client.0.smithi094.stdout:1/989: chown d4/d9/db/d31/d3c/fd7 0 1 2024-03-24T10:46:05.755 INFO:tasks.workunit.client.0.smithi094.stdout:1/990: write d4/d9/db/d39/d43/f11a [443537,110830] 0 2024-03-24T10:46:05.766 INFO:tasks.workunit.client.0.smithi094.stdout:1/991: dread d4/d9/db/d5d/fc3 [0,4194304] 0 2024-03-24T10:46:05.766 INFO:tasks.workunit.client.0.smithi094.stdout:1/992: creat d4/d9/de/d6f/d62/d99/f148 x:0 0 0 2024-03-24T10:46:05.767 INFO:tasks.workunit.client.1.smithi096.stdout:8/97: dwrite d8/f16 [0,4194304] 0 2024-03-24T10:46:05.767 INFO:tasks.workunit.client.1.smithi096.stdout:8/98: write d8/d11/f19 [1002204,130796] 0 2024-03-24T10:46:05.767 INFO:tasks.workunit.client.1.smithi096.stdout:3/180: dwrite d0/d13/f17 [0,4194304] 0 2024-03-24T10:46:05.769 INFO:tasks.workunit.client.1.smithi096.stdout:5/90: dwrite d2/d7/d8/f12 [4194304,4194304] 0 2024-03-24T10:46:05.769 INFO:tasks.workunit.client.1.smithi096.stdout:5/91: fsync d2/d7/f10 0 2024-03-24T10:46:05.769 INFO:tasks.workunit.client.1.smithi096.stdout:5/92: write d2/f3 [2296047,21644] 0 2024-03-24T10:46:05.769 INFO:tasks.workunit.client.1.smithi096.stdout:5/93: fsync d2/d7/f1c 0 2024-03-24T10:46:05.769 INFO:tasks.workunit.client.1.smithi096.stdout:5/94: fsync d2/d7/f10 0 2024-03-24T10:46:05.769 INFO:tasks.workunit.client.0.smithi094.stdout:1/993: rename d4/f7 to d4/d9/de/d6f/d62/d84/f149 0 2024-03-24T10:46:05.772 INFO:tasks.workunit.client.1.smithi096.stdout:3/181: dread d0/d9/f1a [0,4194304] 0 2024-03-24T10:46:05.772 INFO:tasks.workunit.client.1.smithi096.stdout:3/182: readlink d0/l5 0 2024-03-24T10:46:05.772 INFO:tasks.workunit.client.1.smithi096.stdout:8/99: rename d8/ld to d8/d11/l1b 0 2024-03-24T10:46:05.773 INFO:tasks.workunit.client.0.smithi094.stdout:1/994: dread d4/d9/f25 [0,4194304] 0 2024-03-24T10:46:05.773 INFO:tasks.workunit.client.0.smithi094.stdout:1/995: stat d4/d9/d37/d33/d58/f144 0 2024-03-24T10:46:05.777 INFO:tasks.workunit.client.0.smithi094.stdout:1/996: unlink d4/d9/de/d3d/d66/dbe/c13b 0 2024-03-24T10:46:05.777 INFO:tasks.workunit.client.0.smithi094.stdout:1/997: creat d4/d9/de/d6f/d62/f14a x:0 0 0 2024-03-24T10:46:05.777 INFO:tasks.workunit.client.0.smithi094.stdout:1/998: fsync d4/d9/d37/d33/d58/f93 0 2024-03-24T10:46:05.778 INFO:tasks.workunit.client.0.smithi094.stdout:1/999: write d4/d9/db/d31/f145 [820467,47698] 0 2024-03-24T10:46:05.779 INFO:tasks.workunit.client.1.smithi096.stdout:5/95: creat d2/d7/d8/df/f1f x:0 0 0 2024-03-24T10:46:05.781 INFO:tasks.workunit.client.1.smithi096.stdout:8/100: rename d8/ff to d8/d11/d17/f1c 0 2024-03-24T10:46:05.783 INFO:tasks.workunit.client.0.smithi094.stderr:+ rm -rf -- ./tmp.ndA6na18Wr 2024-03-24T10:46:05.784 INFO:tasks.workunit.client.1.smithi096.stdout:5/96: rename d2/d7/f1c to d2/d7/d8/d18/f20 0 2024-03-24T10:46:05.784 INFO:tasks.workunit.client.1.smithi096.stdout:5/97: readlink d2/d7/d8/lb 0 2024-03-24T10:46:05.784 INFO:tasks.workunit.client.1.smithi096.stdout:8/101: mkdir d8/d11/d17/d1d 0 2024-03-24T10:46:05.786 INFO:tasks.workunit.client.1.smithi096.stdout:3/183: dwrite d0/d13/d1e/f2a [0,4194304] 0 2024-03-24T10:46:05.788 INFO:tasks.workunit.client.1.smithi096.stdout:3/184: mknod d0/d13/d41/c42 0 2024-03-24T10:46:05.790 INFO:tasks.workunit.client.1.smithi096.stdout:5/98: dread d2/f3 [0,4194304] 0 2024-03-24T10:46:05.791 INFO:tasks.workunit.client.1.smithi096.stdout:6/103: sync 2024-03-24T10:46:05.791 INFO:tasks.workunit.client.1.smithi096.stdout:0/75: sync 2024-03-24T10:46:05.791 INFO:tasks.workunit.client.1.smithi096.stdout:6/104: fdatasync d8/d10/d22/f21 0 2024-03-24T10:46:05.791 INFO:tasks.workunit.client.1.smithi096.stdout:3/185: creat d0/d9/d34/d39/f43 x:0 0 0 2024-03-24T10:46:05.793 INFO:tasks.workunit.client.1.smithi096.stdout:5/99: unlink d2/d7/d8/df/l11 0 2024-03-24T10:46:05.795 INFO:tasks.workunit.client.1.smithi096.stdout:3/186: creat d0/f44 x:0 0 0 2024-03-24T10:46:05.796 INFO:tasks.workunit.client.1.smithi096.stdout:3/187: write d0/d9/d34/f37 [332379,14282] 0 2024-03-24T10:46:05.797 INFO:tasks.workunit.client.1.smithi096.stdout:3/188: dread d0/d9/f1a [0,4194304] 0 2024-03-24T10:46:05.799 INFO:tasks.workunit.client.1.smithi096.stdout:5/100: mknod d2/c21 0 2024-03-24T10:46:05.800 INFO:tasks.workunit.client.1.smithi096.stdout:3/189: mkdir d0/d13/d45 0 2024-03-24T10:46:05.801 INFO:tasks.workunit.client.1.smithi096.stdout:8/102: dwrite d8/f16 [0,4194304] 0 2024-03-24T10:46:05.803 INFO:tasks.workunit.client.1.smithi096.stdout:5/101: rename d2/le to d2/d7/l22 0 2024-03-24T10:46:05.803 INFO:tasks.workunit.client.1.smithi096.stdout:5/102: chown d2/d7/d8/d18 696758 1 2024-03-24T10:46:05.804 INFO:tasks.workunit.client.1.smithi096.stdout:3/190: dread d0/d13/d1e/f2d [0,4194304] 0 2024-03-24T10:46:05.804 INFO:tasks.workunit.client.1.smithi096.stdout:3/191: fsync d0/d13/f17 0 2024-03-24T10:46:05.806 INFO:tasks.workunit.client.1.smithi096.stdout:3/192: dread d0/d13/f1f [0,4194304] 0 2024-03-24T10:46:05.809 INFO:tasks.workunit.client.1.smithi096.stdout:8/103: mkdir d8/d11/d17/d1e 0 2024-03-24T10:46:05.809 INFO:tasks.workunit.client.1.smithi096.stdout:6/105: dread d8/f18 [0,4194304] 0 2024-03-24T10:46:05.809 INFO:tasks.workunit.client.1.smithi096.stdout:6/106: write d8/d12/f27 [45492,34271] 0 2024-03-24T10:46:05.811 INFO:tasks.workunit.client.1.smithi096.stdout:6/107: chown l0 7 1 2024-03-24T10:46:05.812 INFO:tasks.workunit.client.1.smithi096.stdout:6/108: rmdir d8/d10/d22 39 2024-03-24T10:46:05.812 INFO:tasks.workunit.client.1.smithi096.stdout:6/109: chown d8/d10/c1a 8514 1 2024-03-24T10:46:05.815 INFO:tasks.workunit.client.1.smithi096.stdout:3/193: write d0/d13/f17 [2467491,74611] 0 2024-03-24T10:46:05.816 INFO:tasks.workunit.client.1.smithi096.stdout:3/194: write d0/d13/d1e/f2a [4987501,74163] 0 2024-03-24T10:46:05.816 INFO:tasks.workunit.client.1.smithi096.stdout:3/195: write d0/d13/f1f [277209,39746] 0 2024-03-24T10:46:05.816 INFO:tasks.workunit.client.1.smithi096.stdout:0/76: dwrite d0/f5 [0,4194304] 0 2024-03-24T10:46:05.816 INFO:tasks.workunit.client.1.smithi096.stdout:6/110: mkdir d8/d12/d28 0 2024-03-24T10:46:05.819 INFO:tasks.workunit.client.1.smithi096.stdout:6/111: dread f2 [0,4194304] 0 2024-03-24T10:46:05.819 INFO:tasks.workunit.client.1.smithi096.stdout:6/112: write d8/d10/d22/f21 [789922,40735] 0 2024-03-24T10:46:05.819 INFO:tasks.workunit.client.1.smithi096.stdout:6/113: chown f3 111967 1 2024-03-24T10:46:05.823 INFO:tasks.workunit.client.1.smithi096.stdout:3/196: creat d0/d13/f46 x:0 0 0 2024-03-24T10:46:05.823 INFO:tasks.workunit.client.1.smithi096.stdout:3/197: dread - d0/d9/d34/d39/f43 zero size 2024-03-24T10:46:05.827 INFO:tasks.workunit.client.1.smithi096.stdout:8/104: dwrite f1 [0,4194304] 0 2024-03-24T10:46:05.832 INFO:tasks.workunit.client.1.smithi096.stdout:5/103: rmdir d2/d7/d8 39 2024-03-24T10:46:05.832 INFO:tasks.workunit.client.1.smithi096.stdout:5/104: chown d2/d7/d8 6017 1 2024-03-24T10:46:05.840 INFO:tasks.workunit.client.1.smithi096.stdout:0/77: mknod d0/c13 0 2024-03-24T10:46:05.840 INFO:tasks.workunit.client.1.smithi096.stdout:0/78: read d0/d2/fb [2929269,6027] 0 2024-03-24T10:46:05.842 INFO:tasks.workunit.client.1.smithi096.stdout:6/114: creat d8/f29 x:0 0 0 2024-03-24T10:46:05.842 INFO:tasks.workunit.client.1.smithi096.stdout:6/115: write f3 [4510201,41827] 0 2024-03-24T10:46:05.842 INFO:tasks.workunit.client.1.smithi096.stdout:6/116: fdatasync d8/f14 0 2024-03-24T10:46:05.843 INFO:tasks.workunit.client.1.smithi096.stdout:3/198: mkdir d0/d13/d45/d47 0 2024-03-24T10:46:05.844 INFO:tasks.workunit.client.1.smithi096.stdout:8/105: creat d8/d11/d17/d1a/f1f x:0 0 0 2024-03-24T10:46:05.846 INFO:tasks.workunit.client.1.smithi096.stdout:5/105: rename d2/d7/fd to d2/d7/d8/df/d14/f23 0 2024-03-24T10:46:05.847 INFO:tasks.workunit.client.1.smithi096.stdout:6/117: write d8/d12/f1f [5105654,98991] 0 2024-03-24T10:46:05.847 INFO:tasks.workunit.client.1.smithi096.stdout:6/118: truncate d8/d12/f27 231507 0 2024-03-24T10:46:05.848 INFO:tasks.workunit.client.1.smithi096.stdout:0/79: write d0/f5 [4146448,67543] 0 2024-03-24T10:46:05.848 INFO:tasks.workunit.client.1.smithi096.stdout:3/199: mkdir d0/d9/d34/d39/d48 0 2024-03-24T10:46:05.866 INFO:tasks.workunit.client.1.smithi096.stdout:8/106: rename d8/d11/d17/l18 to d8/l20 0 2024-03-24T10:46:05.872 INFO:tasks.workunit.client.1.smithi096.stdout:6/119: creat d8/d10/f2a x:0 0 0 2024-03-24T10:46:05.874 INFO:tasks.workunit.client.1.smithi096.stdout:5/106: dwrite d2/d7/d8/df/f19 [0,4194304] 0 2024-03-24T10:46:05.874 INFO:tasks.workunit.client.1.smithi096.stdout:0/80: link d0/c12 d0/d11/c14 0 2024-03-24T10:46:05.875 INFO:tasks.workunit.client.1.smithi096.stdout:3/200: dwrite d0/d9/d34/f37 [0,4194304] 0 2024-03-24T10:46:05.875 INFO:tasks.workunit.client.1.smithi096.stdout:3/201: chown d0/d9/d34 1769499 1 2024-03-24T10:46:05.875 INFO:tasks.workunit.client.1.smithi096.stdout:6/120: write d8/d12/f27 [112469,122924] 0 2024-03-24T10:46:05.875 INFO:tasks.workunit.client.1.smithi096.stdout:0/81: mkdir d0/d2/d15 0 2024-03-24T10:46:05.882 INFO:tasks.workunit.client.1.smithi096.stdout:5/107: write d2/d7/d8/f12 [8346084,42525] 0 2024-03-24T10:46:05.882 INFO:tasks.workunit.client.1.smithi096.stdout:8/107: dwrite f1 [8388608,4194304] 0 2024-03-24T10:46:05.882 INFO:tasks.workunit.client.1.smithi096.stdout:6/121: creat d8/d10/d13/f2b x:0 0 0 2024-03-24T10:46:05.883 INFO:tasks.workunit.client.1.smithi096.stdout:0/82: mkdir d0/d11/d16 0 2024-03-24T10:46:05.883 INFO:tasks.workunit.client.1.smithi096.stdout:0/83: write d0/f5 [4667893,92331] 0 2024-03-24T10:46:05.883 INFO:tasks.workunit.client.1.smithi096.stdout:0/84: stat d0/c8 0 2024-03-24T10:46:05.884 INFO:tasks.workunit.client.1.smithi096.stdout:3/202: getdents d0/d9/d34/d39 0 2024-03-24T10:46:05.894 INFO:tasks.workunit.client.1.smithi096.stdout:5/108: mknod d2/d7/d8/df/d14/c24 0 2024-03-24T10:46:05.898 INFO:tasks.workunit.client.1.smithi096.stdout:8/108: mkdir d8/d11/d17/d1d/d21 0 2024-03-24T10:46:05.898 INFO:tasks.workunit.client.1.smithi096.stdout:8/109: truncate d8/fe 885303 0 2024-03-24T10:46:05.900 INFO:tasks.workunit.client.1.smithi096.stdout:0/85: dwrite d0/d2/fb [0,4194304] 0 2024-03-24T10:46:05.902 INFO:tasks.workunit.client.1.smithi096.stdout:6/122: creat d8/d10/d22/f2c x:0 0 0 2024-03-24T10:46:05.905 INFO:tasks.workunit.client.1.smithi096.stdout:9/73: sync 2024-03-24T10:46:05.905 INFO:tasks.workunit.client.1.smithi096.stdout:9/74: write d9/db/f14 [295178,67027] 0 2024-03-24T10:46:05.905 INFO:tasks.workunit.client.1.smithi096.stdout:9/75: creat d9/f16 x:0 0 0 2024-03-24T10:46:05.905 INFO:tasks.workunit.client.1.smithi096.stdout:9/76: dread - d9/f16 zero size 2024-03-24T10:46:05.905 INFO:tasks.workunit.client.1.smithi096.stdout:9/77: write d9/f16 [387651,68950] 0 2024-03-24T10:46:05.906 INFO:tasks.workunit.client.1.smithi096.stdout:3/203: rename d0/f7 to d0/d9/f49 0 2024-03-24T10:46:05.909 INFO:tasks.workunit.client.1.smithi096.stdout:5/109: creat d2/d7/d8/df/f25 x:0 0 0 2024-03-24T10:46:05.909 INFO:tasks.workunit.client.1.smithi096.stdout:5/110: write d2/d7/d8/df/f1b [858528,99903] 0 2024-03-24T10:46:05.910 INFO:tasks.workunit.client.1.smithi096.stdout:9/78: dread d9/f16 [0,4194304] 0 2024-03-24T10:46:05.911 INFO:tasks.workunit.client.1.smithi096.stdout:8/110: rmdir d8/d11/d17/d1d 39 2024-03-24T10:46:05.911 INFO:tasks.workunit.client.1.smithi096.stdout:9/79: read d9/db/f14 [39051,54648] 0 2024-03-24T10:46:05.913 INFO:tasks.workunit.client.1.smithi096.stdout:0/86: mknod d0/d11/d16/c17 0 2024-03-24T10:46:05.918 INFO:tasks.workunit.client.1.smithi096.stdout:3/204: unlink d0/c10 0 2024-03-24T10:46:05.921 INFO:tasks.workunit.client.1.smithi096.stdout:3/205: write d0/d9/f2e [389760,35385] 0 2024-03-24T10:46:05.924 INFO:tasks.workunit.client.1.smithi096.stdout:5/111: dwrite d2/f3 [0,4194304] 0 2024-03-24T10:46:05.927 INFO:tasks.workunit.client.1.smithi096.stdout:8/111: rename d8/d11/d17/d1d/d21 to d8/d11/d22 0 2024-03-24T10:46:05.930 INFO:tasks.workunit.client.1.smithi096.stdout:9/80: symlink d9/d15/l17 0 2024-03-24T10:46:05.933 INFO:tasks.workunit.client.1.smithi096.stdout:6/123: getdents d8/d10/d22 0 2024-03-24T10:46:05.933 INFO:tasks.workunit.client.1.smithi096.stdout:6/124: fdatasync d8/f14 0 2024-03-24T10:46:05.933 INFO:tasks.workunit.client.1.smithi096.stdout:6/125: write d8/d10/fd [887157,104016] 0 2024-03-24T10:46:05.933 INFO:tasks.workunit.client.1.smithi096.stdout:6/126: write f7 [4288695,1477] 0 2024-03-24T10:46:05.933 INFO:tasks.workunit.client.1.smithi096.stdout:6/127: dread - d8/f26 zero size 2024-03-24T10:46:05.938 INFO:tasks.workunit.client.1.smithi096.stdout:9/81: dread d9/db/f14 [0,4194304] 0 2024-03-24T10:46:05.938 INFO:tasks.workunit.client.1.smithi096.stdout:9/82: write d9/f13 [1408548,62579] 0 2024-03-24T10:46:05.938 INFO:tasks.workunit.client.1.smithi096.stdout:9/83: read d9/db/fe [2509821,50896] 0 2024-03-24T10:46:05.938 INFO:tasks.workunit.client.1.smithi096.stdout:9/84: creat d9/f18 x:0 0 0 2024-03-24T10:46:05.938 INFO:tasks.workunit.client.1.smithi096.stdout:9/85: dread - d9/f18 zero size 2024-03-24T10:46:05.940 INFO:tasks.workunit.client.1.smithi096.stdout:0/87: dwrite d0/d2/fb [0,4194304] 0 2024-03-24T10:46:05.940 INFO:tasks.workunit.client.1.smithi096.stdout:0/88: rename d0/d2 to d0/d2/d18 22 2024-03-24T10:46:05.940 INFO:tasks.workunit.client.1.smithi096.stdout:8/112: write d8/d11/d17/f1c [2894134,104182] 0 2024-03-24T10:46:05.944 INFO:tasks.workunit.client.1.smithi096.stdout:3/206: mknod d0/c4a 0 2024-03-24T10:46:05.945 INFO:tasks.workunit.client.1.smithi096.stdout:6/128: creat d8/d12/f2d x:0 0 0 2024-03-24T10:46:05.953 INFO:tasks.workunit.client.1.smithi096.stdout:9/86: unlink d9/d15/l17 0 2024-03-24T10:46:05.953 INFO:tasks.workunit.client.1.smithi096.stdout:9/87: fsync d9/db/f14 0 2024-03-24T10:46:05.953 INFO:tasks.workunit.client.1.smithi096.stdout:6/129: dread d8/f18 [0,4194304] 0 2024-03-24T10:46:05.953 INFO:tasks.workunit.client.1.smithi096.stdout:6/130: write f7 [4923473,104911] 0 2024-03-24T10:46:05.960 INFO:tasks.workunit.client.1.smithi096.stdout:8/113: unlink d8/d11/c13 0 2024-03-24T10:46:05.960 INFO:tasks.workunit.client.1.smithi096.stdout:5/112: dwrite d2/d7/d8/df/f1b [0,4194304] 0 2024-03-24T10:46:05.960 INFO:tasks.workunit.client.1.smithi096.stdout:5/113: creat d2/d7/f26 x:0 0 0 2024-03-24T10:46:05.963 INFO:tasks.workunit.client.1.smithi096.stdout:3/207: symlink d0/d9/d34/d39/d48/l4b 0 2024-03-24T10:46:05.969 INFO:tasks.workunit.client.1.smithi096.stdout:6/131: symlink d8/d10/d13/l2e 0 2024-03-24T10:46:05.969 INFO:tasks.workunit.client.1.smithi096.stdout:6/132: dread - d8/f26 zero size 2024-03-24T10:46:05.969 INFO:tasks.workunit.client.1.smithi096.stdout:6/133: dread - d8/f29 zero size 2024-03-24T10:46:05.970 INFO:tasks.workunit.client.1.smithi096.stdout:7/104: sync 2024-03-24T10:46:05.970 INFO:tasks.workunit.client.1.smithi096.stdout:7/105: creat d0/d21/f2d x:0 0 0 2024-03-24T10:46:05.971 INFO:tasks.workunit.client.1.smithi096.stdout:7/106: creat d0/d21/f2e x:0 0 0 2024-03-24T10:46:05.982 INFO:tasks.workunit.client.1.smithi096.stdout:5/114: symlink d2/d7/d8/df/d14/l27 0 2024-03-24T10:46:05.985 INFO:tasks.workunit.client.1.smithi096.stdout:3/208: creat d0/d13/d45/f4c x:0 0 0 2024-03-24T10:46:05.985 INFO:tasks.workunit.client.1.smithi096.stdout:3/209: read - d0/d9/d34/d39/f43 zero size 2024-03-24T10:46:05.985 INFO:tasks.workunit.client.1.smithi096.stdout:9/88: rmdir d9/db 39 2024-03-24T10:46:05.985 INFO:tasks.workunit.client.1.smithi096.stdout:9/89: fsync d9/db/f14 0 2024-03-24T10:46:05.987 INFO:tasks.workunit.client.1.smithi096.stdout:9/90: write d9/f13 [321954,82366] 0 2024-03-24T10:46:05.987 INFO:tasks.workunit.client.1.smithi096.stdout:9/91: creat d9/f19 x:0 0 0 2024-03-24T10:46:05.987 INFO:tasks.workunit.client.1.smithi096.stdout:6/134: symlink d8/d10/d13/l2f 0 2024-03-24T10:46:05.988 INFO:tasks.workunit.client.1.smithi096.stdout:9/92: write d9/f16 [12874,61902] 0 2024-03-24T10:46:05.988 INFO:tasks.workunit.client.1.smithi096.stdout:8/114: rename d8/d11/d17 to d8/d11/d22/d23 0 2024-03-24T10:46:05.988 INFO:tasks.workunit.client.1.smithi096.stdout:8/115: truncate d8/fc 1664861 0 2024-03-24T10:46:05.990 INFO:tasks.workunit.client.1.smithi096.stdout:9/93: fsync d9/f16 0 2024-03-24T10:46:05.990 INFO:tasks.workunit.client.1.smithi096.stdout:9/94: dread - d9/f18 zero size 2024-03-24T10:46:05.990 INFO:tasks.workunit.client.1.smithi096.stdout:9/95: creat d9/f1a x:0 0 0 2024-03-24T10:46:05.992 INFO:tasks.workunit.client.1.smithi096.stdout:7/107: link d0/d21/l2a d0/d21/l2f 0 2024-03-24T10:46:05.992 INFO:tasks.workunit.client.1.smithi096.stdout:7/108: read - d0/d13/f18 zero size 2024-03-24T10:46:05.993 INFO:tasks.workunit.client.1.smithi096.stdout:3/210: unlink d0/d9/c38 0 2024-03-24T10:46:05.998 INFO:tasks.workunit.client.1.smithi096.stdout:6/135: unlink d8/f26 0 2024-03-24T10:46:05.999 INFO:tasks.workunit.client.1.smithi096.stdout:6/136: dread d8/d12/f27 [0,4194304] 0 2024-03-24T10:46:06.002 INFO:tasks.workunit.client.1.smithi096.stdout:5/115: rename d2/d7/d8/lc to d2/d7/d8/d18/l28 0 2024-03-24T10:46:06.005 INFO:tasks.workunit.client.1.smithi096.stdout:8/116: mknod d8/d11/d22/d23/c24 0 2024-03-24T10:46:06.005 INFO:tasks.workunit.client.1.smithi096.stdout:8/117: write d8/f15 [874452,28532] 0 2024-03-24T10:46:06.010 INFO:tasks.workunit.client.1.smithi096.stdout:9/96: symlink d9/d15/l1b 0 2024-03-24T10:46:06.010 INFO:tasks.workunit.client.1.smithi096.stdout:9/97: write d9/f18 [409757,30735] 0 2024-03-24T10:46:06.010 INFO:tasks.workunit.client.1.smithi096.stdout:9/98: truncate d9/db/f14 1094082 0 2024-03-24T10:46:06.010 INFO:tasks.workunit.client.1.smithi096.stdout:9/99: readlink d9/d15/l12 0 2024-03-24T10:46:06.010 INFO:tasks.workunit.client.1.smithi096.stdout:9/100: truncate d9/f1a 478758 0 2024-03-24T10:46:06.011 INFO:tasks.workunit.client.1.smithi096.stdout:5/116: dread d2/d7/d8/df/f1b [0,4194304] 0 2024-03-24T10:46:06.011 INFO:tasks.workunit.client.1.smithi096.stdout:2/106: sync 2024-03-24T10:46:06.011 INFO:tasks.workunit.client.1.smithi096.stdout:2/107: creat d5/dc/f2f x:0 0 0 2024-03-24T10:46:06.011 INFO:tasks.workunit.client.1.smithi096.stdout:2/108: readlink d5/dc/d12/l1b 0 2024-03-24T10:46:06.011 INFO:tasks.workunit.client.1.smithi096.stdout:4/126: sync 2024-03-24T10:46:06.012 INFO:tasks.workunit.client.1.smithi096.stdout:1/94: sync 2024-03-24T10:46:06.012 INFO:tasks.workunit.client.1.smithi096.stdout:4/127: dread d1/fd [0,4194304] 0 2024-03-24T10:46:06.013 INFO:tasks.workunit.client.1.smithi096.stdout:1/95: read d3/fc [36091,14416] 0 2024-03-24T10:46:06.013 INFO:tasks.workunit.client.1.smithi096.stdout:1/96: chown d3/f9 3 1 2024-03-24T10:46:06.013 INFO:tasks.workunit.client.1.smithi096.stdout:7/109: mknod d0/d21/c30 0 2024-03-24T10:46:06.013 INFO:tasks.workunit.client.1.smithi096.stdout:8/118: dread d8/f12 [0,4194304] 0 2024-03-24T10:46:06.014 INFO:tasks.workunit.client.1.smithi096.stdout:3/211: dwrite d0/d13/f21 [0,4194304] 0 2024-03-24T10:46:06.015 INFO:tasks.workunit.client.1.smithi096.stdout:7/110: fsync d0/d21/f2d 0 2024-03-24T10:46:06.019 INFO:tasks.workunit.client.1.smithi096.stdout:6/137: rename f3 to d8/d10/d19/f30 0 2024-03-24T10:46:06.024 INFO:tasks.workunit.client.1.smithi096.stdout:2/109: symlink d5/dc/l30 0 2024-03-24T10:46:06.025 INFO:tasks.workunit.client.1.smithi096.stdout:2/110: dread - d5/f1f zero size 2024-03-24T10:46:06.034 INFO:tasks.workunit.client.1.smithi096.stdout:4/128: dwrite d1/f10 [0,4194304] 0 2024-03-24T10:46:06.034 INFO:tasks.workunit.client.1.smithi096.stdout:8/119: mkdir d8/d11/d22/d23/d1e/d25 0 2024-03-24T10:46:06.035 INFO:tasks.workunit.client.1.smithi096.stdout:1/97: dwrite d3/f13 [0,4194304] 0 2024-03-24T10:46:06.038 INFO:tasks.workunit.client.1.smithi096.stdout:5/117: mkdir d2/d29 0 2024-03-24T10:46:06.040 INFO:tasks.workunit.client.1.smithi096.stdout:2/111: unlink d5/dc/d12/l1b 0 2024-03-24T10:46:06.040 INFO:tasks.workunit.client.1.smithi096.stdout:2/112: creat d5/dc/d12/f31 x:0 0 0 2024-03-24T10:46:06.040 INFO:tasks.workunit.client.1.smithi096.stdout:2/113: dread - d5/dc/f21 zero size 2024-03-24T10:46:06.042 INFO:tasks.workunit.client.1.smithi096.stdout:4/129: write d1/f10 [3821244,83882] 0 2024-03-24T10:46:06.046 INFO:tasks.workunit.client.1.smithi096.stdout:5/118: dread d2/f17 [0,4194304] 0 2024-03-24T10:46:06.046 INFO:tasks.workunit.client.1.smithi096.stdout:5/119: chown d2/d7/d8/lb 7732 1 2024-03-24T10:46:06.046 INFO:tasks.workunit.client.1.smithi096.stdout:4/130: write d1/f3 [7217902,84739] 0 2024-03-24T10:46:06.046 INFO:tasks.workunit.client.1.smithi096.stdout:4/131: fdatasync d1/fe 0 2024-03-24T10:46:06.046 INFO:tasks.workunit.client.1.smithi096.stdout:4/132: chown d1/f24 15 1 2024-03-24T10:46:06.046 INFO:tasks.workunit.client.1.smithi096.stdout:5/120: creat d2/d7/f2a x:0 0 0 2024-03-24T10:46:06.047 INFO:tasks.workunit.client.1.smithi096.stdout:4/133: dread d1/fd [0,4194304] 0 2024-03-24T10:46:06.048 INFO:tasks.workunit.client.1.smithi096.stdout:7/111: dwrite d0/d21/f2d [0,4194304] 0 2024-03-24T10:46:06.049 INFO:tasks.workunit.client.1.smithi096.stdout:6/138: dwrite f7 [0,4194304] 0 2024-03-24T10:46:06.051 INFO:tasks.workunit.client.1.smithi096.stdout:8/120: creat d8/d11/d22/d23/d1a/f26 x:0 0 0 2024-03-24T10:46:06.053 INFO:tasks.workunit.client.1.smithi096.stdout:9/101: rename c6 to d9/db/c1c 0 2024-03-24T10:46:06.060 INFO:tasks.workunit.client.1.smithi096.stdout:3/212: truncate d0/d13/f21 853755 0 2024-03-24T10:46:06.060 INFO:tasks.workunit.client.1.smithi096.stdout:3/213: fsync d0/d13/d1e/f2c 0 2024-03-24T10:46:06.062 INFO:tasks.workunit.client.1.smithi096.stdout:1/98: creat d3/f21 x:0 0 0 2024-03-24T10:46:06.065 INFO:tasks.workunit.client.1.smithi096.stdout:2/114: creat d5/de/d2a/f32 x:0 0 0 2024-03-24T10:46:06.065 INFO:tasks.workunit.client.1.smithi096.stdout:2/115: fsync d5/de/d2a/f2b 0 2024-03-24T10:46:06.074 INFO:tasks.workunit.client.1.smithi096.stdout:7/112: rmdir d0 39 2024-03-24T10:46:06.074 INFO:tasks.workunit.client.1.smithi096.stdout:7/113: truncate d0/d13/f15 495040 0 2024-03-24T10:46:06.075 INFO:tasks.workunit.client.1.smithi096.stdout:6/139: creat d8/f31 x:0 0 0 2024-03-24T10:46:06.078 INFO:tasks.workunit.client.1.smithi096.stdout:5/121: dwrite d2/d7/d8/df/f1b [0,4194304] 0 2024-03-24T10:46:06.078 INFO:tasks.workunit.client.1.smithi096.stdout:3/214: dwrite d0/d9/f2e [0,4194304] 0 2024-03-24T10:46:06.086 INFO:tasks.workunit.client.1.smithi096.stdout:6/140: dread d8/f1c [0,4194304] 0 2024-03-24T10:46:06.087 INFO:tasks.workunit.client.1.smithi096.stdout:1/99: creat d3/f22 x:0 0 0 2024-03-24T10:46:06.087 INFO:tasks.workunit.client.1.smithi096.stdout:5/122: write d2/f3 [1395995,87700] 0 2024-03-24T10:46:06.088 INFO:tasks.workunit.client.1.smithi096.stdout:2/116: symlink d5/dc/d12/d23/l33 0 2024-03-24T10:46:06.095 INFO:tasks.workunit.client.1.smithi096.stdout:3/215: dread d0/d13/d1e/f2a [0,4194304] 0 2024-03-24T10:46:06.101 INFO:tasks.workunit.client.1.smithi096.stdout:8/121: rmdir d8 39 2024-03-24T10:46:06.101 INFO:tasks.workunit.client.1.smithi096.stdout:8/122: chown d8/d11/d22/d23 318 1 2024-03-24T10:46:06.101 INFO:tasks.workunit.client.1.smithi096.stdout:8/123: truncate d8/f15 1735681 0 2024-03-24T10:46:06.102 INFO:tasks.workunit.client.1.smithi096.stdout:9/102: link d9/db/c1c d9/db/c1d 0 2024-03-24T10:46:06.104 INFO:tasks.workunit.client.1.smithi096.stdout:6/141: stat d8/d10/c25 0 2024-03-24T10:46:06.107 INFO:tasks.workunit.client.1.smithi096.stdout:5/123: symlink d2/d7/l2b 0 2024-03-24T10:46:06.107 INFO:tasks.workunit.client.1.smithi096.stdout:2/117: rmdir d5/de 39 2024-03-24T10:46:06.107 INFO:tasks.workunit.client.1.smithi096.stdout:2/118: write d5/dc/f2f [901176,85828] 0 2024-03-24T10:46:06.109 INFO:tasks.workunit.client.1.smithi096.stdout:3/216: dwrite d0/d9/f2e [0,4194304] 0 2024-03-24T10:46:06.110 INFO:tasks.workunit.client.1.smithi096.stdout:7/114: rename d0/d21/f2d to d0/db/d26/f31 0 2024-03-24T10:46:06.110 INFO:tasks.workunit.client.1.smithi096.stdout:7/115: fsync d0/db/d1c/f29 0 2024-03-24T10:46:06.111 INFO:tasks.workunit.client.1.smithi096.stdout:8/124: creat d8/d11/f27 x:0 0 0 2024-03-24T10:46:06.111 INFO:tasks.workunit.client.1.smithi096.stdout:7/116: dread d0/d13/f15 [0,4194304] 0 2024-03-24T10:46:06.111 INFO:tasks.workunit.client.1.smithi096.stdout:7/117: truncate d0/f20 881102 0 2024-03-24T10:46:06.113 INFO:tasks.workunit.client.1.smithi096.stdout:9/103: creat d9/d15/f1e x:0 0 0 2024-03-24T10:46:06.114 INFO:tasks.workunit.client.1.smithi096.stdout:1/100: mkdir d3/d23 0 2024-03-24T10:46:06.114 INFO:tasks.workunit.client.1.smithi096.stdout:1/101: fsync d3/f1f 0 2024-03-24T10:46:06.114 INFO:tasks.workunit.client.1.smithi096.stdout:1/102: fdatasync d3/f9 0 2024-03-24T10:46:06.117 INFO:tasks.workunit.client.1.smithi096.stdout:2/119: write d5/fb [412265,47202] 0 2024-03-24T10:46:06.123 INFO:tasks.workunit.client.1.smithi096.stdout:5/124: write d2/d7/d8/df/d14/f23 [2430602,23616] 0 2024-03-24T10:46:06.127 INFO:tasks.workunit.client.1.smithi096.stdout:8/125: creat d8/d11/d22/d23/d1d/f28 x:0 0 0 2024-03-24T10:46:06.128 INFO:tasks.workunit.client.1.smithi096.stdout:7/118: unlink d0/f1f 0 2024-03-24T10:46:06.129 INFO:tasks.workunit.client.1.smithi096.stdout:6/142: mkdir d8/d12/d32 0 2024-03-24T10:46:06.132 INFO:tasks.workunit.client.1.smithi096.stdout:1/103: mkdir d3/d24 0 2024-03-24T10:46:06.132 INFO:tasks.workunit.client.1.smithi096.stdout:1/104: chown c2 30 1 2024-03-24T10:46:06.134 INFO:tasks.workunit.client.1.smithi096.stdout:6/143: getdents d8 0 2024-03-24T10:46:06.134 INFO:tasks.workunit.client.1.smithi096.stdout:7/119: read d0/d13/f9 [359636,52283] 0 2024-03-24T10:46:06.136 INFO:tasks.workunit.client.1.smithi096.stdout:3/217: dwrite d0/f1 [0,4194304] 0 2024-03-24T10:46:06.136 INFO:tasks.workunit.client.1.smithi096.stdout:3/218: dread - d0/f44 zero size 2024-03-24T10:46:06.136 INFO:tasks.workunit.client.1.smithi096.stdout:6/144: creat d8/d12/d28/f33 x:0 0 0 2024-03-24T10:46:06.137 INFO:tasks.workunit.client.1.smithi096.stdout:9/104: dwrite d9/f16 [0,4194304] 0 2024-03-24T10:46:06.139 INFO:tasks.workunit.client.1.smithi096.stdout:3/219: mknod d0/d13/d45/c4d 0 2024-03-24T10:46:06.147 INFO:tasks.workunit.client.1.smithi096.stdout:9/105: mknod d9/c1f 0 2024-03-24T10:46:06.147 INFO:tasks.workunit.client.1.smithi096.stdout:9/106: creat d9/d15/f20 x:0 0 0 2024-03-24T10:46:06.147 INFO:tasks.workunit.client.1.smithi096.stdout:9/107: creat d9/d15/f21 x:0 0 0 2024-03-24T10:46:06.148 INFO:tasks.workunit.client.1.smithi096.stdout:7/120: write d0/db/f1b [777589,42963] 0 2024-03-24T10:46:06.148 INFO:tasks.workunit.client.1.smithi096.stdout:3/220: creat d0/d13/d45/d47/f4e x:0 0 0 2024-03-24T10:46:06.149 INFO:tasks.workunit.client.1.smithi096.stdout:6/145: write d8/d10/d19/f30 [1107931,87927] 0 2024-03-24T10:46:06.150 INFO:tasks.workunit.client.1.smithi096.stdout:6/146: chown d8/d10/d22/f2c 425639 1 2024-03-24T10:46:06.150 INFO:tasks.workunit.client.1.smithi096.stdout:6/147: creat d8/d10/d19/f34 x:0 0 0 2024-03-24T10:46:06.150 INFO:tasks.workunit.client.1.smithi096.stdout:6/148: chown d8/d10/d19/f30 41 1 2024-03-24T10:46:06.155 INFO:tasks.workunit.client.1.smithi096.stdout:5/125: dwrite d2/d7/f26 [0,4194304] 0 2024-03-24T10:46:06.155 INFO:tasks.workunit.client.1.smithi096.stdout:5/126: readlink d2/d7/l2b 0 2024-03-24T10:46:06.156 INFO:tasks.workunit.client.1.smithi096.stdout:8/126: dwrite d8/d11/f19 [0,4194304] 0 2024-03-24T10:46:06.156 INFO:tasks.workunit.client.1.smithi096.stdout:8/127: creat d8/d11/f29 x:0 0 0 2024-03-24T10:46:06.158 INFO:tasks.workunit.client.1.smithi096.stdout:7/121: mkdir d0/d13/d32 0 2024-03-24T10:46:06.160 INFO:tasks.workunit.client.1.smithi096.stdout:7/122: dread d0/f20 [0,4194304] 0 2024-03-24T10:46:06.160 INFO:tasks.workunit.client.1.smithi096.stdout:7/123: fdatasync d0/d13/f2c 0 2024-03-24T10:46:06.162 INFO:tasks.workunit.client.1.smithi096.stdout:6/149: dread d8/d10/d19/f1b [0,4194304] 0 2024-03-24T10:46:06.166 INFO:tasks.workunit.client.1.smithi096.stdout:5/127: unlink d2/d7/d8/df/d14/l1e 0 2024-03-24T10:46:06.167 INFO:tasks.workunit.client.1.smithi096.stdout:1/105: dwrite d3/f6 [0,4194304] 0 2024-03-24T10:46:06.170 INFO:tasks.workunit.client.1.smithi096.stdout:9/108: dwrite d9/f1a [0,4194304] 0 2024-03-24T10:46:06.173 INFO:tasks.workunit.client.1.smithi096.stdout:8/128: mknod d8/d11/d22/d23/c2a 0 2024-03-24T10:46:06.176 INFO:tasks.workunit.client.1.smithi096.stdout:7/124: symlink d0/db/l33 0 2024-03-24T10:46:06.176 INFO:tasks.workunit.client.1.smithi096.stdout:7/125: fsync d0/d13/f15 0 2024-03-24T10:46:06.177 INFO:tasks.workunit.client.1.smithi096.stdout:7/126: dread - d0/d13/f2c zero size 2024-03-24T10:46:06.177 INFO:tasks.workunit.client.1.smithi096.stdout:6/150: dread d8/d10/d19/f30 [0,4194304] 0 2024-03-24T10:46:06.177 INFO:tasks.workunit.client.1.smithi096.stdout:6/151: dread - d8/d10/f2a zero size 2024-03-24T10:46:06.178 INFO:tasks.workunit.client.1.smithi096.stdout:5/128: getdents d2/d7/d8/d18 0 2024-03-24T10:46:06.179 INFO:tasks.workunit.client.1.smithi096.stdout:6/152: dread d8/d12/f27 [0,4194304] 0 2024-03-24T10:46:06.179 INFO:tasks.workunit.client.1.smithi096.stdout:6/153: write d8/d10/fd [1637759,107535] 0 2024-03-24T10:46:06.186 INFO:tasks.workunit.client.1.smithi096.stdout:7/127: dread d0/d13/f5 [0,4194304] 0 2024-03-24T10:46:06.186 INFO:tasks.workunit.client.1.smithi096.stdout:8/129: link f1 d8/d11/d22/d23/d1d/f2b 0 2024-03-24T10:46:06.187 INFO:tasks.workunit.client.1.smithi096.stdout:5/129: truncate d2/d7/d8/f12 1982068 0 2024-03-24T10:46:06.190 INFO:tasks.workunit.client.1.smithi096.stdout:6/154: symlink d8/d10/l35 0 2024-03-24T10:46:06.190 INFO:tasks.workunit.client.1.smithi096.stdout:6/155: truncate d8/d10/d19/f34 179114 0 2024-03-24T10:46:06.190 INFO:tasks.workunit.client.1.smithi096.stdout:6/156: write d8/d10/f2a [604715,130118] 0 2024-03-24T10:46:06.190 INFO:tasks.workunit.client.1.smithi096.stdout:1/106: rmdir d3 39 2024-03-24T10:46:06.192 INFO:tasks.workunit.client.1.smithi096.stdout:9/109: getdents d9 0 2024-03-24T10:46:06.195 INFO:tasks.workunit.client.1.smithi096.stdout:6/157: mkdir d8/d12/d28/d36 0 2024-03-24T10:46:06.195 INFO:tasks.workunit.client.1.smithi096.stdout:6/158: readlink d8/d10/d13/l2f 0 2024-03-24T10:46:06.196 INFO:tasks.workunit.client.1.smithi096.stdout:6/159: read - d8/d10/d22/f2c zero size 2024-03-24T10:46:06.196 INFO:tasks.workunit.client.1.smithi096.stdout:6/160: truncate d8/f31 429821 0 2024-03-24T10:46:06.197 INFO:tasks.workunit.client.1.smithi096.stdout:6/161: truncate f6 269073 0 2024-03-24T10:46:06.197 INFO:tasks.workunit.client.1.smithi096.stdout:6/162: chown d8/d12/d32 1710 1 2024-03-24T10:46:06.197 INFO:tasks.workunit.client.1.smithi096.stdout:9/110: dread d9/db/fe [0,4194304] 0 2024-03-24T10:46:06.197 INFO:tasks.workunit.client.1.smithi096.stdout:6/163: chown d8/d12/f2d 3536 1 2024-03-24T10:46:06.198 INFO:tasks.workunit.client.1.smithi096.stdout:9/111: truncate d9/db/f14 1650101 0 2024-03-24T10:46:06.198 INFO:tasks.workunit.client.1.smithi096.stdout:9/112: chown d9/d15 59663 1 2024-03-24T10:46:06.199 INFO:tasks.workunit.client.1.smithi096.stdout:6/164: creat d8/d12/f37 x:0 0 0 2024-03-24T10:46:06.199 INFO:tasks.workunit.client.1.smithi096.stdout:6/165: dread - d8/d12/f2d zero size 2024-03-24T10:46:06.199 INFO:tasks.workunit.client.1.smithi096.stdout:6/166: readlink d8/d10/d13/l2e 0 2024-03-24T10:46:06.199 INFO:tasks.workunit.client.1.smithi096.stdout:6/167: getdents d8/d12/d28/d36 0 2024-03-24T10:46:06.199 INFO:tasks.workunit.client.1.smithi096.stdout:6/168: fdatasync d8/d12/f37 0 2024-03-24T10:46:06.200 INFO:tasks.workunit.client.1.smithi096.stdout:9/113: creat d9/db/f22 x:0 0 0 2024-03-24T10:46:06.202 INFO:tasks.workunit.client.1.smithi096.stdout:6/169: mkdir d8/d12/d32/d38 0 2024-03-24T10:46:06.206 INFO:tasks.workunit.client.1.smithi096.stdout:8/130: write d8/d11/f19 [950701,50160] 0 2024-03-24T10:46:06.207 INFO:tasks.workunit.client.1.smithi096.stdout:6/170: getdents d8/d10/d19 0 2024-03-24T10:46:06.209 INFO:tasks.workunit.client.1.smithi096.stdout:9/114: dread d9/f16 [0,4194304] 0 2024-03-24T10:46:06.210 INFO:tasks.workunit.client.1.smithi096.stdout:6/171: symlink d8/d12/d28/l39 0 2024-03-24T10:46:06.210 INFO:tasks.workunit.client.1.smithi096.stdout:6/172: chown d8/d10/d19/f30 2383826 1 2024-03-24T10:46:06.211 INFO:tasks.workunit.client.1.smithi096.stdout:6/173: rename d8 to d8/d10/d3a 22 2024-03-24T10:46:06.211 INFO:tasks.workunit.client.1.smithi096.stdout:6/174: write d8/f1c [4762122,26169] 0 2024-03-24T10:46:06.211 INFO:tasks.workunit.client.1.smithi096.stdout:6/175: fdatasync d8/d10/d19/f34 0 2024-03-24T10:46:06.211 INFO:tasks.workunit.client.1.smithi096.stdout:7/128: dwrite d0/db/f1b [4194304,4194304] 0 2024-03-24T10:46:06.213 INFO:tasks.workunit.client.1.smithi096.stdout:8/131: getdents d8/d11/d22/d23 0 2024-03-24T10:46:06.221 INFO:tasks.workunit.client.1.smithi096.stdout:7/129: link d0/d13/l4 d0/d13/d32/l34 0 2024-03-24T10:46:06.221 INFO:tasks.workunit.client.1.smithi096.stdout:7/130: creat d0/d13/f35 x:0 0 0 2024-03-24T10:46:06.221 INFO:tasks.workunit.client.1.smithi096.stdout:7/131: readlink d0/d21/l22 0 2024-03-24T10:46:06.222 INFO:tasks.workunit.client.1.smithi096.stdout:7/132: dread d0/f14 [0,4194304] 0 2024-03-24T10:46:06.222 INFO:tasks.workunit.client.1.smithi096.stdout:7/133: stat d0/db/d1c 0 2024-03-24T10:46:06.222 INFO:tasks.workunit.client.1.smithi096.stdout:1/107: dwrite d3/f9 [0,4194304] 0 2024-03-24T10:46:06.222 INFO:tasks.workunit.client.1.smithi096.stdout:1/108: getdents d3/d24 0 2024-03-24T10:46:06.224 INFO:tasks.workunit.client.1.smithi096.stdout:9/115: mkdir d9/db/d23 0 2024-03-24T10:46:06.226 INFO:tasks.workunit.client.1.smithi096.stdout:5/130: dread d2/d7/d8/f12 [0,4194304] 0 2024-03-24T10:46:06.237 INFO:tasks.workunit.client.1.smithi096.stdout:8/132: truncate d8/f12 1014340 0 2024-03-24T10:46:06.238 INFO:tasks.workunit.client.1.smithi096.stdout:6/176: truncate d8/f1c 2437065 0 2024-03-24T10:46:06.246 INFO:tasks.workunit.client.1.smithi096.stdout:7/134: rename d0/d13/ca to d0/db/c36 0 2024-03-24T10:46:06.246 INFO:tasks.workunit.client.1.smithi096.stdout:8/133: dread d8/d11/d22/d23/d1d/f2b [8388608,4194304] 0 2024-03-24T10:46:06.252 INFO:tasks.workunit.client.1.smithi096.stdout:9/116: creat d9/f24 x:0 0 0 2024-03-24T10:46:06.252 INFO:tasks.workunit.client.1.smithi096.stdout:9/117: creat d9/d15/f25 x:0 0 0 2024-03-24T10:46:06.253 INFO:tasks.workunit.client.1.smithi096.stdout:6/177: rmdir d8/d10/d19 39 2024-03-24T10:46:06.253 INFO:tasks.workunit.client.1.smithi096.stdout:6/178: chown d8/f18 7 1 2024-03-24T10:46:06.256 INFO:tasks.workunit.client.1.smithi096.stdout:7/135: creat d0/db/d26/f37 x:0 0 0 2024-03-24T10:46:06.256 INFO:tasks.workunit.client.1.smithi096.stdout:7/136: dread - d0/d13/f18 zero size 2024-03-24T10:46:06.258 INFO:tasks.workunit.client.1.smithi096.stdout:8/134: rename d8/d11/f29 to d8/d11/d22/d23/d1e/f2c 0 2024-03-24T10:46:06.265 INFO:tasks.workunit.client.1.smithi096.stdout:7/137: creat d0/f38 x:0 0 0 2024-03-24T10:46:06.269 INFO:tasks.workunit.client.1.smithi096.stdout:8/135: rmdir d8/d11/d22/d23/d1e 39 2024-03-24T10:46:06.271 INFO:tasks.workunit.client.1.smithi096.stdout:6/179: dwrite d8/f9 [4194304,4194304] 0 2024-03-24T10:46:06.275 INFO:tasks.workunit.client.1.smithi096.stdout:7/138: mkdir d0/db/d1c/d39 0 2024-03-24T10:46:06.277 INFO:tasks.workunit.client.1.smithi096.stdout:7/139: dread - d0/d21/f27 zero size 2024-03-24T10:46:06.278 INFO:tasks.workunit.client.1.smithi096.stdout:9/118: rmdir d9/d15 39 2024-03-24T10:46:06.282 INFO:tasks.workunit.client.1.smithi096.stdout:8/136: creat d8/d11/d22/d23/d1a/f2d x:0 0 0 2024-03-24T10:46:06.285 INFO:tasks.workunit.client.1.smithi096.stdout:1/109: dwrite d3/f9 [4194304,4194304] 0 2024-03-24T10:46:06.286 INFO:tasks.workunit.client.1.smithi096.stdout:6/180: unlink d8/d10/fd 0 2024-03-24T10:46:06.288 INFO:tasks.workunit.client.1.smithi096.stdout:8/137: mkdir d8/d11/d22/d23/d1e/d25/d2e 0 2024-03-24T10:46:06.290 INFO:tasks.workunit.client.1.smithi096.stdout:1/110: symlink d3/d23/l25 0 2024-03-24T10:46:06.290 INFO:tasks.workunit.client.1.smithi096.stdout:1/111: write d3/ff [768976,118001] 0 2024-03-24T10:46:06.291 INFO:tasks.workunit.client.1.smithi096.stdout:1/112: write d3/f9 [8759541,89878] 0 2024-03-24T10:46:06.295 INFO:tasks.workunit.client.1.smithi096.stdout:6/181: getdents d8/d12 0 2024-03-24T10:46:06.295 INFO:tasks.workunit.client.1.smithi096.stdout:7/140: dwrite d0/d13/f15 [0,4194304] 0 2024-03-24T10:46:06.298 INFO:tasks.workunit.client.1.smithi096.stdout:6/182: dread d8/d10/d19/f34 [0,4194304] 0 2024-03-24T10:46:06.298 INFO:tasks.workunit.client.1.smithi096.stdout:6/183: chown d8/d12 25169713 1 2024-03-24T10:46:06.298 INFO:tasks.workunit.client.1.smithi096.stdout:6/184: creat d8/d10/f3b x:0 0 0 2024-03-24T10:46:06.298 INFO:tasks.workunit.client.1.smithi096.stdout:6/185: write d8/d10/d22/f21 [1390889,50144] 0 2024-03-24T10:46:06.298 INFO:tasks.workunit.client.1.smithi096.stdout:9/119: dwrite d9/d15/f1e [0,4194304] 0 2024-03-24T10:46:06.298 INFO:tasks.workunit.client.1.smithi096.stdout:9/120: write d9/f18 [804377,122384] 0 2024-03-24T10:46:06.301 INFO:tasks.workunit.client.1.smithi096.stdout:8/138: dread f1 [4194304,4194304] 0 2024-03-24T10:46:06.308 INFO:tasks.workunit.client.1.smithi096.stdout:6/186: rename d8/d10/f2a to d8/d12/d32/f3c 0 2024-03-24T10:46:06.308 INFO:tasks.workunit.client.1.smithi096.stdout:6/187: getdents d8 0 2024-03-24T10:46:06.309 INFO:tasks.workunit.client.1.smithi096.stdout:6/188: dread - d8/d12/f2d zero size 2024-03-24T10:46:06.309 INFO:tasks.workunit.client.1.smithi096.stdout:6/189: creat d8/d10/f3d x:0 0 0 2024-03-24T10:46:06.309 INFO:tasks.workunit.client.1.smithi096.stdout:6/190: chown d8/d12/c1d 12672 1 2024-03-24T10:46:06.320 INFO:tasks.workunit.client.1.smithi096.stdout:8/139: rmdir d8/d11/d22/d23/d1a 39 2024-03-24T10:46:06.320 INFO:tasks.workunit.client.1.smithi096.stdout:8/140: chown d8/fe 225 1 2024-03-24T10:46:06.327 INFO:tasks.workunit.client.1.smithi096.stdout:7/141: dwrite d0/d13/f35 [0,4194304] 0 2024-03-24T10:46:06.327 INFO:tasks.workunit.client.1.smithi096.stdout:7/142: dread - d0/f24 zero size 2024-03-24T10:46:06.327 INFO:tasks.workunit.client.1.smithi096.stdout:1/113: dwrite d3/ff [0,4194304] 0 2024-03-24T10:46:06.327 INFO:tasks.workunit.client.1.smithi096.stdout:6/191: dwrite d8/d10/f3b [0,4194304] 0 2024-03-24T10:46:06.332 INFO:tasks.workunit.client.1.smithi096.stdout:8/141: read d8/f16 [2626907,114572] 0 2024-03-24T10:46:06.332 INFO:tasks.workunit.client.1.smithi096.stdout:8/142: dread - d8/d11/d22/d23/d1a/f2d zero size 2024-03-24T10:46:06.332 INFO:tasks.workunit.client.1.smithi096.stdout:8/143: creat d8/d11/d22/d23/d1d/f2f x:0 0 0 2024-03-24T10:46:06.337 INFO:tasks.workunit.client.1.smithi096.stdout:7/143: dread d0/d13/f15 [0,4194304] 0 2024-03-24T10:46:06.342 INFO:tasks.workunit.client.1.smithi096.stdout:6/192: truncate d8/d10/d19/f30 4234822 0 2024-03-24T10:46:06.342 INFO:tasks.workunit.client.1.smithi096.stdout:6/193: creat d8/d10/f3e x:0 0 0 2024-03-24T10:46:06.342 INFO:tasks.workunit.client.1.smithi096.stdout:6/194: creat d8/d10/f3f x:0 0 0 2024-03-24T10:46:06.346 INFO:tasks.workunit.client.1.smithi096.stdout:8/144: rename d8/f12 to d8/d11/d22/d23/d1d/f30 0 2024-03-24T10:46:06.346 INFO:tasks.workunit.client.1.smithi096.stdout:8/145: write d8/f14 [783533,91911] 0 2024-03-24T10:46:06.346 INFO:tasks.workunit.client.1.smithi096.stdout:1/114: link d3/c1d d3/c26 0 2024-03-24T10:46:06.355 INFO:tasks.workunit.client.1.smithi096.stdout:8/146: unlink d8/d11/d22/d23/c2a 0 2024-03-24T10:46:06.355 INFO:tasks.workunit.client.1.smithi096.stdout:8/147: dread - d8/d11/d22/d23/d1d/f28 zero size 2024-03-24T10:46:06.355 INFO:tasks.workunit.client.1.smithi096.stdout:8/148: chown d8/d11/d22/d23/d1a/f26 46950968 1 2024-03-24T10:46:06.359 INFO:tasks.workunit.client.1.smithi096.stdout:7/144: getdents d0/d21 0 2024-03-24T10:46:06.359 INFO:tasks.workunit.client.1.smithi096.stdout:7/145: fsync d0/f24 0 2024-03-24T10:46:06.359 INFO:tasks.workunit.client.1.smithi096.stdout:7/146: write d0/d13/f2c [727896,2084] 0 2024-03-24T10:46:06.361 INFO:tasks.workunit.client.1.smithi096.stdout:1/115: mkdir d3/d27 0 2024-03-24T10:46:06.364 INFO:tasks.workunit.client.1.smithi096.stdout:1/116: unlink d3/fc 0 2024-03-24T10:46:06.370 INFO:tasks.workunit.client.1.smithi096.stdout:8/149: getdents d8/d11/d22/d23 0 2024-03-24T10:46:06.371 INFO:tasks.workunit.client.1.smithi096.stdout:1/117: link d3/l1a d3/l28 0 2024-03-24T10:46:06.373 INFO:tasks.workunit.client.1.smithi096.stdout:7/147: dwrite d0/d13/f18 [0,4194304] 0 2024-03-24T10:46:06.374 INFO:tasks.workunit.client.1.smithi096.stdout:7/148: chown d0/d13/f9 0 1 2024-03-24T10:46:06.374 INFO:tasks.workunit.client.1.smithi096.stdout:7/149: chown d0/db/d1c/d39 905 1 2024-03-24T10:46:06.374 INFO:tasks.workunit.client.1.smithi096.stdout:1/118: symlink d3/d24/l29 0 2024-03-24T10:46:06.374 INFO:tasks.workunit.client.1.smithi096.stdout:8/150: dread d8/f14 [0,4194304] 0 2024-03-24T10:46:06.382 INFO:tasks.workunit.client.1.smithi096.stdout:1/119: dread d3/ff [0,4194304] 0 2024-03-24T10:46:06.383 INFO:tasks.workunit.client.1.smithi096.stdout:7/150: mkdir d0/db/d3a 0 2024-03-24T10:46:06.385 INFO:tasks.workunit.client.1.smithi096.stdout:1/120: symlink d3/d27/l2a 0 2024-03-24T10:46:06.387 INFO:tasks.workunit.client.1.smithi096.stdout:7/151: creat d0/d13/d32/f3b x:0 0 0 2024-03-24T10:46:06.387 INFO:tasks.workunit.client.1.smithi096.stdout:7/152: chown d0/db/d26/f37 0 1 2024-03-24T10:46:06.391 INFO:tasks.workunit.client.1.smithi096.stdout:7/153: mkdir d0/d21/d3c 0 2024-03-24T10:46:06.391 INFO:tasks.workunit.client.1.smithi096.stdout:7/154: write d0/f14 [1392339,51243] 0 2024-03-24T10:46:06.392 INFO:tasks.workunit.client.1.smithi096.stdout:7/155: truncate d0/db/ff 516934 0 2024-03-24T10:46:06.392 INFO:tasks.workunit.client.1.smithi096.stdout:8/151: dwrite d8/f15 [0,4194304] 0 2024-03-24T10:46:06.392 INFO:tasks.workunit.client.1.smithi096.stdout:8/152: fdatasync d8/f15 0 2024-03-24T10:46:06.393 INFO:tasks.workunit.client.1.smithi096.stdout:1/121: getdents d3 0 2024-03-24T10:46:06.399 INFO:tasks.workunit.client.1.smithi096.stdout:1/122: mknod d3/d23/c2b 0 2024-03-24T10:46:06.399 INFO:tasks.workunit.client.1.smithi096.stdout:7/156: write d0/db/f1b [6063313,127417] 0 2024-03-24T10:46:06.400 INFO:tasks.workunit.client.1.smithi096.stdout:8/153: dread d8/f15 [0,4194304] 0 2024-03-24T10:46:06.400 INFO:tasks.workunit.client.1.smithi096.stdout:8/154: readlink l6 0 2024-03-24T10:46:06.402 INFO:tasks.workunit.client.1.smithi096.stdout:1/123: rename d3/l16 to d3/d23/l2c 0 2024-03-24T10:46:06.409 INFO:tasks.workunit.client.1.smithi096.stdout:8/155: link d8/d11/d22/d23/d1d/f2b d8/d11/f31 0 2024-03-24T10:46:06.409 INFO:tasks.workunit.client.1.smithi096.stdout:8/156: fsync d8/d11/d22/d23/d1d/f28 0 2024-03-24T10:46:06.409 INFO:tasks.workunit.client.1.smithi096.stdout:8/157: write d8/d11/d22/d23/d1d/f2f [286966,54498] 0 2024-03-24T10:46:06.409 INFO:tasks.workunit.client.1.smithi096.stdout:8/158: chown d8/d11/d22/d23/d1a/f2d 54 1 2024-03-24T10:46:06.410 INFO:tasks.workunit.client.1.smithi096.stdout:8/159: read d8/fc [895511,112424] 0 2024-03-24T10:46:06.410 INFO:tasks.workunit.client.1.smithi096.stdout:8/160: symlink d8/d11/d22/d23/d1d/l32 0 2024-03-24T10:46:06.411 INFO:tasks.workunit.client.1.smithi096.stdout:1/124: read d3/f13 [1888303,44443] 0 2024-03-24T10:46:06.414 INFO:tasks.workunit.client.1.smithi096.stdout:8/161: creat d8/d11/d22/d23/d1e/f33 x:0 0 0 2024-03-24T10:46:06.414 INFO:tasks.workunit.client.1.smithi096.stdout:8/162: chown d8/d11/d22/d23 117495 1 2024-03-24T10:46:06.414 INFO:tasks.workunit.client.1.smithi096.stdout:8/163: write d8/d11/d22/d23/d1d/f28 [984086,102263] 0 2024-03-24T10:46:06.414 INFO:tasks.workunit.client.1.smithi096.stdout:7/157: dwrite d0/d13/f15 [0,4194304] 0 2024-03-24T10:46:06.414 INFO:tasks.workunit.client.1.smithi096.stdout:1/125: symlink d3/d27/l2d 0 2024-03-24T10:46:06.418 INFO:tasks.workunit.client.1.smithi096.stdout:8/164: rename l7 to d8/d11/d22/d23/d1e/l34 0 2024-03-24T10:46:06.419 INFO:tasks.workunit.client.1.smithi096.stdout:8/165: readlink l2 0 2024-03-24T10:46:06.422 INFO:tasks.workunit.client.1.smithi096.stdout:8/166: mknod d8/d11/d22/d23/d1e/d25/c35 0 2024-03-24T10:46:06.423 INFO:tasks.workunit.client.1.smithi096.stdout:7/158: dread d0/db/f1b [4194304,4194304] 0 2024-03-24T10:46:06.423 INFO:tasks.workunit.client.1.smithi096.stdout:7/159: chown d0/d13 1537 1 2024-03-24T10:46:06.424 INFO:tasks.workunit.client.1.smithi096.stdout:7/160: write d0/d13/f2c [380942,81263] 0 2024-03-24T10:46:06.424 INFO:tasks.workunit.client.1.smithi096.stdout:7/161: readlink d0/l10 0 2024-03-24T10:46:06.424 INFO:tasks.workunit.client.1.smithi096.stdout:7/162: chown d0/f38 39 1 2024-03-24T10:46:06.427 INFO:tasks.workunit.client.1.smithi096.stdout:8/167: dread d8/f15 [0,4194304] 0 2024-03-24T10:46:06.431 INFO:tasks.workunit.client.1.smithi096.stdout:8/168: mknod d8/d11/d22/d23/c36 0 2024-03-24T10:46:06.433 INFO:tasks.workunit.client.1.smithi096.stdout:1/126: dwrite d3/f6 [0,4194304] 0 2024-03-24T10:46:06.435 INFO:tasks.workunit.client.1.smithi096.stdout:8/169: mkdir d8/d11/d22/d23/d37 0 2024-03-24T10:46:06.435 INFO:tasks.workunit.client.1.smithi096.stdout:8/170: write d8/d11/f19 [4909114,92302] 0 2024-03-24T10:46:06.435 INFO:tasks.workunit.client.1.smithi096.stdout:8/171: readlink d8/la 0 2024-03-24T10:46:06.435 INFO:tasks.workunit.client.1.smithi096.stdout:8/172: write d8/d11/d22/d23/d1e/f33 [549894,116216] 0 2024-03-24T10:46:06.435 INFO:tasks.workunit.client.1.smithi096.stdout:8/173: creat d8/d11/d22/d23/d1d/f38 x:0 0 0 2024-03-24T10:46:06.449 INFO:tasks.workunit.client.1.smithi096.stdout:7/163: dwrite d0/db/f1b [0,4194304] 0 2024-03-24T10:46:06.449 INFO:tasks.workunit.client.1.smithi096.stdout:7/164: creat d0/d13/f3d x:0 0 0 2024-03-24T10:46:06.449 INFO:tasks.workunit.client.1.smithi096.stdout:1/127: dwrite d3/f21 [0,4194304] 0 2024-03-24T10:46:06.452 INFO:tasks.workunit.client.1.smithi096.stdout:8/174: link d8/f14 d8/d11/d22/d23/d37/f39 0 2024-03-24T10:46:06.460 INFO:tasks.workunit.client.1.smithi096.stdout:1/128: unlink d3/l28 0 2024-03-24T10:46:06.460 INFO:tasks.workunit.client.1.smithi096.stdout:1/129: truncate d3/f22 528528 0 2024-03-24T10:46:06.464 INFO:tasks.workunit.client.1.smithi096.stdout:8/175: getdents d8/d11/d22/d23/d1a 0 2024-03-24T10:46:06.486 INFO:tasks.workunit.client.1.smithi096.stdout:7/165: dwrite d0/d21/f27 [0,4194304] 0 2024-03-24T10:46:06.486 INFO:tasks.workunit.client.1.smithi096.stdout:8/176: dwrite d8/f14 [0,4194304] 0 2024-03-24T10:46:06.495 INFO:tasks.workunit.client.1.smithi096.stdout:7/166: dread d0/d13/f18 [0,4194304] 0 2024-03-24T10:46:06.496 INFO:tasks.workunit.client.1.smithi096.stdout:8/177: dread d8/d11/d22/d23/d37/f39 [0,4194304] 0 2024-03-24T10:46:06.496 INFO:tasks.workunit.client.1.smithi096.stdout:8/178: write d8/d11/d22/d23/d1d/f2f [314331,109674] 0 2024-03-24T10:46:06.503 INFO:tasks.workunit.client.1.smithi096.stdout:4/134: sync 2024-03-24T10:46:06.504 INFO:tasks.workunit.client.1.smithi096.stdout:0/89: sync 2024-03-24T10:46:06.504 INFO:tasks.workunit.client.1.smithi096.stdout:0/90: read d0/fe [163576,65819] 0 2024-03-24T10:46:06.504 INFO:tasks.workunit.client.1.smithi096.stdout:0/91: chown d0/d11/d16/c17 65773 1 2024-03-24T10:46:06.506 INFO:tasks.workunit.client.1.smithi096.stdout:4/135: dread d1/fe [0,4194304] 0 2024-03-24T10:46:06.547 INFO:tasks.workunit.client.1.smithi096.stdout:8/179: mknod d8/d11/d22/d23/d1d/c3a 0 2024-03-24T10:46:06.548 INFO:tasks.workunit.client.1.smithi096.stdout:0/92: rmdir d0/d11/d16 39 2024-03-24T10:46:06.553 INFO:tasks.workunit.client.1.smithi096.stdout:4/136: fsync d1/d11/f19 0 2024-03-24T10:46:06.553 INFO:tasks.workunit.client.1.smithi096.stdout:4/137: readlink d1/la 0 2024-03-24T10:46:06.554 INFO:tasks.workunit.client.1.smithi096.stdout:7/167: getdents d0 0 2024-03-24T10:46:06.554 INFO:tasks.workunit.client.1.smithi096.stdout:7/168: write d0/f14 [1452804,15449] 0 2024-03-24T10:46:06.555 INFO:tasks.workunit.client.1.smithi096.stdout:8/180: truncate d8/fe 103459 0 2024-03-24T10:46:06.555 INFO:tasks.workunit.client.1.smithi096.stdout:8/181: chown d8/d11/f27 0 1 2024-03-24T10:46:06.555 INFO:tasks.workunit.client.1.smithi096.stdout:8/182: creat d8/d11/d22/d23/d1d/f3b x:0 0 0 2024-03-24T10:46:06.556 INFO:tasks.workunit.client.1.smithi096.stdout:0/93: dread d0/d2/fb [0,4194304] 0 2024-03-24T10:46:06.559 INFO:tasks.workunit.client.1.smithi096.stdout:7/169: symlink d0/db/l3e 0 2024-03-24T10:46:06.564 INFO:tasks.workunit.client.1.smithi096.stdout:4/138: rename d1/l1b to d1/d11/l2c 0 2024-03-24T10:46:06.566 INFO:tasks.workunit.client.1.smithi096.stdout:7/170: symlink d0/db/l3f 0 2024-03-24T10:46:06.567 INFO:tasks.workunit.client.1.smithi096.stdout:7/171: fsync d0/f24 0 2024-03-24T10:46:06.567 INFO:tasks.workunit.client.1.smithi096.stdout:7/172: truncate d0/db/d1c/f29 66181 0 2024-03-24T10:46:06.568 INFO:tasks.workunit.client.1.smithi096.stdout:8/183: rename d8/d11/l1b to d8/d11/d22/d23/d1e/d25/d2e/l3c 0 2024-03-24T10:46:06.568 INFO:tasks.workunit.client.1.smithi096.stdout:4/139: symlink d1/d2b/l2d 0 2024-03-24T10:46:06.570 INFO:tasks.workunit.client.1.smithi096.stdout:7/173: symlink d0/d13/l40 0 2024-03-24T10:46:06.570 INFO:tasks.workunit.client.1.smithi096.stdout:7/174: dread - d0/d21/f2e zero size 2024-03-24T10:46:06.570 INFO:tasks.workunit.client.1.smithi096.stdout:4/140: write d1/d11/f19 [579517,97528] 0 2024-03-24T10:46:06.573 INFO:tasks.workunit.client.1.smithi096.stdout:0/94: rename d0/c13 to d0/d2/d15/c19 0 2024-03-24T10:46:06.574 INFO:tasks.workunit.client.1.smithi096.stdout:8/184: mkdir d8/d11/d22/d23/d1d/d3d 0 2024-03-24T10:46:06.574 INFO:tasks.workunit.client.1.smithi096.stdout:8/185: creat d8/d11/d22/d23/d1d/f3e x:0 0 0 2024-03-24T10:46:06.575 INFO:tasks.workunit.client.1.smithi096.stdout:7/175: creat d0/db/d3a/f41 x:0 0 0 2024-03-24T10:46:06.575 INFO:tasks.workunit.client.1.smithi096.stdout:7/176: stat d0/db/d1c/f29 0 2024-03-24T10:46:06.578 INFO:tasks.workunit.client.1.smithi096.stdout:8/186: rename l2 to d8/d11/d22/d23/d1e/d25/d2e/l3f 0 2024-03-24T10:46:06.580 INFO:tasks.workunit.client.1.smithi096.stdout:7/177: mknod d0/db/d26/c42 0 2024-03-24T10:46:06.580 INFO:tasks.workunit.client.1.smithi096.stdout:0/95: dread d0/d2/fb [0,4194304] 0 2024-03-24T10:46:06.581 INFO:tasks.workunit.client.1.smithi096.stdout:8/187: mknod d8/d11/d22/d23/d1e/d25/c40 0 2024-03-24T10:46:06.581 INFO:tasks.workunit.client.1.smithi096.stdout:8/188: fdatasync d8/d11/d22/d23/d1d/f38 0 2024-03-24T10:46:06.582 INFO:tasks.workunit.client.1.smithi096.stdout:7/178: dread d0/d13/f2c [0,4194304] 0 2024-03-24T10:46:06.583 INFO:tasks.workunit.client.1.smithi096.stdout:0/96: symlink d0/d11/l1a 0 2024-03-24T10:46:06.584 INFO:tasks.workunit.client.1.smithi096.stdout:8/189: rmdir d8 39 2024-03-24T10:46:06.588 INFO:tasks.workunit.client.1.smithi096.stdout:8/190: dread d8/d11/d22/d23/d1d/f28 [0,4194304] 0 2024-03-24T10:46:06.589 INFO:tasks.workunit.client.1.smithi096.stdout:0/97: write d0/d2/fb [2097566,80229] 0 2024-03-24T10:46:06.589 INFO:tasks.workunit.client.1.smithi096.stdout:4/141: dwrite d1/f24 [0,4194304] 0 2024-03-24T10:46:06.590 INFO:tasks.workunit.client.1.smithi096.stdout:8/191: symlink d8/d11/d22/d23/d1e/l41 0 2024-03-24T10:46:06.590 INFO:tasks.workunit.client.1.smithi096.stdout:7/179: dread d0/d13/f5 [0,4194304] 0 2024-03-24T10:46:06.592 INFO:tasks.workunit.client.1.smithi096.stdout:0/98: mkdir d0/d11/d1b 0 2024-03-24T10:46:06.594 INFO:tasks.workunit.client.1.smithi096.stdout:4/142: rmdir d1/d2b 39 2024-03-24T10:46:06.595 INFO:tasks.workunit.client.1.smithi096.stdout:7/180: read d0/d13/f18 [2108521,12013] 0 2024-03-24T10:46:06.597 INFO:tasks.workunit.client.1.smithi096.stdout:0/99: fsync d0/d2/fb 0 2024-03-24T10:46:06.597 INFO:tasks.workunit.client.1.smithi096.stdout:0/100: chown d0/d2/d15 117 1 2024-03-24T10:46:06.597 INFO:tasks.workunit.client.1.smithi096.stdout:8/192: symlink d8/d11/l42 0 2024-03-24T10:46:06.597 INFO:tasks.workunit.client.1.smithi096.stdout:8/193: dread - d8/d11/d22/d23/d1d/f3b zero size 2024-03-24T10:46:06.597 INFO:tasks.workunit.client.1.smithi096.stdout:8/194: creat d8/d11/f43 x:0 0 0 2024-03-24T10:46:06.597 INFO:tasks.workunit.client.1.smithi096.stdout:8/195: write d8/d11/d22/d23/d1d/f3e [120029,71910] 0 2024-03-24T10:46:06.601 INFO:tasks.workunit.client.1.smithi096.stdout:7/181: creat d0/d21/d3c/f43 x:0 0 0 2024-03-24T10:46:06.605 INFO:tasks.workunit.client.1.smithi096.stdout:7/182: dread d0/d13/f18 [0,4194304] 0 2024-03-24T10:46:06.605 INFO:tasks.workunit.client.1.smithi096.stdout:7/183: read d0/f20 [840545,111241] 0 2024-03-24T10:46:06.605 INFO:tasks.workunit.client.1.smithi096.stdout:7/184: dread - d0/d13/d32/f3b zero size 2024-03-24T10:46:06.608 INFO:tasks.workunit.client.1.smithi096.stdout:0/101: creat d0/d11/f1c x:0 0 0 2024-03-24T10:46:06.613 INFO:tasks.workunit.client.1.smithi096.stdout:8/196: mkdir d8/d11/d44 0 2024-03-24T10:46:06.641 INFO:tasks.workunit.client.1.smithi096.stdout:0/102: dwrite d0/f5 [0,4194304] 0 2024-03-24T10:46:06.641 INFO:tasks.workunit.client.1.smithi096.stdout:8/197: dwrite d8/d11/d22/d23/d1d/f38 [0,4194304] 0 2024-03-24T10:46:06.702 INFO:tasks.workunit.client.1.smithi096.stdout:4/143: mknod d1/d11/c2e 0 2024-03-24T10:46:06.705 INFO:tasks.workunit.client.1.smithi096.stdout:7/185: creat d0/db/d26/f44 x:0 0 0 2024-03-24T10:46:06.705 INFO:tasks.workunit.client.1.smithi096.stdout:7/186: write d0/f20 [1489296,31204] 0 2024-03-24T10:46:06.705 INFO:tasks.workunit.client.1.smithi096.stdout:7/187: chown d0/db/d3a 2 1 2024-03-24T10:46:06.710 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:06 smithi096 ceph-mon[38097]: pgmap v246: 65 pgs: 65 active+clean; 1.8 GiB data, 7.2 GiB used, 529 GiB / 536 GiB avail; 224 MiB/s rd, 262 MiB/s wr, 368 op/s 2024-03-24T10:46:06.722 INFO:tasks.workunit.client.1.smithi096.stdout:0/103: symlink d0/d11/l1d 0 2024-03-24T10:46:06.722 INFO:tasks.workunit.client.1.smithi096.stdout:0/104: stat d0/d11/d16 0 2024-03-24T10:46:06.722 INFO:tasks.workunit.client.1.smithi096.stdout:0/105: write d0/f5 [5352576,51912] 0 2024-03-24T10:46:06.725 INFO:tasks.workunit.client.1.smithi096.stdout:8/198: mkdir d8/d11/d44/d45 0 2024-03-24T10:46:06.738 INFO:tasks.workunit.client.1.smithi096.stdout:7/188: mkdir d0/d13/d32/d45 0 2024-03-24T10:46:06.746 INFO:tasks.workunit.client.1.smithi096.stdout:0/106: symlink d0/d2/d15/l1e 0 2024-03-24T10:46:06.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:06 smithi094 ceph-mon[29446]: pgmap v246: 65 pgs: 65 active+clean; 1.8 GiB data, 7.2 GiB used, 529 GiB / 536 GiB avail; 224 MiB/s rd, 262 MiB/s wr, 368 op/s 2024-03-24T10:46:06.750 INFO:tasks.workunit.client.1.smithi096.stdout:8/199: unlink d8/d11/d22/d23/d1a/f1f 0 2024-03-24T10:46:06.755 INFO:tasks.workunit.client.1.smithi096.stdout:0/107: fdatasync d0/f5 0 2024-03-24T10:46:06.755 INFO:tasks.workunit.client.1.smithi096.stdout:7/189: dwrite d0/d21/f2e [0,4194304] 0 2024-03-24T10:46:06.755 INFO:tasks.workunit.client.1.smithi096.stdout:7/190: write d0/db/ff [1138045,45737] 0 2024-03-24T10:46:06.755 INFO:tasks.workunit.client.1.smithi096.stdout:7/191: fdatasync d0/db/d26/f37 0 2024-03-24T10:46:06.756 INFO:tasks.workunit.client.1.smithi096.stdout:7/192: truncate d0/d13/d32/f3b 889863 0 2024-03-24T10:46:06.756 INFO:tasks.workunit.client.1.smithi096.stdout:7/193: write d0/d13/f18 [966118,69602] 0 2024-03-24T10:46:06.756 INFO:tasks.workunit.client.1.smithi096.stdout:8/200: rename d8/d11/d22/d23/d1e/d25 to d8/d11/d22/d23/d1a/d46 0 2024-03-24T10:46:06.767 INFO:tasks.workunit.client.1.smithi096.stdout:7/194: mknod d0/d21/d3c/c46 0 2024-03-24T10:46:06.772 INFO:tasks.workunit.client.1.smithi096.stdout:8/201: link d8/d11/l42 d8/d11/d22/d23/d37/l47 0 2024-03-24T10:46:06.775 INFO:tasks.workunit.client.1.smithi096.stdout:7/195: mknod d0/db/d26/c47 0 2024-03-24T10:46:06.776 INFO:tasks.workunit.client.1.smithi096.stdout:7/196: readlink d0/d13/d32/l34 0 2024-03-24T10:46:06.779 INFO:tasks.workunit.client.1.smithi096.stdout:8/202: symlink d8/d11/d22/d23/d1e/l48 0 2024-03-24T10:46:06.779 INFO:tasks.workunit.client.1.smithi096.stdout:8/203: fsync d8/d11/f31 0 2024-03-24T10:46:06.780 INFO:tasks.workunit.client.1.smithi096.stdout:0/108: dwrite d0/f5 [0,4194304] 0 2024-03-24T10:46:06.782 INFO:tasks.workunit.client.1.smithi096.stdout:7/197: mkdir d0/db/d26/d48 0 2024-03-24T10:46:06.784 INFO:tasks.workunit.client.1.smithi096.stdout:8/204: read d8/d11/f31 [9262852,51025] 0 2024-03-24T10:46:06.785 INFO:tasks.workunit.client.1.smithi096.stdout:0/109: dread d0/f5 [4194304,4194304] 0 2024-03-24T10:46:06.785 INFO:tasks.workunit.client.1.smithi096.stdout:0/110: dread - d0/d11/f1c zero size 2024-03-24T10:46:06.786 INFO:tasks.workunit.client.1.smithi096.stdout:8/205: dread d8/d11/d22/d23/d1d/f28 [0,4194304] 0 2024-03-24T10:46:06.786 INFO:tasks.workunit.client.1.smithi096.stdout:8/206: dread - d8/d11/d22/d23/d1d/f3b zero size 2024-03-24T10:46:06.791 INFO:tasks.workunit.client.1.smithi096.stdout:7/198: unlink d0/d21/c30 0 2024-03-24T10:46:06.791 INFO:tasks.workunit.client.1.smithi096.stdout:7/199: fsync d0/f38 0 2024-03-24T10:46:06.794 INFO:tasks.workunit.client.1.smithi096.stdout:0/111: truncate d0/d2/fb 1656038 0 2024-03-24T10:46:06.797 INFO:tasks.workunit.client.1.smithi096.stdout:7/200: dread d0/d13/f5 [0,4194304] 0 2024-03-24T10:46:06.800 INFO:tasks.workunit.client.1.smithi096.stdout:0/112: creat d0/d2/d15/f1f x:0 0 0 2024-03-24T10:46:06.802 INFO:tasks.workunit.client.1.smithi096.stdout:8/207: dwrite d8/fe [0,4194304] 0 2024-03-24T10:46:06.802 INFO:tasks.workunit.client.1.smithi096.stdout:7/201: symlink d0/l49 0 2024-03-24T10:46:06.828 INFO:tasks.workunit.client.1.smithi096.stdout:8/208: mkdir d8/d11/d22/d23/d1a/d49 0 2024-03-24T10:46:06.831 INFO:tasks.workunit.client.1.smithi096.stdout:2/120: sync 2024-03-24T10:46:06.831 INFO:tasks.workunit.client.1.smithi096.stdout:3/221: sync 2024-03-24T10:46:06.831 INFO:tasks.workunit.client.1.smithi096.stdout:2/121: creat d5/dc/f34 x:0 0 0 2024-03-24T10:46:06.832 INFO:tasks.workunit.client.1.smithi096.stdout:7/202: creat d0/db/d1c/d39/f4a x:0 0 0 2024-03-24T10:46:06.834 INFO:tasks.workunit.client.1.smithi096.stdout:8/209: rmdir d8/d11/d44 39 2024-03-24T10:46:06.836 INFO:tasks.workunit.client.1.smithi096.stdout:3/222: creat d0/d9/d22/f4f x:0 0 0 2024-03-24T10:46:06.838 INFO:tasks.workunit.client.1.smithi096.stdout:8/210: dread d8/f15 [0,4194304] 0 2024-03-24T10:46:06.843 INFO:tasks.workunit.client.1.smithi096.stdout:3/223: unlink d0/d9/d34/f37 0 2024-03-24T10:46:06.846 INFO:tasks.workunit.client.1.smithi096.stdout:8/211: creat d8/d11/d22/d23/d1d/d3d/f4a x:0 0 0 2024-03-24T10:46:06.846 INFO:tasks.workunit.client.1.smithi096.stdout:8/212: creat d8/d11/d22/d23/d1d/d3d/f4b x:0 0 0 2024-03-24T10:46:06.850 INFO:tasks.workunit.client.1.smithi096.stdout:3/224: symlink d0/d9/d22/l50 0 2024-03-24T10:46:06.855 INFO:tasks.workunit.client.1.smithi096.stdout:3/225: mknod d0/d9/d22/c51 0 2024-03-24T10:46:06.855 INFO:tasks.workunit.client.1.smithi096.stdout:8/213: dread d8/d11/d22/d23/d1d/f38 [0,4194304] 0 2024-03-24T10:46:06.855 INFO:tasks.workunit.client.1.smithi096.stdout:8/214: dread - d8/d11/d22/d23/d1d/d3d/f4b zero size 2024-03-24T10:46:06.855 INFO:tasks.workunit.client.1.smithi096.stdout:8/215: read d8/d11/d22/d23/f1c [3089558,12497] 0 2024-03-24T10:46:06.855 INFO:tasks.workunit.client.1.smithi096.stdout:8/216: dread - d8/d11/d22/d23/d1a/f26 zero size 2024-03-24T10:46:06.856 INFO:tasks.workunit.client.1.smithi096.stdout:3/226: mkdir d0/d13/d45/d52 0 2024-03-24T10:46:06.856 INFO:tasks.workunit.client.1.smithi096.stdout:8/217: dread d8/d11/d22/d23/d1d/f28 [0,4194304] 0 2024-03-24T10:46:06.856 INFO:tasks.workunit.client.1.smithi096.stdout:8/218: read - d8/d11/f27 zero size 2024-03-24T10:46:06.859 INFO:tasks.workunit.client.1.smithi096.stdout:8/219: mkdir d8/d11/d22/d4c 0 2024-03-24T10:46:06.860 INFO:tasks.workunit.client.1.smithi096.stdout:8/220: write d8/d11/f43 [532236,93789] 0 2024-03-24T10:46:06.860 INFO:tasks.workunit.client.1.smithi096.stdout:8/221: write d8/d11/d22/d23/d1d/f3b [504824,46649] 0 2024-03-24T10:46:06.863 INFO:tasks.workunit.client.1.smithi096.stdout:9/121: sync 2024-03-24T10:46:06.863 INFO:tasks.workunit.client.1.smithi096.stdout:1/130: sync 2024-03-24T10:46:06.863 INFO:tasks.workunit.client.1.smithi096.stdout:6/195: sync 2024-03-24T10:46:06.863 INFO:tasks.workunit.client.1.smithi096.stdout:1/131: chown c1 102481680 1 2024-03-24T10:46:06.863 INFO:tasks.workunit.client.1.smithi096.stdout:9/122: dread - d9/d15/f25 zero size 2024-03-24T10:46:06.864 INFO:tasks.workunit.client.1.smithi096.stdout:5/131: sync 2024-03-24T10:46:06.865 INFO:tasks.workunit.client.1.smithi096.stdout:5/132: dread - d2/d7/d8/df/f25 zero size 2024-03-24T10:46:06.866 INFO:tasks.workunit.client.1.smithi096.stdout:8/222: dread d8/fc [0,4194304] 0 2024-03-24T10:46:06.866 INFO:tasks.workunit.client.1.smithi096.stdout:9/123: mkdir d9/d15/d26 0 2024-03-24T10:46:06.868 INFO:tasks.workunit.client.1.smithi096.stdout:5/133: symlink d2/d7/d8/df/l2c 0 2024-03-24T10:46:06.868 INFO:tasks.workunit.client.1.smithi096.stdout:5/134: stat d2 0 2024-03-24T10:46:06.868 INFO:tasks.workunit.client.1.smithi096.stdout:5/135: write d2/d7/d8/d18/f20 [5195202,109996] 0 2024-03-24T10:46:06.870 INFO:tasks.workunit.client.1.smithi096.stdout:6/196: dread f6 [0,4194304] 0 2024-03-24T10:46:06.871 INFO:tasks.workunit.client.1.smithi096.stdout:9/124: creat d9/d15/f27 x:0 0 0 2024-03-24T10:46:06.875 INFO:tasks.workunit.client.1.smithi096.stdout:3/227: dwrite d0/d13/d45/f4c [0,4194304] 0 2024-03-24T10:46:06.875 INFO:tasks.workunit.client.1.smithi096.stdout:3/228: chown d0/d9/d22/c40 111 1 2024-03-24T10:46:06.875 INFO:tasks.workunit.client.1.smithi096.stdout:3/229: chown d0/d9/f36 2815521 1 2024-03-24T10:46:06.877 INFO:tasks.workunit.client.1.smithi096.stdout:6/197: unlink d8/d12/f27 0 2024-03-24T10:46:06.878 INFO:tasks.workunit.client.1.smithi096.stdout:6/198: fdatasync d8/f18 0 2024-03-24T10:46:06.882 INFO:tasks.workunit.client.1.smithi096.stdout:1/132: dwrite d3/f6 [0,4194304] 0 2024-03-24T10:46:06.882 INFO:tasks.workunit.client.1.smithi096.stdout:1/133: dread - d3/f1f zero size 2024-03-24T10:46:06.882 INFO:tasks.workunit.client.1.smithi096.stdout:1/134: fdatasync d3/f1f 0 2024-03-24T10:46:06.882 INFO:tasks.workunit.client.1.smithi096.stdout:1/135: fsync d3/f1f 0 2024-03-24T10:46:06.882 INFO:tasks.workunit.client.1.smithi096.stdout:3/230: unlink d0/d9/d22/c24 0 2024-03-24T10:46:06.882 INFO:tasks.workunit.client.1.smithi096.stdout:1/136: dread d3/f22 [0,4194304] 0 2024-03-24T10:46:06.882 INFO:tasks.workunit.client.1.smithi096.stdout:1/137: chown c2 2134360098 1 2024-03-24T10:46:06.883 INFO:tasks.workunit.client.1.smithi096.stdout:9/125: getdents d9 0 2024-03-24T10:46:06.885 INFO:tasks.workunit.client.1.smithi096.stdout:3/231: creat d0/d13/d45/d52/f53 x:0 0 0 2024-03-24T10:46:06.886 INFO:tasks.workunit.client.1.smithi096.stdout:1/138: mknod d3/d27/c2e 0 2024-03-24T10:46:06.887 INFO:tasks.workunit.client.1.smithi096.stdout:1/139: chown d3/c19 217751 1 2024-03-24T10:46:06.887 INFO:tasks.workunit.client.1.smithi096.stdout:9/126: stat d9/db/c1c 0 2024-03-24T10:46:06.887 INFO:tasks.workunit.client.1.smithi096.stdout:9/127: chown d9/f16 51658 1 2024-03-24T10:46:06.888 INFO:tasks.workunit.client.1.smithi096.stdout:3/232: stat d0/d13/c1d 0 2024-03-24T10:46:06.888 INFO:tasks.workunit.client.1.smithi096.stdout:3/233: truncate d0/f44 623652 0 2024-03-24T10:46:06.888 INFO:tasks.workunit.client.1.smithi096.stdout:3/234: write d0/d9/d22/f3d [495632,115082] 0 2024-03-24T10:46:06.888 INFO:tasks.workunit.client.1.smithi096.stdout:3/235: fdatasync d0/d9/f3b 0 2024-03-24T10:46:06.889 INFO:tasks.workunit.client.1.smithi096.stdout:9/128: creat d9/db/f28 x:0 0 0 2024-03-24T10:46:06.889 INFO:tasks.workunit.client.1.smithi096.stdout:6/199: dread d8/d10/f3b [0,4194304] 0 2024-03-24T10:46:06.890 INFO:tasks.workunit.client.1.smithi096.stdout:3/236: unlink d0/d13/f17 0 2024-03-24T10:46:06.892 INFO:tasks.workunit.client.1.smithi096.stdout:9/129: unlink d9/d15/l1b 0 2024-03-24T10:46:06.892 INFO:tasks.workunit.client.1.smithi096.stdout:9/130: fsync d9/f19 0 2024-03-24T10:46:06.892 INFO:tasks.workunit.client.1.smithi096.stdout:9/131: fsync d9/db/fe 0 2024-03-24T10:46:06.894 INFO:tasks.workunit.client.1.smithi096.stdout:6/200: symlink d8/d10/d19/l40 0 2024-03-24T10:46:06.894 INFO:tasks.workunit.client.1.smithi096.stdout:6/201: truncate f6 706877 0 2024-03-24T10:46:06.896 INFO:tasks.workunit.client.1.smithi096.stdout:3/237: symlink d0/d13/d45/d52/l54 0 2024-03-24T10:46:06.897 INFO:tasks.workunit.client.1.smithi096.stdout:9/132: mkdir d9/d15/d26/d29 0 2024-03-24T10:46:06.898 INFO:tasks.workunit.client.1.smithi096.stdout:9/133: write d9/db/f14 [1037875,18795] 0 2024-03-24T10:46:06.899 INFO:tasks.workunit.client.1.smithi096.stdout:6/202: symlink d8/d12/d32/l41 0 2024-03-24T10:46:06.899 INFO:tasks.workunit.client.1.smithi096.stdout:6/203: stat d8/d10/f3b 0 2024-03-24T10:46:06.902 INFO:tasks.workunit.client.1.smithi096.stdout:3/238: symlink d0/d9/d34/d39/d48/l55 0 2024-03-24T10:46:06.903 INFO:tasks.workunit.client.1.smithi096.stdout:1/140: dwrite d3/f1f [0,4194304] 0 2024-03-24T10:46:06.912 INFO:tasks.workunit.client.1.smithi096.stdout:1/141: unlink d3/f13 0 2024-03-24T10:46:06.918 INFO:tasks.workunit.client.1.smithi096.stdout:0/113: rename d0/d2/d15 to d0/d11/d20 0 2024-03-24T10:46:06.921 INFO:tasks.workunit.client.1.smithi096.stdout:5/136: rename d2/f17 to d2/d7/d8/df/f2d 0 2024-03-24T10:46:06.923 INFO:tasks.workunit.client.1.smithi096.stdout:9/134: dwrite d9/db/fe [0,4194304] 0 2024-03-24T10:46:06.923 INFO:tasks.workunit.client.1.smithi096.stdout:9/135: dread - d9/d15/f25 zero size 2024-03-24T10:46:06.923 INFO:tasks.workunit.client.1.smithi096.stdout:9/136: truncate d9/db/f28 328203 0 2024-03-24T10:46:06.924 INFO:tasks.workunit.client.1.smithi096.stdout:6/204: dwrite d8/d10/d19/f30 [0,4194304] 0 2024-03-24T10:46:06.924 INFO:tasks.workunit.client.1.smithi096.stdout:6/205: chown d8/d10/f3e 38216383 1 2024-03-24T10:46:06.924 INFO:tasks.workunit.client.1.smithi096.stdout:6/206: fdatasync d8/d12/d32/f3c 0 2024-03-24T10:46:06.924 INFO:tasks.workunit.client.1.smithi096.stdout:6/207: fdatasync d8/d12/f2d 0 2024-03-24T10:46:06.924 INFO:tasks.workunit.client.1.smithi096.stdout:6/208: fsync d8/d10/d19/f34 0 2024-03-24T10:46:06.936 INFO:tasks.workunit.client.1.smithi096.stdout:7/203: getdents d0/db/d1c/d39 0 2024-03-24T10:46:06.937 INFO:tasks.workunit.client.1.smithi096.stdout:7/204: mkdir d0/db/d3a/d4b 0 2024-03-24T10:46:06.937 INFO:tasks.workunit.client.1.smithi096.stdout:7/205: fsync d0/d13/f2c 0 2024-03-24T10:46:06.949 INFO:tasks.workunit.client.1.smithi096.stdout:5/137: dwrite d2/d7/d8/df/f25 [0,4194304] 0 2024-03-24T10:46:06.949 INFO:tasks.workunit.client.1.smithi096.stdout:5/138: chown d2/d7/d8/lb 0 1 2024-03-24T10:46:06.950 INFO:tasks.workunit.client.1.smithi096.stdout:8/223: rmdir d8/d11/d22/d23/d1d 39 2024-03-24T10:46:06.953 INFO:tasks.workunit.client.1.smithi096.stdout:6/209: dwrite d8/d10/d19/f34 [0,4194304] 0 2024-03-24T10:46:06.954 INFO:tasks.workunit.client.1.smithi096.stdout:7/206: dwrite d0/db/f1b [0,4194304] 0 2024-03-24T10:46:06.957 INFO:tasks.workunit.client.1.smithi096.stdout:5/139: symlink d2/d7/d8/df/l2e 0 2024-03-24T10:46:06.959 INFO:tasks.workunit.client.1.smithi096.stdout:8/224: truncate d8/d11/d22/d23/d1d/f2b 7894735 0 2024-03-24T10:46:06.959 INFO:tasks.workunit.client.1.smithi096.stdout:8/225: fsync d8/d11/d22/d23/d1d/f2f 0 2024-03-24T10:46:06.959 INFO:tasks.workunit.client.1.smithi096.stdout:8/226: write d8/d11/d22/d23/d1d/d3d/f4a [385376,79693] 0 2024-03-24T10:46:06.966 INFO:tasks.workunit.client.1.smithi096.stdout:5/140: write d2/d7/d8/df/f2d [3443959,97863] 0 2024-03-24T10:46:06.966 INFO:tasks.workunit.client.1.smithi096.stdout:7/207: write d0/d13/f35 [2140301,90680] 0 2024-03-24T10:46:06.967 INFO:tasks.workunit.client.1.smithi096.stdout:6/210: unlink d8/f18 0 2024-03-24T10:46:06.976 INFO:tasks.workunit.client.1.smithi096.stdout:6/211: getdents d8/d12 0 2024-03-24T10:46:06.977 INFO:tasks.workunit.client.1.smithi096.stdout:6/212: read f6 [545403,105627] 0 2024-03-24T10:46:06.977 INFO:tasks.workunit.client.1.smithi096.stdout:6/213: chown d8/d12/d28 41219436 1 2024-03-24T10:46:06.979 INFO:tasks.workunit.client.1.smithi096.stdout:1/142: rename d3/d27 to d3/d23/d2f 0 2024-03-24T10:46:06.979 INFO:tasks.workunit.client.1.smithi096.stdout:1/143: write d3/f22 [759185,82315] 0 2024-03-24T10:46:06.980 INFO:tasks.workunit.client.1.smithi096.stdout:5/141: dwrite d2/d7/f10 [0,4194304] 0 2024-03-24T10:46:06.983 INFO:tasks.workunit.client.1.smithi096.stdout:0/114: rename d0/d11/d1b to d0/d11/d16/d21 0 2024-03-24T10:46:06.983 INFO:tasks.workunit.client.1.smithi096.stdout:0/115: creat d0/f22 x:0 0 0 2024-03-24T10:46:06.983 INFO:tasks.workunit.client.1.smithi096.stdout:0/116: chown d0/d2/cf 676839 1 2024-03-24T10:46:06.987 INFO:tasks.workunit.client.1.smithi096.stdout:1/144: write d3/ff [63307,41125] 0 2024-03-24T10:46:06.988 INFO:tasks.workunit.client.1.smithi096.stdout:5/142: symlink d2/d7/l2f 0 2024-03-24T10:46:06.994 INFO:tasks.workunit.client.1.smithi096.stdout:6/214: dwrite d8/d10/d22/f21 [0,4194304] 0 2024-03-24T10:46:06.994 INFO:tasks.workunit.client.1.smithi096.stdout:6/215: truncate d8/d10/f3d 933023 0 2024-03-24T10:46:06.994 INFO:tasks.workunit.client.1.smithi096.stdout:6/216: chown d8/d12/c1d 59926 1 2024-03-24T10:46:06.994 INFO:tasks.workunit.client.1.smithi096.stdout:6/217: truncate d8/d10/d22/f21 5043560 0 2024-03-24T10:46:06.995 INFO:tasks.workunit.client.1.smithi096.stdout:0/117: rename d0/l10 to d0/l23 0 2024-03-24T10:46:06.995 INFO:tasks.workunit.client.1.smithi096.stdout:0/118: truncate d0/fe 642316 0 2024-03-24T10:46:06.996 INFO:tasks.workunit.client.1.smithi096.stdout:1/145: read d3/f15 [2027627,124520] 0 2024-03-24T10:46:06.996 INFO:tasks.workunit.client.1.smithi096.stdout:1/146: chown d3/d23/l25 0 1 2024-03-24T10:46:06.996 INFO:tasks.workunit.client.1.smithi096.stdout:9/137: rename d9/db/c1c to d9/d15/c2a 0 2024-03-24T10:46:07.000 INFO:tasks.workunit.client.1.smithi096.stdout:0/119: unlink d0/d11/d16/c17 0 2024-03-24T10:46:07.006 INFO:tasks.workunit.client.1.smithi096.stdout:1/147: getdents d3/d23/d2f 0 2024-03-24T10:46:07.006 INFO:tasks.workunit.client.1.smithi096.stdout:1/148: chown d3/f1f 6466469 1 2024-03-24T10:46:07.008 INFO:tasks.workunit.client.1.smithi096.stdout:1/149: mknod d3/d23/c30 0 2024-03-24T10:46:07.010 INFO:tasks.workunit.client.1.smithi096.stdout:6/218: dread f7 [0,4194304] 0 2024-03-24T10:46:07.013 INFO:tasks.workunit.client.1.smithi096.stdout:1/150: link d3/d23/d2f/c2e d3/c31 0 2024-03-24T10:46:07.014 INFO:tasks.workunit.client.1.smithi096.stdout:6/219: mknod d8/d10/c42 0 2024-03-24T10:46:07.014 INFO:tasks.workunit.client.1.smithi096.stdout:6/220: dread - d8/d12/d28/f33 zero size 2024-03-24T10:46:07.014 INFO:tasks.workunit.client.1.smithi096.stdout:6/221: chown d8/d10/d22 17 1 2024-03-24T10:46:07.015 INFO:tasks.workunit.client.1.smithi096.stdout:1/151: rename d3/ld to d3/d23/l32 0 2024-03-24T10:46:07.017 INFO:tasks.workunit.client.1.smithi096.stdout:6/222: unlink l0 0 2024-03-24T10:46:07.019 INFO:tasks.workunit.client.1.smithi096.stdout:1/152: mknod d3/c33 0 2024-03-24T10:46:07.024 INFO:tasks.workunit.client.1.smithi096.stdout:5/143: dwrite d2/d7/f10 [0,4194304] 0 2024-03-24T10:46:07.024 INFO:tasks.workunit.client.1.smithi096.stdout:5/144: fdatasync d2/f3 0 2024-03-24T10:46:07.027 INFO:tasks.workunit.client.1.smithi096.stdout:1/153: dread d3/f9 [4194304,4194304] 0 2024-03-24T10:46:07.031 INFO:tasks.workunit.client.1.smithi096.stdout:9/138: dwrite d9/f1a [0,4194304] 0 2024-03-24T10:46:07.031 INFO:tasks.workunit.client.1.smithi096.stdout:9/139: write d9/d15/f27 [530599,59518] 0 2024-03-24T10:46:07.031 INFO:tasks.workunit.client.1.smithi096.stdout:9/140: getdents d9/d15/d26/d29 0 2024-03-24T10:46:07.036 INFO:tasks.workunit.client.1.smithi096.stdout:0/120: dwrite d0/d11/f1c [0,4194304] 0 2024-03-24T10:46:07.036 INFO:tasks.workunit.client.1.smithi096.stdout:0/121: chown d0/fe 411 1 2024-03-24T10:46:07.036 INFO:tasks.workunit.client.1.smithi096.stdout:0/122: dread - d0/d11/d20/f1f zero size 2024-03-24T10:46:07.036 INFO:tasks.workunit.client.1.smithi096.stdout:0/123: fsync d0/d11/d20/f1f 0 2024-03-24T10:46:07.036 INFO:tasks.workunit.client.1.smithi096.stdout:0/124: fdatasync d0/d2/fb 0 2024-03-24T10:46:07.036 INFO:tasks.workunit.client.1.smithi096.stdout:0/125: write d0/d11/d20/f1f [798282,85252] 0 2024-03-24T10:46:07.037 INFO:tasks.workunit.client.1.smithi096.stdout:6/223: dwrite d8/d12/f2d [0,4194304] 0 2024-03-24T10:46:07.037 INFO:tasks.workunit.client.1.smithi096.stdout:8/227: dread f1 [0,4194304] 0 2024-03-24T10:46:07.048 INFO:tasks.workunit.client.1.smithi096.stdout:5/145: mknod d2/d7/d8/df/d14/c30 0 2024-03-24T10:46:07.056 INFO:tasks.workunit.client.1.smithi096.stdout:8/228: dwrite d8/d11/d22/d23/d1d/f38 [0,4194304] 0 2024-03-24T10:46:07.056 INFO:tasks.workunit.client.1.smithi096.stdout:8/229: write d8/d11/d22/d23/d1a/f2d [459457,101240] 0 2024-03-24T10:46:07.058 INFO:tasks.workunit.client.1.smithi096.stdout:9/141: creat d9/d15/d26/f2b x:0 0 0 2024-03-24T10:46:07.058 INFO:tasks.workunit.client.1.smithi096.stdout:9/142: readlink d9/d15/l12 0 2024-03-24T10:46:07.065 INFO:tasks.workunit.client.1.smithi096.stdout:0/126: mkdir d0/d11/d24 0 2024-03-24T10:46:07.070 INFO:tasks.workunit.client.1.smithi096.stdout:6/224: rmdir d8/d10/d19 39 2024-03-24T10:46:07.075 INFO:tasks.workunit.client.1.smithi096.stdout:5/146: link d2/f3 d2/d7/d8/df/d14/f31 0 2024-03-24T10:46:07.076 INFO:tasks.workunit.client.1.smithi096.stdout:5/147: chown d2/d7/f26 111 1 2024-03-24T10:46:07.079 INFO:tasks.workunit.client.1.smithi096.stdout:1/154: truncate d3/f15 660291 0 2024-03-24T10:46:07.079 INFO:tasks.workunit.client.1.smithi096.stdout:1/155: write d3/ff [4506118,118967] 0 2024-03-24T10:46:07.080 INFO:tasks.workunit.client.1.smithi096.stdout:0/127: dwrite d0/f5 [4194304,4194304] 0 2024-03-24T10:46:07.080 INFO:tasks.workunit.client.1.smithi096.stdout:0/128: getdents d0/d11/d16/d21 0 2024-03-24T10:46:07.080 INFO:tasks.workunit.client.1.smithi096.stdout:0/129: chown d0/d11 5511 1 2024-03-24T10:46:07.081 INFO:tasks.workunit.client.1.smithi096.stdout:0/130: write d0/d11/d20/f1f [191356,64188] 0 2024-03-24T10:46:07.081 INFO:tasks.workunit.client.1.smithi096.stdout:0/131: chown d0/l23 0 1 2024-03-24T10:46:07.082 INFO:tasks.workunit.client.1.smithi096.stdout:0/132: chown d0/d11/l1a 257670 1 2024-03-24T10:46:07.082 INFO:tasks.workunit.client.1.smithi096.stdout:0/133: stat d0/d2/cf 0 2024-03-24T10:46:07.083 INFO:tasks.workunit.client.1.smithi096.stdout:4/144: sync 2024-03-24T10:46:07.084 INFO:tasks.workunit.client.1.smithi096.stdout:5/148: dread d2/d7/d8/df/f1b [0,4194304] 0 2024-03-24T10:46:07.084 INFO:tasks.workunit.client.1.smithi096.stdout:5/149: truncate d2/d7/d8/df/f1f 509693 0 2024-03-24T10:46:07.089 INFO:tasks.workunit.client.1.smithi096.stdout:8/230: creat d8/d11/d44/d45/f4d x:0 0 0 2024-03-24T10:46:07.090 INFO:tasks.workunit.client.1.smithi096.stdout:6/225: symlink d8/d12/l43 0 2024-03-24T10:46:07.092 INFO:tasks.workunit.client.1.smithi096.stdout:1/156: creat d3/f34 x:0 0 0 2024-03-24T10:46:07.092 INFO:tasks.workunit.client.1.smithi096.stdout:1/157: stat d3/f21 0 2024-03-24T10:46:07.094 INFO:tasks.workunit.client.1.smithi096.stdout:0/134: mkdir d0/d11/d16/d21/d25 0 2024-03-24T10:46:07.094 INFO:tasks.workunit.client.1.smithi096.stdout:0/135: chown d0/l23 5 1 2024-03-24T10:46:07.097 INFO:tasks.workunit.client.1.smithi096.stdout:4/145: creat d1/d2b/f2f x:0 0 0 2024-03-24T10:46:07.098 INFO:tasks.workunit.client.1.smithi096.stdout:8/231: dread d8/d11/d22/d23/d1d/f38 [0,4194304] 0 2024-03-24T10:46:07.098 INFO:tasks.workunit.client.1.smithi096.stdout:8/232: truncate d8/d11/d22/d23/f1c 4605539 0 2024-03-24T10:46:07.101 INFO:tasks.workunit.client.1.smithi096.stdout:9/143: rename d9/d15/d26/f2b to d9/f2c 0 2024-03-24T10:46:07.102 INFO:tasks.workunit.client.1.smithi096.stdout:0/136: write d0/f5 [6898932,129560] 0 2024-03-24T10:46:07.102 INFO:tasks.workunit.client.1.smithi096.stdout:0/137: chown d0/d11/d20/f1f 3578 1 2024-03-24T10:46:07.104 INFO:tasks.workunit.client.1.smithi096.stdout:1/158: creat d3/d24/f35 x:0 0 0 2024-03-24T10:46:07.106 INFO:tasks.workunit.client.1.smithi096.stdout:4/146: readlink d1/d11/l26 0 2024-03-24T10:46:07.108 INFO:tasks.workunit.client.1.smithi096.stdout:8/233: symlink d8/d11/d22/d23/d1a/d49/l4e 0 2024-03-24T10:46:07.108 INFO:tasks.workunit.client.1.smithi096.stdout:8/234: chown d8/d11/d22/d23/d1a 949208658 1 2024-03-24T10:46:07.110 INFO:tasks.workunit.client.1.smithi096.stdout:9/144: link d9/d15/c2a d9/d15/d26/c2d 0 2024-03-24T10:46:07.110 INFO:tasks.workunit.client.1.smithi096.stdout:9/145: stat d9/db/fe 0 2024-03-24T10:46:07.110 INFO:tasks.workunit.client.1.smithi096.stdout:9/146: stat d9 0 2024-03-24T10:46:07.114 INFO:tasks.workunit.client.1.smithi096.stdout:4/147: write d1/f24 [3126475,82735] 0 2024-03-24T10:46:07.114 INFO:tasks.workunit.client.1.smithi096.stdout:6/226: dwrite d8/d10/f3d [0,4194304] 0 2024-03-24T10:46:07.115 INFO:tasks.workunit.client.1.smithi096.stdout:6/227: chown d8/lf 3 1 2024-03-24T10:46:07.116 INFO:tasks.workunit.client.1.smithi096.stdout:0/138: creat d0/d11/d20/f26 x:0 0 0 2024-03-24T10:46:07.116 INFO:tasks.workunit.client.1.smithi096.stdout:0/139: truncate d0/d11/d20/f26 89590 0 2024-03-24T10:46:07.117 INFO:tasks.workunit.client.1.smithi096.stdout:1/159: dread d3/f12 [0,4194304] 0 2024-03-24T10:46:07.121 INFO:tasks.workunit.client.1.smithi096.stdout:9/147: write d9/f16 [237385,25585] 0 2024-03-24T10:46:07.121 INFO:tasks.workunit.client.1.smithi096.stdout:9/148: truncate d9/f13 2337402 0 2024-03-24T10:46:07.123 INFO:tasks.workunit.client.1.smithi096.stdout:4/148: unlink d1/f3 0 2024-03-24T10:46:07.123 INFO:tasks.workunit.client.1.smithi096.stdout:4/149: chown l0 0 1 2024-03-24T10:46:07.127 INFO:tasks.workunit.client.1.smithi096.stdout:6/228: creat d8/d12/d32/d38/f44 x:0 0 0 2024-03-24T10:46:07.132 INFO:tasks.workunit.client.1.smithi096.stdout:0/140: mknod d0/d2/c27 0 2024-03-24T10:46:07.132 INFO:tasks.workunit.client.1.smithi096.stdout:0/141: dread - d0/f22 zero size 2024-03-24T10:46:07.137 INFO:tasks.workunit.client.1.smithi096.stdout:6/229: dread d8/d10/d19/f1b [0,4194304] 0 2024-03-24T10:46:07.137 INFO:tasks.workunit.client.1.smithi096.stdout:1/160: mkdir d3/d36 0 2024-03-24T10:46:07.141 INFO:tasks.workunit.client.1.smithi096.stdout:4/150: dwrite d1/ff [0,4194304] 0 2024-03-24T10:46:07.141 INFO:tasks.workunit.client.1.smithi096.stdout:8/235: rmdir d8/d11/d22/d23/d1e 39 2024-03-24T10:46:07.144 INFO:tasks.workunit.client.1.smithi096.stdout:9/149: creat d9/f2e x:0 0 0 2024-03-24T10:46:07.153 INFO:tasks.workunit.client.1.smithi096.stdout:1/161: creat d3/d23/d2f/f37 x:0 0 0 2024-03-24T10:46:07.157 INFO:tasks.workunit.client.1.smithi096.stdout:4/151: symlink d1/d2b/l30 0 2024-03-24T10:46:07.157 INFO:tasks.workunit.client.1.smithi096.stdout:4/152: dread - d1/d2b/f2f zero size 2024-03-24T10:46:07.159 INFO:tasks.workunit.client.1.smithi096.stdout:8/236: symlink d8/d11/d22/d4c/l4f 0 2024-03-24T10:46:07.165 INFO:tasks.workunit.client.1.smithi096.stdout:6/230: dwrite d8/f31 [0,4194304] 0 2024-03-24T10:46:07.165 INFO:tasks.workunit.client.1.smithi096.stdout:6/231: write d8/d12/d32/f3c [918810,37171] 0 2024-03-24T10:46:07.165 INFO:tasks.workunit.client.1.smithi096.stdout:9/150: dwrite d9/d15/f25 [0,4194304] 0 2024-03-24T10:46:07.165 INFO:tasks.workunit.client.1.smithi096.stdout:9/151: truncate d9/db/f22 222209 0 2024-03-24T10:46:07.167 INFO:tasks.workunit.client.1.smithi096.stdout:0/142: rmdir d0/d11 39 2024-03-24T10:46:07.171 INFO:tasks.workunit.client.1.smithi096.stdout:9/152: dread d9/f16 [0,4194304] 0 2024-03-24T10:46:07.171 INFO:tasks.workunit.client.1.smithi096.stdout:9/153: creat d9/f2f x:0 0 0 2024-03-24T10:46:07.171 INFO:tasks.workunit.client.1.smithi096.stdout:9/154: write d9/f19 [70725,129545] 0 2024-03-24T10:46:07.171 INFO:tasks.workunit.client.1.smithi096.stdout:9/155: write d9/f2e [309342,124394] 0 2024-03-24T10:46:07.173 INFO:tasks.workunit.client.1.smithi096.stdout:1/162: dwrite d3/f34 [0,4194304] 0 2024-03-24T10:46:07.174 INFO:tasks.workunit.client.1.smithi096.stdout:6/232: read d8/f14 [3732763,23343] 0 2024-03-24T10:46:07.178 INFO:tasks.workunit.client.1.smithi096.stdout:8/237: dwrite d8/d11/d22/d23/d1d/f38 [0,4194304] 0 2024-03-24T10:46:07.179 INFO:tasks.workunit.client.1.smithi096.stdout:0/143: mknod d0/d11/d16/c28 0 2024-03-24T10:46:07.180 INFO:tasks.workunit.client.1.smithi096.stdout:4/153: symlink d1/d11/l31 0 2024-03-24T10:46:07.180 INFO:tasks.workunit.client.1.smithi096.stdout:4/154: readlink d1/d11/l31 0 2024-03-24T10:46:07.180 INFO:tasks.workunit.client.1.smithi096.stdout:4/155: fdatasync d1/fd 0 2024-03-24T10:46:07.188 INFO:tasks.workunit.client.1.smithi096.stdout:4/156: write d1/ff [1586651,103811] 0 2024-03-24T10:46:07.189 INFO:tasks.workunit.client.1.smithi096.stdout:1/163: creat d3/d23/f38 x:0 0 0 2024-03-24T10:46:07.191 INFO:tasks.workunit.client.1.smithi096.stdout:6/233: unlink d8/d12/f37 0 2024-03-24T10:46:07.192 INFO:tasks.workunit.client.1.smithi096.stdout:1/164: dread d3/f22 [0,4194304] 0 2024-03-24T10:46:07.196 INFO:tasks.workunit.client.1.smithi096.stdout:4/157: dread d1/f8 [0,4194304] 0 2024-03-24T10:46:07.196 INFO:tasks.workunit.client.1.smithi096.stdout:4/158: write d1/f1f [477593,102755] 0 2024-03-24T10:46:07.201 INFO:tasks.workunit.client.1.smithi096.stdout:4/159: dread d1/f10 [0,4194304] 0 2024-03-24T10:46:07.201 INFO:tasks.workunit.client.1.smithi096.stdout:4/160: write d1/f24 [5119284,109193] 0 2024-03-24T10:46:07.201 INFO:tasks.workunit.client.1.smithi096.stdout:4/161: write d1/d2b/f2f [51328,117070] 0 2024-03-24T10:46:07.205 INFO:tasks.workunit.client.1.smithi096.stdout:8/238: symlink d8/d11/d22/d23/d37/l50 0 2024-03-24T10:46:07.206 INFO:tasks.workunit.client.1.smithi096.stdout:0/144: symlink d0/d2/l29 0 2024-03-24T10:46:07.208 INFO:tasks.workunit.client.1.smithi096.stdout:9/156: rename d9/db/c1d to d9/db/c30 0 2024-03-24T10:46:07.210 INFO:tasks.workunit.client.1.smithi096.stdout:6/234: link d8/d12/f1f d8/d12/d28/d36/f45 0 2024-03-24T10:46:07.214 INFO:tasks.workunit.client.1.smithi096.stdout:3/239: sync 2024-03-24T10:46:07.214 INFO:tasks.workunit.client.1.smithi096.stdout:2/122: sync 2024-03-24T10:46:07.214 INFO:tasks.workunit.client.1.smithi096.stdout:3/240: chown d0/d13/d45/d52/f53 1 1 2024-03-24T10:46:07.214 INFO:tasks.workunit.client.1.smithi096.stdout:3/241: rename d0/d13 to d0/d13/d45/d52/d56 22 2024-03-24T10:46:07.215 INFO:tasks.workunit.client.1.smithi096.stdout:1/165: unlink d3/d23/l32 0 2024-03-24T10:46:07.219 INFO:tasks.workunit.client.1.smithi096.stdout:4/162: mkdir d1/d32 0 2024-03-24T10:46:07.223 INFO:tasks.workunit.client.1.smithi096.stdout:8/239: mknod d8/d11/d22/d23/d1a/d46/d2e/c51 0 2024-03-24T10:46:07.225 INFO:tasks.workunit.client.1.smithi096.stdout:5/150: sync 2024-03-24T10:46:07.225 INFO:tasks.workunit.client.1.smithi096.stdout:7/208: sync 2024-03-24T10:46:07.226 INFO:tasks.workunit.client.1.smithi096.stdout:0/145: creat d0/d11/d16/d21/f2a x:0 0 0 2024-03-24T10:46:07.226 INFO:tasks.workunit.client.1.smithi096.stdout:0/146: fsync d0/f22 0 2024-03-24T10:46:07.227 INFO:tasks.workunit.client.1.smithi096.stdout:9/157: mkdir d9/db/d23/d31 0 2024-03-24T10:46:07.234 INFO:tasks.workunit.client.1.smithi096.stdout:6/235: link d8/d10/c25 d8/d10/d19/c46 0 2024-03-24T10:46:07.237 INFO:tasks.workunit.client.1.smithi096.stdout:3/242: dwrite d0/d9/f1a [0,4194304] 0 2024-03-24T10:46:07.237 INFO:tasks.workunit.client.1.smithi096.stdout:3/243: creat d0/d13/f57 x:0 0 0 2024-03-24T10:46:07.237 INFO:tasks.workunit.client.1.smithi096.stdout:3/244: chown d0/d9/f2e 871 1 2024-03-24T10:46:07.237 INFO:tasks.workunit.client.1.smithi096.stdout:3/245: write d0/d13/d45/d47/f4e [545145,78208] 0 2024-03-24T10:46:07.239 INFO:tasks.workunit.client.1.smithi096.stdout:5/151: mkdir d2/d29/d32 0 2024-03-24T10:46:07.239 INFO:tasks.workunit.client.1.smithi096.stdout:4/163: symlink d1/d32/l33 0 2024-03-24T10:46:07.239 INFO:tasks.workunit.client.1.smithi096.stdout:5/152: write d2/d7/d8/df/f2d [3954533,89404] 0 2024-03-24T10:46:07.240 INFO:tasks.workunit.client.1.smithi096.stdout:7/209: symlink d0/db/d3a/l4c 0 2024-03-24T10:46:07.241 INFO:tasks.workunit.client.1.smithi096.stdout:0/147: mkdir d0/d11/d20/d2b 0 2024-03-24T10:46:07.241 INFO:tasks.workunit.client.1.smithi096.stdout:7/210: chown d0/d21/f2e 22 1 2024-03-24T10:46:07.243 INFO:tasks.workunit.client.1.smithi096.stdout:6/236: creat d8/d12/d32/f47 x:0 0 0 2024-03-24T10:46:07.243 INFO:tasks.workunit.client.1.smithi096.stdout:6/237: write f7 [5253812,121630] 0 2024-03-24T10:46:07.243 INFO:tasks.workunit.client.1.smithi096.stdout:6/238: stat d8/d10/f3d 0 2024-03-24T10:46:07.246 INFO:tasks.workunit.client.1.smithi096.stdout:7/211: dread d0/d13/f35 [0,4194304] 0 2024-03-24T10:46:07.246 INFO:tasks.workunit.client.1.smithi096.stdout:7/212: chown d0/d13/f9 425 1 2024-03-24T10:46:07.248 INFO:tasks.workunit.client.1.smithi096.stdout:0/148: dread d0/d11/d20/f1f [0,4194304] 0 2024-03-24T10:46:07.248 INFO:tasks.workunit.client.1.smithi096.stdout:0/149: write d0/fe [1239240,91095] 0 2024-03-24T10:46:07.249 INFO:tasks.workunit.client.1.smithi096.stdout:0/150: dread d0/d11/d20/f26 [0,4194304] 0 2024-03-24T10:46:07.249 INFO:tasks.workunit.client.1.smithi096.stdout:0/151: chown d0/c12 3069 1 2024-03-24T10:46:07.249 INFO:tasks.workunit.client.1.smithi096.stdout:4/164: mknod d1/d2b/c34 0 2024-03-24T10:46:07.250 INFO:tasks.workunit.client.1.smithi096.stdout:0/152: dread d0/d11/d20/f26 [0,4194304] 0 2024-03-24T10:46:07.250 INFO:tasks.workunit.client.1.smithi096.stdout:0/153: read - d0/f22 zero size 2024-03-24T10:46:07.253 INFO:tasks.workunit.client.1.smithi096.stdout:3/246: dread d0/f1 [0,4194304] 0 2024-03-24T10:46:07.254 INFO:tasks.workunit.client.1.smithi096.stdout:6/239: link d8/f31 d8/d10/d19/f48 0 2024-03-24T10:46:07.254 INFO:tasks.workunit.client.1.smithi096.stdout:6/240: chown d8/d10/d19/f34 667610 1 2024-03-24T10:46:07.254 INFO:tasks.workunit.client.1.smithi096.stdout:4/165: creat d1/f35 x:0 0 0 2024-03-24T10:46:07.254 INFO:tasks.workunit.client.1.smithi096.stdout:4/166: chown d1/d11/l2c 25297959 1 2024-03-24T10:46:07.254 INFO:tasks.workunit.client.1.smithi096.stdout:4/167: write d1/f8 [4069874,95961] 0 2024-03-24T10:46:07.254 INFO:tasks.workunit.client.1.smithi096.stdout:4/168: read d1/f8 [3293333,61352] 0 2024-03-24T10:46:07.254 INFO:tasks.workunit.client.1.smithi096.stdout:0/154: mkdir d0/d11/d24/d2c 0 2024-03-24T10:46:07.257 INFO:tasks.workunit.client.1.smithi096.stdout:6/241: link d8/d10/c42 d8/d10/d19/c49 0 2024-03-24T10:46:07.258 INFO:tasks.workunit.client.1.smithi096.stdout:2/123: dwrite d5/dc/f21 [0,4194304] 0 2024-03-24T10:46:07.258 INFO:tasks.workunit.client.1.smithi096.stdout:2/124: truncate d5/dc/d12/f31 416588 0 2024-03-24T10:46:07.258 INFO:tasks.workunit.client.1.smithi096.stdout:2/125: creat d5/de/d2a/f35 x:0 0 0 2024-03-24T10:46:07.260 INFO:tasks.workunit.client.1.smithi096.stdout:8/240: dwrite f1 [4194304,4194304] 0 2024-03-24T10:46:07.260 INFO:tasks.workunit.client.1.smithi096.stdout:8/241: write d8/d11/d44/d45/f4d [1039429,29021] 0 2024-03-24T10:46:07.260 INFO:tasks.workunit.client.1.smithi096.stdout:8/242: readlink d8/l20 0 2024-03-24T10:46:07.260 INFO:tasks.workunit.client.1.smithi096.stdout:5/153: dwrite d2/d7/d8/df/f1f [0,4194304] 0 2024-03-24T10:46:07.263 INFO:tasks.workunit.client.1.smithi096.stdout:4/169: truncate d1/f10 2654172 0 2024-03-24T10:46:07.263 INFO:tasks.workunit.client.1.smithi096.stdout:1/166: rmdir d3 39 2024-03-24T10:46:07.266 INFO:tasks.workunit.client.1.smithi096.stdout:0/155: creat d0/d11/d24/f2d x:0 0 0 2024-03-24T10:46:07.267 INFO:tasks.workunit.client.1.smithi096.stdout:7/213: dwrite d0/db/d26/f44 [0,4194304] 0 2024-03-24T10:46:07.268 INFO:tasks.workunit.client.1.smithi096.stdout:7/214: chown d0/d21/d3c 1939 1 2024-03-24T10:46:07.268 INFO:tasks.workunit.client.1.smithi096.stdout:7/215: write d0/d21/f2e [5024072,96745] 0 2024-03-24T10:46:07.268 INFO:tasks.workunit.client.1.smithi096.stdout:7/216: creat d0/db/d3a/f4d x:0 0 0 2024-03-24T10:46:07.271 INFO:tasks.workunit.client.1.smithi096.stdout:6/242: creat d8/f4a x:0 0 0 2024-03-24T10:46:07.272 INFO:tasks.workunit.client.1.smithi096.stdout:4/170: read d1/f1c [3704533,102428] 0 2024-03-24T10:46:07.277 INFO:tasks.workunit.client.1.smithi096.stdout:3/247: dwrite d0/d13/d45/d52/f53 [0,4194304] 0 2024-03-24T10:46:07.286 INFO:tasks.workunit.client.1.smithi096.stdout:3/248: dread d0/d13/d1e/f2c [0,4194304] 0 2024-03-24T10:46:07.286 INFO:tasks.workunit.client.1.smithi096.stdout:3/249: readlink d0/d9/d34/d39/d48/l55 0 2024-03-24T10:46:07.287 INFO:tasks.workunit.client.1.smithi096.stdout:2/126: creat d5/dc/d12/d23/f36 x:0 0 0 2024-03-24T10:46:07.290 INFO:tasks.workunit.client.1.smithi096.stdout:8/243: mkdir d8/d11/d22/d23/d52 0 2024-03-24T10:46:07.297 INFO:tasks.workunit.client.1.smithi096.stdout:5/154: mkdir d2/d29/d32/d33 0 2024-03-24T10:46:07.298 INFO:tasks.workunit.client.1.smithi096.stdout:1/167: creat d3/d24/f39 x:0 0 0 2024-03-24T10:46:07.298 INFO:tasks.workunit.client.1.smithi096.stdout:1/168: chown d3/la 11542 1 2024-03-24T10:46:07.298 INFO:tasks.workunit.client.1.smithi096.stdout:1/169: dread - d3/d23/d2f/f37 zero size 2024-03-24T10:46:07.298 INFO:tasks.workunit.client.1.smithi096.stdout:0/156: creat d0/d11/d20/f2e x:0 0 0 2024-03-24T10:46:07.298 INFO:tasks.workunit.client.1.smithi096.stdout:0/157: dread - d0/f22 zero size 2024-03-24T10:46:07.299 INFO:tasks.workunit.client.1.smithi096.stdout:6/243: dwrite d8/d12/d32/f47 [0,4194304] 0 2024-03-24T10:46:07.299 INFO:tasks.workunit.client.1.smithi096.stdout:6/244: truncate d8/f29 427249 0 2024-03-24T10:46:07.299 INFO:tasks.workunit.client.1.smithi096.stdout:7/217: dwrite d0/db/f1b [0,4194304] 0 2024-03-24T10:46:07.308 INFO:tasks.workunit.client.1.smithi096.stdout:4/171: link d1/d2b/c34 d1/d2b/c36 0 2024-03-24T10:46:07.308 INFO:tasks.workunit.client.1.smithi096.stdout:6/245: write f7 [2788177,11482] 0 2024-03-24T10:46:07.308 INFO:tasks.workunit.client.1.smithi096.stdout:6/246: read - d8/f4a zero size 2024-03-24T10:46:07.314 INFO:tasks.workunit.client.1.smithi096.stdout:4/172: dread d1/f1c [0,4194304] 0 2024-03-24T10:46:07.314 INFO:tasks.workunit.client.1.smithi096.stdout:8/244: dwrite d8/d11/d44/d45/f4d [0,4194304] 0 2024-03-24T10:46:07.314 INFO:tasks.workunit.client.1.smithi096.stdout:0/158: dwrite d0/f22 [0,4194304] 0 2024-03-24T10:46:07.315 INFO:tasks.workunit.client.1.smithi096.stdout:4/173: dread d1/d11/f19 [0,4194304] 0 2024-03-24T10:46:07.317 INFO:tasks.workunit.client.1.smithi096.stdout:4/174: dread d1/f1f [0,4194304] 0 2024-03-24T10:46:07.321 INFO:tasks.workunit.client.1.smithi096.stdout:3/250: rename d0/d13/d45/d52/l54 to d0/d13/d1e/l58 0 2024-03-24T10:46:07.322 INFO:tasks.workunit.client.1.smithi096.stdout:8/245: dread d8/d11/d22/d23/d1d/f38 [0,4194304] 0 2024-03-24T10:46:07.323 INFO:tasks.workunit.client.1.smithi096.stdout:8/246: truncate d8/d11/d22/d23/d1a/f2d 1465992 0 2024-03-24T10:46:07.324 INFO:tasks.workunit.client.1.smithi096.stdout:2/127: link f2 d5/de/d2a/f37 0 2024-03-24T10:46:07.324 INFO:tasks.workunit.client.1.smithi096.stdout:2/128: readlink d5/l10 0 2024-03-24T10:46:07.329 INFO:tasks.workunit.client.1.smithi096.stdout:1/170: link d3/c4 d3/d24/c3a 0 2024-03-24T10:46:07.334 INFO:tasks.workunit.client.1.smithi096.stdout:7/218: mknod d0/d13/c4e 0 2024-03-24T10:46:07.334 INFO:tasks.workunit.client.1.smithi096.stdout:7/219: readlink d0/db/l3f 0 2024-03-24T10:46:07.338 INFO:tasks.workunit.client.1.smithi096.stdout:1/171: write d3/f1f [2226175,81603] 0 2024-03-24T10:46:07.341 INFO:tasks.workunit.client.1.smithi096.stdout:0/159: link d0/d11/d16/d21/f2a d0/d11/d16/f2f 0 2024-03-24T10:46:07.341 INFO:tasks.workunit.client.1.smithi096.stdout:0/160: chown d0/d11/d16/d21/d25 16991545 1 2024-03-24T10:46:07.341 INFO:tasks.workunit.client.1.smithi096.stdout:0/161: creat d0/d11/d24/f30 x:0 0 0 2024-03-24T10:46:07.341 INFO:tasks.workunit.client.1.smithi096.stdout:0/162: chown d0/fe 98 1 2024-03-24T10:46:07.344 INFO:tasks.workunit.client.1.smithi096.stdout:5/155: rename d2/d7/c1a to d2/d7/d8/df/d14/c34 0 2024-03-24T10:46:07.346 INFO:tasks.workunit.client.1.smithi096.stdout:3/251: truncate d0/d9/f2e 935079 0 2024-03-24T10:46:07.346 INFO:tasks.workunit.client.1.smithi096.stdout:3/252: fdatasync d0/d13/d1e/f2d 0 2024-03-24T10:46:07.348 INFO:tasks.workunit.client.1.smithi096.stdout:8/247: mknod d8/d11/d22/d23/d1d/c53 0 2024-03-24T10:46:07.350 INFO:tasks.workunit.client.1.smithi096.stdout:7/220: mkdir d0/db/d4f 0 2024-03-24T10:46:07.354 INFO:tasks.workunit.client.1.smithi096.stdout:1/172: creat d3/d23/d2f/f3b x:0 0 0 2024-03-24T10:46:07.357 INFO:tasks.workunit.client.1.smithi096.stdout:0/163: getdents d0/d11/d16/d21 0 2024-03-24T10:46:07.357 INFO:tasks.workunit.client.1.smithi096.stdout:0/164: readlink d0/d2/l29 0 2024-03-24T10:46:07.357 INFO:tasks.workunit.client.1.smithi096.stdout:0/165: chown d0/d11/d16 15556 1 2024-03-24T10:46:07.359 INFO:tasks.workunit.client.1.smithi096.stdout:7/221: write d0/d21/f27 [3649103,99826] 0 2024-03-24T10:46:07.359 INFO:tasks.workunit.client.1.smithi096.stdout:6/247: rename d8/d10/f3f to d8/d12/d32/d38/f4b 0 2024-03-24T10:46:07.363 INFO:tasks.workunit.client.1.smithi096.stdout:3/253: rmdir d0/d9/d34/d39 39 2024-03-24T10:46:07.364 INFO:tasks.workunit.client.1.smithi096.stdout:5/156: link d2/d7/d8/df/d14/c34 d2/d7/d8/d18/c35 0 2024-03-24T10:46:07.367 INFO:tasks.workunit.client.1.smithi096.stdout:8/248: unlink d8/d11/f31 0 2024-03-24T10:46:07.369 INFO:tasks.workunit.client.1.smithi096.stdout:2/129: mkdir d5/d38 0 2024-03-24T10:46:07.371 INFO:tasks.workunit.client.1.smithi096.stdout:8/249: dread d8/f14 [0,4194304] 0 2024-03-24T10:46:07.371 INFO:tasks.workunit.client.1.smithi096.stdout:8/250: fsync d8/f16 0 2024-03-24T10:46:07.371 INFO:tasks.workunit.client.1.smithi096.stdout:1/173: symlink d3/d23/d2f/l3c 0 2024-03-24T10:46:07.371 INFO:tasks.workunit.client.1.smithi096.stdout:1/174: getdents d3/d36 0 2024-03-24T10:46:07.372 INFO:tasks.workunit.client.1.smithi096.stdout:1/175: creat d3/d23/d2f/f3d x:0 0 0 2024-03-24T10:46:07.372 INFO:tasks.workunit.client.1.smithi096.stdout:0/166: truncate d0/f5 376631 0 2024-03-24T10:46:07.376 INFO:tasks.workunit.client.1.smithi096.stdout:7/222: mknod d0/d13/d32/c50 0 2024-03-24T10:46:07.381 INFO:tasks.workunit.client.1.smithi096.stdout:6/248: truncate d8/d10/d19/f1b 2173846 0 2024-03-24T10:46:07.383 INFO:tasks.workunit.client.1.smithi096.stdout:3/254: mknod d0/d9/d34/d39/d48/c59 0 2024-03-24T10:46:07.388 INFO:tasks.workunit.client.1.smithi096.stdout:5/157: stat d2/d7/l22 0 2024-03-24T10:46:07.391 INFO:tasks.workunit.client.1.smithi096.stdout:7/223: dwrite d0/d21/d3c/f43 [0,4194304] 0 2024-03-24T10:46:07.392 INFO:tasks.workunit.client.1.smithi096.stdout:2/130: mknod d5/de/d2a/c39 0 2024-03-24T10:46:07.394 INFO:tasks.workunit.client.1.smithi096.stdout:8/251: creat d8/d11/d22/d23/d1a/d46/d2e/f54 x:0 0 0 2024-03-24T10:46:07.399 INFO:tasks.workunit.client.1.smithi096.stdout:1/176: rename d3/f6 to d3/d24/f3e 0 2024-03-24T10:46:07.399 INFO:tasks.workunit.client.1.smithi096.stdout:5/158: rename d2/d7 to d2/d7/d8/df/d14/d36 22 2024-03-24T10:46:07.400 INFO:tasks.workunit.client.1.smithi096.stdout:0/167: stat d0/c6 0 2024-03-24T10:46:07.404 INFO:tasks.workunit.client.1.smithi096.stdout:7/224: unlink d0/d21/d3c/f43 0 2024-03-24T10:46:07.407 INFO:tasks.workunit.client.1.smithi096.stdout:2/131: link d5/dc/f27 d5/d38/f3a 0 2024-03-24T10:46:07.407 INFO:tasks.workunit.client.1.smithi096.stdout:2/132: chown d5/dc/d12/d23/f2c 6967992 1 2024-03-24T10:46:07.408 INFO:tasks.workunit.client.1.smithi096.stdout:8/252: mknod d8/d11/d22/d23/d1d/c55 0 2024-03-24T10:46:07.413 INFO:tasks.workunit.client.1.smithi096.stdout:5/159: mkdir d2/d29/d32/d33/d37 0 2024-03-24T10:46:07.415 INFO:tasks.workunit.client.1.smithi096.stdout:6/249: dwrite d8/f9 [0,4194304] 0 2024-03-24T10:46:07.415 INFO:tasks.workunit.client.1.smithi096.stdout:0/168: creat d0/d11/d16/f31 x:0 0 0 2024-03-24T10:46:07.417 INFO:tasks.workunit.client.1.smithi096.stdout:2/133: rename d5/dc/l2d to d5/dc/d12/l3b 0 2024-03-24T10:46:07.418 INFO:tasks.workunit.client.1.smithi096.stdout:3/255: dwrite d0/d13/d45/f4c [0,4194304] 0 2024-03-24T10:46:07.418 INFO:tasks.workunit.client.1.smithi096.stdout:3/256: chown d0/d13/d1e/l58 27 1 2024-03-24T10:46:07.418 INFO:tasks.workunit.client.1.smithi096.stdout:7/225: dread d0/d13/f15 [0,4194304] 0 2024-03-24T10:46:07.418 INFO:tasks.workunit.client.1.smithi096.stdout:3/257: creat d0/d9/d34/f5a x:0 0 0 2024-03-24T10:46:07.418 INFO:tasks.workunit.client.1.smithi096.stdout:3/258: chown d0/d9/d22 87042 1 2024-03-24T10:46:07.418 INFO:tasks.workunit.client.1.smithi096.stdout:3/259: creat d0/d13/d45/d52/f5b x:0 0 0 2024-03-24T10:46:07.418 INFO:tasks.workunit.client.1.smithi096.stdout:3/260: creat d0/f5c x:0 0 0 2024-03-24T10:46:07.419 INFO:tasks.workunit.client.1.smithi096.stdout:8/253: creat d8/d11/d22/d23/d1d/d3d/f56 x:0 0 0 2024-03-24T10:46:07.422 INFO:tasks.workunit.client.1.smithi096.stdout:5/160: creat d2/d29/f38 x:0 0 0 2024-03-24T10:46:07.425 INFO:tasks.workunit.client.1.smithi096.stdout:0/169: creat d0/d11/d24/d2c/f32 x:0 0 0 2024-03-24T10:46:07.426 INFO:tasks.workunit.client.1.smithi096.stdout:2/134: mknod d5/dc/c3c 0 2024-03-24T10:46:07.426 INFO:tasks.workunit.client.1.smithi096.stdout:2/135: read - d5/dc/f27 zero size 2024-03-24T10:46:07.427 INFO:tasks.workunit.client.1.smithi096.stdout:1/177: getdents d3/d23/d2f 0 2024-03-24T10:46:07.431 INFO:tasks.workunit.client.1.smithi096.stdout:7/226: creat d0/d21/f51 x:0 0 0 2024-03-24T10:46:07.434 INFO:tasks.workunit.client.1.smithi096.stdout:1/178: write d3/f21 [164683,117734] 0 2024-03-24T10:46:07.436 INFO:tasks.workunit.client.1.smithi096.stdout:8/254: dwrite d8/d11/d22/d23/d1a/d46/d2e/f54 [0,4194304] 0 2024-03-24T10:46:07.436 INFO:tasks.workunit.client.1.smithi096.stdout:8/255: write d8/d11/f27 [692094,24026] 0 2024-03-24T10:46:07.436 INFO:tasks.workunit.client.1.smithi096.stdout:5/161: creat d2/f39 x:0 0 0 2024-03-24T10:46:07.438 INFO:tasks.workunit.client.1.smithi096.stdout:0/170: rename d0/d11/d16/f31 to d0/d11/d16/d21/f33 0 2024-03-24T10:46:07.438 INFO:tasks.workunit.client.1.smithi096.stdout:0/171: fdatasync d0/d11/d24/f2d 0 2024-03-24T10:46:07.441 INFO:tasks.workunit.client.1.smithi096.stdout:2/136: creat d5/de/f3d x:0 0 0 2024-03-24T10:46:07.441 INFO:tasks.workunit.client.1.smithi096.stdout:8/256: dread d8/d11/d22/d23/f1c [0,4194304] 0 2024-03-24T10:46:07.443 INFO:tasks.workunit.client.1.smithi096.stdout:7/227: link d0/d13/c12 d0/db/d3a/d4b/c52 0 2024-03-24T10:46:07.447 INFO:tasks.workunit.client.1.smithi096.stdout:7/228: dread d0/d13/f15 [0,4194304] 0 2024-03-24T10:46:07.447 INFO:tasks.workunit.client.1.smithi096.stdout:1/179: rename d3/f12 to d3/f3f 0 2024-03-24T10:46:07.447 INFO:tasks.workunit.client.1.smithi096.stdout:1/180: dread - d3/d23/f38 zero size 2024-03-24T10:46:07.453 INFO:tasks.workunit.client.1.smithi096.stdout:5/162: rmdir d2/d7/d8 39 2024-03-24T10:46:07.454 INFO:tasks.workunit.client.1.smithi096.stdout:0/172: symlink d0/d11/l34 0 2024-03-24T10:46:07.459 INFO:tasks.workunit.client.1.smithi096.stdout:8/257: dwrite d8/d11/f19 [0,4194304] 0 2024-03-24T10:46:07.459 INFO:tasks.workunit.client.1.smithi096.stdout:8/258: chown d8/d11/d22/d23/d1d/f30 7550946 1 2024-03-24T10:46:07.462 INFO:tasks.workunit.client.1.smithi096.stdout:7/229: symlink d0/db/d1c/d39/l53 0 2024-03-24T10:46:07.465 INFO:tasks.workunit.client.1.smithi096.stdout:1/181: symlink d3/d23/d2f/l40 0 2024-03-24T10:46:07.468 INFO:tasks.workunit.client.1.smithi096.stdout:5/163: symlink d2/d7/d8/df/l3a 0 2024-03-24T10:46:07.468 INFO:tasks.workunit.client.1.smithi096.stdout:5/164: truncate d2/d7/d8/f12 2603314 0 2024-03-24T10:46:07.470 INFO:tasks.workunit.client.1.smithi096.stdout:0/173: symlink d0/d11/l35 0 2024-03-24T10:46:07.470 INFO:tasks.workunit.client.1.smithi096.stdout:0/174: write d0/d11/f1c [4932505,127882] 0 2024-03-24T10:46:07.474 INFO:tasks.workunit.client.1.smithi096.stdout:2/137: rename c1 to d5/d38/c3e 0 2024-03-24T10:46:07.474 INFO:tasks.workunit.client.1.smithi096.stdout:2/138: fsync f3 0 2024-03-24T10:46:07.474 INFO:tasks.workunit.client.1.smithi096.stdout:2/139: dread - d5/dc/f34 zero size 2024-03-24T10:46:07.474 INFO:tasks.workunit.client.1.smithi096.stdout:2/140: write d5/de/f15 [1364999,87588] 0 2024-03-24T10:46:07.474 INFO:tasks.workunit.client.1.smithi096.stdout:2/141: chown d5/dc/d12/f31 358828571 1 2024-03-24T10:46:07.475 INFO:tasks.workunit.client.1.smithi096.stdout:8/259: creat d8/d11/d22/d23/d52/f57 x:0 0 0 2024-03-24T10:46:07.482 INFO:tasks.workunit.client.1.smithi096.stdout:7/230: link d0/d13/l6 d0/db/d1c/l54 0 2024-03-24T10:46:07.482 INFO:tasks.workunit.client.1.smithi096.stdout:7/231: dread - d0/db/d3a/f4d zero size 2024-03-24T10:46:07.482 INFO:tasks.workunit.client.1.smithi096.stdout:9/158: sync 2024-03-24T10:46:07.486 INFO:tasks.workunit.client.1.smithi096.stdout:0/175: creat d0/d11/d16/d21/d25/f36 x:0 0 0 2024-03-24T10:46:07.488 INFO:tasks.workunit.client.1.smithi096.stdout:1/182: rename c2 to d3/d23/d2f/c41 0 2024-03-24T10:46:07.488 INFO:tasks.workunit.client.1.smithi096.stdout:1/183: write d3/ff [3188758,100387] 0 2024-03-24T10:46:07.489 INFO:tasks.workunit.client.1.smithi096.stdout:8/260: creat d8/f58 x:0 0 0 2024-03-24T10:46:07.490 INFO:tasks.workunit.client.1.smithi096.stdout:5/165: symlink d2/l3b 0 2024-03-24T10:46:07.491 INFO:tasks.workunit.client.1.smithi096.stdout:2/142: dwrite f2 [0,4194304] 0 2024-03-24T10:46:07.491 INFO:tasks.workunit.client.1.smithi096.stdout:2/143: fdatasync d5/de/d2a/f2b 0 2024-03-24T10:46:07.491 INFO:tasks.workunit.client.1.smithi096.stdout:2/144: dread - d5/de/d2a/f35 zero size 2024-03-24T10:46:07.491 INFO:tasks.workunit.client.1.smithi096.stdout:0/176: stat d0/d11/c14 0 2024-03-24T10:46:07.492 INFO:tasks.workunit.client.1.smithi096.stdout:1/184: creat d3/d23/f42 x:0 0 0 2024-03-24T10:46:07.492 INFO:tasks.workunit.client.1.smithi096.stdout:1/185: dread - d3/d24/f35 zero size 2024-03-24T10:46:07.495 INFO:tasks.workunit.client.1.smithi096.stdout:1/186: mknod d3/d36/c43 0 2024-03-24T10:46:07.496 INFO:tasks.workunit.client.1.smithi096.stdout:9/159: write d9/db/fe [856146,96428] 0 2024-03-24T10:46:07.496 INFO:tasks.workunit.client.1.smithi096.stdout:9/160: fdatasync d9/f19 0 2024-03-24T10:46:07.496 INFO:tasks.workunit.client.1.smithi096.stdout:9/161: write d9/f16 [1383741,15504] 0 2024-03-24T10:46:07.497 INFO:tasks.workunit.client.1.smithi096.stdout:2/145: dread d5/dc/f2f [0,4194304] 0 2024-03-24T10:46:07.498 INFO:tasks.workunit.client.1.smithi096.stdout:1/187: mknod d3/d36/c44 0 2024-03-24T10:46:07.498 INFO:tasks.workunit.client.1.smithi096.stdout:1/188: creat d3/d23/f45 x:0 0 0 2024-03-24T10:46:07.498 INFO:tasks.workunit.client.1.smithi096.stdout:1/189: readlink d3/l1a 0 2024-03-24T10:46:07.498 INFO:tasks.workunit.client.1.smithi096.stdout:1/190: write d3/f3f [5068275,68064] 0 2024-03-24T10:46:07.498 INFO:tasks.workunit.client.1.smithi096.stdout:1/191: creat d3/d24/f46 x:0 0 0 2024-03-24T10:46:07.499 INFO:tasks.workunit.client.1.smithi096.stdout:7/232: dwrite d0/d21/f51 [0,4194304] 0 2024-03-24T10:46:07.500 INFO:tasks.workunit.client.1.smithi096.stdout:8/261: dread d8/d11/d22/d23/d1a/d46/d2e/f54 [0,4194304] 0 2024-03-24T10:46:07.507 INFO:tasks.workunit.client.1.smithi096.stdout:1/192: dread d3/f21 [0,4194304] 0 2024-03-24T10:46:07.507 INFO:tasks.workunit.client.1.smithi096.stdout:1/193: chown d3/d23/d2f/l2d 9295 1 2024-03-24T10:46:07.507 INFO:tasks.workunit.client.1.smithi096.stdout:1/194: chown d3/d23/f38 433 1 2024-03-24T10:46:07.515 INFO:tasks.workunit.client.1.smithi096.stdout:7/233: getdents d0/db/d26 0 2024-03-24T10:46:07.518 INFO:tasks.workunit.client.1.smithi096.stdout:9/162: rename d9/db/c30 to d9/db/c32 0 2024-03-24T10:46:07.520 INFO:tasks.workunit.client.1.smithi096.stdout:2/146: symlink d5/de/l3f 0 2024-03-24T10:46:07.520 INFO:tasks.workunit.client.1.smithi096.stdout:2/147: dread - d5/d38/f3a zero size 2024-03-24T10:46:07.520 INFO:tasks.workunit.client.1.smithi096.stdout:2/148: dread - d5/de/d2a/f32 zero size 2024-03-24T10:46:07.523 INFO:tasks.workunit.client.1.smithi096.stdout:7/234: unlink d0/f24 0 2024-03-24T10:46:07.523 INFO:tasks.workunit.client.1.smithi096.stdout:0/177: dwrite d0/d11/d20/f1f [0,4194304] 0 2024-03-24T10:46:07.533 INFO:tasks.workunit.client.1.smithi096.stdout:2/149: rmdir d5/dc/d12/d23 39 2024-03-24T10:46:07.533 INFO:tasks.workunit.client.1.smithi096.stdout:2/150: creat d5/de/d2a/f40 x:0 0 0 2024-03-24T10:46:07.544 INFO:tasks.workunit.client.1.smithi096.stdout:5/166: dwrite d2/f5 [0,4194304] 0 2024-03-24T10:46:07.544 INFO:tasks.workunit.client.1.smithi096.stdout:5/167: stat d2/d7/d8 0 2024-03-24T10:46:07.545 INFO:tasks.workunit.client.1.smithi096.stdout:8/262: dwrite d8/d11/f27 [0,4194304] 0 2024-03-24T10:46:07.549 INFO:tasks.workunit.client.1.smithi096.stdout:0/178: rmdir d0 39 2024-03-24T10:46:07.549 INFO:tasks.workunit.client.1.smithi096.stdout:0/179: truncate d0/d11/d24/f2d 279376 0 2024-03-24T10:46:07.553 INFO:tasks.workunit.client.1.smithi096.stdout:9/163: dwrite d9/f2c [0,4194304] 0 2024-03-24T10:46:07.554 INFO:tasks.workunit.client.1.smithi096.stdout:1/195: dwrite d3/d23/d2f/f37 [0,4194304] 0 2024-03-24T10:46:07.557 INFO:tasks.workunit.client.1.smithi096.stdout:7/235: dwrite d0/db/d1c/f29 [0,4194304] 0 2024-03-24T10:46:07.557 INFO:tasks.workunit.client.1.smithi096.stdout:2/151: dwrite d5/dc/d12/f26 [0,4194304] 0 2024-03-24T10:46:07.557 INFO:tasks.workunit.client.1.smithi096.stdout:7/236: write d0/d21/f27 [3404508,63290] 0 2024-03-24T10:46:07.558 INFO:tasks.workunit.client.1.smithi096.stdout:8/263: getdents d8/d11/d44 0 2024-03-24T10:46:07.561 INFO:tasks.workunit.client.1.smithi096.stdout:8/264: dread d8/d11/d22/d23/d1d/f28 [0,4194304] 0 2024-03-24T10:46:07.568 INFO:tasks.workunit.client.1.smithi096.stdout:9/164: unlink d9/f24 0 2024-03-24T10:46:07.570 INFO:tasks.workunit.client.1.smithi096.stdout:0/180: dwrite d0/d11/f1c [0,4194304] 0 2024-03-24T10:46:07.570 INFO:tasks.workunit.client.1.smithi096.stdout:0/181: write d0/d11/d16/d21/f2a [840972,13378] 0 2024-03-24T10:46:07.570 INFO:tasks.workunit.client.1.smithi096.stdout:0/182: truncate d0/d11/d24/f30 384681 0 2024-03-24T10:46:07.573 INFO:tasks.workunit.client.1.smithi096.stdout:2/152: dwrite d5/dc/f25 [4194304,4194304] 0 2024-03-24T10:46:07.578 INFO:tasks.workunit.client.1.smithi096.stdout:9/165: read d9/d15/f1e [2782525,14090] 0 2024-03-24T10:46:07.578 INFO:tasks.workunit.client.1.smithi096.stdout:9/166: chown d9/d15/f27 43028065 1 2024-03-24T10:46:07.578 INFO:tasks.workunit.client.1.smithi096.stdout:9/167: fdatasync d9/f19 0 2024-03-24T10:46:07.578 INFO:tasks.workunit.client.1.smithi096.stdout:9/168: fsync d9/d15/f21 0 2024-03-24T10:46:07.579 INFO:tasks.workunit.client.1.smithi096.stdout:8/265: dwrite d8/d11/d22/d23/d1e/f33 [0,4194304] 0 2024-03-24T10:46:07.579 INFO:tasks.workunit.client.1.smithi096.stdout:8/266: fsync f1 0 2024-03-24T10:46:07.579 INFO:tasks.workunit.client.1.smithi096.stdout:8/267: write d8/d11/d22/d23/f1c [1915506,96356] 0 2024-03-24T10:46:07.579 INFO:tasks.workunit.client.1.smithi096.stdout:8/268: chown d8/d11/d22/d23/d1d/f2f 6 1 2024-03-24T10:46:07.579 INFO:tasks.workunit.client.1.smithi096.stdout:8/269: fsync d8/fc 0 2024-03-24T10:46:07.579 INFO:tasks.workunit.client.1.smithi096.stdout:8/270: stat d8/d11/d22/d23/d1a/d46/c40 0 2024-03-24T10:46:07.579 INFO:tasks.workunit.client.1.smithi096.stdout:8/271: write d8/d11/d22/d23/d1d/d3d/f4a [1151137,120300] 0 2024-03-24T10:46:07.582 INFO:tasks.workunit.client.1.smithi096.stdout:7/237: truncate d0/d13/f18 1599253 0 2024-03-24T10:46:07.586 INFO:tasks.workunit.client.1.smithi096.stdout:0/183: creat d0/d11/d20/d2b/f37 x:0 0 0 2024-03-24T10:46:07.587 INFO:tasks.workunit.client.1.smithi096.stdout:1/196: rmdir d3/d23 39 2024-03-24T10:46:07.589 INFO:tasks.workunit.client.1.smithi096.stdout:2/153: rename d5/dc/d12/d23/f36 to d5/dc/d12/f41 0 2024-03-24T10:46:07.590 INFO:tasks.workunit.client.1.smithi096.stdout:9/169: link d9/f16 d9/d15/d26/f33 0 2024-03-24T10:46:07.597 INFO:tasks.workunit.client.1.smithi096.stdout:8/272: stat d8/l20 0 2024-03-24T10:46:07.597 INFO:tasks.workunit.client.1.smithi096.stdout:8/273: write d8/d11/d22/d23/d1d/f2f [595805,34973] 0 2024-03-24T10:46:07.597 INFO:tasks.workunit.client.1.smithi096.stdout:8/274: write d8/d11/d22/d23/d1a/f26 [355323,101405] 0 2024-03-24T10:46:07.597 INFO:tasks.workunit.client.1.smithi096.stdout:4/175: sync 2024-03-24T10:46:07.600 INFO:tasks.workunit.client.1.smithi096.stdout:0/184: creat d0/d11/d24/f38 x:0 0 0 2024-03-24T10:46:07.601 INFO:tasks.workunit.client.1.smithi096.stdout:1/197: symlink d3/d23/l47 0 2024-03-24T10:46:07.603 INFO:tasks.workunit.client.1.smithi096.stdout:2/154: mknod d5/d38/c42 0 2024-03-24T10:46:07.604 INFO:tasks.workunit.client.1.smithi096.stdout:6/250: sync 2024-03-24T10:46:07.605 INFO:tasks.workunit.client.1.smithi096.stdout:9/170: creat d9/db/d23/d31/f34 x:0 0 0 2024-03-24T10:46:07.606 INFO:tasks.workunit.client.1.smithi096.stdout:1/198: dread d3/f21 [0,4194304] 0 2024-03-24T10:46:07.609 INFO:tasks.workunit.client.1.smithi096.stdout:8/275: getdents d8/d11/d22/d23/d1a/d46 0 2024-03-24T10:46:07.611 INFO:tasks.workunit.client.1.smithi096.stdout:7/238: getdents d0/d13/d32 0 2024-03-24T10:46:07.614 INFO:tasks.workunit.client.1.smithi096.stdout:0/185: rename d0/d2/l9 to d0/d11/d20/d2b/l39 0 2024-03-24T10:46:07.617 INFO:tasks.workunit.client.1.smithi096.stdout:6/251: truncate f6 526354 0 2024-03-24T10:46:07.617 INFO:tasks.workunit.client.1.smithi096.stdout:6/252: chown d8/d12/d32/d38/f4b 98 1 2024-03-24T10:46:07.619 INFO:tasks.workunit.client.1.smithi096.stdout:9/171: creat d9/db/d23/f35 x:0 0 0 2024-03-24T10:46:07.619 INFO:tasks.workunit.client.1.smithi096.stdout:9/172: truncate d9/db/d23/f35 68338 0 2024-03-24T10:46:07.620 INFO:tasks.workunit.client.1.smithi096.stdout:1/199: symlink d3/d24/l48 0 2024-03-24T10:46:07.620 INFO:tasks.workunit.client.1.smithi096.stdout:1/200: write d3/d24/f3e [45453,97237] 0 2024-03-24T10:46:07.622 INFO:tasks.workunit.client.1.smithi096.stdout:4/176: getdents d1 0 2024-03-24T10:46:07.622 INFO:tasks.workunit.client.1.smithi096.stdout:1/201: dread d3/f22 [0,4194304] 0 2024-03-24T10:46:07.623 INFO:tasks.workunit.client.1.smithi096.stdout:7/239: write d0/db/d26/f44 [3515860,42294] 0 2024-03-24T10:46:07.625 INFO:tasks.workunit.client.1.smithi096.stdout:2/155: mknod d5/c43 0 2024-03-24T10:46:07.625 INFO:tasks.workunit.client.1.smithi096.stdout:2/156: write d5/de/d2a/f2b [778564,101988] 0 2024-03-24T10:46:07.628 INFO:tasks.workunit.client.1.smithi096.stdout:9/173: creat d9/d15/d26/d29/f36 x:0 0 0 2024-03-24T10:46:07.628 INFO:tasks.workunit.client.1.smithi096.stdout:8/276: dwrite d8/d11/d22/d23/d1a/d46/d2e/f54 [0,4194304] 0 2024-03-24T10:46:07.632 INFO:tasks.workunit.client.1.smithi096.stdout:6/253: truncate f6 800619 0 2024-03-24T10:46:07.632 INFO:tasks.workunit.client.1.smithi096.stdout:6/254: fdatasync d8/d10/d19/f34 0 2024-03-24T10:46:07.632 INFO:tasks.workunit.client.1.smithi096.stdout:6/255: getdents d8/d10/d22 0 2024-03-24T10:46:07.632 INFO:tasks.workunit.client.1.smithi096.stdout:6/256: chown d8/f31 2914 1 2024-03-24T10:46:07.632 INFO:tasks.workunit.client.1.smithi096.stdout:9/174: dread d9/f13 [0,4194304] 0 2024-03-24T10:46:07.632 INFO:tasks.workunit.client.1.smithi096.stdout:9/175: write d9/db/d23/d31/f34 [461412,75516] 0 2024-03-24T10:46:07.636 INFO:tasks.workunit.client.1.smithi096.stdout:1/202: symlink d3/d23/d2f/l49 0 2024-03-24T10:46:07.640 INFO:tasks.workunit.client.1.smithi096.stdout:7/240: creat d0/db/d26/f55 x:0 0 0 2024-03-24T10:46:07.640 INFO:tasks.workunit.client.1.smithi096.stdout:0/186: rename d0/d11/d24/f38 to d0/d11/d16/d21/d25/f3a 0 2024-03-24T10:46:07.640 INFO:tasks.workunit.client.1.smithi096.stdout:0/187: fsync d0/d11/d24/d2c/f32 0 2024-03-24T10:46:07.640 INFO:tasks.workunit.client.1.smithi096.stdout:0/188: write d0/d11/d16/d21/f33 [326965,130815] 0 2024-03-24T10:46:07.645 INFO:tasks.workunit.client.1.smithi096.stdout:2/157: unlink d5/c43 0 2024-03-24T10:46:07.645 INFO:tasks.workunit.client.1.smithi096.stdout:8/277: dwrite d8/d11/f43 [0,4194304] 0 2024-03-24T10:46:07.649 INFO:tasks.workunit.client.1.smithi096.stdout:9/176: mknod d9/c37 0 2024-03-24T10:46:07.650 INFO:tasks.workunit.client.1.smithi096.stdout:9/177: write d9/db/f14 [298727,74510] 0 2024-03-24T10:46:07.650 INFO:tasks.workunit.client.1.smithi096.stdout:9/178: write d9/f2e [403636,53425] 0 2024-03-24T10:46:07.650 INFO:tasks.workunit.client.1.smithi096.stdout:9/179: chown d9/f18 56117771 1 2024-03-24T10:46:07.650 INFO:tasks.workunit.client.1.smithi096.stdout:9/180: chown d9/db/f28 13 1 2024-03-24T10:46:07.650 INFO:tasks.workunit.client.1.smithi096.stdout:4/177: mknod d1/d11/c37 0 2024-03-24T10:46:07.651 INFO:tasks.workunit.client.1.smithi096.stdout:1/203: creat d3/d23/d2f/f4a x:0 0 0 2024-03-24T10:46:07.653 INFO:tasks.workunit.client.1.smithi096.stdout:7/241: dwrite d0/db/d3a/f4d [0,4194304] 0 2024-03-24T10:46:07.654 INFO:tasks.workunit.client.1.smithi096.stdout:0/189: symlink d0/d11/d16/l3b 0 2024-03-24T10:46:07.655 INFO:tasks.workunit.client.1.smithi096.stdout:6/257: rename d8/lf to d8/d12/d32/d38/l4c 0 2024-03-24T10:46:07.666 INFO:tasks.workunit.client.1.smithi096.stdout:8/278: mknod d8/d11/d22/d23/d1e/c59 0 2024-03-24T10:46:07.666 INFO:tasks.workunit.client.1.smithi096.stdout:9/181: mknod d9/d15/d26/c38 0 2024-03-24T10:46:07.667 INFO:tasks.workunit.client.1.smithi096.stdout:4/178: chown d1/d11/f12 1292101941 1 2024-03-24T10:46:07.672 INFO:tasks.workunit.client.1.smithi096.stdout:9/182: dread d9/d15/f1e [0,4194304] 0 2024-03-24T10:46:07.672 INFO:tasks.workunit.client.1.smithi096.stdout:9/183: dread - d9/d15/d26/d29/f36 zero size 2024-03-24T10:46:07.677 INFO:tasks.workunit.client.1.smithi096.stdout:7/242: link d0/d13/c4e d0/db/d26/d48/c56 0 2024-03-24T10:46:07.679 INFO:tasks.workunit.client.1.smithi096.stdout:0/190: getdents d0/d11/d16/d21 0 2024-03-24T10:46:07.681 INFO:tasks.workunit.client.1.smithi096.stdout:2/158: rename d5/dc/d12/l3b to d5/d38/l44 0 2024-03-24T10:46:07.682 INFO:tasks.workunit.client.1.smithi096.stdout:4/179: rename d1/d2b to d1/d2b/d38 22 2024-03-24T10:46:07.683 INFO:tasks.workunit.client.1.smithi096.stdout:6/258: mknod d8/d10/d13/c4d 0 2024-03-24T10:46:07.683 INFO:tasks.workunit.client.1.smithi096.stdout:6/259: fdatasync d8/d10/f3e 0 2024-03-24T10:46:07.683 INFO:tasks.workunit.client.1.smithi096.stdout:6/260: creat d8/d12/d28/f4e x:0 0 0 2024-03-24T10:46:07.684 INFO:tasks.workunit.client.1.smithi096.stdout:8/279: mknod d8/d11/d44/d45/c5a 0 2024-03-24T10:46:07.684 INFO:tasks.workunit.client.1.smithi096.stdout:8/280: readlink d8/d11/d22/d23/d37/l50 0 2024-03-24T10:46:07.684 INFO:tasks.workunit.client.1.smithi096.stdout:8/281: truncate d8/d11/d22/d23/d1d/d3d/f56 554627 0 2024-03-24T10:46:07.684 INFO:tasks.workunit.client.1.smithi096.stdout:8/282: chown d8/f14 13141 1 2024-03-24T10:46:07.685 INFO:tasks.workunit.client.1.smithi096.stdout:1/204: getdents d3/d24 0 2024-03-24T10:46:07.686 INFO:tasks.workunit.client.1.smithi096.stdout:6/261: read d8/f1c [977487,52238] 0 2024-03-24T10:46:07.686 INFO:tasks.workunit.client.1.smithi096.stdout:6/262: truncate d8/f9 4656358 0 2024-03-24T10:46:07.687 INFO:tasks.workunit.client.1.smithi096.stdout:7/243: mknod d0/d21/c57 0 2024-03-24T10:46:07.687 INFO:tasks.workunit.client.1.smithi096.stdout:7/244: truncate d0/f20 1556573 0 2024-03-24T10:46:07.688 INFO:tasks.workunit.client.1.smithi096.stdout:0/191: mknod d0/d11/c3c 0 2024-03-24T10:46:07.689 INFO:tasks.workunit.client.1.smithi096.stdout:4/180: creat d1/d11/f39 x:0 0 0 2024-03-24T10:46:07.690 INFO:tasks.workunit.client.1.smithi096.stdout:2/159: creat d5/d38/f45 x:0 0 0 2024-03-24T10:46:07.692 INFO:tasks.workunit.client.1.smithi096.stdout:9/184: dwrite d9/d15/d26/f33 [0,4194304] 0 2024-03-24T10:46:07.692 INFO:tasks.workunit.client.1.smithi096.stdout:9/185: readlink d9/d15/l12 0 2024-03-24T10:46:07.692 INFO:tasks.workunit.client.1.smithi096.stdout:9/186: fdatasync d9/d15/f20 0 2024-03-24T10:46:07.692 INFO:tasks.workunit.client.1.smithi096.stdout:9/187: fsync d9/db/f14 0 2024-03-24T10:46:07.700 INFO:tasks.workunit.client.1.smithi096.stdout:1/205: rename d3/la to d3/d23/l4b 0 2024-03-24T10:46:07.701 INFO:tasks.workunit.client.1.smithi096.stdout:7/245: creat d0/db/d4f/f58 x:0 0 0 2024-03-24T10:46:07.705 INFO:tasks.workunit.client.1.smithi096.stdout:0/192: unlink d0/d11/d16/d21/d25/f3a 0 2024-03-24T10:46:07.705 INFO:tasks.workunit.client.1.smithi096.stdout:0/193: creat d0/d11/d20/d2b/f3d x:0 0 0 2024-03-24T10:46:07.706 INFO:tasks.workunit.client.1.smithi096.stdout:8/283: read d8/d11/d22/d23/d1d/f2b [762557,94396] 0 2024-03-24T10:46:07.706 INFO:tasks.workunit.client.1.smithi096.stdout:8/284: write f1 [2676023,35208] 0 2024-03-24T10:46:07.709 INFO:tasks.workunit.client.1.smithi096.stdout:4/181: mknod d1/d2b/c3a 0 2024-03-24T10:46:07.709 INFO:tasks.workunit.client.1.smithi096.stdout:4/182: write d1/d2b/f2f [652753,94496] 0 2024-03-24T10:46:07.710 INFO:tasks.workunit.client.1.smithi096.stdout:2/160: mknod d5/d38/c46 0 2024-03-24T10:46:07.710 INFO:tasks.workunit.client.1.smithi096.stdout:2/161: creat d5/dc/f47 x:0 0 0 2024-03-24T10:46:07.710 INFO:tasks.workunit.client.1.smithi096.stdout:4/183: read d1/f24 [4712871,111304] 0 2024-03-24T10:46:07.716 INFO:tasks.workunit.client.1.smithi096.stdout:9/188: link d9/d15/d26/c2d d9/c39 0 2024-03-24T10:46:07.716 INFO:tasks.workunit.client.1.smithi096.stdout:3/261: sync 2024-03-24T10:46:07.716 INFO:tasks.workunit.client.1.smithi096.stdout:3/262: readlink d0/d9/l20 0 2024-03-24T10:46:07.717 INFO:tasks.workunit.client.1.smithi096.stdout:9/189: dread d9/db/f28 [0,4194304] 0 2024-03-24T10:46:07.726 INFO:tasks.workunit.client.1.smithi096.stdout:0/194: link d0/cc d0/d11/d24/d2c/c3e 0 2024-03-24T10:46:07.732 INFO:tasks.workunit.client.1.smithi096.stdout:4/184: mknod d1/c3b 0 2024-03-24T10:46:07.732 INFO:tasks.workunit.client.1.smithi096.stdout:4/185: write d1/d11/f19 [1442457,40964] 0 2024-03-24T10:46:07.738 INFO:tasks.workunit.client.1.smithi096.stdout:3/263: mkdir d0/d13/d45/d52/d5d 0 2024-03-24T10:46:07.738 INFO:tasks.workunit.client.1.smithi096.stdout:3/264: creat d0/d9/f5e x:0 0 0 2024-03-24T10:46:07.740 INFO:tasks.workunit.client.1.smithi096.stdout:1/206: getdents d3/d23/d2f 0 2024-03-24T10:46:07.744 INFO:tasks.workunit.client.1.smithi096.stdout:7/246: getdents d0/db/d1c 0 2024-03-24T10:46:07.745 INFO:tasks.workunit.client.1.smithi096.stdout:4/186: dwrite d1/f35 [0,4194304] 0 2024-03-24T10:46:07.746 INFO:tasks.workunit.client.1.smithi096.stdout:5/168: sync 2024-03-24T10:46:07.748 INFO:tasks.workunit.client.1.smithi096.stdout:0/195: mknod d0/d11/c3f 0 2024-03-24T10:46:07.756 INFO:tasks.workunit.client.1.smithi096.stdout:5/169: dread d2/f3 [0,4194304] 0 2024-03-24T10:46:07.756 INFO:tasks.workunit.client.1.smithi096.stdout:3/265: creat d0/d9/d34/d39/d48/f5f x:0 0 0 2024-03-24T10:46:07.756 INFO:tasks.workunit.client.1.smithi096.stdout:3/266: rename d0/d13/d45 to d0/d13/d45/d52/d60 22 2024-03-24T10:46:07.756 INFO:tasks.workunit.client.1.smithi096.stdout:3/267: truncate d0/d9/f49 2004471 0 2024-03-24T10:46:07.756 INFO:tasks.workunit.client.1.smithi096.stdout:3/268: creat d0/f61 x:0 0 0 2024-03-24T10:46:07.756 INFO:tasks.workunit.client.1.smithi096.stdout:3/269: stat d0/d13/d45/d52/f53 0 2024-03-24T10:46:07.756 INFO:tasks.workunit.client.1.smithi096.stdout:3/270: truncate d0/f44 1506553 0 2024-03-24T10:46:07.756 INFO:tasks.workunit.client.1.smithi096.stdout:3/271: fsync d0/d9/f36 0 2024-03-24T10:46:07.757 INFO:tasks.workunit.client.1.smithi096.stdout:3/272: write d0/f4 [1276628,87372] 0 2024-03-24T10:46:07.759 INFO:tasks.workunit.client.1.smithi096.stdout:1/207: creat d3/d24/f4c x:0 0 0 2024-03-24T10:46:07.759 INFO:tasks.workunit.client.1.smithi096.stdout:1/208: dread - d3/d24/f35 zero size 2024-03-24T10:46:07.759 INFO:tasks.workunit.client.1.smithi096.stdout:1/209: chown d3/d23/d2f/c41 15846 1 2024-03-24T10:46:07.763 INFO:tasks.workunit.client.1.smithi096.stdout:9/190: rmdir d9/d15/d26/d29 39 2024-03-24T10:46:07.771 INFO:tasks.workunit.client.1.smithi096.stdout:4/187: dwrite d1/f24 [0,4194304] 0 2024-03-24T10:46:07.772 INFO:tasks.workunit.client.1.smithi096.stdout:3/273: dwrite d0/d9/f32 [0,4194304] 0 2024-03-24T10:46:07.775 INFO:tasks.workunit.client.1.smithi096.stdout:0/196: mkdir d0/d11/d40 0 2024-03-24T10:46:07.775 INFO:tasks.workunit.client.1.smithi096.stdout:0/197: fsync d0/d11/d20/f1f 0 2024-03-24T10:46:07.775 INFO:tasks.workunit.client.1.smithi096.stdout:0/198: readlink d0/d11/d16/l3b 0 2024-03-24T10:46:07.775 INFO:tasks.workunit.client.1.smithi096.stdout:0/199: fdatasync d0/f22 0 2024-03-24T10:46:07.775 INFO:tasks.workunit.client.1.smithi096.stdout:0/200: dread - d0/d11/d16/d21/d25/f36 zero size 2024-03-24T10:46:07.779 INFO:tasks.workunit.client.1.smithi096.stdout:5/170: mkdir d2/d29/d32/d33/d3c 0 2024-03-24T10:46:07.784 INFO:tasks.workunit.client.1.smithi096.stdout:1/210: creat d3/d36/f4d x:0 0 0 2024-03-24T10:46:07.784 INFO:tasks.workunit.client.1.smithi096.stdout:1/211: rename d3 to d3/d4e 22 2024-03-24T10:46:07.784 INFO:tasks.workunit.client.1.smithi096.stdout:1/212: stat d3/d24/f4c 0 2024-03-24T10:46:07.784 INFO:tasks.workunit.client.1.smithi096.stdout:1/213: write d3/d23/f45 [782871,125474] 0 2024-03-24T10:46:07.786 INFO:tasks.workunit.client.1.smithi096.stdout:9/191: link d9/d15/d26/d29/f36 d9/f3a 0 2024-03-24T10:46:07.789 INFO:tasks.workunit.client.1.smithi096.stdout:3/274: dwrite d0/f5c [0,4194304] 0 2024-03-24T10:46:07.789 INFO:tasks.workunit.client.1.smithi096.stdout:7/247: rmdir d0/d13 39 2024-03-24T10:46:07.795 INFO:tasks.workunit.client.1.smithi096.stdout:4/188: mknod d1/c3c 0 2024-03-24T10:46:07.798 INFO:tasks.workunit.client.1.smithi096.stdout:0/201: creat d0/d11/d40/f41 x:0 0 0 2024-03-24T10:46:07.798 INFO:tasks.workunit.client.1.smithi096.stdout:0/202: stat d0/d11/d20 0 2024-03-24T10:46:07.799 INFO:tasks.workunit.client.1.smithi096.stdout:5/171: truncate d2/d7/d8/df/d14/f23 2524510 0 2024-03-24T10:46:07.800 INFO:tasks.workunit.client.1.smithi096.stdout:1/214: mknod d3/c4f 0 2024-03-24T10:46:07.804 INFO:tasks.workunit.client.1.smithi096.stdout:9/192: symlink d9/db/d23/l3b 0 2024-03-24T10:46:07.813 INFO:tasks.workunit.client.1.smithi096.stdout:3/275: rmdir d0/d13/d45/d52/d5d 0 2024-03-24T10:46:07.816 INFO:tasks.workunit.client.1.smithi096.stdout:7/248: unlink d0/d13/f2c 0 2024-03-24T10:46:07.821 INFO:tasks.workunit.client.1.smithi096.stdout:9/193: dwrite d9/d15/d26/d29/f36 [0,4194304] 0 2024-03-24T10:46:07.821 INFO:tasks.workunit.client.1.smithi096.stdout:9/194: chown d9/f2e 2 1 2024-03-24T10:46:07.821 INFO:tasks.workunit.client.1.smithi096.stdout:9/195: write d9/f18 [1365511,83482] 0 2024-03-24T10:46:07.832 INFO:tasks.workunit.client.1.smithi096.stdout:7/249: dwrite d0/f14 [0,4194304] 0 2024-03-24T10:46:07.853 INFO:tasks.workunit.client.1.smithi096.stdout:0/203: mkdir d0/d11/d42 0 2024-03-24T10:46:07.854 INFO:tasks.workunit.client.1.smithi096.stdout:0/204: readlink d0/d2/l29 0 2024-03-24T10:46:07.854 INFO:tasks.workunit.client.1.smithi096.stdout:0/205: readlink d0/d11/l35 0 2024-03-24T10:46:07.854 INFO:tasks.workunit.client.1.smithi096.stdout:0/206: read d0/d2/fb [464567,88780] 0 2024-03-24T10:46:07.855 INFO:tasks.workunit.client.1.smithi096.stdout:7/250: dwrite d0/f38 [0,4194304] 0 2024-03-24T10:46:07.855 INFO:tasks.workunit.client.1.smithi096.stdout:5/172: mkdir d2/d29/d32/d33/d3c/d3d 0 2024-03-24T10:46:07.856 INFO:tasks.workunit.client.1.smithi096.stdout:5/173: readlink d2/d7/l2f 0 2024-03-24T10:46:07.856 INFO:tasks.workunit.client.1.smithi096.stdout:5/174: creat d2/d7/f3e x:0 0 0 2024-03-24T10:46:07.862 INFO:tasks.workunit.client.1.smithi096.stdout:0/207: write d0/d11/f1c [2936665,68786] 0 2024-03-24T10:46:07.862 INFO:tasks.workunit.client.1.smithi096.stdout:0/208: truncate d0/d11/d24/f30 724388 0 2024-03-24T10:46:07.863 INFO:tasks.workunit.client.1.smithi096.stdout:7/251: dread d0/f14 [0,4194304] 0 2024-03-24T10:46:07.863 INFO:tasks.workunit.client.1.smithi096.stdout:3/276: symlink d0/d13/d41/l62 0 2024-03-24T10:46:07.864 INFO:tasks.workunit.client.1.smithi096.stdout:4/189: getdents d1/d2b 0 2024-03-24T10:46:07.877 INFO:tasks.workunit.client.1.smithi096.stdout:5/175: mkdir d2/d29/d32/d3f 0 2024-03-24T10:46:07.877 INFO:tasks.workunit.client.1.smithi096.stdout:5/176: dread - d2/d29/f38 zero size 2024-03-24T10:46:07.878 INFO:tasks.workunit.client.1.smithi096.stdout:0/209: mkdir d0/d11/d43 0 2024-03-24T10:46:07.878 INFO:tasks.workunit.client.1.smithi096.stdout:0/210: chown d0/ca 0 1 2024-03-24T10:46:07.879 INFO:tasks.workunit.client.1.smithi096.stdout:7/252: mkdir d0/d21/d59 0 2024-03-24T10:46:07.880 INFO:tasks.workunit.client.1.smithi096.stdout:3/277: link d0/d9/f5e d0/d9/d34/f63 0 2024-03-24T10:46:07.880 INFO:tasks.workunit.client.1.smithi096.stdout:3/278: write d0/d9/d34/f5a [574294,76259] 0 2024-03-24T10:46:07.880 INFO:tasks.workunit.client.1.smithi096.stdout:3/279: chown d0/d13/d41 3446 1 2024-03-24T10:46:07.882 INFO:tasks.workunit.client.1.smithi096.stdout:4/190: rename d1/c3c to d1/d32/c3d 0 2024-03-24T10:46:07.882 INFO:tasks.workunit.client.1.smithi096.stdout:4/191: readlink d1/d11/l31 0 2024-03-24T10:46:07.884 INFO:tasks.workunit.client.1.smithi096.stdout:9/196: getdents d9 0 2024-03-24T10:46:07.884 INFO:tasks.workunit.client.1.smithi096.stdout:9/197: chown d9/db/f22 2 1 2024-03-24T10:46:07.884 INFO:tasks.workunit.client.1.smithi096.stdout:9/198: write d9/f2f [245740,103922] 0 2024-03-24T10:46:07.884 INFO:tasks.workunit.client.1.smithi096.stdout:9/199: chown d9/d15/f1e 1 1 2024-03-24T10:46:07.884 INFO:tasks.workunit.client.1.smithi096.stdout:9/200: write d9/d15/f20 [181782,47291] 0 2024-03-24T10:46:07.884 INFO:tasks.workunit.client.1.smithi096.stdout:9/201: creat d9/d15/f3c x:0 0 0 2024-03-24T10:46:07.885 INFO:tasks.workunit.client.1.smithi096.stdout:5/177: dread d2/d7/f10 [0,4194304] 0 2024-03-24T10:46:07.885 INFO:tasks.workunit.client.1.smithi096.stdout:5/178: stat d2/d7/l2b 0 2024-03-24T10:46:07.885 INFO:tasks.workunit.client.1.smithi096.stdout:0/211: truncate d0/fe 221494 0 2024-03-24T10:46:07.886 INFO:tasks.workunit.client.1.smithi096.stdout:7/253: creat d0/f5a x:0 0 0 2024-03-24T10:46:07.892 INFO:tasks.workunit.client.1.smithi096.stdout:3/280: rename d0/d9/d34/f5a to d0/d13/d45/d47/f64 0 2024-03-24T10:46:07.893 INFO:tasks.workunit.client.1.smithi096.stdout:4/192: creat d1/f3e x:0 0 0 2024-03-24T10:46:07.893 INFO:tasks.workunit.client.1.smithi096.stdout:4/193: readlink d1/l20 0 2024-03-24T10:46:07.910 INFO:tasks.workunit.client.1.smithi096.stdout:5/179: dwrite d2/d7/d8/df/d14/f23 [0,4194304] 0 2024-03-24T10:46:07.910 INFO:tasks.workunit.client.1.smithi096.stdout:5/180: readlink d2/d7/d8/lb 0 2024-03-24T10:46:07.910 INFO:tasks.workunit.client.1.smithi096.stdout:0/212: dwrite d0/f22 [0,4194304] 0 2024-03-24T10:46:07.910 INFO:tasks.workunit.client.1.smithi096.stdout:0/213: creat d0/d11/d40/f44 x:0 0 0 2024-03-24T10:46:07.985 INFO:tasks.workunit.client.1.smithi096.stdout:9/202: mkdir d9/db/d23/d31/d3d 0 2024-03-24T10:46:07.987 INFO:tasks.workunit.client.1.smithi096.stdout:3/281: unlink d0/d9/d34/d39/f43 0 2024-03-24T10:46:07.996 INFO:tasks.workunit.client.1.smithi096.stdout:4/194: rename d1/d11/l2a to d1/d2b/l3f 0 2024-03-24T10:46:07.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:07 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:46:07.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:07 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:46:07.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:07 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:46:07.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:07 smithi094 ceph-mon[29446]: pgmap v247: 65 pgs: 65 active+clean; 2.2 GiB data, 7.9 GiB used, 528 GiB / 536 GiB avail; 255 MiB/s rd, 304 MiB/s wr, 415 op/s 2024-03-24T10:46:08.016 INFO:tasks.workunit.client.1.smithi096.stdout:5/181: creat d2/d29/d32/f40 x:0 0 0 2024-03-24T10:46:08.016 INFO:tasks.workunit.client.1.smithi096.stdout:5/182: write d2/d29/f38 [541723,53515] 0 2024-03-24T10:46:08.016 INFO:tasks.workunit.client.1.smithi096.stdout:5/183: stat d2 0 2024-03-24T10:46:08.016 INFO:tasks.workunit.client.1.smithi096.stdout:4/195: dwrite d1/f24 [0,4194304] 0 2024-03-24T10:46:08.018 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:07 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:46:08.018 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:07 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:46:08.018 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:07 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:46:08.018 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:07 smithi096 ceph-mon[38097]: pgmap v247: 65 pgs: 65 active+clean; 2.2 GiB data, 7.9 GiB used, 528 GiB / 536 GiB avail; 255 MiB/s rd, 304 MiB/s wr, 415 op/s 2024-03-24T10:46:08.021 INFO:tasks.workunit.client.1.smithi096.stdout:9/203: rename d9/db/d23/d31/d3d to d9/db/d23/d31/d3e 0 2024-03-24T10:46:08.021 INFO:tasks.workunit.client.1.smithi096.stdout:9/204: read d9/f13 [1415234,95936] 0 2024-03-24T10:46:08.022 INFO:tasks.workunit.client.1.smithi096.stdout:3/282: symlink d0/d13/d41/l65 0 2024-03-24T10:46:08.026 INFO:tasks.workunit.client.1.smithi096.stdout:0/214: rename d0/f22 to d0/d11/d24/f45 0 2024-03-24T10:46:08.026 INFO:tasks.workunit.client.1.smithi096.stdout:0/215: chown d0/d11/d16/f2f 584 1 2024-03-24T10:46:08.027 INFO:tasks.workunit.client.1.smithi096.stdout:0/216: write d0/d11/d20/f2e [718492,20106] 0 2024-03-24T10:46:08.027 INFO:tasks.workunit.client.1.smithi096.stdout:0/217: write d0/fe [526270,122654] 0 2024-03-24T10:46:08.027 INFO:tasks.workunit.client.1.smithi096.stdout:0/218: stat d0/d11/d20/d2b/f3d 0 2024-03-24T10:46:08.029 INFO:tasks.workunit.client.1.smithi096.stdout:3/283: write d0/d13/d1e/f2a [2506131,27852] 0 2024-03-24T10:46:08.029 INFO:tasks.workunit.client.1.smithi096.stdout:3/284: dread - d0/d9/d34/d39/d48/f5f zero size 2024-03-24T10:46:08.029 INFO:tasks.workunit.client.1.smithi096.stdout:3/285: truncate d0/d13/d1e/f2a 5558514 0 2024-03-24T10:46:08.029 INFO:tasks.workunit.client.1.smithi096.stdout:4/196: mkdir d1/d2b/d40 0 2024-03-24T10:46:08.032 INFO:tasks.workunit.client.1.smithi096.stdout:9/205: creat d9/db/d23/d31/d3e/f3f x:0 0 0 2024-03-24T10:46:08.036 INFO:tasks.workunit.client.1.smithi096.stdout:5/184: truncate d2/d7/d8/df/d14/f23 339369 0 2024-03-24T10:46:08.036 INFO:tasks.workunit.client.1.smithi096.stdout:5/185: write d2/d29/f38 [1540087,37172] 0 2024-03-24T10:46:08.036 INFO:tasks.workunit.client.1.smithi096.stdout:3/286: dread d0/f5c [0,4194304] 0 2024-03-24T10:46:08.036 INFO:tasks.workunit.client.1.smithi096.stdout:3/287: creat d0/d13/d45/d52/f66 x:0 0 0 2024-03-24T10:46:08.036 INFO:tasks.workunit.client.1.smithi096.stdout:3/288: creat d0/d9/d34/f67 x:0 0 0 2024-03-24T10:46:08.037 INFO:tasks.workunit.client.1.smithi096.stdout:5/186: read d2/d29/f38 [311922,91070] 0 2024-03-24T10:46:08.037 INFO:tasks.workunit.client.1.smithi096.stdout:5/187: stat d2/d29/f38 0 2024-03-24T10:46:08.038 INFO:tasks.workunit.client.1.smithi096.stdout:4/197: creat d1/d11/f41 x:0 0 0 2024-03-24T10:46:08.042 INFO:tasks.workunit.client.1.smithi096.stdout:9/206: symlink d9/db/d23/d31/d3e/l40 0 2024-03-24T10:46:08.047 INFO:tasks.workunit.client.1.smithi096.stdout:0/219: rmdir d0/d11/d40 39 2024-03-24T10:46:08.053 INFO:tasks.workunit.client.1.smithi096.stdout:4/198: mkdir d1/d32/d42 0 2024-03-24T10:46:08.053 INFO:tasks.workunit.client.1.smithi096.stdout:4/199: stat d1/d32/l33 0 2024-03-24T10:46:08.054 INFO:tasks.workunit.client.1.smithi096.stdout:5/188: dwrite d2/d29/d32/f40 [0,4194304] 0 2024-03-24T10:46:08.054 INFO:tasks.workunit.client.1.smithi096.stdout:5/189: truncate d2/d7/d8/df/d14/f23 989556 0 2024-03-24T10:46:08.057 INFO:tasks.workunit.client.1.smithi096.stdout:9/207: symlink d9/d15/d26/l41 0 2024-03-24T10:46:08.060 INFO:tasks.workunit.client.1.smithi096.stdout:0/220: dread d0/d11/d24/f45 [0,4194304] 0 2024-03-24T10:46:08.060 INFO:tasks.workunit.client.1.smithi096.stdout:0/221: stat d0/fe 0 2024-03-24T10:46:08.060 INFO:tasks.workunit.client.1.smithi096.stdout:4/200: mknod d1/d2b/c43 0 2024-03-24T10:46:08.070 INFO:tasks.workunit.client.1.smithi096.stdout:5/190: dwrite d2/d7/d8/f12 [0,4194304] 0 2024-03-24T10:46:08.133 INFO:tasks.workunit.client.1.smithi096.stdout:9/208: creat d9/d15/d26/d29/f42 x:0 0 0 2024-03-24T10:46:08.133 INFO:tasks.workunit.client.1.smithi096.stdout:9/209: write d9/d15/d26/f33 [4619907,75828] 0 2024-03-24T10:46:08.135 INFO:tasks.workunit.client.1.smithi096.stdout:0/222: creat d0/d11/d16/d21/d25/f46 x:0 0 0 2024-03-24T10:46:08.137 INFO:tasks.workunit.client.1.smithi096.stdout:4/201: mkdir d1/d32/d42/d44 0 2024-03-24T10:46:08.139 INFO:tasks.workunit.client.1.smithi096.stdout:5/191: link d2/f5 d2/d7/f41 0 2024-03-24T10:46:08.139 INFO:tasks.workunit.client.1.smithi096.stdout:5/192: dread - d2/f39 zero size 2024-03-24T10:46:08.146 INFO:tasks.workunit.client.1.smithi096.stdout:0/223: mkdir d0/d11/d16/d21/d25/d47 0 2024-03-24T10:46:08.147 INFO:tasks.workunit.client.1.smithi096.stdout:4/202: mkdir d1/d32/d42/d44/d45 0 2024-03-24T10:46:08.150 INFO:tasks.workunit.client.1.smithi096.stdout:0/224: link d0/d11/d24/f2d d0/f48 0 2024-03-24T10:46:08.152 INFO:tasks.workunit.client.1.smithi096.stdout:4/203: creat d1/d2b/d40/f46 x:0 0 0 2024-03-24T10:46:08.155 INFO:tasks.workunit.client.1.smithi096.stdout:5/193: dwrite d2/d7/f2a [0,4194304] 0 2024-03-24T10:46:08.159 INFO:tasks.workunit.client.1.smithi096.stdout:4/204: link d1/cc d1/d32/c47 0 2024-03-24T10:46:08.168 INFO:tasks.workunit.client.1.smithi096.stdout:5/194: mknod d2/d29/d32/c42 0 2024-03-24T10:46:08.168 INFO:tasks.workunit.client.1.smithi096.stdout:6/263: sync 2024-03-24T10:46:08.170 INFO:tasks.workunit.client.1.smithi096.stdout:0/225: dwrite d0/d11/d20/d2b/f37 [0,4194304] 0 2024-03-24T10:46:08.171 INFO:tasks.workunit.client.1.smithi096.stdout:5/195: creat d2/f43 x:0 0 0 2024-03-24T10:46:08.173 INFO:tasks.workunit.client.1.smithi096.stdout:0/226: dread d0/d11/d20/f2e [0,4194304] 0 2024-03-24T10:46:08.173 INFO:tasks.workunit.client.1.smithi096.stdout:0/227: truncate d0/d11/d20/d2b/f3d 749893 0 2024-03-24T10:46:08.174 INFO:tasks.workunit.client.1.smithi096.stdout:6/264: rename d8/f14 to d8/d10/f4f 0 2024-03-24T10:46:08.181 INFO:tasks.workunit.client.1.smithi096.stdout:4/205: dwrite d1/ff [0,4194304] 0 2024-03-24T10:46:08.181 INFO:tasks.workunit.client.1.smithi096.stdout:4/206: fsync d1/d2b/d40/f46 0 2024-03-24T10:46:08.185 INFO:tasks.workunit.client.1.smithi096.stdout:0/228: mkdir d0/d11/d16/d21/d25/d47/d49 0 2024-03-24T10:46:08.190 INFO:tasks.workunit.client.1.smithi096.stdout:5/196: dwrite d2/d7/d8/df/f2d [0,4194304] 0 2024-03-24T10:46:08.194 INFO:tasks.workunit.client.1.smithi096.stdout:4/207: mknod d1/d32/c48 0 2024-03-24T10:46:08.194 INFO:tasks.workunit.client.1.smithi096.stdout:4/208: dread - d1/d11/f41 zero size 2024-03-24T10:46:08.198 INFO:tasks.workunit.client.1.smithi096.stdout:5/197: rename d2/d7/d8/df/f1b to d2/d29/d32/d33/f44 0 2024-03-24T10:46:08.206 INFO:tasks.workunit.client.1.smithi096.stdout:0/229: dwrite d0/d11/d16/d21/f2a [0,4194304] 0 2024-03-24T10:46:08.206 INFO:tasks.workunit.client.1.smithi096.stdout:0/230: write d0/d11/d24/f30 [1402873,38590] 0 2024-03-24T10:46:08.206 INFO:tasks.workunit.client.1.smithi096.stdout:0/231: truncate d0/d11/d20/d2b/f37 4677336 0 2024-03-24T10:46:08.206 INFO:tasks.workunit.client.1.smithi096.stdout:0/232: truncate d0/d11/d24/d2c/f32 439688 0 2024-03-24T10:46:08.210 INFO:tasks.workunit.client.1.smithi096.stdout:0/233: rename d0/d2/fb to d0/d2/f4a 0 2024-03-24T10:46:08.211 INFO:tasks.workunit.client.1.smithi096.stdout:5/198: dread d2/f3 [0,4194304] 0 2024-03-24T10:46:08.213 INFO:tasks.workunit.client.1.smithi096.stdout:4/209: dwrite d1/d11/f19 [0,4194304] 0 2024-03-24T10:46:08.214 INFO:tasks.workunit.client.1.smithi096.stdout:4/210: write d1/fd [1392526,7276] 0 2024-03-24T10:46:08.216 INFO:tasks.workunit.client.1.smithi096.stdout:8/285: sync 2024-03-24T10:46:08.216 INFO:tasks.workunit.client.1.smithi096.stdout:8/286: chown d8/d11/d22/d23/d37/l47 1 1 2024-03-24T10:46:08.216 INFO:tasks.workunit.client.1.smithi096.stdout:5/199: dread d2/d29/f38 [0,4194304] 0 2024-03-24T10:46:08.226 INFO:tasks.workunit.client.1.smithi096.stdout:8/287: creat d8/d11/d22/d23/f5b x:0 0 0 2024-03-24T10:46:08.228 INFO:tasks.workunit.client.1.smithi096.stdout:0/234: dwrite d0/d11/d16/d21/d25/f46 [0,4194304] 0 2024-03-24T10:46:08.229 INFO:tasks.workunit.client.1.smithi096.stdout:5/200: rename d2/d7/l2b to d2/d29/d32/d33/d3c/l45 0 2024-03-24T10:46:08.234 INFO:tasks.workunit.client.1.smithi096.stdout:4/211: unlink d1/d11/l16 0 2024-03-24T10:46:08.234 INFO:tasks.workunit.client.1.smithi096.stdout:4/212: fdatasync d1/f10 0 2024-03-24T10:46:08.234 INFO:tasks.workunit.client.1.smithi096.stdout:4/213: chown d1/d11 13741 1 2024-03-24T10:46:08.237 INFO:tasks.workunit.client.1.smithi096.stdout:0/235: mknod d0/d11/d42/c4b 0 2024-03-24T10:46:08.237 INFO:tasks.workunit.client.1.smithi096.stdout:0/236: stat d0/d11/d42 0 2024-03-24T10:46:08.237 INFO:tasks.workunit.client.1.smithi096.stdout:0/237: creat d0/d11/d24/f4c x:0 0 0 2024-03-24T10:46:08.326 INFO:tasks.workunit.client.1.smithi096.stdout:5/201: mknod d2/d29/d32/d3f/c46 0 2024-03-24T10:46:08.326 INFO:tasks.workunit.client.1.smithi096.stdout:5/202: read - d2/f39 zero size 2024-03-24T10:46:08.333 INFO:tasks.workunit.client.1.smithi096.stdout:8/288: getdents d8/d11/d22/d23/d1a 0 2024-03-24T10:46:08.334 INFO:tasks.workunit.client.1.smithi096.stdout:5/203: write d2/d7/f26 [1379238,6784] 0 2024-03-24T10:46:08.337 INFO:tasks.workunit.client.1.smithi096.stdout:0/238: link d0/d11/d24/f30 d0/d11/d16/d21/d25/f4d 0 2024-03-24T10:46:08.341 INFO:tasks.workunit.client.1.smithi096.stdout:8/289: truncate d8/d11/d22/d23/f1c 3458955 0 2024-03-24T10:46:08.344 INFO:tasks.workunit.client.1.smithi096.stdout:5/204: creat d2/d7/d8/d18/f47 x:0 0 0 2024-03-24T10:46:08.345 INFO:tasks.workunit.client.1.smithi096.stdout:0/239: truncate d0/d11/d16/d21/f2a 3351388 0 2024-03-24T10:46:08.345 INFO:tasks.workunit.client.1.smithi096.stdout:0/240: write d0/d11/d16/d21/d25/f36 [173797,13670] 0 2024-03-24T10:46:08.345 INFO:tasks.workunit.client.1.smithi096.stdout:0/241: write d0/d11/d24/f2d [181710,117084] 0 2024-03-24T10:46:08.346 INFO:tasks.workunit.client.1.smithi096.stdout:0/242: write d0/d11/d40/f41 [256159,84298] 0 2024-03-24T10:46:08.346 INFO:tasks.workunit.client.1.smithi096.stdout:0/243: write d0/d11/d16/d21/f33 [210247,77743] 0 2024-03-24T10:46:08.349 INFO:tasks.workunit.client.1.smithi096.stdout:8/290: creat d8/d11/d22/d23/d1d/f5c x:0 0 0 2024-03-24T10:46:08.351 INFO:tasks.workunit.client.1.smithi096.stdout:5/205: rename d2/d7/d8/df/d14/l15 to d2/d29/d32/d33/l48 0 2024-03-24T10:46:08.351 INFO:tasks.workunit.client.1.smithi096.stdout:5/206: chown d2/d29/d32/d33/d37 436 1 2024-03-24T10:46:08.361 INFO:tasks.workunit.client.1.smithi096.stdout:5/207: dread d2/d7/d8/df/f2d [0,4194304] 0 2024-03-24T10:46:08.362 INFO:tasks.workunit.client.1.smithi096.stdout:5/208: getdents d2/d29/d32/d33/d3c 0 2024-03-24T10:46:08.363 INFO:tasks.workunit.client.1.smithi096.stdout:5/209: truncate d2/f39 992670 0 2024-03-24T10:46:08.363 INFO:tasks.workunit.client.1.smithi096.stdout:5/210: write d2/f43 [748144,44211] 0 2024-03-24T10:46:08.365 INFO:tasks.workunit.client.1.smithi096.stdout:0/244: dwrite d0/d11/d24/f30 [0,4194304] 0 2024-03-24T10:46:08.367 INFO:tasks.workunit.client.1.smithi096.stdout:0/245: write d0/f5 [346289,5805] 0 2024-03-24T10:46:08.372 INFO:tasks.workunit.client.1.smithi096.stdout:0/246: stat d0/d11/d24/d2c/c3e 0 2024-03-24T10:46:08.375 INFO:tasks.workunit.client.1.smithi096.stdout:2/162: sync 2024-03-24T10:46:08.375 INFO:tasks.workunit.client.1.smithi096.stdout:2/163: read - d5/de/f3d zero size 2024-03-24T10:46:08.375 INFO:tasks.workunit.client.1.smithi096.stdout:2/164: creat d5/dc/d12/f48 x:0 0 0 2024-03-24T10:46:08.379 INFO:tasks.workunit.client.1.smithi096.stdout:0/247: creat d0/d11/d16/f4e x:0 0 0 2024-03-24T10:46:08.379 INFO:tasks.workunit.client.1.smithi096.stdout:8/291: dwrite d8/d11/d22/d23/f1c [0,4194304] 0 2024-03-24T10:46:08.379 INFO:tasks.workunit.client.1.smithi096.stdout:8/292: stat d8/d11/d22/d23/d1e/f33 0 2024-03-24T10:46:08.383 INFO:tasks.workunit.client.1.smithi096.stdout:5/211: dwrite d2/d7/d8/df/f19 [0,4194304] 0 2024-03-24T10:46:08.386 INFO:tasks.workunit.client.1.smithi096.stdout:2/165: dread f3 [0,4194304] 0 2024-03-24T10:46:08.387 INFO:tasks.workunit.client.1.smithi096.stdout:5/212: dread d2/d29/f38 [0,4194304] 0 2024-03-24T10:46:08.387 INFO:tasks.workunit.client.1.smithi096.stdout:5/213: chown d2/d7/c16 266414 1 2024-03-24T10:46:08.396 INFO:tasks.workunit.client.1.smithi096.stdout:8/293: rmdir d8/d11/d44 39 2024-03-24T10:46:08.397 INFO:tasks.workunit.client.1.smithi096.stdout:0/248: dwrite d0/d11/d24/f30 [0,4194304] 0 2024-03-24T10:46:08.399 INFO:tasks.workunit.client.1.smithi096.stdout:2/166: rename d5/dc/c24 to d5/dc/d12/d23/c49 0 2024-03-24T10:46:08.402 INFO:tasks.workunit.client.1.smithi096.stdout:0/249: symlink d0/l4f 0 2024-03-24T10:46:08.402 INFO:tasks.workunit.client.1.smithi096.stdout:0/250: fsync d0/d11/d24/f45 0 2024-03-24T10:46:08.408 INFO:tasks.workunit.client.1.smithi096.stdout:0/251: rename d0/fe to d0/d11/d24/f50 0 2024-03-24T10:46:08.411 INFO:tasks.workunit.client.1.smithi096.stdout:0/252: creat d0/d11/d43/f51 x:0 0 0 2024-03-24T10:46:08.414 INFO:tasks.workunit.client.1.smithi096.stdout:5/214: dwrite d2/d29/d32/d33/f44 [4194304,4194304] 0 2024-03-24T10:46:08.415 INFO:tasks.workunit.client.1.smithi096.stdout:8/294: dwrite d8/d11/d22/d23/d1a/d46/d2e/f54 [0,4194304] 0 2024-03-24T10:46:08.415 INFO:tasks.workunit.client.1.smithi096.stdout:8/295: fsync d8/d11/d22/d23/d1d/f2b 0 2024-03-24T10:46:08.418 INFO:tasks.workunit.client.1.smithi096.stdout:2/167: rmdir d5/dc/d12 39 2024-03-24T10:46:08.422 INFO:tasks.workunit.client.1.smithi096.stdout:5/215: mkdir d2/d49 0 2024-03-24T10:46:08.425 INFO:tasks.workunit.client.1.smithi096.stdout:2/168: creat d5/dc/d12/f4a x:0 0 0 2024-03-24T10:46:08.436 INFO:tasks.workunit.client.1.smithi096.stdout:8/296: truncate d8/d11/d22/d23/d1a/d46/d2e/f54 1332127 0 2024-03-24T10:46:08.439 INFO:tasks.workunit.client.1.smithi096.stdout:0/253: dwrite d0/f5 [0,4194304] 0 2024-03-24T10:46:08.440 INFO:tasks.workunit.client.1.smithi096.stdout:0/254: rmdir d0/d11 39 2024-03-24T10:46:08.461 INFO:tasks.workunit.client.1.smithi096.stdout:8/297: dwrite d8/d11/d44/d45/f4d [0,4194304] 0 2024-03-24T10:46:08.463 INFO:tasks.workunit.client.1.smithi096.stdout:8/298: unlink f1 0 2024-03-24T10:46:08.463 INFO:tasks.workunit.client.1.smithi096.stdout:8/299: truncate d8/d11/d22/d23/d52/f57 673353 0 2024-03-24T10:46:08.463 INFO:tasks.workunit.client.1.smithi096.stdout:8/300: readlink d8/la 0 2024-03-24T10:46:08.463 INFO:tasks.workunit.client.1.smithi096.stdout:8/301: dread - d8/d11/d22/d23/d1d/f5c zero size 2024-03-24T10:46:08.466 INFO:tasks.workunit.client.1.smithi096.stdout:2/169: dwrite d5/dc/f21 [0,4194304] 0 2024-03-24T10:46:08.479 INFO:tasks.workunit.client.1.smithi096.stdout:1/215: sync 2024-03-24T10:46:08.480 INFO:tasks.workunit.client.1.smithi096.stdout:2/170: rmdir d5 39 2024-03-24T10:46:08.481 INFO:tasks.workunit.client.1.smithi096.stdout:1/216: dread d3/f22 [0,4194304] 0 2024-03-24T10:46:08.481 INFO:tasks.workunit.client.1.smithi096.stdout:1/217: fdatasync d3/f22 0 2024-03-24T10:46:08.488 INFO:tasks.workunit.client.1.smithi096.stdout:2/171: write d5/f7 [3027837,93017] 0 2024-03-24T10:46:08.489 INFO:tasks.workunit.client.1.smithi096.stdout:2/172: symlink d5/dc/d12/l4b 0 2024-03-24T10:46:08.490 INFO:tasks.workunit.client.1.smithi096.stdout:2/173: creat d5/de/d2a/f4c x:0 0 0 2024-03-24T10:46:08.505 INFO:tasks.workunit.client.1.smithi096.stdout:2/174: dwrite d5/dc/d12/f20 [0,4194304] 0 2024-03-24T10:46:08.508 INFO:tasks.workunit.client.1.smithi096.stdout:2/175: rename d5/de/l18 to d5/dc/d12/l4d 0 2024-03-24T10:46:08.509 INFO:tasks.workunit.client.1.smithi096.stdout:2/176: write d5/dc/d12/f16 [272853,14586] 0 2024-03-24T10:46:08.511 INFO:tasks.workunit.client.1.smithi096.stdout:2/177: dread d5/de/f15 [0,4194304] 0 2024-03-24T10:46:08.511 INFO:tasks.workunit.client.1.smithi096.stdout:2/178: read - d5/d38/f45 zero size 2024-03-24T10:46:08.513 INFO:tasks.workunit.client.1.smithi096.stdout:2/179: rename d5/d38/l44 to d5/dc/d12/d23/l4e 0 2024-03-24T10:46:08.513 INFO:tasks.workunit.client.1.smithi096.stdout:2/180: mkdir d5/dc/d12/d23/d4f 0 2024-03-24T10:46:08.514 INFO:tasks.workunit.client.1.smithi096.stdout:2/181: mknod d5/dc/c50 0 2024-03-24T10:46:08.515 INFO:tasks.workunit.client.1.smithi096.stdout:2/182: creat d5/de/f51 x:0 0 0 2024-03-24T10:46:08.546 INFO:tasks.workunit.client.1.smithi096.stdout:1/218: dread d3/d23/d2f/f37 [0,4194304] 0 2024-03-24T10:46:08.546 INFO:tasks.workunit.client.1.smithi096.stdout:1/219: getdents d3/d23/d2f 0 2024-03-24T10:46:08.546 INFO:tasks.workunit.client.1.smithi096.stdout:1/220: chown d3/d23/d2f/c41 229 1 2024-03-24T10:46:08.549 INFO:tasks.workunit.client.1.smithi096.stdout:1/221: rename d3/f21 to d3/d36/f50 0 2024-03-24T10:46:08.549 INFO:tasks.workunit.client.1.smithi096.stdout:1/222: write d3/d23/f38 [985538,23683] 0 2024-03-24T10:46:08.550 INFO:tasks.workunit.client.1.smithi096.stdout:1/223: symlink d3/l51 0 2024-03-24T10:46:08.552 INFO:tasks.workunit.client.1.smithi096.stdout:1/224: dread d3/f9 [8388608,4194304] 0 2024-03-24T10:46:08.554 INFO:tasks.workunit.client.1.smithi096.stdout:1/225: symlink d3/d24/l52 0 2024-03-24T10:46:08.554 INFO:tasks.workunit.client.1.smithi096.stdout:1/226: readlink d3/l20 0 2024-03-24T10:46:08.569 INFO:tasks.workunit.client.1.smithi096.stdout:1/227: dwrite d3/f15 [0,4194304] 0 2024-03-24T10:46:08.569 INFO:tasks.workunit.client.1.smithi096.stdout:1/228: read d3/ff [2981567,121146] 0 2024-03-24T10:46:08.741 INFO:tasks.workunit.client.1.smithi096.stdout:3/289: sync 2024-03-24T10:46:08.741 INFO:tasks.workunit.client.1.smithi096.stdout:6/265: sync 2024-03-24T10:46:08.741 INFO:tasks.workunit.client.1.smithi096.stdout:4/214: sync 2024-03-24T10:46:08.741 INFO:tasks.workunit.client.1.smithi096.stdout:3/290: truncate d0/f61 409517 0 2024-03-24T10:46:08.742 INFO:tasks.workunit.client.1.smithi096.stdout:9/210: sync 2024-03-24T10:46:08.743 INFO:tasks.workunit.client.1.smithi096.stdout:7/254: sync 2024-03-24T10:46:08.743 INFO:tasks.workunit.client.1.smithi096.stdout:7/255: creat d0/db/d26/f5b x:0 0 0 2024-03-24T10:46:08.743 INFO:tasks.workunit.client.1.smithi096.stdout:5/216: sync 2024-03-24T10:46:08.747 INFO:tasks.workunit.client.1.smithi096.stdout:6/266: symlink d8/l50 0 2024-03-24T10:46:08.748 INFO:tasks.workunit.client.1.smithi096.stdout:4/215: mknod d1/c49 0 2024-03-24T10:46:08.753 INFO:tasks.workunit.client.1.smithi096.stdout:9/211: write d9/d15/d26/d29/f36 [2823478,129240] 0 2024-03-24T10:46:08.757 INFO:tasks.workunit.client.1.smithi096.stdout:4/216: link d1/c17 d1/d32/d42/c4a 0 2024-03-24T10:46:08.757 INFO:tasks.workunit.client.1.smithi096.stdout:4/217: read - d1/d11/f29 zero size 2024-03-24T10:46:08.761 INFO:tasks.workunit.client.1.smithi096.stdout:4/218: symlink d1/l4b 0 2024-03-24T10:46:08.762 INFO:tasks.workunit.client.1.smithi096.stdout:6/267: read d8/f9 [2157582,117060] 0 2024-03-24T10:46:08.766 INFO:tasks.workunit.client.1.smithi096.stdout:4/219: mkdir d1/d32/d4c 0 2024-03-24T10:46:08.769 INFO:tasks.workunit.client.1.smithi096.stdout:4/220: unlink d1/c49 0 2024-03-24T10:46:08.778 INFO:tasks.workunit.client.1.smithi096.stdout:6/268: write d8/d12/f2d [2519071,55276] 0 2024-03-24T10:46:08.779 INFO:tasks.workunit.client.1.smithi096.stdout:7/256: dwrite d0/f20 [0,4194304] 0 2024-03-24T10:46:08.780 INFO:tasks.workunit.client.1.smithi096.stdout:5/217: dwrite d2/d7/d8/df/f1f [0,4194304] 0 2024-03-24T10:46:08.780 INFO:tasks.workunit.client.1.smithi096.stdout:5/218: creat d2/d7/f4a x:0 0 0 2024-03-24T10:46:08.781 INFO:tasks.workunit.client.1.smithi096.stdout:6/269: truncate d8/d10/d19/f30 3669669 0 2024-03-24T10:46:08.783 INFO:tasks.workunit.client.1.smithi096.stdout:7/257: mknod d0/d21/d59/c5c 0 2024-03-24T10:46:08.785 INFO:tasks.workunit.client.1.smithi096.stdout:3/291: dwrite d0/d13/d45/d47/f64 [0,4194304] 0 2024-03-24T10:46:08.787 INFO:tasks.workunit.client.1.smithi096.stdout:5/219: dread d2/f3 [0,4194304] 0 2024-03-24T10:46:08.787 INFO:tasks.workunit.client.1.smithi096.stdout:5/220: read d2/d7/d8/df/d14/f31 [352987,8018] 0 2024-03-24T10:46:08.788 INFO:tasks.workunit.client.1.smithi096.stdout:3/292: symlink d0/d13/l68 0 2024-03-24T10:46:08.794 INFO:tasks.workunit.client.1.smithi096.stdout:9/212: truncate d9/f3a 938975 0 2024-03-24T10:46:08.794 INFO:tasks.workunit.client.1.smithi096.stdout:9/213: truncate d9/db/f22 1063822 0 2024-03-24T10:46:08.796 INFO:tasks.workunit.client.1.smithi096.stdout:9/214: mkdir d9/db/d23/d31/d43 0 2024-03-24T10:46:08.796 INFO:tasks.workunit.client.1.smithi096.stdout:6/270: write d8/d12/d32/f47 [613319,6941] 0 2024-03-24T10:46:08.803 INFO:tasks.workunit.client.1.smithi096.stdout:9/215: rmdir d9/db/d23/d31/d3e 39 2024-03-24T10:46:08.811 INFO:tasks.workunit.client.1.smithi096.stdout:6/271: truncate d8/f31 1834503 0 2024-03-24T10:46:08.817 INFO:tasks.workunit.client.1.smithi096.stdout:6/272: mkdir d8/d10/d22/d51 0 2024-03-24T10:46:08.830 INFO:tasks.workunit.client.1.smithi096.stdout:5/221: dwrite d2/f5 [4194304,4194304] 0 2024-03-24T10:46:08.831 INFO:tasks.workunit.client.1.smithi096.stdout:5/222: write d2/d29/f38 [739443,3181] 0 2024-03-24T10:46:08.841 INFO:tasks.workunit.client.1.smithi096.stdout:9/216: dwrite d9/f16 [0,4194304] 0 2024-03-24T10:46:08.842 INFO:tasks.workunit.client.1.smithi096.stdout:6/273: dwrite d8/d10/d19/f1b [0,4194304] 0 2024-03-24T10:46:08.843 INFO:tasks.workunit.client.1.smithi096.stdout:3/293: dwrite d0/d13/d45/d52/f5b [0,4194304] 0 2024-03-24T10:46:08.843 INFO:tasks.workunit.client.1.smithi096.stdout:3/294: readlink d0/d9/l20 0 2024-03-24T10:46:08.847 INFO:tasks.workunit.client.1.smithi096.stdout:9/217: unlink d9/d15/d26/c2d 0 2024-03-24T10:46:08.854 INFO:tasks.workunit.client.1.smithi096.stdout:9/218: creat d9/db/d23/d31/d43/f44 x:0 0 0 2024-03-24T10:46:08.854 INFO:tasks.workunit.client.1.smithi096.stdout:9/219: chown d9/db/d23 908 1 2024-03-24T10:46:08.857 INFO:tasks.workunit.client.1.smithi096.stdout:9/220: link d9/f1a d9/db/d23/d31/d43/f45 0 2024-03-24T10:46:08.859 INFO:tasks.workunit.client.1.smithi096.stdout:9/221: mknod d9/db/d23/d31/c46 0 2024-03-24T10:46:08.863 INFO:tasks.workunit.client.1.smithi096.stdout:3/295: dwrite d0/d13/d1e/f2d [0,4194304] 0 2024-03-24T10:46:08.867 INFO:tasks.workunit.client.1.smithi096.stdout:5/223: dwrite d2/d29/f38 [0,4194304] 0 2024-03-24T10:46:08.867 INFO:tasks.workunit.client.1.smithi096.stdout:5/224: write d2/d7/f26 [1908470,85216] 0 2024-03-24T10:46:08.868 INFO:tasks.workunit.client.1.smithi096.stdout:3/296: dread d0/f1 [0,4194304] 0 2024-03-24T10:46:08.873 INFO:tasks.workunit.client.1.smithi096.stdout:3/297: symlink d0/d13/d41/l69 0 2024-03-24T10:46:08.878 INFO:tasks.workunit.client.1.smithi096.stdout:9/222: dwrite d9/f3a [0,4194304] 0 2024-03-24T10:46:08.879 INFO:tasks.workunit.client.1.smithi096.stdout:5/225: write d2/d7/d8/f12 [107416,24007] 0 2024-03-24T10:46:08.879 INFO:tasks.workunit.client.1.smithi096.stdout:9/223: write d9/db/d23/f35 [933976,61634] 0 2024-03-24T10:46:08.879 INFO:tasks.workunit.client.1.smithi096.stdout:5/226: truncate d2/d7/f3e 946646 0 2024-03-24T10:46:08.883 INFO:tasks.workunit.client.1.smithi096.stdout:3/298: dread d0/d13/d45/d52/f53 [0,4194304] 0 2024-03-24T10:46:08.883 INFO:tasks.workunit.client.1.smithi096.stdout:3/299: write d0/f61 [557380,13644] 0 2024-03-24T10:46:08.883 INFO:tasks.workunit.client.1.smithi096.stdout:3/300: write d0/f5c [5058621,112498] 0 2024-03-24T10:46:08.883 INFO:tasks.workunit.client.1.smithi096.stdout:3/301: write d0/f44 [96838,38944] 0 2024-03-24T10:46:08.886 INFO:tasks.workunit.client.1.smithi096.stdout:3/302: dread d0/d13/d45/d47/f4e [0,4194304] 0 2024-03-24T10:46:08.886 INFO:tasks.workunit.client.1.smithi096.stdout:3/303: chown d0/c31 88261868 1 2024-03-24T10:46:08.886 INFO:tasks.workunit.client.1.smithi096.stdout:3/304: creat d0/d9/d34/d39/d48/f6a x:0 0 0 2024-03-24T10:46:08.886 INFO:tasks.workunit.client.1.smithi096.stdout:3/305: stat d0/d9/d34 0 2024-03-24T10:46:08.894 INFO:tasks.workunit.client.1.smithi096.stdout:5/227: unlink d2/f3 0 2024-03-24T10:46:08.894 INFO:tasks.workunit.client.1.smithi096.stdout:5/228: readlink d2/d29/d32/d33/d3c/l45 0 2024-03-24T10:46:08.902 INFO:tasks.workunit.client.1.smithi096.stdout:9/224: dwrite d9/d15/f1e [0,4194304] 0 2024-03-24T10:46:08.902 INFO:tasks.workunit.client.1.smithi096.stdout:9/225: creat d9/d15/f47 x:0 0 0 2024-03-24T10:46:08.902 INFO:tasks.workunit.client.1.smithi096.stdout:9/226: readlink d9/db/d23/l3b 0 2024-03-24T10:46:08.903 INFO:tasks.workunit.client.1.smithi096.stdout:3/306: rename d0/d9/l20 to d0/d13/d45/l6b 0 2024-03-24T10:46:08.903 INFO:tasks.workunit.client.1.smithi096.stdout:3/307: readlink d0/d13/d41/l65 0 2024-03-24T10:46:08.903 INFO:tasks.workunit.client.1.smithi096.stdout:3/308: fdatasync d0/d9/d34/d39/d48/f6a 0 2024-03-24T10:46:08.910 INFO:tasks.workunit.client.1.smithi096.stdout:5/229: dwrite d2/d7/d8/d18/f20 [0,4194304] 0 2024-03-24T10:46:08.910 INFO:tasks.workunit.client.1.smithi096.stdout:5/230: readlink d2/d7/d8/df/l3a 0 2024-03-24T10:46:08.911 INFO:tasks.workunit.client.1.smithi096.stdout:5/231: dread d2/f39 [0,4194304] 0 2024-03-24T10:46:08.916 INFO:tasks.workunit.client.1.smithi096.stdout:9/227: mkdir d9/db/d23/d31/d43/d48 0 2024-03-24T10:46:08.916 INFO:tasks.workunit.client.1.smithi096.stdout:0/255: sync 2024-03-24T10:46:08.916 INFO:tasks.workunit.client.1.smithi096.stdout:1/229: sync 2024-03-24T10:46:08.916 INFO:tasks.workunit.client.1.smithi096.stdout:1/230: stat d3 0 2024-03-24T10:46:08.916 INFO:tasks.workunit.client.1.smithi096.stdout:0/256: rename d0/d11 to d0/d11/d16/d21/d25/d52 22 2024-03-24T10:46:08.916 INFO:tasks.workunit.client.1.smithi096.stdout:1/231: dread - d3/d24/f35 zero size 2024-03-24T10:46:08.917 INFO:tasks.workunit.client.1.smithi096.stdout:1/232: dread - d3/d36/f4d zero size 2024-03-24T10:46:08.917 INFO:tasks.workunit.client.1.smithi096.stdout:1/233: dread - d3/d24/f4c zero size 2024-03-24T10:46:08.917 INFO:tasks.workunit.client.1.smithi096.stdout:0/257: creat d0/d11/d16/f53 x:0 0 0 2024-03-24T10:46:08.917 INFO:tasks.workunit.client.1.smithi096.stdout:1/234: truncate d3/d24/f3e 4985886 0 2024-03-24T10:46:08.917 INFO:tasks.workunit.client.1.smithi096.stdout:0/258: write d0/d11/d24/f4c [71698,43204] 0 2024-03-24T10:46:08.917 INFO:tasks.workunit.client.1.smithi096.stdout:0/259: write d0/d11/d40/f44 [496106,35233] 0 2024-03-24T10:46:08.917 INFO:tasks.workunit.client.1.smithi096.stdout:0/260: creat d0/d11/d16/f54 x:0 0 0 2024-03-24T10:46:08.917 INFO:tasks.workunit.client.1.smithi096.stdout:0/261: creat d0/d11/d24/f55 x:0 0 0 2024-03-24T10:46:08.919 INFO:tasks.workunit.client.1.smithi096.stdout:2/183: sync 2024-03-24T10:46:08.924 INFO:tasks.workunit.client.1.smithi096.stdout:9/228: truncate d9/db/fe 1902511 0 2024-03-24T10:46:08.929 INFO:tasks.workunit.client.1.smithi096.stdout:0/262: write d0/d11/d20/d2b/f37 [3029645,41773] 0 2024-03-24T10:46:08.929 INFO:tasks.workunit.client.1.smithi096.stdout:8/302: sync 2024-03-24T10:46:08.935 INFO:tasks.workunit.client.1.smithi096.stdout:5/232: dwrite d2/f43 [0,4194304] 0 2024-03-24T10:46:08.935 INFO:tasks.workunit.client.1.smithi096.stdout:1/235: dwrite d3/f22 [0,4194304] 0 2024-03-24T10:46:08.935 INFO:tasks.workunit.client.1.smithi096.stdout:9/229: read d9/d15/f1e [2485016,2417] 0 2024-03-24T10:46:08.935 INFO:tasks.workunit.client.1.smithi096.stdout:9/230: fsync d9/db/f14 0 2024-03-24T10:46:08.935 INFO:tasks.workunit.client.1.smithi096.stdout:9/231: write d9/d15/f20 [659732,4773] 0 2024-03-24T10:46:08.935 INFO:tasks.workunit.client.1.smithi096.stdout:9/232: fdatasync d9/f1a 0 2024-03-24T10:46:08.935 INFO:tasks.workunit.client.1.smithi096.stdout:9/233: readlink d9/d15/l12 0 2024-03-24T10:46:08.936 INFO:tasks.workunit.client.1.smithi096.stdout:3/309: truncate d0/f61 187885 0 2024-03-24T10:46:08.943 INFO:tasks.workunit.client.1.smithi096.stdout:0/263: unlink d0/d11/d20/l1e 0 2024-03-24T10:46:08.944 INFO:tasks.workunit.client.1.smithi096.stdout:8/303: link d8/d11/d22/d23/d37/f39 d8/f5d 0 2024-03-24T10:46:08.952 INFO:tasks.workunit.client.1.smithi096.stdout:5/233: rename d2/d7/f26 to d2/d7/f4b 0 2024-03-24T10:46:08.952 INFO:tasks.workunit.client.1.smithi096.stdout:9/234: dwrite d9/d15/f21 [0,4194304] 0 2024-03-24T10:46:08.952 INFO:tasks.workunit.client.1.smithi096.stdout:9/235: write d9/f2e [598191,55367] 0 2024-03-24T10:46:08.955 INFO:tasks.workunit.client.1.smithi096.stdout:9/236: dread d9/db/f14 [0,4194304] 0 2024-03-24T10:46:08.961 INFO:tasks.workunit.client.1.smithi096.stdout:1/236: rename d3/d24/c3a to d3/d23/c53 0 2024-03-24T10:46:08.961 INFO:tasks.workunit.client.1.smithi096.stdout:1/237: creat d3/d36/f54 x:0 0 0 2024-03-24T10:46:08.964 INFO:tasks.workunit.client.1.smithi096.stdout:3/310: mknod d0/d13/d41/c6c 0 2024-03-24T10:46:08.964 INFO:tasks.workunit.client.1.smithi096.stdout:3/311: fsync d0/d9/f1a 0 2024-03-24T10:46:08.965 INFO:tasks.workunit.client.1.smithi096.stdout:3/312: creat d0/d13/d45/d52/f6d x:0 0 0 2024-03-24T10:46:08.965 INFO:tasks.workunit.client.1.smithi096.stdout:3/313: chown d0/d9/d34/d39/d48/f5f 21638605 1 2024-03-24T10:46:08.968 INFO:tasks.workunit.client.1.smithi096.stdout:0/264: rmdir d0/d11 39 2024-03-24T10:46:08.969 INFO:tasks.workunit.client.1.smithi096.stdout:0/265: read d0/d11/d16/d21/d25/f36 [156410,56630] 0 2024-03-24T10:46:08.970 INFO:tasks.workunit.client.1.smithi096.stdout:0/266: fdatasync d0/d11/d16/d21/d25/f36 0 2024-03-24T10:46:08.970 INFO:tasks.workunit.client.1.smithi096.stdout:3/314: dread d0/d13/d45/d52/f53 [0,4194304] 0 2024-03-24T10:46:08.970 INFO:tasks.workunit.client.1.smithi096.stdout:0/267: write d0/d11/d16/f4e [511513,109129] 0 2024-03-24T10:46:08.983 INFO:tasks.workunit.client.1.smithi096.stdout:5/234: rename d2/d7/d8/lb to d2/d29/d32/d33/l4c 0 2024-03-24T10:46:08.989 INFO:tasks.workunit.client.1.smithi096.stdout:1/238: symlink d3/d24/l55 0 2024-03-24T10:46:08.992 INFO:tasks.workunit.client.1.smithi096.stdout:5/235: dread d2/d7/d8/df/f2d [4194304,4194304] 0 2024-03-24T10:46:08.996 INFO:tasks.workunit.client.1.smithi096.stdout:0/268: mkdir d0/d11/d20/d56 0 2024-03-24T10:46:09.002 INFO:tasks.workunit.client.1.smithi096.stdout:9/237: truncate d9/d15/f21 625315 0 2024-03-24T10:46:09.005 INFO:tasks.workunit.client.1.smithi096.stdout:5/236: mkdir d2/d7/d8/d18/d4d 0 2024-03-24T10:46:09.009 INFO:tasks.workunit.client.1.smithi096.stdout:0/269: mknod d0/d11/d20/d56/c57 0 2024-03-24T10:46:09.010 INFO:tasks.workunit.client.1.smithi096.stdout:0/270: fsync d0/d11/d24/f55 0 2024-03-24T10:46:09.011 INFO:tasks.workunit.client.1.smithi096.stdout:1/239: dwrite d3/d23/d2f/f37 [0,4194304] 0 2024-03-24T10:46:09.013 INFO:tasks.workunit.client.1.smithi096.stdout:0/271: write d0/d11/d16/f4e [501000,122804] 0 2024-03-24T10:46:09.016 INFO:tasks.workunit.client.1.smithi096.stdout:1/240: creat d3/d23/f56 x:0 0 0 2024-03-24T10:46:09.016 INFO:tasks.workunit.client.1.smithi096.stdout:1/241: creat d3/d24/f57 x:0 0 0 2024-03-24T10:46:09.016 INFO:tasks.workunit.client.1.smithi096.stdout:9/238: dread d9/d15/d26/f33 [0,4194304] 0 2024-03-24T10:46:09.017 INFO:tasks.workunit.client.1.smithi096.stdout:0/272: rmdir d0/d11/d16/d21/d25/d47 39 2024-03-24T10:46:09.018 INFO:tasks.workunit.client.1.smithi096.stdout:0/273: write d0/d11/d24/f50 [168309,62489] 0 2024-03-24T10:46:09.018 INFO:tasks.workunit.client.1.smithi096.stdout:0/274: truncate d0/d11/d24/d2c/f32 580909 0 2024-03-24T10:46:09.022 INFO:tasks.workunit.client.1.smithi096.stdout:1/242: dread d3/d23/f45 [0,4194304] 0 2024-03-24T10:46:09.025 INFO:tasks.workunit.client.1.smithi096.stdout:0/275: symlink d0/l58 0 2024-03-24T10:46:09.025 INFO:tasks.workunit.client.1.smithi096.stdout:0/276: truncate d0/d11/d20/f26 546384 0 2024-03-24T10:46:09.025 INFO:tasks.workunit.client.1.smithi096.stdout:0/277: dread - d0/d11/d16/f54 zero size 2024-03-24T10:46:09.027 INFO:tasks.workunit.client.1.smithi096.stdout:0/278: rename d0/d11/l34 to d0/d2/l59 0 2024-03-24T10:46:09.027 INFO:tasks.workunit.client.1.smithi096.stdout:0/279: truncate d0/d11/d24/f2d 719663 0 2024-03-24T10:46:09.034 INFO:tasks.workunit.client.1.smithi096.stdout:9/239: dwrite d9/db/d23/f35 [0,4194304] 0 2024-03-24T10:46:09.034 INFO:tasks.workunit.client.1.smithi096.stdout:5/237: dwrite d2/d7/f41 [0,4194304] 0 2024-03-24T10:46:09.035 INFO:tasks.workunit.client.1.smithi096.stdout:3/315: dwrite d0/d9/f49 [0,4194304] 0 2024-03-24T10:46:09.040 INFO:tasks.workunit.client.1.smithi096.stdout:1/243: dwrite d3/d23/f56 [0,4194304] 0 2024-03-24T10:46:09.041 INFO:tasks.workunit.client.1.smithi096.stdout:9/240: unlink d9/d15/f25 0 2024-03-24T10:46:09.042 INFO:tasks.workunit.client.1.smithi096.stdout:9/241: chown d9/d15/f1e 6469914 1 2024-03-24T10:46:09.049 INFO:tasks.workunit.client.1.smithi096.stdout:3/316: unlink d0/d13/d45/f4c 0 2024-03-24T10:46:09.049 INFO:tasks.workunit.client.1.smithi096.stdout:3/317: write d0/f61 [684858,92164] 0 2024-03-24T10:46:09.049 INFO:tasks.workunit.client.1.smithi096.stdout:3/318: write d0/f1 [5232992,34315] 0 2024-03-24T10:46:09.049 INFO:tasks.workunit.client.1.smithi096.stdout:3/319: readlink d0/d13/d41/l62 0 2024-03-24T10:46:09.055 INFO:tasks.workunit.client.1.smithi096.stdout:3/320: dread d0/f5c [4194304,4194304] 0 2024-03-24T10:46:09.055 INFO:tasks.workunit.client.1.smithi096.stdout:3/321: chown d0/f1 1548991 1 2024-03-24T10:46:09.055 INFO:tasks.workunit.client.1.smithi096.stdout:0/280: dwrite d0/f5 [0,4194304] 0 2024-03-24T10:46:09.074 INFO:tasks.workunit.client.1.smithi096.stdout:5/238: dwrite d2/d29/d32/f40 [4194304,4194304] 0 2024-03-24T10:46:09.074 INFO:tasks.workunit.client.1.smithi096.stdout:5/239: stat d2/d7/c16 0 2024-03-24T10:46:09.074 INFO:tasks.workunit.client.1.smithi096.stdout:1/244: dwrite d3/f34 [0,4194304] 0 2024-03-24T10:46:09.078 INFO:tasks.workunit.client.1.smithi096.stdout:5/240: link d2/d29/d32/d33/l48 d2/d29/d32/d33/d3c/d3d/l4e 0 2024-03-24T10:46:09.078 INFO:tasks.workunit.client.1.smithi096.stdout:5/241: write d2/f39 [1249970,96992] 0 2024-03-24T10:46:09.078 INFO:tasks.workunit.client.1.smithi096.stdout:3/322: dwrite d0/d9/d34/d39/d48/f5f [0,4194304] 0 2024-03-24T10:46:09.078 INFO:tasks.workunit.client.1.smithi096.stdout:0/281: dwrite d0/d11/d40/f44 [0,4194304] 0 2024-03-24T10:46:09.080 INFO:tasks.workunit.client.1.smithi096.stdout:1/245: symlink d3/d23/d2f/l58 0 2024-03-24T10:46:09.081 INFO:tasks.workunit.client.1.smithi096.stdout:5/242: symlink d2/d29/d32/d33/d3c/l4f 0 2024-03-24T10:46:09.084 INFO:tasks.workunit.client.1.smithi096.stdout:3/323: dread d0/d13/d45/d52/f53 [0,4194304] 0 2024-03-24T10:46:09.087 INFO:tasks.workunit.client.1.smithi096.stdout:5/243: dread d2/d7/f41 [4194304,4194304] 0 2024-03-24T10:46:09.099 INFO:tasks.workunit.client.1.smithi096.stdout:0/282: mkdir d0/d11/d20/d5a 0 2024-03-24T10:46:09.100 INFO:tasks.workunit.client.1.smithi096.stdout:1/246: creat d3/d23/d2f/f59 x:0 0 0 2024-03-24T10:46:09.100 INFO:tasks.workunit.client.1.smithi096.stdout:1/247: stat d3/f15 0 2024-03-24T10:46:09.102 INFO:tasks.workunit.client.1.smithi096.stdout:3/324: rename d0/d9/f2e to d0/d13/d1e/f6e 0 2024-03-24T10:46:09.103 INFO:tasks.workunit.client.1.smithi096.stdout:0/283: symlink d0/d11/d20/d56/l5b 0 2024-03-24T10:46:09.105 INFO:tasks.workunit.client.1.smithi096.stdout:1/248: link d3/d23/f42 d3/d23/d2f/f5a 0 2024-03-24T10:46:09.105 INFO:tasks.workunit.client.1.smithi096.stdout:1/249: fsync d3/d24/f4c 0 2024-03-24T10:46:09.105 INFO:tasks.workunit.client.1.smithi096.stdout:1/250: truncate d3/d24/f4c 42814 0 2024-03-24T10:46:09.105 INFO:tasks.workunit.client.1.smithi096.stdout:1/251: chown d3/f1f 75 1 2024-03-24T10:46:09.111 INFO:tasks.workunit.client.1.smithi096.stdout:0/284: dread d0/d11/d40/f44 [0,4194304] 0 2024-03-24T10:46:09.111 INFO:tasks.workunit.client.1.smithi096.stdout:5/244: rename d2/d7/d8/df/d14/l27 to d2/d7/d8/d18/l50 0 2024-03-24T10:46:09.118 INFO:tasks.workunit.client.1.smithi096.stdout:3/325: rmdir d0/d9/d34 39 2024-03-24T10:46:09.118 INFO:tasks.workunit.client.1.smithi096.stdout:3/326: dread - d0/d9/d34/f67 zero size 2024-03-24T10:46:09.118 INFO:tasks.workunit.client.1.smithi096.stdout:3/327: fdatasync d0/d13/f46 0 2024-03-24T10:46:09.120 INFO:tasks.workunit.client.1.smithi096.stdout:5/245: write d2/d7/f10 [2917638,119805] 0 2024-03-24T10:46:09.123 INFO:tasks.workunit.client.1.smithi096.stdout:1/252: rename d3/d23/d2f/l3c to d3/d36/l5b 0 2024-03-24T10:46:09.123 INFO:tasks.workunit.client.1.smithi096.stdout:1/253: chown d3/d23/f42 31 1 2024-03-24T10:46:09.127 INFO:tasks.workunit.client.1.smithi096.stdout:5/246: truncate d2/f5 2195115 0 2024-03-24T10:46:09.127 INFO:tasks.workunit.client.1.smithi096.stdout:3/328: read d0/d9/f49 [4103421,41784] 0 2024-03-24T10:46:09.129 INFO:tasks.workunit.client.1.smithi096.stdout:1/254: mkdir d3/d24/d5c 0 2024-03-24T10:46:09.129 INFO:tasks.workunit.client.1.smithi096.stdout:1/255: truncate d3/d24/f4c 373715 0 2024-03-24T10:46:09.136 INFO:tasks.workunit.client.1.smithi096.stdout:3/329: write d0/d9/d34/d39/d48/f5f [1659819,104622] 0 2024-03-24T10:46:09.136 INFO:tasks.workunit.client.1.smithi096.stdout:3/330: truncate d0/d9/d34/f67 490465 0 2024-03-24T10:46:09.138 INFO:tasks.workunit.client.1.smithi096.stdout:3/331: mknod d0/d9/c6f 0 2024-03-24T10:46:09.144 INFO:tasks.workunit.client.1.smithi096.stdout:3/332: write d0/d13/d45/d52/f53 [3359009,61479] 0 2024-03-24T10:46:09.145 INFO:tasks.workunit.client.1.smithi096.stdout:3/333: read - d0/d13/d45/d52/f66 zero size 2024-03-24T10:46:09.146 INFO:tasks.workunit.client.1.smithi096.stdout:3/334: rename d0/l5 to d0/l70 0 2024-03-24T10:46:09.147 INFO:tasks.workunit.client.1.smithi096.stdout:3/335: write d0/d13/d1e/f2a [4518094,6965] 0 2024-03-24T10:46:09.148 INFO:tasks.workunit.client.1.smithi096.stdout:3/336: truncate d0/d13/f1f 852848 0 2024-03-24T10:46:09.148 INFO:tasks.workunit.client.1.smithi096.stdout:3/337: chown d0/d9/f32 122515848 1 2024-03-24T10:46:09.148 INFO:tasks.workunit.client.1.smithi096.stdout:3/338: mkdir d0/d9/d34/d71 0 2024-03-24T10:46:09.148 INFO:tasks.workunit.client.1.smithi096.stdout:3/339: read - d0/d9/f3b zero size 2024-03-24T10:46:09.149 INFO:tasks.workunit.client.1.smithi096.stdout:3/340: rename d0/d13/d1e/f6e to d0/f72 0 2024-03-24T10:46:09.149 INFO:tasks.workunit.client.1.smithi096.stdout:3/341: read - d0/d9/d34/f63 zero size 2024-03-24T10:46:09.149 INFO:tasks.workunit.client.1.smithi096.stdout:3/342: truncate d0/f61 1281783 0 2024-03-24T10:46:09.151 INFO:tasks.workunit.client.1.smithi096.stdout:5/247: dwrite d2/d7/d8/df/f25 [0,4194304] 0 2024-03-24T10:46:09.152 INFO:tasks.workunit.client.1.smithi096.stdout:1/256: dwrite d3/d24/f57 [0,4194304] 0 2024-03-24T10:46:09.152 INFO:tasks.workunit.client.1.smithi096.stdout:1/257: truncate d3/d36/f54 475135 0 2024-03-24T10:46:09.152 INFO:tasks.workunit.client.1.smithi096.stdout:1/258: fsync d3/f15 0 2024-03-24T10:46:09.155 INFO:tasks.workunit.client.1.smithi096.stdout:5/248: symlink d2/d29/d32/l51 0 2024-03-24T10:46:09.156 INFO:tasks.workunit.client.1.smithi096.stdout:5/249: creat d2/d7/f52 x:0 0 0 2024-03-24T10:46:09.156 INFO:tasks.workunit.client.1.smithi096.stdout:5/250: write d2/d7/d8/df/f1f [5146642,84556] 0 2024-03-24T10:46:09.164 INFO:tasks.workunit.client.1.smithi096.stdout:5/251: mknod d2/d29/d32/c53 0 2024-03-24T10:46:09.166 INFO:tasks.workunit.client.1.smithi096.stdout:3/343: dwrite d0/d13/f57 [0,4194304] 0 2024-03-24T10:46:09.167 INFO:tasks.workunit.client.1.smithi096.stdout:5/252: link d2/f39 d2/d29/f54 0 2024-03-24T10:46:09.171 INFO:tasks.workunit.client.1.smithi096.stdout:3/344: symlink d0/d9/d34/l73 0 2024-03-24T10:46:09.186 INFO:tasks.workunit.client.1.smithi096.stdout:5/253: dwrite d2/d7/d8/df/f1f [4194304,4194304] 0 2024-03-24T10:46:09.188 INFO:tasks.workunit.client.1.smithi096.stdout:5/254: creat d2/d29/d32/d3f/f55 x:0 0 0 2024-03-24T10:46:09.191 INFO:tasks.workunit.client.1.smithi096.stdout:5/255: creat d2/d7/d8/f56 x:0 0 0 2024-03-24T10:46:09.199 INFO:tasks.workunit.client.1.smithi096.stdout:5/256: dread d2/d7/f4b [0,4194304] 0 2024-03-24T10:46:09.199 INFO:tasks.workunit.client.1.smithi096.stdout:5/257: truncate d2/d7/f10 5100654 0 2024-03-24T10:46:09.207 INFO:tasks.workunit.client.1.smithi096.stdout:5/258: dread d2/d29/d32/d33/f44 [4194304,4194304] 0 2024-03-24T10:46:09.208 INFO:tasks.workunit.client.1.smithi096.stdout:5/259: fdatasync d2/d7/d8/df/d14/f23 0 2024-03-24T10:46:09.208 INFO:tasks.workunit.client.1.smithi096.stdout:5/260: dread - d2/d7/d8/f56 zero size 2024-03-24T10:46:09.208 INFO:tasks.workunit.client.1.smithi096.stdout:5/261: rename d2/d29/d32/d3f to d2/d29/d32/d3f/d57 22 2024-03-24T10:46:09.224 INFO:tasks.workunit.client.1.smithi096.stdout:5/262: dwrite d2/d7/f10 [0,4194304] 0 2024-03-24T10:46:09.231 INFO:tasks.workunit.client.1.smithi096.stdout:5/263: mknod d2/d29/d32/d33/d3c/d3d/c58 0 2024-03-24T10:46:09.231 INFO:tasks.workunit.client.1.smithi096.stdout:5/264: read - d2/d7/f52 zero size 2024-03-24T10:46:09.231 INFO:tasks.workunit.client.1.smithi096.stdout:5/265: dread - d2/d7/f4a zero size 2024-03-24T10:46:09.231 INFO:tasks.workunit.client.1.smithi096.stdout:5/266: readlink d2/d7/l2f 0 2024-03-24T10:46:09.233 INFO:tasks.workunit.client.1.smithi096.stdout:5/267: mknod d2/d29/d32/c59 0 2024-03-24T10:46:09.235 INFO:tasks.workunit.client.1.smithi096.stdout:5/268: creat d2/f5a x:0 0 0 2024-03-24T10:46:09.240 INFO:tasks.workunit.client.1.smithi096.stdout:5/269: symlink d2/l5b 0 2024-03-24T10:46:09.247 INFO:tasks.workunit.client.1.smithi096.stdout:5/270: dread d2/d7/d8/df/f25 [0,4194304] 0 2024-03-24T10:46:09.255 INFO:tasks.workunit.client.1.smithi096.stdout:5/271: dread d2/f43 [0,4194304] 0 2024-03-24T10:46:09.260 INFO:tasks.workunit.client.1.smithi096.stdout:5/272: dread d2/d7/d8/df/f2d [0,4194304] 0 2024-03-24T10:46:09.318 INFO:tasks.workunit.client.1.smithi096.stdout:7/258: sync 2024-03-24T10:46:09.319 INFO:tasks.workunit.client.1.smithi096.stdout:4/221: sync 2024-03-24T10:46:09.319 INFO:tasks.workunit.client.1.smithi096.stdout:2/184: sync 2024-03-24T10:46:09.319 INFO:tasks.workunit.client.1.smithi096.stdout:6/274: sync 2024-03-24T10:46:09.319 INFO:tasks.workunit.client.1.smithi096.stdout:2/185: truncate d5/dc/f34 614610 0 2024-03-24T10:46:09.319 INFO:tasks.workunit.client.1.smithi096.stdout:2/186: chown d5/dc 897 1 2024-03-24T10:46:09.321 INFO:tasks.workunit.client.1.smithi096.stdout:2/187: dread d5/de/f1a [0,4194304] 0 2024-03-24T10:46:09.321 INFO:tasks.workunit.client.1.smithi096.stdout:2/188: stat d5/de/d2a/c39 0 2024-03-24T10:46:09.323 INFO:tasks.workunit.client.1.smithi096.stdout:7/259: mknod d0/d13/c5d 0 2024-03-24T10:46:09.325 INFO:tasks.workunit.client.1.smithi096.stdout:4/222: rename d1/d11/c28 to d1/d2b/d40/c4d 0 2024-03-24T10:46:09.328 INFO:tasks.workunit.client.1.smithi096.stdout:7/260: dread d0/d13/f5 [0,4194304] 0 2024-03-24T10:46:09.328 INFO:tasks.workunit.client.1.smithi096.stdout:7/261: stat d0/db/d26/f55 0 2024-03-24T10:46:09.330 INFO:tasks.workunit.client.1.smithi096.stdout:2/189: write d5/fa [3430763,56017] 0 2024-03-24T10:46:09.334 INFO:tasks.workunit.client.1.smithi096.stdout:7/262: dread d0/f14 [0,4194304] 0 2024-03-24T10:46:09.339 INFO:tasks.workunit.client.1.smithi096.stdout:7/263: dread d0/d13/f5 [0,4194304] 0 2024-03-24T10:46:09.340 INFO:tasks.workunit.client.1.smithi096.stdout:7/264: getdents d0/d21/d3c 0 2024-03-24T10:46:09.342 INFO:tasks.workunit.client.1.smithi096.stdout:2/190: read d5/dc/f25 [252305,104525] 0 2024-03-24T10:46:09.343 INFO:tasks.workunit.client.1.smithi096.stdout:2/191: mknod d5/dc/d12/c52 0 2024-03-24T10:46:09.344 INFO:tasks.workunit.client.1.smithi096.stdout:4/223: dwrite d1/fd [0,4194304] 0 2024-03-24T10:46:09.357 INFO:tasks.workunit.client.1.smithi096.stdout:7/265: dwrite d0/d13/d32/f3b [0,4194304] 0 2024-03-24T10:46:09.365 INFO:tasks.workunit.client.1.smithi096.stdout:2/192: dwrite d5/de/d2a/f37 [4194304,4194304] 0 2024-03-24T10:46:09.473 INFO:tasks.workunit.client.1.smithi096.stdout:4/224: write d1/d11/f23 [1820180,62425] 0 2024-03-24T10:46:09.479 INFO:tasks.workunit.client.1.smithi096.stdout:7/266: symlink d0/db/d26/l5e 0 2024-03-24T10:46:09.479 INFO:tasks.workunit.client.1.smithi096.stdout:4/225: dread d1/d11/f23 [0,4194304] 0 2024-03-24T10:46:09.481 INFO:tasks.workunit.client.1.smithi096.stdout:2/193: truncate d5/dc/d12/f20 3534091 0 2024-03-24T10:46:09.489 INFO:tasks.workunit.client.1.smithi096.stdout:2/194: creat d5/de/f53 x:0 0 0 2024-03-24T10:46:09.491 INFO:tasks.workunit.client.1.smithi096.stdout:2/195: creat d5/dc/d12/d23/f54 x:0 0 0 2024-03-24T10:46:09.492 INFO:tasks.workunit.client.1.smithi096.stdout:2/196: getdents d5 0 2024-03-24T10:46:09.493 INFO:tasks.workunit.client.1.smithi096.stdout:7/267: dwrite d0/db/f1b [0,4194304] 0 2024-03-24T10:46:09.493 INFO:tasks.workunit.client.1.smithi096.stdout:7/268: chown d0/d21/d59 82417686 1 2024-03-24T10:46:09.494 INFO:tasks.workunit.client.1.smithi096.stdout:2/197: mknod d5/d38/c55 0 2024-03-24T10:46:09.506 INFO:tasks.workunit.client.1.smithi096.stdout:7/269: dwrite d0/f5a [0,4194304] 0 2024-03-24T10:46:09.514 INFO:tasks.workunit.client.1.smithi096.stdout:7/270: write d0/f20 [3091033,97205] 0 2024-03-24T10:46:09.514 INFO:tasks.workunit.client.1.smithi096.stdout:7/271: fsync d0/d13/d32/f3b 0 2024-03-24T10:46:09.517 INFO:tasks.workunit.client.1.smithi096.stdout:7/272: mknod d0/d13/c5f 0 2024-03-24T10:46:09.517 INFO:tasks.workunit.client.1.smithi096.stdout:8/304: sync 2024-03-24T10:46:09.517 INFO:tasks.workunit.client.1.smithi096.stdout:9/242: sync 2024-03-24T10:46:09.520 INFO:tasks.workunit.client.1.smithi096.stdout:7/273: mkdir d0/d13/d60 0 2024-03-24T10:46:09.522 INFO:tasks.workunit.client.1.smithi096.stdout:9/243: mkdir d9/db/d23/d31/d49 0 2024-03-24T10:46:09.523 INFO:tasks.workunit.client.1.smithi096.stdout:7/274: creat d0/db/d1c/f61 x:0 0 0 2024-03-24T10:46:09.523 INFO:tasks.workunit.client.1.smithi096.stdout:7/275: write d0/d21/f27 [4289369,30921] 0 2024-03-24T10:46:09.529 INFO:tasks.workunit.client.1.smithi096.stdout:9/244: dread d9/d15/d26/f33 [0,4194304] 0 2024-03-24T10:46:09.532 INFO:tasks.workunit.client.1.smithi096.stdout:9/245: truncate d9/f3a 3806461 0 2024-03-24T10:46:09.532 INFO:tasks.workunit.client.1.smithi096.stdout:9/246: write d9/d15/f27 [1498781,104716] 0 2024-03-24T10:46:09.532 INFO:tasks.workunit.client.1.smithi096.stdout:9/247: creat d9/d15/f4a x:0 0 0 2024-03-24T10:46:09.532 INFO:tasks.workunit.client.1.smithi096.stdout:9/248: creat d9/db/d23/d31/f4b x:0 0 0 2024-03-24T10:46:09.546 INFO:tasks.workunit.client.1.smithi096.stdout:8/305: dwrite d8/d11/d22/d23/d1d/f5c [0,4194304] 0 2024-03-24T10:46:09.546 INFO:tasks.workunit.client.1.smithi096.stdout:7/276: dwrite d0/d13/d32/f3b [0,4194304] 0 2024-03-24T10:46:09.546 INFO:tasks.workunit.client.1.smithi096.stdout:7/277: write d0/d21/f51 [4810105,14557] 0 2024-03-24T10:46:09.547 INFO:tasks.workunit.client.1.smithi096.stdout:8/306: creat d8/d11/d22/d23/d1e/f5e x:0 0 0 2024-03-24T10:46:09.547 INFO:tasks.workunit.client.1.smithi096.stdout:8/307: chown d8/d11/d22/d4c/l4f 1553754 1 2024-03-24T10:46:09.550 INFO:tasks.workunit.client.1.smithi096.stdout:7/278: dread d0/d13/f35 [0,4194304] 0 2024-03-24T10:46:09.550 INFO:tasks.workunit.client.1.smithi096.stdout:7/279: dread - d0/db/d3a/f41 zero size 2024-03-24T10:46:09.550 INFO:tasks.workunit.client.1.smithi096.stdout:8/308: fsync d8/d11/d22/d23/d1d/d3d/f4a 0 2024-03-24T10:46:09.550 INFO:tasks.workunit.client.1.smithi096.stdout:7/280: read - d0/db/d3a/f41 zero size 2024-03-24T10:46:09.550 INFO:tasks.workunit.client.1.smithi096.stdout:7/281: chown d0/db/l28 5 1 2024-03-24T10:46:09.550 INFO:tasks.workunit.client.1.smithi096.stdout:7/282: write d0/db/d4f/f58 [713029,111444] 0 2024-03-24T10:46:09.550 INFO:tasks.workunit.client.1.smithi096.stdout:7/283: fdatasync d0/d13/f3d 0 2024-03-24T10:46:09.550 INFO:tasks.workunit.client.1.smithi096.stdout:7/284: write d0/f20 [3113209,90743] 0 2024-03-24T10:46:09.559 INFO:tasks.workunit.client.1.smithi096.stdout:8/309: write d8/d11/f43 [3904144,20781] 0 2024-03-24T10:46:09.560 INFO:tasks.workunit.client.1.smithi096.stdout:7/285: write d0/d21/f51 [1001573,16500] 0 2024-03-24T10:46:09.562 INFO:tasks.workunit.client.1.smithi096.stdout:8/310: mkdir d8/d11/d22/d23/d1d/d3d/d5f 0 2024-03-24T10:46:09.566 INFO:tasks.workunit.client.1.smithi096.stdout:7/286: dread d0/d13/f35 [0,4194304] 0 2024-03-24T10:46:09.566 INFO:tasks.workunit.client.1.smithi096.stdout:7/287: write d0/db/d1c/d39/f4a [167344,73091] 0 2024-03-24T10:46:09.566 INFO:tasks.workunit.client.1.smithi096.stdout:8/311: write d8/f14 [219970,38959] 0 2024-03-24T10:46:09.566 INFO:tasks.workunit.client.1.smithi096.stdout:8/312: write d8/d11/d22/d23/d1a/f26 [510308,46231] 0 2024-03-24T10:46:09.566 INFO:tasks.workunit.client.1.smithi096.stdout:8/313: chown d8/d11/d22/d23/d1a/d49 132260625 1 2024-03-24T10:46:09.568 INFO:tasks.workunit.client.1.smithi096.stdout:7/288: mkdir d0/db/d3a/d4b/d62 0 2024-03-24T10:46:09.569 INFO:tasks.workunit.client.1.smithi096.stdout:8/314: mknod d8/d11/d22/d23/d1d/d3d/d5f/c60 0 2024-03-24T10:46:09.569 INFO:tasks.workunit.client.1.smithi096.stdout:9/249: dwrite d9/d15/d26/d29/f36 [0,4194304] 0 2024-03-24T10:46:09.571 INFO:tasks.workunit.client.1.smithi096.stdout:8/315: fdatasync d8/f14 0 2024-03-24T10:46:09.574 INFO:tasks.workunit.client.1.smithi096.stdout:9/250: creat d9/db/d23/d31/d49/f4c x:0 0 0 2024-03-24T10:46:09.576 INFO:tasks.workunit.client.1.smithi096.stdout:7/289: write d0/db/d26/f31 [3526272,115793] 0 2024-03-24T10:46:09.576 INFO:tasks.workunit.client.1.smithi096.stdout:7/290: write d0/db/d26/f5b [995252,110063] 0 2024-03-24T10:46:09.578 INFO:tasks.workunit.client.1.smithi096.stdout:8/316: rename d8/d11/d22/d23/d1d/c53 to d8/d11/d22/d23/c61 0 2024-03-24T10:46:09.579 INFO:tasks.workunit.client.1.smithi096.stdout:8/317: write d8/d11/d22/d23/d1d/d3d/f4a [324721,15969] 0 2024-03-24T10:46:09.580 INFO:tasks.workunit.client.1.smithi096.stdout:8/318: symlink d8/d11/d22/d23/d1d/d3d/l62 0 2024-03-24T10:46:09.580 INFO:tasks.workunit.client.1.smithi096.stdout:8/319: write d8/d11/d22/d23/d52/f57 [270208,115196] 0 2024-03-24T10:46:09.580 INFO:tasks.workunit.client.1.smithi096.stdout:8/320: truncate d8/d11/d22/d23/d1d/d3d/f4b 938523 0 2024-03-24T10:46:09.582 INFO:tasks.workunit.client.1.smithi096.stdout:8/321: mknod d8/d11/d22/c63 0 2024-03-24T10:46:09.587 INFO:tasks.workunit.client.1.smithi096.stdout:7/291: dread d0/f38 [0,4194304] 0 2024-03-24T10:46:09.588 INFO:tasks.workunit.client.1.smithi096.stdout:8/322: dread d8/d11/d22/d23/d37/f39 [0,4194304] 0 2024-03-24T10:46:09.591 INFO:tasks.workunit.client.1.smithi096.stdout:9/251: dwrite d9/d15/d26/d29/f42 [0,4194304] 0 2024-03-24T10:46:09.591 INFO:tasks.workunit.client.1.smithi096.stdout:9/252: fdatasync d9/d15/f47 0 2024-03-24T10:46:09.591 INFO:tasks.workunit.client.1.smithi096.stdout:9/253: readlink d9/d15/l12 0 2024-03-24T10:46:09.600 INFO:tasks.workunit.client.1.smithi096.stdout:8/323: mkdir d8/d11/d22/d4c/d64 0 2024-03-24T10:46:09.601 INFO:tasks.workunit.client.1.smithi096.stdout:9/254: dread d9/db/d23/f35 [0,4194304] 0 2024-03-24T10:46:09.614 INFO:tasks.workunit.client.1.smithi096.stdout:8/324: write d8/d11/d22/d23/f1c [3940946,81412] 0 2024-03-24T10:46:09.614 INFO:tasks.workunit.client.1.smithi096.stdout:8/325: chown d8/d11/d22/d23/d1a/d46/d2e/l3f 522682581 1 2024-03-24T10:46:09.623 INFO:tasks.workunit.client.1.smithi096.stdout:7/292: dwrite d0/db/d4f/f58 [0,4194304] 0 2024-03-24T10:46:09.627 INFO:tasks.workunit.client.1.smithi096.stdout:8/326: getdents d8/d11/d22/d23 0 2024-03-24T10:46:09.628 INFO:tasks.workunit.client.1.smithi096.stdout:9/255: dwrite d9/d15/f3c [0,4194304] 0 2024-03-24T10:46:09.628 INFO:tasks.workunit.client.1.smithi096.stdout:7/293: dread d0/db/d26/f31 [0,4194304] 0 2024-03-24T10:46:09.632 INFO:tasks.workunit.client.1.smithi096.stdout:9/256: dread d9/d15/f27 [0,4194304] 0 2024-03-24T10:46:09.637 INFO:tasks.workunit.client.1.smithi096.stdout:7/294: dread d0/d13/f5 [0,4194304] 0 2024-03-24T10:46:09.639 INFO:tasks.workunit.client.1.smithi096.stdout:7/295: write d0/db/d26/f5b [342943,12676] 0 2024-03-24T10:46:09.639 INFO:tasks.workunit.client.1.smithi096.stdout:7/296: read - d0/db/d26/f37 zero size 2024-03-24T10:46:09.647 INFO:tasks.workunit.client.1.smithi096.stdout:7/297: creat d0/d13/d32/d45/f63 x:0 0 0 2024-03-24T10:46:09.648 INFO:tasks.workunit.client.1.smithi096.stdout:8/327: rename d8/d11/d22/d23 to d8/d11/d22/d4c/d65 0 2024-03-24T10:46:09.649 INFO:tasks.workunit.client.1.smithi096.stdout:9/257: dwrite d9/db/d23/d31/f34 [0,4194304] 0 2024-03-24T10:46:09.651 INFO:tasks.workunit.client.1.smithi096.stdout:7/298: unlink d0/d13/f15 0 2024-03-24T10:46:09.652 INFO:tasks.workunit.client.1.smithi096.stdout:8/328: mknod d8/d11/d44/c66 0 2024-03-24T10:46:09.654 INFO:tasks.workunit.client.1.smithi096.stdout:7/299: mknod d0/db/d26/c64 0 2024-03-24T10:46:09.654 INFO:tasks.workunit.client.1.smithi096.stdout:7/300: creat d0/db/d4f/f65 x:0 0 0 2024-03-24T10:46:09.656 INFO:tasks.workunit.client.1.smithi096.stdout:8/329: rmdir d8/d11/d22/d4c/d64 0 2024-03-24T10:46:09.659 INFO:tasks.workunit.client.1.smithi096.stdout:7/301: mknod d0/d13/d32/d45/c66 0 2024-03-24T10:46:09.661 INFO:tasks.workunit.client.1.smithi096.stdout:7/302: creat d0/db/d26/d48/f67 x:0 0 0 2024-03-24T10:46:09.665 INFO:tasks.workunit.client.1.smithi096.stdout:7/303: rename d0/db/d3a to d0/d21/d3c/d68 0 2024-03-24T10:46:09.667 INFO:tasks.workunit.client.1.smithi096.stdout:9/258: dwrite d9/d15/f27 [0,4194304] 0 2024-03-24T10:46:09.668 INFO:tasks.workunit.client.1.smithi096.stdout:9/259: stat d9/f3a 0 2024-03-24T10:46:09.668 INFO:tasks.workunit.client.1.smithi096.stdout:9/260: chown d9/db/d23/d31/d3e/l40 2 1 2024-03-24T10:46:09.682 INFO:tasks.workunit.client.1.smithi096.stdout:7/304: dwrite d0/d13/f35 [0,4194304] 0 2024-03-24T10:46:09.684 INFO:tasks.workunit.client.1.smithi096.stdout:7/305: dread d0/db/d1c/d39/f4a [0,4194304] 0 2024-03-24T10:46:09.684 INFO:tasks.workunit.client.1.smithi096.stdout:7/306: chown d0/db/l33 0 1 2024-03-24T10:46:09.706 INFO:tasks.workunit.client.1.smithi096.stdout:7/307: dwrite d0/d21/f27 [0,4194304] 0 2024-03-24T10:46:09.712 INFO:tasks.workunit.client.1.smithi096.stdout:9/261: symlink d9/d15/l4d 0 2024-03-24T10:46:09.713 INFO:tasks.workunit.client.1.smithi096.stdout:7/308: dread d0/f14 [0,4194304] 0 2024-03-24T10:46:09.720 INFO:tasks.workunit.client.1.smithi096.stdout:7/309: truncate d0/db/f1b 4875308 0 2024-03-24T10:46:09.723 INFO:tasks.workunit.client.1.smithi096.stdout:9/262: dread d9/d15/f27 [0,4194304] 0 2024-03-24T10:46:09.725 INFO:tasks.workunit.client.1.smithi096.stdout:9/263: rename d9/db/d23/d31/f34 to d9/d15/d26/d29/f4e 0 2024-03-24T10:46:09.725 INFO:tasks.workunit.client.1.smithi096.stdout:9/264: fsync d9/db/d23/d31/f4b 0 2024-03-24T10:46:09.727 INFO:tasks.workunit.client.1.smithi096.stdout:9/265: mkdir d9/db/d23/d4f 0 2024-03-24T10:46:09.731 INFO:tasks.workunit.client.1.smithi096.stdout:9/266: link d9/d15/d26/d29/f42 d9/db/d23/d31/d49/f50 0 2024-03-24T10:46:09.734 INFO:tasks.workunit.client.1.smithi096.stdout:9/267: mkdir d9/db/d23/d31/d43/d48/d51 0 2024-03-24T10:46:09.737 INFO:tasks.workunit.client.1.smithi096.stdout:8/330: getdents d8/d11/d22/d4c 0 2024-03-24T10:46:09.737 INFO:tasks.workunit.client.1.smithi096.stdout:8/331: truncate d8/d11/d44/d45/f4d 4606552 0 2024-03-24T10:46:09.738 INFO:tasks.workunit.client.1.smithi096.stdout:7/310: dwrite d0/db/d26/f44 [0,4194304] 0 2024-03-24T10:46:09.739 INFO:tasks.workunit.client.1.smithi096.stdout:7/311: readlink d0/d13/l40 0 2024-03-24T10:46:09.741 INFO:tasks.workunit.client.1.smithi096.stdout:9/268: mknod d9/db/c52 0 2024-03-24T10:46:09.746 INFO:tasks.workunit.client.1.smithi096.stdout:8/332: mkdir d8/d11/d22/d4c/d67 0 2024-03-24T10:46:09.751 INFO:tasks.workunit.client.1.smithi096.stdout:7/312: dwrite d0/d13/d32/f3b [4194304,4194304] 0 2024-03-24T10:46:09.756 INFO:tasks.workunit.client.1.smithi096.stdout:9/269: rename d9/f16 to d9/db/f53 0 2024-03-24T10:46:09.756 INFO:tasks.workunit.client.1.smithi096.stdout:9/270: fdatasync d9/d15/d26/d29/f42 0 2024-03-24T10:46:09.764 INFO:tasks.workunit.client.1.smithi096.stdout:9/271: creat d9/db/d23/d4f/f54 x:0 0 0 2024-03-24T10:46:09.770 INFO:tasks.workunit.client.1.smithi096.stdout:8/333: rename d8/d11/d22/d4c/d65/d1d/f38 to d8/d11/d22/d4c/f68 0 2024-03-24T10:46:09.771 INFO:tasks.workunit.client.1.smithi096.stdout:7/313: write d0/db/f1b [3049895,29951] 0 2024-03-24T10:46:09.771 INFO:tasks.workunit.client.1.smithi096.stdout:7/314: write d0/d13/f5 [487097,127114] 0 2024-03-24T10:46:09.775 INFO:tasks.workunit.client.1.smithi096.stdout:0/285: sync 2024-03-24T10:46:09.776 INFO:tasks.workunit.client.1.smithi096.stdout:1/259: sync 2024-03-24T10:46:09.777 INFO:tasks.workunit.client.1.smithi096.stdout:8/334: unlink d8/d11/d22/d4c/d65/d1a/d46/c35 0 2024-03-24T10:46:09.787 INFO:tasks.workunit.client.1.smithi096.stdout:9/272: rename d9/db/d23/d31/d3e/l40 to d9/db/l55 0 2024-03-24T10:46:09.787 INFO:tasks.workunit.client.1.smithi096.stdout:9/273: write d9/db/d23/d31/d3e/f3f [228719,99537] 0 2024-03-24T10:46:09.787 INFO:tasks.workunit.client.1.smithi096.stdout:9/274: write d9/d15/f4a [1002713,81899] 0 2024-03-24T10:46:09.793 INFO:tasks.workunit.client.1.smithi096.stdout:0/286: dwrite d0/d11/d16/d21/d25/f36 [0,4194304] 0 2024-03-24T10:46:09.794 INFO:tasks.workunit.client.1.smithi096.stdout:8/335: write d8/d11/d22/d4c/d65/d1d/f3e [861824,2234] 0 2024-03-24T10:46:09.795 INFO:tasks.workunit.client.1.smithi096.stdout:1/260: dwrite d3/f22 [4194304,4194304] 0 2024-03-24T10:46:09.812 INFO:tasks.workunit.client.1.smithi096.stdout:7/315: rmdir d0 39 2024-03-24T10:46:09.812 INFO:tasks.workunit.client.1.smithi096.stdout:7/316: write d0/d13/f3d [928490,106642] 0 2024-03-24T10:46:09.812 INFO:tasks.workunit.client.1.smithi096.stdout:7/317: chown d0/d13 460 1 2024-03-24T10:46:09.814 INFO:tasks.workunit.client.1.smithi096.stdout:9/275: truncate d9/d15/d26/f33 3132190 0 2024-03-24T10:46:09.814 INFO:tasks.workunit.client.1.smithi096.stdout:9/276: truncate d9/db/fe 2662029 0 2024-03-24T10:46:09.814 INFO:tasks.workunit.client.1.smithi096.stdout:9/277: chown d9/c1f 4967745 1 2024-03-24T10:46:09.815 INFO:tasks.workunit.client.1.smithi096.stdout:9/278: read d9/d15/f21 [466514,19664] 0 2024-03-24T10:46:09.816 INFO:tasks.workunit.client.1.smithi096.stdout:9/279: creat d9/db/d23/f56 x:0 0 0 2024-03-24T10:46:09.824 INFO:tasks.workunit.client.1.smithi096.stdout:0/287: link d0/d11/d40/f44 d0/d11/d20/d56/f5c 0 2024-03-24T10:46:09.824 INFO:tasks.workunit.client.1.smithi096.stdout:0/288: stat d0/d2 0 2024-03-24T10:46:09.824 INFO:tasks.workunit.client.1.smithi096.stdout:0/289: stat d0/d11/d40/f41 0 2024-03-24T10:46:09.824 INFO:tasks.workunit.client.1.smithi096.stdout:0/290: stat d0/d11/d20/d2b 0 2024-03-24T10:46:09.824 INFO:tasks.workunit.client.1.smithi096.stdout:0/291: write d0/d11/d40/f44 [5213560,63376] 0 2024-03-24T10:46:09.825 INFO:tasks.workunit.client.1.smithi096.stdout:0/292: chown d0/ca 7186 1 2024-03-24T10:46:09.825 INFO:tasks.workunit.client.1.smithi096.stdout:0/293: fsync d0/d11/d16/d21/f2a 0 2024-03-24T10:46:09.825 INFO:tasks.workunit.client.1.smithi096.stdout:0/294: fsync d0/d2/f4a 0 2024-03-24T10:46:09.825 INFO:tasks.workunit.client.1.smithi096.stdout:0/295: dread d0/d11/d20/d2b/f3d [0,4194304] 0 2024-03-24T10:46:09.825 INFO:tasks.workunit.client.1.smithi096.stdout:0/296: write d0/d11/f1c [1087925,13291] 0 2024-03-24T10:46:09.832 INFO:tasks.workunit.client.1.smithi096.stdout:7/318: mknod d0/d21/d3c/d68/d4b/d62/c69 0 2024-03-24T10:46:09.832 INFO:tasks.workunit.client.1.smithi096.stdout:7/319: chown d0/db/c36 90 1 2024-03-24T10:46:09.832 INFO:tasks.workunit.client.1.smithi096.stdout:7/320: stat d0/d13/d32/d45 0 2024-03-24T10:46:09.837 INFO:tasks.workunit.client.1.smithi096.stdout:7/321: write d0/f14 [2468578,31275] 0 2024-03-24T10:46:09.839 INFO:tasks.workunit.client.1.smithi096.stdout:0/297: creat d0/d11/d40/f5d x:0 0 0 2024-03-24T10:46:09.840 INFO:tasks.workunit.client.1.smithi096.stdout:8/336: rename d8/d11/d22/d4c/d65/d1d/f2f to d8/f69 0 2024-03-24T10:46:09.845 INFO:tasks.workunit.client.1.smithi096.stdout:7/322: write d0/db/d1c/f29 [3169065,26680] 0 2024-03-24T10:46:09.845 INFO:tasks.workunit.client.1.smithi096.stdout:7/323: fdatasync d0/d21/f51 0 2024-03-24T10:46:09.845 INFO:tasks.workunit.client.1.smithi096.stdout:7/324: readlink d0/l49 0 2024-03-24T10:46:09.847 INFO:tasks.workunit.client.1.smithi096.stdout:0/298: rmdir d0/d11/d20/d5a 0 2024-03-24T10:46:09.847 INFO:tasks.workunit.client.1.smithi096.stdout:0/299: fdatasync d0/f5 0 2024-03-24T10:46:09.849 INFO:tasks.workunit.client.1.smithi096.stdout:8/337: creat d8/d11/d22/d4c/d65/d1a/d49/f6a x:0 0 0 2024-03-24T10:46:09.850 INFO:tasks.workunit.client.1.smithi096.stdout:7/325: link d0/db/c16 d0/db/c6a 0 2024-03-24T10:46:09.852 INFO:tasks.workunit.client.1.smithi096.stdout:0/300: creat d0/d2/f5e x:0 0 0 2024-03-24T10:46:09.854 INFO:tasks.workunit.client.1.smithi096.stdout:9/280: truncate d9/db/fe 2356701 0 2024-03-24T10:46:09.855 INFO:tasks.workunit.client.1.smithi096.stdout:8/338: creat d8/d11/d22/d4c/d65/d37/f6b x:0 0 0 2024-03-24T10:46:09.855 INFO:tasks.workunit.client.1.smithi096.stdout:8/339: chown d8/d11/d22 1292 1 2024-03-24T10:46:09.855 INFO:tasks.workunit.client.1.smithi096.stdout:8/340: fdatasync d8/d11/f43 0 2024-03-24T10:46:09.856 INFO:tasks.workunit.client.1.smithi096.stdout:0/301: symlink d0/d11/d16/d21/d25/d47/l5f 0 2024-03-24T10:46:09.856 INFO:tasks.workunit.client.1.smithi096.stdout:0/302: chown d0/d11/d20/c19 6 1 2024-03-24T10:46:09.859 INFO:tasks.workunit.client.1.smithi096.stdout:7/326: write d0/d21/d3c/d68/f4d [2269858,109378] 0 2024-03-24T10:46:09.859 INFO:tasks.workunit.client.1.smithi096.stdout:7/327: readlink d0/l10 0 2024-03-24T10:46:09.861 INFO:tasks.workunit.client.1.smithi096.stdout:8/341: creat d8/d11/d22/d4c/d65/d1a/f6c x:0 0 0 2024-03-24T10:46:09.863 INFO:tasks.workunit.client.1.smithi096.stdout:9/281: symlink d9/db/l57 0 2024-03-24T10:46:09.865 INFO:tasks.workunit.client.1.smithi096.stdout:3/345: sync 2024-03-24T10:46:09.866 INFO:tasks.workunit.client.1.smithi096.stdout:8/342: chown d8/d11/d22/d4c/d65/d1d/l32 813 1 2024-03-24T10:46:09.866 INFO:tasks.workunit.client.1.smithi096.stdout:8/343: write d8/d11/d22/d4c/d65/d37/f39 [3419772,70930] 0 2024-03-24T10:46:09.869 INFO:tasks.workunit.client.1.smithi096.stdout:7/328: dread d0/f38 [0,4194304] 0 2024-03-24T10:46:09.869 INFO:tasks.workunit.client.1.smithi096.stdout:7/329: chown d0/db/f1b 1423217 1 2024-03-24T10:46:09.869 INFO:tasks.workunit.client.1.smithi096.stdout:7/330: readlink d0/l49 0 2024-03-24T10:46:09.869 INFO:tasks.workunit.client.1.smithi096.stdout:7/331: truncate d0/db/d26/f5b 1551793 0 2024-03-24T10:46:09.870 INFO:tasks.workunit.client.1.smithi096.stdout:9/282: link d9/db/f22 d9/db/f58 0 2024-03-24T10:46:09.870 INFO:tasks.workunit.client.1.smithi096.stdout:9/283: write d9/db/f53 [3896362,66706] 0 2024-03-24T10:46:09.875 INFO:tasks.workunit.client.1.smithi096.stdout:8/344: rename d8/d11/d22/d4c/d65/d1d/d3d/f4b to d8/d11/d22/d4c/f6d 0 2024-03-24T10:46:09.877 INFO:tasks.workunit.client.1.smithi096.stdout:3/346: write d0/d13/f57 [2725219,104428] 0 2024-03-24T10:46:09.878 INFO:tasks.workunit.client.1.smithi096.stdout:7/332: rename d0/c25 to d0/c6b 0 2024-03-24T10:46:09.878 INFO:tasks.workunit.client.1.smithi096.stdout:3/347: creat d0/d13/d45/d52/f74 x:0 0 0 2024-03-24T10:46:09.878 INFO:tasks.workunit.client.1.smithi096.stdout:3/348: chown d0/d9/d22/f4f 1 1 2024-03-24T10:46:09.878 INFO:tasks.workunit.client.1.smithi096.stdout:3/349: stat d0/d13/d45/d47 0 2024-03-24T10:46:09.878 INFO:tasks.workunit.client.1.smithi096.stdout:3/350: readlink d0/d13/d1e/l58 0 2024-03-24T10:46:09.880 INFO:tasks.workunit.client.1.smithi096.stdout:0/303: dwrite d0/d11/d16/d21/f2a [0,4194304] 0 2024-03-24T10:46:09.880 INFO:tasks.workunit.client.1.smithi096.stdout:9/284: creat d9/db/d23/d31/d3e/f59 x:0 0 0 2024-03-24T10:46:09.880 INFO:tasks.workunit.client.1.smithi096.stdout:0/304: fdatasync d0/d11/d16/f54 0 2024-03-24T10:46:09.880 INFO:tasks.workunit.client.1.smithi096.stdout:9/285: creat d9/db/d23/d31/f5a x:0 0 0 2024-03-24T10:46:09.880 INFO:tasks.workunit.client.1.smithi096.stdout:9/286: write d9/db/d23/d31/d49/f50 [4407731,68368] 0 2024-03-24T10:46:09.880 INFO:tasks.workunit.client.1.smithi096.stdout:9/287: fsync d9/db/fe 0 2024-03-24T10:46:09.884 INFO:tasks.workunit.client.1.smithi096.stdout:7/333: truncate d0/db/d4f/f58 1281939 0 2024-03-24T10:46:09.885 INFO:tasks.workunit.client.1.smithi096.stdout:7/334: chown d0/d13/d32/f3b 798 1 2024-03-24T10:46:09.888 INFO:tasks.workunit.client.1.smithi096.stdout:8/345: read d8/f15 [1558832,80961] 0 2024-03-24T10:46:09.888 INFO:tasks.workunit.client.1.smithi096.stdout:8/346: dread - d8/d11/d22/d4c/d65/d1a/d49/f6a zero size 2024-03-24T10:46:09.893 INFO:tasks.workunit.client.1.smithi096.stdout:9/288: symlink d9/db/d23/d31/d3e/l5b 0 2024-03-24T10:46:09.894 INFO:tasks.workunit.client.1.smithi096.stdout:3/351: dwrite d0/d13/d1e/f2d [0,4194304] 0 2024-03-24T10:46:09.894 INFO:tasks.workunit.client.1.smithi096.stdout:3/352: creat d0/f75 x:0 0 0 2024-03-24T10:46:09.894 INFO:tasks.workunit.client.1.smithi096.stdout:3/353: creat d0/d13/d45/d47/f76 x:0 0 0 2024-03-24T10:46:09.894 INFO:tasks.workunit.client.1.smithi096.stdout:3/354: readlink d0/d13/d1e/l58 0 2024-03-24T10:46:09.894 INFO:tasks.workunit.client.1.smithi096.stdout:3/355: stat d0/d13/d41 0 2024-03-24T10:46:09.894 INFO:tasks.workunit.client.1.smithi096.stdout:3/356: fdatasync d0/d13/d45/d47/f76 0 2024-03-24T10:46:09.894 INFO:tasks.workunit.client.1.smithi096.stdout:7/335: truncate d0/db/d26/f44 1893520 0 2024-03-24T10:46:09.898 INFO:tasks.workunit.client.1.smithi096.stdout:8/347: link d8/f14 d8/d11/d22/d4c/d65/d37/f6e 0 2024-03-24T10:46:09.902 INFO:tasks.workunit.client.1.smithi096.stdout:8/348: write d8/d11/d22/d4c/d65/d1a/d46/d2e/f54 [1133030,97703] 0 2024-03-24T10:46:09.903 INFO:tasks.workunit.client.1.smithi096.stdout:9/289: write d9/f2c [2866809,107352] 0 2024-03-24T10:46:09.912 INFO:tasks.workunit.client.1.smithi096.stdout:7/336: dwrite d0/d21/d3c/d68/f4d [4194304,4194304] 0 2024-03-24T10:46:09.917 INFO:tasks.workunit.client.1.smithi096.stdout:3/357: mkdir d0/d13/d77 0 2024-03-24T10:46:09.917 INFO:tasks.workunit.client.1.smithi096.stdout:9/290: write d9/db/d23/d31/d43/f45 [564689,24097] 0 2024-03-24T10:46:09.922 INFO:tasks.workunit.client.1.smithi096.stdout:7/337: write d0/d21/f27 [3977305,34199] 0 2024-03-24T10:46:09.922 INFO:tasks.workunit.client.1.smithi096.stdout:8/349: dwrite d8/d11/d22/d4c/d65/d37/f6b [0,4194304] 0 2024-03-24T10:46:09.922 INFO:tasks.workunit.client.1.smithi096.stdout:7/338: read d0/db/ff [667087,87098] 0 2024-03-24T10:46:09.924 INFO:tasks.workunit.client.1.smithi096.stdout:9/291: creat d9/f5c x:0 0 0 2024-03-24T10:46:09.929 INFO:tasks.workunit.client.1.smithi096.stdout:8/350: mknod d8/d11/d22/d4c/d67/c6f 0 2024-03-24T10:46:09.929 INFO:tasks.workunit.client.1.smithi096.stdout:8/351: fdatasync d8/f15 0 2024-03-24T10:46:09.932 INFO:tasks.workunit.client.1.smithi096.stdout:7/339: creat d0/d13/d32/d45/f6c x:0 0 0 2024-03-24T10:46:09.932 INFO:tasks.workunit.client.1.smithi096.stdout:7/340: chown d0/f20 20697875 1 2024-03-24T10:46:09.933 INFO:tasks.workunit.client.1.smithi096.stdout:9/292: symlink d9/db/d23/l5d 0 2024-03-24T10:46:09.933 INFO:tasks.workunit.client.1.smithi096.stdout:9/293: chown d9/db/d23/d31/d43/f44 5 1 2024-03-24T10:46:09.935 INFO:tasks.workunit.client.1.smithi096.stdout:9/294: dread d9/d15/f21 [0,4194304] 0 2024-03-24T10:46:09.935 INFO:tasks.workunit.client.1.smithi096.stdout:9/295: truncate d9/d15/f4a 1314054 0 2024-03-24T10:46:09.939 INFO:tasks.workunit.client.1.smithi096.stdout:8/352: creat d8/d11/d22/d4c/d65/d1e/f70 x:0 0 0 2024-03-24T10:46:09.943 INFO:tasks.workunit.client.1.smithi096.stdout:9/296: dread d9/d15/d26/d29/f4e [0,4194304] 0 2024-03-24T10:46:09.945 INFO:tasks.workunit.client.1.smithi096.stdout:7/341: link d0/db/ff d0/db/f6d 0 2024-03-24T10:46:09.945 INFO:tasks.workunit.client.1.smithi096.stdout:7/342: chown d0/d13/c12 246430 1 2024-03-24T10:46:09.945 INFO:tasks.workunit.client.1.smithi096.stdout:7/343: readlink d0/db/l3e 0 2024-03-24T10:46:09.947 INFO:tasks.workunit.client.1.smithi096.stdout:8/353: rename d8/d11/d22/d4c/d65/d1d/f3e to d8/d11/d22/d4c/d65/d1a/d46/d2e/f71 0 2024-03-24T10:46:09.947 INFO:tasks.workunit.client.1.smithi096.stdout:8/354: fdatasync d8/d11/d44/d45/f4d 0 2024-03-24T10:46:09.948 INFO:tasks.workunit.client.1.smithi096.stdout:8/355: dread d8/d11/d22/d4c/f6d [0,4194304] 0 2024-03-24T10:46:09.948 INFO:tasks.workunit.client.1.smithi096.stdout:8/356: dread - d8/d11/d22/d4c/d65/d1e/f5e zero size 2024-03-24T10:46:09.948 INFO:tasks.workunit.client.1.smithi096.stdout:8/357: write d8/d11/d22/d4c/d65/d1a/f2d [597047,96100] 0 2024-03-24T10:46:09.949 INFO:tasks.workunit.client.1.smithi096.stdout:8/358: dread - d8/d11/d22/d4c/d65/d1e/f5e zero size 2024-03-24T10:46:09.950 INFO:tasks.workunit.client.1.smithi096.stdout:8/359: mkdir d8/d72 0 2024-03-24T10:46:09.954 INFO:tasks.workunit.client.1.smithi096.stdout:9/297: dread d9/d15/d26/d29/f42 [0,4194304] 0 2024-03-24T10:46:09.973 INFO:tasks.workunit.client.1.smithi096.stdout:7/344: dwrite d0/db/ff [0,4194304] 0 2024-03-24T10:46:09.973 INFO:tasks.workunit.client.1.smithi096.stdout:7/345: fdatasync d0/d13/d32/d45/f6c 0 2024-03-24T10:46:09.974 INFO:tasks.workunit.client.1.smithi096.stdout:7/346: dread - d0/db/d26/d48/f67 zero size 2024-03-24T10:46:09.974 INFO:tasks.workunit.client.1.smithi096.stdout:7/347: write d0/d13/f9 [2090414,103566] 0 2024-03-24T10:46:09.974 INFO:tasks.workunit.client.1.smithi096.stdout:8/360: dwrite d8/d11/d22/d4c/d65/f5b [0,4194304] 0 2024-03-24T10:46:09.980 INFO:tasks.workunit.client.1.smithi096.stdout:8/361: mknod d8/d11/d22/d4c/d65/d52/c73 0 2024-03-24T10:46:09.982 INFO:tasks.workunit.client.1.smithi096.stdout:8/362: creat d8/d11/d22/d4c/d67/f74 x:0 0 0 2024-03-24T10:46:09.982 INFO:tasks.workunit.client.1.smithi096.stdout:8/363: chown d8/d11/d22/d4c/d65/d37/f39 539211 1 2024-03-24T10:46:09.982 INFO:tasks.workunit.client.1.smithi096.stdout:8/364: write d8/d11/d22/d4c/d65/d1a/f6c [5801,56448] 0 2024-03-24T10:46:09.983 INFO:tasks.workunit.client.1.smithi096.stdout:8/365: fsync d8/d11/d22/d4c/d65/d1d/f28 0 2024-03-24T10:46:09.984 INFO:tasks.workunit.client.1.smithi096.stdout:8/366: getdents d8/d11/d22/d4c 0 2024-03-24T10:46:09.986 INFO:tasks.workunit.client.1.smithi096.stdout:8/367: mknod d8/d11/d22/d4c/d65/d1a/c75 0 2024-03-24T10:46:09.987 INFO:tasks.workunit.client.1.smithi096.stdout:7/348: dwrite d0/d13/d32/d45/f6c [0,4194304] 0 2024-03-24T10:46:09.987 INFO:tasks.workunit.client.1.smithi096.stdout:8/368: dread d8/d11/d22/d4c/d65/d52/f57 [0,4194304] 0 2024-03-24T10:46:09.992 INFO:tasks.workunit.client.1.smithi096.stdout:7/349: creat d0/d21/d3c/d68/d4b/d62/f6e x:0 0 0 2024-03-24T10:46:09.997 INFO:tasks.workunit.client.1.smithi096.stdout:7/350: dread d0/f38 [0,4194304] 0 2024-03-24T10:46:10.002 INFO:tasks.workunit.client.1.smithi096.stdout:2/198: sync 2024-03-24T10:46:10.002 INFO:tasks.workunit.client.1.smithi096.stdout:6/275: sync 2024-03-24T10:46:10.002 INFO:tasks.workunit.client.1.smithi096.stdout:6/276: fsync d8/d12/f2d 0 2024-03-24T10:46:10.002 INFO:tasks.workunit.client.1.smithi096.stdout:2/199: chown d5/dc/d12/d23/l4e 35910034 1 2024-03-24T10:46:10.002 INFO:tasks.workunit.client.1.smithi096.stdout:6/277: chown d8/d10/d19/f48 7 1 2024-03-24T10:46:10.002 INFO:tasks.workunit.client.1.smithi096.stdout:2/200: readlink d5/dc/d12/d23/l4e 0 2024-03-24T10:46:10.002 INFO:tasks.workunit.client.1.smithi096.stdout:4/226: sync 2024-03-24T10:46:10.002 INFO:tasks.workunit.client.1.smithi096.stdout:4/227: dread - d1/d11/f41 zero size 2024-03-24T10:46:10.002 INFO:tasks.workunit.client.1.smithi096.stdout:5/273: sync 2024-03-24T10:46:10.005 INFO:tasks.workunit.client.1.smithi096.stdout:2/201: dread f3 [0,4194304] 0 2024-03-24T10:46:10.008 INFO:tasks.workunit.client.1.smithi096.stdout:8/369: rmdir d8 39 2024-03-24T10:46:10.010 INFO:tasks.workunit.client.1.smithi096.stdout:4/228: dread d1/f8 [0,4194304] 0 2024-03-24T10:46:10.010 INFO:tasks.workunit.client.1.smithi096.stdout:4/229: chown d1/d2b/l3f 485901506 1 2024-03-24T10:46:10.012 INFO:tasks.workunit.client.1.smithi096.stdout:7/351: mknod d0/db/d1c/d39/c6f 0 2024-03-24T10:46:10.012 INFO:tasks.workunit.client.1.smithi096.stdout:7/352: dread - d0/db/d26/f55 zero size 2024-03-24T10:46:10.017 INFO:tasks.workunit.client.1.smithi096.stdout:6/278: rename d8/d10/c1a to d8/d12/d32/c52 0 2024-03-24T10:46:10.017 INFO:tasks.workunit.client.1.smithi096.stdout:6/279: write d8/d12/d32/d38/f44 [729258,84096] 0 2024-03-24T10:46:10.021 INFO:tasks.workunit.client.1.smithi096.stdout:5/274: creat d2/d7/d8/df/d14/f5c x:0 0 0 2024-03-24T10:46:10.026 INFO:tasks.workunit.client.1.smithi096.stdout:4/230: creat d1/d2b/f4e x:0 0 0 2024-03-24T10:46:10.027 INFO:tasks.workunit.client.1.smithi096.stdout:7/353: symlink d0/db/l70 0 2024-03-24T10:46:10.027 INFO:tasks.workunit.client.1.smithi096.stdout:4/231: write d1/d2b/f2f [325923,58372] 0 2024-03-24T10:46:10.027 INFO:tasks.workunit.client.1.smithi096.stdout:4/232: stat d1/d32/d42/d44/d45 0 2024-03-24T10:46:10.027 INFO:tasks.workunit.client.1.smithi096.stdout:4/233: readlink d1/l21 0 2024-03-24T10:46:10.027 INFO:tasks.workunit.client.1.smithi096.stdout:4/234: chown d1/d2b 3925 1 2024-03-24T10:46:10.028 INFO:tasks.workunit.client.1.smithi096.stdout:4/235: write d1/f10 [1493473,65998] 0 2024-03-24T10:46:10.028 INFO:tasks.workunit.client.1.smithi096.stdout:6/280: link d8/d10/d22/f21 d8/d12/d28/f53 0 2024-03-24T10:46:10.032 INFO:tasks.workunit.client.1.smithi096.stdout:8/370: rename d8/d11/d22/d4c/d65/d1d/d3d/f4a to d8/d72/f76 0 2024-03-24T10:46:10.032 INFO:tasks.workunit.client.1.smithi096.stdout:5/275: creat d2/d49/f5d x:0 0 0 2024-03-24T10:46:10.032 INFO:tasks.workunit.client.1.smithi096.stdout:5/276: chown d2/d7/d8/f56 1 1 2024-03-24T10:46:10.037 INFO:tasks.workunit.client.1.smithi096.stdout:2/202: mknod d5/de/c56 0 2024-03-24T10:46:10.037 INFO:tasks.workunit.client.1.smithi096.stdout:7/354: truncate d0/d13/f35 2263610 0 2024-03-24T10:46:10.041 INFO:tasks.workunit.client.1.smithi096.stdout:4/236: mkdir d1/d2b/d40/d4f 0 2024-03-24T10:46:10.041 INFO:tasks.workunit.client.1.smithi096.stdout:6/281: mknod d8/d12/d32/c54 0 2024-03-24T10:46:10.042 INFO:tasks.workunit.client.1.smithi096.stdout:6/282: dread d8/f29 [0,4194304] 0 2024-03-24T10:46:10.042 INFO:tasks.workunit.client.1.smithi096.stdout:6/283: creat d8/d10/d13/f55 x:0 0 0 2024-03-24T10:46:10.042 INFO:tasks.workunit.client.1.smithi096.stdout:6/284: readlink d8/l50 0 2024-03-24T10:46:10.047 INFO:tasks.workunit.client.1.smithi096.stdout:8/371: rename d8/d11/d22/d4c/d65/d1e to d8/d77 0 2024-03-24T10:46:10.048 INFO:tasks.workunit.client.1.smithi096.stdout:2/203: symlink d5/d38/l57 0 2024-03-24T10:46:10.052 INFO:tasks.workunit.client.1.smithi096.stdout:7/355: creat d0/d21/d3c/f71 x:0 0 0 2024-03-24T10:46:10.052 INFO:tasks.workunit.client.1.smithi096.stdout:7/356: chown d0/d13/d32/c50 7898 1 2024-03-24T10:46:10.052 INFO:tasks.workunit.client.1.smithi096.stdout:7/357: write d0/d21/f27 [2888550,9763] 0 2024-03-24T10:46:10.056 INFO:tasks.workunit.client.1.smithi096.stdout:4/237: rmdir d1/d32/d42 39 2024-03-24T10:46:10.056 INFO:tasks.workunit.client.1.smithi096.stdout:4/238: dread - d1/d2b/d40/f46 zero size 2024-03-24T10:46:10.064 INFO:tasks.workunit.client.1.smithi096.stdout:4/239: dread d1/d11/f19 [0,4194304] 0 2024-03-24T10:46:10.064 INFO:tasks.workunit.client.1.smithi096.stdout:4/240: readlink d1/d32/l33 0 2024-03-24T10:46:10.064 INFO:tasks.workunit.client.1.smithi096.stdout:5/277: rename d2/d7/d8/d18/c35 to d2/d7/d8/c5e 0 2024-03-24T10:46:10.065 INFO:tasks.workunit.client.1.smithi096.stdout:5/278: getdents d2/d29/d32/d33/d37 0 2024-03-24T10:46:10.065 INFO:tasks.workunit.client.1.smithi096.stdout:8/372: symlink d8/d11/d22/d4c/d65/d1a/d46/l78 0 2024-03-24T10:46:10.068 INFO:tasks.workunit.client.1.smithi096.stdout:2/204: creat d5/dc/f58 x:0 0 0 2024-03-24T10:46:10.070 INFO:tasks.workunit.client.1.smithi096.stdout:7/358: creat d0/db/f72 x:0 0 0 2024-03-24T10:46:10.074 INFO:tasks.workunit.client.1.smithi096.stdout:6/285: rename d8/l50 to d8/d12/d32/d38/l56 0 2024-03-24T10:46:10.076 INFO:tasks.workunit.client.1.smithi096.stdout:5/279: mkdir d2/d29/d32/d3f/d5f 0 2024-03-24T10:46:10.076 INFO:tasks.workunit.client.1.smithi096.stdout:5/280: chown d2/d29/d32/d3f/d5f 5711 1 2024-03-24T10:46:10.076 INFO:tasks.workunit.client.1.smithi096.stdout:5/281: fdatasync d2/d7/d8/f56 0 2024-03-24T10:46:10.077 INFO:tasks.workunit.client.1.smithi096.stdout:8/373: symlink d8/d11/d44/d45/l79 0 2024-03-24T10:46:10.080 INFO:tasks.workunit.client.1.smithi096.stdout:2/205: creat d5/de/f59 x:0 0 0 2024-03-24T10:46:10.081 INFO:tasks.workunit.client.1.smithi096.stdout:2/206: creat d5/de/d2a/f5a x:0 0 0 2024-03-24T10:46:10.081 INFO:tasks.workunit.client.1.smithi096.stdout:2/207: read - d5/de/d2a/f4c zero size 2024-03-24T10:46:10.088 INFO:tasks.workunit.client.1.smithi096.stdout:7/359: mkdir d0/db/d73 0 2024-03-24T10:46:10.088 INFO:tasks.workunit.client.1.smithi096.stdout:7/360: truncate d0/d13/d32/d45/f63 731150 0 2024-03-24T10:46:10.094 INFO:tasks.workunit.client.1.smithi096.stdout:5/282: dwrite d2/d7/f52 [0,4194304] 0 2024-03-24T10:46:10.098 INFO:tasks.workunit.client.1.smithi096.stdout:7/361: write d0/db/ff [301529,110126] 0 2024-03-24T10:46:10.100 INFO:tasks.workunit.client.1.smithi096.stdout:5/283: write d2/d7/f41 [2132936,22621] 0 2024-03-24T10:46:10.100 INFO:tasks.workunit.client.1.smithi096.stdout:4/241: rmdir d1 39 2024-03-24T10:46:10.100 INFO:tasks.workunit.client.1.smithi096.stdout:4/242: write d1/d11/f41 [762637,126723] 0 2024-03-24T10:46:10.101 INFO:tasks.workunit.client.1.smithi096.stdout:6/286: symlink d8/d10/l57 0 2024-03-24T10:46:10.102 INFO:tasks.workunit.client.1.smithi096.stdout:6/287: fsync d8/f4a 0 2024-03-24T10:46:10.102 INFO:tasks.workunit.client.1.smithi096.stdout:6/288: stat d8/d10/d13 0 2024-03-24T10:46:10.102 INFO:tasks.workunit.client.1.smithi096.stdout:6/289: fsync d8/d12/d32/f3c 0 2024-03-24T10:46:10.102 INFO:tasks.workunit.client.1.smithi096.stdout:6/290: chown d8/d10/d22/f2c 383445 1 2024-03-24T10:46:10.102 INFO:tasks.workunit.client.1.smithi096.stdout:8/374: creat d8/d11/d22/d4c/d67/f7a x:0 0 0 2024-03-24T10:46:10.103 INFO:tasks.workunit.client.1.smithi096.stdout:8/375: fsync d8/d11/d22/d4c/d65/d1d/f28 0 2024-03-24T10:46:10.106 INFO:tasks.workunit.client.1.smithi096.stdout:2/208: dwrite d5/de/f59 [0,4194304] 0 2024-03-24T10:46:10.107 INFO:tasks.workunit.client.1.smithi096.stdout:2/209: dread d5/dc/f34 [0,4194304] 0 2024-03-24T10:46:10.108 INFO:tasks.workunit.client.1.smithi096.stdout:2/210: write d5/d38/f45 [650818,62907] 0 2024-03-24T10:46:10.108 INFO:tasks.workunit.client.1.smithi096.stdout:4/243: link d1/f1c d1/d32/d4c/f50 0 2024-03-24T10:46:10.110 INFO:tasks.workunit.client.1.smithi096.stdout:6/291: mkdir d8/d10/d58 0 2024-03-24T10:46:10.111 INFO:tasks.workunit.client.1.smithi096.stdout:8/376: creat d8/d11/d22/d4c/d65/d1d/f7b x:0 0 0 2024-03-24T10:46:10.117 INFO:tasks.workunit.client.1.smithi096.stdout:5/284: symlink d2/d7/d8/df/d14/l60 0 2024-03-24T10:46:10.119 INFO:tasks.workunit.client.1.smithi096.stdout:7/362: dwrite d0/d21/f51 [0,4194304] 0 2024-03-24T10:46:10.119 INFO:tasks.workunit.client.1.smithi096.stdout:6/292: write d8/f31 [283095,10861] 0 2024-03-24T10:46:10.120 INFO:tasks.workunit.client.1.smithi096.stdout:2/211: link d5/de/d2a/f37 d5/dc/f5b 0 2024-03-24T10:46:10.121 INFO:tasks.workunit.client.1.smithi096.stdout:8/377: stat d8/d77/l34 0 2024-03-24T10:46:10.123 INFO:tasks.workunit.client.1.smithi096.stdout:7/363: creat d0/db/d4f/f74 x:0 0 0 2024-03-24T10:46:10.124 INFO:tasks.workunit.client.1.smithi096.stdout:6/293: mkdir d8/d12/d32/d38/d59 0 2024-03-24T10:46:10.127 INFO:tasks.workunit.client.1.smithi096.stdout:2/212: symlink d5/d38/l5c 0 2024-03-24T10:46:10.127 INFO:tasks.workunit.client.1.smithi096.stdout:4/244: dwrite d1/d2b/f4e [0,4194304] 0 2024-03-24T10:46:10.128 INFO:tasks.workunit.client.1.smithi096.stdout:8/378: symlink d8/d11/l7c 0 2024-03-24T10:46:10.130 INFO:tasks.workunit.client.1.smithi096.stdout:4/245: dread d1/d2b/f2f [0,4194304] 0 2024-03-24T10:46:10.135 INFO:tasks.workunit.client.1.smithi096.stdout:6/294: creat d8/d12/d32/d38/d59/f5a x:0 0 0 2024-03-24T10:46:10.139 INFO:tasks.workunit.client.1.smithi096.stdout:2/213: getdents d5/d38 0 2024-03-24T10:46:10.146 INFO:tasks.workunit.client.1.smithi096.stdout:8/379: unlink d8/d11/l42 0 2024-03-24T10:46:10.146 INFO:tasks.workunit.client.1.smithi096.stdout:8/380: truncate d8/d11/d22/d4c/d67/f7a 442925 0 2024-03-24T10:46:10.148 INFO:tasks.workunit.client.1.smithi096.stdout:4/246: link d1/f24 d1/f51 0 2024-03-24T10:46:10.149 INFO:tasks.workunit.client.1.smithi096.stdout:4/247: read d1/d11/f23 [303331,61384] 0 2024-03-24T10:46:10.149 INFO:tasks.workunit.client.1.smithi096.stdout:7/364: rmdir d0/db/d4f 39 2024-03-24T10:46:10.151 INFO:tasks.workunit.client.1.smithi096.stdout:6/295: unlink d8/d10/f4f 0 2024-03-24T10:46:10.151 INFO:tasks.workunit.client.1.smithi096.stdout:6/296: write d8/d10/d13/f2b [961321,103791] 0 2024-03-24T10:46:10.151 INFO:tasks.workunit.client.1.smithi096.stdout:6/297: dread - d8/d10/d22/f2c zero size 2024-03-24T10:46:10.152 INFO:tasks.workunit.client.1.smithi096.stdout:2/214: symlink d5/dc/l5d 0 2024-03-24T10:46:10.154 INFO:tasks.workunit.client.1.smithi096.stdout:7/365: rename d0/d13/f18 to d0/db/f75 0 2024-03-24T10:46:10.154 INFO:tasks.workunit.client.1.smithi096.stdout:7/366: truncate d0/db/d1c/f61 626471 0 2024-03-24T10:46:10.157 INFO:tasks.workunit.client.1.smithi096.stdout:2/215: mkdir d5/de/d2a/d5e 0 2024-03-24T10:46:10.159 INFO:tasks.workunit.client.1.smithi096.stdout:2/216: rmdir d5/dc/d12/d23 39 2024-03-24T10:46:10.161 INFO:tasks.workunit.client.1.smithi096.stdout:2/217: creat d5/f5f x:0 0 0 2024-03-24T10:46:10.161 INFO:tasks.workunit.client.1.smithi096.stdout:6/298: write d8/d10/d19/f34 [3753052,10032] 0 2024-03-24T10:46:10.163 INFO:tasks.workunit.client.1.smithi096.stdout:2/218: mknod d5/de/d2a/d5e/c60 0 2024-03-24T10:46:10.168 INFO:tasks.workunit.client.1.smithi096.stdout:4/248: dwrite d1/f1c [0,4194304] 0 2024-03-24T10:46:10.170 INFO:tasks.workunit.client.1.smithi096.stdout:2/219: creat d5/dc/d12/d23/d4f/f61 x:0 0 0 2024-03-24T10:46:10.170 INFO:tasks.workunit.client.1.smithi096.stdout:2/220: creat d5/de/d2a/f62 x:0 0 0 2024-03-24T10:46:10.171 INFO:tasks.workunit.client.1.smithi096.stdout:2/221: read d5/de/f13 [713000,15175] 0 2024-03-24T10:46:10.177 INFO:tasks.workunit.client.1.smithi096.stdout:4/249: unlink d1/la 0 2024-03-24T10:46:10.181 INFO:tasks.workunit.client.1.smithi096.stdout:4/250: creat d1/d32/d42/d44/d45/f52 x:0 0 0 2024-03-24T10:46:10.182 INFO:tasks.workunit.client.1.smithi096.stdout:7/367: dwrite d0/d13/f3d [0,4194304] 0 2024-03-24T10:46:10.183 INFO:tasks.workunit.client.1.smithi096.stdout:6/299: dwrite d8/f4a [0,4194304] 0 2024-03-24T10:46:10.185 INFO:tasks.workunit.client.1.smithi096.stdout:4/251: dread d1/f10 [0,4194304] 0 2024-03-24T10:46:10.187 INFO:tasks.workunit.client.1.smithi096.stdout:7/368: creat d0/db/d1c/d39/f76 x:0 0 0 2024-03-24T10:46:10.189 INFO:tasks.workunit.client.1.smithi096.stdout:4/252: dread d1/d11/f19 [0,4194304] 0 2024-03-24T10:46:10.190 INFO:tasks.workunit.client.1.smithi096.stdout:6/300: dread d8/d12/f2d [0,4194304] 0 2024-03-24T10:46:10.190 INFO:tasks.workunit.client.1.smithi096.stdout:6/301: write d8/d12/d28/f33 [455772,128671] 0 2024-03-24T10:46:10.190 INFO:tasks.workunit.client.1.smithi096.stdout:6/302: stat d8/d12/f2d 0 2024-03-24T10:46:10.190 INFO:tasks.workunit.client.1.smithi096.stdout:6/303: truncate d8/d10/d19/f1b 4612832 0 2024-03-24T10:46:10.192 INFO:tasks.workunit.client.1.smithi096.stdout:7/369: mkdir d0/db/d1c/d77 0 2024-03-24T10:46:10.192 INFO:tasks.workunit.client.1.smithi096.stdout:7/370: chown d0/db/d73 5966 1 2024-03-24T10:46:10.192 INFO:tasks.workunit.client.1.smithi096.stdout:7/371: readlink d0/db/d1c/d39/l53 0 2024-03-24T10:46:10.194 INFO:tasks.workunit.client.1.smithi096.stdout:4/253: rmdir d1/d2b/d40/d4f 0 2024-03-24T10:46:10.195 INFO:tasks.workunit.client.1.smithi096.stdout:6/304: creat d8/d12/d28/d36/f5b x:0 0 0 2024-03-24T10:46:10.197 INFO:tasks.workunit.client.1.smithi096.stdout:7/372: dread d0/f38 [0,4194304] 0 2024-03-24T10:46:10.197 INFO:tasks.workunit.client.1.smithi096.stdout:7/373: chown d0/db/d26/f55 9828 1 2024-03-24T10:46:10.197 INFO:tasks.workunit.client.1.smithi096.stdout:7/374: fsync d0/d13/d32/f3b 0 2024-03-24T10:46:10.199 INFO:tasks.workunit.client.1.smithi096.stdout:4/254: rename d1/l20 to d1/d2b/l53 0 2024-03-24T10:46:10.201 INFO:tasks.workunit.client.1.smithi096.stdout:7/375: symlink d0/db/d73/l78 0 2024-03-24T10:46:10.202 INFO:tasks.workunit.client.1.smithi096.stdout:7/376: chown d0/d13/c12 390 1 2024-03-24T10:46:10.207 INFO:tasks.workunit.client.1.smithi096.stdout:7/377: write d0/db/d4f/f58 [178269,91844] 0 2024-03-24T10:46:10.209 INFO:tasks.workunit.client.1.smithi096.stdout:7/378: creat d0/d21/d59/f79 x:0 0 0 2024-03-24T10:46:10.209 INFO:tasks.workunit.client.1.smithi096.stdout:7/379: dread - d0/db/d26/d48/f67 zero size 2024-03-24T10:46:10.211 INFO:tasks.workunit.client.1.smithi096.stdout:7/380: unlink d0/db/d1c/l54 0 2024-03-24T10:46:10.215 INFO:tasks.workunit.client.1.smithi096.stdout:7/381: unlink d0/db/c36 0 2024-03-24T10:46:10.216 INFO:tasks.workunit.client.1.smithi096.stdout:7/382: creat d0/db/d1c/d77/f7a x:0 0 0 2024-03-24T10:46:10.217 INFO:tasks.workunit.client.1.smithi096.stdout:7/383: symlink d0/db/d4f/l7b 0 2024-03-24T10:46:10.219 INFO:tasks.workunit.client.1.smithi096.stdout:7/384: symlink d0/d21/l7c 0 2024-03-24T10:46:10.219 INFO:tasks.workunit.client.1.smithi096.stdout:7/385: readlink d0/d21/d3c/d68/l4c 0 2024-03-24T10:46:10.219 INFO:tasks.workunit.client.1.smithi096.stdout:7/386: creat d0/d21/d59/f7d x:0 0 0 2024-03-24T10:46:10.219 INFO:tasks.workunit.client.1.smithi096.stdout:6/305: dwrite d8/d10/f3d [0,4194304] 0 2024-03-24T10:46:10.219 INFO:tasks.workunit.client.1.smithi096.stdout:4/255: dwrite d1/ff [4194304,4194304] 0 2024-03-24T10:46:10.221 INFO:tasks.workunit.client.1.smithi096.stdout:7/387: mkdir d0/d21/d3c/d68/d7e 0 2024-03-24T10:46:10.221 INFO:tasks.workunit.client.1.smithi096.stdout:7/388: fdatasync d0/db/d1c/d39/f76 0 2024-03-24T10:46:10.223 INFO:tasks.workunit.client.1.smithi096.stdout:6/306: creat d8/d12/f5c x:0 0 0 2024-03-24T10:46:10.224 INFO:tasks.workunit.client.1.smithi096.stdout:7/389: link d0/d13/d32/c50 d0/d13/d60/c7f 0 2024-03-24T10:46:10.224 INFO:tasks.workunit.client.1.smithi096.stdout:6/307: symlink d8/d10/d22/l5d 0 2024-03-24T10:46:10.225 INFO:tasks.workunit.client.1.smithi096.stdout:6/308: unlink d8/d10/d19/c49 0 2024-03-24T10:46:10.226 INFO:tasks.workunit.client.1.smithi096.stdout:6/309: read d8/f29 [109655,72502] 0 2024-03-24T10:46:10.226 INFO:tasks.workunit.client.1.smithi096.stdout:6/310: write d8/d10/d19/f1b [5246933,84873] 0 2024-03-24T10:46:10.228 INFO:tasks.workunit.client.1.smithi096.stdout:7/390: dread d0/db/d1c/f29 [0,4194304] 0 2024-03-24T10:46:10.229 INFO:tasks.workunit.client.1.smithi096.stdout:4/256: dread d1/f1c [0,4194304] 0 2024-03-24T10:46:10.230 INFO:tasks.workunit.client.1.smithi096.stdout:7/391: mkdir d0/d21/d3c/d80 0 2024-03-24T10:46:10.230 INFO:tasks.workunit.client.1.smithi096.stdout:6/311: dread d8/d10/d19/f48 [0,4194304] 0 2024-03-24T10:46:10.230 INFO:tasks.workunit.client.1.smithi096.stdout:6/312: write d8/d12/d32/f3c [1390772,68260] 0 2024-03-24T10:46:10.230 INFO:tasks.workunit.client.1.smithi096.stdout:6/313: chown d8 20 1 2024-03-24T10:46:10.230 INFO:tasks.workunit.client.1.smithi096.stdout:6/314: chown d8/d10/l57 265410735 1 2024-03-24T10:46:10.231 INFO:tasks.workunit.client.1.smithi096.stdout:4/257: symlink d1/d2b/d40/l54 0 2024-03-24T10:46:10.233 INFO:tasks.workunit.client.1.smithi096.stdout:7/392: mknod d0/d13/d32/c81 0 2024-03-24T10:46:10.236 INFO:tasks.workunit.client.1.smithi096.stdout:6/315: creat d8/d10/d22/f5e x:0 0 0 2024-03-24T10:46:10.237 INFO:tasks.workunit.client.1.smithi096.stdout:1/261: sync 2024-03-24T10:46:10.237 INFO:tasks.workunit.client.1.smithi096.stdout:1/262: getdents d3/d24/d5c 0 2024-03-24T10:46:10.237 INFO:tasks.workunit.client.1.smithi096.stdout:0/305: sync 2024-03-24T10:46:10.237 INFO:tasks.workunit.client.1.smithi096.stdout:0/306: readlink d0/d11/l1a 0 2024-03-24T10:46:10.237 INFO:tasks.workunit.client.1.smithi096.stdout:0/307: chown d0/d2/l59 326 1 2024-03-24T10:46:10.237 INFO:tasks.workunit.client.1.smithi096.stdout:9/298: sync 2024-03-24T10:46:10.237 INFO:tasks.workunit.client.1.smithi096.stdout:1/263: read d3/d36/f50 [2162933,113871] 0 2024-03-24T10:46:10.237 INFO:tasks.workunit.client.1.smithi096.stdout:1/264: chown d3/c4f 18 1 2024-03-24T10:46:10.237 INFO:tasks.workunit.client.1.smithi096.stdout:1/265: read - d3/d23/d2f/f5a zero size 2024-03-24T10:46:10.237 INFO:tasks.workunit.client.1.smithi096.stdout:3/358: sync 2024-03-24T10:46:10.237 INFO:tasks.workunit.client.1.smithi096.stdout:1/266: write d3/d23/d2f/f5a [333341,120297] 0 2024-03-24T10:46:10.237 INFO:tasks.workunit.client.1.smithi096.stdout:3/359: creat d0/d13/d45/d52/f78 x:0 0 0 2024-03-24T10:46:10.238 INFO:tasks.workunit.client.1.smithi096.stdout:7/393: getdents d0/db/d26/d48 0 2024-03-24T10:46:10.238 INFO:tasks.workunit.client.1.smithi096.stdout:7/394: dread - d0/db/d26/f55 zero size 2024-03-24T10:46:10.240 INFO:tasks.workunit.client.1.smithi096.stdout:6/316: unlink d8/d10/d19/l40 0 2024-03-24T10:46:10.243 INFO:tasks.workunit.client.1.smithi096.stdout:0/308: creat d0/f60 x:0 0 0 2024-03-24T10:46:10.246 INFO:tasks.workunit.client.1.smithi096.stdout:1/267: mknod d3/d24/c5d 0 2024-03-24T10:46:10.246 INFO:tasks.workunit.client.1.smithi096.stdout:9/299: write d9/f3a [1078888,58669] 0 2024-03-24T10:46:10.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:09 smithi094 ceph-mon[29446]: pgmap v248: 65 pgs: 65 active+clean; 2.4 GiB data, 8.5 GiB used, 528 GiB / 536 GiB avail; 211 MiB/s rd, 258 MiB/s wr, 374 op/s 2024-03-24T10:46:10.250 INFO:tasks.workunit.client.1.smithi096.stdout:7/395: rename d0/d21/d3c/d68/d4b/d62/f6e to d0/d21/d3c/f82 0 2024-03-24T10:46:10.251 INFO:tasks.workunit.client.1.smithi096.stdout:3/360: dread d0/d9/f49 [0,4194304] 0 2024-03-24T10:46:10.251 INFO:tasks.workunit.client.1.smithi096.stdout:4/258: dwrite d1/f1f [0,4194304] 0 2024-03-24T10:46:10.256 INFO:tasks.workunit.client.1.smithi096.stdout:3/361: dread d0/f5c [4194304,4194304] 0 2024-03-24T10:46:10.257 INFO:tasks.workunit.client.1.smithi096.stdout:4/259: dread d1/f10 [0,4194304] 0 2024-03-24T10:46:10.257 INFO:tasks.workunit.client.1.smithi096.stdout:9/300: mkdir d9/db/d23/d31/d5e 0 2024-03-24T10:46:10.257 INFO:tasks.workunit.client.1.smithi096.stdout:9/301: write d9/d15/f1e [2242765,106574] 0 2024-03-24T10:46:10.257 INFO:tasks.workunit.client.1.smithi096.stdout:9/302: write d9/db/d23/d31/d3e/f59 [757093,3679] 0 2024-03-24T10:46:10.258 INFO:tasks.workunit.client.1.smithi096.stdout:9/303: write d9/f2c [3651136,75135] 0 2024-03-24T10:46:10.258 INFO:tasks.workunit.client.1.smithi096.stdout:9/304: fsync d9/db/d23/d31/f4b 0 2024-03-24T10:46:10.260 INFO:tasks.workunit.client.1.smithi096.stdout:3/362: mknod d0/d13/d77/c79 0 2024-03-24T10:46:10.261 INFO:tasks.workunit.client.1.smithi096.stdout:3/363: fdatasync d0/f1 0 2024-03-24T10:46:10.261 INFO:tasks.workunit.client.1.smithi096.stdout:7/396: write d0/d13/d32/f3b [4747079,118276] 0 2024-03-24T10:46:10.261 INFO:tasks.workunit.client.1.smithi096.stdout:7/397: fdatasync d0/d13/d32/d45/f63 0 2024-03-24T10:46:10.261 INFO:tasks.workunit.client.1.smithi096.stdout:7/398: creat d0/d21/d59/f83 x:0 0 0 2024-03-24T10:46:10.262 INFO:tasks.workunit.client.1.smithi096.stdout:4/260: symlink d1/d32/d4c/l55 0 2024-03-24T10:46:10.265 INFO:tasks.workunit.client.1.smithi096.stdout:9/305: symlink d9/db/d23/d31/d49/l5f 0 2024-03-24T10:46:10.266 INFO:tasks.workunit.client.1.smithi096.stdout:3/364: creat d0/d13/d77/f7a x:0 0 0 2024-03-24T10:46:10.267 INFO:tasks.workunit.client.1.smithi096.stdout:3/365: read - d0/d13/d45/d47/f76 zero size 2024-03-24T10:46:10.270 INFO:tasks.workunit.client.1.smithi096.stdout:6/317: dwrite d8/d10/d13/f2b [0,4194304] 0 2024-03-24T10:46:10.270 INFO:tasks.workunit.client.1.smithi096.stdout:1/268: dwrite d3/d23/f38 [0,4194304] 0 2024-03-24T10:46:10.271 INFO:tasks.workunit.client.1.smithi096.stdout:7/399: link d0/d13/d32/c81 d0/db/d4f/c84 0 2024-03-24T10:46:10.271 INFO:tasks.workunit.client.1.smithi096.stdout:7/400: write d0/db/d26/f5b [737044,112106] 0 2024-03-24T10:46:10.271 INFO:tasks.workunit.client.1.smithi096.stdout:4/261: getdents d1/d2b/d40 0 2024-03-24T10:46:10.273 INFO:tasks.workunit.client.1.smithi096.stdout:7/401: dread d0/db/f75 [0,4194304] 0 2024-03-24T10:46:10.277 INFO:tasks.workunit.client.1.smithi096.stdout:9/306: rmdir d9/db/d23/d31/d5e 0 2024-03-24T10:46:10.279 INFO:tasks.workunit.client.1.smithi096.stdout:3/366: mkdir d0/d13/d41/d7b 0 2024-03-24T10:46:10.283 INFO:tasks.workunit.client.1.smithi096.stdout:9/307: dread d9/d15/d26/d29/f42 [0,4194304] 0 2024-03-24T10:46:10.283 INFO:tasks.workunit.client.1.smithi096.stdout:9/308: write d9/db/d23/d31/d43/f44 [699073,74958] 0 2024-03-24T10:46:10.283 INFO:tasks.workunit.client.1.smithi096.stdout:9/309: truncate d9/d15/d26/d29/f36 4278136 0 2024-03-24T10:46:10.283 INFO:tasks.workunit.client.1.smithi096.stdout:9/310: dread - d9/db/d23/d31/d49/f4c zero size 2024-03-24T10:46:10.283 INFO:tasks.workunit.client.1.smithi096.stdout:9/311: stat d9/db/l57 0 2024-03-24T10:46:10.286 INFO:tasks.workunit.client.1.smithi096.stdout:1/269: symlink d3/d23/d2f/l5e 0 2024-03-24T10:46:10.286 INFO:tasks.workunit.client.1.smithi096.stdout:1/270: getdents d3/d24/d5c 0 2024-03-24T10:46:10.287 INFO:tasks.workunit.client.1.smithi096.stdout:1/271: fsync d3/d23/f56 0 2024-03-24T10:46:10.287 INFO:tasks.workunit.client.1.smithi096.stdout:4/262: getdents d1/d2b 0 2024-03-24T10:46:10.287 INFO:tasks.workunit.client.1.smithi096.stdout:4/263: chown d1/d2b/l3f 84203285 1 2024-03-24T10:46:10.287 INFO:tasks.workunit.client.1.smithi096.stdout:4/264: chown d1/d32/l33 132491521 1 2024-03-24T10:46:10.289 INFO:tasks.workunit.client.1.smithi096.stdout:7/402: mkdir d0/d21/d3c/d68/d85 0 2024-03-24T10:46:10.290 INFO:tasks.workunit.client.1.smithi096.stdout:7/403: dread - d0/db/d26/f55 zero size 2024-03-24T10:46:10.293 INFO:tasks.workunit.client.1.smithi096.stdout:6/318: getdents d8/d10/d19 0 2024-03-24T10:46:10.293 INFO:tasks.workunit.client.1.smithi096.stdout:6/319: write d8/f1c [2497177,102262] 0 2024-03-24T10:46:10.295 INFO:tasks.workunit.client.1.smithi096.stdout:9/312: mkdir d9/d60 0 2024-03-24T10:46:10.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:09 smithi096 ceph-mon[38097]: pgmap v248: 65 pgs: 65 active+clean; 2.4 GiB data, 8.5 GiB used, 528 GiB / 536 GiB avail; 211 MiB/s rd, 258 MiB/s wr, 374 op/s 2024-03-24T10:46:10.339 INFO:tasks.workunit.client.1.smithi096.stdout:1/272: rename d3/d23/f45 to d3/d36/f5f 0 2024-03-24T10:46:10.339 INFO:tasks.workunit.client.1.smithi096.stdout:1/273: creat d3/d23/d2f/f60 x:0 0 0 2024-03-24T10:46:10.342 INFO:tasks.workunit.client.1.smithi096.stdout:4/265: unlink d1/d2b/c34 0 2024-03-24T10:46:10.346 INFO:tasks.workunit.client.1.smithi096.stdout:6/320: truncate d8/d12/f2d 3222050 0 2024-03-24T10:46:10.347 INFO:tasks.workunit.client.1.smithi096.stdout:3/367: dwrite d0/d13/d45/d52/f5b [0,4194304] 0 2024-03-24T10:46:10.351 INFO:tasks.workunit.client.1.smithi096.stdout:9/313: symlink d9/db/d23/d31/l61 0 2024-03-24T10:46:10.354 INFO:tasks.workunit.client.1.smithi096.stdout:4/266: dread d1/f24 [0,4194304] 0 2024-03-24T10:46:10.354 INFO:tasks.workunit.client.1.smithi096.stdout:4/267: write d1/f3e [106889,42931] 0 2024-03-24T10:46:10.357 INFO:tasks.workunit.client.1.smithi096.stdout:6/321: write d8/d10/d13/f2b [30585,68381] 0 2024-03-24T10:46:10.358 INFO:tasks.workunit.client.1.smithi096.stdout:9/314: dread d9/d15/d26/d29/f36 [0,4194304] 0 2024-03-24T10:46:10.358 INFO:tasks.workunit.client.1.smithi096.stdout:9/315: truncate d9/d15/f21 1210955 0 2024-03-24T10:46:10.366 INFO:tasks.workunit.client.1.smithi096.stdout:1/274: dwrite d3/f15 [4194304,4194304] 0 2024-03-24T10:46:10.372 INFO:tasks.workunit.client.1.smithi096.stdout:3/368: mknod d0/d13/c7c 0 2024-03-24T10:46:10.374 INFO:tasks.workunit.client.1.smithi096.stdout:1/275: dread d3/f34 [0,4194304] 0 2024-03-24T10:46:10.374 INFO:tasks.workunit.client.1.smithi096.stdout:4/268: mknod d1/d32/d42/c56 0 2024-03-24T10:46:10.376 INFO:tasks.workunit.client.1.smithi096.stdout:6/322: truncate d8/d10/d19/f30 1528436 0 2024-03-24T10:46:10.378 INFO:tasks.workunit.client.1.smithi096.stdout:9/316: symlink d9/d15/d26/l62 0 2024-03-24T10:46:10.379 INFO:tasks.workunit.client.1.smithi096.stdout:9/317: write d9/db/fe [2857594,58486] 0 2024-03-24T10:46:10.379 INFO:tasks.workunit.client.1.smithi096.stdout:9/318: fdatasync d9/db/d23/d31/f5a 0 2024-03-24T10:46:10.379 INFO:tasks.workunit.client.1.smithi096.stdout:9/319: chown d9/d15 10849501 1 2024-03-24T10:46:10.379 INFO:tasks.workunit.client.1.smithi096.stdout:4/269: dread d1/f8 [0,4194304] 0 2024-03-24T10:46:10.385 INFO:tasks.workunit.client.1.smithi096.stdout:3/369: rename d0/d9/d34 to d0/d13/d45/d7d 0 2024-03-24T10:46:10.389 INFO:tasks.workunit.client.1.smithi096.stdout:1/276: link d3/f1f d3/d24/d5c/f61 0 2024-03-24T10:46:10.389 INFO:tasks.workunit.client.1.smithi096.stdout:1/277: write d3/d36/f4d [94700,53147] 0 2024-03-24T10:46:10.392 INFO:tasks.workunit.client.1.smithi096.stdout:4/270: creat d1/d32/d42/d44/f57 x:0 0 0 2024-03-24T10:46:10.392 INFO:tasks.workunit.client.1.smithi096.stdout:4/271: fsync d1/d11/f12 0 2024-03-24T10:46:10.392 INFO:tasks.workunit.client.1.smithi096.stdout:4/272: read d1/f51 [4663310,88580] 0 2024-03-24T10:46:10.394 INFO:tasks.workunit.client.1.smithi096.stdout:9/320: dwrite d9/db/f58 [0,4194304] 0 2024-03-24T10:46:10.394 INFO:tasks.workunit.client.1.smithi096.stdout:9/321: write d9/d15/d26/d29/f36 [2540208,107718] 0 2024-03-24T10:46:10.396 INFO:tasks.workunit.client.1.smithi096.stdout:6/323: rename d8/d12/d28/d36/f5b to d8/d12/d32/d38/d59/f5f 0 2024-03-24T10:46:10.399 INFO:tasks.workunit.client.1.smithi096.stdout:3/370: mkdir d0/d13/d45/d47/d7e 0 2024-03-24T10:46:10.399 INFO:tasks.workunit.client.1.smithi096.stdout:3/371: getdents d0/d13/d45/d7d/d71 0 2024-03-24T10:46:10.408 INFO:tasks.workunit.client.1.smithi096.stdout:1/278: link d3/d23/d2f/f3b d3/d23/f62 0 2024-03-24T10:46:10.412 INFO:tasks.workunit.client.1.smithi096.stdout:1/279: dread d3/f34 [0,4194304] 0 2024-03-24T10:46:10.415 INFO:tasks.workunit.client.1.smithi096.stdout:9/322: unlink d9/d15/f47 0 2024-03-24T10:46:10.417 INFO:tasks.workunit.client.1.smithi096.stdout:6/324: creat d8/d10/d19/f60 x:0 0 0 2024-03-24T10:46:10.417 INFO:tasks.workunit.client.1.smithi096.stdout:6/325: fsync d8/d12/d32/d38/d59/f5a 0 2024-03-24T10:46:10.418 INFO:tasks.workunit.client.1.smithi096.stdout:3/372: dwrite d0/d13/f21 [0,4194304] 0 2024-03-24T10:46:10.418 INFO:tasks.workunit.client.1.smithi096.stdout:3/373: dread - d0/d9/d22/f4f zero size 2024-03-24T10:46:10.419 INFO:tasks.workunit.client.1.smithi096.stdout:4/273: mknod d1/d2b/c58 0 2024-03-24T10:46:10.419 INFO:tasks.workunit.client.1.smithi096.stdout:1/280: symlink d3/d23/l63 0 2024-03-24T10:46:10.419 INFO:tasks.workunit.client.1.smithi096.stdout:1/281: readlink d3/d23/d2f/l40 0 2024-03-24T10:46:10.420 INFO:tasks.workunit.client.1.smithi096.stdout:1/282: write d3/d24/f35 [111195,25063] 0 2024-03-24T10:46:10.423 INFO:tasks.workunit.client.1.smithi096.stdout:9/323: rename d9/f2c to d9/db/d23/d4f/f63 0 2024-03-24T10:46:10.423 INFO:tasks.workunit.client.1.smithi096.stdout:9/324: fsync d9/f13 0 2024-03-24T10:46:10.423 INFO:tasks.workunit.client.1.smithi096.stdout:9/325: readlink d9/db/d23/d31/d3e/l5b 0 2024-03-24T10:46:10.424 INFO:tasks.workunit.client.1.smithi096.stdout:6/326: mkdir d8/d12/d61 0 2024-03-24T10:46:10.424 INFO:tasks.workunit.client.1.smithi096.stdout:6/327: write d8/d10/d22/f5e [260442,96846] 0 2024-03-24T10:46:10.425 INFO:tasks.workunit.client.1.smithi096.stdout:9/326: dread d9/db/f28 [0,4194304] 0 2024-03-24T10:46:10.425 INFO:tasks.workunit.client.1.smithi096.stdout:9/327: write d9/d15/f21 [599996,25423] 0 2024-03-24T10:46:10.425 INFO:tasks.workunit.client.1.smithi096.stdout:9/328: dread - d9/db/d23/f56 zero size 2024-03-24T10:46:10.426 INFO:tasks.workunit.client.1.smithi096.stdout:3/374: mkdir d0/d13/d41/d7f 0 2024-03-24T10:46:10.428 INFO:tasks.workunit.client.1.smithi096.stdout:3/375: dread d0/d13/d45/d47/f4e [0,4194304] 0 2024-03-24T10:46:10.429 INFO:tasks.workunit.client.1.smithi096.stdout:6/328: rename d8/f1c to d8/f62 0 2024-03-24T10:46:10.430 INFO:tasks.workunit.client.1.smithi096.stdout:9/329: mkdir d9/d15/d64 0 2024-03-24T10:46:10.433 INFO:tasks.workunit.client.1.smithi096.stdout:4/274: mkdir d1/d2b/d59 0 2024-03-24T10:46:10.433 INFO:tasks.workunit.client.1.smithi096.stdout:4/275: dread - d1/d32/d42/d44/d45/f52 zero size 2024-03-24T10:46:10.434 INFO:tasks.workunit.client.1.smithi096.stdout:3/376: symlink d0/d13/d45/d7d/d39/l80 0 2024-03-24T10:46:10.434 INFO:tasks.workunit.client.1.smithi096.stdout:1/283: dwrite d3/d24/f57 [4194304,4194304] 0 2024-03-24T10:46:10.436 INFO:tasks.workunit.client.1.smithi096.stdout:9/330: unlink d9/d15/f21 0 2024-03-24T10:46:10.436 INFO:tasks.workunit.client.1.smithi096.stdout:9/331: stat d9/db/d23/d31/l61 0 2024-03-24T10:46:10.437 INFO:tasks.workunit.client.1.smithi096.stdout:9/332: read d9/f2f [40435,44027] 0 2024-03-24T10:46:10.438 INFO:tasks.workunit.client.1.smithi096.stdout:4/276: chown d1/d2b/c3a 1 1 2024-03-24T10:46:10.439 INFO:tasks.workunit.client.1.smithi096.stdout:3/377: symlink d0/d13/d41/d7f/l81 0 2024-03-24T10:46:10.441 INFO:tasks.workunit.client.1.smithi096.stdout:1/284: truncate d3/d23/d2f/f37 1496919 0 2024-03-24T10:46:10.443 INFO:tasks.workunit.client.1.smithi096.stdout:9/333: mkdir d9/d65 0 2024-03-24T10:46:10.443 INFO:tasks.workunit.client.1.smithi096.stdout:9/334: write d9/d15/f20 [1388215,70273] 0 2024-03-24T10:46:10.443 INFO:tasks.workunit.client.1.smithi096.stdout:9/335: chown d9/f13 7 1 2024-03-24T10:46:10.443 INFO:tasks.workunit.client.1.smithi096.stdout:9/336: dread - d9/db/d23/f56 zero size 2024-03-24T10:46:10.443 INFO:tasks.workunit.client.1.smithi096.stdout:9/337: creat d9/d15/f66 x:0 0 0 2024-03-24T10:46:10.444 INFO:tasks.workunit.client.1.smithi096.stdout:6/329: dwrite d8/d12/f1f [0,4194304] 0 2024-03-24T10:46:10.444 INFO:tasks.workunit.client.1.smithi096.stdout:6/330: write d8/d12/d28/f4e [80606,78583] 0 2024-03-24T10:46:10.444 INFO:tasks.workunit.client.1.smithi096.stdout:6/331: write d8/f9 [2182158,71823] 0 2024-03-24T10:46:10.445 INFO:tasks.workunit.client.1.smithi096.stdout:6/332: write d8/d10/d13/f2b [99785,38901] 0 2024-03-24T10:46:10.445 INFO:tasks.workunit.client.1.smithi096.stdout:4/277: link d1/f51 d1/d32/d4c/f5a 0 2024-03-24T10:46:10.446 INFO:tasks.workunit.client.1.smithi096.stdout:1/285: stat d3/l1a 0 2024-03-24T10:46:10.446 INFO:tasks.workunit.client.1.smithi096.stdout:4/278: write d1/d11/f12 [488171,67557] 0 2024-03-24T10:46:10.446 INFO:tasks.workunit.client.1.smithi096.stdout:3/378: dread d0/d13/f57 [0,4194304] 0 2024-03-24T10:46:10.449 INFO:tasks.workunit.client.1.smithi096.stdout:9/338: dread d9/db/d23/d31/d43/f45 [0,4194304] 0 2024-03-24T10:46:10.450 INFO:tasks.workunit.client.1.smithi096.stdout:9/339: write d9/f18 [787181,91634] 0 2024-03-24T10:46:10.455 INFO:tasks.workunit.client.1.smithi096.stdout:9/340: write d9/db/f53 [2869627,59440] 0 2024-03-24T10:46:10.460 INFO:tasks.workunit.client.1.smithi096.stdout:1/286: dwrite d3/d23/d2f/f59 [0,4194304] 0 2024-03-24T10:46:10.481 INFO:tasks.workunit.client.1.smithi096.stdout:4/279: creat d1/d2b/d59/f5b x:0 0 0 2024-03-24T10:46:10.481 INFO:tasks.workunit.client.1.smithi096.stdout:4/280: stat d1/d2b 0 2024-03-24T10:46:10.489 INFO:tasks.workunit.client.1.smithi096.stdout:1/287: symlink d3/d24/d5c/l64 0 2024-03-24T10:46:10.490 INFO:tasks.workunit.client.1.smithi096.stdout:1/288: fdatasync d3/d23/d2f/f4a 0 2024-03-24T10:46:10.490 INFO:tasks.workunit.client.1.smithi096.stdout:1/289: truncate d3/d23/d2f/f3b 988154 0 2024-03-24T10:46:10.493 INFO:tasks.workunit.client.1.smithi096.stdout:6/333: rename d8/d10/d22 to d8/d10/d13/d63 0 2024-03-24T10:46:10.493 INFO:tasks.workunit.client.1.smithi096.stdout:6/334: read d8/d12/f2d [241529,44311] 0 2024-03-24T10:46:10.498 INFO:tasks.workunit.client.1.smithi096.stdout:3/379: rename d0/d13/d45/l6b to d0/l82 0 2024-03-24T10:46:10.498 INFO:tasks.workunit.client.1.smithi096.stdout:3/380: chown d0/d13/d45/d52/f53 881 1 2024-03-24T10:46:10.499 INFO:tasks.workunit.client.1.smithi096.stdout:3/381: chown d0/d13/d45/d47/f4e 63902289 1 2024-03-24T10:46:10.499 INFO:tasks.workunit.client.1.smithi096.stdout:3/382: creat d0/d13/f83 x:0 0 0 2024-03-24T10:46:10.500 INFO:tasks.workunit.client.1.smithi096.stdout:9/341: rename d9/db/f58 to d9/db/d23/d31/d43/f67 0 2024-03-24T10:46:10.502 INFO:tasks.workunit.client.1.smithi096.stdout:3/383: link d0/c31 d0/d13/d45/d47/c84 0 2024-03-24T10:46:10.503 INFO:tasks.workunit.client.1.smithi096.stdout:3/384: dread - d0/f75 zero size 2024-03-24T10:46:10.503 INFO:tasks.workunit.client.1.smithi096.stdout:4/281: rename d1/l7 to d1/d32/d4c/l5c 0 2024-03-24T10:46:10.504 INFO:tasks.workunit.client.1.smithi096.stdout:1/290: dwrite d3/d36/f5f [0,4194304] 0 2024-03-24T10:46:10.506 INFO:tasks.workunit.client.1.smithi096.stdout:3/385: rmdir d0/d13/d41/d7b 0 2024-03-24T10:46:10.508 INFO:tasks.workunit.client.1.smithi096.stdout:4/282: mknod d1/c5d 0 2024-03-24T10:46:10.508 INFO:tasks.workunit.client.1.smithi096.stdout:4/283: fsync d1/d32/d4c/f50 0 2024-03-24T10:46:10.508 INFO:tasks.workunit.client.1.smithi096.stdout:4/284: readlink d1/d11/l31 0 2024-03-24T10:46:10.508 INFO:tasks.workunit.client.1.smithi096.stdout:4/285: truncate d1/f24 5810164 0 2024-03-24T10:46:10.512 INFO:tasks.workunit.client.1.smithi096.stdout:6/335: dwrite d8/d12/d32/d38/d59/f5a [0,4194304] 0 2024-03-24T10:46:10.512 INFO:tasks.workunit.client.1.smithi096.stdout:6/336: chown d8/d12/d61 0 1 2024-03-24T10:46:10.514 INFO:tasks.workunit.client.1.smithi096.stdout:4/286: unlink d1/d11/f39 0 2024-03-24T10:46:10.518 INFO:tasks.workunit.client.1.smithi096.stdout:6/337: symlink d8/d12/d32/d38/d59/l64 0 2024-03-24T10:46:10.521 INFO:tasks.workunit.client.1.smithi096.stdout:4/287: creat d1/d32/d42/f5e x:0 0 0 2024-03-24T10:46:10.529 INFO:tasks.workunit.client.1.smithi096.stdout:9/342: dwrite d9/d15/f66 [0,4194304] 0 2024-03-24T10:46:10.530 INFO:tasks.workunit.client.1.smithi096.stdout:6/338: mkdir d8/d12/d28/d36/d65 0 2024-03-24T10:46:10.533 INFO:tasks.workunit.client.1.smithi096.stdout:1/291: dwrite d3/d36/f54 [0,4194304] 0 2024-03-24T10:46:10.540 INFO:tasks.workunit.client.1.smithi096.stdout:3/386: dwrite d0/d13/d45/d52/f53 [0,4194304] 0 2024-03-24T10:46:10.542 INFO:tasks.workunit.client.1.smithi096.stdout:6/339: mkdir d8/d12/d32/d66 0 2024-03-24T10:46:10.545 INFO:tasks.workunit.client.1.smithi096.stdout:1/292: dread d3/d36/f54 [0,4194304] 0 2024-03-24T10:46:10.551 INFO:tasks.workunit.client.1.smithi096.stdout:5/285: sync 2024-03-24T10:46:10.551 INFO:tasks.workunit.client.1.smithi096.stdout:5/286: truncate d2/d49/f5d 670249 0 2024-03-24T10:46:10.551 INFO:tasks.workunit.client.1.smithi096.stdout:5/287: chown d2/d29/d32/d33/d3c/d3d/c58 15839 1 2024-03-24T10:46:10.552 INFO:tasks.workunit.client.1.smithi096.stdout:4/288: dwrite d1/d32/d42/d44/d45/f52 [0,4194304] 0 2024-03-24T10:46:10.552 INFO:tasks.workunit.client.1.smithi096.stdout:4/289: dread - d1/d2b/d59/f5b zero size 2024-03-24T10:46:10.554 INFO:tasks.workunit.client.1.smithi096.stdout:9/343: dwrite d9/db/d23/f56 [0,4194304] 0 2024-03-24T10:46:10.555 INFO:tasks.workunit.client.1.smithi096.stdout:1/293: dread d3/d24/f3e [0,4194304] 0 2024-03-24T10:46:10.559 INFO:tasks.workunit.client.1.smithi096.stdout:6/340: dwrite d8/d10/d13/d63/f21 [4194304,4194304] 0 2024-03-24T10:46:10.559 INFO:tasks.workunit.client.1.smithi096.stdout:6/341: dread - d8/d12/d32/d38/d59/f5f zero size 2024-03-24T10:46:10.559 INFO:tasks.workunit.client.1.smithi096.stdout:6/342: dread - d8/d10/d19/f60 zero size 2024-03-24T10:46:10.562 INFO:tasks.workunit.client.1.smithi096.stdout:5/288: mknod d2/d7/c61 0 2024-03-24T10:46:10.562 INFO:tasks.workunit.client.1.smithi096.stdout:5/289: getdents d2/d7/d8/d18/d4d 0 2024-03-24T10:46:10.562 INFO:tasks.workunit.client.1.smithi096.stdout:5/290: creat d2/d7/d8/f62 x:0 0 0 2024-03-24T10:46:10.570 INFO:tasks.workunit.client.1.smithi096.stdout:6/343: dread d8/d10/d13/d63/f21 [4194304,4194304] 0 2024-03-24T10:46:10.572 INFO:tasks.workunit.client.1.smithi096.stdout:4/290: mkdir d1/d32/d42/d44/d45/d5f 0 2024-03-24T10:46:10.573 INFO:tasks.workunit.client.1.smithi096.stdout:4/291: write d1/d32/d4c/f5a [5612503,87827] 0 2024-03-24T10:46:10.579 INFO:tasks.workunit.client.1.smithi096.stdout:9/344: dwrite d9/db/d23/d4f/f63 [4194304,4194304] 0 2024-03-24T10:46:10.587 INFO:tasks.workunit.client.1.smithi096.stdout:1/294: mkdir d3/d23/d2f/d65 0 2024-03-24T10:46:10.589 INFO:tasks.workunit.client.1.smithi096.stdout:4/292: dwrite d1/f35 [0,4194304] 0 2024-03-24T10:46:10.594 INFO:tasks.workunit.client.1.smithi096.stdout:3/387: getdents d0/d13/d45/d52 0 2024-03-24T10:46:10.596 INFO:tasks.workunit.client.1.smithi096.stdout:3/388: write d0/f61 [575374,13311] 0 2024-03-24T10:46:10.596 INFO:tasks.workunit.client.1.smithi096.stdout:3/389: read d0/f44 [1117205,60455] 0 2024-03-24T10:46:10.596 INFO:tasks.workunit.client.1.smithi096.stdout:3/390: write d0/d13/d45/d52/f66 [546920,48053] 0 2024-03-24T10:46:10.596 INFO:tasks.workunit.client.1.smithi096.stdout:3/391: write d0/f75 [366346,117434] 0 2024-03-24T10:46:10.597 INFO:tasks.workunit.client.1.smithi096.stdout:5/291: creat d2/d29/d32/d33/f63 x:0 0 0 2024-03-24T10:46:10.598 INFO:tasks.workunit.client.1.smithi096.stdout:6/344: mknod d8/d10/c67 0 2024-03-24T10:46:10.598 INFO:tasks.workunit.client.1.smithi096.stdout:4/293: dread d1/d11/f19 [0,4194304] 0 2024-03-24T10:46:10.606 INFO:tasks.workunit.client.1.smithi096.stdout:9/345: symlink d9/db/d23/d31/d43/d48/d51/l68 0 2024-03-24T10:46:10.606 INFO:tasks.workunit.client.1.smithi096.stdout:1/295: rmdir d3/d24/d5c 39 2024-03-24T10:46:10.608 INFO:tasks.workunit.client.1.smithi096.stdout:3/392: mkdir d0/d13/d45/d7d/d39/d48/d85 0 2024-03-24T10:46:10.612 INFO:tasks.workunit.client.1.smithi096.stdout:6/345: dwrite d8/d12/d32/d38/f44 [0,4194304] 0 2024-03-24T10:46:10.613 INFO:tasks.workunit.client.1.smithi096.stdout:5/292: mknod d2/d29/d32/d33/d37/c64 0 2024-03-24T10:46:10.616 INFO:tasks.workunit.client.1.smithi096.stdout:6/346: dread d8/f62 [0,4194304] 0 2024-03-24T10:46:10.616 INFO:tasks.workunit.client.1.smithi096.stdout:1/296: creat d3/f66 x:0 0 0 2024-03-24T10:46:10.618 INFO:tasks.workunit.client.1.smithi096.stdout:5/293: dread d2/f43 [0,4194304] 0 2024-03-24T10:46:10.618 INFO:tasks.workunit.client.1.smithi096.stdout:5/294: chown d2/d7/d8/df/f2d 50095481 1 2024-03-24T10:46:10.630 INFO:tasks.workunit.client.1.smithi096.stdout:4/294: creat d1/d2b/d40/f60 x:0 0 0 2024-03-24T10:46:10.630 INFO:tasks.workunit.client.1.smithi096.stdout:4/295: stat d1/fe 0 2024-03-24T10:46:10.630 INFO:tasks.workunit.client.1.smithi096.stdout:9/346: symlink d9/db/l69 0 2024-03-24T10:46:10.631 INFO:tasks.workunit.client.1.smithi096.stdout:6/347: mkdir d8/d12/d32/d68 0 2024-03-24T10:46:10.632 INFO:tasks.workunit.client.1.smithi096.stdout:1/297: creat d3/d24/d5c/f67 x:0 0 0 2024-03-24T10:46:10.634 INFO:tasks.workunit.client.1.smithi096.stdout:5/295: unlink d2/c21 0 2024-03-24T10:46:10.634 INFO:tasks.workunit.client.1.smithi096.stdout:5/296: readlink d2/d29/d32/d33/d3c/d3d/l4e 0 2024-03-24T10:46:10.635 INFO:tasks.workunit.client.1.smithi096.stdout:4/296: symlink d1/d32/d4c/l61 0 2024-03-24T10:46:10.635 INFO:tasks.workunit.client.1.smithi096.stdout:4/297: creat d1/d32/d42/d44/f62 x:0 0 0 2024-03-24T10:46:10.636 INFO:tasks.workunit.client.1.smithi096.stdout:6/348: getdents d8/d10/d13/d63 0 2024-03-24T10:46:10.637 INFO:tasks.workunit.client.1.smithi096.stdout:6/349: creat d8/d12/f69 x:0 0 0 2024-03-24T10:46:10.639 INFO:tasks.workunit.client.1.smithi096.stdout:5/297: rename d2/d7/d8/df/l2c to d2/d29/l65 0 2024-03-24T10:46:10.639 INFO:tasks.workunit.client.1.smithi096.stdout:5/298: creat d2/d29/d32/d3f/f66 x:0 0 0 2024-03-24T10:46:10.641 INFO:tasks.workunit.client.1.smithi096.stdout:1/298: dread d3/d24/d5c/f61 [0,4194304] 0 2024-03-24T10:46:10.642 INFO:tasks.workunit.client.1.smithi096.stdout:9/347: getdents d9/d15 0 2024-03-24T10:46:10.646 INFO:tasks.workunit.client.1.smithi096.stdout:9/348: dread d9/d15/f4a [0,4194304] 0 2024-03-24T10:46:10.647 INFO:tasks.workunit.client.1.smithi096.stdout:9/349: creat d9/d15/d26/d29/f6a x:0 0 0 2024-03-24T10:46:10.647 INFO:tasks.workunit.client.1.smithi096.stdout:9/350: write d9/d15/f1e [2498953,52727] 0 2024-03-24T10:46:10.647 INFO:tasks.workunit.client.1.smithi096.stdout:6/350: unlink d8/d12/d32/d38/f44 0 2024-03-24T10:46:10.652 INFO:tasks.workunit.client.1.smithi096.stdout:9/351: mknod d9/db/d23/d31/d43/c6b 0 2024-03-24T10:46:10.652 INFO:tasks.workunit.client.1.smithi096.stdout:9/352: creat d9/db/d23/d31/d43/f6c x:0 0 0 2024-03-24T10:46:10.654 INFO:tasks.workunit.client.1.smithi096.stdout:6/351: rmdir d8/d10/d13/d63 39 2024-03-24T10:46:10.658 INFO:tasks.workunit.client.1.smithi096.stdout:6/352: truncate d8/d10/d13/f2b 1917880 0 2024-03-24T10:46:10.659 INFO:tasks.workunit.client.1.smithi096.stdout:6/353: fsync f2 0 2024-03-24T10:46:10.659 INFO:tasks.workunit.client.1.smithi096.stdout:6/354: fsync d8/d12/d28/f33 0 2024-03-24T10:46:10.659 INFO:tasks.workunit.client.1.smithi096.stdout:6/355: fdatasync d8/d12/f5c 0 2024-03-24T10:46:10.660 INFO:tasks.workunit.client.1.smithi096.stdout:1/299: dwrite d3/d36/f54 [0,4194304] 0 2024-03-24T10:46:10.660 INFO:tasks.workunit.client.1.smithi096.stdout:1/300: creat d3/d23/f68 x:0 0 0 2024-03-24T10:46:10.660 INFO:tasks.workunit.client.1.smithi096.stdout:1/301: stat d3/d23 0 2024-03-24T10:46:10.661 INFO:tasks.workunit.client.1.smithi096.stdout:5/299: dwrite d2/d7/d8/d18/f47 [0,4194304] 0 2024-03-24T10:46:10.668 INFO:tasks.workunit.client.1.smithi096.stdout:1/302: write d3/d36/f5f [3050990,28805] 0 2024-03-24T10:46:10.668 INFO:tasks.workunit.client.1.smithi096.stdout:1/303: creat d3/d23/f69 x:0 0 0 2024-03-24T10:46:10.671 INFO:tasks.workunit.client.1.smithi096.stdout:5/300: rename d2/d7/d8/d18/l50 to d2/d29/d32/d3f/d5f/l67 0 2024-03-24T10:46:10.684 INFO:tasks.workunit.client.1.smithi096.stdout:1/304: mkdir d3/d24/d6a 0 2024-03-24T10:46:10.684 INFO:tasks.workunit.client.1.smithi096.stdout:1/305: write d3/f66 [251622,112121] 0 2024-03-24T10:46:10.684 INFO:tasks.workunit.client.1.smithi096.stdout:1/306: truncate d3/d24/f39 931015 0 2024-03-24T10:46:10.684 INFO:tasks.workunit.client.1.smithi096.stdout:6/356: dwrite d8/f31 [0,4194304] 0 2024-03-24T10:46:10.684 INFO:tasks.workunit.client.1.smithi096.stdout:1/307: write d3/d24/f39 [772722,63271] 0 2024-03-24T10:46:10.684 INFO:tasks.workunit.client.1.smithi096.stdout:1/308: write d3/d23/d2f/f3d [688206,25172] 0 2024-03-24T10:46:10.684 INFO:tasks.workunit.client.1.smithi096.stdout:1/309: creat d3/d24/f6b x:0 0 0 2024-03-24T10:46:10.686 INFO:tasks.workunit.client.1.smithi096.stdout:5/301: mknod d2/d7/d8/df/c68 0 2024-03-24T10:46:10.689 INFO:tasks.workunit.client.1.smithi096.stdout:6/357: link d8/d12/d32/d38/d59/l64 d8/d10/d13/d63/d51/l6a 0 2024-03-24T10:46:10.691 INFO:tasks.workunit.client.1.smithi096.stdout:1/310: symlink d3/d36/l6c 0 2024-03-24T10:46:10.692 INFO:tasks.workunit.client.1.smithi096.stdout:5/302: mkdir d2/d7/d69 0 2024-03-24T10:46:10.693 INFO:tasks.workunit.client.1.smithi096.stdout:5/303: dread d2/d7/f3e [0,4194304] 0 2024-03-24T10:46:10.693 INFO:tasks.workunit.client.1.smithi096.stdout:5/304: write d2/d7/d8/df/f19 [745538,111199] 0 2024-03-24T10:46:10.695 INFO:tasks.workunit.client.1.smithi096.stdout:4/298: getdents d1/d32/d42/d44 0 2024-03-24T10:46:10.700 INFO:tasks.workunit.client.1.smithi096.stdout:6/358: mkdir d8/d12/d28/d36/d65/d6b 0 2024-03-24T10:46:10.704 INFO:tasks.workunit.client.1.smithi096.stdout:4/299: rename d1/c5d to d1/d32/c63 0 2024-03-24T10:46:10.708 INFO:tasks.workunit.client.1.smithi096.stdout:1/311: dwrite d3/d36/f5f [0,4194304] 0 2024-03-24T10:46:10.710 INFO:tasks.workunit.client.1.smithi096.stdout:0/309: sync 2024-03-24T10:46:10.710 INFO:tasks.workunit.client.1.smithi096.stdout:8/381: sync 2024-03-24T10:46:10.710 INFO:tasks.workunit.client.1.smithi096.stdout:2/222: sync 2024-03-24T10:46:10.710 INFO:tasks.workunit.client.1.smithi096.stdout:0/310: chown d0/d11/d16/d21/d25/f46 6672 1 2024-03-24T10:46:10.711 INFO:tasks.workunit.client.1.smithi096.stdout:2/223: write d5/fb [959622,111283] 0 2024-03-24T10:46:10.711 INFO:tasks.workunit.client.1.smithi096.stdout:2/224: chown d5/dc/d12/d23/d4f 6770 1 2024-03-24T10:46:10.711 INFO:tasks.workunit.client.1.smithi096.stdout:2/225: fsync d5/dc/f21 0 2024-03-24T10:46:10.717 INFO:tasks.workunit.client.1.smithi096.stdout:1/312: mkdir d3/d24/d5c/d6d 0 2024-03-24T10:46:10.722 INFO:tasks.workunit.client.1.smithi096.stdout:0/311: rmdir d0/d11/d20/d56 39 2024-03-24T10:46:10.723 INFO:tasks.workunit.client.1.smithi096.stdout:2/226: mknod d5/c63 0 2024-03-24T10:46:10.723 INFO:tasks.workunit.client.1.smithi096.stdout:2/227: stat d5/d38/c3e 0 2024-03-24T10:46:10.724 INFO:tasks.workunit.client.1.smithi096.stdout:9/353: rename d9/db/d23 to d9/db/d6d 0 2024-03-24T10:46:10.725 INFO:tasks.workunit.client.1.smithi096.stdout:4/300: unlink d1/d2b/f4e 0 2024-03-24T10:46:10.725 INFO:tasks.workunit.client.1.smithi096.stdout:4/301: fdatasync d1/f51 0 2024-03-24T10:46:10.726 INFO:tasks.workunit.client.1.smithi096.stdout:4/302: creat d1/d2b/d40/f64 x:0 0 0 2024-03-24T10:46:10.727 INFO:tasks.workunit.client.1.smithi096.stdout:1/313: symlink d3/d24/l6e 0 2024-03-24T10:46:10.728 INFO:tasks.workunit.client.1.smithi096.stdout:8/382: mknod d8/d11/d22/d4c/d65/d1d/c7d 0 2024-03-24T10:46:10.728 INFO:tasks.workunit.client.1.smithi096.stdout:5/305: write d2/d7/d8/d18/f47 [4884033,102331] 0 2024-03-24T10:46:10.729 INFO:tasks.workunit.client.1.smithi096.stdout:0/312: mknod d0/d11/d24/d2c/c61 0 2024-03-24T10:46:10.730 INFO:tasks.workunit.client.1.smithi096.stdout:0/313: creat d0/d11/d16/d21/d25/f62 x:0 0 0 2024-03-24T10:46:10.731 INFO:tasks.workunit.client.1.smithi096.stdout:9/354: symlink d9/db/d6d/d31/d43/d48/l6e 0 2024-03-24T10:46:10.733 INFO:tasks.workunit.client.1.smithi096.stdout:4/303: mknod d1/d32/d42/c65 0 2024-03-24T10:46:10.734 INFO:tasks.workunit.client.1.smithi096.stdout:4/304: write d1/d11/f29 [177248,41121] 0 2024-03-24T10:46:10.735 INFO:tasks.workunit.client.1.smithi096.stdout:1/314: link d3/d24/c5d d3/d24/d6a/c6f 0 2024-03-24T10:46:10.736 INFO:tasks.workunit.client.1.smithi096.stdout:8/383: write d8/d11/d22/d4c/d65/d37/f6e [3962098,92709] 0 2024-03-24T10:46:10.740 INFO:tasks.workunit.client.1.smithi096.stdout:2/228: getdents d5/dc/d12/d23/d4f 0 2024-03-24T10:46:10.744 INFO:tasks.workunit.client.1.smithi096.stdout:4/305: creat d1/d11/f66 x:0 0 0 2024-03-24T10:46:10.744 INFO:tasks.workunit.client.1.smithi096.stdout:4/306: fsync d1/d11/f23 0 2024-03-24T10:46:10.744 INFO:tasks.workunit.client.1.smithi096.stdout:4/307: chown d1/d11/l31 19 1 2024-03-24T10:46:10.745 INFO:tasks.workunit.client.1.smithi096.stdout:4/308: truncate d1/d32/d42/d44/f62 385864 0 2024-03-24T10:46:10.750 INFO:tasks.workunit.client.1.smithi096.stdout:2/229: symlink d5/de/d2a/l64 0 2024-03-24T10:46:10.750 INFO:tasks.workunit.client.1.smithi096.stdout:2/230: write d5/dc/f27 [542118,21097] 0 2024-03-24T10:46:10.751 INFO:tasks.workunit.client.1.smithi096.stdout:2/231: rmdir d5/dc/d12/d23/d4f 39 2024-03-24T10:46:10.751 INFO:tasks.workunit.client.1.smithi096.stdout:2/232: read d5/fa [3649257,42707] 0 2024-03-24T10:46:10.752 INFO:tasks.workunit.client.1.smithi096.stdout:2/233: creat d5/de/d2a/d5e/f65 x:0 0 0 2024-03-24T10:46:10.761 INFO:tasks.workunit.client.1.smithi096.stdout:5/306: dwrite d2/d7/d8/f62 [0,4194304] 0 2024-03-24T10:46:10.761 INFO:tasks.workunit.client.1.smithi096.stdout:5/307: dread - d2/f5a zero size 2024-03-24T10:46:10.761 INFO:tasks.workunit.client.1.smithi096.stdout:5/308: creat d2/d7/d8/df/f6a x:0 0 0 2024-03-24T10:46:10.764 INFO:tasks.workunit.client.1.smithi096.stdout:8/384: dread d8/d11/d22/d4c/d65/d37/f6b [0,4194304] 0 2024-03-24T10:46:10.766 INFO:tasks.workunit.client.1.smithi096.stdout:0/314: dwrite d0/d11/d40/f5d [0,4194304] 0 2024-03-24T10:46:10.770 INFO:tasks.workunit.client.1.smithi096.stdout:5/309: symlink d2/d29/d32/d33/d3c/l6b 0 2024-03-24T10:46:10.771 INFO:tasks.workunit.client.1.smithi096.stdout:1/315: dwrite d3/d36/f50 [0,4194304] 0 2024-03-24T10:46:10.779 INFO:tasks.workunit.client.1.smithi096.stdout:2/234: dwrite d5/dc/d12/f4a [0,4194304] 0 2024-03-24T10:46:10.779 INFO:tasks.workunit.client.1.smithi096.stdout:2/235: read - d5/de/f51 zero size 2024-03-24T10:46:10.780 INFO:tasks.workunit.client.1.smithi096.stdout:1/316: creat d3/d24/d6a/f70 x:0 0 0 2024-03-24T10:46:10.784 INFO:tasks.workunit.client.1.smithi096.stdout:0/315: getdents d0/d11/d20/d2b 0 2024-03-24T10:46:10.786 INFO:tasks.workunit.client.1.smithi096.stdout:2/236: write f3 [2069272,10001] 0 2024-03-24T10:46:10.789 INFO:tasks.workunit.client.1.smithi096.stdout:0/316: truncate d0/d11/d16/d21/d25/f46 3800228 0 2024-03-24T10:46:10.790 INFO:tasks.workunit.client.1.smithi096.stdout:8/385: dwrite d8/d11/d44/d45/f4d [0,4194304] 0 2024-03-24T10:46:10.791 INFO:tasks.workunit.client.1.smithi096.stdout:2/237: dread d5/fa [0,4194304] 0 2024-03-24T10:46:10.792 INFO:tasks.workunit.client.1.smithi096.stdout:0/317: mkdir d0/d11/d16/d21/d25/d47/d63 0 2024-03-24T10:46:10.797 INFO:tasks.workunit.client.1.smithi096.stdout:2/238: link d5/d38/f3a d5/d38/f66 0 2024-03-24T10:46:10.797 INFO:tasks.workunit.client.1.smithi096.stdout:2/239: chown d5/dc/c50 5 1 2024-03-24T10:46:10.798 INFO:tasks.workunit.client.1.smithi096.stdout:0/318: rmdir d0/d11/d16/d21/d25/d47/d49 0 2024-03-24T10:46:10.799 INFO:tasks.workunit.client.1.smithi096.stdout:0/319: creat d0/d2/f64 x:0 0 0 2024-03-24T10:46:10.804 INFO:tasks.workunit.client.1.smithi096.stdout:1/317: write d3/d36/f54 [155003,59028] 0 2024-03-24T10:46:10.807 INFO:tasks.workunit.client.1.smithi096.stdout:0/320: mkdir d0/d11/d20/d65 0 2024-03-24T10:46:10.807 INFO:tasks.workunit.client.1.smithi096.stdout:5/310: dwrite d2/d29/d32/d3f/f55 [0,4194304] 0 2024-03-24T10:46:10.809 INFO:tasks.workunit.client.1.smithi096.stdout:0/321: creat d0/d11/f66 x:0 0 0 2024-03-24T10:46:10.815 INFO:tasks.workunit.client.1.smithi096.stdout:0/322: creat d0/d11/d24/d2c/f67 x:0 0 0 2024-03-24T10:46:10.815 INFO:tasks.workunit.client.1.smithi096.stdout:0/323: creat d0/d11/d24/f68 x:0 0 0 2024-03-24T10:46:10.815 INFO:tasks.workunit.client.1.smithi096.stdout:0/324: fsync d0/d11/d20/d56/f5c 0 2024-03-24T10:46:10.820 INFO:tasks.workunit.client.1.smithi096.stdout:2/240: dwrite d5/fb [0,4194304] 0 2024-03-24T10:46:10.821 INFO:tasks.workunit.client.1.smithi096.stdout:0/325: mknod d0/d11/d20/d65/c69 0 2024-03-24T10:46:10.822 INFO:tasks.workunit.client.1.smithi096.stdout:2/241: dread d5/dc/d12/d23/f2c [0,4194304] 0 2024-03-24T10:46:10.824 INFO:tasks.workunit.client.1.smithi096.stdout:0/326: truncate d0/d11/d24/f50 348193 0 2024-03-24T10:46:10.826 INFO:tasks.workunit.client.1.smithi096.stdout:2/242: rename d5/de/f59 to d5/d38/f67 0 2024-03-24T10:46:10.826 INFO:tasks.workunit.client.1.smithi096.stdout:2/243: chown d5/dc/f58 952654398 1 2024-03-24T10:46:10.828 INFO:tasks.workunit.client.1.smithi096.stdout:1/318: dread d3/d36/f5f [0,4194304] 0 2024-03-24T10:46:10.828 INFO:tasks.workunit.client.1.smithi096.stdout:0/327: unlink d0/d11/d16/d21/d25/f62 0 2024-03-24T10:46:10.828 INFO:tasks.workunit.client.1.smithi096.stdout:1/319: read d3/d24/d5c/f61 [1776600,110646] 0 2024-03-24T10:46:10.830 INFO:tasks.workunit.client.1.smithi096.stdout:7/404: sync 2024-03-24T10:46:10.831 INFO:tasks.workunit.client.1.smithi096.stdout:7/405: chown d0 296313375 1 2024-03-24T10:46:10.831 INFO:tasks.workunit.client.1.smithi096.stdout:7/406: write d0/db/d1c/f61 [968371,119737] 0 2024-03-24T10:46:10.831 INFO:tasks.workunit.client.1.smithi096.stdout:0/328: creat d0/d11/d20/d2b/f6a x:0 0 0 2024-03-24T10:46:10.832 INFO:tasks.workunit.client.1.smithi096.stdout:0/329: stat d0/f48 0 2024-03-24T10:46:10.832 INFO:tasks.workunit.client.1.smithi096.stdout:0/330: write d0/d11/d16/f54 [839765,27029] 0 2024-03-24T10:46:10.835 INFO:tasks.workunit.client.1.smithi096.stdout:2/244: symlink d5/de/l68 0 2024-03-24T10:46:10.835 INFO:tasks.workunit.client.1.smithi096.stdout:7/407: creat d0/d13/d60/f86 x:0 0 0 2024-03-24T10:46:10.840 INFO:tasks.workunit.client.1.smithi096.stdout:9/355: rmdir d9/db/d6d 39 2024-03-24T10:46:10.840 INFO:tasks.workunit.client.1.smithi096.stdout:9/356: creat d9/d15/d26/d29/f6f x:0 0 0 2024-03-24T10:46:10.856 INFO:tasks.workunit.client.1.smithi096.stdout:0/331: dwrite d0/f60 [0,4194304] 0 2024-03-24T10:46:10.862 INFO:tasks.workunit.client.1.smithi096.stdout:7/408: dwrite d0/db/d1c/d39/f76 [0,4194304] 0 2024-03-24T10:46:10.862 INFO:tasks.workunit.client.1.smithi096.stdout:0/332: link d0/d11/d16/f4e d0/d11/d16/d21/d25/d47/f6b 0 2024-03-24T10:46:10.867 INFO:tasks.workunit.client.1.smithi096.stdout:9/357: dwrite d9/f19 [0,4194304] 0 2024-03-24T10:46:10.867 INFO:tasks.workunit.client.1.smithi096.stdout:9/358: fdatasync d9/db/d6d/d31/d43/f44 0 2024-03-24T10:46:10.867 INFO:tasks.workunit.client.1.smithi096.stdout:7/409: write d0/d13/f35 [1262446,78520] 0 2024-03-24T10:46:10.867 INFO:tasks.workunit.client.1.smithi096.stdout:7/410: write d0/db/d1c/d77/f7a [355026,108713] 0 2024-03-24T10:46:10.867 INFO:tasks.workunit.client.1.smithi096.stdout:0/333: mknod d0/d11/d20/d56/c6c 0 2024-03-24T10:46:10.867 INFO:tasks.workunit.client.1.smithi096.stdout:0/334: creat d0/d11/f6d x:0 0 0 2024-03-24T10:46:10.872 INFO:tasks.workunit.client.1.smithi096.stdout:9/359: dread d9/f1a [0,4194304] 0 2024-03-24T10:46:10.872 INFO:tasks.workunit.client.1.smithi096.stdout:9/360: dread - d9/db/d6d/d31/d43/f6c zero size 2024-03-24T10:46:10.876 INFO:tasks.workunit.client.1.smithi096.stdout:7/411: mkdir d0/d21/d59/d87 0 2024-03-24T10:46:10.878 INFO:tasks.workunit.client.1.smithi096.stdout:0/335: creat d0/d11/d20/d65/f6e x:0 0 0 2024-03-24T10:46:10.882 INFO:tasks.workunit.client.1.smithi096.stdout:9/361: rename d9/f19 to d9/d15/d26/f70 0 2024-03-24T10:46:10.882 INFO:tasks.workunit.client.1.smithi096.stdout:9/362: write d9/db/d6d/f35 [4286829,19538] 0 2024-03-24T10:46:10.882 INFO:tasks.workunit.client.1.smithi096.stdout:9/363: fsync d9/db/d6d/d31/f4b 0 2024-03-24T10:46:10.882 INFO:tasks.workunit.client.1.smithi096.stdout:9/364: readlink d9/db/d6d/d31/d49/l5f 0 2024-03-24T10:46:10.885 INFO:tasks.workunit.client.1.smithi096.stdout:7/412: symlink d0/d13/d32/l88 0 2024-03-24T10:46:10.888 INFO:tasks.workunit.client.1.smithi096.stdout:0/336: mknod d0/d11/d24/d2c/c6f 0 2024-03-24T10:46:10.890 INFO:tasks.workunit.client.1.smithi096.stdout:7/413: symlink d0/db/d1c/l89 0 2024-03-24T10:46:10.890 INFO:tasks.workunit.client.1.smithi096.stdout:7/414: fsync d0/d21/f2e 0 2024-03-24T10:46:10.890 INFO:tasks.workunit.client.1.smithi096.stdout:7/415: readlink d0/db/d73/l78 0 2024-03-24T10:46:10.891 INFO:tasks.workunit.client.1.smithi096.stdout:5/311: rmdir d2/d7/d8/df 39 2024-03-24T10:46:10.894 INFO:tasks.workunit.client.1.smithi096.stdout:0/337: rename d0/d11/d20/d2b/f37 to d0/d11/d16/d21/d25/d47/f70 0 2024-03-24T10:46:10.894 INFO:tasks.workunit.client.1.smithi096.stdout:0/338: fdatasync d0/f5 0 2024-03-24T10:46:10.894 INFO:tasks.workunit.client.1.smithi096.stdout:0/339: stat d0/d11/d20/f1f 0 2024-03-24T10:46:10.895 INFO:tasks.workunit.client.1.smithi096.stdout:0/340: fsync d0/d11/d16/f53 0 2024-03-24T10:46:10.895 INFO:tasks.workunit.client.1.smithi096.stdout:0/341: dread - d0/d11/f6d zero size 2024-03-24T10:46:10.895 INFO:tasks.workunit.client.1.smithi096.stdout:0/342: creat d0/d2/f71 x:0 0 0 2024-03-24T10:46:10.895 INFO:tasks.workunit.client.1.smithi096.stdout:0/343: creat d0/f72 x:0 0 0 2024-03-24T10:46:10.896 INFO:tasks.workunit.client.1.smithi096.stdout:7/416: creat d0/d21/d59/d87/f8a x:0 0 0 2024-03-24T10:46:10.896 INFO:tasks.workunit.client.1.smithi096.stdout:7/417: dread - d0/db/d4f/f65 zero size 2024-03-24T10:46:10.896 INFO:tasks.workunit.client.1.smithi096.stdout:7/418: chown d0/db/d4f/l7b 5373 1 2024-03-24T10:46:10.897 INFO:tasks.workunit.client.1.smithi096.stdout:5/312: creat d2/d29/d32/d3f/d5f/f6c x:0 0 0 2024-03-24T10:46:10.901 INFO:tasks.workunit.client.1.smithi096.stdout:9/365: dwrite d9/f5c [0,4194304] 0 2024-03-24T10:46:10.901 INFO:tasks.workunit.client.1.smithi096.stdout:0/344: dread d0/d11/f1c [0,4194304] 0 2024-03-24T10:46:10.901 INFO:tasks.workunit.client.1.smithi096.stdout:0/345: chown d0/d11/d20/d2b 59 1 2024-03-24T10:46:10.902 INFO:tasks.workunit.client.1.smithi096.stdout:7/419: rename d0/db/f1b to d0/d21/f8b 0 2024-03-24T10:46:10.902 INFO:tasks.workunit.client.1.smithi096.stdout:7/420: fdatasync d0/d21/d59/f7d 0 2024-03-24T10:46:10.902 INFO:tasks.workunit.client.1.smithi096.stdout:7/421: creat d0/db/f8c x:0 0 0 2024-03-24T10:46:10.902 INFO:tasks.workunit.client.1.smithi096.stdout:7/422: truncate d0/db/f72 506838 0 2024-03-24T10:46:10.902 INFO:tasks.workunit.client.1.smithi096.stdout:7/423: readlink d0/d21/l7c 0 2024-03-24T10:46:10.904 INFO:tasks.workunit.client.1.smithi096.stdout:0/346: dread d0/d11/d16/d21/f33 [0,4194304] 0 2024-03-24T10:46:10.904 INFO:tasks.workunit.client.1.smithi096.stdout:5/313: creat d2/d29/d32/d33/d37/f6d x:0 0 0 2024-03-24T10:46:10.908 INFO:tasks.workunit.client.1.smithi096.stdout:2/245: fsync d5/d38/f66 0 2024-03-24T10:46:10.912 INFO:tasks.workunit.client.1.smithi096.stdout:9/366: symlink d9/db/d6d/d31/l71 0 2024-03-24T10:46:10.916 INFO:tasks.workunit.client.1.smithi096.stdout:7/424: rename d0/d21/f51 to d0/db/d26/d48/f8d 0 2024-03-24T10:46:10.919 INFO:tasks.workunit.client.1.smithi096.stdout:0/347: unlink d0/d11/c14 0 2024-03-24T10:46:10.921 INFO:tasks.workunit.client.1.smithi096.stdout:9/367: write d9/d15/d26/f70 [971297,96134] 0 2024-03-24T10:46:10.922 INFO:tasks.workunit.client.1.smithi096.stdout:2/246: dwrite d5/de/d2a/d5e/f65 [0,4194304] 0 2024-03-24T10:46:10.922 INFO:tasks.workunit.client.1.smithi096.stdout:5/314: symlink d2/d7/d8/d18/l6e 0 2024-03-24T10:46:10.922 INFO:tasks.workunit.client.1.smithi096.stdout:7/425: creat d0/d21/d3c/d68/d85/f8e x:0 0 0 2024-03-24T10:46:10.924 INFO:tasks.workunit.client.1.smithi096.stdout:0/348: rename d0/d11/d16/f4e to d0/f73 0 2024-03-24T10:46:10.925 INFO:tasks.workunit.client.1.smithi096.stdout:0/349: chown d0/d2/l29 12340357 1 2024-03-24T10:46:10.930 INFO:tasks.workunit.client.1.smithi096.stdout:2/247: link d5/dc/f5b d5/dc/d12/d23/d4f/f69 0 2024-03-24T10:46:10.930 INFO:tasks.workunit.client.1.smithi096.stdout:2/248: truncate d5/dc/d12/f41 5104 0 2024-03-24T10:46:10.931 INFO:tasks.workunit.client.1.smithi096.stdout:5/315: stat d2/d7/d8/df/d14/c24 0 2024-03-24T10:46:10.934 INFO:tasks.workunit.client.1.smithi096.stdout:7/426: dread d0/db/d26/f31 [0,4194304] 0 2024-03-24T10:46:10.934 INFO:tasks.workunit.client.1.smithi096.stdout:0/350: unlink d0/d11/d20/f2e 0 2024-03-24T10:46:10.936 INFO:tasks.workunit.client.1.smithi096.stdout:2/249: mknod d5/c6a 0 2024-03-24T10:46:10.937 INFO:tasks.workunit.client.1.smithi096.stdout:2/250: dread d5/dc/d12/f31 [0,4194304] 0 2024-03-24T10:46:10.938 INFO:tasks.workunit.client.1.smithi096.stdout:9/368: dwrite d9/d15/f66 [0,4194304] 0 2024-03-24T10:46:10.938 INFO:tasks.workunit.client.1.smithi096.stdout:9/369: readlink d9/db/d6d/d31/d43/d48/d51/l68 0 2024-03-24T10:46:10.938 INFO:tasks.workunit.client.1.smithi096.stdout:5/316: rename d2/d29/d32/c59 to d2/d7/d8/c6f 0 2024-03-24T10:46:10.938 INFO:tasks.workunit.client.1.smithi096.stdout:5/317: chown d2/d29/d32/d3f/c46 249 1 2024-03-24T10:46:10.939 INFO:tasks.workunit.client.1.smithi096.stdout:2/251: read d5/dc/d12/d23/f2c [775764,121568] 0 2024-03-24T10:46:10.941 INFO:tasks.workunit.client.1.smithi096.stdout:0/351: mknod d0/d11/d20/d2b/c74 0 2024-03-24T10:46:10.943 INFO:tasks.workunit.client.1.smithi096.stdout:9/370: creat d9/db/d6d/d31/d43/d48/d51/f72 x:0 0 0 2024-03-24T10:46:10.944 INFO:tasks.workunit.client.1.smithi096.stdout:9/371: read d9/d15/f20 [8425,52634] 0 2024-03-24T10:46:10.945 INFO:tasks.workunit.client.1.smithi096.stdout:5/318: symlink d2/d29/d32/d33/d37/l70 0 2024-03-24T10:46:10.945 INFO:tasks.workunit.client.1.smithi096.stdout:5/319: read d2/d7/f4b [625122,94252] 0 2024-03-24T10:46:10.947 INFO:tasks.workunit.client.1.smithi096.stdout:2/252: mkdir d5/dc/d6b 0 2024-03-24T10:46:10.948 INFO:tasks.workunit.client.1.smithi096.stdout:0/352: truncate d0/d11/d16/d21/d25/d47/f70 1978075 0 2024-03-24T10:46:10.948 INFO:tasks.workunit.client.1.smithi096.stdout:0/353: fdatasync d0/d11/d20/f1f 0 2024-03-24T10:46:10.949 INFO:tasks.workunit.client.1.smithi096.stdout:7/427: dwrite d0/d13/d32/d45/f6c [0,4194304] 0 2024-03-24T10:46:10.951 INFO:tasks.workunit.client.1.smithi096.stdout:5/320: mknod d2/d7/d8/df/c71 0 2024-03-24T10:46:10.956 INFO:tasks.workunit.client.1.smithi096.stdout:0/354: mkdir d0/d11/d43/d75 0 2024-03-24T10:46:10.956 INFO:tasks.workunit.client.1.smithi096.stdout:0/355: chown d0/d11/d20/d56/f5c 56 1 2024-03-24T10:46:10.959 INFO:tasks.workunit.client.1.smithi096.stdout:7/428: creat d0/d21/f8f x:0 0 0 2024-03-24T10:46:10.962 INFO:tasks.workunit.client.1.smithi096.stdout:7/429: symlink d0/db/l90 0 2024-03-24T10:46:10.970 INFO:tasks.workunit.client.1.smithi096.stdout:5/321: dwrite d2/d7/f10 [0,4194304] 0 2024-03-24T10:46:10.976 INFO:tasks.workunit.client.1.smithi096.stdout:0/356: dwrite d0/d11/d24/f55 [0,4194304] 0 2024-03-24T10:46:10.976 INFO:tasks.workunit.client.1.smithi096.stdout:0/357: stat d0/d11/d20/f1f 0 2024-03-24T10:46:10.976 INFO:tasks.workunit.client.1.smithi096.stdout:0/358: creat d0/d11/d24/d2c/f76 x:0 0 0 2024-03-24T10:46:10.980 INFO:tasks.workunit.client.1.smithi096.stdout:7/430: link d0/d21/f8b d0/db/d1c/f91 0 2024-03-24T10:46:10.980 INFO:tasks.workunit.client.1.smithi096.stdout:7/431: readlink d0/d13/d32/l88 0 2024-03-24T10:46:10.980 INFO:tasks.workunit.client.1.smithi096.stdout:7/432: chown d0/db/d26/f5b 1630723172 1 2024-03-24T10:46:10.984 INFO:tasks.workunit.client.1.smithi096.stdout:0/359: truncate d0/d11/d16/d21/d25/f36 2495629 0 2024-03-24T10:46:10.984 INFO:tasks.workunit.client.1.smithi096.stdout:0/360: write d0/d11/d24/f50 [1371585,31271] 0 2024-03-24T10:46:10.987 INFO:tasks.workunit.client.1.smithi096.stdout:3/393: sync 2024-03-24T10:46:10.987 INFO:tasks.workunit.client.1.smithi096.stdout:3/394: dread - d0/d13/d45/d7d/f63 zero size 2024-03-24T10:46:10.998 INFO:tasks.workunit.client.1.smithi096.stdout:3/395: dread d0/d13/d45/d52/f53 [0,4194304] 0 2024-03-24T10:46:10.998 INFO:tasks.workunit.client.1.smithi096.stdout:7/433: dwrite d0/db/ff [4194304,4194304] 0 2024-03-24T10:46:10.998 INFO:tasks.workunit.client.1.smithi096.stdout:3/396: chown d0/c11 33739889 1 2024-03-24T10:46:10.998 INFO:tasks.workunit.client.1.smithi096.stdout:3/397: write d0/d9/f1a [4254471,28421] 0 2024-03-24T10:46:11.011 INFO:tasks.workunit.client.1.smithi096.stdout:7/434: rmdir d0/d13/d60 39 2024-03-24T10:46:11.012 INFO:tasks.workunit.client.1.smithi096.stdout:3/398: dwrite d0/d9/f49 [0,4194304] 0 2024-03-24T10:46:11.012 INFO:tasks.workunit.client.1.smithi096.stdout:9/372: rename d9/db/d6d/d31 to d9/d15/d26/d29/d73 0 2024-03-24T10:46:11.013 INFO:tasks.workunit.client.1.smithi096.stdout:7/435: symlink d0/d13/d32/d45/l92 0 2024-03-24T10:46:11.017 INFO:tasks.workunit.client.1.smithi096.stdout:3/399: creat d0/d13/d45/d7d/d39/d48/f86 x:0 0 0 2024-03-24T10:46:11.017 INFO:tasks.workunit.client.1.smithi096.stdout:3/400: getdents d0/d13/d45/d52 0 2024-03-24T10:46:11.017 INFO:tasks.workunit.client.1.smithi096.stdout:3/401: chown d0/d13/d45/d7d/d39/d48/l4b 4671 1 2024-03-24T10:46:11.017 INFO:tasks.workunit.client.1.smithi096.stdout:3/402: chown d0/d13/d1e/f2d 516700869 1 2024-03-24T10:46:11.017 INFO:tasks.workunit.client.1.smithi096.stdout:3/403: dread - d0/d13/d45/d52/f74 zero size 2024-03-24T10:46:11.020 INFO:tasks.workunit.client.1.smithi096.stdout:2/253: rename d5/de/f1a to d5/dc/f6c 0 2024-03-24T10:46:11.020 INFO:tasks.workunit.client.1.smithi096.stdout:2/254: chown d5/dc/f21 7 1 2024-03-24T10:46:11.021 INFO:tasks.workunit.client.1.smithi096.stdout:3/404: symlink d0/d13/l87 0 2024-03-24T10:46:11.021 INFO:tasks.workunit.client.1.smithi096.stdout:9/373: read d9/db/d6d/f56 [2972827,67856] 0 2024-03-24T10:46:11.024 INFO:tasks.workunit.client.1.smithi096.stdout:5/322: rename d2/d7/d8/df/f1f to d2/d7/d8/d18/f72 0 2024-03-24T10:46:11.025 INFO:tasks.workunit.client.1.smithi096.stdout:5/323: dread d2/d49/f5d [0,4194304] 0 2024-03-24T10:46:11.025 INFO:tasks.workunit.client.1.smithi096.stdout:2/255: mknod d5/dc/d6b/c6d 0 2024-03-24T10:46:11.025 INFO:tasks.workunit.client.1.smithi096.stdout:2/256: fsync d5/d38/f45 0 2024-03-24T10:46:11.025 INFO:tasks.workunit.client.1.smithi096.stdout:2/257: fsync d5/de/d2a/f40 0 2024-03-24T10:46:11.027 INFO:tasks.workunit.client.1.smithi096.stdout:3/405: mkdir d0/d13/d45/d52/d88 0 2024-03-24T10:46:11.027 INFO:tasks.workunit.client.1.smithi096.stdout:5/324: write d2/f39 [1161253,45177] 0 2024-03-24T10:46:11.028 INFO:tasks.workunit.client.1.smithi096.stdout:2/258: mknod d5/de/d2a/d5e/c6e 0 2024-03-24T10:46:11.030 INFO:tasks.workunit.client.1.smithi096.stdout:7/436: dwrite d0/db/d1c/d39/f76 [0,4194304] 0 2024-03-24T10:46:11.030 INFO:tasks.workunit.client.1.smithi096.stdout:7/437: chown d0/db/l90 68240 1 2024-03-24T10:46:11.031 INFO:tasks.workunit.client.1.smithi096.stdout:3/406: rename d0/d13/d45/d7d/d39/d48/f5f to d0/d13/d45/d47/f89 0 2024-03-24T10:46:11.032 INFO:tasks.workunit.client.1.smithi096.stdout:5/325: mknod d2/d7/d8/d18/d4d/c73 0 2024-03-24T10:46:11.032 INFO:tasks.workunit.client.1.smithi096.stdout:5/326: write d2/d7/f3e [1739735,66359] 0 2024-03-24T10:46:11.033 INFO:tasks.workunit.client.1.smithi096.stdout:2/259: creat d5/d38/f6f x:0 0 0 2024-03-24T10:46:11.038 INFO:tasks.workunit.client.1.smithi096.stdout:7/438: dread d0/db/d26/d48/f8d [0,4194304] 0 2024-03-24T10:46:11.038 INFO:tasks.workunit.client.1.smithi096.stdout:7/439: dread - d0/db/d4f/f65 zero size 2024-03-24T10:46:11.038 INFO:tasks.workunit.client.1.smithi096.stdout:7/440: fsync d0/db/d4f/f74 0 2024-03-24T10:46:11.038 INFO:tasks.workunit.client.1.smithi096.stdout:7/441: stat d0/db/d4f/l7b 0 2024-03-24T10:46:11.040 INFO:tasks.workunit.client.1.smithi096.stdout:3/407: symlink d0/d13/d45/d7d/d71/l8a 0 2024-03-24T10:46:11.042 INFO:tasks.workunit.client.1.smithi096.stdout:0/361: truncate d0/d11/d24/f55 2311818 0 2024-03-24T10:46:11.044 INFO:tasks.workunit.client.1.smithi096.stdout:5/327: truncate d2/d29/d32/d33/f44 8145381 0 2024-03-24T10:46:11.045 INFO:tasks.workunit.client.1.smithi096.stdout:5/328: chown d2/d7/d8/df/f25 89772 1 2024-03-24T10:46:11.045 INFO:tasks.workunit.client.1.smithi096.stdout:5/329: chown d2/d7/f52 109758 1 2024-03-24T10:46:11.046 INFO:tasks.workunit.client.1.smithi096.stdout:2/260: creat d5/dc/d12/d23/d4f/f70 x:0 0 0 2024-03-24T10:46:11.048 INFO:tasks.workunit.client.1.smithi096.stdout:7/442: creat d0/d13/d60/f93 x:0 0 0 2024-03-24T10:46:11.050 INFO:tasks.workunit.client.1.smithi096.stdout:0/362: truncate d0/f5 1044873 0 2024-03-24T10:46:11.050 INFO:tasks.workunit.client.1.smithi096.stdout:0/363: dread - d0/d11/d20/d65/f6e zero size 2024-03-24T10:46:11.054 INFO:tasks.workunit.client.1.smithi096.stdout:7/443: mknod d0/d13/c94 0 2024-03-24T10:46:11.056 INFO:tasks.workunit.client.1.smithi096.stdout:2/261: link d5/d38/c3e d5/de/c71 0 2024-03-24T10:46:11.057 INFO:tasks.workunit.client.1.smithi096.stdout:7/444: symlink d0/l95 0 2024-03-24T10:46:11.062 INFO:tasks.workunit.client.1.smithi096.stdout:7/445: unlink d0/d13/c94 0 2024-03-24T10:46:11.062 INFO:tasks.workunit.client.1.smithi096.stdout:7/446: fsync d0/db/d26/f37 0 2024-03-24T10:46:11.062 INFO:tasks.workunit.client.1.smithi096.stdout:7/447: rename d0/d13/d60 to d0/d13/d60/d96 22 2024-03-24T10:46:11.062 INFO:tasks.workunit.client.1.smithi096.stdout:7/448: dread - d0/db/d4f/f65 zero size 2024-03-24T10:46:11.062 INFO:tasks.workunit.client.1.smithi096.stdout:7/449: chown d0/l1d 204429 1 2024-03-24T10:46:11.063 INFO:tasks.workunit.client.1.smithi096.stdout:7/450: readlink d0/d13/l4 0 2024-03-24T10:46:11.063 INFO:tasks.workunit.client.1.smithi096.stdout:7/451: write d0/db/d26/f31 [4401371,90415] 0 2024-03-24T10:46:11.063 INFO:tasks.workunit.client.1.smithi096.stdout:7/452: chown d0/d21/d3c 104640718 1 2024-03-24T10:46:11.063 INFO:tasks.workunit.client.1.smithi096.stdout:3/408: dwrite d0/d13/f21 [0,4194304] 0 2024-03-24T10:46:11.063 INFO:tasks.workunit.client.1.smithi096.stdout:3/409: creat d0/d9/f8b x:0 0 0 2024-03-24T10:46:11.064 INFO:tasks.workunit.client.1.smithi096.stdout:3/410: write d0/d9/d22/f3d [433739,86519] 0 2024-03-24T10:46:11.064 INFO:tasks.workunit.client.1.smithi096.stdout:3/411: dread - d0/d9/f8b zero size 2024-03-24T10:46:11.064 INFO:tasks.workunit.client.1.smithi096.stdout:3/412: write d0/d13/f57 [1972009,79732] 0 2024-03-24T10:46:11.064 INFO:tasks.workunit.client.1.smithi096.stdout:2/262: getdents d5/de/d2a 0 2024-03-24T10:46:11.064 INFO:tasks.workunit.client.1.smithi096.stdout:2/263: write d5/dc/d12/f48 [362470,116581] 0 2024-03-24T10:46:11.064 INFO:tasks.workunit.client.1.smithi096.stdout:2/264: chown d5/de/d2a/d5e 8183 1 2024-03-24T10:46:11.066 INFO:tasks.workunit.client.1.smithi096.stdout:0/364: dwrite d0/d11/d24/d2c/f32 [0,4194304] 0 2024-03-24T10:46:11.069 INFO:tasks.workunit.client.1.smithi096.stdout:3/413: unlink d0/c11 0 2024-03-24T10:46:11.070 INFO:tasks.workunit.client.1.smithi096.stdout:2/265: mkdir d5/dc/d6b/d72 0 2024-03-24T10:46:11.080 INFO:tasks.workunit.client.1.smithi096.stdout:0/365: symlink d0/d11/d24/d2c/l77 0 2024-03-24T10:46:11.083 INFO:tasks.workunit.client.1.smithi096.stdout:3/414: write d0/d9/f49 [3823534,61728] 0 2024-03-24T10:46:11.083 INFO:tasks.workunit.client.1.smithi096.stdout:3/415: chown d0/d13/d45/d7d/d39/d48 375894 1 2024-03-24T10:46:11.084 INFO:tasks.workunit.client.1.smithi096.stdout:0/366: mkdir d0/d11/d16/d21/d78 0 2024-03-24T10:46:11.087 INFO:tasks.workunit.client.1.smithi096.stdout:3/416: symlink d0/d9/d22/l8c 0 2024-03-24T10:46:11.087 INFO:tasks.workunit.client.1.smithi096.stdout:3/417: chown d0/d13/d45/d47 1416 1 2024-03-24T10:46:11.090 INFO:tasks.workunit.client.1.smithi096.stdout:0/367: rename d0/d11/l1d to d0/d11/d43/d75/l79 0 2024-03-24T10:46:11.090 INFO:tasks.workunit.client.1.smithi096.stdout:0/368: chown d0/d11/d20/d2b/c74 2 1 2024-03-24T10:46:11.094 INFO:tasks.workunit.client.1.smithi096.stdout:3/418: rename d0/d13/d45/d47/d7e to d0/d13/d45/d7d/d39/d48/d8d 0 2024-03-24T10:46:11.094 INFO:tasks.workunit.client.1.smithi096.stdout:3/419: fsync d0/d13/d45/d7d/f63 0 2024-03-24T10:46:11.094 INFO:tasks.workunit.client.1.smithi096.stdout:3/420: creat d0/d13/d45/d47/f8e x:0 0 0 2024-03-24T10:46:11.103 INFO:tasks.workunit.client.1.smithi096.stdout:8/386: sync 2024-03-24T10:46:11.103 INFO:tasks.workunit.client.1.smithi096.stdout:4/309: sync 2024-03-24T10:46:11.103 INFO:tasks.workunit.client.1.smithi096.stdout:4/310: truncate d1/d32/d42/d44/f57 1001032 0 2024-03-24T10:46:11.103 INFO:tasks.workunit.client.1.smithi096.stdout:6/359: sync 2024-03-24T10:46:11.103 INFO:tasks.workunit.client.1.smithi096.stdout:1/320: sync 2024-03-24T10:46:11.104 INFO:tasks.workunit.client.1.smithi096.stdout:0/369: dread d0/f60 [0,4194304] 0 2024-03-24T10:46:11.104 INFO:tasks.workunit.client.1.smithi096.stdout:0/370: truncate d0/d11/d20/d65/f6e 590968 0 2024-03-24T10:46:11.104 INFO:tasks.workunit.client.1.smithi096.stdout:2/266: dwrite d5/dc/f47 [0,4194304] 0 2024-03-24T10:46:11.104 INFO:tasks.workunit.client.1.smithi096.stdout:7/453: dwrite d0/db/f8c [0,4194304] 0 2024-03-24T10:46:11.109 INFO:tasks.workunit.client.1.smithi096.stdout:8/387: write d8/d11/d22/d4c/d65/d1d/f30 [1350636,35971] 0 2024-03-24T10:46:11.111 INFO:tasks.workunit.client.1.smithi096.stdout:1/321: mkdir d3/d23/d71 0 2024-03-24T10:46:11.111 INFO:tasks.workunit.client.1.smithi096.stdout:1/322: fsync d3/d36/f54 0 2024-03-24T10:46:11.111 INFO:tasks.workunit.client.1.smithi096.stdout:1/323: chown d3/d24/f6b 10626138 1 2024-03-24T10:46:11.111 INFO:tasks.workunit.client.1.smithi096.stdout:1/324: readlink d3/d23/d2f/l2a 0 2024-03-24T10:46:11.112 INFO:tasks.workunit.client.1.smithi096.stdout:1/325: fsync d3/d23/d2f/f37 0 2024-03-24T10:46:11.114 INFO:tasks.workunit.client.1.smithi096.stdout:6/360: dread d8/d10/d19/f34 [0,4194304] 0 2024-03-24T10:46:11.120 INFO:tasks.workunit.client.1.smithi096.stdout:0/371: mknod d0/d11/d16/d21/d25/d47/d63/c7a 0 2024-03-24T10:46:11.121 INFO:tasks.workunit.client.1.smithi096.stdout:2/267: mknod d5/dc/d12/d23/c73 0 2024-03-24T10:46:11.122 INFO:tasks.workunit.client.1.smithi096.stdout:3/421: dwrite d0/d13/f57 [4194304,4194304] 0 2024-03-24T10:46:11.122 INFO:tasks.workunit.client.1.smithi096.stdout:3/422: chown d0/d13/d45/d47 1592 1 2024-03-24T10:46:11.122 INFO:tasks.workunit.client.1.smithi096.stdout:4/311: dwrite d1/f35 [0,4194304] 0 2024-03-24T10:46:11.125 INFO:tasks.workunit.client.1.smithi096.stdout:7/454: symlink d0/d21/d3c/d68/d4b/d62/l97 0 2024-03-24T10:46:11.125 INFO:tasks.workunit.client.1.smithi096.stdout:7/455: fsync d0/d13/d32/d45/f63 0 2024-03-24T10:46:11.126 INFO:tasks.workunit.client.1.smithi096.stdout:1/326: unlink d3/d24/f6b 0 2024-03-24T10:46:11.127 INFO:tasks.workunit.client.1.smithi096.stdout:4/312: dread d1/d11/f41 [0,4194304] 0 2024-03-24T10:46:11.127 INFO:tasks.workunit.client.1.smithi096.stdout:4/313: fdatasync d1/d2b/d40/f64 0 2024-03-24T10:46:11.128 INFO:tasks.workunit.client.1.smithi096.stdout:6/361: rename d8/d12/d28/l39 to d8/d10/d13/d63/l6c 0 2024-03-24T10:46:11.128 INFO:tasks.workunit.client.1.smithi096.stdout:6/362: read - d8/d10/f3e zero size 2024-03-24T10:46:11.128 INFO:tasks.workunit.client.1.smithi096.stdout:6/363: getdents d8/d12/d61 0 2024-03-24T10:46:11.129 INFO:tasks.workunit.client.1.smithi096.stdout:0/372: symlink d0/d11/d20/l7b 0 2024-03-24T10:46:11.130 INFO:tasks.workunit.client.1.smithi096.stdout:4/314: dread d1/fe [0,4194304] 0 2024-03-24T10:46:11.139 INFO:tasks.workunit.client.1.smithi096.stdout:3/423: mkdir d0/d13/d45/d47/d8f 0 2024-03-24T10:46:11.139 INFO:tasks.workunit.client.1.smithi096.stdout:3/424: chown d0/d9/d22/c40 40984 1 2024-03-24T10:46:11.139 INFO:tasks.workunit.client.1.smithi096.stdout:3/425: creat d0/d13/d77/f90 x:0 0 0 2024-03-24T10:46:11.139 INFO:tasks.workunit.client.1.smithi096.stdout:3/426: truncate d0/d9/d22/f3d 921165 0 2024-03-24T10:46:11.140 INFO:tasks.workunit.client.1.smithi096.stdout:7/456: unlink d0/l3 0 2024-03-24T10:46:11.143 INFO:tasks.workunit.client.1.smithi096.stdout:2/268: rename d5/dc/d12/d23/l33 to d5/dc/d6b/l74 0 2024-03-24T10:46:11.145 INFO:tasks.workunit.client.1.smithi096.stdout:0/373: creat d0/d11/d16/d21/d25/d47/f7c x:0 0 0 2024-03-24T10:46:11.145 INFO:tasks.workunit.client.1.smithi096.stdout:0/374: creat d0/d11/d16/d21/d25/d47/f7d x:0 0 0 2024-03-24T10:46:11.148 INFO:tasks.workunit.client.1.smithi096.stdout:7/457: write d0/f38 [279512,50538] 0 2024-03-24T10:46:11.148 INFO:tasks.workunit.client.1.smithi096.stdout:7/458: fsync d0/f14 0 2024-03-24T10:46:11.148 INFO:tasks.workunit.client.1.smithi096.stdout:4/315: readlink d1/d2b/l53 0 2024-03-24T10:46:11.151 INFO:tasks.workunit.client.1.smithi096.stdout:3/427: write d0/f4 [3313007,10718] 0 2024-03-24T10:46:11.153 INFO:tasks.workunit.client.1.smithi096.stdout:2/269: dread f3 [0,4194304] 0 2024-03-24T10:46:11.154 INFO:tasks.workunit.client.1.smithi096.stdout:4/316: dread d1/d32/d4c/f50 [0,4194304] 0 2024-03-24T10:46:11.154 INFO:tasks.workunit.client.1.smithi096.stdout:4/317: write d1/f10 [1602857,113284] 0 2024-03-24T10:46:11.154 INFO:tasks.workunit.client.1.smithi096.stdout:6/364: dwrite d8/d10/d13/d63/f5e [0,4194304] 0 2024-03-24T10:46:11.154 INFO:tasks.workunit.client.1.smithi096.stdout:6/365: fdatasync d8/d10/d13/d63/f21 0 2024-03-24T10:46:11.155 INFO:tasks.workunit.client.1.smithi096.stdout:1/327: dwrite d3/f66 [0,4194304] 0 2024-03-24T10:46:11.155 INFO:tasks.workunit.client.1.smithi096.stdout:1/328: readlink d3/d24/l52 0 2024-03-24T10:46:11.155 INFO:tasks.workunit.client.1.smithi096.stdout:0/375: mknod d0/d11/d24/d2c/c7e 0 2024-03-24T10:46:11.160 INFO:tasks.workunit.client.1.smithi096.stdout:7/459: symlink d0/d21/d3c/d68/d85/l98 0 2024-03-24T10:46:11.160 INFO:tasks.workunit.client.1.smithi096.stdout:7/460: stat d0/d21/f8f 0 2024-03-24T10:46:11.169 INFO:tasks.workunit.client.1.smithi096.stdout:6/366: truncate d8/d12/d28/f53 7362455 0 2024-03-24T10:46:11.169 INFO:tasks.workunit.client.1.smithi096.stdout:6/367: stat d8/d12/f69 0 2024-03-24T10:46:11.176 INFO:tasks.workunit.client.1.smithi096.stdout:1/329: dwrite d3/d23/d2f/f4a [0,4194304] 0 2024-03-24T10:46:11.178 INFO:tasks.workunit.client.1.smithi096.stdout:1/330: dread d3/d24/f35 [0,4194304] 0 2024-03-24T10:46:11.178 INFO:tasks.workunit.client.1.smithi096.stdout:1/331: read d3/d36/f4d [129346,91105] 0 2024-03-24T10:46:11.178 INFO:tasks.workunit.client.1.smithi096.stdout:1/332: chown d3/d24/f39 29401 1 2024-03-24T10:46:11.178 INFO:tasks.workunit.client.1.smithi096.stdout:1/333: write d3/d23/d2f/f5a [614717,114352] 0 2024-03-24T10:46:11.178 INFO:tasks.workunit.client.1.smithi096.stdout:1/334: fdatasync d3/ff 0 2024-03-24T10:46:11.182 INFO:tasks.workunit.client.1.smithi096.stdout:6/368: dread d8/d12/d28/d36/f45 [0,4194304] 0 2024-03-24T10:46:11.183 INFO:tasks.workunit.client.1.smithi096.stdout:2/270: dwrite d5/dc/d12/f31 [0,4194304] 0 2024-03-24T10:46:11.183 INFO:tasks.workunit.client.1.smithi096.stdout:2/271: write d5/dc/f25 [3218582,5045] 0 2024-03-24T10:46:11.183 INFO:tasks.workunit.client.1.smithi096.stdout:7/461: dwrite d0/d13/d32/f3b [4194304,4194304] 0 2024-03-24T10:46:11.191 INFO:tasks.workunit.client.1.smithi096.stdout:2/272: write d5/dc/d12/f20 [1694512,35507] 0 2024-03-24T10:46:11.199 INFO:tasks.workunit.client.1.smithi096.stdout:0/376: rmdir d0/d11 39 2024-03-24T10:46:11.200 INFO:tasks.workunit.client.1.smithi096.stdout:1/335: mknod d3/d23/d71/c72 0 2024-03-24T10:46:11.209 INFO:tasks.workunit.client.1.smithi096.stdout:6/369: dwrite d8/f62 [0,4194304] 0 2024-03-24T10:46:11.209 INFO:tasks.workunit.client.1.smithi096.stdout:6/370: write d8/d12/d28/f53 [7381883,114952] 0 2024-03-24T10:46:11.209 INFO:tasks.workunit.client.1.smithi096.stdout:7/462: dwrite d0/d21/d3c/d68/f4d [8388608,4194304] 0 2024-03-24T10:46:11.209 INFO:tasks.workunit.client.1.smithi096.stdout:2/273: link d5/d38/c55 d5/dc/d12/d23/c75 0 2024-03-24T10:46:11.209 INFO:tasks.workunit.client.1.smithi096.stdout:2/274: read d5/dc/d12/f20 [3057358,116096] 0 2024-03-24T10:46:11.210 INFO:tasks.workunit.client.1.smithi096.stdout:2/275: creat d5/d38/f76 x:0 0 0 2024-03-24T10:46:11.211 INFO:tasks.workunit.client.1.smithi096.stdout:1/336: truncate d3/d23/d2f/f59 4688 0 2024-03-24T10:46:11.213 INFO:tasks.workunit.client.1.smithi096.stdout:4/318: rename d1/d32/d42/d44/d45 to d1/d2b/d67 0 2024-03-24T10:46:11.218 INFO:tasks.workunit.client.1.smithi096.stdout:2/276: creat d5/dc/d12/d23/f77 x:0 0 0 2024-03-24T10:46:11.218 INFO:tasks.workunit.client.1.smithi096.stdout:2/277: write d5/dc/f27 [1338103,61473] 0 2024-03-24T10:46:11.220 INFO:tasks.workunit.client.1.smithi096.stdout:0/377: dwrite d0/d2/f4a [0,4194304] 0 2024-03-24T10:46:11.220 INFO:tasks.workunit.client.1.smithi096.stdout:1/337: dread d3/f15 [0,4194304] 0 2024-03-24T10:46:11.222 INFO:tasks.workunit.client.1.smithi096.stdout:7/463: rmdir d0/d21/d59/d87 39 2024-03-24T10:46:11.224 INFO:tasks.workunit.client.1.smithi096.stdout:2/278: truncate d5/dc/f21 3347812 0 2024-03-24T10:46:11.224 INFO:tasks.workunit.client.1.smithi096.stdout:2/279: fdatasync d5/de/d2a/f62 0 2024-03-24T10:46:11.224 INFO:tasks.workunit.client.1.smithi096.stdout:2/280: write d5/de/d2a/f32 [658361,10916] 0 2024-03-24T10:46:11.226 INFO:tasks.workunit.client.1.smithi096.stdout:1/338: mkdir d3/d73 0 2024-03-24T10:46:11.229 INFO:tasks.workunit.client.1.smithi096.stdout:7/464: mkdir d0/d13/d32/d99 0 2024-03-24T10:46:11.254 INFO:tasks.workunit.client.1.smithi096.stdout:7/465: dwrite d0/d13/f5 [0,4194304] 0 2024-03-24T10:46:11.255 INFO:tasks.workunit.client.1.smithi096.stdout:2/281: dwrite d5/fa [0,4194304] 0 2024-03-24T10:46:11.255 INFO:tasks.workunit.client.1.smithi096.stdout:2/282: getdents d5/de/d2a 0 2024-03-24T10:46:11.272 INFO:tasks.workunit.client.1.smithi096.stdout:7/466: dwrite d0/d13/f9 [0,4194304] 0 2024-03-24T10:46:11.272 INFO:tasks.workunit.client.1.smithi096.stdout:7/467: read - d0/db/d26/f37 zero size 2024-03-24T10:46:11.275 INFO:tasks.workunit.client.1.smithi096.stdout:7/468: symlink d0/db/d73/l9a 0 2024-03-24T10:46:11.277 INFO:tasks.workunit.client.1.smithi096.stdout:7/469: dread d0/db/d1c/d39/f4a [0,4194304] 0 2024-03-24T10:46:11.280 INFO:tasks.workunit.client.1.smithi096.stdout:7/470: link d0/d21/d3c/d68/d4b/d62/l97 d0/d21/d3c/d68/d85/l9b 0 2024-03-24T10:46:11.280 INFO:tasks.workunit.client.1.smithi096.stdout:7/471: creat d0/db/d1c/d77/f9c x:0 0 0 2024-03-24T10:46:11.280 INFO:tasks.workunit.client.1.smithi096.stdout:7/472: chown d0/db/f72 411041 1 2024-03-24T10:46:11.281 INFO:tasks.workunit.client.1.smithi096.stdout:7/473: truncate d0/db/d26/f55 1018285 0 2024-03-24T10:46:11.281 INFO:tasks.workunit.client.1.smithi096.stdout:7/474: truncate d0/d21/d3c/d68/d85/f8e 1011523 0 2024-03-24T10:46:11.281 INFO:tasks.workunit.client.1.smithi096.stdout:7/475: creat d0/d13/d60/f9d x:0 0 0 2024-03-24T10:46:11.296 INFO:tasks.workunit.client.1.smithi096.stdout:7/476: dwrite d0/db/d26/f44 [0,4194304] 0 2024-03-24T10:46:11.296 INFO:tasks.workunit.client.1.smithi096.stdout:7/477: truncate d0/db/d1c/d39/f76 4902558 0 2024-03-24T10:46:11.296 INFO:tasks.workunit.client.1.smithi096.stdout:7/478: write d0/d21/d3c/f71 [738413,99104] 0 2024-03-24T10:46:11.296 INFO:tasks.workunit.client.1.smithi096.stdout:7/479: fdatasync d0/d21/f8f 0 2024-03-24T10:46:11.299 INFO:tasks.workunit.client.1.smithi096.stdout:7/480: creat d0/f9e x:0 0 0 2024-03-24T10:46:11.299 INFO:tasks.workunit.client.1.smithi096.stdout:7/481: truncate d0/db/d26/f5b 2276528 0 2024-03-24T10:46:11.326 INFO:tasks.workunit.client.1.smithi096.stdout:9/374: sync 2024-03-24T10:46:11.328 INFO:tasks.workunit.client.1.smithi096.stdout:9/375: dread d9/d15/d26/d29/d73/d3e/f59 [0,4194304] 0 2024-03-24T10:46:11.328 INFO:tasks.workunit.client.1.smithi096.stdout:9/376: rmdir d9/db/d6d/d4f 39 2024-03-24T10:46:11.338 INFO:tasks.workunit.client.1.smithi096.stdout:7/482: dread d0/d13/d32/d45/f6c [0,4194304] 0 2024-03-24T10:46:11.338 INFO:tasks.workunit.client.1.smithi096.stdout:7/483: write d0/d21/d3c/f82 [546324,105430] 0 2024-03-24T10:46:11.340 INFO:tasks.workunit.client.1.smithi096.stdout:2/283: dwrite d5/fa [0,4194304] 0 2024-03-24T10:46:11.342 INFO:tasks.workunit.client.1.smithi096.stdout:2/284: unlink d5/de/d2a/l64 0 2024-03-24T10:46:11.342 INFO:tasks.workunit.client.1.smithi096.stdout:2/285: truncate d5/dc/f1c 59921 0 2024-03-24T10:46:11.344 INFO:tasks.workunit.client.1.smithi096.stdout:2/286: creat d5/de/d2a/f78 x:0 0 0 2024-03-24T10:46:11.355 INFO:tasks.workunit.client.1.smithi096.stdout:9/377: dwrite d9/d15/d26/d29/d73/d43/f44 [0,4194304] 0 2024-03-24T10:46:11.355 INFO:tasks.workunit.client.1.smithi096.stdout:7/484: dwrite d0/db/d26/f44 [0,4194304] 0 2024-03-24T10:46:11.362 INFO:tasks.workunit.client.1.smithi096.stdout:2/287: mkdir d5/d79 0 2024-03-24T10:46:11.367 INFO:tasks.workunit.client.1.smithi096.stdout:9/378: mknod d9/c74 0 2024-03-24T10:46:11.373 INFO:tasks.workunit.client.1.smithi096.stdout:7/485: dwrite d0/d21/f8b [0,4194304] 0 2024-03-24T10:46:11.373 INFO:tasks.workunit.client.1.smithi096.stdout:7/486: truncate d0/d21/d59/d87/f8a 75433 0 2024-03-24T10:46:11.373 INFO:tasks.workunit.client.1.smithi096.stdout:7/487: chown d0/d21/d3c/c46 51252313 1 2024-03-24T10:46:11.374 INFO:tasks.workunit.client.1.smithi096.stdout:7/488: read d0/d13/d32/d45/f63 [282539,65175] 0 2024-03-24T10:46:11.377 INFO:tasks.workunit.client.1.smithi096.stdout:9/379: dread d9/d15/d26/d29/f42 [0,4194304] 0 2024-03-24T10:46:11.377 INFO:tasks.workunit.client.1.smithi096.stdout:9/380: fsync d9/d15/f1e 0 2024-03-24T10:46:11.384 INFO:tasks.workunit.client.1.smithi096.stdout:7/489: dread d0/db/f6d [4194304,4194304] 0 2024-03-24T10:46:11.386 INFO:tasks.workunit.client.1.smithi096.stdout:7/490: creat d0/db/d1c/f9f x:0 0 0 2024-03-24T10:46:11.386 INFO:tasks.workunit.client.1.smithi096.stdout:7/491: chown d0/db/l3f 5409 1 2024-03-24T10:46:11.388 INFO:tasks.workunit.client.1.smithi096.stdout:7/492: unlink d0/d13/d32/c50 0 2024-03-24T10:46:11.390 INFO:tasks.workunit.client.1.smithi096.stdout:7/493: rename d0/db/d73 to d0/d21/d59/d87/da0 0 2024-03-24T10:46:11.392 INFO:tasks.workunit.client.1.smithi096.stdout:7/494: rename d0/d21/d3c/d68/d85/l9b to d0/d21/d59/d87/da0/la1 0 2024-03-24T10:46:11.394 INFO:tasks.workunit.client.1.smithi096.stdout:7/495: mkdir d0/db/d1c/d39/da2 0 2024-03-24T10:46:11.395 INFO:tasks.workunit.client.1.smithi096.stdout:7/496: symlink d0/d13/d32/la3 0 2024-03-24T10:46:11.395 INFO:tasks.workunit.client.1.smithi096.stdout:7/497: write d0/db/f6d [2340712,114974] 0 2024-03-24T10:46:11.397 INFO:tasks.workunit.client.1.smithi096.stdout:7/498: rename d0/d21/d59/f7d to d0/d21/d3c/d68/d4b/fa4 0 2024-03-24T10:46:11.398 INFO:tasks.workunit.client.1.smithi096.stdout:7/499: symlink d0/db/d4f/la5 0 2024-03-24T10:46:11.399 INFO:tasks.workunit.client.1.smithi096.stdout:7/500: write d0/db/d26/f55 [894131,85465] 0 2024-03-24T10:46:11.400 INFO:tasks.workunit.client.1.smithi096.stdout:7/501: symlink d0/d21/la6 0 2024-03-24T10:46:11.401 INFO:tasks.workunit.client.1.smithi096.stdout:7/502: write d0/db/d26/f5b [2057209,26133] 0 2024-03-24T10:46:11.404 INFO:tasks.workunit.client.1.smithi096.stdout:2/288: dwrite d5/d38/f3a [0,4194304] 0 2024-03-24T10:46:11.405 INFO:tasks.workunit.client.1.smithi096.stdout:9/381: dwrite d9/f1a [0,4194304] 0 2024-03-24T10:46:11.409 INFO:tasks.workunit.client.1.smithi096.stdout:7/503: dread d0/d13/f9 [0,4194304] 0 2024-03-24T10:46:11.409 INFO:tasks.workunit.client.1.smithi096.stdout:7/504: truncate d0/d21/f8f 782419 0 2024-03-24T10:46:11.424 INFO:tasks.workunit.client.1.smithi096.stdout:7/505: dwrite d0/d21/d59/f79 [0,4194304] 0 2024-03-24T10:46:11.429 INFO:tasks.workunit.client.1.smithi096.stdout:2/289: rmdir d5/dc 39 2024-03-24T10:46:11.438 INFO:tasks.workunit.client.1.smithi096.stdout:2/290: rename d5/dc/d12/d23/f54 to d5/dc/d12/d23/f7a 0 2024-03-24T10:46:11.439 INFO:tasks.workunit.client.1.smithi096.stdout:7/506: truncate d0/db/d26/f55 198796 0 2024-03-24T10:46:11.441 INFO:tasks.workunit.client.1.smithi096.stdout:2/291: link d5/d38/f66 d5/d38/f7b 0 2024-03-24T10:46:11.446 INFO:tasks.workunit.client.1.smithi096.stdout:7/507: creat d0/d21/d3c/d68/fa7 x:0 0 0 2024-03-24T10:46:11.446 INFO:tasks.workunit.client.1.smithi096.stdout:7/508: write d0/db/d1c/d77/f9c [34169,98033] 0 2024-03-24T10:46:11.449 INFO:tasks.workunit.client.1.smithi096.stdout:7/509: rmdir d0/d21/d59/d87/da0 39 2024-03-24T10:46:11.449 INFO:tasks.workunit.client.1.smithi096.stdout:2/292: write d5/dc/d12/f4a [402055,22710] 0 2024-03-24T10:46:11.449 INFO:tasks.workunit.client.1.smithi096.stdout:2/293: stat d5/d38/l57 0 2024-03-24T10:46:11.450 INFO:tasks.workunit.client.1.smithi096.stdout:2/294: write f2 [661397,14661] 0 2024-03-24T10:46:11.452 INFO:tasks.workunit.client.1.smithi096.stdout:7/510: mkdir d0/db/d26/d48/da8 0 2024-03-24T10:46:11.452 INFO:tasks.workunit.client.1.smithi096.stdout:2/295: dread d5/d38/f45 [0,4194304] 0 2024-03-24T10:46:11.456 INFO:tasks.workunit.client.1.smithi096.stdout:2/296: mkdir d5/d7c 0 2024-03-24T10:46:11.460 INFO:tasks.workunit.client.1.smithi096.stdout:7/511: write d0/db/d26/f44 [2485853,71614] 0 2024-03-24T10:46:11.460 INFO:tasks.workunit.client.1.smithi096.stdout:2/297: rmdir d5/de/d2a/d5e 39 2024-03-24T10:46:11.463 INFO:tasks.workunit.client.1.smithi096.stdout:7/512: mkdir d0/da9 0 2024-03-24T10:46:11.463 INFO:tasks.workunit.client.1.smithi096.stdout:7/513: stat d0/d13/l40 0 2024-03-24T10:46:11.464 INFO:tasks.workunit.client.1.smithi096.stdout:2/298: symlink d5/dc/d12/l7d 0 2024-03-24T10:46:11.464 INFO:tasks.workunit.client.1.smithi096.stdout:2/299: write d5/dc/d12/f4a [4813435,57501] 0 2024-03-24T10:46:11.465 INFO:tasks.workunit.client.1.smithi096.stdout:7/514: creat d0/d13/d32/faa x:0 0 0 2024-03-24T10:46:11.468 INFO:tasks.workunit.client.1.smithi096.stdout:2/300: creat d5/d7c/f7e x:0 0 0 2024-03-24T10:46:11.468 INFO:tasks.workunit.client.1.smithi096.stdout:7/515: truncate d0/d13/d60/f86 674029 0 2024-03-24T10:46:11.468 INFO:tasks.workunit.client.1.smithi096.stdout:7/516: fsync d0/d21/d59/f83 0 2024-03-24T10:46:11.468 INFO:tasks.workunit.client.1.smithi096.stdout:7/517: write d0/d13/d60/f9d [707740,28885] 0 2024-03-24T10:46:11.471 INFO:tasks.workunit.client.1.smithi096.stdout:7/518: unlink d0/d13/f9 0 2024-03-24T10:46:11.472 INFO:tasks.workunit.client.1.smithi096.stdout:7/519: mkdir d0/db/d1c/d77/dab 0 2024-03-24T10:46:11.477 INFO:tasks.workunit.client.1.smithi096.stdout:2/301: dread d5/de/d2a/d5e/f65 [0,4194304] 0 2024-03-24T10:46:11.477 INFO:tasks.workunit.client.1.smithi096.stdout:8/388: sync 2024-03-24T10:46:11.477 INFO:tasks.workunit.client.1.smithi096.stdout:5/330: sync 2024-03-24T10:46:11.477 INFO:tasks.workunit.client.1.smithi096.stdout:8/389: chown d8/d11/d22/d4c/d65/d1d 205029548 1 2024-03-24T10:46:11.477 INFO:tasks.workunit.client.1.smithi096.stdout:5/331: readlink d2/d7/l22 0 2024-03-24T10:46:11.477 INFO:tasks.workunit.client.1.smithi096.stdout:5/332: chown d2/d7/d8/df/d14/c24 1675364 1 2024-03-24T10:46:11.477 INFO:tasks.workunit.client.1.smithi096.stdout:5/333: dread - d2/d7/d8/df/d14/f5c zero size 2024-03-24T10:46:11.477 INFO:tasks.workunit.client.1.smithi096.stdout:2/302: truncate d5/de/f13 637486 0 2024-03-24T10:46:11.479 INFO:tasks.workunit.client.1.smithi096.stdout:8/390: truncate d8/f16 3344602 0 2024-03-24T10:46:11.479 INFO:tasks.workunit.client.1.smithi096.stdout:8/391: dread - d8/d11/d22/d4c/d65/d1a/d49/f6a zero size 2024-03-24T10:46:11.480 INFO:tasks.workunit.client.1.smithi096.stdout:7/520: write d0/d21/d3c/d68/f4d [4272313,88350] 0 2024-03-24T10:46:11.480 INFO:tasks.workunit.client.1.smithi096.stdout:5/334: creat d2/d29/d32/d33/d3c/f74 x:0 0 0 2024-03-24T10:46:11.483 INFO:tasks.workunit.client.1.smithi096.stdout:2/303: creat d5/de/d2a/d5e/f7f x:0 0 0 2024-03-24T10:46:11.483 INFO:tasks.workunit.client.1.smithi096.stdout:2/304: chown d5/de/d2a/f37 2093605109 1 2024-03-24T10:46:11.484 INFO:tasks.workunit.client.1.smithi096.stdout:8/392: rename d8/d11/d22/c63 to d8/c7e 0 2024-03-24T10:46:11.485 INFO:tasks.workunit.client.1.smithi096.stdout:7/521: symlink d0/d13/d32/lac 0 2024-03-24T10:46:11.486 INFO:tasks.workunit.client.1.smithi096.stdout:5/335: creat d2/d29/d32/d33/d3c/d3d/f75 x:0 0 0 2024-03-24T10:46:11.487 INFO:tasks.workunit.client.1.smithi096.stdout:2/305: symlink d5/d7c/l80 0 2024-03-24T10:46:11.490 INFO:tasks.workunit.client.1.smithi096.stdout:8/393: rmdir d8 39 2024-03-24T10:46:11.490 INFO:tasks.workunit.client.1.smithi096.stdout:8/394: readlink d8/d11/d22/d4c/l4f 0 2024-03-24T10:46:11.492 INFO:tasks.workunit.client.1.smithi096.stdout:7/522: unlink d0/d21/f2e 0 2024-03-24T10:46:11.492 INFO:tasks.workunit.client.1.smithi096.stdout:7/523: fsync d0/d13/f3d 0 2024-03-24T10:46:11.494 INFO:tasks.workunit.client.1.smithi096.stdout:8/395: creat d8/d11/f7f x:0 0 0 2024-03-24T10:46:11.498 INFO:tasks.workunit.client.1.smithi096.stdout:8/396: write d8/f16 [1128168,117661] 0 2024-03-24T10:46:11.498 INFO:tasks.workunit.client.1.smithi096.stdout:8/397: write d8/d11/d22/d4c/d65/d1a/d46/d2e/f54 [1379694,54171] 0 2024-03-24T10:46:11.498 INFO:tasks.workunit.client.1.smithi096.stdout:8/398: chown d8/d11/d22/d4c/d65/d1a/d46/d2e/f54 243618 1 2024-03-24T10:46:11.506 INFO:tasks.workunit.client.1.smithi096.stdout:7/524: rename d0/d21/c57 to d0/d13/d60/cad 0 2024-03-24T10:46:11.509 INFO:tasks.workunit.client.1.smithi096.stdout:8/399: rename d8/d11/d22/d4c/d65/d1d/f28 to d8/d11/d22/d4c/d65/d1a/f80 0 2024-03-24T10:46:11.510 INFO:tasks.workunit.client.1.smithi096.stdout:7/525: symlink d0/db/d1c/d39/lae 0 2024-03-24T10:46:11.510 INFO:tasks.workunit.client.1.smithi096.stdout:7/526: fsync d0/d21/d3c/d68/d4b/fa4 0 2024-03-24T10:46:11.511 INFO:tasks.workunit.client.1.smithi096.stdout:5/336: dwrite d2/d7/d8/df/d14/f31 [0,4194304] 0 2024-03-24T10:46:11.513 INFO:tasks.workunit.client.1.smithi096.stdout:8/400: rename d8/d11/d22/d4c/d65/c36 to d8/d11/d44/d45/c81 0 2024-03-24T10:46:11.516 INFO:tasks.workunit.client.1.smithi096.stdout:5/337: symlink d2/d29/d32/d33/l76 0 2024-03-24T10:46:11.520 INFO:tasks.workunit.client.1.smithi096.stdout:8/401: rename d8/d77 to d8/d11/d22/d4c/d82 0 2024-03-24T10:46:11.524 INFO:tasks.workunit.client.1.smithi096.stdout:7/527: dread d0/d21/d3c/d68/f4d [8388608,4194304] 0 2024-03-24T10:46:11.524 INFO:tasks.workunit.client.1.smithi096.stdout:7/528: truncate d0/db/d4f/f74 834869 0 2024-03-24T10:46:11.524 INFO:tasks.workunit.client.1.smithi096.stdout:7/529: creat d0/d21/d3c/faf x:0 0 0 2024-03-24T10:46:11.524 INFO:tasks.workunit.client.1.smithi096.stdout:7/530: write d0/db/d26/d48/f67 [279314,73898] 0 2024-03-24T10:46:11.529 INFO:tasks.workunit.client.1.smithi096.stdout:5/338: getdents d2/d49 0 2024-03-24T10:46:11.536 INFO:tasks.workunit.client.1.smithi096.stdout:5/339: mknod d2/d29/c77 0 2024-03-24T10:46:11.537 INFO:tasks.workunit.client.1.smithi096.stdout:7/531: getdents d0/d21/d3c/d68/d4b/d62 0 2024-03-24T10:46:11.539 INFO:tasks.workunit.client.1.smithi096.stdout:8/402: dwrite d8/d11/d22/d4c/d65/d1d/f3b [0,4194304] 0 2024-03-24T10:46:11.540 INFO:tasks.workunit.client.1.smithi096.stdout:8/403: fsync d8/d11/d22/d4c/d65/d1a/d46/d2e/f71 0 2024-03-24T10:46:11.540 INFO:tasks.workunit.client.1.smithi096.stdout:8/404: read d8/f14 [2520828,533] 0 2024-03-24T10:46:11.541 INFO:tasks.workunit.client.1.smithi096.stdout:7/532: rmdir d0/d13/d60 39 2024-03-24T10:46:11.541 INFO:tasks.workunit.client.1.smithi096.stdout:7/533: chown d0/f5a 2 1 2024-03-24T10:46:11.548 INFO:tasks.workunit.client.1.smithi096.stdout:8/405: stat d8/d11/d22/d4c/d65/d1d/f2b 0 2024-03-24T10:46:11.551 INFO:tasks.workunit.client.1.smithi096.stdout:5/340: dwrite d2/d29/d32/d3f/d5f/f6c [0,4194304] 0 2024-03-24T10:46:11.552 INFO:tasks.workunit.client.1.smithi096.stdout:7/534: symlink d0/d13/d32/lb0 0 2024-03-24T10:46:11.554 INFO:tasks.workunit.client.1.smithi096.stdout:8/406: creat d8/d72/f83 x:0 0 0 2024-03-24T10:46:11.559 INFO:tasks.workunit.client.1.smithi096.stdout:5/341: dread d2/d7/d8/df/d14/f31 [0,4194304] 0 2024-03-24T10:46:11.559 INFO:tasks.workunit.client.1.smithi096.stdout:5/342: write d2/d7/d8/f56 [340885,91083] 0 2024-03-24T10:46:11.563 INFO:tasks.workunit.client.1.smithi096.stdout:7/535: mkdir d0/db/d26/d48/da8/db1 0 2024-03-24T10:46:11.563 INFO:tasks.workunit.client.1.smithi096.stdout:7/536: readlink d0/d13/d32/l88 0 2024-03-24T10:46:11.564 INFO:tasks.workunit.client.1.smithi096.stdout:8/407: creat d8/d11/d44/f84 x:0 0 0 2024-03-24T10:46:11.566 INFO:tasks.workunit.client.1.smithi096.stdout:7/537: unlink d0/d21/d3c/d68/d4b/c52 0 2024-03-24T10:46:11.568 INFO:tasks.workunit.client.1.smithi096.stdout:8/408: mkdir d8/d11/d85 0 2024-03-24T10:46:11.568 INFO:tasks.workunit.client.1.smithi096.stdout:7/538: symlink d0/db/d4f/lb2 0 2024-03-24T10:46:11.571 INFO:tasks.workunit.client.1.smithi096.stdout:8/409: mkdir d8/d11/d22/d4c/d65/d1d/d86 0 2024-03-24T10:46:11.571 INFO:tasks.workunit.client.1.smithi096.stdout:8/410: write d8/d11/d22/d4c/d65/d1a/d49/f6a [807035,95196] 0 2024-03-24T10:46:11.571 INFO:tasks.workunit.client.1.smithi096.stdout:8/411: dread - d8/d11/d22/d4c/d82/f70 zero size 2024-03-24T10:46:11.573 INFO:tasks.workunit.client.1.smithi096.stdout:8/412: creat d8/d11/d22/d4c/d65/d1d/d86/f87 x:0 0 0 2024-03-24T10:46:11.575 INFO:tasks.workunit.client.1.smithi096.stdout:5/343: dwrite d2/d7/f3e [0,4194304] 0 2024-03-24T10:46:11.578 INFO:tasks.workunit.client.1.smithi096.stdout:5/344: rmdir d2/d7/d8/df 39 2024-03-24T10:46:11.578 INFO:tasks.workunit.client.1.smithi096.stdout:5/345: write d2/d29/d32/f40 [8578232,68605] 0 2024-03-24T10:46:11.578 INFO:tasks.workunit.client.1.smithi096.stdout:5/346: readlink d2/d29/d32/d33/d3c/d3d/l4e 0 2024-03-24T10:46:11.597 INFO:tasks.workunit.client.1.smithi096.stdout:7/539: dwrite d0/db/d26/f5b [0,4194304] 0 2024-03-24T10:46:11.600 INFO:tasks.workunit.client.1.smithi096.stdout:7/540: unlink d0/db/d26/f44 0 2024-03-24T10:46:11.600 INFO:tasks.workunit.client.1.smithi096.stdout:8/413: dwrite d8/d11/f7f [0,4194304] 0 2024-03-24T10:46:11.601 INFO:tasks.workunit.client.1.smithi096.stdout:5/347: dwrite d2/d7/d8/df/d14/f5c [0,4194304] 0 2024-03-24T10:46:11.606 INFO:tasks.workunit.client.1.smithi096.stdout:7/541: mknod d0/d21/d3c/d68/d85/cb3 0 2024-03-24T10:46:11.607 INFO:tasks.workunit.client.1.smithi096.stdout:7/542: dread - d0/db/d26/f37 zero size 2024-03-24T10:46:11.620 INFO:tasks.workunit.client.1.smithi096.stdout:3/428: sync 2024-03-24T10:46:11.628 INFO:tasks.workunit.client.1.smithi096.stdout:5/348: dwrite d2/d7/d8/d18/f20 [4194304,4194304] 0 2024-03-24T10:46:11.628 INFO:tasks.workunit.client.1.smithi096.stdout:5/349: chown d2/d7/d8/d18/l6e 30149 1 2024-03-24T10:46:11.630 INFO:tasks.workunit.client.1.smithi096.stdout:8/414: dwrite d8/d11/d22/d4c/d65/f1c [4194304,4194304] 0 2024-03-24T10:46:11.631 INFO:tasks.workunit.client.1.smithi096.stdout:7/543: dwrite d0/db/d1c/f61 [0,4194304] 0 2024-03-24T10:46:11.633 INFO:tasks.workunit.client.1.smithi096.stdout:5/350: dread d2/d7/f41 [0,4194304] 0 2024-03-24T10:46:11.633 INFO:tasks.workunit.client.1.smithi096.stdout:5/351: chown d2/d7/d8/df/l3a 798 1 2024-03-24T10:46:11.634 INFO:tasks.workunit.client.1.smithi096.stdout:5/352: write d2/d7/d8/d18/f47 [4557371,39952] 0 2024-03-24T10:46:11.637 INFO:tasks.workunit.client.1.smithi096.stdout:3/429: write d0/d13/d1e/f2f [3341477,68770] 0 2024-03-24T10:46:11.639 INFO:tasks.workunit.client.1.smithi096.stdout:8/415: dread d8/d11/d22/d4c/d65/d37/f6e [0,4194304] 0 2024-03-24T10:46:11.639 INFO:tasks.workunit.client.1.smithi096.stdout:8/416: readlink d8/d11/d22/d4c/d82/l41 0 2024-03-24T10:46:11.676 INFO:tasks.workunit.client.1.smithi096.stdout:7/544: creat d0/db/d26/d48/fb4 x:0 0 0 2024-03-24T10:46:11.692 INFO:tasks.workunit.client.1.smithi096.stdout:3/430: mknod d0/d13/d45/d52/d88/c91 0 2024-03-24T10:46:11.695 INFO:tasks.workunit.client.1.smithi096.stdout:8/417: creat d8/d11/d22/d4c/d65/d52/f88 x:0 0 0 2024-03-24T10:46:11.772 INFO:tasks.workunit.client.1.smithi096.stdout:7/545: mknod d0/da9/cb5 0 2024-03-24T10:46:11.772 INFO:tasks.workunit.client.1.smithi096.stdout:7/546: creat d0/d21/d3c/fb6 x:0 0 0 2024-03-24T10:46:11.772 INFO:tasks.workunit.client.1.smithi096.stdout:5/353: getdents d2/d29/d32/d3f/d5f 0 2024-03-24T10:46:11.776 INFO:tasks.workunit.client.1.smithi096.stdout:3/431: unlink d0/d9/d22/l50 0 2024-03-24T10:46:11.780 INFO:tasks.workunit.client.1.smithi096.stdout:7/547: creat d0/db/d4f/fb7 x:0 0 0 2024-03-24T10:46:11.782 INFO:tasks.workunit.client.1.smithi096.stdout:5/354: getdents d2 0 2024-03-24T10:46:11.782 INFO:tasks.workunit.client.1.smithi096.stdout:5/355: chown d2/d29/d32/d33 21296 1 2024-03-24T10:46:11.782 INFO:tasks.workunit.client.1.smithi096.stdout:5/356: truncate d2/d29/d32/d33/d37/f6d 740282 0 2024-03-24T10:46:11.783 INFO:tasks.workunit.client.1.smithi096.stdout:7/548: truncate d0/f38 576038 0 2024-03-24T10:46:11.783 INFO:tasks.workunit.client.1.smithi096.stdout:7/549: chown d0/db/l28 1 1 2024-03-24T10:46:11.783 INFO:tasks.workunit.client.1.smithi096.stdout:7/550: dread - d0/db/d4f/f65 zero size 2024-03-24T10:46:11.783 INFO:tasks.workunit.client.1.smithi096.stdout:7/551: readlink d0/db/l3e 0 2024-03-24T10:46:11.784 INFO:tasks.workunit.client.1.smithi096.stdout:5/357: creat d2/d7/d8/df/f78 x:0 0 0 2024-03-24T10:46:11.784 INFO:tasks.workunit.client.1.smithi096.stdout:5/358: chown d2/d29/d32/d33/f44 1711905 1 2024-03-24T10:46:11.784 INFO:tasks.workunit.client.1.smithi096.stdout:5/359: readlink d2/l3b 0 2024-03-24T10:46:11.784 INFO:tasks.workunit.client.1.smithi096.stdout:7/552: dread d0/db/d1c/d39/f4a [0,4194304] 0 2024-03-24T10:46:11.785 INFO:tasks.workunit.client.1.smithi096.stdout:3/432: dread d0/f1 [0,4194304] 0 2024-03-24T10:46:11.786 INFO:tasks.workunit.client.1.smithi096.stdout:5/360: mknod d2/d29/d32/d3f/c79 0 2024-03-24T10:46:11.790 INFO:tasks.workunit.client.1.smithi096.stdout:5/361: symlink d2/d29/d32/d3f/l7a 0 2024-03-24T10:46:11.790 INFO:tasks.workunit.client.1.smithi096.stdout:5/362: dread d2/d7/d8/df/d14/f23 [0,4194304] 0 2024-03-24T10:46:11.795 INFO:tasks.workunit.client.1.smithi096.stdout:7/553: read d0/d13/f3d [1129117,26476] 0 2024-03-24T10:46:11.795 INFO:tasks.workunit.client.1.smithi096.stdout:7/554: write d0/d13/d32/faa [720692,6704] 0 2024-03-24T10:46:11.796 INFO:tasks.workunit.client.1.smithi096.stdout:7/555: mknod d0/db/d26/d48/da8/cb8 0 2024-03-24T10:46:11.797 INFO:tasks.workunit.client.1.smithi096.stdout:7/556: write d0/d21/d3c/d68/d85/f8e [96491,53797] 0 2024-03-24T10:46:11.797 INFO:tasks.workunit.client.1.smithi096.stdout:7/557: creat d0/d13/d32/fb9 x:0 0 0 2024-03-24T10:46:11.797 INFO:tasks.workunit.client.1.smithi096.stdout:7/558: chown d0/d21/d3c/d68/d4b/fa4 1 1 2024-03-24T10:46:11.798 INFO:tasks.workunit.client.1.smithi096.stdout:7/559: chown d0/d13/l6 445 1 2024-03-24T10:46:11.800 INFO:tasks.workunit.client.1.smithi096.stdout:7/560: creat d0/db/d1c/d77/dab/fba x:0 0 0 2024-03-24T10:46:11.800 INFO:tasks.workunit.client.1.smithi096.stdout:7/561: chown d0/d13/d32/l88 6447 1 2024-03-24T10:46:11.801 INFO:tasks.workunit.client.1.smithi096.stdout:3/433: dwrite d0/f1 [0,4194304] 0 2024-03-24T10:46:11.802 INFO:tasks.workunit.client.1.smithi096.stdout:3/434: readlink d0/d13/d41/l69 0 2024-03-24T10:46:11.802 INFO:tasks.workunit.client.1.smithi096.stdout:3/435: readlink d0/d9/d22/l8c 0 2024-03-24T10:46:11.806 INFO:tasks.workunit.client.1.smithi096.stdout:0/378: sync 2024-03-24T10:46:11.806 INFO:tasks.workunit.client.1.smithi096.stdout:6/371: sync 2024-03-24T10:46:11.806 INFO:tasks.workunit.client.1.smithi096.stdout:9/382: sync 2024-03-24T10:46:11.806 INFO:tasks.workunit.client.1.smithi096.stdout:1/339: sync 2024-03-24T10:46:11.806 INFO:tasks.workunit.client.1.smithi096.stdout:2/306: sync 2024-03-24T10:46:11.807 INFO:tasks.workunit.client.1.smithi096.stdout:4/319: sync 2024-03-24T10:46:11.807 INFO:tasks.workunit.client.1.smithi096.stdout:4/320: dread - d1/d2b/d40/f64 zero size 2024-03-24T10:46:11.807 INFO:tasks.workunit.client.1.smithi096.stdout:4/321: fdatasync d1/d11/f29 0 2024-03-24T10:46:11.810 INFO:tasks.workunit.client.1.smithi096.stdout:9/383: mkdir d9/db/d6d/d4f/d75 0 2024-03-24T10:46:11.810 INFO:tasks.workunit.client.1.smithi096.stdout:9/384: write d9/d15/d26/d29/d73/f4b [662411,15419] 0 2024-03-24T10:46:11.810 INFO:tasks.workunit.client.1.smithi096.stdout:4/322: dread d1/f3e [0,4194304] 0 2024-03-24T10:46:11.812 INFO:tasks.workunit.client.1.smithi096.stdout:5/363: dwrite d2/d29/d32/d33/d3c/d3d/f75 [0,4194304] 0 2024-03-24T10:46:11.813 INFO:tasks.workunit.client.1.smithi096.stdout:0/379: write d0/d11/d24/f50 [1257922,59522] 0 2024-03-24T10:46:11.813 INFO:tasks.workunit.client.1.smithi096.stdout:2/307: symlink d5/d79/l81 0 2024-03-24T10:46:11.814 INFO:tasks.workunit.client.1.smithi096.stdout:5/364: dread d2/d7/d8/df/d14/f23 [0,4194304] 0 2024-03-24T10:46:11.815 INFO:tasks.workunit.client.1.smithi096.stdout:9/385: creat d9/db/d6d/d4f/d75/f76 x:0 0 0 2024-03-24T10:46:11.816 INFO:tasks.workunit.client.1.smithi096.stdout:4/323: truncate d1/f1f 2829106 0 2024-03-24T10:46:11.816 INFO:tasks.workunit.client.1.smithi096.stdout:0/380: dread d0/f48 [0,4194304] 0 2024-03-24T10:46:11.831 INFO:tasks.workunit.client.1.smithi096.stdout:5/365: mknod d2/d29/d32/d33/d3c/c7b 0 2024-03-24T10:46:11.831 INFO:tasks.workunit.client.1.smithi096.stdout:5/366: creat d2/d29/f7c x:0 0 0 2024-03-24T10:46:11.831 INFO:tasks.workunit.client.1.smithi096.stdout:9/386: mknod d9/d15/d26/d29/d73/c77 0 2024-03-24T10:46:11.833 INFO:tasks.workunit.client.1.smithi096.stdout:2/308: dwrite d5/dc/f2e [0,4194304] 0 2024-03-24T10:46:11.834 INFO:tasks.workunit.client.1.smithi096.stdout:6/372: dwrite d8/d12/d28/f33 [0,4194304] 0 2024-03-24T10:46:11.836 INFO:tasks.workunit.client.1.smithi096.stdout:5/367: mknod d2/d29/d32/d33/d3c/d3d/c7d 0 2024-03-24T10:46:11.839 INFO:tasks.workunit.client.1.smithi096.stdout:9/387: link d9/d15/d26/d29/d73/d43/d48/l6e d9/db/d6d/d4f/l78 0 2024-03-24T10:46:11.862 INFO:tasks.workunit.client.1.smithi096.stdout:5/368: dwrite d2/d7/d8/f56 [0,4194304] 0 2024-03-24T10:46:11.862 INFO:tasks.workunit.client.1.smithi096.stdout:6/373: dwrite f2 [0,4194304] 0 2024-03-24T10:46:11.862 INFO:tasks.workunit.client.1.smithi096.stdout:9/388: mkdir d9/d79 0 2024-03-24T10:46:11.864 INFO:tasks.workunit.client.1.smithi096.stdout:2/309: dwrite d5/dc/f2f [0,4194304] 0 2024-03-24T10:46:11.869 INFO:tasks.workunit.client.1.smithi096.stdout:9/389: dread d9/d15/d26/d29/f42 [0,4194304] 0 2024-03-24T10:46:11.874 INFO:tasks.workunit.client.1.smithi096.stdout:6/374: getdents d8/d10/d13 0 2024-03-24T10:46:11.874 INFO:tasks.workunit.client.1.smithi096.stdout:6/375: chown d8/le 119342771 1 2024-03-24T10:46:11.875 INFO:tasks.workunit.client.1.smithi096.stdout:9/390: truncate d9/db/f14 695362 0 2024-03-24T10:46:11.878 INFO:tasks.workunit.client.1.smithi096.stdout:2/310: rmdir d5/de/d2a 39 2024-03-24T10:46:11.878 INFO:tasks.workunit.client.1.smithi096.stdout:2/311: dread - d5/de/d2a/f62 zero size 2024-03-24T10:46:11.878 INFO:tasks.workunit.client.1.smithi096.stdout:6/376: dread d8/d10/d13/d63/f21 [4194304,4194304] 0 2024-03-24T10:46:11.879 INFO:tasks.workunit.client.1.smithi096.stdout:6/377: readlink d8/le 0 2024-03-24T10:46:11.880 INFO:tasks.workunit.client.1.smithi096.stdout:2/312: mknod d5/dc/d6b/c82 0 2024-03-24T10:46:11.882 INFO:tasks.workunit.client.1.smithi096.stdout:2/313: creat d5/dc/f83 x:0 0 0 2024-03-24T10:46:11.882 INFO:tasks.workunit.client.1.smithi096.stdout:2/314: getdents d5/dc/d12/d23/d4f 0 2024-03-24T10:46:11.882 INFO:tasks.workunit.client.1.smithi096.stdout:2/315: truncate d5/de/f3d 472180 0 2024-03-24T10:46:11.883 INFO:tasks.workunit.client.1.smithi096.stdout:2/316: truncate d5/dc/d12/f31 4629199 0 2024-03-24T10:46:11.889 INFO:tasks.workunit.client.1.smithi096.stdout:8/418: symlink d8/l89 0 2024-03-24T10:46:11.891 INFO:tasks.workunit.client.1.smithi096.stdout:2/317: dread d5/dc/f47 [0,4194304] 0 2024-03-24T10:46:11.892 INFO:tasks.workunit.client.1.smithi096.stdout:8/419: getdents d8/d11/d22/d4c/d65/d1d 0 2024-03-24T10:46:11.894 INFO:tasks.workunit.client.1.smithi096.stdout:8/420: creat d8/d11/d85/f8a x:0 0 0 2024-03-24T10:46:11.902 INFO:tasks.workunit.client.1.smithi096.stdout:9/391: dwrite d9/d15/d26/d29/d73/d43/f45 [4194304,4194304] 0 2024-03-24T10:46:11.902 INFO:tasks.workunit.client.1.smithi096.stdout:9/392: chown d9/db/f14 3111 1 2024-03-24T10:46:11.902 INFO:tasks.workunit.client.1.smithi096.stdout:9/393: read d9/f3a [98730,93363] 0 2024-03-24T10:46:11.934 INFO:tasks.workunit.client.1.smithi096.stdout:3/436: mkdir d0/d92 0 2024-03-24T10:46:11.936 INFO:tasks.workunit.client.1.smithi096.stdout:2/318: creat d5/f84 x:0 0 0 2024-03-24T10:46:11.936 INFO:tasks.workunit.client.1.smithi096.stdout:2/319: chown d5/dc/d12/f26 106065015 1 2024-03-24T10:46:11.937 INFO:tasks.workunit.client.1.smithi096.stdout:2/320: symlink d5/dc/l85 0 2024-03-24T10:46:11.942 INFO:tasks.workunit.client.1.smithi096.stdout:2/321: dread d5/dc/f47 [0,4194304] 0 2024-03-24T10:46:11.943 INFO:tasks.workunit.client.1.smithi096.stdout:5/369: symlink d2/d7/d8/l7e 0 2024-03-24T10:46:11.944 INFO:tasks.workunit.client.1.smithi096.stdout:7/562: rename d0/db/c6a to d0/d21/d3c/cbb 0 2024-03-24T10:46:11.948 INFO:tasks.workunit.client.1.smithi096.stdout:0/381: rename d0/d2 to d0/d11/d42/d7f 0 2024-03-24T10:46:11.956 INFO:tasks.workunit.client.1.smithi096.stdout:6/378: rename d8/d10/c25 to d8/d12/d32/d68/c6d 0 2024-03-24T10:46:11.956 INFO:tasks.workunit.client.1.smithi096.stdout:6/379: creat d8/d12/d32/f6e x:0 0 0 2024-03-24T10:46:11.956 INFO:tasks.workunit.client.1.smithi096.stdout:6/380: getdents d8/d10/d58 0 2024-03-24T10:46:11.956 INFO:tasks.workunit.client.1.smithi096.stdout:6/381: chown d8/d10/d13/d63/f5e 142 1 2024-03-24T10:46:11.956 INFO:tasks.workunit.client.1.smithi096.stdout:6/382: truncate f2 4539808 0 2024-03-24T10:46:11.956 INFO:tasks.workunit.client.1.smithi096.stdout:2/322: dread d5/dc/f5b [0,4194304] 0 2024-03-24T10:46:11.957 INFO:tasks.workunit.client.1.smithi096.stdout:3/437: dwrite d0/d13/d45/d7d/d39/d48/f6a [0,4194304] 0 2024-03-24T10:46:11.957 INFO:tasks.workunit.client.1.smithi096.stdout:0/382: getdents d0/d11 0 2024-03-24T10:46:11.957 INFO:tasks.workunit.client.1.smithi096.stdout:0/383: chown d0/d11/d20/f1f 2180 1 2024-03-24T10:46:11.957 INFO:tasks.workunit.client.1.smithi096.stdout:0/384: chown d0/d11/d20/d2b/l39 260059841 1 2024-03-24T10:46:11.957 INFO:tasks.workunit.client.1.smithi096.stdout:0/385: chown d0/d11/d43 26475516 1 2024-03-24T10:46:11.961 INFO:tasks.workunit.client.1.smithi096.stdout:2/323: dread d5/f7 [0,4194304] 0 2024-03-24T10:46:11.963 INFO:tasks.workunit.client.1.smithi096.stdout:6/383: creat d8/d12/d28/d36/d65/f6f x:0 0 0 2024-03-24T10:46:11.969 INFO:tasks.workunit.client.1.smithi096.stdout:8/421: rename d8/c7e to d8/d11/d22/d4c/d65/d37/c8b 0 2024-03-24T10:46:11.970 INFO:tasks.workunit.client.1.smithi096.stdout:5/370: dwrite d2/d49/f5d [0,4194304] 0 2024-03-24T10:46:11.973 INFO:tasks.workunit.client.1.smithi096.stdout:2/324: unlink d5/de/f15 0 2024-03-24T10:46:11.973 INFO:tasks.workunit.client.1.smithi096.stdout:2/325: read - d5/de/d2a/f35 zero size 2024-03-24T10:46:11.973 INFO:tasks.workunit.client.1.smithi096.stdout:2/326: truncate d5/de/f3d 1453420 0 2024-03-24T10:46:11.973 INFO:tasks.workunit.client.1.smithi096.stdout:2/327: truncate d5/de/d2a/d5e/f65 4200014 0 2024-03-24T10:46:11.974 INFO:tasks.workunit.client.1.smithi096.stdout:2/328: dread d5/dc/d12/f41 [0,4194304] 0 2024-03-24T10:46:11.975 INFO:tasks.workunit.client.1.smithi096.stdout:6/384: mknod d8/d12/d28/d36/d65/c70 0 2024-03-24T10:46:11.975 INFO:tasks.workunit.client.1.smithi096.stdout:6/385: write d8/d10/d19/f60 [436794,52222] 0 2024-03-24T10:46:11.976 INFO:tasks.workunit.client.1.smithi096.stdout:9/394: rename d9/d15/d26/c38 to d9/d60/c7a 0 2024-03-24T10:46:11.978 INFO:tasks.workunit.client.1.smithi096.stdout:7/563: dwrite d0/db/d26/f55 [0,4194304] 0 2024-03-24T10:46:11.978 INFO:tasks.workunit.client.1.smithi096.stdout:7/564: fdatasync d0/d21/d3c/faf 0 2024-03-24T10:46:11.978 INFO:tasks.workunit.client.1.smithi096.stdout:7/565: chown d0/db/d1c/l89 485281 1 2024-03-24T10:46:11.978 INFO:tasks.workunit.client.1.smithi096.stdout:3/438: dwrite d0/d13/d45/d47/f64 [0,4194304] 0 2024-03-24T10:46:11.983 INFO:tasks.workunit.client.1.smithi096.stdout:5/371: creat d2/d7/f7f x:0 0 0 2024-03-24T10:46:11.984 INFO:tasks.workunit.client.1.smithi096.stdout:5/372: chown d2/d29/d32/d33/d3c/l4f 23 1 2024-03-24T10:46:11.987 INFO:tasks.workunit.client.1.smithi096.stdout:6/386: rmdir d8 39 2024-03-24T10:46:11.987 INFO:tasks.workunit.client.1.smithi096.stdout:6/387: write d8/d12/d28/d36/d65/f6f [183084,8254] 0 2024-03-24T10:46:11.987 INFO:tasks.workunit.client.1.smithi096.stdout:6/388: chown d8/d10/d19/f1b 31357 1 2024-03-24T10:46:11.994 INFO:tasks.workunit.client.1.smithi096.stdout:0/386: rename d0/d11/d40 to d0/d11/d16/d21/d25/d47/d63/d80 0 2024-03-24T10:46:11.995 INFO:tasks.workunit.client.1.smithi096.stdout:9/395: creat d9/db/d6d/d4f/f7b x:0 0 0 2024-03-24T10:46:11.997 INFO:tasks.workunit.client.1.smithi096.stdout:9/396: write d9/db/fe [2673851,82879] 0 2024-03-24T10:46:11.998 INFO:tasks.workunit.client.1.smithi096.stdout:5/373: dwrite d2/d29/d32/d33/d3c/f74 [0,4194304] 0 2024-03-24T10:46:12.003 INFO:tasks.workunit.client.1.smithi096.stdout:7/566: truncate d0/db/d1c/d39/f76 4401560 0 2024-03-24T10:46:12.006 INFO:tasks.workunit.client.1.smithi096.stdout:3/439: symlink d0/d13/d45/d7d/d39/l93 0 2024-03-24T10:46:12.010 INFO:tasks.workunit.client.1.smithi096.stdout:6/389: mkdir d8/d12/d28/d36/d65/d71 0 2024-03-24T10:46:12.011 INFO:tasks.workunit.client.1.smithi096.stdout:6/390: truncate d8/d12/d32/d38/f4b 333848 0 2024-03-24T10:46:12.017 INFO:tasks.workunit.client.1.smithi096.stdout:5/374: rename d2/d29/f7c to d2/d29/d32/f80 0 2024-03-24T10:46:12.017 INFO:tasks.workunit.client.1.smithi096.stdout:7/567: link d0/db/d26/f5b d0/d13/d32/d99/fbc 0 2024-03-24T10:46:12.020 INFO:tasks.workunit.client.1.smithi096.stdout:3/440: rename d0/d13/d45/d7d/f67 to d0/d13/d45/d7d/f94 0 2024-03-24T10:46:12.020 INFO:tasks.workunit.client.1.smithi096.stdout:0/387: dwrite d0/d11/d16/d21/d25/d47/f70 [0,4194304] 0 2024-03-24T10:46:12.021 INFO:tasks.workunit.client.1.smithi096.stdout:6/391: link d8/f62 d8/d10/d58/f72 0 2024-03-24T10:46:12.022 INFO:tasks.workunit.client.1.smithi096.stdout:7/568: dread d0/f14 [0,4194304] 0 2024-03-24T10:46:12.024 INFO:tasks.workunit.client.1.smithi096.stdout:2/329: truncate d5/de/d2a/d5e/f65 183088 0 2024-03-24T10:46:12.026 INFO:tasks.workunit.client.1.smithi096.stdout:9/397: dwrite d9/d15/d26/d29/d73/d3e/f59 [0,4194304] 0 2024-03-24T10:46:12.029 INFO:tasks.workunit.client.1.smithi096.stdout:0/388: rename d0/d11/d42 to d0/d11/d16/d21/d81 0 2024-03-24T10:46:12.029 INFO:tasks.workunit.client.1.smithi096.stdout:0/389: readlink d0/d11/d24/d2c/l77 0 2024-03-24T10:46:12.034 INFO:tasks.workunit.client.1.smithi096.stdout:6/392: creat d8/d12/d32/d66/f73 x:0 0 0 2024-03-24T10:46:12.036 INFO:tasks.workunit.client.1.smithi096.stdout:7/569: mknod d0/d21/d3c/cbd 0 2024-03-24T10:46:12.037 INFO:tasks.workunit.client.1.smithi096.stdout:7/570: dread d0/d13/d32/d45/f63 [0,4194304] 0 2024-03-24T10:46:12.050 INFO:tasks.workunit.client.1.smithi096.stdout:9/398: mkdir d9/d7c 0 2024-03-24T10:46:12.052 INFO:tasks.workunit.client.1.smithi096.stdout:7/571: dwrite d0/db/d1c/f29 [0,4194304] 0 2024-03-24T10:46:12.053 INFO:tasks.workunit.client.1.smithi096.stdout:7/572: read d0/d21/d59/d87/f8a [63099,24403] 0 2024-03-24T10:46:12.053 INFO:tasks.workunit.client.1.smithi096.stdout:7/573: write d0/d21/d3c/d68/d4b/fa4 [224238,73750] 0 2024-03-24T10:46:12.054 INFO:tasks.workunit.client.1.smithi096.stdout:6/393: rmdir d8 39 2024-03-24T10:46:12.062 INFO:tasks.workunit.client.1.smithi096.stdout:2/330: rename d5/dc/d12/d23/d4f to d5/de/d86 0 2024-03-24T10:46:12.062 INFO:tasks.workunit.client.1.smithi096.stdout:2/331: fsync d5/de/f13 0 2024-03-24T10:46:12.063 INFO:tasks.workunit.client.1.smithi096.stdout:7/574: creat d0/d21/d3c/d68/d7e/fbe x:0 0 0 2024-03-24T10:46:12.064 INFO:tasks.workunit.client.1.smithi096.stdout:7/575: readlink d0/d13/l4 0 2024-03-24T10:46:12.066 INFO:tasks.workunit.client.1.smithi096.stdout:7/576: creat d0/da9/fbf x:0 0 0 2024-03-24T10:46:12.066 INFO:tasks.workunit.client.1.smithi096.stdout:7/577: fdatasync d0/db/d26/f55 0 2024-03-24T10:46:12.076 INFO:tasks.workunit.client.1.smithi096.stdout:2/332: dwrite d5/d7c/f7e [0,4194304] 0 2024-03-24T10:46:12.155 INFO:tasks.workunit.client.1.smithi096.stdout:7/578: creat d0/d21/d3c/d68/d4b/fc0 x:0 0 0 2024-03-24T10:46:12.158 INFO:tasks.workunit.client.1.smithi096.stdout:2/333: getdents d5/dc/d12 0 2024-03-24T10:46:12.158 INFO:tasks.workunit.client.1.smithi096.stdout:2/334: fsync d5/d38/f45 0 2024-03-24T10:46:12.158 INFO:tasks.workunit.client.1.smithi096.stdout:2/335: write d5/dc/f58 [717172,66557] 0 2024-03-24T10:46:12.159 INFO:tasks.workunit.client.1.smithi096.stdout:7/579: creat d0/d21/d3c/d80/fc1 x:0 0 0 2024-03-24T10:46:12.160 INFO:tasks.workunit.client.1.smithi096.stdout:2/336: symlink d5/d79/l87 0 2024-03-24T10:46:12.163 INFO:tasks.workunit.client.1.smithi096.stdout:7/580: link d0/l1d d0/db/d26/d48/lc2 0 2024-03-24T10:46:12.163 INFO:tasks.workunit.client.1.smithi096.stdout:7/581: truncate d0/db/f72 1402846 0 2024-03-24T10:46:12.165 INFO:tasks.workunit.client.1.smithi096.stdout:2/337: rmdir d5/dc 39 2024-03-24T10:46:12.167 INFO:tasks.workunit.client.1.smithi096.stdout:2/338: rmdir d5/dc/d12 39 2024-03-24T10:46:12.178 INFO:tasks.workunit.client.1.smithi096.stdout:2/339: write d5/d38/f7b [2989655,110451] 0 2024-03-24T10:46:12.178 INFO:tasks.workunit.client.1.smithi096.stdout:2/340: truncate d5/de/f51 939824 0 2024-03-24T10:46:12.178 INFO:tasks.workunit.client.1.smithi096.stdout:2/341: dread - d5/de/d2a/f62 zero size 2024-03-24T10:46:12.178 INFO:tasks.workunit.client.1.smithi096.stdout:2/342: fsync d5/dc/f5b 0 2024-03-24T10:46:12.178 INFO:tasks.workunit.client.1.smithi096.stdout:2/343: truncate d5/de/d86/f70 456953 0 2024-03-24T10:46:12.178 INFO:tasks.workunit.client.1.smithi096.stdout:2/344: fdatasync d5/dc/f21 0 2024-03-24T10:46:12.182 INFO:tasks.workunit.client.1.smithi096.stdout:7/582: dwrite d0/f14 [0,4194304] 0 2024-03-24T10:46:12.184 INFO:tasks.workunit.client.1.smithi096.stdout:7/583: write d0/d13/d60/f9d [134392,118453] 0 2024-03-24T10:46:12.188 INFO:tasks.workunit.client.1.smithi096.stdout:2/345: dread d5/d38/f66 [0,4194304] 0 2024-03-24T10:46:12.192 INFO:tasks.workunit.client.1.smithi096.stdout:7/584: dread d0/d13/d32/f3b [4194304,4194304] 0 2024-03-24T10:46:12.209 INFO:tasks.workunit.client.1.smithi096.stdout:7/585: dwrite d0/db/d1c/f91 [4194304,4194304] 0 2024-03-24T10:46:12.223 INFO:tasks.workunit.client.1.smithi096.stdout:7/586: dwrite d0/db/d1c/d39/f4a [0,4194304] 0 2024-03-24T10:46:12.223 INFO:tasks.workunit.client.1.smithi096.stdout:7/587: write d0/d21/d3c/d68/f41 [510207,48298] 0 2024-03-24T10:46:12.225 INFO:tasks.workunit.client.1.smithi096.stdout:7/588: rename d0/d13/d60 to d0/db/d26/dc3 0 2024-03-24T10:46:12.225 INFO:tasks.workunit.client.1.smithi096.stdout:7/589: write d0/db/d1c/d77/dab/fba [220008,36351] 0 2024-03-24T10:46:12.228 INFO:tasks.workunit.client.1.smithi096.stdout:7/590: mknod d0/d21/d59/d87/cc4 0 2024-03-24T10:46:12.231 INFO:tasks.workunit.client.1.smithi096.stdout:7/591: symlink d0/db/d26/dc3/lc5 0 2024-03-24T10:46:12.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:11 smithi094 ceph-mon[29446]: pgmap v249: 65 pgs: 65 active+clean; 2.6 GiB data, 9.0 GiB used, 527 GiB / 536 GiB avail; 239 MiB/s rd, 298 MiB/s wr, 409 op/s 2024-03-24T10:46:12.255 INFO:tasks.workunit.client.1.smithi096.stdout:7/592: dwrite d0/f9e [0,4194304] 0 2024-03-24T10:46:12.255 INFO:tasks.workunit.client.1.smithi096.stdout:7/593: dread - d0/d21/d3c/fb6 zero size 2024-03-24T10:46:12.255 INFO:tasks.workunit.client.1.smithi096.stdout:7/594: fdatasync d0/d21/d3c/d68/d4b/fc0 0 2024-03-24T10:46:12.255 INFO:tasks.workunit.client.1.smithi096.stdout:7/595: write d0/db/f72 [2203774,128877] 0 2024-03-24T10:46:12.259 INFO:tasks.workunit.client.1.smithi096.stdout:7/596: symlink d0/d21/d3c/d68/d7e/lc6 0 2024-03-24T10:46:12.259 INFO:tasks.workunit.client.1.smithi096.stdout:7/597: chown d0/da9/fbf 5 1 2024-03-24T10:46:12.260 INFO:tasks.workunit.client.1.smithi096.stdout:7/598: dread - d0/db/d26/dc3/f93 zero size 2024-03-24T10:46:12.262 INFO:tasks.workunit.client.1.smithi096.stdout:7/599: rmdir d0/db/d1c/d77 39 2024-03-24T10:46:12.283 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:11 smithi096 ceph-mon[38097]: pgmap v249: 65 pgs: 65 active+clean; 2.6 GiB data, 9.0 GiB used, 527 GiB / 536 GiB avail; 239 MiB/s rd, 298 MiB/s wr, 409 op/s 2024-03-24T10:46:12.314 INFO:tasks.workunit.client.1.smithi096.stdout:1/340: sync 2024-03-24T10:46:12.320 INFO:tasks.workunit.client.1.smithi096.stdout:1/341: write d3/f66 [929190,50789] 0 2024-03-24T10:46:12.321 INFO:tasks.workunit.client.1.smithi096.stdout:1/342: write d3/d24/d6a/f70 [242960,29587] 0 2024-03-24T10:46:12.339 INFO:tasks.workunit.client.1.smithi096.stdout:7/600: link d0/d13/l4 d0/d21/lc7 0 2024-03-24T10:46:12.345 INFO:tasks.workunit.client.1.smithi096.stdout:7/601: write d0/db/ff [7836445,89096] 0 2024-03-24T10:46:12.346 INFO:tasks.workunit.client.1.smithi096.stdout:7/602: chown d0/d21/d3c/faf 27872 1 2024-03-24T10:46:12.346 INFO:tasks.workunit.client.1.smithi096.stdout:1/343: mknod d3/d36/c74 0 2024-03-24T10:46:12.346 INFO:tasks.workunit.client.1.smithi096.stdout:1/344: creat d3/d24/d5c/f75 x:0 0 0 2024-03-24T10:46:12.346 INFO:tasks.workunit.client.1.smithi096.stdout:1/345: chown d3/d24/f3e 15191739 1 2024-03-24T10:46:12.349 INFO:tasks.workunit.client.1.smithi096.stdout:7/603: mknod d0/db/d26/d48/da8/db1/cc8 0 2024-03-24T10:46:12.349 INFO:tasks.workunit.client.1.smithi096.stdout:7/604: write d0/db/d26/d48/fb4 [772619,75989] 0 2024-03-24T10:46:12.349 INFO:tasks.workunit.client.1.smithi096.stdout:7/605: chown d0/d21/d3c/d68/d4b/fc0 125 1 2024-03-24T10:46:12.351 INFO:tasks.workunit.client.1.smithi096.stdout:1/346: symlink d3/d24/d5c/d6d/l76 0 2024-03-24T10:46:12.354 INFO:tasks.workunit.client.1.smithi096.stdout:7/606: symlink d0/d21/d59/lc9 0 2024-03-24T10:46:12.354 INFO:tasks.workunit.client.1.smithi096.stdout:7/607: fdatasync d0/f9e 0 2024-03-24T10:46:12.355 INFO:tasks.workunit.client.1.smithi096.stdout:7/608: getdents d0/d21/d59/d87/da0 0 2024-03-24T10:46:12.357 INFO:tasks.workunit.client.1.smithi096.stdout:7/609: symlink d0/d21/d3c/lca 0 2024-03-24T10:46:12.358 INFO:tasks.workunit.client.1.smithi096.stdout:7/610: creat d0/d21/fcb x:0 0 0 2024-03-24T10:46:12.362 INFO:tasks.workunit.client.1.smithi096.stdout:7/611: write d0/db/f72 [466287,125255] 0 2024-03-24T10:46:12.364 INFO:tasks.workunit.client.1.smithi096.stdout:7/612: rename d0/db/d4f/la5 to d0/db/d26/dc3/lcc 0 2024-03-24T10:46:12.367 INFO:tasks.workunit.client.1.smithi096.stdout:1/347: dwrite d3/d23/d2f/f4a [0,4194304] 0 2024-03-24T10:46:12.368 INFO:tasks.workunit.client.1.smithi096.stdout:1/348: dread d3/d24/f35 [0,4194304] 0 2024-03-24T10:46:12.373 INFO:tasks.workunit.client.1.smithi096.stdout:7/613: dread d0/db/f72 [0,4194304] 0 2024-03-24T10:46:12.374 INFO:tasks.workunit.client.1.smithi096.stdout:7/614: mkdir d0/d13/d32/d45/dcd 0 2024-03-24T10:46:12.374 INFO:tasks.workunit.client.1.smithi096.stdout:7/615: creat d0/db/d26/d48/fce x:0 0 0 2024-03-24T10:46:12.374 INFO:tasks.workunit.client.1.smithi096.stdout:1/349: dread d3/ff [0,4194304] 0 2024-03-24T10:46:12.383 INFO:tasks.workunit.client.1.smithi096.stdout:7/616: dread d0/f5a [0,4194304] 0 2024-03-24T10:46:12.383 INFO:tasks.workunit.client.1.smithi096.stdout:7/617: creat d0/d21/d3c/d80/fcf x:0 0 0 2024-03-24T10:46:12.389 INFO:tasks.workunit.client.1.smithi096.stdout:7/618: rename d0/d21/d3c/fb6 to d0/db/d26/d48/fd0 0 2024-03-24T10:46:12.393 INFO:tasks.workunit.client.1.smithi096.stdout:1/350: dwrite d3/d24/d5c/f67 [0,4194304] 0 2024-03-24T10:46:12.393 INFO:tasks.workunit.client.1.smithi096.stdout:1/351: creat d3/d24/d6a/f77 x:0 0 0 2024-03-24T10:46:12.393 INFO:tasks.workunit.client.1.smithi096.stdout:1/352: chown d3/d36 121 1 2024-03-24T10:46:12.393 INFO:tasks.workunit.client.1.smithi096.stdout:1/353: creat d3/d23/d2f/f78 x:0 0 0 2024-03-24T10:46:12.406 INFO:tasks.workunit.client.1.smithi096.stdout:4/324: sync 2024-03-24T10:46:12.406 INFO:tasks.workunit.client.1.smithi096.stdout:8/422: sync 2024-03-24T10:46:12.406 INFO:tasks.workunit.client.1.smithi096.stdout:8/423: chown d8/d11/d44/d45/l79 0 1 2024-03-24T10:46:12.407 INFO:tasks.workunit.client.1.smithi096.stdout:8/424: chown d8/d11/d22/d4c/d65/d52/f57 61229 1 2024-03-24T10:46:12.407 INFO:tasks.workunit.client.1.smithi096.stdout:7/619: dwrite d0/db/d26/f55 [0,4194304] 0 2024-03-24T10:46:12.407 INFO:tasks.workunit.client.1.smithi096.stdout:1/354: mkdir d3/d24/d79 0 2024-03-24T10:46:12.407 INFO:tasks.workunit.client.1.smithi096.stdout:7/620: rename d0/db/d1c/l89 to d0/d21/d59/ld1 0 2024-03-24T10:46:12.407 INFO:tasks.workunit.client.1.smithi096.stdout:8/425: dread d8/d72/f76 [0,4194304] 0 2024-03-24T10:46:12.410 INFO:tasks.workunit.client.1.smithi096.stdout:7/621: dread d0/d21/d3c/f82 [0,4194304] 0 2024-03-24T10:46:12.410 INFO:tasks.workunit.client.1.smithi096.stdout:1/355: link d3/d23/d2f/f3b d3/d73/f7a 0 2024-03-24T10:46:12.412 INFO:tasks.workunit.client.1.smithi096.stdout:4/325: write d1/f1f [546957,26052] 0 2024-03-24T10:46:12.415 INFO:tasks.workunit.client.1.smithi096.stdout:7/622: stat d0/db/d4f/c84 0 2024-03-24T10:46:12.415 INFO:tasks.workunit.client.1.smithi096.stdout:7/623: readlink d0/d13/l4 0 2024-03-24T10:46:12.419 INFO:tasks.workunit.client.1.smithi096.stdout:1/356: mkdir d3/d23/d71/d7b 0 2024-03-24T10:46:12.419 INFO:tasks.workunit.client.1.smithi096.stdout:1/357: fsync d3/f1f 0 2024-03-24T10:46:12.419 INFO:tasks.workunit.client.1.smithi096.stdout:1/358: truncate d3/d73/f7a 1316438 0 2024-03-24T10:46:12.419 INFO:tasks.workunit.client.1.smithi096.stdout:1/359: truncate d3/d24/f35 289009 0 2024-03-24T10:46:12.423 INFO:tasks.workunit.client.1.smithi096.stdout:1/360: dread d3/d23/f42 [0,4194304] 0 2024-03-24T10:46:12.423 INFO:tasks.workunit.client.1.smithi096.stdout:1/361: getdents d3/d23/d2f/d65 0 2024-03-24T10:46:12.424 INFO:tasks.workunit.client.1.smithi096.stdout:8/426: rmdir d8/d11/d44 39 2024-03-24T10:46:12.424 INFO:tasks.workunit.client.1.smithi096.stdout:8/427: readlink d8/l20 0 2024-03-24T10:46:12.428 INFO:tasks.workunit.client.1.smithi096.stdout:8/428: dread d8/d11/d22/d4c/d65/d37/f6b [0,4194304] 0 2024-03-24T10:46:12.428 INFO:tasks.workunit.client.1.smithi096.stdout:8/429: truncate d8/d72/f83 406020 0 2024-03-24T10:46:12.428 INFO:tasks.workunit.client.1.smithi096.stdout:8/430: read d8/d72/f76 [623727,94383] 0 2024-03-24T10:46:12.430 INFO:tasks.workunit.client.1.smithi096.stdout:4/326: symlink d1/d2b/l68 0 2024-03-24T10:46:12.431 INFO:tasks.workunit.client.1.smithi096.stdout:8/431: dread d8/f69 [0,4194304] 0 2024-03-24T10:46:12.431 INFO:tasks.workunit.client.1.smithi096.stdout:8/432: fsync d8/d11/d22/d4c/d65/d1a/f2d 0 2024-03-24T10:46:12.431 INFO:tasks.workunit.client.1.smithi096.stdout:8/433: readlink d8/la 0 2024-03-24T10:46:12.432 INFO:tasks.workunit.client.1.smithi096.stdout:1/362: mkdir d3/d23/d2f/d65/d7c 0 2024-03-24T10:46:12.433 INFO:tasks.workunit.client.1.smithi096.stdout:7/624: dwrite d0/d21/fcb [0,4194304] 0 2024-03-24T10:46:12.433 INFO:tasks.workunit.client.1.smithi096.stdout:7/625: chown d0/db/d1c/d39/c6f 0 1 2024-03-24T10:46:12.433 INFO:tasks.workunit.client.1.smithi096.stdout:7/626: chown d0/db/f75 865338 1 2024-03-24T10:46:12.433 INFO:tasks.workunit.client.1.smithi096.stdout:7/627: truncate d0/f38 1569631 0 2024-03-24T10:46:12.434 INFO:tasks.workunit.client.1.smithi096.stdout:4/327: symlink d1/d32/d42/l69 0 2024-03-24T10:46:12.435 INFO:tasks.workunit.client.1.smithi096.stdout:1/363: creat d3/d23/d71/f7d x:0 0 0 2024-03-24T10:46:12.439 INFO:tasks.workunit.client.1.smithi096.stdout:8/434: dread d8/f14 [0,4194304] 0 2024-03-24T10:46:12.443 INFO:tasks.workunit.client.1.smithi096.stdout:3/441: sync 2024-03-24T10:46:12.443 INFO:tasks.workunit.client.1.smithi096.stdout:5/375: sync 2024-03-24T10:46:12.445 INFO:tasks.workunit.client.1.smithi096.stdout:4/328: creat d1/d2b/d67/f6a x:0 0 0 2024-03-24T10:46:12.447 INFO:tasks.workunit.client.1.smithi096.stdout:1/364: link d3/d24/f35 d3/d23/d2f/f7e 0 2024-03-24T10:46:12.449 INFO:tasks.workunit.client.1.smithi096.stdout:8/435: creat d8/d11/d22/d4c/d82/f8c x:0 0 0 2024-03-24T10:46:12.451 INFO:tasks.workunit.client.1.smithi096.stdout:5/376: write d2/d29/d32/f40 [7384047,62596] 0 2024-03-24T10:46:12.452 INFO:tasks.workunit.client.1.smithi096.stdout:5/377: truncate d2/f39 2307649 0 2024-03-24T10:46:12.453 INFO:tasks.workunit.client.1.smithi096.stdout:3/442: mknod d0/d13/d45/d7d/d39/c95 0 2024-03-24T10:46:12.453 INFO:tasks.workunit.client.1.smithi096.stdout:3/443: chown d0/l26 71127905 1 2024-03-24T10:46:12.453 INFO:tasks.workunit.client.1.smithi096.stdout:3/444: truncate d0/d9/f5e 915084 0 2024-03-24T10:46:12.453 INFO:tasks.workunit.client.1.smithi096.stdout:3/445: write d0/f61 [602996,85007] 0 2024-03-24T10:46:12.456 INFO:tasks.workunit.client.1.smithi096.stdout:1/365: rename d3/d23/d2f/l58 to d3/d23/d71/d7b/l7f 0 2024-03-24T10:46:12.459 INFO:tasks.workunit.client.1.smithi096.stdout:5/378: symlink d2/d7/d8/d18/d4d/l81 0 2024-03-24T10:46:12.460 INFO:tasks.workunit.client.1.smithi096.stdout:5/379: chown d2/d7/d8/df/f25 23733 1 2024-03-24T10:46:12.460 INFO:tasks.workunit.client.1.smithi096.stdout:5/380: readlink d2/d7/d8/df/l2e 0 2024-03-24T10:46:12.460 INFO:tasks.workunit.client.1.smithi096.stdout:3/446: creat d0/d13/d45/d7d/d71/f96 x:0 0 0 2024-03-24T10:46:12.467 INFO:tasks.workunit.client.1.smithi096.stdout:4/329: getdents d1/d11 0 2024-03-24T10:46:12.472 INFO:tasks.workunit.client.1.smithi096.stdout:1/366: mkdir d3/d23/d2f/d80 0 2024-03-24T10:46:12.472 INFO:tasks.workunit.client.1.smithi096.stdout:1/367: fdatasync d3/ff 0 2024-03-24T10:46:12.473 INFO:tasks.workunit.client.1.smithi096.stdout:8/436: symlink d8/d11/d22/d4c/d65/d37/l8d 0 2024-03-24T10:46:12.473 INFO:tasks.workunit.client.1.smithi096.stdout:3/447: dwrite d0/d13/f83 [0,4194304] 0 2024-03-24T10:46:12.477 INFO:tasks.workunit.client.1.smithi096.stdout:1/368: mknod d3/d24/d5c/c81 0 2024-03-24T10:46:12.477 INFO:tasks.workunit.client.1.smithi096.stdout:1/369: write d3/d36/f54 [955334,64609] 0 2024-03-24T10:46:12.480 INFO:tasks.workunit.client.1.smithi096.stdout:8/437: getdents d8/d11/d22/d4c/d65 0 2024-03-24T10:46:12.480 INFO:tasks.workunit.client.1.smithi096.stdout:8/438: fsync d8/d11/d22/d4c/d65/d1d/f7b 0 2024-03-24T10:46:12.484 INFO:tasks.workunit.client.1.smithi096.stdout:5/381: rename d2/d7/d8/c5e to d2/d7/d8/df/d14/c82 0 2024-03-24T10:46:12.484 INFO:tasks.workunit.client.1.smithi096.stdout:5/382: write d2/d7/d8/df/f78 [161163,76079] 0 2024-03-24T10:46:12.485 INFO:tasks.workunit.client.1.smithi096.stdout:4/330: mkdir d1/d2b/d6b 0 2024-03-24T10:46:12.487 INFO:tasks.workunit.client.1.smithi096.stdout:8/439: mknod d8/d72/c8e 0 2024-03-24T10:46:12.491 INFO:tasks.workunit.client.1.smithi096.stdout:3/448: dwrite d0/d9/f1a [0,4194304] 0 2024-03-24T10:46:12.495 INFO:tasks.workunit.client.1.smithi096.stdout:5/383: rename d2/f5a to d2/d29/d32/d33/f83 0 2024-03-24T10:46:12.495 INFO:tasks.workunit.client.1.smithi096.stdout:5/384: dread - d2/d29/d32/d33/f83 zero size 2024-03-24T10:46:12.495 INFO:tasks.workunit.client.1.smithi096.stdout:5/385: stat d2/d7/d8/df/l2e 0 2024-03-24T10:46:12.496 INFO:tasks.workunit.client.1.smithi096.stdout:4/331: symlink d1/d32/d42/l6c 0 2024-03-24T10:46:12.503 INFO:tasks.workunit.client.1.smithi096.stdout:4/332: write d1/f51 [3517023,33434] 0 2024-03-24T10:46:12.509 INFO:tasks.workunit.client.1.smithi096.stdout:3/449: dwrite d0/d13/f21 [4194304,4194304] 0 2024-03-24T10:46:12.512 INFO:tasks.workunit.client.1.smithi096.stdout:8/440: rename d8/f5d to d8/d72/f8f 0 2024-03-24T10:46:12.514 INFO:tasks.workunit.client.1.smithi096.stdout:5/386: truncate d2/d29/d32/d33/d37/f6d 52969 0 2024-03-24T10:46:12.518 INFO:tasks.workunit.client.1.smithi096.stdout:3/450: dread d0/f5c [0,4194304] 0 2024-03-24T10:46:12.523 INFO:tasks.workunit.client.1.smithi096.stdout:4/333: dwrite d1/f35 [0,4194304] 0 2024-03-24T10:46:12.523 INFO:tasks.workunit.client.1.smithi096.stdout:4/334: creat d1/d2b/d40/f6d x:0 0 0 2024-03-24T10:46:12.532 INFO:tasks.workunit.client.1.smithi096.stdout:5/387: dwrite d2/d29/d32/d33/d3c/f74 [0,4194304] 0 2024-03-24T10:46:12.540 INFO:tasks.workunit.client.1.smithi096.stdout:4/335: dwrite d1/ff [4194304,4194304] 0 2024-03-24T10:46:12.540 INFO:tasks.workunit.client.1.smithi096.stdout:4/336: stat d1 0 2024-03-24T10:46:12.545 INFO:tasks.workunit.client.1.smithi096.stdout:5/388: mkdir d2/d29/d32/d3f/d84 0 2024-03-24T10:46:12.549 INFO:tasks.workunit.client.1.smithi096.stdout:4/337: symlink d1/d11/l6e 0 2024-03-24T10:46:12.549 INFO:tasks.workunit.client.1.smithi096.stdout:4/338: readlink d1/d32/d42/l6c 0 2024-03-24T10:46:12.550 INFO:tasks.workunit.client.1.smithi096.stdout:5/389: dread d2/d29/d32/f40 [4194304,4194304] 0 2024-03-24T10:46:12.559 INFO:tasks.workunit.client.1.smithi096.stdout:3/451: truncate d0/d13/f21 6775064 0 2024-03-24T10:46:12.559 INFO:tasks.workunit.client.1.smithi096.stdout:3/452: fdatasync d0/d13/d1e/f2a 0 2024-03-24T10:46:12.561 INFO:tasks.workunit.client.1.smithi096.stdout:3/453: link d0/c3f d0/d13/d45/d47/c97 0 2024-03-24T10:46:12.563 INFO:tasks.workunit.client.1.smithi096.stdout:3/454: mknod d0/d13/d45/d7d/d39/d48/d85/c98 0 2024-03-24T10:46:12.569 INFO:tasks.workunit.client.1.smithi096.stdout:2/346: sync 2024-03-24T10:46:12.569 INFO:tasks.workunit.client.1.smithi096.stdout:6/394: sync 2024-03-24T10:46:12.569 INFO:tasks.workunit.client.1.smithi096.stdout:9/399: sync 2024-03-24T10:46:12.569 INFO:tasks.workunit.client.1.smithi096.stdout:9/400: stat d9/d15/d26/d29/d73/d3e/f3f 0 2024-03-24T10:46:12.569 INFO:tasks.workunit.client.1.smithi096.stdout:9/401: fdatasync d9/f2f 0 2024-03-24T10:46:12.569 INFO:tasks.workunit.client.1.smithi096.stdout:0/390: sync 2024-03-24T10:46:12.569 INFO:tasks.workunit.client.1.smithi096.stdout:4/339: dwrite d1/f35 [0,4194304] 0 2024-03-24T10:46:12.570 INFO:tasks.workunit.client.1.smithi096.stdout:6/395: mknod d8/d12/d28/d36/c74 0 2024-03-24T10:46:12.571 INFO:tasks.workunit.client.1.smithi096.stdout:0/391: dread d0/d11/d16/d21/d25/d47/f6b [0,4194304] 0 2024-03-24T10:46:12.571 INFO:tasks.workunit.client.1.smithi096.stdout:0/392: chown d0/d11/d16/d21/d25/d47/d63/d80/f5d 139231 1 2024-03-24T10:46:12.572 INFO:tasks.workunit.client.1.smithi096.stdout:5/390: dwrite d2/d7/f52 [0,4194304] 0 2024-03-24T10:46:12.574 INFO:tasks.workunit.client.1.smithi096.stdout:3/455: dread d0/d13/d1e/f2a [0,4194304] 0 2024-03-24T10:46:12.574 INFO:tasks.workunit.client.1.smithi096.stdout:3/456: stat d0/d13/d45/d7d/f94 0 2024-03-24T10:46:12.574 INFO:tasks.workunit.client.1.smithi096.stdout:9/402: unlink d9/d15/f1e 0 2024-03-24T10:46:12.574 INFO:tasks.workunit.client.1.smithi096.stdout:9/403: truncate d9/d15/d26/d29/f6f 692283 0 2024-03-24T10:46:12.575 INFO:tasks.workunit.client.1.smithi096.stdout:4/340: unlink d1/d32/d4c/f50 0 2024-03-24T10:46:12.576 INFO:tasks.workunit.client.1.smithi096.stdout:3/457: dread d0/d13/f1f [0,4194304] 0 2024-03-24T10:46:12.577 INFO:tasks.workunit.client.1.smithi096.stdout:6/396: creat d8/d12/d28/d36/d65/d6b/f75 x:0 0 0 2024-03-24T10:46:12.578 INFO:tasks.workunit.client.1.smithi096.stdout:0/393: mkdir d0/d11/d16/d21/d78/d82 0 2024-03-24T10:46:12.581 INFO:tasks.workunit.client.1.smithi096.stdout:5/391: creat d2/d7/d8/df/d14/f85 x:0 0 0 2024-03-24T10:46:12.584 INFO:tasks.workunit.client.1.smithi096.stdout:9/404: creat d9/d15/d26/d29/d73/d49/f7d x:0 0 0 2024-03-24T10:46:12.585 INFO:tasks.workunit.client.1.smithi096.stdout:5/392: dread d2/f43 [0,4194304] 0 2024-03-24T10:46:12.585 INFO:tasks.workunit.client.1.smithi096.stdout:4/341: rmdir d1/d11 39 2024-03-24T10:46:12.586 INFO:tasks.workunit.client.1.smithi096.stdout:4/342: read d1/d32/d4c/f5a [3155714,17751] 0 2024-03-24T10:46:12.587 INFO:tasks.workunit.client.1.smithi096.stdout:3/458: symlink d0/d13/d45/d7d/d71/l99 0 2024-03-24T10:46:12.587 INFO:tasks.workunit.client.1.smithi096.stdout:4/343: dread d1/d32/d42/d44/f62 [0,4194304] 0 2024-03-24T10:46:12.587 INFO:tasks.workunit.client.1.smithi096.stdout:3/459: write d0/d13/d45/d7d/f94 [1149232,56400] 0 2024-03-24T10:46:12.597 INFO:tasks.workunit.client.1.smithi096.stdout:6/397: mkdir d8/d10/d76 0 2024-03-24T10:46:12.598 INFO:tasks.workunit.client.1.smithi096.stdout:9/405: write d9/db/d6d/f35 [2708599,73275] 0 2024-03-24T10:46:12.599 INFO:tasks.workunit.client.1.smithi096.stdout:2/347: dwrite d5/d38/f7b [0,4194304] 0 2024-03-24T10:46:12.601 INFO:tasks.workunit.client.1.smithi096.stdout:0/394: unlink d0/d11/d20/d2b/l39 0 2024-03-24T10:46:12.604 INFO:tasks.workunit.client.1.smithi096.stdout:5/393: rename d2/d7/f41 to d2/d7/f86 0 2024-03-24T10:46:12.608 INFO:tasks.workunit.client.1.smithi096.stdout:4/344: creat d1/f6f x:0 0 0 2024-03-24T10:46:12.610 INFO:tasks.workunit.client.1.smithi096.stdout:2/348: rename d5/de/d86 to d5/d7c/d88 0 2024-03-24T10:46:12.614 INFO:tasks.workunit.client.1.smithi096.stdout:0/395: rmdir d0/d11/d16 39 2024-03-24T10:46:12.615 INFO:tasks.workunit.client.1.smithi096.stdout:9/406: write d9/d15/d26/d29/d73/d43/f45 [3476991,128986] 0 2024-03-24T10:46:12.615 INFO:tasks.workunit.client.1.smithi096.stdout:9/407: truncate d9/d15/d26/d29/d73/d43/f6c 618110 0 2024-03-24T10:46:12.617 INFO:tasks.workunit.client.1.smithi096.stdout:5/394: dread d2/d7/f10 [0,4194304] 0 2024-03-24T10:46:12.617 INFO:tasks.workunit.client.1.smithi096.stdout:5/395: dread - d2/d29/d32/d3f/f66 zero size 2024-03-24T10:46:12.617 INFO:tasks.workunit.client.1.smithi096.stdout:5/396: stat d2/d7/d8/d18/d4d 0 2024-03-24T10:46:12.617 INFO:tasks.workunit.client.1.smithi096.stdout:5/397: getdents d2/d49 0 2024-03-24T10:46:12.617 INFO:tasks.workunit.client.1.smithi096.stdout:2/349: symlink d5/dc/d6b/l89 0 2024-03-24T10:46:12.618 INFO:tasks.workunit.client.1.smithi096.stdout:0/396: symlink d0/d11/d43/l83 0 2024-03-24T10:46:12.619 INFO:tasks.workunit.client.1.smithi096.stdout:9/408: creat d9/d60/f7e x:0 0 0 2024-03-24T10:46:12.621 INFO:tasks.workunit.client.1.smithi096.stdout:2/350: unlink d5/dc/f27 0 2024-03-24T10:46:12.621 INFO:tasks.workunit.client.1.smithi096.stdout:2/351: fsync d5/de/f3d 0 2024-03-24T10:46:12.622 INFO:tasks.workunit.client.1.smithi096.stdout:3/460: dwrite d0/d13/d45/d52/f78 [0,4194304] 0 2024-03-24T10:46:12.623 INFO:tasks.workunit.client.1.smithi096.stdout:0/397: mkdir d0/d11/d24/d2c/d84 0 2024-03-24T10:46:12.623 INFO:tasks.workunit.client.1.smithi096.stdout:2/352: creat d5/dc/d6b/f8a x:0 0 0 2024-03-24T10:46:12.627 INFO:tasks.workunit.client.1.smithi096.stdout:5/398: dread d2/d7/f4b [0,4194304] 0 2024-03-24T10:46:12.627 INFO:tasks.workunit.client.1.smithi096.stdout:5/399: fsync d2/d29/d32/d33/d3c/d3d/f75 0 2024-03-24T10:46:12.627 INFO:tasks.workunit.client.1.smithi096.stdout:2/353: mknod d5/d79/c8b 0 2024-03-24T10:46:12.628 INFO:tasks.workunit.client.1.smithi096.stdout:0/398: write d0/f5 [999392,124905] 0 2024-03-24T10:46:12.628 INFO:tasks.workunit.client.1.smithi096.stdout:6/398: dwrite f6 [0,4194304] 0 2024-03-24T10:46:12.629 INFO:tasks.workunit.client.1.smithi096.stdout:4/345: dwrite d1/d2b/d40/f46 [0,4194304] 0 2024-03-24T10:46:12.630 INFO:tasks.workunit.client.1.smithi096.stdout:4/346: dread d1/d32/d42/d44/f62 [0,4194304] 0 2024-03-24T10:46:12.631 INFO:tasks.workunit.client.1.smithi096.stdout:9/409: write d9/d15/d26/d29/d73/d3e/f59 [1512589,81668] 0 2024-03-24T10:46:12.631 INFO:tasks.workunit.client.1.smithi096.stdout:3/461: getdents d0/d13/d1e 0 2024-03-24T10:46:12.634 INFO:tasks.workunit.client.1.smithi096.stdout:5/400: creat d2/d7/d69/f87 x:0 0 0 2024-03-24T10:46:12.638 INFO:tasks.workunit.client.1.smithi096.stdout:2/354: truncate d5/de/d2a/f2b 511279 0 2024-03-24T10:46:12.638 INFO:tasks.workunit.client.1.smithi096.stdout:2/355: creat d5/d7c/f8c x:0 0 0 2024-03-24T10:46:12.639 INFO:tasks.workunit.client.1.smithi096.stdout:0/399: symlink d0/d11/d43/d75/l85 0 2024-03-24T10:46:12.643 INFO:tasks.workunit.client.1.smithi096.stdout:6/399: rename d8/d12/l23 to d8/d12/d32/d66/l77 0 2024-03-24T10:46:12.643 INFO:tasks.workunit.client.1.smithi096.stdout:6/400: fsync d8/d10/d58/f72 0 2024-03-24T10:46:12.643 INFO:tasks.workunit.client.1.smithi096.stdout:6/401: readlink d8/d12/d32/d66/l77 0 2024-03-24T10:46:12.643 INFO:tasks.workunit.client.1.smithi096.stdout:9/410: dwrite d9/f3a [4194304,4194304] 0 2024-03-24T10:46:12.648 INFO:tasks.workunit.client.1.smithi096.stdout:0/400: write d0/d11/d24/f30 [934784,54774] 0 2024-03-24T10:46:12.662 INFO:tasks.workunit.client.1.smithi096.stdout:3/462: link d0/d9/l23 d0/d13/d45/d7d/d39/d48/d85/l9a 0 2024-03-24T10:46:12.663 INFO:tasks.workunit.client.1.smithi096.stdout:5/401: link d2/d29/d32/d33/l76 d2/d7/d8/l88 0 2024-03-24T10:46:12.669 INFO:tasks.workunit.client.1.smithi096.stdout:4/347: rename d1/d2b/d59/f5b to d1/d2b/d40/f70 0 2024-03-24T10:46:12.669 INFO:tasks.workunit.client.1.smithi096.stdout:4/348: chown d1/d32/d4c/l5c 21364086 1 2024-03-24T10:46:12.670 INFO:tasks.workunit.client.1.smithi096.stdout:4/349: readlink d1/d32/l33 0 2024-03-24T10:46:12.671 INFO:tasks.workunit.client.1.smithi096.stdout:6/402: symlink d8/d12/l78 0 2024-03-24T10:46:12.676 INFO:tasks.workunit.client.1.smithi096.stdout:0/401: link d0/d11/d43/d75/l85 d0/d11/d24/l86 0 2024-03-24T10:46:12.676 INFO:tasks.workunit.client.1.smithi096.stdout:0/402: dread - d0/d11/f66 zero size 2024-03-24T10:46:12.682 INFO:tasks.workunit.client.1.smithi096.stdout:3/463: unlink d0/d9/d22/c3a 0 2024-03-24T10:46:12.684 INFO:tasks.workunit.client.1.smithi096.stdout:9/411: rmdir d9/d15/d26/d29 39 2024-03-24T10:46:12.686 INFO:tasks.workunit.client.1.smithi096.stdout:0/403: creat d0/d11/d24/f87 x:0 0 0 2024-03-24T10:46:12.687 INFO:tasks.workunit.client.1.smithi096.stdout:6/403: write f2 [4115493,60262] 0 2024-03-24T10:46:12.688 INFO:tasks.workunit.client.1.smithi096.stdout:0/404: write d0/d11/f1c [4882837,97436] 0 2024-03-24T10:46:12.689 INFO:tasks.workunit.client.1.smithi096.stdout:5/402: dwrite d2/d7/d8/f12 [4194304,4194304] 0 2024-03-24T10:46:12.694 INFO:tasks.workunit.client.1.smithi096.stdout:0/405: creat d0/d11/d43/d75/f88 x:0 0 0 2024-03-24T10:46:12.696 INFO:tasks.workunit.client.1.smithi096.stdout:5/403: dread d2/d29/f54 [0,4194304] 0 2024-03-24T10:46:12.698 INFO:tasks.workunit.client.1.smithi096.stdout:5/404: write d2/f39 [1921642,99143] 0 2024-03-24T10:46:12.698 INFO:tasks.workunit.client.1.smithi096.stdout:5/405: chown d2/d29/d32/d33/d3c/d3d 269512 1 2024-03-24T10:46:12.698 INFO:tasks.workunit.client.1.smithi096.stdout:5/406: fsync d2/d49/f5d 0 2024-03-24T10:46:12.698 INFO:tasks.workunit.client.1.smithi096.stdout:6/404: dread d8/d10/d58/f72 [0,4194304] 0 2024-03-24T10:46:12.698 INFO:tasks.workunit.client.1.smithi096.stdout:0/406: truncate d0/d11/d16/d21/d81/d7f/f4a 2076620 0 2024-03-24T10:46:12.699 INFO:tasks.workunit.client.1.smithi096.stdout:0/407: fdatasync d0/d11/d24/f4c 0 2024-03-24T10:46:12.699 INFO:tasks.workunit.client.1.smithi096.stdout:2/356: dwrite d5/d38/f3a [0,4194304] 0 2024-03-24T10:46:12.703 INFO:tasks.workunit.client.1.smithi096.stdout:0/408: creat d0/d11/d20/d2b/f89 x:0 0 0 2024-03-24T10:46:12.706 INFO:tasks.workunit.client.1.smithi096.stdout:2/357: creat d5/de/d2a/d5e/f8d x:0 0 0 2024-03-24T10:46:12.706 INFO:tasks.workunit.client.1.smithi096.stdout:2/358: dread - d5/dc/f83 zero size 2024-03-24T10:46:12.710 INFO:tasks.workunit.client.1.smithi096.stdout:0/409: mkdir d0/d11/d20/d65/d8a 0 2024-03-24T10:46:12.712 INFO:tasks.workunit.client.1.smithi096.stdout:0/410: dread d0/f5 [0,4194304] 0 2024-03-24T10:46:12.714 INFO:tasks.workunit.client.1.smithi096.stdout:0/411: mknod d0/c8b 0 2024-03-24T10:46:12.714 INFO:tasks.workunit.client.1.smithi096.stdout:0/412: fsync d0/d11/d24/f50 0 2024-03-24T10:46:12.715 INFO:tasks.workunit.client.1.smithi096.stdout:0/413: dread d0/d11/d20/f26 [0,4194304] 0 2024-03-24T10:46:12.715 INFO:tasks.workunit.client.1.smithi096.stdout:0/414: readlink d0/l23 0 2024-03-24T10:46:12.715 INFO:tasks.workunit.client.1.smithi096.stdout:6/405: rmdir d8/d12/d32 39 2024-03-24T10:46:12.717 INFO:tasks.workunit.client.1.smithi096.stdout:0/415: creat d0/d11/d20/d65/f8c x:0 0 0 2024-03-24T10:46:12.719 INFO:tasks.workunit.client.1.smithi096.stdout:6/406: rmdir d8/d12/d28/d36/d65/d6b 39 2024-03-24T10:46:12.721 INFO:tasks.workunit.client.1.smithi096.stdout:9/412: dwrite d9/f1a [4194304,4194304] 0 2024-03-24T10:46:12.721 INFO:tasks.workunit.client.1.smithi096.stdout:9/413: readlink d9/d15/d26/d29/d73/d3e/l5b 0 2024-03-24T10:46:12.721 INFO:tasks.workunit.client.1.smithi096.stdout:5/407: dwrite d2/d7/f4a [0,4194304] 0 2024-03-24T10:46:12.721 INFO:tasks.workunit.client.1.smithi096.stdout:5/408: chown d2/d7/d8/d18/f72 19 1 2024-03-24T10:46:12.721 INFO:tasks.workunit.client.1.smithi096.stdout:6/407: creat d8/d12/d32/d38/d59/f79 x:0 0 0 2024-03-24T10:46:12.723 INFO:tasks.workunit.client.1.smithi096.stdout:9/414: rename d9/d15/d26/d29/d73/d3e to d9/db/d6d/d7f 0 2024-03-24T10:46:12.725 INFO:tasks.workunit.client.1.smithi096.stdout:5/409: mkdir d2/d29/d32/d89 0 2024-03-24T10:46:12.728 INFO:tasks.workunit.client.1.smithi096.stdout:6/408: symlink d8/d12/d28/d36/d65/d71/l7a 0 2024-03-24T10:46:12.730 INFO:tasks.workunit.client.1.smithi096.stdout:9/415: creat d9/db/f80 x:0 0 0 2024-03-24T10:46:12.731 INFO:tasks.workunit.client.1.smithi096.stdout:9/416: dread d9/d15/d26/d29/f6f [0,4194304] 0 2024-03-24T10:46:12.732 INFO:tasks.workunit.client.1.smithi096.stdout:6/409: mkdir d8/d7b 0 2024-03-24T10:46:12.732 INFO:tasks.workunit.client.1.smithi096.stdout:6/410: truncate d8/d12/f69 561288 0 2024-03-24T10:46:12.734 INFO:tasks.workunit.client.1.smithi096.stdout:9/417: mknod d9/db/d6d/c81 0 2024-03-24T10:46:12.736 INFO:tasks.workunit.client.1.smithi096.stdout:6/411: unlink d8/d10/l57 0 2024-03-24T10:46:12.737 INFO:tasks.workunit.client.1.smithi096.stdout:0/416: dwrite d0/d11/d16/d21/d81/d7f/f64 [0,4194304] 0 2024-03-24T10:46:12.739 INFO:tasks.workunit.client.1.smithi096.stdout:6/412: mknod d8/d12/d28/d36/c7c 0 2024-03-24T10:46:12.740 INFO:tasks.workunit.client.1.smithi096.stdout:5/410: dwrite d2/d7/f4a [0,4194304] 0 2024-03-24T10:46:12.745 INFO:tasks.workunit.client.1.smithi096.stdout:0/417: rename d0/d11/d24/d2c/c7e to d0/d11/d16/d21/d25/d47/c8d 0 2024-03-24T10:46:12.748 INFO:tasks.workunit.client.1.smithi096.stdout:5/411: mkdir d2/d7/d8/df/d8a 0 2024-03-24T10:46:12.750 INFO:tasks.workunit.client.1.smithi096.stdout:6/413: rename d8/d12/f69 to d8/d12/d28/d36/f7d 0 2024-03-24T10:46:12.750 INFO:tasks.workunit.client.1.smithi096.stdout:6/414: fdatasync d8/d10/d13/d63/f21 0 2024-03-24T10:46:12.752 INFO:tasks.workunit.client.1.smithi096.stdout:9/418: dwrite d9/f5c [0,4194304] 0 2024-03-24T10:46:12.753 INFO:tasks.workunit.client.1.smithi096.stdout:0/418: creat d0/f8e x:0 0 0 2024-03-24T10:46:12.755 INFO:tasks.workunit.client.1.smithi096.stdout:5/412: truncate d2/d7/d8/df/d14/f31 2674727 0 2024-03-24T10:46:12.757 INFO:tasks.workunit.client.1.smithi096.stdout:6/415: mkdir d8/d12/d61/d7e 0 2024-03-24T10:46:12.767 INFO:tasks.workunit.client.1.smithi096.stdout:9/419: creat d9/d7c/f82 x:0 0 0 2024-03-24T10:46:12.768 INFO:tasks.workunit.client.1.smithi096.stdout:0/419: mkdir d0/d11/d16/d21/d81/d8f 0 2024-03-24T10:46:12.770 INFO:tasks.workunit.client.1.smithi096.stdout:5/413: creat d2/f8b x:0 0 0 2024-03-24T10:46:12.770 INFO:tasks.workunit.client.1.smithi096.stdout:5/414: creat d2/d7/f8c x:0 0 0 2024-03-24T10:46:12.774 INFO:tasks.workunit.client.1.smithi096.stdout:9/420: link d9/d15/d26/d29/d73/d43/c6b d9/d15/d26/d29/c83 0 2024-03-24T10:46:12.776 INFO:tasks.workunit.client.1.smithi096.stdout:6/416: dwrite d8/f62 [0,4194304] 0 2024-03-24T10:46:12.776 INFO:tasks.workunit.client.1.smithi096.stdout:0/420: write d0/d11/d16/d21/d25/d47/d63/d80/f5d [4120292,71614] 0 2024-03-24T10:46:12.779 INFO:tasks.workunit.client.1.smithi096.stdout:5/415: unlink d2/d29/d32/d33/d3c/f74 0 2024-03-24T10:46:12.783 INFO:tasks.workunit.client.1.smithi096.stdout:9/421: unlink d9/db/f22 0 2024-03-24T10:46:12.784 INFO:tasks.workunit.client.1.smithi096.stdout:6/417: symlink d8/d10/l7f 0 2024-03-24T10:46:12.786 INFO:tasks.workunit.client.1.smithi096.stdout:0/421: symlink d0/d11/d16/d21/d25/d47/d63/l90 0 2024-03-24T10:46:12.789 INFO:tasks.workunit.client.1.smithi096.stdout:9/422: link d9/db/l55 d9/d15/d26/d29/d73/d49/l84 0 2024-03-24T10:46:12.791 INFO:tasks.workunit.client.1.smithi096.stdout:6/418: creat d8/d12/d32/f80 x:0 0 0 2024-03-24T10:46:12.793 INFO:tasks.workunit.client.1.smithi096.stdout:0/422: unlink d0/d11/d16/d21/d25/d47/l5f 0 2024-03-24T10:46:12.796 INFO:tasks.workunit.client.1.smithi096.stdout:9/423: link d9/d15/d26/d29/d73/d43/f44 d9/d15/d26/d29/d73/d43/f85 0 2024-03-24T10:46:12.796 INFO:tasks.workunit.client.1.smithi096.stdout:5/416: dwrite d2/d49/f5d [4194304,4194304] 0 2024-03-24T10:46:12.797 INFO:tasks.workunit.client.1.smithi096.stdout:5/417: chown d2/d29 104053 1 2024-03-24T10:46:12.798 INFO:tasks.workunit.client.1.smithi096.stdout:6/419: creat d8/f81 x:0 0 0 2024-03-24T10:46:12.800 INFO:tasks.workunit.client.1.smithi096.stdout:9/424: mkdir d9/d15/d26/d29/d73/d49/d86 0 2024-03-24T10:46:12.800 INFO:tasks.workunit.client.1.smithi096.stdout:9/425: chown d9/f1a 23219306 1 2024-03-24T10:46:12.800 INFO:tasks.workunit.client.1.smithi096.stdout:9/426: read d9/db/d6d/d7f/f3f [253644,24812] 0 2024-03-24T10:46:12.803 INFO:tasks.workunit.client.1.smithi096.stdout:2/359: mknod d5/d38/c8e 0 2024-03-24T10:46:12.809 INFO:tasks.workunit.client.1.smithi096.stdout:5/418: unlink d2/d7/d8/df/f19 0 2024-03-24T10:46:12.810 INFO:tasks.workunit.client.1.smithi096.stdout:9/427: mknod d9/c87 0 2024-03-24T10:46:12.814 INFO:tasks.workunit.client.1.smithi096.stdout:2/360: mkdir d5/d7c/d8f 0 2024-03-24T10:46:12.814 INFO:tasks.workunit.client.1.smithi096.stdout:2/361: fdatasync d5/de/d2a/f5a 0 2024-03-24T10:46:12.814 INFO:tasks.workunit.client.1.smithi096.stdout:5/419: mkdir d2/d29/d32/d33/d8d 0 2024-03-24T10:46:12.814 INFO:tasks.workunit.client.1.smithi096.stdout:9/428: creat d9/db/d6d/d7f/f88 x:0 0 0 2024-03-24T10:46:12.816 INFO:tasks.workunit.client.1.smithi096.stdout:2/362: mkdir d5/d7c/d88/d90 0 2024-03-24T10:46:12.816 INFO:tasks.workunit.client.1.smithi096.stdout:9/429: symlink d9/d79/l89 0 2024-03-24T10:46:12.819 INFO:tasks.workunit.client.1.smithi096.stdout:9/430: unlink d9/d15/d26/d29/d73/l71 0 2024-03-24T10:46:12.825 INFO:tasks.workunit.client.1.smithi096.stdout:0/423: dwrite d0/d11/d24/f45 [0,4194304] 0 2024-03-24T10:46:12.827 INFO:tasks.workunit.client.1.smithi096.stdout:2/363: read d5/d38/f66 [3552530,63668] 0 2024-03-24T10:46:12.829 INFO:tasks.workunit.client.1.smithi096.stdout:0/424: mkdir d0/d11/d20/d91 0 2024-03-24T10:46:12.829 INFO:tasks.workunit.client.1.smithi096.stdout:0/425: dread - d0/d11/d43/d75/f88 zero size 2024-03-24T10:46:12.829 INFO:tasks.workunit.client.1.smithi096.stdout:6/420: dwrite d8/d12/f1f [4194304,4194304] 0 2024-03-24T10:46:12.832 INFO:tasks.workunit.client.1.smithi096.stdout:2/364: mkdir d5/dc/d6b/d72/d91 0 2024-03-24T10:46:12.832 INFO:tasks.workunit.client.1.smithi096.stdout:2/365: fdatasync d5/dc/d12/f16 0 2024-03-24T10:46:12.832 INFO:tasks.workunit.client.1.smithi096.stdout:2/366: stat d5/d7c/f7e 0 2024-03-24T10:46:12.832 INFO:tasks.workunit.client.1.smithi096.stdout:2/367: chown d5/dc/d12/d23/l4e 5 1 2024-03-24T10:46:12.832 INFO:tasks.workunit.client.1.smithi096.stdout:2/368: fsync d5/de/f51 0 2024-03-24T10:46:12.832 INFO:tasks.workunit.client.1.smithi096.stdout:2/369: dread - d5/dc/d12/d23/f7a zero size 2024-03-24T10:46:12.835 INFO:tasks.workunit.client.1.smithi096.stdout:0/426: mkdir d0/d11/d16/d21/d78/d92 0 2024-03-24T10:46:12.838 INFO:tasks.workunit.client.1.smithi096.stdout:5/420: dwrite d2/d29/d32/d33/d3c/d3d/f75 [0,4194304] 0 2024-03-24T10:46:12.839 INFO:tasks.workunit.client.1.smithi096.stdout:6/421: creat d8/d12/d32/d38/f82 x:0 0 0 2024-03-24T10:46:12.839 INFO:tasks.workunit.client.1.smithi096.stdout:6/422: creat d8/f83 x:0 0 0 2024-03-24T10:46:12.839 INFO:tasks.workunit.client.1.smithi096.stdout:6/423: write d8/d12/f5c [863816,127151] 0 2024-03-24T10:46:12.842 INFO:tasks.workunit.client.1.smithi096.stdout:2/370: mknod d5/d7c/d88/d90/c92 0 2024-03-24T10:46:12.843 INFO:tasks.workunit.client.1.smithi096.stdout:0/427: symlink d0/d11/d24/l93 0 2024-03-24T10:46:12.843 INFO:tasks.workunit.client.1.smithi096.stdout:0/428: fsync d0/d11/d24/d2c/f32 0 2024-03-24T10:46:12.844 INFO:tasks.workunit.client.1.smithi096.stdout:5/421: creat d2/d29/d32/d89/f8e x:0 0 0 2024-03-24T10:46:12.848 INFO:tasks.workunit.client.1.smithi096.stdout:0/429: creat d0/d11/d16/d21/d81/d7f/f94 x:0 0 0 2024-03-24T10:46:12.855 INFO:tasks.workunit.client.1.smithi096.stdout:2/371: dread d5/dc/d12/f31 [0,4194304] 0 2024-03-24T10:46:12.863 INFO:tasks.workunit.client.1.smithi096.stdout:6/424: dwrite d8/d12/d32/f80 [0,4194304] 0 2024-03-24T10:46:12.867 INFO:tasks.workunit.client.1.smithi096.stdout:6/425: mknod d8/d10/d19/c84 0 2024-03-24T10:46:12.868 INFO:tasks.workunit.client.1.smithi096.stdout:6/426: chown d8 308783 1 2024-03-24T10:46:12.870 INFO:tasks.workunit.client.1.smithi096.stdout:5/422: dwrite d2/d7/f52 [0,4194304] 0 2024-03-24T10:46:12.870 INFO:tasks.workunit.client.1.smithi096.stdout:5/423: write d2/d7/f7f [453707,125905] 0 2024-03-24T10:46:12.873 INFO:tasks.workunit.client.1.smithi096.stdout:5/424: creat d2/d29/d32/d3f/d5f/f8f x:0 0 0 2024-03-24T10:46:12.874 INFO:tasks.workunit.client.1.smithi096.stdout:0/430: dwrite d0/d11/d24/d2c/f76 [0,4194304] 0 2024-03-24T10:46:12.875 INFO:tasks.workunit.client.1.smithi096.stdout:0/431: read - d0/d11/d24/d2c/f67 zero size 2024-03-24T10:46:12.875 INFO:tasks.workunit.client.1.smithi096.stdout:0/432: chown d0/d11/d20/d2b/f3d 3 1 2024-03-24T10:46:12.875 INFO:tasks.workunit.client.1.smithi096.stdout:0/433: write d0/d11/d24/f4c [612732,7986] 0 2024-03-24T10:46:12.876 INFO:tasks.workunit.client.1.smithi096.stdout:5/425: unlink d2/d29/d32/d33/l76 0 2024-03-24T10:46:12.876 INFO:tasks.workunit.client.1.smithi096.stdout:5/426: write d2/d29/f54 [39649,36315] 0 2024-03-24T10:46:12.876 INFO:tasks.workunit.client.1.smithi096.stdout:5/427: write d2/d7/d8/df/d14/f85 [133267,33289] 0 2024-03-24T10:46:12.885 INFO:tasks.workunit.client.1.smithi096.stdout:2/372: dwrite d5/de/d2a/f4c [0,4194304] 0 2024-03-24T10:46:12.891 INFO:tasks.workunit.client.1.smithi096.stdout:2/373: mkdir d5/d93 0 2024-03-24T10:46:12.906 INFO:tasks.workunit.client.1.smithi096.stdout:5/428: dwrite d2/d7/d8/df/f2d [0,4194304] 0 2024-03-24T10:46:12.906 INFO:tasks.workunit.client.1.smithi096.stdout:5/429: creat d2/d29/f90 x:0 0 0 2024-03-24T10:46:12.906 INFO:tasks.workunit.client.1.smithi096.stdout:5/430: dread - d2/d7/d69/f87 zero size 2024-03-24T10:46:12.909 INFO:tasks.workunit.client.1.smithi096.stdout:0/434: dwrite d0/f5 [0,4194304] 0 2024-03-24T10:46:12.909 INFO:tasks.workunit.client.1.smithi096.stdout:0/435: creat d0/d11/d24/f95 x:0 0 0 2024-03-24T10:46:12.909 INFO:tasks.workunit.client.1.smithi096.stdout:0/436: fsync d0/d11/d20/f26 0 2024-03-24T10:46:12.918 INFO:tasks.workunit.client.1.smithi096.stdout:0/437: unlink d0/d11/d16/f2f 0 2024-03-24T10:46:12.930 INFO:tasks.workunit.client.1.smithi096.stdout:0/438: write d0/d11/d16/d21/d81/d7f/f64 [2744553,102433] 0 2024-03-24T10:46:12.930 INFO:tasks.workunit.client.1.smithi096.stdout:0/439: stat d0/d11/d16/d21/d25/d47/d63/d80 0 2024-03-24T10:46:12.930 INFO:tasks.workunit.client.1.smithi096.stdout:0/440: read - d0/d11/d24/f95 zero size 2024-03-24T10:46:12.930 INFO:tasks.workunit.client.1.smithi096.stdout:0/441: read - d0/d11/f6d zero size 2024-03-24T10:46:12.930 INFO:tasks.workunit.client.1.smithi096.stdout:0/442: dread - d0/d11/d16/d21/d81/d7f/f94 zero size 2024-03-24T10:46:12.931 INFO:tasks.workunit.client.1.smithi096.stdout:5/431: dwrite d2/d29/f38 [4194304,4194304] 0 2024-03-24T10:46:12.936 INFO:tasks.workunit.client.1.smithi096.stdout:0/443: link d0/d11/d24/f4c d0/d11/d20/d91/f96 0 2024-03-24T10:46:12.936 INFO:tasks.workunit.client.1.smithi096.stdout:0/444: truncate d0/d11/d20/d2b/f89 776109 0 2024-03-24T10:46:12.936 INFO:tasks.workunit.client.1.smithi096.stdout:0/445: chown d0 3 1 2024-03-24T10:46:12.943 INFO:tasks.workunit.client.1.smithi096.stdout:5/432: link d2/d29/f54 d2/d7/d69/f91 0 2024-03-24T10:46:12.943 INFO:tasks.workunit.client.1.smithi096.stdout:5/433: readlink d2/d29/l65 0 2024-03-24T10:46:12.945 INFO:tasks.workunit.client.1.smithi096.stdout:0/446: dread d0/d11/d24/f30 [0,4194304] 0 2024-03-24T10:46:12.948 INFO:tasks.workunit.client.1.smithi096.stdout:0/447: mknod d0/d11/c97 0 2024-03-24T10:46:12.951 INFO:tasks.workunit.client.1.smithi096.stdout:1/370: sync 2024-03-24T10:46:12.951 INFO:tasks.workunit.client.1.smithi096.stdout:7/628: sync 2024-03-24T10:46:12.954 INFO:tasks.workunit.client.1.smithi096.stdout:1/371: dread d3/d23/d2f/f5a [0,4194304] 0 2024-03-24T10:46:12.954 INFO:tasks.workunit.client.1.smithi096.stdout:0/448: symlink d0/d11/d16/d21/d78/d92/l98 0 2024-03-24T10:46:12.955 INFO:tasks.workunit.client.1.smithi096.stdout:0/449: write d0/d11/d20/d65/f8c [624650,35668] 0 2024-03-24T10:46:12.955 INFO:tasks.workunit.client.1.smithi096.stdout:0/450: chown d0/d11/d24/f55 17 1 2024-03-24T10:46:12.955 INFO:tasks.workunit.client.1.smithi096.stdout:0/451: write d0/f48 [1664269,82888] 0 2024-03-24T10:46:12.956 INFO:tasks.workunit.client.1.smithi096.stdout:1/372: dread d3/d23/d2f/f5a [0,4194304] 0 2024-03-24T10:46:12.957 INFO:tasks.workunit.client.1.smithi096.stdout:8/441: sync 2024-03-24T10:46:12.957 INFO:tasks.workunit.client.1.smithi096.stdout:8/442: readlink d8/d11/l7c 0 2024-03-24T10:46:12.957 INFO:tasks.workunit.client.1.smithi096.stdout:4/350: sync 2024-03-24T10:46:12.957 INFO:tasks.workunit.client.1.smithi096.stdout:4/351: write d1/d11/f12 [2862078,105844] 0 2024-03-24T10:46:12.958 INFO:tasks.workunit.client.1.smithi096.stdout:4/352: write d1/f8 [56716,81559] 0 2024-03-24T10:46:12.962 INFO:tasks.workunit.client.1.smithi096.stdout:7/629: creat d0/fd2 x:0 0 0 2024-03-24T10:46:12.962 INFO:tasks.workunit.client.1.smithi096.stdout:7/630: fsync d0/d13/f35 0 2024-03-24T10:46:12.963 INFO:tasks.workunit.client.1.smithi096.stdout:0/452: creat d0/d11/d16/d21/d78/d92/f99 x:0 0 0 2024-03-24T10:46:12.964 INFO:tasks.workunit.client.1.smithi096.stdout:9/431: fsync d9/db/d6d/d7f/f88 0 2024-03-24T10:46:12.965 INFO:tasks.workunit.client.1.smithi096.stdout:7/631: dread d0/d13/d32/faa [0,4194304] 0 2024-03-24T10:46:12.965 INFO:tasks.workunit.client.1.smithi096.stdout:1/373: rename d3/c26 to d3/d36/c82 0 2024-03-24T10:46:12.965 INFO:tasks.workunit.client.1.smithi096.stdout:5/434: dwrite d2/f8b [0,4194304] 0 2024-03-24T10:46:12.967 INFO:tasks.workunit.client.1.smithi096.stdout:4/353: symlink d1/d32/d42/l71 0 2024-03-24T10:46:12.969 INFO:tasks.workunit.client.1.smithi096.stdout:0/453: mkdir d0/d11/d16/d21/d81/d8f/d9a 0 2024-03-24T10:46:12.970 INFO:tasks.workunit.client.1.smithi096.stdout:0/454: chown d0/d11/d43/d75 0 1 2024-03-24T10:46:12.970 INFO:tasks.workunit.client.1.smithi096.stdout:9/432: mkdir d9/d15/d26/d29/d73/d49/d8a 0 2024-03-24T10:46:12.974 INFO:tasks.workunit.client.1.smithi096.stdout:8/443: rename c4 to d8/d11/d22/d4c/d65/d1d/c90 0 2024-03-24T10:46:12.974 INFO:tasks.workunit.client.1.smithi096.stdout:8/444: fdatasync d8/d11/d22/d4c/d67/f7a 0 2024-03-24T10:46:12.976 INFO:tasks.workunit.client.1.smithi096.stdout:1/374: write d3/d24/d5c/f67 [634404,109075] 0 2024-03-24T10:46:12.976 INFO:tasks.workunit.client.1.smithi096.stdout:1/375: creat d3/d23/d2f/f83 x:0 0 0 2024-03-24T10:46:12.980 INFO:tasks.workunit.client.1.smithi096.stdout:2/374: rmdir d5/dc/d6b 39 2024-03-24T10:46:12.981 INFO:tasks.workunit.client.1.smithi096.stdout:3/464: sync 2024-03-24T10:46:12.984 INFO:tasks.workunit.client.1.smithi096.stdout:9/433: mkdir d9/db/d6d/d4f/d75/d8b 0 2024-03-24T10:46:12.991 INFO:tasks.workunit.client.1.smithi096.stdout:7/632: rmdir d0/d21/d3c 39 2024-03-24T10:46:12.995 INFO:tasks.workunit.client.1.smithi096.stdout:1/376: mkdir d3/d84 0 2024-03-24T10:46:12.996 INFO:tasks.workunit.client.1.smithi096.stdout:5/435: truncate d2/d7/d8/df/f78 101698 0 2024-03-24T10:46:12.998 INFO:tasks.workunit.client.1.smithi096.stdout:4/354: unlink d1/d2b/c43 0 2024-03-24T10:46:12.999 INFO:tasks.workunit.client.1.smithi096.stdout:2/375: truncate d5/dc/f1c 356352 0 2024-03-24T10:46:12.999 INFO:tasks.workunit.client.1.smithi096.stdout:2/376: chown d5/de/d2a/f78 16700 1 2024-03-24T10:46:13.001 INFO:tasks.workunit.client.1.smithi096.stdout:9/434: getdents d9/db/d6d/d7f 0 2024-03-24T10:46:13.001 INFO:tasks.workunit.client.1.smithi096.stdout:9/435: getdents d9/d15/d26/d29/d73/d49/d86 0 2024-03-24T10:46:13.002 INFO:tasks.workunit.client.1.smithi096.stdout:8/445: dwrite d8/fe [0,4194304] 0 2024-03-24T10:46:13.004 INFO:tasks.workunit.client.1.smithi096.stdout:7/633: unlink d0/d21/d59/ld1 0 2024-03-24T10:46:13.007 INFO:tasks.workunit.client.1.smithi096.stdout:1/377: mkdir d3/d23/d71/d85 0 2024-03-24T10:46:13.007 INFO:tasks.workunit.client.1.smithi096.stdout:2/377: read d5/de/d2a/f37 [5357270,12344] 0 2024-03-24T10:46:13.009 INFO:tasks.workunit.client.1.smithi096.stdout:4/355: rename d1/d11/l26 to d1/d11/l72 0 2024-03-24T10:46:13.010 INFO:tasks.workunit.client.1.smithi096.stdout:8/446: dread d8/d11/d22/d4c/d65/d1d/f30 [0,4194304] 0 2024-03-24T10:46:13.011 INFO:tasks.workunit.client.1.smithi096.stdout:3/465: dwrite d0/d13/d1e/f2f [0,4194304] 0 2024-03-24T10:46:13.011 INFO:tasks.workunit.client.1.smithi096.stdout:3/466: chown d0/c4a 1 1 2024-03-24T10:46:13.011 INFO:tasks.workunit.client.1.smithi096.stdout:3/467: write d0/d13/d45/d47/f76 [935003,47466] 0 2024-03-24T10:46:13.018 INFO:tasks.workunit.client.1.smithi096.stdout:1/378: dread d3/d23/f62 [0,4194304] 0 2024-03-24T10:46:13.018 INFO:tasks.workunit.client.1.smithi096.stdout:2/378: rmdir d5/dc/d6b 39 2024-03-24T10:46:13.018 INFO:tasks.workunit.client.1.smithi096.stdout:2/379: fsync d5/d7c/d88/f69 0 2024-03-24T10:46:13.018 INFO:tasks.workunit.client.1.smithi096.stdout:2/380: chown d5/d7c 0 1 2024-03-24T10:46:13.019 INFO:tasks.workunit.client.1.smithi096.stdout:5/436: truncate d2/d7/f2a 1634807 0 2024-03-24T10:46:13.028 INFO:tasks.workunit.client.1.smithi096.stdout:8/447: mkdir d8/d11/d22/d4c/d65/d1d/d86/d91 0 2024-03-24T10:46:13.028 INFO:tasks.workunit.client.1.smithi096.stdout:8/448: chown d8/d11/d22/d4c/f68 5589402 1 2024-03-24T10:46:13.028 INFO:tasks.workunit.client.1.smithi096.stdout:8/449: write d8/f16 [1283810,38392] 0 2024-03-24T10:46:13.028 INFO:tasks.workunit.client.1.smithi096.stdout:8/450: stat d8/d11/l7c 0 2024-03-24T10:46:13.028 INFO:tasks.workunit.client.1.smithi096.stdout:8/451: write d8/d11/d22/d4c/d65/d1d/f7b [323453,103960] 0 2024-03-24T10:46:13.032 INFO:tasks.workunit.client.1.smithi096.stdout:9/436: rename d9/d15/d26/d29/f6f to d9/db/f8c 0 2024-03-24T10:46:13.034 INFO:tasks.workunit.client.1.smithi096.stdout:3/468: mkdir d0/d9b 0 2024-03-24T10:46:13.034 INFO:tasks.workunit.client.1.smithi096.stdout:3/469: truncate d0/d13/d45/d7d/d71/f96 826559 0 2024-03-24T10:46:13.035 INFO:tasks.workunit.client.1.smithi096.stdout:7/634: symlink d0/db/d26/dc3/ld3 0 2024-03-24T10:46:13.040 INFO:tasks.workunit.client.1.smithi096.stdout:5/437: readlink d2/d7/d8/l7e 0 2024-03-24T10:46:13.040 INFO:tasks.workunit.client.1.smithi096.stdout:5/438: dread - d2/d29/d32/d33/f83 zero size 2024-03-24T10:46:13.042 INFO:tasks.workunit.client.1.smithi096.stdout:4/356: mknod d1/d2b/c73 0 2024-03-24T10:46:13.042 INFO:tasks.workunit.client.1.smithi096.stdout:4/357: write d1/d2b/d67/f6a [891430,81456] 0 2024-03-24T10:46:13.045 INFO:tasks.workunit.client.1.smithi096.stdout:4/358: dread d1/f24 [4194304,4194304] 0 2024-03-24T10:46:13.045 INFO:tasks.workunit.client.1.smithi096.stdout:4/359: chown d1/ff 0 1 2024-03-24T10:46:13.045 INFO:tasks.workunit.client.1.smithi096.stdout:4/360: readlink d1/d32/d4c/l61 0 2024-03-24T10:46:13.049 INFO:tasks.workunit.client.1.smithi096.stdout:8/452: dwrite d8/d11/d22/d4c/d65/d1d/f2b [0,4194304] 0 2024-03-24T10:46:13.049 INFO:tasks.workunit.client.1.smithi096.stdout:8/453: truncate d8/d11/d22/d4c/d65/d1d/d86/f87 246126 0 2024-03-24T10:46:13.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.059+0000 7f3c50ab7700 1 -- 172.21.15.94:0/2034941070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c4c0fbce0 msgr2=0x7f3c4c0fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:13.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.059+0000 7f3c50ab7700 1 --2- 172.21.15.94:0/2034941070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c4c0fbce0 0x7f3c4c0fc0f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c004940 tx=0x7f3c3c009e30 comp rx=0 tx=0).stop 2024-03-24T10:46:13.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.059+0000 7f3c50ab7700 1 -- 172.21.15.94:0/2034941070 shutdown_connections 2024-03-24T10:46:13.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.059+0000 7f3c50ab7700 1 --2- 172.21.15.94:0/2034941070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c4c0fcf90 0x7f3c4c0fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:13.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.059+0000 7f3c50ab7700 1 --2- 172.21.15.94:0/2034941070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c4c0fbce0 0x7f3c4c0fc0f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:13.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.059+0000 7f3c50ab7700 1 -- 172.21.15.94:0/2034941070 >> 172.21.15.94:0/2034941070 conn(0x7f3c4c0f7440 msgr2=0x7f3c4c0f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:46:13.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.059+0000 7f3c50ab7700 1 -- 172.21.15.94:0/2034941070 shutdown_connections 2024-03-24T10:46:13.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.059+0000 7f3c50ab7700 1 -- 172.21.15.94:0/2034941070 wait complete. 2024-03-24T10:46:13.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.060+0000 7f3c50ab7700 1 Processor -- start 2024-03-24T10:46:13.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.060+0000 7f3c50ab7700 1 -- start start 2024-03-24T10:46:13.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.060+0000 7f3c50ab7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c4c0fbce0 0x7f3c4c103e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:46:13.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.060+0000 7f3c50ab7700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c4c0fcf90 0x7f3c4c1022f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:46:13.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.060+0000 7f3c50ab7700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4c1028a0 con 0x7f3c4c0fcf90 2024-03-24T10:46:13.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.060+0000 7f3c50ab7700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4c102a10 con 0x7f3c4c0fbce0 2024-03-24T10:46:13.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.060+0000 7f3c49d9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c4c0fcf90 0x7f3c4c1022f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:46:13.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.060+0000 7f3c49d9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c4c0fcf90 0x7f3c4c1022f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54262/0 (socket says 172.21.15.94:54262) 2024-03-24T10:46:13.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.060+0000 7f3c49d9b700 1 -- 172.21.15.94:0/621057524 learned_addr learned my addr 172.21.15.94:0/621057524 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:46:13.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.061+0000 7f3c4a59c700 1 --2- 172.21.15.94:0/621057524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c4c0fbce0 0x7f3c4c103e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:46:13.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.061+0000 7f3c49d9b700 1 -- 172.21.15.94:0/621057524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c4c0fbce0 msgr2=0x7f3c4c103e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:13.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.061+0000 7f3c49d9b700 1 --2- 172.21.15.94:0/621057524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c4c0fbce0 0x7f3c4c103e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:13.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.061+0000 7f3c49d9b700 1 -- 172.21.15.94:0/621057524 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c3c005040 con 0x7f3c4c0fcf90 2024-03-24T10:46:13.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.061+0000 7f3c49d9b700 1 --2- 172.21.15.94:0/621057524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c4c0fcf90 0x7f3c4c1022f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3c40002a50 tx=0x7f3c400046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:46:13.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.061+0000 7f3c3affd700 1 -- 172.21.15.94:0/621057524 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c40007cc0 con 0x7f3c4c0fcf90 2024-03-24T10:46:13.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.061+0000 7f3c50ab7700 1 -- 172.21.15.94:0/621057524 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c4c102cf0 con 0x7f3c4c0fcf90 2024-03-24T10:46:13.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.061+0000 7f3c3affd700 1 -- 172.21.15.94:0/621057524 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3c4000f040 con 0x7f3c4c0fcf90 2024-03-24T10:46:13.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.062+0000 7f3c3affd700 1 -- 172.21.15.94:0/621057524 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c40009640 con 0x7f3c4c0fcf90 2024-03-24T10:46:13.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.062+0000 7f3c50ab7700 1 -- 172.21.15.94:0/621057524 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c4c115a60 con 0x7f3c4c0fcf90 2024-03-24T10:46:13.065 INFO:tasks.workunit.client.1.smithi096.stdout:0/455: truncate d0/d11/d20/d65/f8c 536773 0 2024-03-24T10:46:13.065 INFO:tasks.workunit.client.1.smithi096.stdout:0/456: truncate d0/d11/d24/f68 497068 0 2024-03-24T10:46:13.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.062+0000 7f3c50ab7700 1 -- 172.21.15.94:0/621057524 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c4c05f5f0 con 0x7f3c4c0fcf90 2024-03-24T10:46:13.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.063+0000 7f3c3affd700 1 -- 172.21.15.94:0/621057524 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f3c400097a0 con 0x7f3c4c0fcf90 2024-03-24T10:46:13.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.064+0000 7f3c3affd700 1 --2- 172.21.15.94:0/621057524 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f3c340692a0 0x7f3c3406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:46:13.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.064+0000 7f3c3affd700 1 -- 172.21.15.94:0/621057524 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f3c40086f00 con 0x7f3c4c0fcf90 2024-03-24T10:46:13.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.064+0000 7f3c4a59c700 1 --2- 172.21.15.94:0/621057524 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f3c340692a0 0x7f3c3406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:46:13.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.064+0000 7f3c4a59c700 1 --2- 172.21.15.94:0/621057524 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f3c340692a0 0x7f3c3406b750 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c00b970 tx=0x7f3c3c009e30 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:46:13.067 INFO:tasks.workunit.client.1.smithi096.stdout:9/437: rename d9/db/c32 to d9/d15/d26/d29/d73/c8d 0 2024-03-24T10:46:13.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.069+0000 7f3c3affd700 1 -- 172.21.15.94:0/621057524 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f3c400531e0 con 0x7f3c4c0fcf90 2024-03-24T10:46:13.071 INFO:tasks.workunit.client.1.smithi096.stdout:3/470: mkdir d0/d13/d45/d52/d88/d9c 0 2024-03-24T10:46:13.074 INFO:tasks.workunit.client.1.smithi096.stdout:7/635: mkdir d0/db/d26/d48/dd4 0 2024-03-24T10:46:13.080 INFO:tasks.workunit.client.1.smithi096.stdout:2/381: symlink d5/de/l94 0 2024-03-24T10:46:13.080 INFO:tasks.workunit.client.1.smithi096.stdout:1/379: write d3/d73/f7a [1368129,27931] 0 2024-03-24T10:46:13.081 INFO:tasks.workunit.client.1.smithi096.stdout:0/457: dwrite d0/d11/d16/d21/d25/d47/f7c [0,4194304] 0 2024-03-24T10:46:13.087 INFO:tasks.workunit.client.1.smithi096.stdout:8/454: creat d8/f92 x:0 0 0 2024-03-24T10:46:13.095 INFO:tasks.workunit.client.1.smithi096.stdout:9/438: mkdir d9/db/d6d/d4f/d75/d8b/d8e 0 2024-03-24T10:46:13.095 INFO:tasks.workunit.client.1.smithi096.stdout:3/471: mknod d0/d9/d22/c9d 0 2024-03-24T10:46:13.096 INFO:tasks.workunit.client.1.smithi096.stdout:3/472: chown d0/d13/d45/d7d/d39/c95 10 1 2024-03-24T10:46:13.096 INFO:tasks.workunit.client.1.smithi096.stdout:3/473: write d0/f61 [1234885,43656] 0 2024-03-24T10:46:13.098 INFO:tasks.workunit.client.1.smithi096.stdout:1/380: dwrite d3/d24/d6a/f70 [0,4194304] 0 2024-03-24T10:46:13.098 INFO:tasks.workunit.client.1.smithi096.stdout:1/381: truncate d3/d23/f69 54068 0 2024-03-24T10:46:13.098 INFO:tasks.workunit.client.1.smithi096.stdout:7/636: creat d0/db/d26/fd5 x:0 0 0 2024-03-24T10:46:13.098 INFO:tasks.workunit.client.1.smithi096.stdout:7/637: fdatasync d0/db/d4f/f74 0 2024-03-24T10:46:13.098 INFO:tasks.workunit.client.1.smithi096.stdout:7/638: creat d0/d21/d59/fd6 x:0 0 0 2024-03-24T10:46:13.099 INFO:tasks.workunit.client.1.smithi096.stdout:2/382: getdents d5/d79 0 2024-03-24T10:46:13.099 INFO:tasks.workunit.client.1.smithi096.stdout:2/383: creat d5/d7c/f95 x:0 0 0 2024-03-24T10:46:13.104 INFO:tasks.workunit.client.1.smithi096.stdout:5/439: dwrite d2/d7/d8/df/f25 [4194304,4194304] 0 2024-03-24T10:46:13.104 INFO:tasks.workunit.client.1.smithi096.stdout:5/440: fdatasync d2/d29/d32/f80 0 2024-03-24T10:46:13.104 INFO:tasks.workunit.client.1.smithi096.stdout:5/441: fsync d2/d29/d32/d33/f63 0 2024-03-24T10:46:13.104 INFO:tasks.workunit.client.1.smithi096.stdout:9/439: write d9/d15/f27 [2028846,41338] 0 2024-03-24T10:46:13.104 INFO:tasks.workunit.client.1.smithi096.stdout:1/382: write d3/d23/d2f/f3b [488057,51609] 0 2024-03-24T10:46:13.106 INFO:tasks.workunit.client.1.smithi096.stdout:0/458: link d0/c6 d0/c9b 0 2024-03-24T10:46:13.107 INFO:tasks.workunit.client.1.smithi096.stdout:0/459: read - d0/d11/d16/d21/d25/d47/f7d zero size 2024-03-24T10:46:13.107 INFO:tasks.workunit.client.1.smithi096.stdout:0/460: write d0/d11/d20/f26 [1316792,65114] 0 2024-03-24T10:46:13.107 INFO:tasks.workunit.client.1.smithi096.stdout:0/461: chown d0/d11/d16/d21/d25 206 1 2024-03-24T10:46:13.107 INFO:tasks.workunit.client.1.smithi096.stdout:8/455: truncate d8/d11/d22/d4c/d65/d1a/d46/d2e/f71 75319 0 2024-03-24T10:46:13.110 INFO:tasks.workunit.client.1.smithi096.stdout:3/474: unlink d0/f61 0 2024-03-24T10:46:13.112 INFO:tasks.workunit.client.1.smithi096.stdout:9/440: dread d9/d15/f3c [0,4194304] 0 2024-03-24T10:46:13.115 INFO:tasks.workunit.client.1.smithi096.stdout:7/639: symlink d0/d21/d3c/d68/d85/ld7 0 2024-03-24T10:46:13.116 INFO:tasks.workunit.client.1.smithi096.stdout:3/475: dread d0/d9/f49 [0,4194304] 0 2024-03-24T10:46:13.118 INFO:tasks.workunit.client.1.smithi096.stdout:2/384: symlink d5/l96 0 2024-03-24T10:46:13.121 INFO:tasks.workunit.client.1.smithi096.stdout:5/442: symlink d2/l92 0 2024-03-24T10:46:13.122 INFO:tasks.workunit.client.1.smithi096.stdout:1/383: link d3/d36/c74 d3/d84/c86 0 2024-03-24T10:46:13.122 INFO:tasks.workunit.client.1.smithi096.stdout:1/384: write d3/d23/d2f/f83 [640579,37922] 0 2024-03-24T10:46:13.122 INFO:tasks.workunit.client.1.smithi096.stdout:0/462: getdents d0/d11/d20 0 2024-03-24T10:46:13.126 INFO:tasks.workunit.client.1.smithi096.stdout:9/441: creat d9/db/d6d/d4f/d75/d8b/d8e/f8f x:0 0 0 2024-03-24T10:46:13.126 INFO:tasks.workunit.client.1.smithi096.stdout:9/442: write d9/d15/d26/d29/d73/f4b [1690886,197] 0 2024-03-24T10:46:13.126 INFO:tasks.workunit.client.1.smithi096.stdout:7/640: getdents d0/d21/d3c/d68/d7e 0 2024-03-24T10:46:13.128 INFO:tasks.workunit.client.1.smithi096.stdout:3/476: dread d0/d9/f32 [0,4194304] 0 2024-03-24T10:46:13.132 INFO:tasks.workunit.client.1.smithi096.stdout:1/385: write d3/d24/d6a/f70 [1774559,27622] 0 2024-03-24T10:46:13.133 INFO:tasks.workunit.client.1.smithi096.stdout:5/443: creat d2/d7/d8/d18/f93 x:0 0 0 2024-03-24T10:46:13.136 INFO:tasks.workunit.client.1.smithi096.stdout:0/463: mkdir d0/d11/d16/d21/d81/d9c 0 2024-03-24T10:46:13.136 INFO:tasks.workunit.client.1.smithi096.stdout:0/464: dread - d0/d11/d16/d21/d81/d7f/f71 zero size 2024-03-24T10:46:13.137 INFO:tasks.workunit.client.1.smithi096.stdout:8/456: truncate d8/d11/d22/d4c/d65/d1d/f2b 2611059 0 2024-03-24T10:46:13.137 INFO:tasks.workunit.client.1.smithi096.stdout:8/457: chown d8/f15 991 1 2024-03-24T10:46:13.137 INFO:tasks.workunit.client.1.smithi096.stdout:0/465: read d0/d11/d20/d91/f96 [525619,93258] 0 2024-03-24T10:46:13.138 INFO:tasks.workunit.client.1.smithi096.stdout:0/466: dread - d0/d11/d24/d2c/f67 zero size 2024-03-24T10:46:13.140 INFO:tasks.workunit.client.1.smithi096.stdout:7/641: creat d0/db/d1c/d77/fd8 x:0 0 0 2024-03-24T10:46:13.142 INFO:tasks.workunit.client.1.smithi096.stdout:2/385: dwrite d5/de/d2a/f37 [4194304,4194304] 0 2024-03-24T10:46:13.142 INFO:tasks.workunit.client.1.smithi096.stdout:2/386: write d5/de/d2a/f40 [709083,6801] 0 2024-03-24T10:46:13.146 INFO:tasks.workunit.client.1.smithi096.stdout:1/386: mknod d3/d23/d71/d85/c87 0 2024-03-24T10:46:13.146 INFO:tasks.workunit.client.1.smithi096.stdout:1/387: read d3/ff [343447,94081] 0 2024-03-24T10:46:13.146 INFO:tasks.workunit.client.1.smithi096.stdout:1/388: write d3/d73/f7a [322853,24881] 0 2024-03-24T10:46:13.159 INFO:tasks.workunit.client.1.smithi096.stdout:8/458: getdents d8/d11/d22/d4c/d65/d1a/d46 0 2024-03-24T10:46:13.159 INFO:tasks.workunit.client.1.smithi096.stdout:8/459: dread - d8/d11/d44/f84 zero size 2024-03-24T10:46:13.160 INFO:tasks.workunit.client.1.smithi096.stdout:0/467: truncate d0/d11/d20/d91/f96 292896 0 2024-03-24T10:46:13.162 INFO:tasks.workunit.client.1.smithi096.stdout:1/389: write d3/d23/f38 [1441129,63773] 0 2024-03-24T10:46:13.164 INFO:tasks.workunit.client.1.smithi096.stdout:2/387: rename d5/de/c71 to d5/dc/d12/c97 0 2024-03-24T10:46:13.164 INFO:tasks.workunit.client.1.smithi096.stdout:2/388: fdatasync d5/dc/f2f 0 2024-03-24T10:46:13.164 INFO:tasks.workunit.client.1.smithi096.stdout:2/389: chown d5/de/f51 61066942 1 2024-03-24T10:46:13.164 INFO:tasks.workunit.client.1.smithi096.stdout:2/390: write d5/de/f53 [208252,112135] 0 2024-03-24T10:46:13.165 INFO:tasks.workunit.client.1.smithi096.stdout:8/460: mkdir d8/d11/d85/d93 0 2024-03-24T10:46:13.166 INFO:tasks.workunit.client.1.smithi096.stdout:0/468: mkdir d0/d11/d24/d9d 0 2024-03-24T10:46:13.170 INFO:tasks.workunit.client.1.smithi096.stdout:3/477: dwrite d0/d13/f21 [4194304,4194304] 0 2024-03-24T10:46:13.170 INFO:tasks.workunit.client.1.smithi096.stdout:3/478: fdatasync d0/d13/d45/d47/f76 0 2024-03-24T10:46:13.170 INFO:tasks.workunit.client.1.smithi096.stdout:3/479: fdatasync d0/d13/d45/d52/f6d 0 2024-03-24T10:46:13.170 INFO:tasks.workunit.client.1.smithi096.stdout:7/642: dwrite d0/db/d26/f37 [0,4194304] 0 2024-03-24T10:46:13.170 INFO:tasks.workunit.client.1.smithi096.stdout:1/390: rename d3/c4f to d3/d23/d71/c88 0 2024-03-24T10:46:13.173 INFO:tasks.workunit.client.1.smithi096.stdout:2/391: creat d5/dc/d12/d23/f98 x:0 0 0 2024-03-24T10:46:13.176 INFO:tasks.workunit.client.1.smithi096.stdout:5/444: dwrite d2/d7/d8/f62 [4194304,4194304] 0 2024-03-24T10:46:13.182 INFO:tasks.workunit.client.1.smithi096.stdout:0/469: rename d0/d11/d16/d21/d25/d47/d63 to d0/d11/d20/d9e 0 2024-03-24T10:46:13.183 INFO:tasks.workunit.client.1.smithi096.stdout:1/391: mknod d3/d23/d71/d85/c89 0 2024-03-24T10:46:13.186 INFO:tasks.workunit.client.1.smithi096.stdout:2/392: truncate d5/dc/d12/f16 584867 0 2024-03-24T10:46:13.186 INFO:tasks.workunit.client.1.smithi096.stdout:2/393: creat d5/d7c/f99 x:0 0 0 2024-03-24T10:46:13.186 INFO:tasks.workunit.client.1.smithi096.stdout:2/394: truncate d5/d7c/f8c 855133 0 2024-03-24T10:46:13.186 INFO:tasks.workunit.client.1.smithi096.stdout:2/395: chown d5/fb 8568515 1 2024-03-24T10:46:13.186 INFO:tasks.workunit.client.1.smithi096.stdout:8/461: symlink d8/d11/d22/d4c/d65/d37/l94 0 2024-03-24T10:46:13.190 INFO:tasks.workunit.client.1.smithi096.stdout:5/445: mkdir d2/d29/d32/d33/d8d/d94 0 2024-03-24T10:46:13.192 INFO:tasks.workunit.client.1.smithi096.stdout:3/480: dwrite d0/d13/d45/d7d/d39/d48/f6a [0,4194304] 0 2024-03-24T10:46:13.192 INFO:tasks.workunit.client.1.smithi096.stdout:3/481: read - d0/d13/d45/d52/f74 zero size 2024-03-24T10:46:13.194 INFO:tasks.workunit.client.1.smithi096.stdout:7/643: rename d0/d21/d3c/cbd to d0/db/d26/d48/da8/cd9 0 2024-03-24T10:46:13.197 INFO:tasks.workunit.client.1.smithi096.stdout:1/392: unlink d3/d36/l6c 0 2024-03-24T10:46:13.197 INFO:tasks.workunit.client.1.smithi096.stdout:1/393: getdents d3/d23/d2f/d80 0 2024-03-24T10:46:13.197 INFO:tasks.workunit.client.1.smithi096.stdout:1/394: fsync d3/d24/d6a/f77 0 2024-03-24T10:46:13.197 INFO:tasks.workunit.client.1.smithi096.stdout:1/395: dread - d3/d23/d2f/f60 zero size 2024-03-24T10:46:13.197 INFO:tasks.workunit.client.1.smithi096.stdout:1/396: chown d3/d24/c5d 612421 1 2024-03-24T10:46:13.197 INFO:tasks.workunit.client.1.smithi096.stdout:1/397: chown d3/d36/f50 39538 1 2024-03-24T10:46:13.197 INFO:tasks.workunit.client.1.smithi096.stdout:1/398: readlink d3/d24/d5c/l64 0 2024-03-24T10:46:13.201 INFO:tasks.workunit.client.1.smithi096.stdout:2/396: read - d5/d38/f76 zero size 2024-03-24T10:46:13.205 INFO:tasks.workunit.client.1.smithi096.stdout:8/462: link d8/d11/d22/d4c/d65/d1a/d46/d2e/l3c d8/d11/d22/d4c/d65/d1a/d46/l95 0 2024-03-24T10:46:13.205 INFO:tasks.workunit.client.1.smithi096.stdout:1/399: write d3/d23/d2f/f4a [3086219,128972] 0 2024-03-24T10:46:13.205 INFO:tasks.workunit.client.1.smithi096.stdout:1/400: creat d3/d23/d71/f8a x:0 0 0 2024-03-24T10:46:13.206 INFO:tasks.workunit.client.1.smithi096.stdout:0/470: dwrite d0/d11/d24/f30 [0,4194304] 0 2024-03-24T10:46:13.206 INFO:tasks.workunit.client.1.smithi096.stdout:5/446: mkdir d2/d7/d8/df/d95 0 2024-03-24T10:46:13.208 INFO:tasks.workunit.client.1.smithi096.stdout:3/482: dread d0/d13/f21 [0,4194304] 0 2024-03-24T10:46:13.211 INFO:tasks.workunit.client.1.smithi096.stdout:7/644: symlink d0/d21/d3c/lda 0 2024-03-24T10:46:13.211 INFO:tasks.workunit.client.1.smithi096.stdout:7/645: readlink d0/d21/la6 0 2024-03-24T10:46:13.211 INFO:tasks.workunit.client.1.smithi096.stdout:7/646: chown d0/db/d26 52 1 2024-03-24T10:46:13.215 INFO:tasks.workunit.client.1.smithi096.stdout:5/447: dread d2/d7/d8/d18/f72 [0,4194304] 0 2024-03-24T10:46:13.215 INFO:tasks.workunit.client.1.smithi096.stdout:5/448: truncate d2/d7/d8/df/d14/f85 595741 0 2024-03-24T10:46:13.219 INFO:tasks.workunit.client.1.smithi096.stdout:2/397: dread d5/d38/f3a [0,4194304] 0 2024-03-24T10:46:13.220 INFO:tasks.workunit.client.1.smithi096.stdout:2/398: chown d5/fa 416488262 1 2024-03-24T10:46:13.228 INFO:tasks.workunit.client.1.smithi096.stdout:0/471: symlink d0/d11/d16/l9f 0 2024-03-24T10:46:13.228 INFO:tasks.workunit.client.1.smithi096.stdout:0/472: fdatasync d0/d11/f1c 0 2024-03-24T10:46:13.232 INFO:tasks.workunit.client.1.smithi096.stdout:3/483: symlink d0/d13/d45/d7d/d39/d48/d8d/l9e 0 2024-03-24T10:46:13.236 INFO:tasks.workunit.client.1.smithi096.stdout:7/647: mkdir d0/da9/ddb 0 2024-03-24T10:46:13.247 INFO:tasks.workunit.client.1.smithi096.stdout:2/399: mknod d5/d7c/c9a 0 2024-03-24T10:46:13.251 INFO:tasks.workunit.client.1.smithi096.stdout:0/473: link d0/d11/d20/d2b/f89 d0/d11/d20/d91/fa0 0 2024-03-24T10:46:13.255 INFO:tasks.workunit.client.1.smithi096.stdout:3/484: symlink d0/d13/d41/l9f 0 2024-03-24T10:46:13.264 INFO:tasks.workunit.client.1.smithi096.stdout:3/485: dread d0/d13/d45/d47/f64 [0,4194304] 0 2024-03-24T10:46:13.272 INFO:tasks.workunit.client.1.smithi096.stdout:3/486: read d0/d13/d45/d52/f5b [389568,1116] 0 2024-03-24T10:46:13.278 INFO:tasks.workunit.client.1.smithi096.stdout:7/648: unlink d0/db/d4f/f65 0 2024-03-24T10:46:13.279 INFO:tasks.workunit.client.1.smithi096.stdout:5/449: mkdir d2/d7/d8/df/d14/d96 0 2024-03-24T10:46:13.280 INFO:tasks.workunit.client.1.smithi096.stdout:7/649: dread d0/db/d1c/d77/f7a [0,4194304] 0 2024-03-24T10:46:13.282 INFO:tasks.workunit.client.1.smithi096.stdout:8/463: rename d8/d11/d22/d4c/d65/d1d/d86 to d8/d72/d96 0 2024-03-24T10:46:13.284 INFO:tasks.workunit.client.1.smithi096.stdout:0/474: link d0/d11/d20/d56/l5b d0/d11/d43/la1 0 2024-03-24T10:46:13.286 INFO:tasks.workunit.client.1.smithi096.stdout:3/487: link d0/d9/d22/l28 d0/d13/d45/d47/d8f/la0 0 2024-03-24T10:46:13.290 INFO:tasks.workunit.client.1.smithi096.stdout:8/464: write d8/d11/d22/d4c/d82/f33 [416970,18225] 0 2024-03-24T10:46:13.293 INFO:tasks.workunit.client.1.smithi096.stdout:0/475: creat d0/d11/d16/d21/d81/d9c/fa2 x:0 0 0 2024-03-24T10:46:13.293 INFO:tasks.workunit.client.1.smithi096.stdout:0/476: chown d0/d11/d43/d75/l79 197267717 1 2024-03-24T10:46:13.296 INFO:tasks.workunit.client.1.smithi096.stdout:3/488: mknod d0/d13/d45/d7d/d71/ca1 0 2024-03-24T10:46:13.298 INFO:tasks.workunit.client.1.smithi096.stdout:5/450: mknod d2/d7/c97 0 2024-03-24T10:46:13.301 INFO:tasks.workunit.client.1.smithi096.stdout:7/650: rmdir d0/db/d26/d48 39 2024-03-24T10:46:13.309 INFO:tasks.workunit.client.1.smithi096.stdout:2/400: link d5/d38/f67 d5/d38/f9b 0 2024-03-24T10:46:13.310 INFO:tasks.workunit.client.1.smithi096.stdout:3/489: symlink d0/d9/la2 0 2024-03-24T10:46:13.310 INFO:tasks.workunit.client.1.smithi096.stdout:5/451: symlink d2/d7/d8/df/l98 0 2024-03-24T10:46:13.312 INFO:tasks.workunit.client.1.smithi096.stdout:1/401: rename d3/d24 to d3/d23/d8b 0 2024-03-24T10:46:13.313 INFO:tasks.workunit.client.1.smithi096.stdout:2/401: truncate d5/dc/f2e 2862000 0 2024-03-24T10:46:13.314 INFO:tasks.workunit.client.1.smithi096.stdout:2/402: write d5/f7 [4548554,22507] 0 2024-03-24T10:46:13.315 INFO:tasks.workunit.client.1.smithi096.stdout:3/490: creat d0/d13/d41/d7f/fa3 x:0 0 0 2024-03-24T10:46:13.315 INFO:tasks.workunit.client.1.smithi096.stdout:3/491: fsync d0/d13/d45/d52/f5b 0 2024-03-24T10:46:13.315 INFO:tasks.workunit.client.1.smithi096.stdout:5/452: mkdir d2/d29/d32/d33/d8d/d94/d99 0 2024-03-24T10:46:13.316 INFO:tasks.workunit.client.1.smithi096.stdout:0/477: dwrite d0/d11/f6d [0,4194304] 0 2024-03-24T10:46:13.316 INFO:tasks.workunit.client.1.smithi096.stdout:8/465: dwrite d8/f58 [0,4194304] 0 2024-03-24T10:46:13.317 INFO:tasks.workunit.client.1.smithi096.stdout:8/466: write d8/d11/f43 [1280781,6553] 0 2024-03-24T10:46:13.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.317+0000 7f3c50ab7700 1 -- 172.21.15.94:0/621057524 --> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3c4c1012f0 con 0x7f3c340692a0 2024-03-24T10:46:13.318 INFO:tasks.workunit.client.1.smithi096.stdout:8/467: write d8/d11/d22/d4c/d65/d1a/f2d [1457997,35610] 0 2024-03-24T10:46:13.318 INFO:tasks.workunit.client.1.smithi096.stdout:7/651: rename d0/db/d26/dc3/lc5 to d0/db/d26/ldc 0 2024-03-24T10:46:13.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.318+0000 7f3c3affd700 1 -- 172.21.15.94:0/621057524 <== mgr.14232 v2:172.21.15.94:6800/2184357820 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+328 (secure 0 0 0) 0x7f3c4c1012f0 con 0x7f3c340692a0 2024-03-24T10:46:13.320 INFO:tasks.workunit.client.1.smithi096.stdout:3/492: unlink d0/d13/d45/d7d/d39/d48/l4b 0 2024-03-24T10:46:13.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.321+0000 7f3c50ab7700 1 -- 172.21.15.94:0/621057524 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f3c340692a0 msgr2=0x7f3c3406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:13.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.321+0000 7f3c50ab7700 1 --2- 172.21.15.94:0/621057524 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f3c340692a0 0x7f3c3406b750 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c00b970 tx=0x7f3c3c009e30 comp rx=0 tx=0).stop 2024-03-24T10:46:13.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.321+0000 7f3c50ab7700 1 -- 172.21.15.94:0/621057524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c4c0fcf90 msgr2=0x7f3c4c1022f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:13.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.321+0000 7f3c50ab7700 1 --2- 172.21.15.94:0/621057524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c4c0fcf90 0x7f3c4c1022f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3c40002a50 tx=0x7f3c400046b0 comp rx=0 tx=0).stop 2024-03-24T10:46:13.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.322+0000 7f3c50ab7700 1 -- 172.21.15.94:0/621057524 shutdown_connections 2024-03-24T10:46:13.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.322+0000 7f3c50ab7700 1 --2- 172.21.15.94:0/621057524 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f3c340692a0 0x7f3c3406b750 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:13.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.322+0000 7f3c50ab7700 1 --2- 172.21.15.94:0/621057524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c4c0fbce0 0x7f3c4c103e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:13.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.322+0000 7f3c50ab7700 1 --2- 172.21.15.94:0/621057524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c4c0fcf90 0x7f3c4c1022f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:13.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.322+0000 7f3c50ab7700 1 -- 172.21.15.94:0/621057524 >> 172.21.15.94:0/621057524 conn(0x7f3c4c0f7440 msgr2=0x7f3c4c0f8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:46:13.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.322+0000 7f3c50ab7700 1 -- 172.21.15.94:0/621057524 shutdown_connections 2024-03-24T10:46:13.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.322+0000 7f3c50ab7700 1 -- 172.21.15.94:0/621057524 wait complete. 2024-03-24T10:46:13.335 INFO:tasks.workunit.client.1.smithi096.stdout:0/478: dwrite d0/d11/d16/d21/d81/d7f/f4a [0,4194304] 0 2024-03-24T10:46:13.336 INFO:teuthology.orchestra.run.smithi094.stdout:true 2024-03-24T10:46:13.343 INFO:tasks.workunit.client.1.smithi096.stdout:8/468: write d8/fe [1393430,49929] 0 2024-03-24T10:46:13.347 INFO:tasks.workunit.client.1.smithi096.stdout:3/493: dwrite d0/d13/d77/f7a [0,4194304] 0 2024-03-24T10:46:13.352 INFO:tasks.workunit.client.1.smithi096.stdout:7/652: rename d0/db/d1c/f61 to d0/d21/d3c/d68/d85/fdd 0 2024-03-24T10:46:13.352 INFO:tasks.workunit.client.1.smithi096.stdout:7/653: chown d0/d21/d3c/cbb 6 1 2024-03-24T10:46:13.352 INFO:tasks.workunit.client.1.smithi096.stdout:7/654: stat d0/d21/d59/d87/da0 0 2024-03-24T10:46:13.353 INFO:tasks.workunit.client.1.smithi096.stdout:7/655: creat d0/d13/d32/d45/fde x:0 0 0 2024-03-24T10:46:13.353 INFO:tasks.workunit.client.1.smithi096.stdout:7/656: write d0/d13/f3d [2903385,93776] 0 2024-03-24T10:46:13.354 INFO:tasks.workunit.client.1.smithi096.stdout:5/453: rmdir d2/d29/d32/d3f/d5f 39 2024-03-24T10:46:13.354 INFO:tasks.workunit.client.1.smithi096.stdout:5/454: stat d2/d7/d8/df/d14/f31 0 2024-03-24T10:46:13.358 INFO:tasks.workunit.client.1.smithi096.stdout:3/494: dread d0/d13/d1e/f2d [0,4194304] 0 2024-03-24T10:46:13.363 INFO:tasks.workunit.client.1.smithi096.stdout:7/657: dread d0/db/d1c/f29 [0,4194304] 0 2024-03-24T10:46:13.364 INFO:tasks.workunit.client.1.smithi096.stdout:8/469: unlink d8/d11/d22/d4c/d82/l48 0 2024-03-24T10:46:13.370 INFO:tasks.workunit.client.1.smithi096.stdout:0/479: rename d0/d11/d16/d21/d81/d7f/c7 to d0/d11/d16/d21/d78/d82/ca3 0 2024-03-24T10:46:13.373 INFO:tasks.workunit.client.1.smithi096.stdout:1/402: rmdir d3/d23/d2f/d80 0 2024-03-24T10:46:13.374 INFO:tasks.workunit.client.1.smithi096.stdout:5/455: symlink d2/d7/d8/df/d14/d96/l9a 0 2024-03-24T10:46:13.375 INFO:tasks.workunit.client.1.smithi096.stdout:3/495: mkdir d0/d13/d1e/da4 0 2024-03-24T10:46:13.375 INFO:tasks.workunit.client.1.smithi096.stdout:3/496: write d0/d13/d1e/f2f [4214382,118585] 0 2024-03-24T10:46:13.375 INFO:tasks.workunit.client.1.smithi096.stdout:8/470: stat d8/d11/d22/d4c/d65/d1d/c90 0 2024-03-24T10:46:13.377 INFO:tasks.workunit.client.1.smithi096.stdout:1/403: mkdir d3/d23/d8b/d8c 0 2024-03-24T10:46:13.379 INFO:tasks.workunit.client.1.smithi096.stdout:0/480: write d0/d11/d16/d21/d25/d47/f7c [1389789,47743] 0 2024-03-24T10:46:13.379 INFO:tasks.workunit.client.1.smithi096.stdout:0/481: read - d0/d11/f66 zero size 2024-03-24T10:46:13.379 INFO:tasks.workunit.client.1.smithi096.stdout:3/497: dread d0/d9/f32 [0,4194304] 0 2024-03-24T10:46:13.379 INFO:tasks.workunit.client.1.smithi096.stdout:3/498: write d0/d9/f36 [4328475,12237] 0 2024-03-24T10:46:13.381 INFO:tasks.workunit.client.1.smithi096.stdout:5/456: rename d2/d29/d32/f80 to d2/d29/d32/d3f/f9b 0 2024-03-24T10:46:13.381 INFO:tasks.workunit.client.1.smithi096.stdout:5/457: creat d2/d29/d32/d89/f9c x:0 0 0 2024-03-24T10:46:13.381 INFO:tasks.workunit.client.1.smithi096.stdout:8/471: mkdir d8/d11/d22/d4c/d65/d1d/d3d/d97 0 2024-03-24T10:46:13.383 INFO:tasks.workunit.client.1.smithi096.stdout:7/658: dwrite d0/d21/d3c/d68/d4b/fa4 [0,4194304] 0 2024-03-24T10:46:13.385 INFO:tasks.workunit.client.1.smithi096.stdout:3/499: symlink d0/d13/d45/d7d/d39/d48/d8d/la5 0 2024-03-24T10:46:13.386 INFO:tasks.workunit.client.1.smithi096.stdout:3/500: write d0/d9/f5e [799966,36927] 0 2024-03-24T10:46:13.387 INFO:tasks.workunit.client.1.smithi096.stdout:8/472: mkdir d8/d72/d98 0 2024-03-24T10:46:13.396 INFO:tasks.workunit.client.1.smithi096.stdout:3/501: truncate d0/d13/d45/d7d/d71/f96 506269 0 2024-03-24T10:46:13.400 INFO:tasks.workunit.client.1.smithi096.stdout:8/473: write d8/d11/d22/d4c/f68 [2086099,6467] 0 2024-03-24T10:46:13.400 INFO:tasks.workunit.client.1.smithi096.stdout:8/474: write d8/d11/d22/d4c/d65/d1a/f26 [1065490,100564] 0 2024-03-24T10:46:13.404 INFO:tasks.workunit.client.1.smithi096.stdout:8/475: mknod d8/d11/d22/c99 0 2024-03-24T10:46:13.406 INFO:tasks.workunit.client.1.smithi096.stdout:8/476: dread d8/d11/d22/d4c/d65/d1a/d46/d2e/f71 [0,4194304] 0 2024-03-24T10:46:13.409 INFO:tasks.workunit.client.1.smithi096.stdout:8/477: creat d8/d11/d22/d4c/d65/d1d/f9a x:0 0 0 2024-03-24T10:46:13.409 INFO:tasks.workunit.client.1.smithi096.stdout:8/478: readlink d8/d11/d22/d4c/l4f 0 2024-03-24T10:46:13.409 INFO:tasks.workunit.client.1.smithi096.stdout:8/479: fdatasync d8/d11/d22/d4c/d65/d1d/f5c 0 2024-03-24T10:46:13.410 INFO:tasks.workunit.client.1.smithi096.stdout:8/480: getdents d8/d72/d96 0 2024-03-24T10:46:13.412 INFO:tasks.workunit.client.1.smithi096.stdout:8/481: rename d8/d11/d22/d4c/d65/d1a/d46/d2e/f71 to d8/d11/d44/d45/f9b 0 2024-03-24T10:46:13.412 INFO:tasks.workunit.client.1.smithi096.stdout:8/482: chown d8/d11/d22/d4c/d65/d1d/c90 42753 1 2024-03-24T10:46:13.413 INFO:tasks.workunit.client.1.smithi096.stdout:0/482: dwrite d0/d11/d16/d21/d25/f46 [0,4194304] 0 2024-03-24T10:46:13.415 INFO:tasks.workunit.client.1.smithi096.stdout:0/483: stat d0/d11/d16/d21/d25/d47/c8d 0 2024-03-24T10:46:13.423 INFO:tasks.workunit.client.1.smithi096.stdout:3/502: dwrite d0/d9/d22/f3d [0,4194304] 0 2024-03-24T10:46:13.426 INFO:tasks.workunit.client.1.smithi096.stdout:5/458: dwrite d2/d7/f8c [0,4194304] 0 2024-03-24T10:46:13.426 INFO:tasks.workunit.client.1.smithi096.stdout:0/484: dread d0/d11/d24/d2c/f32 [0,4194304] 0 2024-03-24T10:46:13.426 INFO:tasks.workunit.client.1.smithi096.stdout:0/485: chown d0/d11/d20/d56 2520 1 2024-03-24T10:46:13.426 INFO:tasks.workunit.client.1.smithi096.stdout:3/503: rmdir d0/d13/d45/d7d/d39/d48/d8d 39 2024-03-24T10:46:13.426 INFO:tasks.workunit.client.1.smithi096.stdout:3/504: truncate d0/d13/d45/d52/f6d 634127 0 2024-03-24T10:46:13.429 INFO:tasks.workunit.client.1.smithi096.stdout:3/505: creat d0/d13/d45/d52/d88/d9c/fa6 x:0 0 0 2024-03-24T10:46:13.430 INFO:tasks.workunit.client.1.smithi096.stdout:3/506: getdents d0/d13/d41 0 2024-03-24T10:46:13.430 INFO:tasks.workunit.client.1.smithi096.stdout:3/507: dread - d0/d13/d77/f90 zero size 2024-03-24T10:46:13.430 INFO:tasks.workunit.client.1.smithi096.stdout:3/508: creat d0/d9/d22/fa7 x:0 0 0 2024-03-24T10:46:13.432 INFO:tasks.workunit.client.1.smithi096.stdout:3/509: dread d0/d13/d45/d7d/f94 [0,4194304] 0 2024-03-24T10:46:13.432 INFO:tasks.workunit.client.1.smithi096.stdout:7/659: dwrite d0/f38 [0,4194304] 0 2024-03-24T10:46:13.438 INFO:tasks.workunit.client.1.smithi096.stdout:8/483: truncate d8/d11/d22/d4c/f68 2445928 0 2024-03-24T10:46:13.438 INFO:tasks.workunit.client.1.smithi096.stdout:8/484: write d8/d11/d22/d4c/d65/d1a/d49/f6a [1265123,22231] 0 2024-03-24T10:46:13.439 INFO:tasks.workunit.client.1.smithi096.stdout:3/510: rmdir d0/d9 39 2024-03-24T10:46:13.439 INFO:tasks.workunit.client.1.smithi096.stdout:3/511: chown d0/f5c 104 1 2024-03-24T10:46:13.439 INFO:tasks.workunit.client.1.smithi096.stdout:3/512: creat d0/d9/d22/fa8 x:0 0 0 2024-03-24T10:46:13.447 INFO:tasks.workunit.client.1.smithi096.stdout:8/485: write d8/d11/d22/d4c/d65/d1d/f5c [1485457,121287] 0 2024-03-24T10:46:13.447 INFO:tasks.workunit.client.1.smithi096.stdout:8/486: chown d8/d11/d22/d4c/d65/d1a/d46/c40 228968 1 2024-03-24T10:46:13.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.448+0000 7f9314e13700 1 -- 172.21.15.94:0/138107423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93100fcf80 msgr2=0x7f93100fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:13.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.448+0000 7f9314e13700 1 --2- 172.21.15.94:0/138107423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93100fcf80 0x7f93100fd3f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9304004940 tx=0x7f9304009e30 comp rx=0 tx=0).stop 2024-03-24T10:46:13.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.448+0000 7f9314e13700 1 -- 172.21.15.94:0/138107423 shutdown_connections 2024-03-24T10:46:13.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.448+0000 7f9314e13700 1 --2- 172.21.15.94:0/138107423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93100fcf80 0x7f93100fd3f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:13.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.448+0000 7f9314e13700 1 --2- 172.21.15.94:0/138107423 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93100fbcd0 0x7f93100fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:13.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.448+0000 7f9314e13700 1 -- 172.21.15.94:0/138107423 >> 172.21.15.94:0/138107423 conn(0x7f93100f7430 msgr2=0x7f93100f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:46:13.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.448+0000 7f9314e13700 1 -- 172.21.15.94:0/138107423 shutdown_connections 2024-03-24T10:46:13.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.448+0000 7f9314e13700 1 -- 172.21.15.94:0/138107423 wait complete. 2024-03-24T10:46:13.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.448+0000 7f9314e13700 1 Processor -- start 2024-03-24T10:46:13.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.448+0000 7f9314e13700 1 -- start start 2024-03-24T10:46:13.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.449+0000 7f9314e13700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93100fbcd0 0x7f9310066c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:46:13.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.449+0000 7f9314e13700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93100fcf80 0x7f93101126f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:46:13.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.449+0000 7f9314e13700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9310112c30 con 0x7f93100fbcd0 2024-03-24T10:46:13.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.449+0000 7f9314e13700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9310112d70 con 0x7f93100fcf80 2024-03-24T10:46:13.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.449+0000 7f930e59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93100fbcd0 0x7f9310066c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:46:13.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.449+0000 7f930e59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93100fbcd0 0x7f9310066c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54286/0 (socket says 172.21.15.94:54286) 2024-03-24T10:46:13.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.449+0000 7f930e59c700 1 -- 172.21.15.94:0/1189679514 learned_addr learned my addr 172.21.15.94:0/1189679514 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:46:13.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.449+0000 7f930e59c700 1 -- 172.21.15.94:0/1189679514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93100fcf80 msgr2=0x7f93101126f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:13.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.449+0000 7f930e59c700 1 --2- 172.21.15.94:0/1189679514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93100fcf80 0x7f93101126f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:13.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.449+0000 7f930e59c700 1 -- 172.21.15.94:0/1189679514 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9304005040 con 0x7f93100fbcd0 2024-03-24T10:46:13.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.449+0000 7f930e59c700 1 --2- 172.21.15.94:0/1189679514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93100fbcd0 0x7f9310066c20 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f930000eb40 tx=0x7f930000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:46:13.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.449+0000 7f92feffd700 1 -- 172.21.15.94:0/1189679514 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9300007cc0 con 0x7f93100fbcd0 2024-03-24T10:46:13.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.449+0000 7f92feffd700 1 -- 172.21.15.94:0/1189679514 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f930000f040 con 0x7f93100fbcd0 2024-03-24T10:46:13.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.449+0000 7f9314e13700 1 -- 172.21.15.94:0/1189679514 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9310112f70 con 0x7f93100fbcd0 2024-03-24T10:46:13.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.450+0000 7f92feffd700 1 -- 172.21.15.94:0/1189679514 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9300009620 con 0x7f93100fbcd0 2024-03-24T10:46:13.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.450+0000 7f9314e13700 1 -- 172.21.15.94:0/1189679514 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9310113400 con 0x7f93100fbcd0 2024-03-24T10:46:13.453 INFO:tasks.workunit.client.1.smithi096.stdout:7/660: dwrite d0/d21/d59/d87/f8a [0,4194304] 0 2024-03-24T10:46:13.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.451+0000 7f9314e13700 1 -- 172.21.15.94:0/1189679514 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f931004c710 con 0x7f93100fbcd0 2024-03-24T10:46:13.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.451+0000 7f92feffd700 1 -- 172.21.15.94:0/1189679514 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f9300009780 con 0x7f93100fbcd0 2024-03-24T10:46:13.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.451+0000 7f92feffd700 1 --2- 172.21.15.94:0/1189679514 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f92f80692a0 0x7f92f806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:46:13.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.452+0000 7f930dd9b700 1 --2- 172.21.15.94:0/1189679514 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f92f80692a0 0x7f92f806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:46:13.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.452+0000 7f92feffd700 1 -- 172.21.15.94:0/1189679514 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f9300086d50 con 0x7f93100fbcd0 2024-03-24T10:46:13.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.452+0000 7f930dd9b700 1 --2- 172.21.15.94:0/1189679514 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f92f80692a0 0x7f92f806b750 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9304013870 tx=0x7f9304009e30 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:46:13.455 INFO:tasks.workunit.client.1.smithi096.stdout:7/661: truncate d0/d21/fcb 854708 0 2024-03-24T10:46:13.457 INFO:tasks.workunit.client.1.smithi096.stdout:7/662: rename d0/d21/d3c/d68/d85/f8e to d0/d13/d32/d99/fdf 0 2024-03-24T10:46:13.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.458+0000 7f92feffd700 1 -- 172.21.15.94:0/1189679514 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f930004e190 con 0x7f93100fbcd0 2024-03-24T10:46:13.459 INFO:tasks.workunit.client.1.smithi096.stdout:7/663: unlink d0/d21/d3c/lca 0 2024-03-24T10:46:13.466 INFO:tasks.workunit.client.1.smithi096.stdout:8/487: dwrite d8/d11/d22/d4c/d65/d52/f88 [0,4194304] 0 2024-03-24T10:46:13.466 INFO:tasks.workunit.client.1.smithi096.stdout:6/427: sync 2024-03-24T10:46:13.467 INFO:tasks.workunit.client.1.smithi096.stdout:6/428: write d8/d12/d32/f3c [1865603,72403] 0 2024-03-24T10:46:13.469 INFO:tasks.workunit.client.1.smithi096.stdout:7/664: write d0/f38 [1095716,51653] 0 2024-03-24T10:46:13.472 INFO:tasks.workunit.client.1.smithi096.stdout:7/665: dread d0/d21/d3c/f82 [0,4194304] 0 2024-03-24T10:46:13.476 INFO:tasks.workunit.client.1.smithi096.stdout:7/666: creat d0/d21/d59/d87/fe0 x:0 0 0 2024-03-24T10:46:13.476 INFO:tasks.workunit.client.1.smithi096.stdout:7/667: write d0/d21/d3c/d80/fc1 [242106,109427] 0 2024-03-24T10:46:13.476 INFO:tasks.workunit.client.1.smithi096.stdout:7/668: write d0/d21/d3c/d68/f4d [5328445,2079] 0 2024-03-24T10:46:13.476 INFO:tasks.workunit.client.1.smithi096.stdout:8/488: dread d8/f16 [0,4194304] 0 2024-03-24T10:46:13.476 INFO:tasks.workunit.client.1.smithi096.stdout:8/489: dread - d8/d11/d44/f84 zero size 2024-03-24T10:46:13.478 INFO:tasks.workunit.client.1.smithi096.stdout:7/669: dread d0/d13/d32/d99/fdf [0,4194304] 0 2024-03-24T10:46:13.482 INFO:tasks.workunit.client.1.smithi096.stdout:8/490: creat d8/d72/d98/f9c x:0 0 0 2024-03-24T10:46:13.482 INFO:tasks.workunit.client.1.smithi096.stdout:8/491: chown d8/d11/d22/d4c/d65/d1a/d46/d2e/f54 6196 1 2024-03-24T10:46:13.482 INFO:tasks.workunit.client.1.smithi096.stdout:8/492: write d8/d11/d22/d4c/d82/f33 [127098,94438] 0 2024-03-24T10:46:13.482 INFO:tasks.workunit.client.1.smithi096.stdout:8/493: dread - d8/d11/d22/d4c/d82/f70 zero size 2024-03-24T10:46:13.483 INFO:tasks.workunit.client.1.smithi096.stdout:8/494: mknod d8/d11/d22/d4c/d67/c9d 0 2024-03-24T10:46:13.484 INFO:tasks.workunit.client.1.smithi096.stdout:8/495: readlink d8/d11/d22/d4c/d65/d37/l50 0 2024-03-24T10:46:13.484 INFO:tasks.workunit.client.1.smithi096.stdout:8/496: chown d8/d11/d22/d4c/d65/d1d/d3d/d5f/c60 1823 1 2024-03-24T10:46:13.486 INFO:tasks.workunit.client.1.smithi096.stdout:8/497: truncate d8/d11/d22/d4c/d65/d1a/f2d 323374 0 2024-03-24T10:46:13.486 INFO:tasks.workunit.client.1.smithi096.stdout:6/429: dwrite d8/d12/d28/d36/d65/f6f [0,4194304] 0 2024-03-24T10:46:13.488 INFO:tasks.workunit.client.1.smithi096.stdout:8/498: symlink d8/d11/d22/d4c/d65/d1d/l9e 0 2024-03-24T10:46:13.489 INFO:tasks.workunit.client.1.smithi096.stdout:8/499: write d8/f15 [1306836,79742] 0 2024-03-24T10:46:13.489 INFO:tasks.workunit.client.1.smithi096.stdout:8/500: truncate d8/d11/d22/d4c/d65/d1a/f26 1975615 0 2024-03-24T10:46:13.492 INFO:tasks.workunit.client.1.smithi096.stdout:6/430: unlink d8/d12/d32/d66/l77 0 2024-03-24T10:46:13.492 INFO:tasks.workunit.client.1.smithi096.stdout:6/431: write d8/d12/d32/f6e [402416,41875] 0 2024-03-24T10:46:13.494 INFO:tasks.workunit.client.1.smithi096.stdout:7/670: dwrite d0/db/d1c/d77/f7a [0,4194304] 0 2024-03-24T10:46:13.494 INFO:tasks.workunit.client.1.smithi096.stdout:7/671: read d0/d21/d3c/d80/fc1 [125422,64405] 0 2024-03-24T10:46:13.494 INFO:tasks.workunit.client.1.smithi096.stdout:7/672: dread - d0/d13/d32/fb9 zero size 2024-03-24T10:46:13.495 INFO:tasks.workunit.client.1.smithi096.stdout:8/501: link d8/d11/d22/d4c/d82/f70 d8/d11/d22/d4c/f9f 0 2024-03-24T10:46:13.496 INFO:tasks.workunit.client.1.smithi096.stdout:6/432: creat d8/d10/d58/f85 x:0 0 0 2024-03-24T10:46:13.496 INFO:tasks.workunit.client.1.smithi096.stdout:6/433: write d8/f83 [382649,81489] 0 2024-03-24T10:46:13.501 INFO:tasks.workunit.client.1.smithi096.stdout:8/502: rename d8/d11/d22/d4c/d65/d1d/d3d/d97 to d8/d11/d22/d4c/d65/d1a/da0 0 2024-03-24T10:46:13.502 INFO:tasks.workunit.client.1.smithi096.stdout:7/673: write d0/d13/d32/d99/fbc [1806008,128545] 0 2024-03-24T10:46:13.502 INFO:tasks.workunit.client.1.smithi096.stdout:6/434: mknod d8/c86 0 2024-03-24T10:46:13.507 INFO:tasks.workunit.client.1.smithi096.stdout:8/503: mkdir d8/d11/d22/d4c/d82/da1 0 2024-03-24T10:46:13.508 INFO:tasks.workunit.client.1.smithi096.stdout:7/674: symlink d0/db/le1 0 2024-03-24T10:46:13.508 INFO:tasks.workunit.client.1.smithi096.stdout:7/675: write d0/f38 [1904339,82327] 0 2024-03-24T10:46:13.511 INFO:tasks.workunit.client.1.smithi096.stdout:8/504: creat d8/d11/d22/d4c/d65/fa2 x:0 0 0 2024-03-24T10:46:13.516 INFO:tasks.workunit.client.1.smithi096.stdout:8/505: rename d8/d11/d22/d4c/d65/d37/c8b to d8/d11/d85/ca3 0 2024-03-24T10:46:13.519 INFO:tasks.workunit.client.1.smithi096.stdout:8/506: symlink d8/d72/d96/la4 0 2024-03-24T10:46:13.522 INFO:tasks.workunit.client.1.smithi096.stdout:8/507: unlink d8/d11/d22/d4c/d65/c61 0 2024-03-24T10:46:13.523 INFO:tasks.workunit.client.1.smithi096.stdout:8/508: truncate d8/d11/d22/d4c/d82/f2c 968701 0 2024-03-24T10:46:13.530 INFO:tasks.workunit.client.1.smithi096.stdout:6/435: dwrite d8/d12/f5c [0,4194304] 0 2024-03-24T10:46:13.531 INFO:tasks.workunit.client.1.smithi096.stdout:7/676: dwrite d0/db/f72 [0,4194304] 0 2024-03-24T10:46:13.531 INFO:tasks.workunit.client.1.smithi096.stdout:7/677: creat d0/d21/d59/fe2 x:0 0 0 2024-03-24T10:46:13.531 INFO:tasks.workunit.client.1.smithi096.stdout:7/678: truncate d0/d13/d32/d99/fdf 1033882 0 2024-03-24T10:46:13.534 INFO:tasks.workunit.client.1.smithi096.stdout:6/436: symlink d8/d12/d28/d36/d65/d71/l87 0 2024-03-24T10:46:13.534 INFO:tasks.workunit.client.1.smithi096.stdout:6/437: chown d8/d10/d13/d63/f21 129508 1 2024-03-24T10:46:13.536 INFO:tasks.workunit.client.1.smithi096.stdout:7/679: creat d0/db/d26/d48/da8/fe3 x:0 0 0 2024-03-24T10:46:13.538 INFO:tasks.workunit.client.1.smithi096.stdout:8/509: dwrite d8/d11/d22/d4c/f9f [0,4194304] 0 2024-03-24T10:46:13.539 INFO:tasks.workunit.client.1.smithi096.stdout:6/438: rename d8/le to d8/d12/l88 0 2024-03-24T10:46:13.540 INFO:tasks.workunit.client.1.smithi096.stdout:7/680: truncate d0/d21/d3c/d68/f41 436774 0 2024-03-24T10:46:13.540 INFO:tasks.workunit.client.1.smithi096.stdout:7/681: readlink d0/d21/l2a 0 2024-03-24T10:46:13.540 INFO:tasks.workunit.client.1.smithi096.stdout:7/682: write d0/d13/d32/d99/fbc [3764507,82147] 0 2024-03-24T10:46:13.540 INFO:tasks.workunit.client.1.smithi096.stdout:7/683: chown d0/l49 0 1 2024-03-24T10:46:13.546 INFO:tasks.workunit.client.1.smithi096.stdout:8/510: unlink d8/d11/d22/d4c/d82/f33 0 2024-03-24T10:46:13.546 INFO:tasks.workunit.client.1.smithi096.stdout:8/511: write d8/d11/d22/d4c/d65/d1d/f7b [550812,109886] 0 2024-03-24T10:46:13.549 INFO:tasks.workunit.client.1.smithi096.stdout:6/439: mknod d8/d12/d32/d38/c89 0 2024-03-24T10:46:13.554 INFO:tasks.workunit.client.1.smithi096.stdout:7/684: truncate d0/d21/d3c/d68/d85/fdd 3805589 0 2024-03-24T10:46:13.556 INFO:tasks.workunit.client.1.smithi096.stdout:8/512: symlink d8/d11/d22/d4c/la5 0 2024-03-24T10:46:13.557 INFO:tasks.workunit.client.1.smithi096.stdout:8/513: write d8/d11/d22/d4c/d65/d1a/f2d [308144,107700] 0 2024-03-24T10:46:13.557 INFO:tasks.workunit.client.1.smithi096.stdout:8/514: stat d8/d72 0 2024-03-24T10:46:13.557 INFO:tasks.workunit.client.1.smithi096.stdout:7/685: creat d0/d21/d3c/d68/d85/fe4 x:0 0 0 2024-03-24T10:46:13.592 INFO:tasks.workunit.client.1.smithi096.stdout:8/515: mkdir d8/d11/d22/d4c/d65/d1a/da0/da6 0 2024-03-24T10:46:13.592 INFO:tasks.workunit.client.1.smithi096.stdout:8/516: chown d8/d11/d22/d4c/d82/l41 265185974 1 2024-03-24T10:46:13.594 INFO:tasks.workunit.client.1.smithi096.stdout:6/440: dwrite d8/d10/d58/f72 [0,4194304] 0 2024-03-24T10:46:13.598 INFO:tasks.workunit.client.1.smithi096.stdout:8/517: link d8/d11/d22/d4c/d65/d52/f88 d8/d11/d44/fa7 0 2024-03-24T10:46:13.599 INFO:tasks.workunit.client.1.smithi096.stdout:6/441: dread d8/d12/f2d [0,4194304] 0 2024-03-24T10:46:13.614 INFO:tasks.workunit.client.1.smithi096.stdout:8/518: symlink d8/d11/d44/d45/la8 0 2024-03-24T10:46:13.616 INFO:tasks.workunit.client.1.smithi096.stdout:8/519: mknod d8/d11/d22/d4c/d65/d52/ca9 0 2024-03-24T10:46:13.618 INFO:tasks.workunit.client.1.smithi096.stdout:6/442: getdents d8/d12/d28/d36 0 2024-03-24T10:46:13.618 INFO:tasks.workunit.client.1.smithi096.stdout:6/443: readlink d8/d10/d13/d63/l5d 0 2024-03-24T10:46:13.620 INFO:tasks.workunit.client.1.smithi096.stdout:8/520: creat d8/d11/d22/d4c/d65/d1a/d46/d2e/faa x:0 0 0 2024-03-24T10:46:13.620 INFO:tasks.workunit.client.1.smithi096.stdout:8/521: chown d8/d11/d22/d4c/l4f 301 1 2024-03-24T10:46:13.620 INFO:tasks.workunit.client.1.smithi096.stdout:8/522: stat d8/d11/d22/d4c/la5 0 2024-03-24T10:46:13.620 INFO:tasks.workunit.client.1.smithi096.stdout:8/523: chown d8/d11/d22/d4c/d65/d52 24183570 1 2024-03-24T10:46:13.620 INFO:tasks.workunit.client.1.smithi096.stdout:8/524: fsync d8/d11/d22/d4c/f9f 0 2024-03-24T10:46:13.622 INFO:tasks.workunit.client.1.smithi096.stdout:6/444: creat d8/d10/f8a x:0 0 0 2024-03-24T10:46:13.622 INFO:tasks.workunit.client.1.smithi096.stdout:6/445: truncate d8/d10/d58/f85 174403 0 2024-03-24T10:46:13.623 INFO:tasks.workunit.client.1.smithi096.stdout:8/525: mknod d8/cab 0 2024-03-24T10:46:13.624 INFO:tasks.workunit.client.1.smithi096.stdout:4/361: sync 2024-03-24T10:46:13.624 INFO:tasks.workunit.client.1.smithi096.stdout:9/443: sync 2024-03-24T10:46:13.624 INFO:tasks.workunit.client.1.smithi096.stdout:9/444: stat d9/db/c52 0 2024-03-24T10:46:13.624 INFO:tasks.workunit.client.1.smithi096.stdout:9/445: dread - d9/d15/d26/d29/f6a zero size 2024-03-24T10:46:13.625 INFO:tasks.workunit.client.1.smithi096.stdout:6/446: mknod d8/c8b 0 2024-03-24T10:46:13.626 INFO:tasks.workunit.client.1.smithi096.stdout:8/526: rmdir d8/d11/d22/d4c/d65/d1d/d3d/d5f 39 2024-03-24T10:46:13.631 INFO:tasks.workunit.client.1.smithi096.stdout:4/362: rename d1/d32/d42/d44 to d1/d2b/d59/d74 0 2024-03-24T10:46:13.634 INFO:tasks.workunit.client.1.smithi096.stdout:8/527: link d8/d11/d44/d45/c81 d8/d11/d22/d4c/d65/d52/cac 0 2024-03-24T10:46:13.634 INFO:tasks.workunit.client.1.smithi096.stdout:4/363: creat d1/d32/f75 x:0 0 0 2024-03-24T10:46:13.635 INFO:tasks.workunit.client.1.smithi096.stdout:8/528: symlink d8/d11/d22/d4c/d65/d1a/d46/d2e/lad 0 2024-03-24T10:46:13.635 INFO:tasks.workunit.client.1.smithi096.stdout:8/529: fsync d8/d11/d22/d4c/d65/f1c 0 2024-03-24T10:46:13.636 INFO:tasks.workunit.client.1.smithi096.stdout:4/364: getdents d1/d11 0 2024-03-24T10:46:13.638 INFO:tasks.workunit.client.1.smithi096.stdout:8/530: unlink d8/d11/d22/d4c/d65/f1c 0 2024-03-24T10:46:13.639 INFO:tasks.workunit.client.1.smithi096.stdout:4/365: creat d1/d2b/d6b/f76 x:0 0 0 2024-03-24T10:46:13.639 INFO:tasks.workunit.client.1.smithi096.stdout:4/366: readlink d1/d11/l6e 0 2024-03-24T10:46:13.640 INFO:tasks.workunit.client.1.smithi096.stdout:6/447: dwrite d8/d10/d19/f34 [0,4194304] 0 2024-03-24T10:46:13.640 INFO:tasks.workunit.client.1.smithi096.stdout:6/448: write d8/d10/f3e [523574,69808] 0 2024-03-24T10:46:13.641 INFO:tasks.workunit.client.1.smithi096.stdout:8/531: creat d8/d11/d22/d4c/d65/d1a/d49/fae x:0 0 0 2024-03-24T10:46:13.642 INFO:tasks.workunit.client.1.smithi096.stdout:6/449: mknod d8/d12/d32/d38/d59/c8c 0 2024-03-24T10:46:13.645 INFO:tasks.workunit.client.1.smithi096.stdout:8/532: creat d8/d11/d22/d4c/faf x:0 0 0 2024-03-24T10:46:13.645 INFO:tasks.workunit.client.1.smithi096.stdout:8/533: fdatasync d8/d11/d22/d4c/d65/d37/f6b 0 2024-03-24T10:46:13.660 INFO:tasks.workunit.client.1.smithi096.stdout:4/367: dwrite d1/d2b/d40/f60 [0,4194304] 0 2024-03-24T10:46:13.660 INFO:tasks.workunit.client.1.smithi096.stdout:4/368: write d1/d11/f12 [3020111,34951] 0 2024-03-24T10:46:13.660 INFO:tasks.workunit.client.1.smithi096.stdout:4/369: readlink d1/d11/l72 0 2024-03-24T10:46:13.661 INFO:tasks.workunit.client.1.smithi096.stdout:4/370: write d1/f1f [1095995,13628] 0 2024-03-24T10:46:13.662 INFO:tasks.workunit.client.1.smithi096.stdout:4/371: unlink d1/d11/f23 0 2024-03-24T10:46:13.663 INFO:tasks.workunit.client.1.smithi096.stdout:4/372: mknod d1/c77 0 2024-03-24T10:46:13.665 INFO:tasks.workunit.client.1.smithi096.stdout:4/373: link d1/d2b/l30 d1/l78 0 2024-03-24T10:46:13.666 INFO:tasks.workunit.client.1.smithi096.stdout:4/374: getdents d1/d2b/d6b 0 2024-03-24T10:46:13.666 INFO:tasks.workunit.client.1.smithi096.stdout:4/375: creat d1/d32/d42/f79 x:0 0 0 2024-03-24T10:46:13.667 INFO:tasks.workunit.client.1.smithi096.stdout:8/534: dwrite d8/fe [0,4194304] 0 2024-03-24T10:46:13.668 INFO:tasks.workunit.client.1.smithi096.stdout:8/535: mkdir d8/d11/d22/d4c/db0 0 2024-03-24T10:46:13.669 INFO:tasks.workunit.client.1.smithi096.stdout:6/450: dwrite d8/d10/d19/f60 [0,4194304] 0 2024-03-24T10:46:13.669 INFO:tasks.workunit.client.1.smithi096.stdout:8/536: symlink d8/d11/d22/d4c/d65/d1a/da0/lb1 0 2024-03-24T10:46:13.675 INFO:tasks.workunit.client.1.smithi096.stdout:8/537: link d8/d11/d22/d4c/d82/f5e d8/d11/d22/d4c/d82/fb2 0 2024-03-24T10:46:13.677 INFO:tasks.workunit.client.1.smithi096.stdout:8/538: unlink d8/d11/d85/ca3 0 2024-03-24T10:46:13.678 INFO:tasks.workunit.client.1.smithi096.stdout:6/451: dread d8/d12/d28/f33 [0,4194304] 0 2024-03-24T10:46:13.678 INFO:tasks.workunit.client.1.smithi096.stdout:6/452: read d8/d12/d28/d36/f7d [288682,20350] 0 2024-03-24T10:46:13.680 INFO:tasks.workunit.client.1.smithi096.stdout:6/453: creat d8/d10/d76/f8d x:0 0 0 2024-03-24T10:46:13.684 INFO:tasks.workunit.client.1.smithi096.stdout:4/376: dwrite d1/f1c [4194304,4194304] 0 2024-03-24T10:46:13.693 INFO:tasks.workunit.client.1.smithi096.stdout:6/454: rmdir d8/d10 39 2024-03-24T10:46:13.693 INFO:tasks.workunit.client.1.smithi096.stdout:6/455: chown d8/d12/d61 45 1 2024-03-24T10:46:13.695 INFO:tasks.workunit.client.1.smithi096.stdout:4/377: symlink d1/d32/d4c/l7a 0 2024-03-24T10:46:13.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.695+0000 7f9314e13700 1 -- 172.21.15.94:0/1189679514 --> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9310101260 con 0x7f92f80692a0 2024-03-24T10:46:13.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.696+0000 7f92feffd700 1 -- 172.21.15.94:0/1189679514 <== mgr.14232 v2:172.21.15.94:6800/2184357820 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+328 (secure 0 0 0) 0x7f9310101260 con 0x7f92f80692a0 2024-03-24T10:46:13.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.698+0000 7f9314e13700 1 -- 172.21.15.94:0/1189679514 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f92f80692a0 msgr2=0x7f92f806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:13.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.698+0000 7f9314e13700 1 --2- 172.21.15.94:0/1189679514 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f92f80692a0 0x7f92f806b750 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9304013870 tx=0x7f9304009e30 comp rx=0 tx=0).stop 2024-03-24T10:46:13.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.698+0000 7f9314e13700 1 -- 172.21.15.94:0/1189679514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93100fbcd0 msgr2=0x7f9310066c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:13.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.698+0000 7f9314e13700 1 --2- 172.21.15.94:0/1189679514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93100fbcd0 0x7f9310066c20 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f930000eb40 tx=0x7f930000b040 comp rx=0 tx=0).stop 2024-03-24T10:46:13.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.699+0000 7f9314e13700 1 -- 172.21.15.94:0/1189679514 shutdown_connections 2024-03-24T10:46:13.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.699+0000 7f9314e13700 1 --2- 172.21.15.94:0/1189679514 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f92f80692a0 0x7f92f806b750 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:13.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.699+0000 7f9314e13700 1 --2- 172.21.15.94:0/1189679514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93100fbcd0 0x7f9310066c20 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:13.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.699+0000 7f9314e13700 1 --2- 172.21.15.94:0/1189679514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93100fcf80 0x7f93101126f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:13.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.699+0000 7f9314e13700 1 -- 172.21.15.94:0/1189679514 >> 172.21.15.94:0/1189679514 conn(0x7f93100f7430 msgr2=0x7f93100f8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:46:13.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.699+0000 7f9314e13700 1 -- 172.21.15.94:0/1189679514 shutdown_connections 2024-03-24T10:46:13.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.699+0000 7f9314e13700 1 -- 172.21.15.94:0/1189679514 wait complete. 2024-03-24T10:46:13.704 INFO:tasks.workunit.client.1.smithi096.stdout:4/378: symlink d1/d2b/l7b 0 2024-03-24T10:46:13.706 INFO:tasks.workunit.client.1.smithi096.stdout:4/379: mknod d1/d32/d4c/c7c 0 2024-03-24T10:46:13.707 INFO:tasks.workunit.client.1.smithi096.stdout:4/380: mkdir d1/d32/d4c/d7d 0 2024-03-24T10:46:13.708 INFO:tasks.workunit.client.1.smithi096.stdout:4/381: dread d1/d2b/d59/d74/f62 [0,4194304] 0 2024-03-24T10:46:13.708 INFO:tasks.workunit.client.1.smithi096.stdout:4/382: write d1/f10 [373467,53698] 0 2024-03-24T10:46:13.711 INFO:tasks.workunit.client.1.smithi096.stdout:4/383: mkdir d1/d32/d4c/d7e 0 2024-03-24T10:46:13.713 INFO:tasks.workunit.client.1.smithi096.stdout:4/384: mknod d1/d2b/d40/c7f 0 2024-03-24T10:46:13.713 INFO:tasks.workunit.client.1.smithi096.stdout:4/385: fdatasync d1/d2b/d6b/f76 0 2024-03-24T10:46:13.713 INFO:tasks.workunit.client.1.smithi096.stdout:4/386: write d1/d2b/d6b/f76 [663902,71885] 0 2024-03-24T10:46:13.726 INFO:tasks.workunit.client.1.smithi096.stdout:4/387: dwrite d1/d2b/d40/f6d [0,4194304] 0 2024-03-24T10:46:13.729 INFO:tasks.workunit.client.1.smithi096.stdout:4/388: mkdir d1/d2b/d6b/d80 0 2024-03-24T10:46:13.736 INFO:tasks.workunit.client.1.smithi096.stdout:4/389: dread d1/d2b/d40/f60 [0,4194304] 0 2024-03-24T10:46:13.737 INFO:tasks.workunit.client.1.smithi096.stdout:4/390: write d1/d11/f41 [692772,13558] 0 2024-03-24T10:46:13.737 INFO:tasks.workunit.client.1.smithi096.stdout:4/391: fdatasync d1/d11/f66 0 2024-03-24T10:46:13.739 INFO:tasks.workunit.client.1.smithi096.stdout:4/392: write d1/d11/f12 [3539658,6135] 0 2024-03-24T10:46:13.756 INFO:tasks.workunit.client.1.smithi096.stdout:4/393: dwrite d1/f24 [0,4194304] 0 2024-03-24T10:46:13.759 INFO:tasks.workunit.client.1.smithi096.stdout:4/394: unlink d1/d32/c47 0 2024-03-24T10:46:13.760 INFO:tasks.workunit.client.1.smithi096.stdout:4/395: symlink d1/d32/d4c/d7d/l81 0 2024-03-24T10:46:13.760 INFO:tasks.workunit.client.1.smithi096.stdout:4/396: readlink d1/l21 0 2024-03-24T10:46:13.769 INFO:tasks.workunit.client.1.smithi096.stdout:4/397: dread d1/d32/d4c/f5a [0,4194304] 0 2024-03-24T10:46:13.770 INFO:tasks.workunit.client.1.smithi096.stdout:4/398: unlink d1/d11/c37 0 2024-03-24T10:46:13.770 INFO:tasks.workunit.client.1.smithi096.stdout:4/399: write d1/f6f [396118,124836] 0 2024-03-24T10:46:13.787 INFO:tasks.workunit.client.1.smithi096.stdout:4/400: dwrite d1/d2b/d59/d74/f62 [0,4194304] 0 2024-03-24T10:46:13.792 INFO:tasks.workunit.client.1.smithi096.stdout:4/401: dread d1/d32/d4c/f5a [0,4194304] 0 2024-03-24T10:46:13.792 INFO:tasks.workunit.client.1.smithi096.stdout:4/402: creat d1/d2b/d67/f82 x:0 0 0 2024-03-24T10:46:13.793 INFO:tasks.workunit.client.1.smithi096.stdout:4/403: getdents d1/d32/d4c 0 2024-03-24T10:46:13.802 INFO:tasks.workunit.client.1.smithi096.stdout:4/404: write d1/d2b/d59/d74/f62 [1934563,112355] 0 2024-03-24T10:46:13.804 INFO:tasks.workunit.client.1.smithi096.stdout:4/405: rename d1/d32/d4c/f5a to d1/d2b/d6b/f83 0 2024-03-24T10:46:13.804 INFO:tasks.workunit.client.1.smithi096.stdout:4/406: fdatasync d1/d2b/d40/f46 0 2024-03-24T10:46:13.805 INFO:tasks.workunit.client.1.smithi096.stdout:4/407: rename l0 to d1/d2b/d59/d74/l84 0 2024-03-24T10:46:13.806 INFO:tasks.workunit.client.1.smithi096.stdout:4/408: link d1/d11/l22 d1/d2b/d67/d5f/l85 0 2024-03-24T10:46:13.814 INFO:tasks.workunit.client.1.smithi096.stdout:2/403: sync 2024-03-24T10:46:13.814 INFO:tasks.workunit.client.1.smithi096.stdout:4/409: getdents d1/d2b/d67/d5f 0 2024-03-24T10:46:13.815 INFO:tasks.workunit.client.1.smithi096.stdout:4/410: creat d1/d32/d42/f86 x:0 0 0 2024-03-24T10:46:13.817 INFO:tasks.workunit.client.1.smithi096.stdout:2/404: creat d5/dc/d6b/d72/f9c x:0 0 0 2024-03-24T10:46:13.817 INFO:tasks.workunit.client.1.smithi096.stdout:2/405: getdents d5/dc/d6b/d72/d91 0 2024-03-24T10:46:13.817 INFO:tasks.workunit.client.1.smithi096.stdout:2/406: dread - d5/dc/d12/d23/f7a zero size 2024-03-24T10:46:13.819 INFO:tasks.workunit.client.1.smithi096.stdout:4/411: unlink d1/d2b/d40/f6d 0 2024-03-24T10:46:13.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.825+0000 7f6b4037f700 1 -- 172.21.15.94:0/1026079127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b380662e0 msgr2=0x7f6b38101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:13.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.825+0000 7f6b4037f700 1 --2- 172.21.15.94:0/1026079127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b380662e0 0x7f6b38101dc0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6b34004940 tx=0x7f6b34009e30 comp rx=0 tx=0).stop 2024-03-24T10:46:13.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.825+0000 7f6b4037f700 1 -- 172.21.15.94:0/1026079127 shutdown_connections 2024-03-24T10:46:13.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.825+0000 7f6b4037f700 1 --2- 172.21.15.94:0/1026079127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b38102300 0x7f6b381046e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:13.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.825+0000 7f6b4037f700 1 --2- 172.21.15.94:0/1026079127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b380662e0 0x7f6b38101dc0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:13.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.825+0000 7f6b4037f700 1 -- 172.21.15.94:0/1026079127 >> 172.21.15.94:0/1026079127 conn(0x7f6b380f7430 msgr2=0x7f6b380f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:46:13.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.825+0000 7f6b4037f700 1 -- 172.21.15.94:0/1026079127 shutdown_connections 2024-03-24T10:46:13.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.825+0000 7f6b4037f700 1 -- 172.21.15.94:0/1026079127 wait complete. 2024-03-24T10:46:13.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.826+0000 7f6b4037f700 1 Processor -- start 2024-03-24T10:46:13.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.826+0000 7f6b4037f700 1 -- start start 2024-03-24T10:46:13.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.826+0000 7f6b4037f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b380662e0 0x7f6b3810fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:46:13.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.826+0000 7f6b4037f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b38102300 0x7f6b38110160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:46:13.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.826+0000 7f6b4037f700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b38110760 con 0x7f6b380662e0 2024-03-24T10:46:13.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.826+0000 7f6b4037f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b38110fb0 con 0x7f6b38102300 2024-03-24T10:46:13.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.826+0000 7f6b3e11b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b380662e0 0x7f6b3810fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:46:13.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.826+0000 7f6b3e11b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b380662e0 0x7f6b3810fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54310/0 (socket says 172.21.15.94:54310) 2024-03-24T10:46:13.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.826+0000 7f6b3e11b700 1 -- 172.21.15.94:0/2283151362 learned_addr learned my addr 172.21.15.94:0/2283151362 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:46:13.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.826+0000 7f6b3d91a700 1 --2- 172.21.15.94:0/2283151362 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b38102300 0x7f6b38110160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:46:13.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.826+0000 7f6b3e11b700 1 -- 172.21.15.94:0/2283151362 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b38102300 msgr2=0x7f6b38110160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:13.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.826+0000 7f6b3e11b700 1 --2- 172.21.15.94:0/2283151362 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b38102300 0x7f6b38110160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:13.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.826+0000 7f6b3e11b700 1 -- 172.21.15.94:0/2283151362 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b34005040 con 0x7f6b380662e0 2024-03-24T10:46:13.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.826+0000 7f6b3d91a700 1 --2- 172.21.15.94:0/2283151362 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b38102300 0x7f6b38110160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T10:46:13.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.826+0000 7f6b3e11b700 1 --2- 172.21.15.94:0/2283151362 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b380662e0 0x7f6b3810fc20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6b3400b3b0 tx=0x7f6b34009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:46:13.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.827+0000 7f6b2effd700 1 -- 172.21.15.94:0/2283151362 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b3400e040 con 0x7f6b380662e0 2024-03-24T10:46:13.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.827+0000 7f6b4037f700 1 -- 172.21.15.94:0/2283151362 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b38111230 con 0x7f6b380662e0 2024-03-24T10:46:13.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.827+0000 7f6b2effd700 1 -- 172.21.15.94:0/2283151362 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6b3400bcc0 con 0x7f6b380662e0 2024-03-24T10:46:13.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.827+0000 7f6b2effd700 1 -- 172.21.15.94:0/2283151362 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b3401c630 con 0x7f6b380662e0 2024-03-24T10:46:13.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.827+0000 7f6b4037f700 1 -- 172.21.15.94:0/2283151362 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b38111770 con 0x7f6b380662e0 2024-03-24T10:46:13.831 INFO:tasks.workunit.client.1.smithi096.stdout:4/412: dread d1/d2b/d40/f46 [0,4194304] 0 2024-03-24T10:46:13.832 INFO:tasks.workunit.client.1.smithi096.stdout:4/413: symlink d1/d32/d4c/d7d/l87 0 2024-03-24T10:46:13.832 INFO:tasks.workunit.client.1.smithi096.stdout:2/407: dwrite d5/de/d2a/f5a [0,4194304] 0 2024-03-24T10:46:13.832 INFO:tasks.workunit.client.1.smithi096.stdout:2/408: dread - d5/f84 zero size 2024-03-24T10:46:13.832 INFO:tasks.workunit.client.1.smithi096.stdout:4/414: link d1/d11/c2e d1/d2b/d59/d74/c88 0 2024-03-24T10:46:13.832 INFO:tasks.workunit.client.1.smithi096.stdout:4/415: write d1/f6f [315519,114091] 0 2024-03-24T10:46:13.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.828+0000 7f6b4037f700 1 -- 172.21.15.94:0/2283151362 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b3805f5f0 con 0x7f6b380662e0 2024-03-24T10:46:13.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.828+0000 7f6b2effd700 1 -- 172.21.15.94:0/2283151362 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f6b340139f0 con 0x7f6b380662e0 2024-03-24T10:46:13.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.828+0000 7f6b2effd700 1 --2- 172.21.15.94:0/2283151362 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f6b240692a0 0x7f6b2406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:46:13.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.829+0000 7f6b2effd700 1 -- 172.21.15.94:0/2283151362 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f6b34087fe0 con 0x7f6b380662e0 2024-03-24T10:46:13.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.829+0000 7f6b3d91a700 1 --2- 172.21.15.94:0/2283151362 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f6b240692a0 0x7f6b2406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:46:13.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.829+0000 7f6b3d91a700 1 --2- 172.21.15.94:0/2283151362 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f6b240692a0 0x7f6b2406b750 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6b28009fd0 tx=0x7f6b28009ee0 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:46:13.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:13.834+0000 7f6b2effd700 1 -- 172.21.15.94:0/2283151362 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f6b34057d50 con 0x7f6b380662e0 2024-03-24T10:46:13.838 INFO:tasks.workunit.client.1.smithi096.stdout:4/416: dread d1/d11/f19 [0,4194304] 0 2024-03-24T10:46:13.838 INFO:tasks.workunit.client.1.smithi096.stdout:4/417: chown d1/d11/f41 1868674 1 2024-03-24T10:46:13.930 INFO:tasks.workunit.client.1.smithi096.stdout:2/409: stat d5/d38/c55 0 2024-03-24T10:46:13.933 INFO:tasks.workunit.client.1.smithi096.stdout:2/410: dread d5/dc/f25 [0,4194304] 0 2024-03-24T10:46:13.938 INFO:tasks.workunit.client.1.smithi096.stdout:4/418: mkdir d1/d2b/d89 0 2024-03-24T10:46:13.938 INFO:tasks.workunit.client.1.smithi096.stdout:4/419: chown d1/cc 125751 1 2024-03-24T10:46:13.942 INFO:tasks.workunit.client.1.smithi096.stdout:2/411: dread d5/dc/f2f [0,4194304] 0 2024-03-24T10:46:13.942 INFO:tasks.workunit.client.1.smithi096.stdout:2/412: write d5/f1f [247981,40612] 0 2024-03-24T10:46:13.942 INFO:tasks.workunit.client.1.smithi096.stdout:2/413: read d5/dc/d12/f48 [361685,108639] 0 2024-03-24T10:46:13.949 INFO:tasks.workunit.client.1.smithi096.stdout:4/420: truncate d1/d2b/d59/d74/f62 391276 0 2024-03-24T10:46:13.949 INFO:tasks.workunit.client.1.smithi096.stdout:4/421: creat d1/d2b/d67/f8a x:0 0 0 2024-03-24T10:46:13.951 INFO:tasks.workunit.client.1.smithi096.stdout:4/422: dread d1/d2b/d6b/f76 [0,4194304] 0 2024-03-24T10:46:13.956 INFO:tasks.workunit.client.1.smithi096.stdout:4/423: link d1/d2b/l3f d1/d32/d4c/d7e/l8b 0 2024-03-24T10:46:13.958 INFO:tasks.workunit.client.1.smithi096.stdout:4/424: symlink d1/d32/l8c 0 2024-03-24T10:46:13.960 INFO:tasks.workunit.client.1.smithi096.stdout:4/425: symlink d1/d32/d4c/d7e/l8d 0 2024-03-24T10:46:13.962 INFO:tasks.workunit.client.1.smithi096.stdout:2/414: dwrite d5/dc/d12/f26 [0,4194304] 0 2024-03-24T10:46:13.963 INFO:tasks.workunit.client.1.smithi096.stdout:2/415: symlink d5/dc/d6b/d72/d91/l9d 0 2024-03-24T10:46:13.965 INFO:tasks.workunit.client.1.smithi096.stdout:4/426: read d1/d11/f19 [2116654,122017] 0 2024-03-24T10:46:13.967 INFO:tasks.workunit.client.1.smithi096.stdout:2/416: link d5/de/d2a/d5e/c6e d5/dc/d6b/c9e 0 2024-03-24T10:46:13.967 INFO:tasks.workunit.client.1.smithi096.stdout:2/417: stat d5/dc/l5d 0 2024-03-24T10:46:13.968 INFO:tasks.workunit.client.1.smithi096.stdout:4/427: rename d1/d32/d42/l71 to d1/d11/l8e 0 2024-03-24T10:46:13.969 INFO:tasks.workunit.client.1.smithi096.stdout:4/428: write d1/fd [5195226,65739] 0 2024-03-24T10:46:13.971 INFO:tasks.workunit.client.1.smithi096.stdout:1/404: sync 2024-03-24T10:46:13.971 INFO:tasks.workunit.client.1.smithi096.stdout:4/429: rename d1/d11/l31 to d1/d32/l8f 0 2024-03-24T10:46:13.974 INFO:tasks.workunit.client.1.smithi096.stdout:4/430: getdents d1/d2b 0 2024-03-24T10:46:13.978 INFO:tasks.workunit.client.1.smithi096.stdout:4/431: symlink d1/d2b/l90 0 2024-03-24T10:46:13.984 INFO:tasks.workunit.client.1.smithi096.stdout:2/418: dwrite d5/de/d2a/f37 [8388608,4194304] 0 2024-03-24T10:46:13.986 INFO:tasks.workunit.client.1.smithi096.stdout:2/419: rmdir d5/dc/d6b/d72 39 2024-03-24T10:46:13.988 INFO:tasks.workunit.client.1.smithi096.stdout:1/405: dwrite d3/d23/d8b/d6a/f70 [0,4194304] 0 2024-03-24T10:46:13.988 INFO:tasks.workunit.client.1.smithi096.stdout:1/406: readlink d3/d23/l4b 0 2024-03-24T10:46:13.988 INFO:tasks.workunit.client.1.smithi096.stdout:2/420: truncate d5/de/d2a/f2b 938621 0 2024-03-24T10:46:13.988 INFO:tasks.workunit.client.1.smithi096.stdout:2/421: fsync d5/d38/f76 0 2024-03-24T10:46:13.991 INFO:tasks.workunit.client.1.smithi096.stdout:2/422: mkdir d5/d79/d9f 0 2024-03-24T10:46:13.992 INFO:tasks.workunit.client.1.smithi096.stdout:4/432: getdents d1/d2b 0 2024-03-24T10:46:13.993 INFO:tasks.workunit.client.1.smithi096.stdout:4/433: dread - d1/d2b/d67/f8a zero size 2024-03-24T10:46:13.993 INFO:tasks.workunit.client.1.smithi096.stdout:4/434: write d1/d2b/d67/f6a [1365794,5236] 0 2024-03-24T10:46:13.993 INFO:tasks.workunit.client.1.smithi096.stdout:4/435: chown d1/d2b/d40/c7f 124482974 1 2024-03-24T10:46:13.999 INFO:tasks.workunit.client.1.smithi096.stdout:2/423: write d5/de/d2a/f4c [3568175,4481] 0 2024-03-24T10:46:14.000 INFO:tasks.workunit.client.1.smithi096.stdout:5/459: sync 2024-03-24T10:46:14.002 INFO:tasks.workunit.client.1.smithi096.stdout:5/460: mknod d2/d29/d32/d3f/d84/c9d 0 2024-03-24T10:46:14.005 INFO:tasks.workunit.client.1.smithi096.stdout:5/461: creat d2/d29/d32/d33/d8d/f9e x:0 0 0 2024-03-24T10:46:14.006 INFO:tasks.workunit.client.1.smithi096.stdout:5/462: getdents d2/d7/d8/d18/d4d 0 2024-03-24T10:46:14.008 INFO:tasks.workunit.client.1.smithi096.stdout:4/436: dwrite d1/d32/d42/f5e [0,4194304] 0 2024-03-24T10:46:14.009 INFO:tasks.workunit.client.1.smithi096.stdout:2/424: dread d5/dc/d12/f26 [0,4194304] 0 2024-03-24T10:46:14.009 INFO:tasks.workunit.client.1.smithi096.stdout:2/425: chown d5/d79/d9f 786 1 2024-03-24T10:46:14.009 INFO:tasks.workunit.client.1.smithi096.stdout:2/426: readlink d5/dc/d6b/l74 0 2024-03-24T10:46:14.010 INFO:tasks.workunit.client.1.smithi096.stdout:4/437: mknod d1/d2b/d59/c91 0 2024-03-24T10:46:14.010 INFO:tasks.workunit.client.1.smithi096.stdout:4/438: fsync d1/fe 0 2024-03-24T10:46:14.011 INFO:tasks.workunit.client.1.smithi096.stdout:2/427: creat d5/dc/d6b/d72/d91/fa0 x:0 0 0 2024-03-24T10:46:14.012 INFO:tasks.workunit.client.1.smithi096.stdout:2/428: truncate d5/de/d2a/f4c 4579961 0 2024-03-24T10:46:14.012 INFO:tasks.workunit.client.1.smithi096.stdout:2/429: write d5/dc/f5b [12868381,34503] 0 2024-03-24T10:46:14.012 INFO:tasks.workunit.client.1.smithi096.stdout:5/463: getdents d2/d7/d69 0 2024-03-24T10:46:14.014 INFO:tasks.workunit.client.1.smithi096.stdout:2/430: chown d5/d38/c46 743688 1 2024-03-24T10:46:14.018 INFO:tasks.workunit.client.1.smithi096.stdout:5/464: mknod d2/d29/d32/d33/d3c/c9f 0 2024-03-24T10:46:14.021 INFO:tasks.workunit.client.1.smithi096.stdout:5/465: rename d2/d29/d32/d33/d3c/l6b to d2/d29/d32/la0 0 2024-03-24T10:46:14.021 INFO:tasks.workunit.client.1.smithi096.stdout:5/466: creat d2/d29/d32/d89/fa1 x:0 0 0 2024-03-24T10:46:14.025 INFO:tasks.workunit.client.1.smithi096.stdout:5/467: unlink d2/d7/d8/d18/f20 0 2024-03-24T10:46:14.026 INFO:tasks.workunit.client.1.smithi096.stdout:5/468: dread d2/d29/d32/d33/d37/f6d [0,4194304] 0 2024-03-24T10:46:14.026 INFO:tasks.workunit.client.1.smithi096.stdout:5/469: creat d2/d29/d32/d33/d8d/fa2 x:0 0 0 2024-03-24T10:46:14.026 INFO:tasks.workunit.client.1.smithi096.stdout:5/470: read - d2/d29/d32/d89/f9c zero size 2024-03-24T10:46:14.031 INFO:tasks.workunit.client.1.smithi096.stdout:2/431: dwrite d5/dc/f25 [0,4194304] 0 2024-03-24T10:46:14.033 INFO:tasks.workunit.client.1.smithi096.stdout:2/432: link d5/de/d2a/f32 d5/d38/fa1 0 2024-03-24T10:46:14.043 INFO:tasks.workunit.client.1.smithi096.stdout:2/433: dread d5/dc/f5b [8388608,4194304] 0 2024-03-24T10:46:14.045 INFO:tasks.workunit.client.1.smithi096.stdout:2/434: symlink d5/d93/la2 0 2024-03-24T10:46:14.045 INFO:tasks.workunit.client.1.smithi096.stdout:2/435: truncate d5/de/d2a/d5e/f8d 364507 0 2024-03-24T10:46:14.048 INFO:tasks.workunit.client.1.smithi096.stdout:2/436: truncate d5/d7c/f7e 3624255 0 2024-03-24T10:46:14.048 INFO:tasks.workunit.client.1.smithi096.stdout:2/437: readlink d5/d93/la2 0 2024-03-24T10:46:14.049 INFO:tasks.workunit.client.1.smithi096.stdout:5/471: dwrite d2/d29/d32/d33/f44 [0,4194304] 0 2024-03-24T10:46:14.050 INFO:tasks.workunit.client.1.smithi096.stdout:2/438: dread d5/de/d2a/f40 [0,4194304] 0 2024-03-24T10:46:14.053 INFO:tasks.workunit.client.1.smithi096.stdout:5/472: rmdir d2/d29/d32/d33/d8d/d94/d99 0 2024-03-24T10:46:14.054 INFO:tasks.workunit.client.1.smithi096.stdout:2/439: mknod d5/d7c/ca3 0 2024-03-24T10:46:14.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.054+0000 7f6b4037f700 1 -- 172.21.15.94:0/2283151362 --> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6b38111aa0 con 0x7f6b240692a0 2024-03-24T10:46:14.057 INFO:tasks.workunit.client.1.smithi096.stdout:2/440: link d5/dc/f6c d5/fa4 0 2024-03-24T10:46:14.062 INFO:tasks.workunit.client.1.smithi096.stdout:2/441: rename d5/de/d2a/c39 to d5/dc/d12/d23/ca5 0 2024-03-24T10:46:14.062 INFO:tasks.workunit.client.1.smithi096.stdout:2/442: truncate d5/de/d2a/f78 1000775 0 2024-03-24T10:46:14.063 INFO:tasks.workunit.client.1.smithi096.stdout:2/443: creat d5/de/d2a/fa6 x:0 0 0 2024-03-24T10:46:14.063 INFO:tasks.workunit.client.1.smithi096.stdout:2/444: chown d5/de/d2a/d5e/f7f 3 1 2024-03-24T10:46:14.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.067+0000 7f6b2effd700 1 -- 172.21.15.94:0/2283151362 <== mgr.14232 v2:172.21.15.94:6800/2184357820 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3344 (secure 0 0 0) 0x7f6b38111aa0 con 0x7f6b240692a0 2024-03-24T10:46:14.068 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T10:46:14.068 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.smithi094 smithi094 *:9093,9094 running (6m) 2m ago 8m 25.2M - 0.25.0 c8568f914cd2 8072d4b65986 2024-03-24T10:46:14.068 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi094 smithi094 running (8m) 2m ago 8m 6660k - 17.2.7-726-g9c4e0c34 837124da804f fb5772af46c5 2024-03-24T10:46:14.068 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi096 smithi096 running (7m) 2m ago 7m 6669k - 17.2.7-726-g9c4e0c34 837124da804f 6087ad4e0734 2024-03-24T10:46:14.068 INFO:teuthology.orchestra.run.smithi094.stdout:grafana.smithi094 smithi094 *:3000 running (6m) 2m ago 7m 83.6M - 9.4.7 954c08fa6188 7b534968fb36 2024-03-24T10:46:14.068 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.efiqte smithi094 running (2m) 2m ago 2m 10.4M - 17.2.7-726-g9c4e0c34 837124da804f f0df0a056c21 2024-03-24T10:46:14.068 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.llefgu smithi094 running (2m) 2m ago 2m 11.9M - 17.2.7-726-g9c4e0c34 837124da804f 52a9771dc8e0 2024-03-24T10:46:14.068 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi096.egbmoq smithi096 running (2m) 2m ago 2m 14.5M - 17.2.7-726-g9c4e0c34 837124da804f 1cfc930dbab0 2024-03-24T10:46:14.068 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi096.jtebhh smithi096 running (2m) 2m ago 2m 13.5M - 17.2.7-726-g9c4e0c34 837124da804f 4ff3451a6f6e 2024-03-24T10:46:14.068 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi094.yaethu smithi094 *:9283 running (9m) 2m ago 9m 481M - 17.2.7-726-g9c4e0c34 837124da804f 327d527697e0 2024-03-24T10:46:14.068 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi096.gfonnu smithi096 *:8443,9283 starting - - - - 2024-03-24T10:46:14.068 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi094 smithi094 running (9m) 2m ago 10m 43.3M 2048M 17.2.7-726-g9c4e0c34 837124da804f 76584fceb32b 2024-03-24T10:46:14.069 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi096 smithi096 running (6m) 2m ago 6m 36.3M 2048M 17.2.7-726-g9c4e0c34 837124da804f 710030b222cb 2024-03-24T10:46:14.069 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi094 smithi094 *:9100 running (8m) 2m ago 8m 15.4M - 1.5.0 0da6a335fe13 59895363f7e6 2024-03-24T10:46:14.069 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi096 smithi096 *:9100 running (6m) 2m ago 6m 15.3M - 1.5.0 0da6a335fe13 fc629282835f 2024-03-24T10:46:14.069 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (6m) 2m ago 6m 47.2M 4288M 17.2.7-726-g9c4e0c34 837124da804f b664fd3557bd 2024-03-24T10:46:14.069 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi094 running (5m) 2m ago 5m 46.3M 4288M 17.2.7-726-g9c4e0c34 837124da804f 4e737a80d733 2024-03-24T10:46:14.069 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi094 running (5m) 2m ago 5m 47.2M 4288M 17.2.7-726-g9c4e0c34 837124da804f cf0aefd20c76 2024-03-24T10:46:14.069 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi096 running (4m) 2m ago 4m 44.7M 5312M 17.2.7-726-g9c4e0c34 837124da804f f4650d0f6241 2024-03-24T10:46:14.069 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi096 running (4m) 2m ago 4m 44.6M 5312M 17.2.7-726-g9c4e0c34 837124da804f a63d7c9e705f 2024-03-24T10:46:14.069 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi096 running (3m) 2m ago 3m 41.7M 5312M 17.2.7-726-g9c4e0c34 837124da804f f56fad2959c3 2024-03-24T10:46:14.069 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.smithi094 smithi094 *:9095 running (6m) 2m ago 7m 57.0M - 2.43.0 a07b618ecd1d 023eb0a49108 2024-03-24T10:46:14.070 INFO:tasks.workunit.client.1.smithi096.stdout:2/445: write f2 [12489798,80272] 0 2024-03-24T10:46:14.070 INFO:tasks.workunit.client.1.smithi096.stdout:2/446: creat d5/fa7 x:0 0 0 2024-03-24T10:46:14.070 INFO:tasks.workunit.client.1.smithi096.stdout:5/473: dwrite d2/d29/d32/d33/d37/f6d [0,4194304] 0 2024-03-24T10:46:14.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.069+0000 7f6b4037f700 1 -- 172.21.15.94:0/2283151362 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f6b240692a0 msgr2=0x7f6b2406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:14.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.069+0000 7f6b4037f700 1 --2- 172.21.15.94:0/2283151362 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f6b240692a0 0x7f6b2406b750 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6b28009fd0 tx=0x7f6b28009ee0 comp rx=0 tx=0).stop 2024-03-24T10:46:14.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.069+0000 7f6b4037f700 1 -- 172.21.15.94:0/2283151362 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b380662e0 msgr2=0x7f6b3810fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:14.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.069+0000 7f6b4037f700 1 --2- 172.21.15.94:0/2283151362 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b380662e0 0x7f6b3810fc20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6b3400b3b0 tx=0x7f6b34009e30 comp rx=0 tx=0).stop 2024-03-24T10:46:14.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.070+0000 7f6b4037f700 1 -- 172.21.15.94:0/2283151362 shutdown_connections 2024-03-24T10:46:14.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.070+0000 7f6b4037f700 1 --2- 172.21.15.94:0/2283151362 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f6b240692a0 0x7f6b2406b750 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:14.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.070+0000 7f6b4037f700 1 --2- 172.21.15.94:0/2283151362 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b380662e0 0x7f6b3810fc20 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:14.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.070+0000 7f6b4037f700 1 --2- 172.21.15.94:0/2283151362 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b38102300 0x7f6b38110160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:14.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.070+0000 7f6b4037f700 1 -- 172.21.15.94:0/2283151362 >> 172.21.15.94:0/2283151362 conn(0x7f6b380f7430 msgr2=0x7f6b380f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:46:14.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.070+0000 7f6b4037f700 1 -- 172.21.15.94:0/2283151362 shutdown_connections 2024-03-24T10:46:14.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.070+0000 7f6b4037f700 1 -- 172.21.15.94:0/2283151362 wait complete. 2024-03-24T10:46:14.072 INFO:tasks.workunit.client.1.smithi096.stdout:5/474: read d2/f43 [3933564,55] 0 2024-03-24T10:46:14.073 INFO:tasks.workunit.client.1.smithi096.stdout:2/447: creat d5/d38/fa8 x:0 0 0 2024-03-24T10:46:14.073 INFO:tasks.workunit.client.1.smithi096.stdout:2/448: creat d5/de/d2a/d5e/fa9 x:0 0 0 2024-03-24T10:46:14.077 INFO:tasks.workunit.client.1.smithi096.stdout:5/475: symlink d2/d29/d32/d33/d8d/la3 0 2024-03-24T10:46:14.090 INFO:tasks.workunit.client.1.smithi096.stdout:2/449: dwrite d5/dc/f34 [0,4194304] 0 2024-03-24T10:46:14.097 INFO:tasks.workunit.client.1.smithi096.stdout:5/476: dwrite d2/d7/d8/f12 [0,4194304] 0 2024-03-24T10:46:14.102 INFO:tasks.workunit.client.1.smithi096.stdout:2/450: symlink d5/dc/d12/laa 0 2024-03-24T10:46:14.103 INFO:tasks.workunit.client.1.smithi096.stdout:2/451: readlink d5/l96 0 2024-03-24T10:46:14.106 INFO:tasks.workunit.client.1.smithi096.stdout:5/477: dread d2/d7/d8/d18/f47 [0,4194304] 0 2024-03-24T10:46:14.107 INFO:tasks.workunit.client.1.smithi096.stdout:2/452: symlink d5/d7c/d8f/lab 0 2024-03-24T10:46:14.111 INFO:tasks.workunit.client.1.smithi096.stdout:2/453: creat d5/d7c/d88/fac x:0 0 0 2024-03-24T10:46:14.111 INFO:tasks.workunit.client.1.smithi096.stdout:2/454: write d5/dc/d12/f4a [2780140,15251] 0 2024-03-24T10:46:14.113 INFO:tasks.workunit.client.1.smithi096.stdout:5/478: symlink d2/d7/la4 0 2024-03-24T10:46:14.115 INFO:tasks.workunit.client.1.smithi096.stdout:2/455: mkdir d5/d79/d9f/dad 0 2024-03-24T10:46:14.117 INFO:tasks.workunit.client.1.smithi096.stdout:5/479: unlink d2/d7/d8/d18/d4d/c73 0 2024-03-24T10:46:14.119 INFO:tasks.workunit.client.1.smithi096.stdout:2/456: truncate d5/d38/f67 1212295 0 2024-03-24T10:46:14.135 INFO:tasks.workunit.client.1.smithi096.stdout:5/480: dwrite d2/d29/d32/d33/d37/f6d [0,4194304] 0 2024-03-24T10:46:14.136 INFO:tasks.workunit.client.1.smithi096.stdout:5/481: fsync d2/d29/f90 0 2024-03-24T10:46:14.136 INFO:tasks.workunit.client.1.smithi096.stdout:5/482: fsync d2/d29/f90 0 2024-03-24T10:46:14.136 INFO:tasks.workunit.client.1.smithi096.stdout:2/457: write d5/d38/f67 [191040,43899] 0 2024-03-24T10:46:14.141 INFO:tasks.workunit.client.1.smithi096.stdout:5/483: link d2/d7/d69/f87 d2/d29/d32/d3f/d84/fa5 0 2024-03-24T10:46:14.141 INFO:tasks.workunit.client.1.smithi096.stdout:5/484: stat d2/d29/d32/d3f/d84/fa5 0 2024-03-24T10:46:14.141 INFO:tasks.workunit.client.1.smithi096.stdout:5/485: fsync d2/d7/d8/df/d14/f23 0 2024-03-24T10:46:14.143 INFO:tasks.workunit.client.1.smithi096.stdout:2/458: creat d5/dc/d6b/fae x:0 0 0 2024-03-24T10:46:14.147 INFO:tasks.workunit.client.1.smithi096.stdout:2/459: mknod d5/dc/d6b/caf 0 2024-03-24T10:46:14.153 INFO:tasks.workunit.client.1.smithi096.stdout:6/456: sync 2024-03-24T10:46:14.153 INFO:tasks.workunit.client.1.smithi096.stdout:7/686: sync 2024-03-24T10:46:14.153 INFO:tasks.workunit.client.1.smithi096.stdout:9/446: sync 2024-03-24T10:46:14.153 INFO:tasks.workunit.client.1.smithi096.stdout:0/486: sync 2024-03-24T10:46:14.153 INFO:tasks.workunit.client.1.smithi096.stdout:7/687: chown d0/d21/d3c/lda 7773035 1 2024-03-24T10:46:14.153 INFO:tasks.workunit.client.1.smithi096.stdout:0/487: fdatasync d0/d11/d20/d2b/f6a 0 2024-03-24T10:46:14.154 INFO:tasks.workunit.client.1.smithi096.stdout:3/513: sync 2024-03-24T10:46:14.154 INFO:tasks.workunit.client.1.smithi096.stdout:8/539: sync 2024-03-24T10:46:14.155 INFO:tasks.workunit.client.1.smithi096.stdout:5/486: dwrite d2/d7/d8/df/f78 [0,4194304] 0 2024-03-24T10:46:14.155 INFO:tasks.workunit.client.1.smithi096.stdout:5/487: creat d2/fa6 x:0 0 0 2024-03-24T10:46:14.155 INFO:tasks.workunit.client.1.smithi096.stdout:5/488: dread - d2/d29/d32/d3f/d84/fa5 zero size 2024-03-24T10:46:14.155 INFO:tasks.workunit.client.1.smithi096.stdout:6/457: mknod d8/d12/d28/d36/d65/d6b/c8e 0 2024-03-24T10:46:14.156 INFO:tasks.workunit.client.1.smithi096.stdout:9/447: symlink d9/db/d6d/l90 0 2024-03-24T10:46:14.159 INFO:tasks.workunit.client.1.smithi096.stdout:7/688: unlink d0/d21/d3c/d68/l4c 0 2024-03-24T10:46:14.161 INFO:tasks.workunit.client.1.smithi096.stdout:0/488: creat d0/d11/d16/d21/d25/fa4 x:0 0 0 2024-03-24T10:46:14.161 INFO:tasks.workunit.client.1.smithi096.stdout:0/489: write d0/d11/d20/d91/f96 [1272678,31911] 0 2024-03-24T10:46:14.161 INFO:tasks.workunit.client.1.smithi096.stdout:8/540: read d8/d11/d22/d4c/d65/d37/f39 [3128253,30161] 0 2024-03-24T10:46:14.161 INFO:tasks.workunit.client.1.smithi096.stdout:8/541: write d8/d11/d22/d4c/d65/d1a/d49/f6a [1480758,16134] 0 2024-03-24T10:46:14.167 INFO:tasks.workunit.client.1.smithi096.stdout:5/489: creat d2/d7/fa7 x:0 0 0 2024-03-24T10:46:14.167 INFO:tasks.workunit.client.1.smithi096.stdout:5/490: write d2/d29/d32/d89/f8e [146650,15682] 0 2024-03-24T10:46:14.167 INFO:tasks.workunit.client.1.smithi096.stdout:5/491: readlink d2/l3b 0 2024-03-24T10:46:14.167 INFO:tasks.workunit.client.1.smithi096.stdout:5/492: truncate d2/d29/d32/d89/fa1 984165 0 2024-03-24T10:46:14.168 INFO:tasks.workunit.client.1.smithi096.stdout:6/458: truncate d8/d12/d32/f3c 512702 0 2024-03-24T10:46:14.170 INFO:tasks.workunit.client.1.smithi096.stdout:3/514: dwrite d0/d9/d22/fa7 [0,4194304] 0 2024-03-24T10:46:14.170 INFO:tasks.workunit.client.1.smithi096.stdout:3/515: fsync d0/d13/d45/d7d/f94 0 2024-03-24T10:46:14.170 INFO:tasks.workunit.client.1.smithi096.stdout:3/516: write d0/d9/f8b [498906,122742] 0 2024-03-24T10:46:14.171 INFO:tasks.workunit.client.1.smithi096.stdout:9/448: creat d9/d15/d26/d29/d73/f91 x:0 0 0 2024-03-24T10:46:14.171 INFO:tasks.workunit.client.1.smithi096.stdout:9/449: fsync d9/d15/d26/d29/f36 0 2024-03-24T10:46:14.172 INFO:tasks.workunit.client.1.smithi096.stdout:7/689: symlink d0/d21/d59/le5 0 2024-03-24T10:46:14.172 INFO:tasks.workunit.client.1.smithi096.stdout:2/460: dwrite d5/d38/f3a [0,4194304] 0 2024-03-24T10:46:14.173 INFO:tasks.workunit.client.1.smithi096.stdout:2/461: dread - d5/d38/f76 zero size 2024-03-24T10:46:14.175 INFO:tasks.workunit.client.1.smithi096.stdout:5/493: dread d2/d7/d8/f12 [4194304,4194304] 0 2024-03-24T10:46:14.178 INFO:tasks.workunit.client.1.smithi096.stdout:8/542: rename d8/d11/d22/d4c/d65/f5b to d8/fb3 0 2024-03-24T10:46:14.183 INFO:tasks.workunit.client.1.smithi096.stdout:6/459: unlink d8/d12/d32/d38/f4b 0 2024-03-24T10:46:14.185 INFO:tasks.workunit.client.1.smithi096.stdout:5/494: write d2/d7/d8/df/f78 [1721005,40508] 0 2024-03-24T10:46:14.185 INFO:tasks.workunit.client.1.smithi096.stdout:5/495: creat d2/d29/d32/fa8 x:0 0 0 2024-03-24T10:46:14.188 INFO:tasks.workunit.client.1.smithi096.stdout:3/517: getdents d0/d13/d45/d7d/d39 0 2024-03-24T10:46:14.190 INFO:tasks.workunit.client.1.smithi096.stdout:9/450: unlink d9/db/d6d/d4f/l78 0 2024-03-24T10:46:14.194 INFO:tasks.workunit.client.1.smithi096.stdout:7/690: mkdir d0/db/d1c/d77/dab/de6 0 2024-03-24T10:46:14.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.196+0000 7ff397738700 1 -- 172.21.15.94:0/3215206272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff390066b10 msgr2=0x7ff3900ffa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:14.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.196+0000 7ff397738700 1 --2- 172.21.15.94:0/3215206272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff390066b10 0x7ff3900ffa30 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff38c004940 tx=0x7ff38c009e30 comp rx=0 tx=0).stop 2024-03-24T10:46:14.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.196+0000 7ff397738700 1 -- 172.21.15.94:0/3215206272 shutdown_connections 2024-03-24T10:46:14.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.196+0000 7ff397738700 1 --2- 172.21.15.94:0/3215206272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3900fff70 0x7ff390102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:14.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.196+0000 7ff397738700 1 --2- 172.21.15.94:0/3215206272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff390066b10 0x7ff3900ffa30 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:14.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.196+0000 7ff397738700 1 -- 172.21.15.94:0/3215206272 >> 172.21.15.94:0/3215206272 conn(0x7ff3900f7440 msgr2=0x7ff3900f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:46:14.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.196+0000 7ff397738700 1 -- 172.21.15.94:0/3215206272 shutdown_connections 2024-03-24T10:46:14.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.196+0000 7ff397738700 1 -- 172.21.15.94:0/3215206272 wait complete. 2024-03-24T10:46:14.198 INFO:tasks.workunit.client.1.smithi096.stdout:0/490: rmdir d0/d11/d20/d2b 39 2024-03-24T10:46:14.198 INFO:tasks.workunit.client.1.smithi096.stdout:0/491: fsync d0/d11/d24/f4c 0 2024-03-24T10:46:14.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.196+0000 7ff397738700 1 Processor -- start 2024-03-24T10:46:14.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.196+0000 7ff397738700 1 -- start start 2024-03-24T10:46:14.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.197+0000 7ff397738700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff390066b10 0x7ff3900fd410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:46:14.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.197+0000 7ff397738700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3900fff70 0x7ff3900fb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:46:14.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.197+0000 7ff397738700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3900fbec0 con 0x7ff3900fff70 2024-03-24T10:46:14.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.197+0000 7ff397738700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3900fc000 con 0x7ff390066b10 2024-03-24T10:46:14.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.197+0000 7ff394cd3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3900fff70 0x7ff3900fb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:46:14.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.197+0000 7ff394cd3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3900fff70 0x7ff3900fb930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54336/0 (socket says 172.21.15.94:54336) 2024-03-24T10:46:14.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.197+0000 7ff394cd3700 1 -- 172.21.15.94:0/3841449640 learned_addr learned my addr 172.21.15.94:0/3841449640 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:46:14.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.197+0000 7ff3954d4700 1 --2- 172.21.15.94:0/3841449640 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff390066b10 0x7ff3900fd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:46:14.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.197+0000 7ff394cd3700 1 -- 172.21.15.94:0/3841449640 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff390066b10 msgr2=0x7ff3900fd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:14.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.197+0000 7ff394cd3700 1 --2- 172.21.15.94:0/3841449640 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff390066b10 0x7ff3900fd410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:14.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.197+0000 7ff394cd3700 1 -- 172.21.15.94:0/3841449640 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff38c005040 con 0x7ff3900fff70 2024-03-24T10:46:14.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.197+0000 7ff394cd3700 1 --2- 172.21.15.94:0/3841449640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3900fff70 0x7ff3900fb930 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff38000eb40 tx=0x7ff38000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:46:14.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.197+0000 7ff385ffb700 1 -- 172.21.15.94:0/3841449640 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff380007cc0 con 0x7ff3900fff70 2024-03-24T10:46:14.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.197+0000 7ff397738700 1 -- 172.21.15.94:0/3841449640 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3900fc280 con 0x7ff3900fff70 2024-03-24T10:46:14.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.197+0000 7ff385ffb700 1 -- 172.21.15.94:0/3841449640 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff38000f040 con 0x7ff3900fff70 2024-03-24T10:46:14.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.197+0000 7ff385ffb700 1 -- 172.21.15.94:0/3841449640 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff38000ca10 con 0x7ff3900fff70 2024-03-24T10:46:14.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.198+0000 7ff397738700 1 -- 172.21.15.94:0/3841449640 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff390115a40 con 0x7ff3900fff70 2024-03-24T10:46:14.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.199+0000 7ff397738700 1 -- 172.21.15.94:0/3841449640 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff39005f5f0 con 0x7ff3900fff70 2024-03-24T10:46:14.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.199+0000 7ff385ffb700 1 -- 172.21.15.94:0/3841449640 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7ff38000cc80 con 0x7ff3900fff70 2024-03-24T10:46:14.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.199+0000 7ff385ffb700 1 --2- 172.21.15.94:0/3841449640 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7ff37c0692a0 0x7ff37c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:46:14.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.199+0000 7ff385ffb700 1 -- 172.21.15.94:0/3841449640 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7ff380086dd0 con 0x7ff3900fff70 2024-03-24T10:46:14.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.200+0000 7ff3954d4700 1 --2- 172.21.15.94:0/3841449640 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7ff37c0692a0 0x7ff37c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:46:14.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.200+0000 7ff3954d4700 1 --2- 172.21.15.94:0/3841449640 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7ff37c0692a0 0x7ff37c06b750 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff38c013740 tx=0x7ff38c009e30 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:46:14.203 INFO:tasks.workunit.client.1.smithi096.stdout:2/462: dwrite d5/dc/f21 [0,4194304] 0 2024-03-24T10:46:14.203 INFO:tasks.workunit.client.1.smithi096.stdout:6/460: dwrite f2 [0,4194304] 0 2024-03-24T10:46:14.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.204+0000 7ff385ffb700 1 -- 172.21.15.94:0/3841449640 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7ff38004e190 con 0x7ff3900fff70 2024-03-24T10:46:14.208 INFO:tasks.workunit.client.1.smithi096.stdout:3/518: mkdir d0/d13/d45/d52/d88/d9c/da9 0 2024-03-24T10:46:14.211 INFO:tasks.workunit.client.1.smithi096.stdout:9/451: unlink d9/d15/d26/d29/d73/d49/f4c 0 2024-03-24T10:46:14.214 INFO:tasks.workunit.client.1.smithi096.stdout:7/691: creat d0/da9/ddb/fe7 x:0 0 0 2024-03-24T10:46:14.216 INFO:tasks.workunit.client.1.smithi096.stdout:0/492: creat d0/d11/d16/fa5 x:0 0 0 2024-03-24T10:46:14.216 INFO:tasks.workunit.client.1.smithi096.stdout:0/493: write d0/f8e [225738,10478] 0 2024-03-24T10:46:14.222 INFO:tasks.workunit.client.1.smithi096.stdout:2/463: rename d5/dc/f34 to d5/fb0 0 2024-03-24T10:46:14.224 INFO:tasks.workunit.client.1.smithi096.stdout:6/461: mknod d8/d7b/c8f 0 2024-03-24T10:46:14.227 INFO:tasks.workunit.client.1.smithi096.stdout:5/496: rmdir d2/d29/d32/d33/d3c 39 2024-03-24T10:46:14.227 INFO:tasks.workunit.client.1.smithi096.stdout:5/497: write d2/d29/d32/d89/f9c [624378,47125] 0 2024-03-24T10:46:14.227 INFO:tasks.workunit.client.1.smithi096.stdout:5/498: dread - d2/d29/d32/d3f/d5f/f8f zero size 2024-03-24T10:46:14.229 INFO:tasks.workunit.client.1.smithi096.stdout:3/519: truncate d0/d13/f1f 567062 0 2024-03-24T10:46:14.233 INFO:tasks.workunit.client.1.smithi096.stdout:9/452: mknod d9/d60/c92 0 2024-03-24T10:46:14.234 INFO:tasks.workunit.client.1.smithi096.stdout:7/692: mkdir d0/d21/d3c/d68/d4b/de8 0 2024-03-24T10:46:14.235 INFO:tasks.workunit.client.1.smithi096.stdout:7/693: stat d0/db/d1c/d39/f4a 0 2024-03-24T10:46:14.237 INFO:tasks.workunit.client.1.smithi096.stdout:0/494: rename d0/d11/d24/d2c/f67 to d0/d11/d43/fa6 0 2024-03-24T10:46:14.238 INFO:tasks.workunit.client.1.smithi096.stdout:0/495: dread - d0/d11/d20/d2b/f6a zero size 2024-03-24T10:46:14.239 INFO:tasks.workunit.client.1.smithi096.stdout:2/464: creat d5/d7c/d88/d90/fb1 x:0 0 0 2024-03-24T10:46:14.242 INFO:tasks.workunit.client.1.smithi096.stdout:3/520: mkdir d0/d13/d45/d52/d88/daa 0 2024-03-24T10:46:14.246 INFO:tasks.workunit.client.1.smithi096.stdout:7/694: dread d0/f20 [0,4194304] 0 2024-03-24T10:46:14.246 INFO:tasks.workunit.client.1.smithi096.stdout:7/695: chown d0/db/d1c 7153043 1 2024-03-24T10:46:14.246 INFO:tasks.workunit.client.1.smithi096.stdout:0/496: dread d0/d11/d24/f30 [0,4194304] 0 2024-03-24T10:46:14.247 INFO:tasks.workunit.client.1.smithi096.stdout:7/696: read d0/db/d1c/f29 [3450457,79513] 0 2024-03-24T10:46:14.247 INFO:tasks.workunit.client.1.smithi096.stdout:6/462: rename d8/d10/d13/d63/l6c to d8/d12/d32/l90 0 2024-03-24T10:46:14.249 INFO:tasks.workunit.client.1.smithi096.stdout:2/465: mknod d5/dc/d12/cb2 0 2024-03-24T10:46:14.252 INFO:tasks.workunit.client.1.smithi096.stdout:5/499: unlink d2/d7/f4b 0 2024-03-24T10:46:14.254 INFO:tasks.workunit.client.1.smithi096.stdout:9/453: symlink d9/db/l93 0 2024-03-24T10:46:14.254 INFO:tasks.workunit.client.1.smithi096.stdout:9/454: write d9/db/f28 [1118582,125536] 0 2024-03-24T10:46:14.254 INFO:tasks.workunit.client.1.smithi096.stdout:9/455: creat d9/d7c/f94 x:0 0 0 2024-03-24T10:46:14.257 INFO:tasks.workunit.client.1.smithi096.stdout:7/697: link d0/f9e d0/db/d1c/d77/dab/fe9 0 2024-03-24T10:46:14.257 INFO:tasks.workunit.client.1.smithi096.stdout:4/439: sync 2024-03-24T10:46:14.257 INFO:tasks.workunit.client.1.smithi096.stdout:1/407: sync 2024-03-24T10:46:14.259 INFO:tasks.workunit.client.1.smithi096.stdout:3/521: rename d0/d13/d45/d47/f4e to d0/d13/d41/d7f/fab 0 2024-03-24T10:46:14.259 INFO:tasks.workunit.client.1.smithi096.stdout:3/522: creat d0/d13/d45/d7d/d71/fac x:0 0 0 2024-03-24T10:46:14.261 INFO:tasks.workunit.client.1.smithi096.stdout:6/463: mkdir d8/d10/d76/d91 0 2024-03-24T10:46:14.261 INFO:tasks.workunit.client.1.smithi096.stdout:6/464: creat d8/d12/d28/d36/d65/f92 x:0 0 0 2024-03-24T10:46:14.261 INFO:tasks.workunit.client.1.smithi096.stdout:6/465: chown d8/d12/d28/d36/d65/d71/l7a 671257 1 2024-03-24T10:46:14.261 INFO:tasks.workunit.client.1.smithi096.stdout:6/466: fsync d8/d12/d32/d38/d59/f5f 0 2024-03-24T10:46:14.261 INFO:tasks.workunit.client.1.smithi096.stdout:6/467: readlink d8/d12/d32/d38/l4c 0 2024-03-24T10:46:14.264 INFO:tasks.workunit.client.1.smithi096.stdout:2/466: dwrite d5/fa [4194304,4194304] 0 2024-03-24T10:46:14.264 INFO:tasks.workunit.client.1.smithi096.stdout:2/467: dread - d5/dc/d12/d23/f77 zero size 2024-03-24T10:46:14.264 INFO:tasks.workunit.client.1.smithi096.stdout:7/698: symlink d0/db/d1c/d77/dab/de6/lea 0 2024-03-24T10:46:14.267 INFO:tasks.workunit.client.1.smithi096.stdout:1/408: truncate d3/d73/f7a 582322 0 2024-03-24T10:46:14.267 INFO:tasks.workunit.client.1.smithi096.stdout:1/409: chown d3/d73 311587698 1 2024-03-24T10:46:14.272 INFO:tasks.workunit.client.1.smithi096.stdout:3/523: rename d0/d13/d41/d7f/fa3 to d0/fad 0 2024-03-24T10:46:14.272 INFO:tasks.workunit.client.1.smithi096.stdout:9/456: dwrite d9/d15/d26/d29/f42 [0,4194304] 0 2024-03-24T10:46:14.272 INFO:tasks.workunit.client.1.smithi096.stdout:9/457: creat d9/db/d6d/d4f/d75/d8b/d8e/f95 x:0 0 0 2024-03-24T10:46:14.274 INFO:tasks.workunit.client.1.smithi096.stdout:9/458: dread d9/db/d6d/d7f/f3f [0,4194304] 0 2024-03-24T10:46:14.276 INFO:tasks.workunit.client.1.smithi096.stdout:6/468: creat d8/d12/d61/f93 x:0 0 0 2024-03-24T10:46:14.276 INFO:tasks.workunit.client.1.smithi096.stdout:6/469: creat d8/d12/d28/d36/d65/f94 x:0 0 0 2024-03-24T10:46:14.282 INFO:tasks.workunit.client.1.smithi096.stdout:5/500: rmdir d2/d29 39 2024-03-24T10:46:14.282 INFO:tasks.workunit.client.1.smithi096.stdout:5/501: truncate d2/d7/d69/f87 867164 0 2024-03-24T10:46:14.288 INFO:tasks.workunit.client.1.smithi096.stdout:2/468: mknod d5/dc/d12/d23/cb3 0 2024-03-24T10:46:14.289 INFO:tasks.workunit.client.1.smithi096.stdout:7/699: mknod d0/d21/d3c/d68/ceb 0 2024-03-24T10:46:14.291 INFO:tasks.workunit.client.1.smithi096.stdout:1/410: mknod d3/d23/c8d 0 2024-03-24T10:46:14.291 INFO:tasks.workunit.client.1.smithi096.stdout:1/411: fsync d3/d23/d71/f7d 0 2024-03-24T10:46:14.291 INFO:tasks.workunit.client.1.smithi096.stdout:1/412: write d3/d36/f4d [447666,88725] 0 2024-03-24T10:46:14.292 INFO:tasks.workunit.client.1.smithi096.stdout:1/413: dread - d3/d23/d71/f8a zero size 2024-03-24T10:46:14.293 INFO:tasks.workunit.client.1.smithi096.stdout:0/497: rename d0/d11/d16/f54 to d0/d11/d24/d9d/fa7 0 2024-03-24T10:46:14.293 INFO:tasks.workunit.client.1.smithi096.stdout:0/498: truncate d0/f73 1634698 0 2024-03-24T10:46:14.293 INFO:tasks.workunit.client.1.smithi096.stdout:0/499: truncate d0/f72 350514 0 2024-03-24T10:46:14.294 INFO:tasks.workunit.client.1.smithi096.stdout:0/500: write d0/d11/d16/d21/d81/d7f/f5e [356750,22315] 0 2024-03-24T10:46:14.295 INFO:tasks.workunit.client.1.smithi096.stdout:3/524: rmdir d0/d13/d45/d7d/d39/d48/d8d 39 2024-03-24T10:46:14.301 INFO:tasks.workunit.client.1.smithi096.stdout:9/459: unlink d9/d15/c2a 0 2024-03-24T10:46:14.305 INFO:tasks.workunit.client.1.smithi096.stdout:6/470: link d8/d12/d28/f4e d8/d10/d76/f95 0 2024-03-24T10:46:14.306 INFO:tasks.workunit.client.1.smithi096.stdout:6/471: chown d8/d10/c67 0 1 2024-03-24T10:46:14.308 INFO:tasks.workunit.client.1.smithi096.stdout:2/469: mknod d5/dc/d12/d23/cb4 0 2024-03-24T10:46:14.309 INFO:tasks.workunit.client.1.smithi096.stdout:7/700: symlink d0/d21/d3c/d68/d4b/d62/lec 0 2024-03-24T10:46:14.314 INFO:tasks.workunit.client.1.smithi096.stdout:3/525: mknod d0/d13/d1e/da4/cae 0 2024-03-24T10:46:14.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:13 smithi096 ceph-mon[38097]: pgmap v250: 65 pgs: 65 active+clean; 2.6 GiB data, 9.2 GiB used, 527 GiB / 536 GiB avail; 195 MiB/s rd, 240 MiB/s wr, 349 op/s 2024-03-24T10:46:14.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:13 smithi096 ceph-mon[38097]: from='client.14666 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:46:14.330 INFO:tasks.workunit.client.1.smithi096.stdout:9/460: mknod d9/d15/d26/c96 0 2024-03-24T10:46:14.331 INFO:tasks.workunit.client.1.smithi096.stdout:6/472: creat d8/d10/d19/f96 x:0 0 0 2024-03-24T10:46:14.332 INFO:tasks.workunit.client.1.smithi096.stdout:6/473: write d8/f83 [353235,73977] 0 2024-03-24T10:46:14.333 INFO:tasks.workunit.client.1.smithi096.stdout:5/502: getdents d2/d7/d8/d18 0 2024-03-24T10:46:14.334 INFO:tasks.workunit.client.1.smithi096.stdout:2/470: truncate d5/dc/f58 406535 0 2024-03-24T10:46:14.335 INFO:tasks.workunit.client.1.smithi096.stdout:7/701: truncate d0/db/d1c/f29 2792694 0 2024-03-24T10:46:14.337 INFO:tasks.workunit.client.1.smithi096.stdout:3/526: rename d0/d9/d22/c9d to d0/d13/d45/d52/d88/d9c/caf 0 2024-03-24T10:46:14.337 INFO:tasks.workunit.client.1.smithi096.stdout:3/527: chown d0/d13/d45/d52/d88/d9c/da9 1124951523 1 2024-03-24T10:46:14.341 INFO:tasks.workunit.client.1.smithi096.stdout:5/503: write d2/d7/f8c [2184432,69948] 0 2024-03-24T10:46:14.344 INFO:tasks.workunit.client.1.smithi096.stdout:9/461: creat d9/d15/d26/f97 x:0 0 0 2024-03-24T10:46:14.345 INFO:tasks.workunit.client.1.smithi096.stdout:6/474: creat d8/d10/d76/f97 x:0 0 0 2024-03-24T10:46:14.345 INFO:tasks.workunit.client.1.smithi096.stdout:6/475: creat d8/d12/d32/d38/d59/f98 x:0 0 0 2024-03-24T10:46:14.345 INFO:tasks.workunit.client.1.smithi096.stdout:6/476: dread - d8/d12/d28/d36/d65/d6b/f75 zero size 2024-03-24T10:46:14.347 INFO:tasks.workunit.client.1.smithi096.stdout:2/471: mkdir d5/d79/db5 0 2024-03-24T10:46:14.347 INFO:tasks.workunit.client.1.smithi096.stdout:2/472: creat d5/de/d2a/d5e/fb6 x:0 0 0 2024-03-24T10:46:14.348 INFO:tasks.workunit.client.1.smithi096.stdout:3/528: rmdir d0/d13/d45/d52 39 2024-03-24T10:46:14.350 INFO:tasks.workunit.client.1.smithi096.stdout:7/702: dwrite d0/db/d26/fd5 [0,4194304] 0 2024-03-24T10:46:14.350 INFO:tasks.workunit.client.1.smithi096.stdout:5/504: rmdir d2/d29/d32/d3f 39 2024-03-24T10:46:14.361 INFO:tasks.workunit.client.1.smithi096.stdout:6/477: rmdir d8/d12/d28/d36/d65/d6b 39 2024-03-24T10:46:14.361 INFO:tasks.workunit.client.1.smithi096.stdout:6/478: write d8/d12/d32/d66/f73 [666276,76722] 0 2024-03-24T10:46:14.361 INFO:tasks.workunit.client.1.smithi096.stdout:6/479: chown d8/d10/d58 127323 1 2024-03-24T10:46:14.363 INFO:tasks.workunit.client.1.smithi096.stdout:2/473: creat d5/dc/fb7 x:0 0 0 2024-03-24T10:46:14.366 INFO:tasks.workunit.client.1.smithi096.stdout:7/703: rename d0/d13/c4e to d0/d21/ced 0 2024-03-24T10:46:14.367 INFO:tasks.workunit.client.1.smithi096.stdout:3/529: dwrite d0/d13/f57 [0,4194304] 0 2024-03-24T10:46:14.368 INFO:tasks.workunit.client.1.smithi096.stdout:3/530: write d0/d9/f5e [386272,835] 0 2024-03-24T10:46:14.368 INFO:tasks.workunit.client.1.smithi096.stdout:3/531: stat d0/d13/d77 0 2024-03-24T10:46:14.368 INFO:tasks.workunit.client.1.smithi096.stdout:3/532: fsync d0/f44 0 2024-03-24T10:46:14.371 INFO:tasks.workunit.client.1.smithi096.stdout:5/505: unlink d2/d29/d32/d33/d37/c64 0 2024-03-24T10:46:14.371 INFO:tasks.workunit.client.1.smithi096.stdout:5/506: write d2/d7/d69/f87 [1543967,14843] 0 2024-03-24T10:46:14.371 INFO:tasks.workunit.client.1.smithi096.stdout:5/507: write d2/d7/d8/df/f78 [1800395,116941] 0 2024-03-24T10:46:14.374 INFO:tasks.workunit.client.1.smithi096.stdout:9/462: creat d9/d15/d26/d29/d73/d43/d48/f98 x:0 0 0 2024-03-24T10:46:14.374 INFO:tasks.workunit.client.1.smithi096.stdout:9/463: write d9/d15/d26/d29/d73/f5a [341226,99765] 0 2024-03-24T10:46:14.377 INFO:tasks.workunit.client.1.smithi096.stdout:3/533: rename d0/d13/d45/d7d/d71/fac to d0/d13/d45/d7d/d39/d48/fb0 0 2024-03-24T10:46:14.377 INFO:tasks.workunit.client.1.smithi096.stdout:3/534: fdatasync d0/fad 0 2024-03-24T10:46:14.380 INFO:tasks.workunit.client.1.smithi096.stdout:9/464: creat d9/d15/d26/d29/d73/d43/f99 x:0 0 0 2024-03-24T10:46:14.383 INFO:tasks.workunit.client.1.smithi096.stdout:2/474: link d5/d38/c46 d5/d38/cb8 0 2024-03-24T10:46:14.383 INFO:tasks.workunit.client.1.smithi096.stdout:2/475: truncate d5/dc/f83 125350 0 2024-03-24T10:46:14.383 INFO:tasks.workunit.client.1.smithi096.stdout:2/476: getdents d5/d79/d9f/dad 0 2024-03-24T10:46:14.385 INFO:tasks.workunit.client.1.smithi096.stdout:5/508: link d2/d29/c77 d2/d7/ca9 0 2024-03-24T10:46:14.389 INFO:tasks.workunit.client.1.smithi096.stdout:9/465: write d9/db/d6d/d7f/f3f [55201,30096] 0 2024-03-24T10:46:14.389 INFO:tasks.workunit.client.1.smithi096.stdout:9/466: dread - d9/d15/d26/d29/d73/f91 zero size 2024-03-24T10:46:14.392 INFO:tasks.workunit.client.1.smithi096.stdout:5/509: rename d2/d7/c16 to d2/caa 0 2024-03-24T10:46:14.392 INFO:tasks.workunit.client.1.smithi096.stdout:9/467: creat d9/d7c/f9a x:0 0 0 2024-03-24T10:46:14.392 INFO:tasks.workunit.client.1.smithi096.stdout:7/704: dwrite d0/db/d1c/d39/f4a [4194304,4194304] 0 2024-03-24T10:46:14.394 INFO:tasks.workunit.client.1.smithi096.stdout:5/510: mkdir d2/d29/d32/d33/d37/dab 0 2024-03-24T10:46:14.398 INFO:tasks.workunit.client.1.smithi096.stdout:7/705: symlink d0/da9/ddb/lee 0 2024-03-24T10:46:14.398 INFO:tasks.workunit.client.1.smithi096.stdout:7/706: write d0/d13/d32/d45/f63 [531639,20587] 0 2024-03-24T10:46:14.398 INFO:tasks.workunit.client.1.smithi096.stdout:7/707: write d0/db/f75 [1299265,56557] 0 2024-03-24T10:46:14.399 INFO:tasks.workunit.client.1.smithi096.stdout:7/708: chown d0/d21/f8b 56 1 2024-03-24T10:46:14.399 INFO:tasks.workunit.client.1.smithi096.stdout:5/511: unlink d2/f43 0 2024-03-24T10:46:14.399 INFO:tasks.workunit.client.1.smithi096.stdout:6/480: dwrite d8/d10/f8a [0,4194304] 0 2024-03-24T10:46:14.401 INFO:tasks.workunit.client.1.smithi096.stdout:9/468: rename d9/d15/d26/l62 to d9/db/l9b 0 2024-03-24T10:46:14.402 INFO:tasks.workunit.client.1.smithi096.stdout:7/709: unlink d0/d21/d59/lc9 0 2024-03-24T10:46:14.403 INFO:tasks.workunit.client.1.smithi096.stdout:3/535: dwrite d0/d13/f83 [0,4194304] 0 2024-03-24T10:46:14.403 INFO:tasks.workunit.client.1.smithi096.stdout:2/477: dwrite d5/de/d2a/f32 [0,4194304] 0 2024-03-24T10:46:14.407 INFO:tasks.workunit.client.1.smithi096.stdout:6/481: mknod d8/d10/d76/c99 0 2024-03-24T10:46:14.409 INFO:tasks.workunit.client.1.smithi096.stdout:5/512: rename d2/d29/d32/d3f/c46 to d2/d29/d32/cac 0 2024-03-24T10:46:14.410 INFO:tasks.workunit.client.1.smithi096.stdout:5/513: write d2/d29/d32/d33/f63 [362188,4721] 0 2024-03-24T10:46:14.411 INFO:tasks.workunit.client.1.smithi096.stdout:2/478: write f2 [5887866,96120] 0 2024-03-24T10:46:14.421 INFO:tasks.workunit.client.1.smithi096.stdout:3/536: unlink d0/d13/d45/d47/c97 0 2024-03-24T10:46:14.422 INFO:tasks.workunit.client.1.smithi096.stdout:9/469: dread d9/f3a [0,4194304] 0 2024-03-24T10:46:14.422 INFO:tasks.workunit.client.1.smithi096.stdout:9/470: write d9/db/d6d/d4f/f7b [606833,117310] 0 2024-03-24T10:46:14.422 INFO:tasks.workunit.client.1.smithi096.stdout:9/471: creat d9/d7c/f9c x:0 0 0 2024-03-24T10:46:14.422 INFO:tasks.workunit.client.1.smithi096.stdout:9/472: creat d9/d15/d26/d29/d73/d43/d48/f9d x:0 0 0 2024-03-24T10:46:14.426 INFO:tasks.workunit.client.1.smithi096.stdout:5/514: dwrite d2/d7/d8/d18/f72 [0,4194304] 0 2024-03-24T10:46:14.426 INFO:tasks.workunit.client.1.smithi096.stdout:6/482: getdents d8/d10/d58 0 2024-03-24T10:46:14.429 INFO:tasks.workunit.client.1.smithi096.stdout:6/483: dread d8/d12/f2d [0,4194304] 0 2024-03-24T10:46:14.437 INFO:tasks.workunit.client.1.smithi096.stdout:2/479: unlink d5/dc/d12/c52 0 2024-03-24T10:46:14.441 INFO:tasks.workunit.client.1.smithi096.stdout:7/710: mknod d0/db/d26/dc3/cef 0 2024-03-24T10:46:14.442 INFO:tasks.workunit.client.1.smithi096.stdout:3/537: mkdir d0/d13/d45/d52/d88/d9c/db1 0 2024-03-24T10:46:14.446 INFO:tasks.workunit.client.1.smithi096.stdout:6/484: dwrite d8/d10/d19/f1b [0,4194304] 0 2024-03-24T10:46:14.458 INFO:tasks.workunit.client.1.smithi096.stdout:7/711: getdents d0/d13 0 2024-03-24T10:46:14.464 INFO:tasks.workunit.client.1.smithi096.stdout:5/515: dread d2/d7/f4a [0,4194304] 0 2024-03-24T10:46:14.464 INFO:tasks.workunit.client.1.smithi096.stdout:3/538: rename d0/d9/d22/f29 to d0/fb2 0 2024-03-24T10:46:14.469 INFO:tasks.workunit.client.1.smithi096.stdout:6/485: dwrite d8/f62 [0,4194304] 0 2024-03-24T10:46:14.469 INFO:tasks.workunit.client.1.smithi096.stdout:2/480: dwrite d5/dc/f21 [0,4194304] 0 2024-03-24T10:46:14.469 INFO:tasks.workunit.client.1.smithi096.stdout:2/481: chown d5/de/d2a/d5e 13582 1 2024-03-24T10:46:14.477 INFO:tasks.workunit.client.1.smithi096.stdout:6/486: read d8/d10/f3d [1234573,94499] 0 2024-03-24T10:46:14.477 INFO:tasks.workunit.client.1.smithi096.stdout:6/487: getdents d8/d10/d58 0 2024-03-24T10:46:14.477 INFO:tasks.workunit.client.1.smithi096.stdout:7/712: dwrite d0/d21/d59/d87/fe0 [0,4194304] 0 2024-03-24T10:46:14.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:13 smithi094 ceph-mon[29446]: pgmap v250: 65 pgs: 65 active+clean; 2.6 GiB data, 9.2 GiB used, 527 GiB / 536 GiB avail; 195 MiB/s rd, 240 MiB/s wr, 349 op/s 2024-03-24T10:46:14.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:13 smithi094 ceph-mon[29446]: from='client.14666 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:46:14.505 INFO:tasks.workunit.client.1.smithi096.stdout:9/473: rmdir d9/d15 39 2024-03-24T10:46:14.505 INFO:tasks.workunit.client.1.smithi096.stdout:9/474: fdatasync d9/db/d6d/d7f/f88 0 2024-03-24T10:46:14.505 INFO:tasks.workunit.client.1.smithi096.stdout:6/488: dwrite f2 [0,4194304] 0 2024-03-24T10:46:14.509 INFO:tasks.workunit.client.1.smithi096.stdout:5/516: chown d2/d7/f86 49 1 2024-03-24T10:46:14.512 INFO:tasks.workunit.client.1.smithi096.stdout:3/539: symlink d0/d13/d45/d52/d88/d9c/da9/lb3 0 2024-03-24T10:46:14.512 INFO:tasks.workunit.client.1.smithi096.stdout:3/540: write d0/d13/d45/d7d/d39/d48/f86 [956828,69685] 0 2024-03-24T10:46:14.520 INFO:tasks.workunit.client.1.smithi096.stdout:3/541: dread d0/d9/f36 [0,4194304] 0 2024-03-24T10:46:14.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.523+0000 7ff397738700 1 -- 172.21.15.94:0/3841449640 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff39004bed0 con 0x7ff3900fff70 2024-03-24T10:46:14.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.524+0000 7ff385ffb700 1 -- 172.21.15.94:0/3841449640 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7ff380053150 con 0x7ff3900fff70 2024-03-24T10:46:14.525 INFO:tasks.workunit.client.1.smithi096.stdout:6/489: rename d8/d12/d32/f80 to d8/d12/d32/f9a 0 2024-03-24T10:46:14.525 INFO:tasks.workunit.client.1.smithi096.stdout:6/490: write d8/d12/d28/d36/d65/f92 [58878,119590] 0 2024-03-24T10:46:14.525 INFO:tasks.workunit.client.1.smithi096.stdout:6/491: chown d8/d10/l35 243 1 2024-03-24T10:46:14.525 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-03-24T10:46:14.525 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": { 2024-03-24T10:46:14.525 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-24T10:46:14.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:46:14.526 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": { 2024-03-24T10:46:14.526 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-24T10:46:14.526 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:46:14.526 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": { 2024-03-24T10:46:14.526 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-24T10:46:14.526 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:46:14.526 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-03-24T10:46:14.526 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-24T10:46:14.526 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:46:14.526 INFO:teuthology.orchestra.run.smithi094.stdout: "overall": { 2024-03-24T10:46:14.526 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 14 2024-03-24T10:46:14.526 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-03-24T10:46:14.526 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-03-24T10:46:14.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.526+0000 7ff397738700 1 -- 172.21.15.94:0/3841449640 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7ff37c0692a0 msgr2=0x7ff37c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:14.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.526+0000 7ff397738700 1 --2- 172.21.15.94:0/3841449640 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7ff37c0692a0 0x7ff37c06b750 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff38c013740 tx=0x7ff38c009e30 comp rx=0 tx=0).stop 2024-03-24T10:46:14.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.526+0000 7ff397738700 1 -- 172.21.15.94:0/3841449640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3900fff70 msgr2=0x7ff3900fb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:14.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.526+0000 7ff397738700 1 --2- 172.21.15.94:0/3841449640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3900fff70 0x7ff3900fb930 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff38000eb40 tx=0x7ff38000b040 comp rx=0 tx=0).stop 2024-03-24T10:46:14.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.526+0000 7ff397738700 1 -- 172.21.15.94:0/3841449640 shutdown_connections 2024-03-24T10:46:14.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.526+0000 7ff397738700 1 --2- 172.21.15.94:0/3841449640 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7ff37c0692a0 0x7ff37c06b750 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:14.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.526+0000 7ff397738700 1 --2- 172.21.15.94:0/3841449640 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff390066b10 0x7ff3900fd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:14.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.526+0000 7ff397738700 1 --2- 172.21.15.94:0/3841449640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3900fff70 0x7ff3900fb930 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:14.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.526+0000 7ff397738700 1 -- 172.21.15.94:0/3841449640 >> 172.21.15.94:0/3841449640 conn(0x7ff3900f7440 msgr2=0x7ff3900f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:46:14.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.526+0000 7ff397738700 1 -- 172.21.15.94:0/3841449640 shutdown_connections 2024-03-24T10:46:14.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.526+0000 7ff397738700 1 -- 172.21.15.94:0/3841449640 wait complete. 2024-03-24T10:46:14.528 INFO:tasks.workunit.client.1.smithi096.stdout:3/542: dread d0/d9/f1a [0,4194304] 0 2024-03-24T10:46:14.528 INFO:tasks.workunit.client.1.smithi096.stdout:3/543: creat d0/d13/d77/fb4 x:0 0 0 2024-03-24T10:46:14.528 INFO:tasks.workunit.client.1.smithi096.stdout:3/544: read - d0/d13/d45/d52/d88/d9c/fa6 zero size 2024-03-24T10:46:14.528 INFO:tasks.workunit.client.1.smithi096.stdout:3/545: chown d0/c3f 6739 1 2024-03-24T10:46:14.540 INFO:tasks.workunit.client.1.smithi096.stdout:5/517: mkdir d2/d29/d32/d33/d8d/d94/dad 0 2024-03-24T10:46:14.541 INFO:tasks.workunit.client.1.smithi096.stdout:5/518: read d2/d29/d32/d33/f63 [139613,5936] 0 2024-03-24T10:46:14.541 INFO:tasks.workunit.client.1.smithi096.stdout:5/519: fsync d2/d7/d8/df/f78 0 2024-03-24T10:46:14.541 INFO:tasks.workunit.client.1.smithi096.stdout:2/482: rmdir d5/de/d2a 39 2024-03-24T10:46:14.541 INFO:tasks.workunit.client.1.smithi096.stdout:2/483: write d5/de/f13 [1624272,33597] 0 2024-03-24T10:46:14.549 INFO:tasks.workunit.client.1.smithi096.stdout:6/492: dwrite d8/d10/d19/f48 [4194304,4194304] 0 2024-03-24T10:46:14.549 INFO:tasks.workunit.client.1.smithi096.stdout:6/493: creat d8/d12/d28/f9b x:0 0 0 2024-03-24T10:46:14.549 INFO:tasks.workunit.client.1.smithi096.stdout:6/494: chown d8/d12/l43 3673454 1 2024-03-24T10:46:14.549 INFO:tasks.workunit.client.1.smithi096.stdout:6/495: dread - d8/d12/d61/f93 zero size 2024-03-24T10:46:14.552 INFO:tasks.workunit.client.1.smithi096.stdout:3/546: getdents d0/d13 0 2024-03-24T10:46:14.552 INFO:tasks.workunit.client.1.smithi096.stdout:3/547: write d0/d13/d45/d47/f8e [518903,105535] 0 2024-03-24T10:46:14.552 INFO:tasks.workunit.client.1.smithi096.stdout:5/520: dread d2/f8b [0,4194304] 0 2024-03-24T10:46:14.553 INFO:tasks.workunit.client.1.smithi096.stdout:5/521: write d2/d7/d8/df/d14/f85 [218834,75171] 0 2024-03-24T10:46:14.557 INFO:tasks.workunit.client.1.smithi096.stdout:6/496: write d8/f31 [4782039,110646] 0 2024-03-24T10:46:14.558 INFO:tasks.workunit.client.1.smithi096.stdout:7/713: rename d0/d21 to d0/d13/df0 0 2024-03-24T10:46:14.564 INFO:tasks.workunit.client.1.smithi096.stdout:3/548: mknod d0/d13/cb5 0 2024-03-24T10:46:14.567 INFO:tasks.workunit.client.1.smithi096.stdout:5/522: mknod d2/d7/d8/df/d8a/cae 0 2024-03-24T10:46:14.571 INFO:tasks.workunit.client.1.smithi096.stdout:2/484: write d5/dc/f1c [1323893,80897] 0 2024-03-24T10:46:14.571 INFO:tasks.workunit.client.1.smithi096.stdout:2/485: chown d5/dc/f5b 2782 1 2024-03-24T10:46:14.571 INFO:tasks.workunit.client.1.smithi096.stdout:2/486: creat d5/fb9 x:0 0 0 2024-03-24T10:46:14.572 INFO:tasks.workunit.client.1.smithi096.stdout:6/497: symlink d8/d12/d32/d66/l9c 0 2024-03-24T10:46:14.578 INFO:tasks.workunit.client.1.smithi096.stdout:7/714: creat d0/d13/df0/d3c/d68/d4b/de8/ff1 x:0 0 0 2024-03-24T10:46:14.579 INFO:tasks.workunit.client.1.smithi096.stdout:2/487: write d5/fb0 [751348,121960] 0 2024-03-24T10:46:14.579 INFO:tasks.workunit.client.1.smithi096.stdout:2/488: chown d5/d7c/d88/fac 152567 1 2024-03-24T10:46:14.581 INFO:tasks.workunit.client.1.smithi096.stdout:3/549: mkdir d0/d13/d45/d7d/d71/db6 0 2024-03-24T10:46:14.582 INFO:tasks.workunit.client.1.smithi096.stdout:5/523: creat d2/d29/d32/d3f/d84/faf x:0 0 0 2024-03-24T10:46:14.595 INFO:tasks.workunit.client.1.smithi096.stdout:6/498: rmdir d8/d10/d13/d63 39 2024-03-24T10:46:14.595 INFO:tasks.workunit.client.1.smithi096.stdout:2/489: dwrite d5/dc/d12/d23/f2c [0,4194304] 0 2024-03-24T10:46:14.596 INFO:tasks.workunit.client.1.smithi096.stdout:9/475: rename d9/d15/d26/d29/d73/d43/f44 to d9/d15/d26/d29/d73/d49/d86/f9e 0 2024-03-24T10:46:14.599 INFO:tasks.workunit.client.1.smithi096.stdout:9/476: dread d9/db/d6d/d7f/f3f [0,4194304] 0 2024-03-24T10:46:14.599 INFO:tasks.workunit.client.1.smithi096.stdout:9/477: read - d9/d15/d26/d29/d73/d43/f99 zero size 2024-03-24T10:46:14.602 INFO:tasks.workunit.client.1.smithi096.stdout:2/490: read d5/de/d2a/f32 [1338584,23153] 0 2024-03-24T10:46:14.602 INFO:tasks.workunit.client.1.smithi096.stdout:2/491: truncate d5/de/d2a/d5e/fa9 433184 0 2024-03-24T10:46:14.602 INFO:tasks.workunit.client.1.smithi096.stdout:2/492: dread - d5/dc/d6b/d72/f9c zero size 2024-03-24T10:46:14.603 INFO:tasks.workunit.client.1.smithi096.stdout:2/493: read - d5/dc/d12/d23/f77 zero size 2024-03-24T10:46:14.604 INFO:tasks.workunit.client.1.smithi096.stdout:7/715: truncate d0/db/f75 1225487 0 2024-03-24T10:46:14.607 INFO:tasks.workunit.client.1.smithi096.stdout:3/550: link d0/fad d0/d13/d41/d7f/fb7 0 2024-03-24T10:46:14.614 INFO:tasks.workunit.client.1.smithi096.stdout:7/716: read d0/d13/d32/f3b [531313,35585] 0 2024-03-24T10:46:14.614 INFO:tasks.workunit.client.1.smithi096.stdout:7/717: read d0/d13/df0/d3c/d68/f4d [1929015,116375] 0 2024-03-24T10:46:14.614 INFO:tasks.workunit.client.1.smithi096.stdout:7/718: write d0/d13/df0/d3c/d68/d4b/fa4 [4619689,104307] 0 2024-03-24T10:46:14.616 INFO:tasks.workunit.client.1.smithi096.stdout:3/551: unlink d0/d13/d45/d7d/d71/f96 0 2024-03-24T10:46:14.620 INFO:tasks.workunit.client.1.smithi096.stdout:5/524: rename d2/d7/d8/df/d14/d96/l9a to d2/d7/d8/lb0 0 2024-03-24T10:46:14.620 INFO:tasks.workunit.client.1.smithi096.stdout:5/525: stat d2/d29 0 2024-03-24T10:46:14.620 INFO:tasks.workunit.client.1.smithi096.stdout:2/494: dwrite d5/d7c/d88/f69 [4194304,4194304] 0 2024-03-24T10:46:14.620 INFO:tasks.workunit.client.1.smithi096.stdout:2/495: fsync d5/dc/d12/f16 0 2024-03-24T10:46:14.621 INFO:tasks.workunit.client.1.smithi096.stdout:9/478: creat d9/d65/f9f x:0 0 0 2024-03-24T10:46:14.621 INFO:tasks.workunit.client.1.smithi096.stdout:9/479: dread - d9/d15/d26/d29/d73/f91 zero size 2024-03-24T10:46:14.623 INFO:tasks.workunit.client.1.smithi096.stdout:7/719: getdents d0/db 0 2024-03-24T10:46:14.625 INFO:tasks.workunit.client.1.smithi096.stdout:3/552: symlink d0/d13/d45/d52/d88/lb8 0 2024-03-24T10:46:14.626 INFO:tasks.workunit.client.1.smithi096.stdout:3/553: chown d0/d9/d22/fa8 1 1 2024-03-24T10:46:14.626 INFO:tasks.workunit.client.1.smithi096.stdout:3/554: creat d0/d13/d45/d47/fb9 x:0 0 0 2024-03-24T10:46:14.626 INFO:tasks.workunit.client.1.smithi096.stdout:3/555: write d0/d13/f21 [2244856,90141] 0 2024-03-24T10:46:14.626 INFO:tasks.workunit.client.1.smithi096.stdout:3/556: readlink d0/d13/d41/l69 0 2024-03-24T10:46:14.626 INFO:tasks.workunit.client.1.smithi096.stdout:3/557: write d0/d9/f1a [4962641,76003] 0 2024-03-24T10:46:14.626 INFO:tasks.workunit.client.1.smithi096.stdout:3/558: chown d0/d13/d45/d52/f53 27 1 2024-03-24T10:46:14.628 INFO:tasks.workunit.client.1.smithi096.stdout:6/499: rename d8/d12/d32/l41 to d8/d12/d28/d36/d65/l9d 0 2024-03-24T10:46:14.628 INFO:tasks.workunit.client.1.smithi096.stdout:6/500: write f7 [6151297,122931] 0 2024-03-24T10:46:14.628 INFO:tasks.workunit.client.1.smithi096.stdout:3/559: write d0/d13/d45/d47/f64 [846523,105861] 0 2024-03-24T10:46:14.629 INFO:tasks.workunit.client.1.smithi096.stdout:5/526: creat d2/d29/d32/d33/d37/dab/fb1 x:0 0 0 2024-03-24T10:46:14.629 INFO:tasks.workunit.client.1.smithi096.stdout:5/527: truncate d2/d7/d8/df/f78 4352397 0 2024-03-24T10:46:14.630 INFO:tasks.workunit.client.1.smithi096.stdout:2/496: creat d5/d79/d9f/fba x:0 0 0 2024-03-24T10:46:14.630 INFO:tasks.workunit.client.1.smithi096.stdout:2/497: write d5/dc/d12/d23/f98 [355890,5410] 0 2024-03-24T10:46:14.634 INFO:tasks.workunit.client.1.smithi096.stdout:7/720: getdents d0/d13/df0/d59/d87 0 2024-03-24T10:46:14.636 INFO:tasks.workunit.client.1.smithi096.stdout:6/501: write d8/d10/d19/f34 [2292326,82472] 0 2024-03-24T10:46:14.637 INFO:tasks.workunit.client.1.smithi096.stdout:3/560: rmdir d0/d13/d45 39 2024-03-24T10:46:14.638 INFO:tasks.workunit.client.1.smithi096.stdout:7/721: unlink d0/d13/df0/l2a 0 2024-03-24T10:46:14.638 INFO:tasks.workunit.client.1.smithi096.stdout:7/722: creat d0/db/d1c/d77/ff2 x:0 0 0 2024-03-24T10:46:14.640 INFO:tasks.workunit.client.1.smithi096.stdout:6/502: mkdir d8/d10/d9e 0 2024-03-24T10:46:14.640 INFO:tasks.workunit.client.1.smithi096.stdout:6/503: dread - d8/d12/d28/d36/d65/f94 zero size 2024-03-24T10:46:14.641 INFO:tasks.workunit.client.1.smithi096.stdout:9/480: rename d9/f1a to d9/d15/fa0 0 2024-03-24T10:46:14.643 INFO:tasks.workunit.client.1.smithi096.stdout:3/561: mknod d0/d13/d45/d7d/d71/db6/cba 0 2024-03-24T10:46:14.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.644+0000 7f6d17fc2700 1 -- 172.21.15.94:0/2089008886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d10066b70 msgr2=0x7f6d100ffa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:14.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.644+0000 7f6d17fc2700 1 --2- 172.21.15.94:0/2089008886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d10066b70 0x7f6d100ffa50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c004940 tx=0x7f6d0c009e30 comp rx=0 tx=0).stop 2024-03-24T10:46:14.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.645+0000 7f6d17fc2700 1 -- 172.21.15.94:0/2089008886 shutdown_connections 2024-03-24T10:46:14.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.645+0000 7f6d17fc2700 1 --2- 172.21.15.94:0/2089008886 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6d100fff90 0x7f6d101023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:14.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.645+0000 7f6d17fc2700 1 --2- 172.21.15.94:0/2089008886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d10066b70 0x7f6d100ffa50 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:14.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.645+0000 7f6d17fc2700 1 -- 172.21.15.94:0/2089008886 >> 172.21.15.94:0/2089008886 conn(0x7f6d100f74e0 msgr2=0x7f6d100f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:46:14.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.645+0000 7f6d17fc2700 1 -- 172.21.15.94:0/2089008886 shutdown_connections 2024-03-24T10:46:14.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.645+0000 7f6d17fc2700 1 -- 172.21.15.94:0/2089008886 wait complete. 2024-03-24T10:46:14.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.646+0000 7f6d17fc2700 1 Processor -- start 2024-03-24T10:46:14.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.647+0000 7f6d17fc2700 1 -- start start 2024-03-24T10:46:14.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.647+0000 7f6d17fc2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d10066b70 0x7f6d1010dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:46:14.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.647+0000 7f6d17fc2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6d100fff90 0x7f6d1010e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:46:14.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.647+0000 7f6d17fc2700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1010e6b0 con 0x7f6d10066b70 2024-03-24T10:46:14.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.647+0000 7f6d17fc2700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1010ee00 con 0x7f6d100fff90 2024-03-24T10:46:14.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.647+0000 7f6d15d5e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d10066b70 0x7f6d1010dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:46:14.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.647+0000 7f6d15d5e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d10066b70 0x7f6d1010dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54368/0 (socket says 172.21.15.94:54368) 2024-03-24T10:46:14.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.647+0000 7f6d15d5e700 1 -- 172.21.15.94:0/86409610 learned_addr learned my addr 172.21.15.94:0/86409610 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:46:14.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.647+0000 7f6d15d5e700 1 -- 172.21.15.94:0/86409610 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6d100fff90 msgr2=0x7f6d1010e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:14.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.648+0000 7f6d1555d700 1 --2- 172.21.15.94:0/86409610 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6d100fff90 0x7f6d1010e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:46:14.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.648+0000 7f6d15d5e700 1 --2- 172.21.15.94:0/86409610 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6d100fff90 0x7f6d1010e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:14.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.648+0000 7f6d15d5e700 1 -- 172.21.15.94:0/86409610 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d0c005040 con 0x7f6d10066b70 2024-03-24T10:46:14.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.648+0000 7f6d1555d700 1 --2- 172.21.15.94:0/86409610 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6d100fff90 0x7f6d1010e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T10:46:14.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.648+0000 7f6d15d5e700 1 --2- 172.21.15.94:0/86409610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d10066b70 0x7f6d1010dae0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c00b970 tx=0x7f6d0c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:46:14.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.648+0000 7f6d067fc700 1 -- 172.21.15.94:0/86409610 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d0c00e040 con 0x7f6d10066b70 2024-03-24T10:46:14.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.648+0000 7f6d17fc2700 1 -- 172.21.15.94:0/86409610 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d1010f080 con 0x7f6d10066b70 2024-03-24T10:46:14.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.648+0000 7f6d067fc700 1 -- 172.21.15.94:0/86409610 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6d0c00bcc0 con 0x7f6d10066b70 2024-03-24T10:46:14.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.648+0000 7f6d17fc2700 1 -- 172.21.15.94:0/86409610 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d1010f640 con 0x7f6d10066b70 2024-03-24T10:46:14.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.648+0000 7f6d067fc700 1 -- 172.21.15.94:0/86409610 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d0c013660 con 0x7f6d10066b70 2024-03-24T10:46:14.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.649+0000 7f6d17fc2700 1 -- 172.21.15.94:0/86409610 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d1005f5f0 con 0x7f6d10066b70 2024-03-24T10:46:14.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.650+0000 7f6d067fc700 1 -- 172.21.15.94:0/86409610 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f6d0c01ca00 con 0x7f6d10066b70 2024-03-24T10:46:14.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.650+0000 7f6d067fc700 1 --2- 172.21.15.94:0/86409610 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f6cfc0692a0 0x7f6cfc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:46:14.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.650+0000 7f6d067fc700 1 -- 172.21.15.94:0/86409610 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f6d0c087f50 con 0x7f6d10066b70 2024-03-24T10:46:14.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.650+0000 7f6d1555d700 1 --2- 172.21.15.94:0/86409610 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f6cfc0692a0 0x7f6cfc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:46:14.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.651+0000 7f6d1555d700 1 --2- 172.21.15.94:0/86409610 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f6cfc0692a0 0x7f6cfc06b750 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6d00006fd0 tx=0x7f6d00008040 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:46:14.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.655+0000 7f6d067fc700 1 -- 172.21.15.94:0/86409610 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f6d0c0ba190 con 0x7f6d10066b70 2024-03-24T10:46:14.657 INFO:tasks.workunit.client.1.smithi096.stdout:6/504: dwrite d8/d10/d13/f55 [0,4194304] 0 2024-03-24T10:46:14.660 INFO:tasks.workunit.client.1.smithi096.stdout:7/723: mknod d0/db/d26/d48/da8/cf3 0 2024-03-24T10:46:14.663 INFO:tasks.workunit.client.1.smithi096.stdout:5/528: rename d2/d7/d8/d18/f47 to d2/d29/d32/d33/d3c/d3d/fb2 0 2024-03-24T10:46:14.666 INFO:tasks.workunit.client.1.smithi096.stdout:9/481: dread d9/d15/d26/d29/d73/d43/f6c [0,4194304] 0 2024-03-24T10:46:14.666 INFO:tasks.workunit.client.1.smithi096.stdout:8/543: sync 2024-03-24T10:46:14.666 INFO:tasks.workunit.client.1.smithi096.stdout:8/544: chown d8/d72/f83 4430248 1 2024-03-24T10:46:14.666 INFO:tasks.workunit.client.1.smithi096.stdout:9/482: creat d9/db/d6d/d4f/d75/d8b/d8e/fa1 x:0 0 0 2024-03-24T10:46:14.666 INFO:tasks.workunit.client.1.smithi096.stdout:9/483: truncate d9/db/f28 1666969 0 2024-03-24T10:46:14.666 INFO:tasks.workunit.client.1.smithi096.stdout:9/484: read - d9/db/d6d/d4f/f54 zero size 2024-03-24T10:46:14.671 INFO:tasks.workunit.client.1.smithi096.stdout:6/505: symlink d8/d12/d32/d38/d59/l9f 0 2024-03-24T10:46:14.671 INFO:tasks.workunit.client.1.smithi096.stdout:5/529: dread d2/d29/d32/f40 [4194304,4194304] 0 2024-03-24T10:46:14.673 INFO:tasks.workunit.client.1.smithi096.stdout:7/724: creat d0/d13/df0/d3c/d68/d7e/ff4 x:0 0 0 2024-03-24T10:46:14.673 INFO:tasks.workunit.client.1.smithi096.stdout:6/506: dread d8/d10/d19/f1b [4194304,4194304] 0 2024-03-24T10:46:14.675 INFO:tasks.workunit.client.1.smithi096.stdout:7/725: dread d0/db/d1c/d77/dab/fba [0,4194304] 0 2024-03-24T10:46:14.676 INFO:tasks.workunit.client.1.smithi096.stdout:2/498: rename d5/d38/c55 to d5/d93/cbb 0 2024-03-24T10:46:14.681 INFO:tasks.workunit.client.1.smithi096.stdout:6/507: dread d8/d10/d19/f34 [0,4194304] 0 2024-03-24T10:46:14.683 INFO:tasks.workunit.client.1.smithi096.stdout:7/726: dread d0/db/d1c/d39/f4a [0,4194304] 0 2024-03-24T10:46:14.685 INFO:tasks.workunit.client.1.smithi096.stdout:8/545: mkdir d8/d11/d22/d4c/db4 0 2024-03-24T10:46:14.687 INFO:tasks.workunit.client.1.smithi096.stdout:9/485: stat d9/f13 0 2024-03-24T10:46:14.687 INFO:tasks.workunit.client.1.smithi096.stdout:9/486: chown d9/db/d6d/c81 4120121 1 2024-03-24T10:46:14.688 INFO:tasks.workunit.client.1.smithi096.stdout:2/499: symlink d5/d7c/d8f/lbc 0 2024-03-24T10:46:14.688 INFO:tasks.workunit.client.1.smithi096.stdout:2/500: write d5/de/d2a/d5e/fb6 [698056,67524] 0 2024-03-24T10:46:14.688 INFO:tasks.workunit.client.1.smithi096.stdout:5/530: dwrite d2/d7/d8/df/f2d [0,4194304] 0 2024-03-24T10:46:14.689 INFO:tasks.workunit.client.1.smithi096.stdout:5/531: read d2/d29/d32/f40 [8293525,105898] 0 2024-03-24T10:46:14.689 INFO:tasks.workunit.client.1.smithi096.stdout:5/532: chown d2/d7/d8/df/d14/f23 40911 1 2024-03-24T10:46:14.691 INFO:tasks.workunit.client.1.smithi096.stdout:3/562: rename d0/d13/d45/d7d/d39/d48/d85/l9a to d0/d13/d77/lbb 0 2024-03-24T10:46:14.692 INFO:tasks.workunit.client.1.smithi096.stdout:3/563: chown d0/d13/d45/c4d 30 1 2024-03-24T10:46:14.692 INFO:tasks.workunit.client.1.smithi096.stdout:3/564: write d0/d13/d1e/f2c [86180,109062] 0 2024-03-24T10:46:14.704 INFO:tasks.workunit.client.1.smithi096.stdout:7/727: dread d0/d13/df0/d3c/f82 [0,4194304] 0 2024-03-24T10:46:14.705 INFO:tasks.workunit.client.1.smithi096.stdout:8/546: truncate d8/f16 1267341 0 2024-03-24T10:46:14.706 INFO:tasks.workunit.client.1.smithi096.stdout:5/533: dwrite d2/d29/d32/d3f/d84/fa5 [0,4194304] 0 2024-03-24T10:46:14.707 INFO:tasks.workunit.client.1.smithi096.stdout:9/487: rmdir d9/d15/d26/d29 39 2024-03-24T10:46:14.709 INFO:tasks.workunit.client.1.smithi096.stdout:2/501: truncate d5/d7c/d88/f69 8971958 0 2024-03-24T10:46:14.709 INFO:tasks.workunit.client.1.smithi096.stdout:2/502: readlink d5/l10 0 2024-03-24T10:46:14.709 INFO:tasks.workunit.client.1.smithi096.stdout:2/503: read d5/dc/d12/f16 [73443,117744] 0 2024-03-24T10:46:14.712 INFO:tasks.workunit.client.1.smithi096.stdout:1/414: sync 2024-03-24T10:46:14.712 INFO:tasks.workunit.client.1.smithi096.stdout:0/501: sync 2024-03-24T10:46:14.712 INFO:tasks.workunit.client.1.smithi096.stdout:4/440: sync 2024-03-24T10:46:14.712 INFO:tasks.workunit.client.1.smithi096.stdout:1/415: write d3/d23/f62 [1331594,119117] 0 2024-03-24T10:46:14.712 INFO:tasks.workunit.client.1.smithi096.stdout:1/416: readlink d3/d23/d2f/l5e 0 2024-03-24T10:46:14.714 INFO:tasks.workunit.client.1.smithi096.stdout:4/441: dread d1/f3e [0,4194304] 0 2024-03-24T10:46:14.714 INFO:tasks.workunit.client.1.smithi096.stdout:4/442: truncate d1/d32/d42/f86 282541 0 2024-03-24T10:46:14.717 INFO:tasks.workunit.client.1.smithi096.stdout:3/565: mkdir d0/d13/d45/d52/d88/dbc 0 2024-03-24T10:46:14.732 INFO:tasks.workunit.client.1.smithi096.stdout:0/502: dwrite d0/d11/d16/d21/d25/d47/f70 [0,4194304] 0 2024-03-24T10:46:14.732 INFO:tasks.workunit.client.1.smithi096.stdout:0/503: chown d0/d11/d16/d21/d81/d9c 220 1 2024-03-24T10:46:14.732 INFO:tasks.workunit.client.1.smithi096.stdout:7/728: fdatasync d0/d13/df0/d59/fd6 0 2024-03-24T10:46:14.734 INFO:tasks.workunit.client.1.smithi096.stdout:8/547: rmdir d8/d11/d22/d4c/d65/d1d/d3d/d5f 39 2024-03-24T10:46:14.734 INFO:tasks.workunit.client.1.smithi096.stdout:8/548: write d8/d11/d22/d4c/d65/d1a/d46/d2e/f54 [2178493,2931] 0 2024-03-24T10:46:14.736 INFO:tasks.workunit.client.1.smithi096.stdout:6/508: rmdir d8 39 2024-03-24T10:46:14.740 INFO:tasks.workunit.client.1.smithi096.stdout:5/534: mkdir d2/d29/d32/d3f/d5f/db3 0 2024-03-24T10:46:14.745 INFO:tasks.workunit.client.1.smithi096.stdout:1/417: unlink d3/c19 0 2024-03-24T10:46:14.746 INFO:tasks.workunit.client.1.smithi096.stdout:3/566: rmdir d0/d13/d45/d47/d8f 39 2024-03-24T10:46:14.746 INFO:tasks.workunit.client.1.smithi096.stdout:3/567: readlink d0/d13/d45/d7d/d39/d48/d8d/la5 0 2024-03-24T10:46:14.749 INFO:tasks.workunit.client.1.smithi096.stdout:7/729: mkdir d0/d13/df0/d3c/d68/d4b/df5 0 2024-03-24T10:46:14.751 INFO:tasks.workunit.client.1.smithi096.stdout:0/504: dwrite d0/d11/d24/f68 [0,4194304] 0 2024-03-24T10:46:14.751 INFO:tasks.workunit.client.1.smithi096.stdout:0/505: chown d0/d11/l35 368777554 1 2024-03-24T10:46:14.753 INFO:tasks.workunit.client.1.smithi096.stdout:9/488: read d9/f5c [3452514,100143] 0 2024-03-24T10:46:14.754 INFO:tasks.workunit.client.1.smithi096.stdout:9/489: readlink d9/db/l69 0 2024-03-24T10:46:14.754 INFO:tasks.workunit.client.1.smithi096.stdout:6/509: creat d8/d10/d76/fa0 x:0 0 0 2024-03-24T10:46:14.755 INFO:tasks.workunit.client.1.smithi096.stdout:2/504: rmdir d5/dc/d12/d23 39 2024-03-24T10:46:14.757 INFO:tasks.workunit.client.1.smithi096.stdout:1/418: mknod d3/d23/d2f/d65/c8e 0 2024-03-24T10:46:14.759 INFO:tasks.workunit.client.1.smithi096.stdout:5/535: dwrite d2/d7/f86 [0,4194304] 0 2024-03-24T10:46:14.759 INFO:tasks.workunit.client.1.smithi096.stdout:5/536: readlink d2/d7/la4 0 2024-03-24T10:46:14.759 INFO:tasks.workunit.client.1.smithi096.stdout:5/537: write d2/d29/d32/d3f/d5f/f8f [107326,85184] 0 2024-03-24T10:46:14.760 INFO:tasks.workunit.client.1.smithi096.stdout:7/730: symlink d0/db/d26/d48/da8/lf6 0 2024-03-24T10:46:14.764 INFO:tasks.workunit.client.1.smithi096.stdout:0/506: creat d0/d11/d16/d21/d78/fa8 x:0 0 0 2024-03-24T10:46:14.766 INFO:tasks.workunit.client.1.smithi096.stdout:9/490: symlink d9/d15/d26/d29/d73/d49/la2 0 2024-03-24T10:46:14.766 INFO:tasks.workunit.client.1.smithi096.stdout:9/491: write d9/db/d6d/d4f/d75/d8b/d8e/f8f [588948,130380] 0 2024-03-24T10:46:14.767 INFO:tasks.workunit.client.1.smithi096.stdout:2/505: getdents d5/de/d2a 0 2024-03-24T10:46:14.769 INFO:tasks.workunit.client.1.smithi096.stdout:1/419: mknod d3/c8f 0 2024-03-24T10:46:14.770 INFO:tasks.workunit.client.1.smithi096.stdout:6/510: dwrite d8/d12/d32/d66/f73 [0,4194304] 0 2024-03-24T10:46:14.777 INFO:tasks.workunit.client.1.smithi096.stdout:9/492: creat d9/d15/d26/d29/d73/d43/fa3 x:0 0 0 2024-03-24T10:46:14.777 INFO:tasks.workunit.client.1.smithi096.stdout:9/493: fsync d9/db/d6d/d7f/f3f 0 2024-03-24T10:46:14.777 INFO:tasks.workunit.client.1.smithi096.stdout:1/420: symlink d3/d23/d2f/d65/d7c/l90 0 2024-03-24T10:46:14.777 INFO:tasks.workunit.client.1.smithi096.stdout:1/421: stat d3/f9 0 2024-03-24T10:46:14.783 INFO:tasks.workunit.client.1.smithi096.stdout:6/511: link d8/d12/d32/c54 d8/d12/d32/d38/ca1 0 2024-03-24T10:46:14.783 INFO:tasks.workunit.client.1.smithi096.stdout:6/512: write d8/d10/f3e [1381536,28103] 0 2024-03-24T10:46:14.783 INFO:tasks.workunit.client.1.smithi096.stdout:6/513: fsync d8/f83 0 2024-03-24T10:46:14.783 INFO:tasks.workunit.client.1.smithi096.stdout:1/422: mknod d3/d84/c91 0 2024-03-24T10:46:14.784 INFO:tasks.workunit.client.1.smithi096.stdout:2/506: dwrite d5/d38/fa1 [0,4194304] 0 2024-03-24T10:46:14.784 INFO:tasks.workunit.client.1.smithi096.stdout:2/507: chown d5/l96 1137974919 1 2024-03-24T10:46:14.785 INFO:tasks.workunit.client.1.smithi096.stdout:2/508: dread - d5/f84 zero size 2024-03-24T10:46:14.787 INFO:tasks.workunit.client.1.smithi096.stdout:6/514: creat d8/d12/d61/fa2 x:0 0 0 2024-03-24T10:46:14.788 INFO:tasks.workunit.client.1.smithi096.stdout:2/509: getdents d5/dc/d6b/d72 0 2024-03-24T10:46:14.788 INFO:tasks.workunit.client.1.smithi096.stdout:1/423: dread d3/d23/d2f/f83 [0,4194304] 0 2024-03-24T10:46:14.790 INFO:tasks.workunit.client.1.smithi096.stdout:6/515: truncate d8/f83 408308 0 2024-03-24T10:46:14.793 INFO:tasks.workunit.client.1.smithi096.stdout:1/424: getdents d3/d23/d8b/d5c/d6d 0 2024-03-24T10:46:14.793 INFO:tasks.workunit.client.1.smithi096.stdout:1/425: chown d3/d23/d8b/d6a/c6f 1824587399 1 2024-03-24T10:46:14.794 INFO:tasks.workunit.client.1.smithi096.stdout:1/426: fdatasync d3/d23/f68 0 2024-03-24T10:46:14.794 INFO:tasks.workunit.client.1.smithi096.stdout:6/516: mkdir d8/d10/d9e/da3 0 2024-03-24T10:46:14.798 INFO:tasks.workunit.client.1.smithi096.stdout:1/427: creat d3/d23/d8b/d5c/f92 x:0 0 0 2024-03-24T10:46:14.799 INFO:tasks.workunit.client.1.smithi096.stdout:2/510: read d5/dc/d12/d23/f2c [1342587,84610] 0 2024-03-24T10:46:14.799 INFO:tasks.workunit.client.1.smithi096.stdout:2/511: write d5/fa7 [527363,94496] 0 2024-03-24T10:46:14.799 INFO:tasks.workunit.client.1.smithi096.stdout:2/512: dread - d5/de/d2a/f62 zero size 2024-03-24T10:46:14.799 INFO:tasks.workunit.client.1.smithi096.stdout:2/513: chown d5/d38/fa8 30 1 2024-03-24T10:46:14.800 INFO:tasks.workunit.client.1.smithi096.stdout:6/517: dread d8/d12/d28/f33 [0,4194304] 0 2024-03-24T10:46:14.803 INFO:tasks.workunit.client.1.smithi096.stdout:1/428: unlink d3/d23/d71/d85/c89 0 2024-03-24T10:46:14.806 INFO:tasks.workunit.client.1.smithi096.stdout:3/568: rename d0/d13/d45/d52/d88 to d0/d13/d45/d7d/d39/d48/dbd 0 2024-03-24T10:46:14.811 INFO:tasks.workunit.client.1.smithi096.stdout:1/429: mkdir d3/d23/d71/d7b/d93 0 2024-03-24T10:46:14.811 INFO:tasks.workunit.client.1.smithi096.stdout:1/430: read - d3/d23/d8b/d5c/f92 zero size 2024-03-24T10:46:14.811 INFO:tasks.workunit.client.1.smithi096.stdout:1/431: write d3/d23/d8b/f46 [64092,119256] 0 2024-03-24T10:46:14.811 INFO:tasks.workunit.client.1.smithi096.stdout:1/432: chown d3/d23/d2f/f78 10240 1 2024-03-24T10:46:14.818 INFO:tasks.workunit.client.1.smithi096.stdout:6/518: dwrite d8/d10/f3d [4194304,4194304] 0 2024-03-24T10:46:14.822 INFO:tasks.workunit.client.1.smithi096.stdout:3/569: write d0/d13/d45/d7d/d39/d48/f6a [190615,66146] 0 2024-03-24T10:46:14.838 INFO:tasks.workunit.client.1.smithi096.stdout:3/570: dwrite d0/d13/d45/d7d/f94 [0,4194304] 0 2024-03-24T10:46:14.914 INFO:tasks.workunit.client.1.smithi096.stdout:5/538: rename d2/d7/d8/df/l2e to d2/d7/d8/df/d14/lb4 0 2024-03-24T10:46:14.921 INFO:tasks.workunit.client.1.smithi096.stdout:6/519: mkdir d8/d12/d28/d36/d65/da4 0 2024-03-24T10:46:14.924 INFO:tasks.workunit.client.1.smithi096.stdout:5/539: dread d2/d29/d32/d3f/d84/fa5 [0,4194304] 0 2024-03-24T10:46:14.933 INFO:tasks.workunit.client.1.smithi096.stdout:0/507: rename d0/c8b to d0/d11/d16/d21/d81/d8f/d9a/ca9 0 2024-03-24T10:46:14.936 INFO:tasks.workunit.client.1.smithi096.stdout:0/508: read d0/d11/d20/d91/f96 [569323,76642] 0 2024-03-24T10:46:14.938 INFO:tasks.workunit.client.1.smithi096.stdout:6/520: getdents d8/d10/d13/d63 0 2024-03-24T10:46:14.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.939+0000 7f6d17fc2700 1 -- 172.21.15.94:0/86409610 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6d1010f970 con 0x7f6d10066b70 2024-03-24T10:46:14.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.939+0000 7f6d067fc700 1 -- 172.21.15.94:0/86409610 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1833 (secure 0 0 0) 0x7f6d0c057d70 con 0x7f6d10066b70 2024-03-24T10:46:14.940 INFO:teuthology.orchestra.run.smithi094.stdout:e18 2024-03-24T10:46:14.940 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T10:46:14.940 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:46:14.940 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-03-24T10:46:14.940 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:46:14.941 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-03-24T10:46:14.941 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-03-24T10:46:14.941 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 18 2024-03-24T10:46:14.941 INFO:teuthology.orchestra.run.smithi094.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-24T10:46:14.941 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-03-24T10:43:39.435738+0000 2024-03-24T10:46:14.941 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-03-24T10:43:58.133901+0000 2024-03-24T10:46:14.941 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-03-24T10:46:14.941 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-03-24T10:46:14.941 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 60 2024-03-24T10:46:14.941 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-03-24T10:46:14.941 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-03-24T10:46:14.941 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-03-24T10:46:14.941 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-03-24T10:46:14.941 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 46 2024-03-24T10:46:14.941 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:46:14.941 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-03-24T10:46:14.942 INFO:teuthology.orchestra.run.smithi094.stdout:in 0 2024-03-24T10:46:14.942 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=24251} 2024-03-24T10:46:14.942 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-03-24T10:46:14.942 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-03-24T10:46:14.942 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-03-24T10:46:14.942 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-03-24T10:46:14.942 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-03-24T10:46:14.942 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-03-24T10:46:14.942 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-03-24T10:46:14.942 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 1 2024-03-24T10:46:14.942 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi096.egbmoq{0:24251} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.96:6824/1539752208,v1:172.21.15.96:6825/1539752208] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:46:14.942 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.llefgu{0:14500} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.94:6828/126762980,v1:172.21.15.94:6829/126762980] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:46:14.942 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:46:14.943 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:46:14.943 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-03-24T10:46:14.943 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:46:14.943 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.efiqte{-1:14512} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.94:6826/1636528860,v1:172.21.15.94:6827/1636528860] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:46:14.943 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi096.jtebhh{-1:24261} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.96:6826/1321087930,v1:172.21.15.96:6827/1321087930] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:46:14.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.942+0000 7f6d17fc2700 1 -- 172.21.15.94:0/86409610 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f6cfc0692a0 msgr2=0x7f6cfc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:14.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.942+0000 7f6d17fc2700 1 --2- 172.21.15.94:0/86409610 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f6cfc0692a0 0x7f6cfc06b750 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6d00006fd0 tx=0x7f6d00008040 comp rx=0 tx=0).stop 2024-03-24T10:46:14.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.942+0000 7f6d17fc2700 1 -- 172.21.15.94:0/86409610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d10066b70 msgr2=0x7f6d1010dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:14.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.942+0000 7f6d17fc2700 1 --2- 172.21.15.94:0/86409610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d10066b70 0x7f6d1010dae0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c00b970 tx=0x7f6d0c009e30 comp rx=0 tx=0).stop 2024-03-24T10:46:14.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.942+0000 7f6d17fc2700 1 -- 172.21.15.94:0/86409610 shutdown_connections 2024-03-24T10:46:14.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.942+0000 7f6d17fc2700 1 --2- 172.21.15.94:0/86409610 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f6cfc0692a0 0x7f6cfc06b750 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:14.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.942+0000 7f6d17fc2700 1 --2- 172.21.15.94:0/86409610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d10066b70 0x7f6d1010dae0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:14.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.942+0000 7f6d17fc2700 1 --2- 172.21.15.94:0/86409610 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6d100fff90 0x7f6d1010e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:14.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.942+0000 7f6d17fc2700 1 -- 172.21.15.94:0/86409610 >> 172.21.15.94:0/86409610 conn(0x7f6d100f74e0 msgr2=0x7f6d100f78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:46:14.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.942+0000 7f6d17fc2700 1 -- 172.21.15.94:0/86409610 shutdown_connections 2024-03-24T10:46:14.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:14.942+0000 7f6d17fc2700 1 -- 172.21.15.94:0/86409610 wait complete. 2024-03-24T10:46:14.945 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 18 2024-03-24T10:46:14.946 INFO:tasks.workunit.client.1.smithi096.stdout:3/571: rename d0/d13/d45/d52/f78 to d0/d9/fbe 0 2024-03-24T10:46:14.946 INFO:tasks.workunit.client.1.smithi096.stdout:3/572: fdatasync d0/d13/d45/d7d/d39/d48/fb0 0 2024-03-24T10:46:14.946 INFO:tasks.workunit.client.1.smithi096.stdout:6/521: symlink d8/d10/d9e/da3/la5 0 2024-03-24T10:46:14.950 INFO:tasks.workunit.client.1.smithi096.stdout:3/573: unlink d0/fb2 0 2024-03-24T10:46:14.951 INFO:tasks.workunit.client.1.smithi096.stdout:5/540: dwrite d2/d7/d8/df/f25 [0,4194304] 0 2024-03-24T10:46:14.951 INFO:tasks.workunit.client.1.smithi096.stdout:5/541: truncate d2/d29/d32/d89/f8e 942272 0 2024-03-24T10:46:14.951 INFO:tasks.workunit.client.1.smithi096.stdout:6/522: mkdir d8/d12/d28/d36/d65/d6b/da6 0 2024-03-24T10:46:14.953 INFO:tasks.workunit.client.1.smithi096.stdout:1/433: truncate d3/d23/d2f/f3b 1253596 0 2024-03-24T10:46:14.954 INFO:tasks.workunit.client.1.smithi096.stdout:1/434: dread d3/d23/f69 [0,4194304] 0 2024-03-24T10:46:14.955 INFO:tasks.workunit.client.1.smithi096.stdout:0/509: dwrite d0/d11/d20/d91/f96 [0,4194304] 0 2024-03-24T10:46:14.955 INFO:tasks.workunit.client.1.smithi096.stdout:0/510: truncate d0/d11/d24/f30 4661128 0 2024-03-24T10:46:14.955 INFO:tasks.workunit.client.1.smithi096.stdout:0/511: readlink d0/d11/d20/d9e/l90 0 2024-03-24T10:46:14.955 INFO:tasks.workunit.client.1.smithi096.stdout:0/512: write d0/d11/d16/d21/d25/d47/f70 [5135567,82749] 0 2024-03-24T10:46:14.955 INFO:tasks.workunit.client.1.smithi096.stdout:0/513: fdatasync d0/d11/d24/f2d 0 2024-03-24T10:46:14.956 INFO:tasks.workunit.client.1.smithi096.stdout:5/542: creat d2/d29/d32/d33/d3c/d3d/fb5 x:0 0 0 2024-03-24T10:46:14.958 INFO:tasks.workunit.client.1.smithi096.stdout:6/523: symlink d8/d10/d76/la7 0 2024-03-24T10:46:14.965 INFO:tasks.workunit.client.1.smithi096.stdout:1/435: truncate d3/d23/d8b/d5c/f67 3544069 0 2024-03-24T10:46:14.966 INFO:tasks.workunit.client.1.smithi096.stdout:0/514: dread d0/d11/d16/d21/d25/f36 [0,4194304] 0 2024-03-24T10:46:14.966 INFO:tasks.workunit.client.1.smithi096.stdout:0/515: chown d0/d11/d20/d9e/d80/f41 25806 1 2024-03-24T10:46:14.969 INFO:tasks.workunit.client.1.smithi096.stdout:3/574: write d0/d9/fbe [3743397,34120] 0 2024-03-24T10:46:14.978 INFO:tasks.workunit.client.1.smithi096.stdout:5/543: mkdir d2/d7/d8/d18/db6 0 2024-03-24T10:46:14.980 INFO:tasks.workunit.client.1.smithi096.stdout:1/436: mknod d3/d23/d8b/d79/c94 0 2024-03-24T10:46:14.983 INFO:tasks.workunit.client.1.smithi096.stdout:6/524: dwrite d8/d12/d28/d36/d65/f94 [0,4194304] 0 2024-03-24T10:46:14.985 INFO:tasks.workunit.client.1.smithi096.stdout:3/575: creat d0/d13/d45/d7d/d39/fbf x:0 0 0 2024-03-24T10:46:14.992 INFO:tasks.workunit.client.1.smithi096.stdout:5/544: creat d2/d7/d8/df/d95/fb7 x:0 0 0 2024-03-24T10:46:15.000 INFO:tasks.workunit.client.1.smithi096.stdout:1/437: unlink d3/d36/f50 0 2024-03-24T10:46:15.002 INFO:tasks.workunit.client.1.smithi096.stdout:0/516: rmdir d0/d11/d16/d21/d78 39 2024-03-24T10:46:15.002 INFO:tasks.workunit.client.1.smithi096.stdout:0/517: truncate d0/d11/d20/d65/f8c 1555726 0 2024-03-24T10:46:15.002 INFO:tasks.workunit.client.1.smithi096.stdout:0/518: write d0/d11/f1c [2688094,77921] 0 2024-03-24T10:46:15.006 INFO:tasks.workunit.client.1.smithi096.stdout:3/576: creat d0/d13/d45/d7d/d71/db6/fc0 x:0 0 0 2024-03-24T10:46:15.007 INFO:tasks.workunit.client.1.smithi096.stdout:3/577: chown d0/d13/d45/d7d/d39/d48/fb0 140367 1 2024-03-24T10:46:15.009 INFO:tasks.workunit.client.1.smithi096.stdout:5/545: mkdir d2/d29/d32/d3f/d84/db8 0 2024-03-24T10:46:15.011 INFO:tasks.workunit.client.1.smithi096.stdout:3/578: rename d0/d9/f8b to d0/d13/d45/d52/fc1 0 2024-03-24T10:46:15.013 INFO:tasks.workunit.client.1.smithi096.stdout:3/579: truncate d0/d13/f57 4181400 0 2024-03-24T10:46:15.015 INFO:tasks.workunit.client.1.smithi096.stdout:6/525: dwrite d8/d12/d32/f3c [0,4194304] 0 2024-03-24T10:46:15.015 INFO:tasks.workunit.client.1.smithi096.stdout:6/526: chown d8/d10/d76/f8d 292617757 1 2024-03-24T10:46:15.015 INFO:tasks.workunit.client.1.smithi096.stdout:6/527: fsync d8/d12/d28/d36/d65/f6f 0 2024-03-24T10:46:15.015 INFO:tasks.workunit.client.1.smithi096.stdout:6/528: stat d8/d10/d19 0 2024-03-24T10:46:15.022 INFO:tasks.workunit.client.1.smithi096.stdout:3/580: write d0/f5c [4643636,85660] 0 2024-03-24T10:46:15.023 INFO:tasks.workunit.client.1.smithi096.stdout:5/546: dread d2/d29/d32/d33/d3c/d3d/f75 [0,4194304] 0 2024-03-24T10:46:15.026 INFO:tasks.workunit.client.1.smithi096.stdout:3/581: link d0/d13/d1e/f2f d0/d9/d22/fc2 0 2024-03-24T10:46:15.028 INFO:tasks.workunit.client.1.smithi096.stdout:1/438: dwrite d3/d23/d2f/f59 [0,4194304] 0 2024-03-24T10:46:15.030 INFO:tasks.workunit.client.1.smithi096.stdout:1/439: dread d3/d23/d8b/f35 [0,4194304] 0 2024-03-24T10:46:15.031 INFO:tasks.workunit.client.1.smithi096.stdout:1/440: stat d3/d23/d8b/d6a/f77 0 2024-03-24T10:46:15.031 INFO:tasks.workunit.client.1.smithi096.stdout:1/441: fdatasync d3/f34 0 2024-03-24T10:46:15.031 INFO:tasks.workunit.client.1.smithi096.stdout:6/529: dread d8/d12/d32/f3c [0,4194304] 0 2024-03-24T10:46:15.031 INFO:tasks.workunit.client.1.smithi096.stdout:6/530: chown d8/d7b 0 1 2024-03-24T10:46:15.032 INFO:tasks.workunit.client.1.smithi096.stdout:1/442: mknod d3/d73/c95 0 2024-03-24T10:46:15.034 INFO:tasks.workunit.client.1.smithi096.stdout:6/531: truncate d8/d10/d19/f1b 5272415 0 2024-03-24T10:46:15.035 INFO:tasks.workunit.client.1.smithi096.stdout:6/532: dread d8/f29 [0,4194304] 0 2024-03-24T10:46:15.036 INFO:tasks.workunit.client.1.smithi096.stdout:1/443: creat d3/d23/d8b/d6a/f96 x:0 0 0 2024-03-24T10:46:15.038 INFO:tasks.workunit.client.1.smithi096.stdout:6/533: mkdir d8/d12/d32/d66/da8 0 2024-03-24T10:46:15.038 INFO:tasks.workunit.client.1.smithi096.stdout:6/534: readlink d8/d12/d32/d38/d59/l9f 0 2024-03-24T10:46:15.038 INFO:tasks.workunit.client.1.smithi096.stdout:6/535: creat d8/d10/d76/fa9 x:0 0 0 2024-03-24T10:46:15.038 INFO:tasks.workunit.client.1.smithi096.stdout:6/536: write d8/d10/d76/f95 [251503,73090] 0 2024-03-24T10:46:15.040 INFO:tasks.workunit.client.1.smithi096.stdout:1/444: mknod d3/d84/c97 0 2024-03-24T10:46:15.040 INFO:tasks.workunit.client.1.smithi096.stdout:0/519: dwrite d0/d11/d24/f2d [0,4194304] 0 2024-03-24T10:46:15.043 INFO:tasks.workunit.client.1.smithi096.stdout:1/445: unlink d3/d23/d8b/f39 0 2024-03-24T10:46:15.048 INFO:tasks.workunit.client.1.smithi096.stdout:5/547: dwrite d2/d7/f2a [0,4194304] 0 2024-03-24T10:46:15.048 INFO:tasks.workunit.client.1.smithi096.stdout:0/520: creat d0/d11/d16/d21/d81/faa x:0 0 0 2024-03-24T10:46:15.049 INFO:tasks.workunit.client.1.smithi096.stdout:0/521: write d0/d11/d20/d65/f8c [431316,5892] 0 2024-03-24T10:46:15.051 INFO:tasks.workunit.client.1.smithi096.stdout:1/446: mkdir d3/d23/d71/d7b/d98 0 2024-03-24T10:46:15.054 INFO:tasks.workunit.client.1.smithi096.stdout:5/548: rmdir d2/d7/d69 39 2024-03-24T10:46:15.055 INFO:tasks.workunit.client.1.smithi096.stdout:0/522: getdents d0/d11/d24/d9d 0 2024-03-24T10:46:15.055 INFO:tasks.workunit.client.1.smithi096.stdout:0/523: chown d0/d11/d43/d75 24874478 1 2024-03-24T10:46:15.056 INFO:tasks.workunit.client.1.smithi096.stdout:0/524: readlink d0/d11/d43/l83 0 2024-03-24T10:46:15.057 INFO:tasks.workunit.client.1.smithi096.stdout:5/549: getdents d2/d29/d32/d33/d8d 0 2024-03-24T10:46:15.060 INFO:tasks.workunit.client.1.smithi096.stdout:0/525: getdents d0/d11/d16/d21/d25 0 2024-03-24T10:46:15.061 INFO:tasks.workunit.client.1.smithi096.stdout:5/550: read d2/d7/f4a [1276240,99074] 0 2024-03-24T10:46:15.062 INFO:tasks.workunit.client.1.smithi096.stdout:0/526: rmdir d0/d11/d20/d56 39 2024-03-24T10:46:15.062 INFO:tasks.workunit.client.1.smithi096.stdout:0/527: readlink d0/d11/d16/l9f 0 2024-03-24T10:46:15.062 INFO:tasks.workunit.client.1.smithi096.stdout:0/528: chown d0/d11/d20/d65 8242459 1 2024-03-24T10:46:15.067 INFO:tasks.workunit.client.1.smithi096.stdout:0/529: dread d0/d11/d20/f26 [0,4194304] 0 2024-03-24T10:46:15.067 INFO:tasks.workunit.client.1.smithi096.stdout:0/530: chown d0/d11/d16/d21/d81/d7f/l29 32262 1 2024-03-24T10:46:15.067 INFO:tasks.workunit.client.1.smithi096.stdout:0/531: dread - d0/d11/d16/d21/d81/d7f/f94 zero size 2024-03-24T10:46:15.075 INFO:tasks.workunit.client.1.smithi096.stdout:5/551: rename d2/d7/d8/df/d95 to d2/d29/d32/db9 0 2024-03-24T10:46:15.077 INFO:tasks.workunit.client.1.smithi096.stdout:3/582: rmdir d0/d13/d45/d7d/d71/db6 39 2024-03-24T10:46:15.079 INFO:tasks.workunit.client.1.smithi096.stdout:3/583: unlink d0/d9/c1c 0 2024-03-24T10:46:15.080 INFO:tasks.workunit.client.1.smithi096.stdout:3/584: mknod d0/d13/d45/d7d/d39/d48/dbd/daa/cc3 0 2024-03-24T10:46:15.081 INFO:tasks.workunit.client.1.smithi096.stdout:3/585: truncate d0/d13/d1e/f2d 57146 0 2024-03-24T10:46:15.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.086+0000 7f8b6cfea700 1 -- 172.21.15.94:0/1777146840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b680fe060 msgr2=0x7f8b680fe4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:15.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.086+0000 7f8b6cfea700 1 --2- 172.21.15.94:0/1777146840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b680fe060 0x7f8b680fe4d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c004940 tx=0x7f8b5c009e30 comp rx=0 tx=0).stop 2024-03-24T10:46:15.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.086+0000 7f8b6cfea700 1 -- 172.21.15.94:0/1777146840 shutdown_connections 2024-03-24T10:46:15.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.086+0000 7f8b6cfea700 1 --2- 172.21.15.94:0/1777146840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b680fe060 0x7f8b680fe4d0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:15.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.086+0000 7f8b6cfea700 1 --2- 172.21.15.94:0/1777146840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b680fcef0 0x7f8b680fd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:15.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.086+0000 7f8b6cfea700 1 -- 172.21.15.94:0/1777146840 >> 172.21.15.94:0/1777146840 conn(0x7f8b680f85b0 msgr2=0x7f8b680fa9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:46:15.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.086+0000 7f8b6cfea700 1 -- 172.21.15.94:0/1777146840 shutdown_connections 2024-03-24T10:46:15.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.086+0000 7f8b6cfea700 1 -- 172.21.15.94:0/1777146840 wait complete. 2024-03-24T10:46:15.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.087+0000 7f8b6cfea700 1 Processor -- start 2024-03-24T10:46:15.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.087+0000 7f8b6cfea700 1 -- start start 2024-03-24T10:46:15.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.087+0000 7f8b6cfea700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b680fcef0 0x7f8b68071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:46:15.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.087+0000 7f8b6cfea700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b680fe060 0x7f8b68070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:46:15.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.087+0000 7f8b6cfea700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b680708c0 con 0x7f8b680fcef0 2024-03-24T10:46:15.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.087+0000 7f8b6cfea700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b68070a30 con 0x7f8b680fe060 2024-03-24T10:46:15.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.087+0000 7f8b6659c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b680fcef0 0x7f8b68071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:46:15.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.087+0000 7f8b6659c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b680fcef0 0x7f8b68071e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54376/0 (socket says 172.21.15.94:54376) 2024-03-24T10:46:15.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.087+0000 7f8b6659c700 1 -- 172.21.15.94:0/1821191880 learned_addr learned my addr 172.21.15.94:0/1821191880 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:46:15.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.087+0000 7f8b65d9b700 1 --2- 172.21.15.94:0/1821191880 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b680fe060 0x7f8b68070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:46:15.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.087+0000 7f8b6659c700 1 -- 172.21.15.94:0/1821191880 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b680fe060 msgr2=0x7f8b68070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:15.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.087+0000 7f8b6659c700 1 --2- 172.21.15.94:0/1821191880 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b680fe060 0x7f8b68070310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:15.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.087+0000 7f8b6659c700 1 -- 172.21.15.94:0/1821191880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b5c005040 con 0x7f8b680fcef0 2024-03-24T10:46:15.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.087+0000 7f8b6659c700 1 --2- 172.21.15.94:0/1821191880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b680fcef0 0x7f8b68071e40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8b58002a50 tx=0x7f8b580046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:46:15.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.088+0000 7f8b56ffd700 1 -- 172.21.15.94:0/1821191880 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b58007cc0 con 0x7f8b680fcef0 2024-03-24T10:46:15.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.088+0000 7f8b56ffd700 1 -- 172.21.15.94:0/1821191880 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8b5800f040 con 0x7f8b680fcef0 2024-03-24T10:46:15.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.088+0000 7f8b6cfea700 1 -- 172.21.15.94:0/1821191880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b68070d10 con 0x7f8b680fcef0 2024-03-24T10:46:15.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.088+0000 7f8b56ffd700 1 -- 172.21.15.94:0/1821191880 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b5800da30 con 0x7f8b680fcef0 2024-03-24T10:46:15.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.088+0000 7f8b6cfea700 1 -- 172.21.15.94:0/1821191880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b68115a70 con 0x7f8b680fcef0 2024-03-24T10:46:15.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.089+0000 7f8b6cfea700 1 -- 172.21.15.94:0/1821191880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b6805f5f0 con 0x7f8b680fcef0 2024-03-24T10:46:15.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.089+0000 7f8b56ffd700 1 -- 172.21.15.94:0/1821191880 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f8b5800db90 con 0x7f8b680fcef0 2024-03-24T10:46:15.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.089+0000 7f8b56ffd700 1 --2- 172.21.15.94:0/1821191880 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f8b500692a0 0x7f8b5006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:46:15.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.090+0000 7f8b56ffd700 1 -- 172.21.15.94:0/1821191880 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f8b58086f90 con 0x7f8b680fcef0 2024-03-24T10:46:15.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.090+0000 7f8b65d9b700 1 --2- 172.21.15.94:0/1821191880 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f8b500692a0 0x7f8b5006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:46:15.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.090+0000 7f8b65d9b700 1 --2- 172.21.15.94:0/1821191880 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f8b500692a0 0x7f8b5006b750 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c00b970 tx=0x7f8b5c009e30 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:46:15.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.094+0000 7f8b56ffd700 1 -- 172.21.15.94:0/1821191880 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f8b580531f0 con 0x7f8b680fcef0 2024-03-24T10:46:15.099 INFO:tasks.workunit.client.1.smithi096.stdout:4/443: sync 2024-03-24T10:46:15.101 INFO:tasks.workunit.client.1.smithi096.stdout:4/444: symlink d1/d32/d42/l92 0 2024-03-24T10:46:15.101 INFO:tasks.workunit.client.1.smithi096.stdout:6/537: dread d8/d10/d19/f1b [0,4194304] 0 2024-03-24T10:46:15.101 INFO:tasks.workunit.client.1.smithi096.stdout:4/445: mknod d1/d2b/d40/c93 0 2024-03-24T10:46:15.104 INFO:tasks.workunit.client.1.smithi096.stdout:4/446: unlink d1/f10 0 2024-03-24T10:46:15.106 INFO:tasks.workunit.client.1.smithi096.stdout:4/447: mknod d1/d2b/d6b/d80/c94 0 2024-03-24T10:46:15.108 INFO:tasks.workunit.client.1.smithi096.stdout:6/538: read d8/f62 [1472249,30335] 0 2024-03-24T10:46:15.109 INFO:tasks.workunit.client.1.smithi096.stdout:6/539: rename d8/d12/d61/fa2 to d8/d12/d61/faa 0 2024-03-24T10:46:15.110 INFO:tasks.workunit.client.1.smithi096.stdout:6/540: fdatasync d8/d12/d28/f33 0 2024-03-24T10:46:15.118 INFO:tasks.workunit.client.1.smithi096.stdout:6/541: dread d8/d12/d28/d36/d65/f6f [0,4194304] 0 2024-03-24T10:46:15.118 INFO:tasks.workunit.client.1.smithi096.stdout:6/542: stat d8/d10/d13/d63/d51/l6a 0 2024-03-24T10:46:15.119 INFO:tasks.workunit.client.1.smithi096.stdout:6/543: creat d8/d7b/fab x:0 0 0 2024-03-24T10:46:15.128 INFO:tasks.workunit.client.1.smithi096.stdout:6/544: read d8/d10/f8a [3443138,61976] 0 2024-03-24T10:46:15.129 INFO:tasks.workunit.client.1.smithi096.stdout:6/545: getdents d8/d12/d32/d68 0 2024-03-24T10:46:15.192 INFO:tasks.workunit.client.1.smithi096.stdout:6/546: dwrite d8/d12/d32/d38/f82 [0,4194304] 0 2024-03-24T10:46:15.193 INFO:tasks.workunit.client.1.smithi096.stdout:6/547: write d8/d12/d28/d36/d65/d6b/f75 [383177,58571] 0 2024-03-24T10:46:15.199 INFO:tasks.workunit.client.1.smithi096.stdout:0/532: dwrite d0/d11/d24/f55 [0,4194304] 0 2024-03-24T10:46:15.201 INFO:tasks.workunit.client.1.smithi096.stdout:0/533: rmdir d0/d11/d20/d56 39 2024-03-24T10:46:15.202 INFO:tasks.workunit.client.1.smithi096.stdout:0/534: rmdir d0/d11/d16/d21/d81/d8f/d9a 39 2024-03-24T10:46:15.203 INFO:tasks.workunit.client.1.smithi096.stdout:0/535: rename d0/d11/d43/l83 to d0/d11/d20/d65/lab 0 2024-03-24T10:46:15.206 INFO:tasks.workunit.client.1.smithi096.stdout:4/448: dwrite d1/d11/f41 [0,4194304] 0 2024-03-24T10:46:15.208 INFO:tasks.workunit.client.1.smithi096.stdout:6/548: dwrite d8/d12/d32/d38/d59/f79 [0,4194304] 0 2024-03-24T10:46:15.215 INFO:tasks.workunit.client.1.smithi096.stdout:6/549: link d8/d12/d32/d38/d59/f98 d8/d12/d32/d68/fac 0 2024-03-24T10:46:15.217 INFO:tasks.workunit.client.1.smithi096.stdout:6/550: mkdir d8/d10/d9e/dad 0 2024-03-24T10:46:15.226 INFO:tasks.workunit.client.1.smithi096.stdout:6/551: write f2 [1868323,114697] 0 2024-03-24T10:46:15.226 INFO:tasks.workunit.client.1.smithi096.stdout:6/552: chown d8/f62 149962 1 2024-03-24T10:46:15.229 INFO:tasks.workunit.client.1.smithi096.stdout:4/449: dwrite d1/fd [0,4194304] 0 2024-03-24T10:46:15.229 INFO:tasks.workunit.client.1.smithi096.stdout:6/553: getdents d8/d12/d28/d36/d65 0 2024-03-24T10:46:15.230 INFO:tasks.workunit.client.1.smithi096.stdout:0/536: dwrite d0/d11/d16/d21/d25/d47/f70 [4194304,4194304] 0 2024-03-24T10:46:15.231 INFO:tasks.workunit.client.1.smithi096.stdout:4/450: dread d1/d2b/d67/f6a [0,4194304] 0 2024-03-24T10:46:15.233 INFO:tasks.workunit.client.1.smithi096.stdout:6/554: creat d8/d12/d28/d36/fae x:0 0 0 2024-03-24T10:46:15.235 INFO:tasks.workunit.client.1.smithi096.stdout:4/451: link d1/d32/d42/l6c d1/d2b/d6b/l95 0 2024-03-24T10:46:15.237 INFO:tasks.workunit.client.1.smithi096.stdout:6/555: symlink d8/d10/d13/d63/d51/laf 0 2024-03-24T10:46:15.237 INFO:tasks.workunit.client.1.smithi096.stdout:6/556: stat d8/d10/f8a 0 2024-03-24T10:46:15.237 INFO:tasks.workunit.client.1.smithi096.stdout:0/537: write d0/f48 [1928480,42646] 0 2024-03-24T10:46:15.239 INFO:tasks.workunit.client.1.smithi096.stdout:4/452: creat d1/d2b/d6b/f96 x:0 0 0 2024-03-24T10:46:15.240 INFO:tasks.workunit.client.1.smithi096.stdout:6/557: symlink d8/d12/d28/d36/d65/d6b/da6/lb0 0 2024-03-24T10:46:15.243 INFO:tasks.workunit.client.1.smithi096.stdout:0/538: creat d0/d11/d16/d21/d78/d92/fac x:0 0 0 2024-03-24T10:46:15.243 INFO:tasks.workunit.client.1.smithi096.stdout:0/539: write d0/d11/d16/fa5 [96211,64391] 0 2024-03-24T10:46:15.243 INFO:tasks.workunit.client.1.smithi096.stdout:4/453: symlink d1/d2b/d59/l97 0 2024-03-24T10:46:15.246 INFO:tasks.workunit.client.1.smithi096.stdout:6/558: creat d8/d10/d9e/da3/fb1 x:0 0 0 2024-03-24T10:46:15.246 INFO:tasks.workunit.client.1.smithi096.stdout:6/559: fsync d8/d10/d58/f85 0 2024-03-24T10:46:15.247 INFO:tasks.workunit.client.1.smithi096.stdout:0/540: mknod d0/d11/d16/cad 0 2024-03-24T10:46:15.247 INFO:tasks.workunit.client.1.smithi096.stdout:0/541: chown d0/d11/d16/d21/d81/d9c 6103617 1 2024-03-24T10:46:15.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:15 smithi094 ceph-mon[29446]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:46:15.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:15 smithi094 ceph-mon[29446]: from='client.14674 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:46:15.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:15 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/3841449640' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:46:15.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:15 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/86409610' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T10:46:15.251 INFO:tasks.workunit.client.1.smithi096.stdout:0/542: dread d0/d11/f1c [0,4194304] 0 2024-03-24T10:46:15.251 INFO:tasks.workunit.client.1.smithi096.stdout:4/454: rmdir d1/d2b/d59/d74 39 2024-03-24T10:46:15.251 INFO:tasks.workunit.client.1.smithi096.stdout:4/455: readlink d1/d11/l72 0 2024-03-24T10:46:15.252 INFO:tasks.workunit.client.1.smithi096.stdout:6/560: mkdir d8/d12/d28/d36/d65/d6b/da6/db2 0 2024-03-24T10:46:15.270 INFO:tasks.workunit.client.1.smithi096.stdout:0/543: unlink d0/l58 0 2024-03-24T10:46:15.272 INFO:tasks.workunit.client.1.smithi096.stdout:4/456: mknod d1/d11/c98 0 2024-03-24T10:46:15.272 INFO:tasks.workunit.client.1.smithi096.stdout:4/457: dread - d1/d11/f66 zero size 2024-03-24T10:46:15.272 INFO:tasks.workunit.client.1.smithi096.stdout:6/561: mknod d8/d12/d61/d7e/cb3 0 2024-03-24T10:46:15.274 INFO:tasks.workunit.client.1.smithi096.stdout:4/458: creat d1/d32/d4c/d7e/f99 x:0 0 0 2024-03-24T10:46:15.276 INFO:tasks.workunit.client.1.smithi096.stdout:6/562: creat d8/d10/d13/d63/fb4 x:0 0 0 2024-03-24T10:46:15.296 INFO:tasks.workunit.client.1.smithi096.stdout:6/563: dwrite d8/d10/d19/f1b [0,4194304] 0 2024-03-24T10:46:15.297 INFO:tasks.workunit.client.1.smithi096.stdout:4/459: dwrite d1/f1c [4194304,4194304] 0 2024-03-24T10:46:15.297 INFO:tasks.workunit.client.1.smithi096.stdout:4/460: write d1/d2b/d40/f70 [333813,58943] 0 2024-03-24T10:46:15.299 INFO:tasks.workunit.client.1.smithi096.stdout:6/564: rename d8/d10/d19/c46 to d8/d12/d32/d38/d59/cb5 0 2024-03-24T10:46:15.301 INFO:tasks.workunit.client.1.smithi096.stdout:4/461: rename d1/d32/d42/l69 to d1/d2b/d59/d74/l9a 0 2024-03-24T10:46:15.301 INFO:tasks.workunit.client.1.smithi096.stdout:4/462: truncate d1/d2b/d67/f8a 912376 0 2024-03-24T10:46:15.302 INFO:tasks.workunit.client.1.smithi096.stdout:4/463: dread - d1/d11/f66 zero size 2024-03-24T10:46:15.302 INFO:tasks.workunit.client.1.smithi096.stdout:4/464: stat d1/d2b/d67/f82 0 2024-03-24T10:46:15.302 INFO:tasks.workunit.client.1.smithi096.stdout:4/465: chown d1/l78 1545 1 2024-03-24T10:46:15.303 INFO:tasks.workunit.client.1.smithi096.stdout:4/466: write d1/fe [87543,12762] 0 2024-03-24T10:46:15.303 INFO:tasks.workunit.client.1.smithi096.stdout:4/467: chown d1/d32/d42/l92 1182 1 2024-03-24T10:46:15.305 INFO:tasks.workunit.client.1.smithi096.stdout:4/468: mkdir d1/d32/d4c/d7d/d9b 0 2024-03-24T10:46:15.305 INFO:tasks.workunit.client.1.smithi096.stdout:4/469: fdatasync d1/f6f 0 2024-03-24T10:46:15.305 INFO:tasks.workunit.client.1.smithi096.stdout:4/470: dread - d1/d2b/d40/f64 zero size 2024-03-24T10:46:15.305 INFO:tasks.workunit.client.1.smithi096.stdout:4/471: dread - d1/d32/d4c/d7e/f99 zero size 2024-03-24T10:46:15.307 INFO:tasks.workunit.client.1.smithi096.stdout:4/472: mkdir d1/d2b/d6b/d80/d9c 0 2024-03-24T10:46:15.307 INFO:tasks.workunit.client.1.smithi096.stdout:6/565: dread d8/d10/d13/f55 [0,4194304] 0 2024-03-24T10:46:15.307 INFO:tasks.workunit.client.1.smithi096.stdout:6/566: truncate d8/d10/d13/f55 5102671 0 2024-03-24T10:46:15.308 INFO:tasks.workunit.client.1.smithi096.stdout:4/473: mkdir d1/d32/d4c/d9d 0 2024-03-24T10:46:15.320 INFO:tasks.workunit.client.1.smithi096.stdout:4/474: dread d1/f24 [0,4194304] 0 2024-03-24T10:46:15.320 INFO:tasks.workunit.client.1.smithi096.stdout:4/475: stat d1/ff 0 2024-03-24T10:46:15.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:15 smithi096 ceph-mon[38097]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:46:15.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:15 smithi096 ceph-mon[38097]: from='client.14674 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:46:15.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:15 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/3841449640' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:46:15.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:15 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/86409610' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T10:46:15.326 INFO:tasks.workunit.client.1.smithi096.stdout:8/549: sync 2024-03-24T10:46:15.326 INFO:tasks.workunit.client.1.smithi096.stdout:9/494: sync 2024-03-24T10:46:15.326 INFO:tasks.workunit.client.1.smithi096.stdout:7/731: sync 2024-03-24T10:46:15.326 INFO:tasks.workunit.client.1.smithi096.stdout:2/514: sync 2024-03-24T10:46:15.326 INFO:tasks.workunit.client.1.smithi096.stdout:2/515: fdatasync d5/de/d2a/f62 0 2024-03-24T10:46:15.327 INFO:tasks.workunit.client.1.smithi096.stdout:2/516: dread - d5/de/d2a/f35 zero size 2024-03-24T10:46:15.327 INFO:tasks.workunit.client.1.smithi096.stdout:9/495: fsync d9/d60/f7e 0 2024-03-24T10:46:15.327 INFO:tasks.workunit.client.1.smithi096.stdout:8/550: rmdir d8/d11/d44 39 2024-03-24T10:46:15.327 INFO:tasks.workunit.client.1.smithi096.stdout:8/551: chown d8/d11/d22/d4c/d82/l34 72057215 1 2024-03-24T10:46:15.327 INFO:tasks.workunit.client.1.smithi096.stdout:2/517: truncate d5/fa4 3290230 0 2024-03-24T10:46:15.328 INFO:tasks.workunit.client.1.smithi096.stdout:6/567: dwrite d8/d12/d32/d38/f82 [0,4194304] 0 2024-03-24T10:46:15.331 INFO:tasks.workunit.client.1.smithi096.stdout:7/732: link d0/d13/df0/d3c/d68/d4b/d62/c69 d0/db/cf7 0 2024-03-24T10:46:15.333 INFO:tasks.workunit.client.1.smithi096.stdout:9/496: creat d9/db/fa4 x:0 0 0 2024-03-24T10:46:15.337 INFO:tasks.workunit.client.1.smithi096.stdout:7/733: chown d0/d13/df0/ced 314143 1 2024-03-24T10:46:15.339 INFO:tasks.workunit.client.1.smithi096.stdout:9/497: mknod d9/d15/d26/d29/ca5 0 2024-03-24T10:46:15.339 INFO:tasks.workunit.client.1.smithi096.stdout:9/498: read d9/d15/d26/f33 [3317946,93029] 0 2024-03-24T10:46:15.340 INFO:tasks.workunit.client.1.smithi096.stdout:4/476: dwrite d1/d11/f19 [0,4194304] 0 2024-03-24T10:46:15.340 INFO:tasks.workunit.client.1.smithi096.stdout:4/477: creat d1/d32/d4c/d7e/f9e x:0 0 0 2024-03-24T10:46:15.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.341+0000 7f8b6cfea700 1 -- 172.21.15.94:0/1821191880 --> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8b681023c0 con 0x7f8b500692a0 2024-03-24T10:46:15.343 INFO:tasks.workunit.client.1.smithi096.stdout:2/518: mkdir d5/d38/dbd 0 2024-03-24T10:46:15.343 INFO:tasks.workunit.client.1.smithi096.stdout:2/519: write d5/d7c/f95 [708311,61007] 0 2024-03-24T10:46:15.343 INFO:tasks.workunit.client.1.smithi096.stdout:2/520: fdatasync d5/dc/d12/d23/f98 0 2024-03-24T10:46:15.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.343+0000 7f8b56ffd700 1 -- 172.21.15.94:0/1821191880 <== mgr.14232 v2:172.21.15.94:6800/2184357820 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+328 (secure 0 0 0) 0x7f8b681023c0 con 0x7f8b500692a0 2024-03-24T10:46:15.344 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-03-24T10:46:15.344 INFO:teuthology.orchestra.run.smithi094.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919", 2024-03-24T10:46:15.344 INFO:teuthology.orchestra.run.smithi094.stdout: "in_progress": true, 2024-03-24T10:46:15.344 INFO:teuthology.orchestra.run.smithi094.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T10:46:15.344 INFO:teuthology.orchestra.run.smithi094.stdout: "services_complete": [], 2024-03-24T10:46:15.344 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": "", 2024-03-24T10:46:15.344 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "Currently upgrading mgr daemons", 2024-03-24T10:46:15.344 INFO:teuthology.orchestra.run.smithi094.stdout: "is_paused": false 2024-03-24T10:46:15.344 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-03-24T10:46:15.345 INFO:tasks.workunit.client.1.smithi096.stdout:7/734: creat d0/d13/df0/d3c/d80/ff8 x:0 0 0 2024-03-24T10:46:15.346 INFO:tasks.workunit.client.1.smithi096.stdout:6/568: dwrite d8/d10/d13/f55 [0,4194304] 0 2024-03-24T10:46:15.346 INFO:tasks.workunit.client.1.smithi096.stdout:6/569: truncate d8/d10/d76/f8d 717597 0 2024-03-24T10:46:15.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.347+0000 7f8b6cfea700 1 -- 172.21.15.94:0/1821191880 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f8b500692a0 msgr2=0x7f8b5006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:15.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.347+0000 7f8b6cfea700 1 --2- 172.21.15.94:0/1821191880 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f8b500692a0 0x7f8b5006b750 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c00b970 tx=0x7f8b5c009e30 comp rx=0 tx=0).stop 2024-03-24T10:46:15.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.347+0000 7f8b6cfea700 1 -- 172.21.15.94:0/1821191880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b680fcef0 msgr2=0x7f8b68071e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:15.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.347+0000 7f8b6cfea700 1 --2- 172.21.15.94:0/1821191880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b680fcef0 0x7f8b68071e40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8b58002a50 tx=0x7f8b580046b0 comp rx=0 tx=0).stop 2024-03-24T10:46:15.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.347+0000 7f8b6cfea700 1 -- 172.21.15.94:0/1821191880 shutdown_connections 2024-03-24T10:46:15.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.347+0000 7f8b6cfea700 1 --2- 172.21.15.94:0/1821191880 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f8b500692a0 0x7f8b5006b750 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:15.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.347+0000 7f8b6cfea700 1 --2- 172.21.15.94:0/1821191880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b680fcef0 0x7f8b68071e40 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:15.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.347+0000 7f8b6cfea700 1 --2- 172.21.15.94:0/1821191880 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b680fe060 0x7f8b68070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:15.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.347+0000 7f8b6cfea700 1 -- 172.21.15.94:0/1821191880 >> 172.21.15.94:0/1821191880 conn(0x7f8b680f85b0 msgr2=0x7f8b680f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:46:15.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.347+0000 7f8b6cfea700 1 -- 172.21.15.94:0/1821191880 shutdown_connections 2024-03-24T10:46:15.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.347+0000 7f8b6cfea700 1 -- 172.21.15.94:0/1821191880 wait complete. 2024-03-24T10:46:15.350 INFO:tasks.workunit.client.1.smithi096.stdout:4/478: creat d1/d2b/d6b/d80/f9f x:0 0 0 2024-03-24T10:46:15.351 INFO:tasks.workunit.client.1.smithi096.stdout:2/521: mknod d5/d79/d9f/cbe 0 2024-03-24T10:46:15.353 INFO:tasks.workunit.client.1.smithi096.stdout:7/735: creat d0/db/ff9 x:0 0 0 2024-03-24T10:46:15.356 INFO:tasks.workunit.client.1.smithi096.stdout:2/522: symlink d5/dc/d6b/d72/d91/lbf 0 2024-03-24T10:46:15.358 INFO:tasks.workunit.client.1.smithi096.stdout:6/570: dread d8/d10/d13/f55 [0,4194304] 0 2024-03-24T10:46:15.358 INFO:tasks.workunit.client.1.smithi096.stdout:9/499: dwrite d9/db/f14 [0,4194304] 0 2024-03-24T10:46:15.358 INFO:tasks.workunit.client.1.smithi096.stdout:6/571: write d8/d12/d28/d36/fae [454047,83185] 0 2024-03-24T10:46:15.358 INFO:tasks.workunit.client.1.smithi096.stdout:2/523: getdents d5/de/d2a/d5e 0 2024-03-24T10:46:15.359 INFO:tasks.workunit.client.1.smithi096.stdout:2/524: fdatasync d5/dc/d12/f20 0 2024-03-24T10:46:15.362 INFO:tasks.workunit.client.1.smithi096.stdout:7/736: dread d0/f9e [0,4194304] 0 2024-03-24T10:46:15.365 INFO:tasks.workunit.client.1.smithi096.stdout:9/500: link d9/d15/d26/d29/d73/d49/d86/f9e d9/d15/fa6 0 2024-03-24T10:46:15.366 INFO:tasks.workunit.client.1.smithi096.stdout:2/525: symlink d5/d7c/lc0 0 2024-03-24T10:46:15.371 INFO:tasks.workunit.client.1.smithi096.stdout:9/501: rmdir d9/d60 39 2024-03-24T10:46:15.372 INFO:tasks.workunit.client.1.smithi096.stdout:9/502: chown d9/c37 0 1 2024-03-24T10:46:15.376 INFO:tasks.workunit.client.1.smithi096.stdout:9/503: symlink d9/db/la7 0 2024-03-24T10:46:15.379 INFO:tasks.workunit.client.1.smithi096.stdout:2/526: dwrite d5/d38/fa8 [0,4194304] 0 2024-03-24T10:46:15.379 INFO:tasks.workunit.client.1.smithi096.stdout:2/527: fsync d5/d7c/f7e 0 2024-03-24T10:46:15.379 INFO:tasks.workunit.client.1.smithi096.stdout:2/528: chown d5/dc/d12/d23/f98 568307719 1 2024-03-24T10:46:15.383 INFO:tasks.workunit.client.1.smithi096.stdout:2/529: mknod d5/d79/db5/cc1 0 2024-03-24T10:46:15.384 INFO:tasks.workunit.client.1.smithi096.stdout:9/504: read d9/db/f14 [1194347,20893] 0 2024-03-24T10:46:15.384 INFO:tasks.workunit.client.1.smithi096.stdout:9/505: stat d9/d15 0 2024-03-24T10:46:15.384 INFO:tasks.workunit.client.1.smithi096.stdout:9/506: dread - d9/d15/d26/d29/d73/d43/d48/f98 zero size 2024-03-24T10:46:15.386 INFO:tasks.workunit.client.1.smithi096.stdout:2/530: creat d5/d79/db5/fc2 x:0 0 0 2024-03-24T10:46:15.390 INFO:tasks.workunit.client.1.smithi096.stdout:9/507: rmdir d9/d15/d26/d29/d73/d49 39 2024-03-24T10:46:15.394 INFO:tasks.workunit.client.1.smithi096.stdout:9/508: symlink d9/db/la8 0 2024-03-24T10:46:15.395 INFO:tasks.workunit.client.1.smithi096.stdout:2/531: write d5/d38/fa8 [2760230,108018] 0 2024-03-24T10:46:15.395 INFO:tasks.workunit.client.1.smithi096.stdout:2/532: truncate d5/dc/d12/f4a 5847536 0 2024-03-24T10:46:15.395 INFO:tasks.workunit.client.1.smithi096.stdout:2/533: write d5/f5f [795708,28240] 0 2024-03-24T10:46:15.397 INFO:tasks.workunit.client.1.smithi096.stdout:9/509: truncate d9/db/d6d/d4f/f63 1150095 0 2024-03-24T10:46:15.398 INFO:tasks.workunit.client.1.smithi096.stdout:9/510: mknod d9/d15/ca9 0 2024-03-24T10:46:15.399 INFO:tasks.workunit.client.1.smithi096.stdout:2/534: getdents d5/dc/d12 0 2024-03-24T10:46:15.400 INFO:tasks.workunit.client.1.smithi096.stdout:2/535: truncate d5/f1f 1192429 0 2024-03-24T10:46:15.400 INFO:tasks.workunit.client.1.smithi096.stdout:2/536: fsync d5/dc/f58 0 2024-03-24T10:46:15.415 INFO:tasks.workunit.client.1.smithi096.stdout:2/537: dwrite d5/dc/d12/d23/f2c [0,4194304] 0 2024-03-24T10:46:15.417 INFO:tasks.workunit.client.1.smithi096.stdout:2/538: symlink d5/d79/d9f/dad/lc3 0 2024-03-24T10:46:15.420 INFO:tasks.workunit.client.1.smithi096.stdout:2/539: creat d5/d38/fc4 x:0 0 0 2024-03-24T10:46:15.436 INFO:tasks.workunit.client.1.smithi096.stdout:2/540: dwrite d5/dc/d12/d23/f2c [0,4194304] 0 2024-03-24T10:46:15.436 INFO:tasks.workunit.client.1.smithi096.stdout:2/541: dread - d5/fb9 zero size 2024-03-24T10:46:15.436 INFO:tasks.workunit.client.1.smithi096.stdout:2/542: readlink d5/dc/d6b/d72/d91/lbf 0 2024-03-24T10:46:15.452 INFO:tasks.workunit.client.1.smithi096.stdout:2/543: dwrite d5/d7c/d88/f70 [0,4194304] 0 2024-03-24T10:46:15.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.466+0000 7f456b897700 1 -- 172.21.15.94:0/2224366224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45640fdfe0 msgr2=0x7f45640fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:15.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.466+0000 7f456b897700 1 --2- 172.21.15.94:0/2224366224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45640fdfe0 0x7f45640fe450 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4554004940 tx=0x7f4554009e30 comp rx=0 tx=0).stop 2024-03-24T10:46:15.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.466+0000 7f456b897700 1 -- 172.21.15.94:0/2224366224 shutdown_connections 2024-03-24T10:46:15.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.466+0000 7f456b897700 1 --2- 172.21.15.94:0/2224366224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45640fdfe0 0x7f45640fe450 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:15.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.466+0000 7f456b897700 1 --2- 172.21.15.94:0/2224366224 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45640fce70 0x7f45640fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:15.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.466+0000 7f456b897700 1 -- 172.21.15.94:0/2224366224 >> 172.21.15.94:0/2224366224 conn(0x7f45640f8550 msgr2=0x7f45640fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:46:15.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.466+0000 7f456b897700 1 -- 172.21.15.94:0/2224366224 shutdown_connections 2024-03-24T10:46:15.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.466+0000 7f456b897700 1 -- 172.21.15.94:0/2224366224 wait complete. 2024-03-24T10:46:15.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.467+0000 7f456b897700 1 Processor -- start 2024-03-24T10:46:15.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.467+0000 7f456b897700 1 -- start start 2024-03-24T10:46:15.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.467+0000 7f456b897700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45640fce70 0x7f456410d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:46:15.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.467+0000 7f456b897700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45640fdfe0 0x7f456410de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:46:15.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.467+0000 7f456b897700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456410e460 con 0x7f45640fdfe0 2024-03-24T10:46:15.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.467+0000 7f456b897700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456410e5d0 con 0x7f45640fce70 2024-03-24T10:46:15.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.467+0000 7f4568e32700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45640fdfe0 0x7f456410de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:46:15.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.467+0000 7f4568e32700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45640fdfe0 0x7f456410de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54398/0 (socket says 172.21.15.94:54398) 2024-03-24T10:46:15.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.467+0000 7f4568e32700 1 -- 172.21.15.94:0/1785927985 learned_addr learned my addr 172.21.15.94:0/1785927985 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:46:15.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.467+0000 7f4569633700 1 --2- 172.21.15.94:0/1785927985 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45640fce70 0x7f456410d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:46:15.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.467+0000 7f4568e32700 1 -- 172.21.15.94:0/1785927985 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45640fce70 msgr2=0x7f456410d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:15.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.467+0000 7f4568e32700 1 --2- 172.21.15.94:0/1785927985 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45640fce70 0x7f456410d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:15.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.467+0000 7f4568e32700 1 -- 172.21.15.94:0/1785927985 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4554005040 con 0x7f45640fdfe0 2024-03-24T10:46:15.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.467+0000 7f4568e32700 1 --2- 172.21.15.94:0/1785927985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45640fdfe0 0x7f456410de60 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f455400b3b0 tx=0x7f4554009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:46:15.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.467+0000 7f4559ffb700 1 -- 172.21.15.94:0/1785927985 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f455400e040 con 0x7f45640fdfe0 2024-03-24T10:46:15.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.467+0000 7f4559ffb700 1 -- 172.21.15.94:0/1785927985 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4554004290 con 0x7f45640fdfe0 2024-03-24T10:46:15.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.467+0000 7f456b897700 1 -- 172.21.15.94:0/1785927985 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4564112fd0 con 0x7f45640fdfe0 2024-03-24T10:46:15.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.467+0000 7f4559ffb700 1 -- 172.21.15.94:0/1785927985 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f455401c3f0 con 0x7f45640fdfe0 2024-03-24T10:46:15.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.468+0000 7f456b897700 1 -- 172.21.15.94:0/1785927985 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4564113560 con 0x7f45640fdfe0 2024-03-24T10:46:15.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.469+0000 7f456b897700 1 -- 172.21.15.94:0/1785927985 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f456404c710 con 0x7f45640fdfe0 2024-03-24T10:46:15.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.469+0000 7f4559ffb700 1 -- 172.21.15.94:0/1785927985 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f455400bbf0 con 0x7f45640fdfe0 2024-03-24T10:46:15.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.469+0000 7f4559ffb700 1 --2- 172.21.15.94:0/1785927985 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f45500692a0 0x7f455006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:46:15.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.470+0000 7f4559ffb700 1 -- 172.21.15.94:0/1785927985 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f4554088430 con 0x7f45640fdfe0 2024-03-24T10:46:15.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.470+0000 7f4569633700 1 --2- 172.21.15.94:0/1785927985 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f45500692a0 0x7f455006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:46:15.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.470+0000 7f4569633700 1 --2- 172.21.15.94:0/1785927985 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f45500692a0 0x7f455006b750 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4560004f40 tx=0x7f456000b040 comp rx=0 tx=0).ready entity=mgr.14232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:46:15.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.474+0000 7f4559ffb700 1 -- 172.21.15.94:0/1785927985 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f4554058240 con 0x7f45640fdfe0 2024-03-24T10:46:15.506 INFO:tasks.workunit.client.1.smithi096.stdout:2/544: rename d5/d7c/d88/fac to d5/dc/d12/fc5 0 2024-03-24T10:46:15.506 INFO:tasks.workunit.client.1.smithi096.stdout:2/545: write d5/dc/d6b/d72/f9c [302306,122762] 0 2024-03-24T10:46:15.508 INFO:tasks.workunit.client.1.smithi096.stdout:2/546: symlink d5/de/lc6 0 2024-03-24T10:46:15.510 INFO:tasks.workunit.client.1.smithi096.stdout:2/547: creat d5/d7c/d8f/fc7 x:0 0 0 2024-03-24T10:46:15.510 INFO:tasks.workunit.client.1.smithi096.stdout:2/548: mknod d5/d7c/d88/cc8 0 2024-03-24T10:46:15.512 INFO:tasks.workunit.client.1.smithi096.stdout:2/549: rename d5/de/d2a/f37 to d5/d7c/d8f/fc9 0 2024-03-24T10:46:15.512 INFO:tasks.workunit.client.1.smithi096.stdout:2/550: dread - d5/d7c/d88/d90/fb1 zero size 2024-03-24T10:46:15.512 INFO:tasks.workunit.client.1.smithi096.stdout:2/551: fdatasync d5/de/f51 0 2024-03-24T10:46:15.514 INFO:tasks.workunit.client.1.smithi096.stdout:2/552: mkdir d5/de/d2a/d5e/dca 0 2024-03-24T10:46:15.529 INFO:tasks.workunit.client.1.smithi096.stdout:2/553: dwrite d5/dc/f1c [0,4194304] 0 2024-03-24T10:46:15.532 INFO:tasks.workunit.client.1.smithi096.stdout:2/554: rename d5/de/d2a/f78 to d5/d7c/d88/d90/fcb 0 2024-03-24T10:46:15.555 INFO:tasks.workunit.client.1.smithi096.stdout:2/555: dwrite d5/fa7 [0,4194304] 0 2024-03-24T10:46:15.562 INFO:tasks.workunit.client.1.smithi096.stdout:2/556: dread f2 [0,4194304] 0 2024-03-24T10:46:15.567 INFO:tasks.workunit.client.1.smithi096.stdout:2/557: mknod d5/de/ccc 0 2024-03-24T10:46:15.570 INFO:tasks.workunit.client.1.smithi096.stdout:2/558: link d5/de/d2a/fa6 d5/de/d2a/d5e/fcd 0 2024-03-24T10:46:15.582 INFO:tasks.workunit.client.1.smithi096.stdout:2/559: dwrite d5/fb [4194304,4194304] 0 2024-03-24T10:46:15.585 INFO:tasks.workunit.client.1.smithi096.stdout:1/447: sync 2024-03-24T10:46:15.585 INFO:tasks.workunit.client.1.smithi096.stdout:5/552: sync 2024-03-24T10:46:15.585 INFO:tasks.workunit.client.1.smithi096.stdout:1/448: readlink d3/d23/d71/d7b/l7f 0 2024-03-24T10:46:15.585 INFO:tasks.workunit.client.1.smithi096.stdout:2/560: rmdir d5/dc/d12/d23 39 2024-03-24T10:46:15.587 INFO:tasks.workunit.client.1.smithi096.stdout:5/553: truncate d2/d7/f86 1743278 0 2024-03-24T10:46:15.589 INFO:tasks.workunit.client.1.smithi096.stdout:1/449: rename d3/d23/d71/d7b/d98 to d3/d23/d8b/d99 0 2024-03-24T10:46:15.589 INFO:tasks.workunit.client.1.smithi096.stdout:1/450: write d3/ff [1009490,14878] 0 2024-03-24T10:46:15.590 INFO:tasks.workunit.client.1.smithi096.stdout:2/561: creat d5/de/fce x:0 0 0 2024-03-24T10:46:15.591 INFO:tasks.workunit.client.1.smithi096.stdout:1/451: dread d3/d23/d2f/f83 [0,4194304] 0 2024-03-24T10:46:15.593 INFO:tasks.workunit.client.1.smithi096.stdout:1/452: mkdir d3/d84/d9a 0 2024-03-24T10:46:15.596 INFO:tasks.workunit.client.1.smithi096.stdout:5/554: mkdir d2/d7/dba 0 2024-03-24T10:46:15.598 INFO:tasks.workunit.client.1.smithi096.stdout:1/453: mkdir d3/d9b 0 2024-03-24T10:46:15.598 INFO:tasks.workunit.client.1.smithi096.stdout:2/562: write d5/de/d2a/f5a [1107652,115919] 0 2024-03-24T10:46:15.601 INFO:tasks.workunit.client.1.smithi096.stdout:1/454: truncate d3/f15 6954489 0 2024-03-24T10:46:15.606 INFO:tasks.workunit.client.1.smithi096.stdout:2/563: truncate d5/d7c/d88/f69 1727153 0 2024-03-24T10:46:15.608 INFO:tasks.workunit.client.1.smithi096.stdout:2/564: dread d5/de/d2a/f40 [0,4194304] 0 2024-03-24T10:46:15.609 INFO:tasks.workunit.client.1.smithi096.stdout:1/455: getdents d3/d73 0 2024-03-24T10:46:15.610 INFO:tasks.workunit.client.1.smithi096.stdout:1/456: dread d3/d23/d8b/f35 [0,4194304] 0 2024-03-24T10:46:15.611 INFO:tasks.workunit.client.1.smithi096.stdout:5/555: dwrite d2/d29/d32/d33/d3c/d3d/fb5 [0,4194304] 0 2024-03-24T10:46:15.611 INFO:tasks.workunit.client.1.smithi096.stdout:5/556: readlink d2/d7/la4 0 2024-03-24T10:46:15.612 INFO:tasks.workunit.client.1.smithi096.stdout:1/457: mkdir d3/d23/d8b/d5c/d9c 0 2024-03-24T10:46:15.620 INFO:tasks.workunit.client.1.smithi096.stdout:5/557: mkdir d2/d29/d32/dbb 0 2024-03-24T10:46:15.620 INFO:tasks.workunit.client.1.smithi096.stdout:1/458: symlink d3/d23/d8b/l9d 0 2024-03-24T10:46:15.621 INFO:tasks.workunit.client.1.smithi096.stdout:1/459: getdents d3/d23/d8b/d8c 0 2024-03-24T10:46:15.622 INFO:tasks.workunit.client.1.smithi096.stdout:5/558: creat d2/d29/d32/db9/fbc x:0 0 0 2024-03-24T10:46:15.622 INFO:tasks.workunit.client.1.smithi096.stdout:2/565: dwrite d5/dc/d6b/f8a [0,4194304] 0 2024-03-24T10:46:15.622 INFO:tasks.workunit.client.1.smithi096.stdout:1/460: symlink d3/d23/d8b/d8c/l9e 0 2024-03-24T10:46:15.623 INFO:tasks.workunit.client.1.smithi096.stdout:1/461: read d3/d23/d8b/f35 [41727,31987] 0 2024-03-24T10:46:15.623 INFO:tasks.workunit.client.1.smithi096.stdout:5/559: mkdir d2/dbd 0 2024-03-24T10:46:15.629 INFO:tasks.workunit.client.1.smithi096.stdout:2/566: mkdir d5/dc/d6b/dcf 0 2024-03-24T10:46:15.630 INFO:tasks.workunit.client.1.smithi096.stdout:2/567: write d5/dc/f2e [3716268,72882] 0 2024-03-24T10:46:15.630 INFO:tasks.workunit.client.1.smithi096.stdout:2/568: creat d5/dc/d6b/d72/d91/fd0 x:0 0 0 2024-03-24T10:46:15.632 INFO:tasks.workunit.client.1.smithi096.stdout:2/569: rename d5/de/d2a/d5e/fa9 to d5/dc/d12/fd1 0 2024-03-24T10:46:15.635 INFO:tasks.workunit.client.1.smithi096.stdout:2/570: mknod d5/dc/d6b/cd2 0 2024-03-24T10:46:15.637 INFO:tasks.workunit.client.1.smithi096.stdout:2/571: getdents d5/d79/d9f/dad 0 2024-03-24T10:46:15.639 INFO:tasks.workunit.client.1.smithi096.stdout:2/572: symlink d5/de/d2a/d5e/ld3 0 2024-03-24T10:46:15.647 INFO:tasks.workunit.client.1.smithi096.stdout:1/462: dwrite d3/d23/d71/f8a [0,4194304] 0 2024-03-24T10:46:15.647 INFO:tasks.workunit.client.1.smithi096.stdout:5/560: dwrite d2/f8b [0,4194304] 0 2024-03-24T10:46:15.647 INFO:tasks.workunit.client.1.smithi096.stdout:5/561: chown d2/d29/d32/db9 244 1 2024-03-24T10:46:15.656 INFO:tasks.workunit.client.1.smithi096.stdout:1/463: write d3/f15 [3052552,97036] 0 2024-03-24T10:46:15.674 INFO:tasks.workunit.client.1.smithi096.stdout:2/573: dwrite d5/d38/f9b [0,4194304] 0 2024-03-24T10:46:15.675 INFO:tasks.workunit.client.1.smithi096.stdout:5/562: dwrite d2/d7/d8/f56 [0,4194304] 0 2024-03-24T10:46:15.675 INFO:tasks.workunit.client.1.smithi096.stdout:2/574: read d5/de/d2a/d5e/f8d [254678,107097] 0 2024-03-24T10:46:15.675 INFO:tasks.workunit.client.1.smithi096.stdout:2/575: write d5/dc/fb7 [418821,10566] 0 2024-03-24T10:46:15.676 INFO:tasks.workunit.client.1.smithi096.stdout:2/576: mkdir d5/d79/dd4 0 2024-03-24T10:46:15.703 INFO:tasks.workunit.client.1.smithi096.stdout:5/563: dwrite d2/d7/fa7 [0,4194304] 0 2024-03-24T10:46:15.703 INFO:tasks.workunit.client.1.smithi096.stdout:2/577: dwrite d5/d7c/f8c [0,4194304] 0 2024-03-24T10:46:15.707 INFO:tasks.workunit.client.1.smithi096.stdout:5/564: rename d2/d7/d8/d18/d4d/l81 to d2/d29/d32/d3f/d5f/lbe 0 2024-03-24T10:46:15.707 INFO:tasks.workunit.client.1.smithi096.stdout:5/565: truncate d2/d7/d69/f91 2884766 0 2024-03-24T10:46:15.707 INFO:tasks.workunit.client.1.smithi096.stdout:5/566: write d2/d7/f86 [1958547,54355] 0 2024-03-24T10:46:15.711 INFO:tasks.workunit.client.1.smithi096.stdout:2/578: creat d5/de/d2a/fd5 x:0 0 0 2024-03-24T10:46:15.711 INFO:tasks.workunit.client.1.smithi096.stdout:2/579: read - d5/de/d2a/d5e/fcd zero size 2024-03-24T10:46:15.716 INFO:tasks.workunit.client.1.smithi096.stdout:5/567: write d2/d7/d8/f12 [826896,54741] 0 2024-03-24T10:46:15.716 INFO:tasks.workunit.client.1.smithi096.stdout:5/568: truncate d2/fa6 877691 0 2024-03-24T10:46:15.716 INFO:tasks.workunit.client.1.smithi096.stdout:5/569: write d2/d7/f4a [405883,12388] 0 2024-03-24T10:46:15.725 INFO:tasks.workunit.client.1.smithi096.stdout:5/570: read d2/d49/f5d [5233546,23139] 0 2024-03-24T10:46:15.726 INFO:tasks.workunit.client.1.smithi096.stdout:2/580: dwrite f2 [0,4194304] 0 2024-03-24T10:46:15.727 INFO:tasks.workunit.client.1.smithi096.stdout:5/571: symlink d2/d29/d32/d33/d37/dab/lbf 0 2024-03-24T10:46:15.729 INFO:tasks.workunit.client.1.smithi096.stdout:2/581: mkdir d5/de/d2a/d5e/dd6 0 2024-03-24T10:46:15.746 INFO:tasks.workunit.client.1.smithi096.stdout:5/572: dwrite d2/d7/d8/df/f78 [0,4194304] 0 2024-03-24T10:46:15.747 INFO:tasks.workunit.client.1.smithi096.stdout:5/573: fdatasync d2/d29/f54 0 2024-03-24T10:46:15.748 INFO:tasks.workunit.client.1.smithi096.stdout:5/574: readlink d2/d7/l2f 0 2024-03-24T10:46:15.748 INFO:tasks.workunit.client.1.smithi096.stdout:5/575: write d2/d7/d8/df/d14/f85 [800651,71095] 0 2024-03-24T10:46:15.751 INFO:tasks.workunit.client.1.smithi096.stdout:2/582: dwrite d5/d7c/d88/d90/fb1 [0,4194304] 0 2024-03-24T10:46:15.752 INFO:tasks.workunit.client.1.smithi096.stdout:2/583: creat d5/d7c/d88/fd7 x:0 0 0 2024-03-24T10:46:15.752 INFO:tasks.workunit.client.1.smithi096.stdout:2/584: write d5/de/f51 [1612013,12246] 0 2024-03-24T10:46:15.752 INFO:tasks.workunit.client.1.smithi096.stdout:2/585: dread - d5/de/d2a/f62 zero size 2024-03-24T10:46:15.755 INFO:tasks.workunit.client.1.smithi096.stdout:2/586: rename d5/dc/d6b/c9e to d5/d7c/cd8 0 2024-03-24T10:46:15.755 INFO:tasks.workunit.client.1.smithi096.stdout:2/587: readlink d5/de/l3f 0 2024-03-24T10:46:15.762 INFO:tasks.workunit.client.1.smithi096.stdout:3/586: sync 2024-03-24T10:46:15.762 INFO:tasks.workunit.client.1.smithi096.stdout:8/552: sync 2024-03-24T10:46:15.762 INFO:tasks.workunit.client.1.smithi096.stdout:0/544: sync 2024-03-24T10:46:15.762 INFO:tasks.workunit.client.1.smithi096.stdout:4/479: sync 2024-03-24T10:46:15.765 INFO:tasks.workunit.client.1.smithi096.stdout:5/576: dwrite d2/f39 [0,4194304] 0 2024-03-24T10:46:15.774 INFO:tasks.workunit.client.1.smithi096.stdout:0/545: symlink d0/d11/lae 0 2024-03-24T10:46:15.778 INFO:tasks.workunit.client.1.smithi096.stdout:5/577: mkdir d2/d29/d32/d3f/d5f/dc0 0 2024-03-24T10:46:15.779 INFO:tasks.workunit.client.1.smithi096.stdout:5/578: write d2/d29/d32/d33/d37/dab/fb1 [860024,91218] 0 2024-03-24T10:46:15.780 INFO:tasks.workunit.client.1.smithi096.stdout:0/546: creat d0/d11/d20/d9e/d80/faf x:0 0 0 2024-03-24T10:46:15.787 INFO:tasks.workunit.client.1.smithi096.stdout:0/547: rename d0/d11/d16/d21/d25/fa4 to d0/fb0 0 2024-03-24T10:46:15.789 INFO:tasks.workunit.client.1.smithi096.stdout:0/548: link d0/d11/d16/d21/d81/d7f/f5e d0/d11/d20/d2b/fb1 0 2024-03-24T10:46:15.789 INFO:tasks.workunit.client.1.smithi096.stdout:0/549: fsync d0/d11/d20/f1f 0 2024-03-24T10:46:15.789 INFO:tasks.workunit.client.1.smithi096.stdout:0/550: creat d0/d11/d20/d56/fb2 x:0 0 0 2024-03-24T10:46:15.790 INFO:tasks.workunit.client.1.smithi096.stdout:0/551: creat d0/d11/d24/d2c/fb3 x:0 0 0 2024-03-24T10:46:15.792 INFO:tasks.workunit.client.1.smithi096.stdout:0/552: creat d0/d11/d20/d2b/fb4 x:0 0 0 2024-03-24T10:46:15.792 INFO:tasks.workunit.client.1.smithi096.stdout:8/553: dwrite d8/d11/d22/d4c/d65/d1a/f2d [0,4194304] 0 2024-03-24T10:46:15.792 INFO:tasks.workunit.client.1.smithi096.stdout:3/587: dwrite d0/d13/d1e/f2f [0,4194304] 0 2024-03-24T10:46:15.792 INFO:tasks.workunit.client.1.smithi096.stdout:8/554: write d8/d11/d22/d4c/d65/d37/f39 [4691860,130135] 0 2024-03-24T10:46:15.792 INFO:tasks.workunit.client.1.smithi096.stdout:8/555: read - d8/d11/d22/d4c/d82/fb2 zero size 2024-03-24T10:46:15.792 INFO:tasks.workunit.client.1.smithi096.stdout:4/480: dwrite d1/d32/d4c/d7e/f9e [0,4194304] 0 2024-03-24T10:46:15.792 INFO:tasks.workunit.client.1.smithi096.stdout:4/481: readlink d1/d2b/l30 0 2024-03-24T10:46:15.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.791+0000 7f456b897700 1 -- 172.21.15.94:0/1785927985 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f45641058d0 con 0x7f45640fdfe0 2024-03-24T10:46:15.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.792+0000 7f4559ffb700 1 -- 172.21.15.94:0/1785927985 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f4554057dd0 con 0x7f45640fdfe0 2024-03-24T10:46:15.793 INFO:teuthology.orchestra.run.smithi094.stdout:HEALTH_OK 2024-03-24T10:46:15.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.794+0000 7f456b897700 1 -- 172.21.15.94:0/1785927985 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f45500692a0 msgr2=0x7f455006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:15.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.794+0000 7f456b897700 1 --2- 172.21.15.94:0/1785927985 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f45500692a0 0x7f455006b750 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4560004f40 tx=0x7f456000b040 comp rx=0 tx=0).stop 2024-03-24T10:46:15.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.794+0000 7f456b897700 1 -- 172.21.15.94:0/1785927985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45640fdfe0 msgr2=0x7f456410de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:15.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.794+0000 7f456b897700 1 --2- 172.21.15.94:0/1785927985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45640fdfe0 0x7f456410de60 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f455400b3b0 tx=0x7f4554009e30 comp rx=0 tx=0).stop 2024-03-24T10:46:15.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.794+0000 7f456b897700 1 -- 172.21.15.94:0/1785927985 shutdown_connections 2024-03-24T10:46:15.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.794+0000 7f456b897700 1 --2- 172.21.15.94:0/1785927985 >> [v2:172.21.15.94:6800/2184357820,v1:172.21.15.94:6801/2184357820] conn(0x7f45500692a0 0x7f455006b750 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:15.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.794+0000 7f456b897700 1 --2- 172.21.15.94:0/1785927985 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45640fce70 0x7f456410d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:15.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.794+0000 7f456b897700 1 --2- 172.21.15.94:0/1785927985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45640fdfe0 0x7f456410de60 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:15.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.794+0000 7f456b897700 1 -- 172.21.15.94:0/1785927985 >> 172.21.15.94:0/1785927985 conn(0x7f45640f8550 msgr2=0x7f45640f9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:46:15.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.794+0000 7f456b897700 1 -- 172.21.15.94:0/1785927985 shutdown_connections 2024-03-24T10:46:15.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:15.794+0000 7f456b897700 1 -- 172.21.15.94:0/1785927985 wait complete. 2024-03-24T10:46:15.796 INFO:tasks.workunit.client.1.smithi096.stdout:4/482: dread d1/d2b/d67/f6a [0,4194304] 0 2024-03-24T10:46:15.797 INFO:tasks.workunit.client.1.smithi096.stdout:3/588: symlink d0/d13/d45/d7d/d39/d48/d85/lc4 0 2024-03-24T10:46:15.801 INFO:tasks.workunit.client.1.smithi096.stdout:0/553: read d0/d11/d24/d2c/f76 [3975861,14303] 0 2024-03-24T10:46:15.801 INFO:tasks.workunit.client.1.smithi096.stdout:8/556: unlink d8/d11/d22/d4c/d82/l41 0 2024-03-24T10:46:15.805 INFO:tasks.workunit.client.1.smithi096.stdout:4/483: creat d1/d2b/d89/fa0 x:0 0 0 2024-03-24T10:46:15.808 INFO:tasks.workunit.client.1.smithi096.stdout:3/589: symlink d0/lc5 0 2024-03-24T10:46:15.808 INFO:tasks.workunit.client.1.smithi096.stdout:3/590: stat d0/d13/d1e/l2b 0 2024-03-24T10:46:15.808 INFO:tasks.workunit.client.1.smithi096.stdout:3/591: chown d0/d9/d22/c40 0 1 2024-03-24T10:46:15.816 INFO:tasks.workunit.client.1.smithi096.stdout:8/557: dwrite d8/d11/d22/d4c/d65/d1d/f30 [0,4194304] 0 2024-03-24T10:46:15.818 INFO:tasks.workunit.client.1.smithi096.stdout:0/554: symlink d0/d11/d43/lb5 0 2024-03-24T10:46:15.819 INFO:tasks.workunit.client.1.smithi096.stdout:0/555: fdatasync d0/f73 0 2024-03-24T10:46:15.819 INFO:tasks.workunit.client.1.smithi096.stdout:8/558: dread d8/d11/d22/d4c/d65/d1a/f6c [0,4194304] 0 2024-03-24T10:46:15.819 INFO:tasks.workunit.client.1.smithi096.stdout:8/559: dread - d8/d11/d22/d4c/faf zero size 2024-03-24T10:46:15.819 INFO:tasks.workunit.client.1.smithi096.stdout:8/560: fsync d8/d11/d44/fa7 0 2024-03-24T10:46:15.823 INFO:tasks.workunit.client.1.smithi096.stdout:4/484: rename d1/d2b/f2f to d1/d2b/d6b/d80/d9c/fa1 0 2024-03-24T10:46:15.827 INFO:tasks.workunit.client.1.smithi096.stdout:0/556: mkdir d0/d11/d16/d21/d78/db6 0 2024-03-24T10:46:15.828 INFO:tasks.workunit.client.1.smithi096.stdout:8/561: truncate d8/d11/d22/d4c/d82/f2c 285451 0 2024-03-24T10:46:15.837 INFO:tasks.workunit.client.1.smithi096.stdout:0/557: getdents d0/d11 0 2024-03-24T10:46:15.837 INFO:tasks.workunit.client.1.smithi096.stdout:0/558: readlink d0/d11/d43/d75/l85 0 2024-03-24T10:46:15.838 INFO:tasks.workunit.client.1.smithi096.stdout:0/559: dread d0/d11/d20/d2b/f3d [0,4194304] 0 2024-03-24T10:46:15.839 INFO:tasks.workunit.client.1.smithi096.stdout:3/592: dwrite d0/d13/f46 [0,4194304] 0 2024-03-24T10:46:15.840 INFO:tasks.workunit.client.1.smithi096.stdout:3/593: write d0/d13/d45/d47/f64 [756948,123423] 0 2024-03-24T10:46:15.841 INFO:tasks.workunit.client.1.smithi096.stdout:0/560: symlink d0/d11/d24/lb7 0 2024-03-24T10:46:15.844 INFO:tasks.workunit.client.1.smithi096.stdout:3/594: link d0/d13/c7c d0/d13/d45/d7d/d39/d48/d8d/cc6 0 2024-03-24T10:46:15.844 INFO:tasks.workunit.client.1.smithi096.stdout:3/595: creat d0/d13/d45/d47/fc7 x:0 0 0 2024-03-24T10:46:15.844 INFO:tasks.workunit.client.1.smithi096.stdout:3/596: write d0/d9/d22/fa8 [108576,109936] 0 2024-03-24T10:46:15.844 INFO:tasks.workunit.client.1.smithi096.stdout:3/597: dread - d0/d13/d77/f90 zero size 2024-03-24T10:46:15.845 INFO:tasks.workunit.client.1.smithi096.stdout:3/598: truncate d0/d13/d45/d7d/d39/d48/f86 1946002 0 2024-03-24T10:46:15.845 INFO:tasks.workunit.client.1.smithi096.stdout:0/561: creat d0/d11/d24/d2c/d84/fb8 x:0 0 0 2024-03-24T10:46:15.845 INFO:tasks.workunit.client.1.smithi096.stdout:0/562: fsync d0/d11/f6d 0 2024-03-24T10:46:15.846 INFO:tasks.workunit.client.1.smithi096.stdout:0/563: creat d0/d11/d16/d21/d81/d7f/fb9 x:0 0 0 2024-03-24T10:46:15.849 INFO:tasks.workunit.client.1.smithi096.stdout:3/599: creat d0/d13/d45/d7d/d39/d48/d8d/fc8 x:0 0 0 2024-03-24T10:46:15.852 INFO:tasks.workunit.client.1.smithi096.stdout:3/600: rename d0/d9/d22/c51 to d0/d13/d45/d7d/d39/d48/dbd/d9c/db1/cc9 0 2024-03-24T10:46:15.854 INFO:tasks.workunit.client.1.smithi096.stdout:4/485: dwrite d1/d2b/d6b/f96 [0,4194304] 0 2024-03-24T10:46:15.856 INFO:tasks.workunit.client.1.smithi096.stdout:8/562: dwrite d8/d11/d22/d4c/d65/d37/f6b [0,4194304] 0 2024-03-24T10:46:15.856 INFO:tasks.workunit.client.1.smithi096.stdout:3/601: rmdir d0/d13/d45/d7d/d39/d48/dbd/daa 39 2024-03-24T10:46:15.859 INFO:tasks.workunit.client.1.smithi096.stdout:3/602: dread d0/d9/f5e [0,4194304] 0 2024-03-24T10:46:15.859 INFO:tasks.workunit.client.1.smithi096.stdout:3/603: dread - d0/d13/d45/d7d/d39/d48/fb0 zero size 2024-03-24T10:46:15.868 INFO:tasks.workunit.client.1.smithi096.stdout:3/604: dread d0/d9/f32 [0,4194304] 0 2024-03-24T10:46:15.868 INFO:tasks.workunit.client.1.smithi096.stdout:4/486: dread d1/d32/d4c/d7e/f9e [0,4194304] 0 2024-03-24T10:46:15.868 INFO:tasks.workunit.client.1.smithi096.stdout:4/487: stat d1/d2b/d67/d5f 0 2024-03-24T10:46:15.877 INFO:tasks.workunit.client.1.smithi096.stdout:8/563: rmdir d8/d72/d96 39 2024-03-24T10:46:15.879 INFO:tasks.workunit.client.1.smithi096.stdout:4/488: symlink d1/d2b/d6b/d80/la2 0 2024-03-24T10:46:15.884 INFO:tasks.workunit.client.1.smithi096.stdout:8/564: dread d8/fb3 [0,4194304] 0 2024-03-24T10:46:15.888 INFO:tasks.workunit.client.1.smithi096.stdout:8/565: unlink d8/d11/d22/d4c/d82/c59 0 2024-03-24T10:46:15.891 INFO:tasks.workunit.client.1.smithi096.stdout:8/566: creat d8/d11/d22/d4c/d65/d1a/fb5 x:0 0 0 2024-03-24T10:46:15.891 INFO:tasks.workunit.client.1.smithi096.stdout:8/567: write d8/d11/f43 [4647567,126444] 0 2024-03-24T10:46:15.891 INFO:tasks.workunit.client.1.smithi096.stdout:6/572: sync 2024-03-24T10:46:15.891 INFO:tasks.workunit.client.1.smithi096.stdout:6/573: chown d8/d12/d61/faa 17468826 1 2024-03-24T10:46:15.891 INFO:tasks.workunit.client.1.smithi096.stdout:7/737: sync 2024-03-24T10:46:15.895 INFO:tasks.workunit.client.1.smithi096.stdout:6/574: rename d8/d12/d32/d38/d59/c8c to d8/d10/d9e/cb6 0 2024-03-24T10:46:15.899 INFO:tasks.workunit.client.1.smithi096.stdout:7/738: rename d0/db/f72 to d0/d13/ffa 0 2024-03-24T10:46:15.903 INFO:tasks.workunit.client.1.smithi096.stdout:6/575: rename d8/d10/f3b to d8/d12/fb7 0 2024-03-24T10:46:15.903 INFO:tasks.workunit.client.1.smithi096.stdout:6/576: write d8/f4a [4194631,126564] 0 2024-03-24T10:46:15.904 INFO:tasks.workunit.client.1.smithi096.stdout:6/577: stat d8/d12/d28/d36/d65/f6f 0 2024-03-24T10:46:15.904 INFO:tasks.workunit.client.1.smithi096.stdout:7/739: write d0/d13/df0/f27 [2955745,65777] 0 2024-03-24T10:46:15.905 INFO:tasks.workunit.client.1.smithi096.stdout:7/740: fdatasync d0/d13/d32/d45/f63 0 2024-03-24T10:46:15.910 INFO:tasks.workunit.client.1.smithi096.stdout:8/568: dwrite d8/d11/d22/d4c/faf [0,4194304] 0 2024-03-24T10:46:15.910 INFO:tasks.workunit.client.1.smithi096.stdout:8/569: chown d8/d11/d22/d4c/d82/f5e 1003 1 2024-03-24T10:46:15.911 INFO:tasks.workunit.client.1.smithi096.stdout:8/570: creat d8/d11/d22/d4c/d65/d1d/fb6 x:0 0 0 2024-03-24T10:46:15.916 INFO:tasks.workunit.client.1.smithi096.stdout:6/578: write d8/d12/d28/f33 [2324348,98581] 0 2024-03-24T10:46:15.916 INFO:tasks.workunit.client.1.smithi096.stdout:6/579: fdatasync d8/d10/d76/f97 0 2024-03-24T10:46:15.919 INFO:tasks.workunit.client.1.smithi096.stdout:8/571: dread d8/d11/d22/d4c/d65/d1d/f30 [0,4194304] 0 2024-03-24T10:46:15.919 INFO:tasks.workunit.client.1.smithi096.stdout:6/580: getdents d8 0 2024-03-24T10:46:15.919 INFO:tasks.workunit.client.1.smithi096.stdout:6/581: fdatasync d8/d12/d28/d36/f45 0 2024-03-24T10:46:15.919 INFO:tasks.workunit.client.1.smithi096.stdout:6/582: write d8/d10/d19/f34 [3208462,124541] 0 2024-03-24T10:46:15.923 INFO:tasks.workunit.client.1.smithi096.stdout:7/741: dwrite d0/db/d1c/d39/f4a [0,4194304] 0 2024-03-24T10:46:15.924 INFO:tasks.workunit.client.1.smithi096.stdout:7/742: write d0/d13/d32/d99/fbc [532607,120291] 0 2024-03-24T10:46:15.924 INFO:tasks.workunit.client.1.smithi096.stdout:7/743: write d0/d13/f35 [1815264,126533] 0 2024-03-24T10:46:15.925 INFO:tasks.workunit.client.1.smithi096.stdout:6/583: dread f2 [0,4194304] 0 2024-03-24T10:46:15.925 INFO:tasks.workunit.client.1.smithi096.stdout:6/584: readlink d8/d12/d28/d36/d65/d71/l87 0 2024-03-24T10:46:15.930 INFO:tasks.workunit.client.1.smithi096.stdout:7/744: dread d0/db/d1c/d77/dab/fe9 [0,4194304] 0 2024-03-24T10:46:15.935 INFO:tasks.workunit.client.1.smithi096.stdout:8/572: dwrite d8/d11/d22/d4c/d65/d1a/f2d [0,4194304] 0 2024-03-24T10:46:15.938 INFO:tasks.workunit.client.1.smithi096.stdout:8/573: truncate d8/d11/d22/d4c/f9f 760264 0 2024-03-24T10:46:15.946 INFO:tasks.workunit.client.1.smithi096.stdout:6/585: dwrite d8/d10/d13/d63/f2c [0,4194304] 0 2024-03-24T10:46:15.946 INFO:tasks.workunit.client.1.smithi096.stdout:6/586: fdatasync d8/d10/d19/f96 0 2024-03-24T10:46:15.946 INFO:tasks.workunit.client.1.smithi096.stdout:6/587: fdatasync d8/d10/d13/d63/f2c 0 2024-03-24T10:46:15.948 INFO:tasks.workunit.client.1.smithi096.stdout:8/574: symlink d8/d11/d22/d4c/db4/lb7 0 2024-03-24T10:46:15.952 INFO:tasks.workunit.client.1.smithi096.stdout:8/575: mknod d8/d11/d22/d4c/d65/d52/cb8 0 2024-03-24T10:46:15.952 INFO:tasks.workunit.client.1.smithi096.stdout:8/576: write d8/d11/d22/d4c/d67/f74 [889818,22805] 0 2024-03-24T10:46:15.952 INFO:tasks.workunit.client.1.smithi096.stdout:8/577: write d8/d11/d22/d4c/d82/f8c [628634,128709] 0 2024-03-24T10:46:15.952 INFO:tasks.workunit.client.1.smithi096.stdout:6/588: rmdir d8/d12/d61 39 2024-03-24T10:46:15.952 INFO:tasks.workunit.client.1.smithi096.stdout:6/589: fsync d8/f29 0 2024-03-24T10:46:15.963 INFO:tasks.workunit.client.1.smithi096.stdout:8/578: creat d8/d11/d22/d4c/d65/d1a/da0/da6/fb9 x:0 0 0 2024-03-24T10:46:15.963 INFO:tasks.workunit.client.1.smithi096.stdout:8/579: truncate d8/d11/d85/f8a 969349 0 2024-03-24T10:46:15.966 INFO:tasks.workunit.client.1.smithi096.stdout:8/580: creat d8/d72/d96/fba x:0 0 0 2024-03-24T10:46:15.966 INFO:tasks.workunit.client.1.smithi096.stdout:8/581: readlink d8/d11/l7c 0 2024-03-24T10:46:15.967 INFO:tasks.workunit.client.1.smithi096.stdout:9/511: sync 2024-03-24T10:46:15.968 INFO:tasks.workunit.client.1.smithi096.stdout:2/588: sync 2024-03-24T10:46:15.968 INFO:tasks.workunit.client.1.smithi096.stdout:1/464: sync 2024-03-24T10:46:15.968 INFO:tasks.workunit.client.1.smithi096.stdout:1/465: chown d3/l1a 22 1 2024-03-24T10:46:15.970 INFO:tasks.workunit.client.1.smithi096.stdout:1/466: dread d3/d23/d2f/f5a [0,4194304] 0 2024-03-24T10:46:15.970 INFO:tasks.workunit.client.1.smithi096.stdout:8/582: mknod d8/d11/d22/d4c/d65/cbb 0 2024-03-24T10:46:15.972 INFO:tasks.workunit.client.1.smithi096.stdout:6/590: rename d8/d10/d9e/da3 to d8/d12/d32/d38/d59/db8 0 2024-03-24T10:46:15.972 INFO:tasks.workunit.client.1.smithi096.stdout:6/591: write d8/d10/d76/f8d [1014405,128973] 0 2024-03-24T10:46:15.973 INFO:tasks.workunit.client.1.smithi096.stdout:2/589: mknod d5/dc/d6b/d72/d91/cd9 0 2024-03-24T10:46:15.973 INFO:tasks.workunit.client.1.smithi096.stdout:2/590: creat d5/d7c/fda x:0 0 0 2024-03-24T10:46:15.974 INFO:tasks.workunit.client.1.smithi096.stdout:1/467: rmdir d3/d23/d2f/d65 39 2024-03-24T10:46:15.976 INFO:tasks.workunit.client.1.smithi096.stdout:2/591: dread d5/de/d2a/d5e/fb6 [0,4194304] 0 2024-03-24T10:46:15.977 INFO:tasks.workunit.client.1.smithi096.stdout:8/583: rename d8/d72/f8f to d8/d11/d22/d4c/d65/d1a/da0/da6/fbc 0 2024-03-24T10:46:15.979 INFO:tasks.workunit.client.1.smithi096.stdout:6/592: mknod d8/d12/d28/d36/d65/d6b/da6/db2/cb9 0 2024-03-24T10:46:15.982 INFO:tasks.workunit.client.1.smithi096.stdout:2/592: mkdir d5/d38/dbd/ddb 0 2024-03-24T10:46:15.983 INFO:tasks.workunit.client.1.smithi096.stdout:6/593: creat d8/d12/d32/d66/fba x:0 0 0 2024-03-24T10:46:15.983 INFO:tasks.workunit.client.1.smithi096.stdout:6/594: truncate d8/d10/d19/f34 4915563 0 2024-03-24T10:46:15.985 INFO:tasks.workunit.client.1.smithi096.stdout:9/512: dwrite d9/db/d6d/d4f/f54 [0,4194304] 0 2024-03-24T10:46:15.986 INFO:tasks.workunit.client.1.smithi096.stdout:2/593: link d5/dc/d12/f31 d5/dc/d6b/d72/d91/fdc 0 2024-03-24T10:46:15.987 INFO:tasks.workunit.client.1.smithi096.stdout:6/595: creat d8/d12/d32/d38/d59/db8/fbb x:0 0 0 2024-03-24T10:46:15.991 INFO:tasks.workunit.client.1.smithi096.stdout:9/513: stat d9/c87 0 2024-03-24T10:46:15.992 INFO:tasks.workunit.client.1.smithi096.stdout:1/468: dwrite d3/f22 [0,4194304] 0 2024-03-24T10:46:15.993 INFO:tasks.workunit.client.1.smithi096.stdout:1/469: stat d3/d23 0 2024-03-24T10:46:15.994 INFO:tasks.workunit.client.1.smithi096.stdout:2/594: unlink d5/f7 0 2024-03-24T10:46:15.994 INFO:tasks.workunit.client.1.smithi096.stdout:2/595: write d5/dc/d6b/fae [900654,36834] 0 2024-03-24T10:46:15.996 INFO:tasks.workunit.client.1.smithi096.stdout:8/584: dwrite d8/d11/d22/d4c/f6d [0,4194304] 0 2024-03-24T10:46:15.997 INFO:tasks.workunit.client.1.smithi096.stdout:6/596: rename d8/d12/d32/d66/f73 to d8/d12/d32/d38/fbc 0 2024-03-24T10:46:15.999 INFO:tasks.workunit.client.1.smithi096.stdout:1/470: creat d3/d73/f9f x:0 0 0 2024-03-24T10:46:15.999 INFO:tasks.workunit.client.1.smithi096.stdout:1/471: fsync d3/f15 0 2024-03-24T10:46:15.999 INFO:tasks.workunit.client.1.smithi096.stdout:1/472: truncate d3/d23/d8b/d5c/f75 599543 0 2024-03-24T10:46:16.004 INFO:tasks.workunit.client.1.smithi096.stdout:8/585: rename d8/d11/d22/d4c/d67/f74 to d8/d11/d22/d4c/d65/d1a/da0/fbd 0 2024-03-24T10:46:16.004 INFO:tasks.workunit.client.1.smithi096.stdout:8/586: write d8/d11/d44/fa7 [4922082,13780] 0 2024-03-24T10:46:16.006 INFO:tasks.workunit.client.1.smithi096.stdout:1/473: symlink d3/d23/d2f/d65/la0 0 2024-03-24T10:46:16.006 INFO:tasks.workunit.client.1.smithi096.stdout:6/597: read d8/d10/d19/f48 [1749065,690] 0 2024-03-24T10:46:16.010 INFO:tasks.workunit.client.1.smithi096.stdout:9/514: dwrite d9/db/f80 [0,4194304] 0 2024-03-24T10:46:16.010 INFO:tasks.workunit.client.1.smithi096.stdout:9/515: chown d9/db/d6d/d4f/d75/d8b/d8e/fa1 239 1 2024-03-24T10:46:16.010 INFO:tasks.workunit.client.1.smithi096.stdout:1/474: mknod d3/d23/d8b/d79/ca1 0 2024-03-24T10:46:16.010 INFO:tasks.workunit.client.1.smithi096.stdout:2/596: dwrite d5/d7c/f95 [0,4194304] 0 2024-03-24T10:46:16.010 INFO:tasks.workunit.client.1.smithi096.stdout:1/475: creat d3/d23/d8b/d6a/fa2 x:0 0 0 2024-03-24T10:46:16.014 INFO:tasks.workunit.client.1.smithi096.stdout:8/587: write d8/d11/d22/d4c/f6d [2895138,98601] 0 2024-03-24T10:46:16.017 INFO:tasks.workunit.client.1.smithi096.stdout:6/598: creat d8/d12/d61/d7e/fbd x:0 0 0 2024-03-24T10:46:16.018 INFO:tasks.workunit.client.1.smithi096.stdout:6/599: chown d8/d12/d28/d36/d65/f6f 232945857 1 2024-03-24T10:46:16.018 INFO:tasks.workunit.client.1.smithi096.stdout:6/600: fdatasync d8/d10/d13/d63/f5e 0 2024-03-24T10:46:16.018 INFO:tasks.workunit.client.1.smithi096.stdout:6/601: fdatasync d8/d12/d32/d38/d59/f98 0 2024-03-24T10:46:16.026 INFO:tasks.workunit.client.1.smithi096.stdout:9/516: mknod d9/db/d6d/d4f/d75/d8b/d8e/caa 0 2024-03-24T10:46:16.036 INFO:tasks.workunit.client.1.smithi096.stdout:9/517: truncate d9/f2f 1063940 0 2024-03-24T10:46:16.037 INFO:tasks.workunit.client.1.smithi096.stdout:9/518: readlink d9/db/la7 0 2024-03-24T10:46:16.039 INFO:tasks.workunit.client.1.smithi096.stdout:9/519: creat d9/db/fab x:0 0 0 2024-03-24T10:46:16.039 INFO:tasks.workunit.client.1.smithi096.stdout:1/476: dwrite d3/d23/f68 [0,4194304] 0 2024-03-24T10:46:16.040 INFO:tasks.workunit.client.1.smithi096.stdout:1/477: creat d3/d36/fa3 x:0 0 0 2024-03-24T10:46:16.040 INFO:tasks.workunit.client.1.smithi096.stdout:2/597: dwrite d5/d7c/f99 [0,4194304] 0 2024-03-24T10:46:16.040 INFO:tasks.workunit.client.1.smithi096.stdout:2/598: chown d5/d7c/d88/f61 51 1 2024-03-24T10:46:16.040 INFO:tasks.workunit.client.1.smithi096.stdout:8/588: dwrite d8/d11/d22/d4c/d65/d1d/f2b [0,4194304] 0 2024-03-24T10:46:16.040 INFO:tasks.workunit.client.1.smithi096.stdout:8/589: creat d8/d11/d22/d4c/d65/d1a/d46/d2e/fbe x:0 0 0 2024-03-24T10:46:16.040 INFO:tasks.workunit.client.1.smithi096.stdout:8/590: stat d8/d11/d22/d4c/d67/c6f 0 2024-03-24T10:46:16.040 INFO:tasks.workunit.client.1.smithi096.stdout:8/591: chown d8/d11/d22/d4c/d65/d1a/d46/d2e/f54 9149257 1 2024-03-24T10:46:16.040 INFO:tasks.workunit.client.1.smithi096.stdout:8/592: fsync d8/d11/d85/f8a 0 2024-03-24T10:46:16.043 INFO:tasks.workunit.client.1.smithi096.stdout:9/520: truncate d9/d15/d26/f70 2470571 0 2024-03-24T10:46:16.043 INFO:tasks.workunit.client.1.smithi096.stdout:2/599: dread d5/dc/d6b/d72/f9c [0,4194304] 0 2024-03-24T10:46:16.051 INFO:tasks.workunit.client.1.smithi096.stdout:1/478: truncate d3/f66 864324 0 2024-03-24T10:46:16.051 INFO:tasks.workunit.client.1.smithi096.stdout:1/479: fdatasync d3/d23/d8b/d6a/f77 0 2024-03-24T10:46:16.054 INFO:tasks.workunit.client.1.smithi096.stdout:8/593: dwrite d8/d11/d44/d45/f4d [4194304,4194304] 0 2024-03-24T10:46:16.054 INFO:tasks.workunit.client.1.smithi096.stdout:8/594: fdatasync d8/d11/d22/d4c/d65/d1a/da0/da6/fb9 0 2024-03-24T10:46:16.059 INFO:tasks.workunit.client.1.smithi096.stdout:2/600: rename d5/dc/l85 to d5/dc/ldd 0 2024-03-24T10:46:16.059 INFO:tasks.workunit.client.1.smithi096.stdout:2/601: stat d5/dc/d12/f48 0 2024-03-24T10:46:16.059 INFO:tasks.workunit.client.1.smithi096.stdout:1/480: dread d3/d23/f56 [0,4194304] 0 2024-03-24T10:46:16.062 INFO:tasks.workunit.client.1.smithi096.stdout:8/595: dread d8/d11/d22/d4c/d65/d1a/f2d [0,4194304] 0 2024-03-24T10:46:16.063 INFO:tasks.workunit.client.1.smithi096.stdout:2/602: rename d5/de/d2a/fa6 to d5/d79/fde 0 2024-03-24T10:46:16.065 INFO:tasks.workunit.client.1.smithi096.stdout:1/481: creat d3/d9b/fa4 x:0 0 0 2024-03-24T10:46:16.066 INFO:tasks.workunit.client.1.smithi096.stdout:8/596: truncate d8/d11/f19 381452 0 2024-03-24T10:46:16.066 INFO:tasks.workunit.client.1.smithi096.stdout:8/597: truncate d8/d72/f83 1215192 0 2024-03-24T10:46:16.068 INFO:tasks.workunit.client.1.smithi096.stdout:2/603: creat d5/d79/d9f/dad/fdf x:0 0 0 2024-03-24T10:46:16.068 INFO:tasks.workunit.client.1.smithi096.stdout:1/482: rmdir d3/d84 39 2024-03-24T10:46:16.069 INFO:tasks.workunit.client.1.smithi096.stdout:1/483: read d3/d23/d8b/d5c/f75 [236982,83309] 0 2024-03-24T10:46:16.071 INFO:tasks.workunit.client.1.smithi096.stdout:8/598: mkdir d8/d11/d22/d4c/dbf 0 2024-03-24T10:46:16.075 INFO:tasks.workunit.client.1.smithi096.stdout:9/521: dwrite d9/d15/d26/f70 [0,4194304] 0 2024-03-24T10:46:16.084 INFO:tasks.workunit.client.1.smithi096.stdout:9/522: rmdir d9/d15/d26/d29/d73 39 2024-03-24T10:46:16.084 INFO:tasks.workunit.client.1.smithi096.stdout:9/523: readlink d9/d15/l12 0 2024-03-24T10:46:16.087 INFO:tasks.workunit.client.1.smithi096.stdout:8/599: dwrite d8/d11/d22/d4c/d65/d1a/d49/f6a [0,4194304] 0 2024-03-24T10:46:16.087 INFO:tasks.workunit.client.1.smithi096.stdout:8/600: dread - d8/d72/d96/fba zero size 2024-03-24T10:46:16.087 INFO:tasks.workunit.client.1.smithi096.stdout:9/524: mknod d9/d15/d26/cac 0 2024-03-24T10:46:16.090 INFO:tasks.workunit.client.1.smithi096.stdout:8/601: dread d8/d11/d22/d4c/d65/d1a/f2d [0,4194304] 0 2024-03-24T10:46:16.093 INFO:tasks.workunit.client.1.smithi096.stdout:9/525: symlink d9/d7c/lad 0 2024-03-24T10:46:16.094 INFO:tasks.workunit.client.1.smithi096.stdout:8/602: mkdir d8/d11/d22/d4c/d65/d52/dc0 0 2024-03-24T10:46:16.095 INFO:tasks.workunit.client.1.smithi096.stdout:9/526: creat d9/d15/d26/d29/d73/d43/d48/fae x:0 0 0 2024-03-24T10:46:16.096 INFO:tasks.workunit.client.1.smithi096.stdout:8/603: symlink d8/d11/d22/d4c/d65/d1a/d46/lc1 0 2024-03-24T10:46:16.097 INFO:tasks.workunit.client.1.smithi096.stdout:9/527: mknod d9/d15/d26/d29/caf 0 2024-03-24T10:46:16.097 INFO:tasks.workunit.client.1.smithi096.stdout:9/528: fsync d9/db/f53 0 2024-03-24T10:46:16.099 INFO:tasks.workunit.client.1.smithi096.stdout:8/604: creat d8/d11/d85/d93/fc2 x:0 0 0 2024-03-24T10:46:16.100 INFO:tasks.workunit.client.1.smithi096.stdout:9/529: unlink d9/d15/d26/d29/f42 0 2024-03-24T10:46:16.102 INFO:tasks.workunit.client.1.smithi096.stdout:8/605: truncate d8/d11/d22/d4c/d65/d1a/d49/f6a 146713 0 2024-03-24T10:46:16.104 INFO:tasks.workunit.client.1.smithi096.stdout:8/606: mknod d8/d11/d44/d45/cc3 0 2024-03-24T10:46:16.108 INFO:tasks.workunit.client.1.smithi096.stdout:8/607: write d8/d11/d22/d4c/d65/d1a/f2d [3054150,75627] 0 2024-03-24T10:46:16.109 INFO:tasks.workunit.client.1.smithi096.stdout:9/530: dread d9/d15/d26/d29/f36 [0,4194304] 0 2024-03-24T10:46:16.110 INFO:tasks.workunit.client.1.smithi096.stdout:9/531: write d9/f2e [1157968,33161] 0 2024-03-24T10:46:16.111 INFO:tasks.workunit.client.1.smithi096.stdout:8/608: dread d8/d11/d22/d4c/d65/d1d/f7b [0,4194304] 0 2024-03-24T10:46:16.116 INFO:tasks.workunit.client.1.smithi096.stdout:8/609: unlink d8/d11/d44/f84 0 2024-03-24T10:46:16.125 INFO:tasks.workunit.client.1.smithi096.stdout:8/610: read d8/d11/f27 [105647,36282] 0 2024-03-24T10:46:16.126 INFO:tasks.workunit.client.1.smithi096.stdout:9/532: dwrite d9/db/fab [0,4194304] 0 2024-03-24T10:46:16.126 INFO:tasks.workunit.client.1.smithi096.stdout:9/533: stat d9/d15/d26/d29/d73/d43/f99 0 2024-03-24T10:46:16.131 INFO:tasks.workunit.client.1.smithi096.stdout:8/611: read d8/d11/f7f [943706,49536] 0 2024-03-24T10:46:16.142 INFO:tasks.workunit.client.1.smithi096.stdout:9/534: creat d9/db/d6d/fb0 x:0 0 0 2024-03-24T10:46:16.149 INFO:tasks.workunit.client.1.smithi096.stdout:8/612: dwrite d8/d11/d22/d4c/d65/d37/f39 [0,4194304] 0 2024-03-24T10:46:16.150 INFO:tasks.workunit.client.1.smithi096.stdout:9/535: write d9/d15/f66 [1383338,106151] 0 2024-03-24T10:46:16.151 INFO:tasks.workunit.client.1.smithi096.stdout:8/613: link d8/f16 d8/d11/d22/d4c/db0/fc4 0 2024-03-24T10:46:16.152 INFO:tasks.workunit.client.1.smithi096.stdout:8/614: read d8/d11/d22/d4c/d65/d1a/f6c [23206,53221] 0 2024-03-24T10:46:16.152 INFO:tasks.workunit.client.1.smithi096.stdout:8/615: write d8/f15 [4571611,45569] 0 2024-03-24T10:46:16.155 INFO:tasks.workunit.client.1.smithi096.stdout:8/616: symlink d8/d11/d22/d4c/d65/d1a/d46/lc5 0 2024-03-24T10:46:16.155 INFO:tasks.workunit.client.1.smithi096.stdout:8/617: chown d8/d11/l7c 184 1 2024-03-24T10:46:16.157 INFO:tasks.workunit.client.1.smithi096.stdout:9/536: mkdir d9/d15/d26/d29/d73/d43/db1 0 2024-03-24T10:46:16.157 INFO:tasks.workunit.client.1.smithi096.stdout:9/537: creat d9/db/d6d/d4f/fb2 x:0 0 0 2024-03-24T10:46:16.162 INFO:tasks.workunit.client.1.smithi096.stdout:8/618: dread d8/d11/f19 [0,4194304] 0 2024-03-24T10:46:16.162 INFO:tasks.workunit.client.1.smithi096.stdout:8/619: stat d8/d11/d44/d45/f9b 0 2024-03-24T10:46:16.162 INFO:tasks.workunit.client.1.smithi096.stdout:8/620: fsync d8/d11/d22/d4c/d65/d1d/fb6 0 2024-03-24T10:46:16.165 INFO:tasks.workunit.client.1.smithi096.stdout:9/538: creat d9/fb3 x:0 0 0 2024-03-24T10:46:16.166 INFO:tasks.workunit.client.1.smithi096.stdout:8/621: symlink d8/d11/d22/d4c/db0/lc6 0 2024-03-24T10:46:16.169 INFO:tasks.workunit.client.1.smithi096.stdout:9/539: truncate d9/d15/d26/d29/f36 5540179 0 2024-03-24T10:46:16.169 INFO:tasks.workunit.client.1.smithi096.stdout:9/540: chown d9/d15/l4d 246267 1 2024-03-24T10:46:16.170 INFO:tasks.workunit.client.1.smithi096.stdout:8/622: link d8/d11/d22/d4c/l4f d8/d72/lc7 0 2024-03-24T10:46:16.170 INFO:tasks.workunit.client.1.smithi096.stdout:8/623: creat d8/d11/d22/d4c/d65/d1a/d49/fc8 x:0 0 0 2024-03-24T10:46:16.194 INFO:tasks.workunit.client.1.smithi096.stdout:9/541: dwrite d9/d15/f27 [4194304,4194304] 0 2024-03-24T10:46:16.195 INFO:tasks.workunit.client.1.smithi096.stdout:8/624: dwrite d8/d11/d22/d4c/d65/d1a/d46/d2e/fbe [0,4194304] 0 2024-03-24T10:46:16.195 INFO:tasks.workunit.client.1.smithi096.stdout:8/625: write d8/d11/d22/d4c/d65/fa2 [230557,74665] 0 2024-03-24T10:46:16.196 INFO:tasks.workunit.client.1.smithi096.stdout:9/542: truncate d9/d15/f4a 1236272 0 2024-03-24T10:46:16.196 INFO:tasks.workunit.client.1.smithi096.stdout:9/543: dread - d9/db/d6d/d7f/f88 zero size 2024-03-24T10:46:16.199 INFO:tasks.workunit.client.1.smithi096.stdout:8/626: unlink d8/d11/d22/d4c/d65/d52/cb8 0 2024-03-24T10:46:16.199 INFO:tasks.workunit.client.1.smithi096.stdout:8/627: chown d8/d11/d22/d4c/d65/d37/f39 11 1 2024-03-24T10:46:16.199 INFO:tasks.workunit.client.1.smithi096.stdout:8/628: dread - d8/d11/d22/d4c/d65/d1a/da0/da6/fb9 zero size 2024-03-24T10:46:16.199 INFO:tasks.workunit.client.1.smithi096.stdout:8/629: fsync d8/d11/d22/d4c/d65/d1a/da0/fbd 0 2024-03-24T10:46:16.205 INFO:tasks.workunit.client.1.smithi096.stdout:8/630: rename d8/d11/d22/d4c/d67/c6f to d8/d72/d96/d91/cc9 0 2024-03-24T10:46:16.205 INFO:tasks.workunit.client.1.smithi096.stdout:8/631: fsync d8/d11/d22/d4c/d67/f7a 0 2024-03-24T10:46:16.212 INFO:tasks.workunit.client.1.smithi096.stdout:8/632: dread d8/d11/d22/d4c/d65/d1a/f2d [0,4194304] 0 2024-03-24T10:46:16.214 INFO:tasks.workunit.client.1.smithi096.stdout:8/633: rename d8/d11/d22/d4c/d65/d1a/da0/da6/fb9 to d8/d11/d22/d4c/fca 0 2024-03-24T10:46:16.214 INFO:tasks.workunit.client.1.smithi096.stdout:8/634: chown d8/d11/d22/d4c/d65/d1a/d46/d2e/c51 3390 1 2024-03-24T10:46:16.216 INFO:tasks.workunit.client.1.smithi096.stdout:9/544: dwrite d9/d15/f66 [0,4194304] 0 2024-03-24T10:46:16.218 INFO:tasks.workunit.client.1.smithi096.stdout:9/545: mknod d9/cb4 0 2024-03-24T10:46:16.220 INFO:tasks.workunit.client.1.smithi096.stdout:9/546: creat d9/d15/d26/d29/d73/fb5 x:0 0 0 2024-03-24T10:46:16.221 INFO:tasks.workunit.client.1.smithi096.stdout:9/547: read d9/d15/d26/d29/d73/d43/f6c [354925,30333] 0 2024-03-24T10:46:16.221 INFO:tasks.workunit.client.1.smithi096.stdout:9/548: write d9/db/f53 [687869,111515] 0 2024-03-24T10:46:16.222 INFO:tasks.workunit.client.1.smithi096.stdout:9/549: creat d9/db/d6d/d7f/fb6 x:0 0 0 2024-03-24T10:46:16.225 INFO:tasks.workunit.client.1.smithi096.stdout:9/550: mkdir d9/d15/d26/d29/d73/d49/db7 0 2024-03-24T10:46:16.229 INFO:tasks.workunit.client.1.smithi096.stdout:8/635: dwrite d8/fc [0,4194304] 0 2024-03-24T10:46:16.229 INFO:tasks.workunit.client.1.smithi096.stdout:8/636: readlink d8/d11/d22/d4c/d65/d1a/d46/l78 0 2024-03-24T10:46:16.229 INFO:tasks.workunit.client.1.smithi096.stdout:9/551: unlink d9/d15/d26/d29/d73/d43/f67 0 2024-03-24T10:46:16.238 INFO:tasks.workunit.client.1.smithi096.stdout:9/552: write d9/d15/f66 [2251216,36701] 0 2024-03-24T10:46:16.238 INFO:tasks.workunit.client.1.smithi096.stdout:9/553: chown d9/d60/c7a 3517 1 2024-03-24T10:46:16.238 INFO:tasks.workunit.client.1.smithi096.stdout:9/554: write d9/d15/f27 [2736208,71508] 0 2024-03-24T10:46:16.238 INFO:tasks.workunit.client.1.smithi096.stdout:9/555: write d9/db/d6d/d4f/f63 [842438,16927] 0 2024-03-24T10:46:16.242 INFO:tasks.workunit.client.1.smithi096.stdout:8/637: dwrite d8/d11/d85/f8a [0,4194304] 0 2024-03-24T10:46:16.242 INFO:tasks.workunit.client.1.smithi096.stdout:8/638: readlink d8/d11/d22/d4c/d65/d1d/l9e 0 2024-03-24T10:46:16.242 INFO:tasks.workunit.client.1.smithi096.stdout:8/639: readlink d8/d11/d22/d4c/d65/d1d/l9e 0 2024-03-24T10:46:16.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:16 smithi094 ceph-mon[29446]: pgmap v251: 65 pgs: 65 active+clean; 2.7 GiB data, 9.4 GiB used, 527 GiB / 536 GiB avail; 171 MiB/s rd, 214 MiB/s wr, 362 op/s 2024-03-24T10:46:16.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:16 smithi094 ceph-mon[29446]: from='client.14686 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:46:16.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:16 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/1785927985' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T10:46:16.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:16 smithi094 ceph-mon[29446]: Standby manager daemon smithi096.gfonnu restarted 2024-03-24T10:46:16.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:16 smithi094 ceph-mon[29446]: Standby manager daemon smithi096.gfonnu started 2024-03-24T10:46:16.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:16 smithi094 ceph-mon[29446]: from='mgr.? 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi096.gfonnu/crt"}]: dispatch 2024-03-24T10:46:16.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:16 smithi094 ceph-mon[29446]: from='mgr.? 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T10:46:16.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:16 smithi094 ceph-mon[29446]: from='mgr.? 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi096.gfonnu/key"}]: dispatch 2024-03-24T10:46:16.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:16 smithi094 ceph-mon[29446]: from='mgr.? 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T10:46:16.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:16 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:46:16.250 INFO:tasks.workunit.client.1.smithi096.stdout:9/556: link d9/d15/d26/c96 d9/d15/d26/d29/d73/d49/d8a/cb8 0 2024-03-24T10:46:16.252 INFO:tasks.workunit.client.1.smithi096.stdout:8/640: mknod d8/d11/d22/d4c/d65/d37/ccb 0 2024-03-24T10:46:16.255 INFO:tasks.workunit.client.1.smithi096.stdout:8/641: mkdir d8/d11/d22/d4c/db0/dcc 0 2024-03-24T10:46:16.255 INFO:tasks.workunit.client.1.smithi096.stdout:9/557: creat d9/d15/d26/d29/d73/d49/d86/fb9 x:0 0 0 2024-03-24T10:46:16.257 INFO:tasks.workunit.client.1.smithi096.stdout:9/558: truncate d9/f18 544083 0 2024-03-24T10:46:16.275 INFO:tasks.workunit.client.1.smithi096.stdout:8/642: dwrite d8/d11/d22/d4c/d65/d1a/d49/fc8 [0,4194304] 0 2024-03-24T10:46:16.275 INFO:tasks.workunit.client.1.smithi096.stdout:8/643: dread - d8/d11/d22/d4c/d65/d1a/d49/fae zero size 2024-03-24T10:46:16.275 INFO:tasks.workunit.client.1.smithi096.stdout:8/644: write d8/d11/d22/d4c/d65/d1a/f6c [611078,48444] 0 2024-03-24T10:46:16.275 INFO:tasks.workunit.client.1.smithi096.stdout:9/559: dwrite d9/d15/d26/d29/d73/f5a [0,4194304] 0 2024-03-24T10:46:16.279 INFO:tasks.workunit.client.1.smithi096.stdout:9/560: creat d9/d15/d26/d29/d73/d49/db7/fba x:0 0 0 2024-03-24T10:46:16.279 INFO:tasks.workunit.client.1.smithi096.stdout:9/561: write d9/fb3 [560904,48284] 0 2024-03-24T10:46:16.291 INFO:tasks.workunit.client.1.smithi096.stdout:8/645: dwrite d8/d11/d22/d4c/d65/d1a/da0/fbd [0,4194304] 0 2024-03-24T10:46:16.292 INFO:tasks.workunit.client.1.smithi096.stdout:9/562: dread d9/d15/f27 [4194304,4194304] 0 2024-03-24T10:46:16.294 INFO:tasks.workunit.client.1.smithi096.stdout:8/646: symlink d8/d11/d22/d4c/db0/dcc/lcd 0 2024-03-24T10:46:16.296 INFO:tasks.workunit.client.1.smithi096.stdout:8/647: write d8/d11/d22/d4c/d65/d1a/f80 [416548,62585] 0 2024-03-24T10:46:16.298 INFO:tasks.workunit.client.1.smithi096.stdout:8/648: rename d8/d11/d22/d4c/d65/d1a/d46/d2e/faa to d8/d11/d22/d4c/d65/d1d/d3d/fce 0 2024-03-24T10:46:16.299 INFO:tasks.workunit.client.1.smithi096.stdout:8/649: fsync d8/d72/d98/f9c 0 2024-03-24T10:46:16.299 INFO:tasks.workunit.client.1.smithi096.stdout:8/650: creat d8/d11/d22/d4c/d65/d1d/fcf x:0 0 0 2024-03-24T10:46:16.303 INFO:tasks.workunit.client.1.smithi096.stdout:0/564: sync 2024-03-24T10:46:16.303 INFO:tasks.workunit.client.1.smithi096.stdout:0/565: chown d0/d11/l1a 222549 1 2024-03-24T10:46:16.303 INFO:tasks.workunit.client.1.smithi096.stdout:5/579: sync 2024-03-24T10:46:16.304 INFO:tasks.workunit.client.1.smithi096.stdout:0/566: truncate d0/d11/d24/f55 1452220 0 2024-03-24T10:46:16.308 INFO:tasks.workunit.client.1.smithi096.stdout:0/567: creat d0/d11/d16/d21/d78/db6/fba x:0 0 0 2024-03-24T10:46:16.309 INFO:tasks.workunit.client.1.smithi096.stdout:9/563: dwrite d9/db/fa4 [0,4194304] 0 2024-03-24T10:46:16.309 INFO:tasks.workunit.client.1.smithi096.stdout:8/651: dread d8/d11/d22/d4c/d65/d1a/da0/fbd [0,4194304] 0 2024-03-24T10:46:16.315 INFO:tasks.workunit.client.1.smithi096.stdout:9/564: truncate d9/d15/d26/d29/f4e 1768349 0 2024-03-24T10:46:16.315 INFO:tasks.workunit.client.1.smithi096.stdout:9/565: fdatasync d9/d15/f20 0 2024-03-24T10:46:16.316 INFO:tasks.workunit.client.1.smithi096.stdout:8/652: write d8/d72/f76 [1069396,5638] 0 2024-03-24T10:46:16.319 INFO:tasks.workunit.client.1.smithi096.stdout:5/580: dwrite d2/d29/d32/d3f/d5f/f6c [0,4194304] 0 2024-03-24T10:46:16.320 INFO:tasks.workunit.client.1.smithi096.stdout:5/581: fsync d2/d29/d32/db9/fb7 0 2024-03-24T10:46:16.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:16 smithi096 ceph-mon[38097]: pgmap v251: 65 pgs: 65 active+clean; 2.7 GiB data, 9.4 GiB used, 527 GiB / 536 GiB avail; 171 MiB/s rd, 214 MiB/s wr, 362 op/s 2024-03-24T10:46:16.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:16 smithi096 ceph-mon[38097]: from='client.14686 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:46:16.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:16 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/1785927985' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T10:46:16.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:16 smithi096 ceph-mon[38097]: Standby manager daemon smithi096.gfonnu restarted 2024-03-24T10:46:16.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:16 smithi096 ceph-mon[38097]: Standby manager daemon smithi096.gfonnu started 2024-03-24T10:46:16.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:16 smithi096 ceph-mon[38097]: from='mgr.? 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi096.gfonnu/crt"}]: dispatch 2024-03-24T10:46:16.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:16 smithi096 ceph-mon[38097]: from='mgr.? 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T10:46:16.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:16 smithi096 ceph-mon[38097]: from='mgr.? 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi096.gfonnu/key"}]: dispatch 2024-03-24T10:46:16.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:16 smithi096 ceph-mon[38097]: from='mgr.? 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T10:46:16.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:16 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:46:16.326 INFO:tasks.workunit.client.1.smithi096.stdout:8/653: mkdir d8/d72/dd0 0 2024-03-24T10:46:16.326 INFO:tasks.workunit.client.1.smithi096.stdout:5/582: truncate d2/d7/d8/df/f25 155259 0 2024-03-24T10:46:16.327 INFO:tasks.workunit.client.1.smithi096.stdout:0/568: dwrite d0/d11/d20/d56/f5c [0,4194304] 0 2024-03-24T10:46:16.327 INFO:tasks.workunit.client.1.smithi096.stdout:0/569: creat d0/d11/d20/d65/fbb x:0 0 0 2024-03-24T10:46:16.327 INFO:tasks.workunit.client.1.smithi096.stdout:0/570: creat d0/d11/d16/d21/d81/d7f/fbc x:0 0 0 2024-03-24T10:46:16.329 INFO:tasks.workunit.client.1.smithi096.stdout:8/654: creat d8/d11/d22/d4c/d82/da1/fd1 x:0 0 0 2024-03-24T10:46:16.329 INFO:tasks.workunit.client.1.smithi096.stdout:8/655: dread - d8/d11/d22/d4c/d82/f5e zero size 2024-03-24T10:46:16.329 INFO:tasks.workunit.client.1.smithi096.stdout:8/656: fsync d8/d11/d22/d4c/d65/d1a/d46/d2e/fbe 0 2024-03-24T10:46:16.332 INFO:tasks.workunit.client.1.smithi096.stdout:3/605: sync 2024-03-24T10:46:16.332 INFO:tasks.workunit.client.1.smithi096.stdout:3/606: fdatasync d0/f44 0 2024-03-24T10:46:16.339 INFO:tasks.workunit.client.1.smithi096.stdout:3/607: dread d0/d13/f83 [0,4194304] 0 2024-03-24T10:46:16.341 INFO:tasks.workunit.client.1.smithi096.stdout:0/571: creat d0/d11/d16/d21/d81/d8f/d9a/fbd x:0 0 0 2024-03-24T10:46:16.341 INFO:tasks.workunit.client.1.smithi096.stdout:0/572: creat d0/d11/d16/fbe x:0 0 0 2024-03-24T10:46:16.342 INFO:tasks.workunit.client.1.smithi096.stdout:9/566: write d9/d15/d26/d29/d73/d43/f85 [564832,113765] 0 2024-03-24T10:46:16.348 INFO:tasks.workunit.client.1.smithi096.stdout:3/608: write d0/d13/f21 [7671510,49341] 0 2024-03-24T10:46:16.348 INFO:tasks.workunit.client.1.smithi096.stdout:5/583: symlink d2/d7/d8/lc1 0 2024-03-24T10:46:16.349 INFO:tasks.workunit.client.1.smithi096.stdout:3/609: write d0/d9/f1a [4773618,109838] 0 2024-03-24T10:46:16.351 INFO:tasks.workunit.client.1.smithi096.stdout:0/573: symlink d0/lbf 0 2024-03-24T10:46:16.352 INFO:tasks.workunit.client.1.smithi096.stdout:9/567: getdents d9/d15 0 2024-03-24T10:46:16.355 INFO:tasks.workunit.client.1.smithi096.stdout:5/584: mknod d2/d29/cc2 0 2024-03-24T10:46:16.355 INFO:tasks.workunit.client.1.smithi096.stdout:5/585: chown d2/d7/d8/df/c68 509709 1 2024-03-24T10:46:16.356 INFO:tasks.workunit.client.1.smithi096.stdout:3/610: unlink d0/d13/d77/f7a 0 2024-03-24T10:46:16.357 INFO:tasks.workunit.client.1.smithi096.stdout:0/574: symlink d0/d11/d24/lc0 0 2024-03-24T10:46:16.357 INFO:tasks.workunit.client.1.smithi096.stdout:9/568: symlink d9/d15/d26/d29/d73/d43/d48/d51/lbb 0 2024-03-24T10:46:16.357 INFO:tasks.workunit.client.1.smithi096.stdout:9/569: stat d9/db/d6d/d4f/fb2 0 2024-03-24T10:46:16.358 INFO:tasks.workunit.client.1.smithi096.stdout:9/570: readlink d9/db/l69 0 2024-03-24T10:46:16.358 INFO:tasks.workunit.client.1.smithi096.stdout:9/571: creat d9/db/d6d/d4f/d75/d8b/d8e/fbc x:0 0 0 2024-03-24T10:46:16.360 INFO:tasks.workunit.client.1.smithi096.stdout:5/586: mkdir d2/d29/d32/d33/d3c/dc3 0 2024-03-24T10:46:16.360 INFO:tasks.workunit.client.1.smithi096.stdout:5/587: write d2/d29/d32/d3f/f66 [880711,24757] 0 2024-03-24T10:46:16.360 INFO:tasks.workunit.client.1.smithi096.stdout:3/611: creat d0/d9b/fca x:0 0 0 2024-03-24T10:46:16.362 INFO:tasks.workunit.client.1.smithi096.stdout:5/588: mknod d2/d7/d8/d18/d4d/cc4 0 2024-03-24T10:46:16.363 INFO:tasks.workunit.client.1.smithi096.stdout:3/612: symlink d0/d13/d41/lcb 0 2024-03-24T10:46:16.363 INFO:tasks.workunit.client.1.smithi096.stdout:3/613: chown d0/lc5 0 1 2024-03-24T10:46:16.364 INFO:tasks.workunit.client.1.smithi096.stdout:3/614: mkdir d0/d13/d45/d7d/d39/d48/dbd/dbc/dcc 0 2024-03-24T10:46:16.365 INFO:tasks.workunit.client.1.smithi096.stdout:3/615: truncate d0/d13/d45/d7d/d39/d48/f6a 3178640 0 2024-03-24T10:46:16.379 INFO:tasks.workunit.client.1.smithi096.stdout:0/575: dwrite d0/d11/d20/d65/f6e [0,4194304] 0 2024-03-24T10:46:16.379 INFO:tasks.workunit.client.1.smithi096.stdout:5/589: dwrite d2/d7/f3e [0,4194304] 0 2024-03-24T10:46:16.393 INFO:tasks.workunit.client.1.smithi096.stdout:5/590: write d2/d7/d8/f56 [1873705,66878] 0 2024-03-24T10:46:16.404 INFO:tasks.workunit.client.1.smithi096.stdout:5/591: write d2/d29/d32/d33/d3c/d3d/fb5 [2590725,119655] 0 2024-03-24T10:46:16.406 INFO:tasks.workunit.client.1.smithi096.stdout:5/592: link d2/d29/d32/d33/d3c/c7b d2/d29/cc5 0 2024-03-24T10:46:16.407 INFO:tasks.workunit.client.1.smithi096.stdout:0/576: dwrite d0/d11/d16/d21/f33 [0,4194304] 0 2024-03-24T10:46:16.407 INFO:tasks.workunit.client.1.smithi096.stdout:9/572: dwrite d9/d15/f4a [0,4194304] 0 2024-03-24T10:46:16.407 INFO:tasks.workunit.client.1.smithi096.stdout:9/573: chown d9/d15/d26/d29/d73/d43/d48 91514 1 2024-03-24T10:46:16.407 INFO:tasks.workunit.client.1.smithi096.stdout:9/574: readlink d9/db/d6d/l90 0 2024-03-24T10:46:16.407 INFO:tasks.workunit.client.1.smithi096.stdout:5/593: dread d2/d29/d32/d89/f8e [0,4194304] 0 2024-03-24T10:46:16.411 INFO:tasks.workunit.client.1.smithi096.stdout:9/575: mkdir d9/d15/d26/dbd 0 2024-03-24T10:46:16.411 INFO:tasks.workunit.client.1.smithi096.stdout:9/576: stat d9/d7c/f9c 0 2024-03-24T10:46:16.417 INFO:tasks.workunit.client.1.smithi096.stdout:0/577: write d0/d11/d24/f68 [3297519,42844] 0 2024-03-24T10:46:16.417 INFO:tasks.workunit.client.1.smithi096.stdout:0/578: truncate d0/d11/d20/d9e/d80/faf 483849 0 2024-03-24T10:46:16.419 INFO:tasks.workunit.client.1.smithi096.stdout:9/577: write d9/db/fa4 [756594,73799] 0 2024-03-24T10:46:16.501 INFO:tasks.workunit.client.1.smithi096.stdout:5/594: creat d2/d7/d8/df/d8a/fc6 x:0 0 0 2024-03-24T10:46:16.506 INFO:tasks.workunit.client.1.smithi096.stdout:0/579: unlink d0/d11/d16/d21/d25/d47/c8d 0 2024-03-24T10:46:16.506 INFO:tasks.workunit.client.1.smithi096.stdout:0/580: chown d0/d11/f6d 5 1 2024-03-24T10:46:16.507 INFO:tasks.workunit.client.1.smithi096.stdout:9/578: write d9/d15/d26/d29/d73/d49/f7d [189392,27263] 0 2024-03-24T10:46:16.513 INFO:tasks.workunit.client.1.smithi096.stdout:5/595: dread d2/d29/d32/d3f/d5f/f6c [0,4194304] 0 2024-03-24T10:46:16.513 INFO:tasks.workunit.client.1.smithi096.stdout:5/596: chown d2/d7/f86 60 1 2024-03-24T10:46:16.515 INFO:tasks.workunit.client.1.smithi096.stdout:0/581: dread d0/d11/d16/d21/d25/d47/f7c [0,4194304] 0 2024-03-24T10:46:16.518 INFO:tasks.workunit.client.1.smithi096.stdout:0/582: dread d0/d11/d24/d2c/f76 [0,4194304] 0 2024-03-24T10:46:16.523 INFO:tasks.workunit.client.1.smithi096.stdout:9/579: getdents d9/d60 0 2024-03-24T10:46:16.526 INFO:tasks.workunit.client.1.smithi096.stdout:5/597: rename d2/d29/d32/d33/d3c/l4f to d2/lc7 0 2024-03-24T10:46:16.529 INFO:tasks.workunit.client.1.smithi096.stdout:5/598: unlink d2/d7/ca9 0 2024-03-24T10:46:16.530 INFO:tasks.workunit.client.1.smithi096.stdout:5/599: fsync d2/d7/d69/f87 0 2024-03-24T10:46:16.530 INFO:tasks.workunit.client.1.smithi096.stdout:5/600: write d2/d29/d32/d33/d3c/d3d/fb5 [2224365,6128] 0 2024-03-24T10:46:16.530 INFO:tasks.workunit.client.1.smithi096.stdout:5/601: chown d2/d29/d32/d3f/d84 0 1 2024-03-24T10:46:16.532 INFO:tasks.workunit.client.1.smithi096.stdout:9/580: truncate d9/d15/d26/d29/d73/d49/f50 2435013 0 2024-03-24T10:46:16.534 INFO:tasks.workunit.client.1.smithi096.stdout:5/602: mknod d2/d29/d32/d3f/d5f/db3/cc8 0 2024-03-24T10:46:16.534 INFO:tasks.workunit.client.1.smithi096.stdout:5/603: write d2/d7/d8/df/f6a [53784,66524] 0 2024-03-24T10:46:16.538 INFO:tasks.workunit.client.1.smithi096.stdout:5/604: creat d2/d29/d32/d33/d8d/d94/fc9 x:0 0 0 2024-03-24T10:46:16.541 INFO:tasks.workunit.client.1.smithi096.stdout:9/581: rmdir d9/d65 39 2024-03-24T10:46:16.542 INFO:tasks.workunit.client.1.smithi096.stdout:5/605: creat d2/d7/d8/d18/d4d/fca x:0 0 0 2024-03-24T10:46:16.545 INFO:tasks.workunit.client.1.smithi096.stdout:9/582: symlink d9/db/d6d/d4f/d75/d8b/lbe 0 2024-03-24T10:46:16.547 INFO:tasks.workunit.client.1.smithi096.stdout:9/583: mknod d9/d60/cbf 0 2024-03-24T10:46:16.548 INFO:tasks.workunit.client.1.smithi096.stdout:9/584: write d9/d15/d26/d29/d73/d43/fa3 [770981,16962] 0 2024-03-24T10:46:16.560 INFO:tasks.workunit.client.1.smithi096.stdout:2/604: sync 2024-03-24T10:46:16.560 INFO:tasks.workunit.client.1.smithi096.stdout:6/602: sync 2024-03-24T10:46:16.561 INFO:tasks.workunit.client.1.smithi096.stdout:4/489: sync 2024-03-24T10:46:16.561 INFO:tasks.workunit.client.1.smithi096.stdout:1/484: sync 2024-03-24T10:46:16.561 INFO:tasks.workunit.client.1.smithi096.stdout:7/745: sync 2024-03-24T10:46:16.561 INFO:tasks.workunit.client.1.smithi096.stdout:4/490: readlink d1/d32/d4c/d7e/l8b 0 2024-03-24T10:46:16.561 INFO:tasks.workunit.client.1.smithi096.stdout:7/746: read - d0/db/d1c/f9f zero size 2024-03-24T10:46:16.561 INFO:tasks.workunit.client.1.smithi096.stdout:1/485: write d3/d73/f7a [202114,116552] 0 2024-03-24T10:46:16.561 INFO:tasks.workunit.client.1.smithi096.stdout:9/585: dwrite d9/d15/f4a [0,4194304] 0 2024-03-24T10:46:16.562 INFO:tasks.workunit.client.1.smithi096.stdout:2/605: mkdir d5/dc/d6b/d72/de0 0 2024-03-24T10:46:16.564 INFO:tasks.workunit.client.1.smithi096.stdout:4/491: rename d1/d2b/d59/l97 to d1/d32/la3 0 2024-03-24T10:46:16.565 INFO:tasks.workunit.client.1.smithi096.stdout:4/492: readlink d1/d2b/l3f 0 2024-03-24T10:46:16.568 INFO:tasks.workunit.client.1.smithi096.stdout:9/586: mkdir d9/d15/d26/d29/d73/d43/d48/dc0 0 2024-03-24T10:46:16.568 INFO:tasks.workunit.client.1.smithi096.stdout:9/587: write d9/db/d6d/d4f/d75/d8b/d8e/f95 [221625,38473] 0 2024-03-24T10:46:16.570 INFO:tasks.workunit.client.1.smithi096.stdout:2/606: truncate d5/fa4 1131111 0 2024-03-24T10:46:16.572 INFO:tasks.workunit.client.1.smithi096.stdout:1/486: rename d3/f15 to d3/d23/d8b/d5c/fa5 0 2024-03-24T10:46:16.572 INFO:tasks.workunit.client.1.smithi096.stdout:1/487: readlink d3/d23/d8b/l52 0 2024-03-24T10:46:16.573 INFO:tasks.workunit.client.1.smithi096.stdout:6/603: write d8/d10/d19/f60 [804193,106598] 0 2024-03-24T10:46:16.575 INFO:tasks.workunit.client.1.smithi096.stdout:9/588: unlink d9/d7c/f94 0 2024-03-24T10:46:16.576 INFO:tasks.workunit.client.1.smithi096.stdout:4/493: dread d1/d32/d42/f5e [0,4194304] 0 2024-03-24T10:46:16.579 INFO:tasks.workunit.client.1.smithi096.stdout:7/747: dwrite d0/d13/d32/faa [0,4194304] 0 2024-03-24T10:46:16.579 INFO:tasks.workunit.client.1.smithi096.stdout:7/748: truncate d0/d13/df0/d3c/d68/d7e/ff4 541506 0 2024-03-24T10:46:16.579 INFO:tasks.workunit.client.1.smithi096.stdout:7/749: fsync d0/db/d26/d48/fd0 0 2024-03-24T10:46:16.579 INFO:tasks.workunit.client.1.smithi096.stdout:1/488: link d3/d23/f38 d3/d23/d8b/d99/fa6 0 2024-03-24T10:46:16.580 INFO:tasks.workunit.client.1.smithi096.stdout:1/489: truncate d3/d23/d71/f7d 198616 0 2024-03-24T10:46:16.580 INFO:tasks.workunit.client.1.smithi096.stdout:1/490: chown d3/d23/l2c 60062051 1 2024-03-24T10:46:16.580 INFO:tasks.workunit.client.1.smithi096.stdout:1/491: write d3/d23/d2f/f60 [413364,40001] 0 2024-03-24T10:46:16.580 INFO:tasks.workunit.client.1.smithi096.stdout:1/492: write d3/d23/f62 [1907907,107723] 0 2024-03-24T10:46:16.581 INFO:tasks.workunit.client.1.smithi096.stdout:2/607: write f2 [3820741,117908] 0 2024-03-24T10:46:16.581 INFO:tasks.workunit.client.1.smithi096.stdout:2/608: chown d5/dc/d6b/d72/d91/fa0 24159898 1 2024-03-24T10:46:16.582 INFO:tasks.workunit.client.1.smithi096.stdout:9/589: mkdir d9/d15/d26/d29/d73/d49/db7/dc1 0 2024-03-24T10:46:16.583 INFO:tasks.workunit.client.1.smithi096.stdout:2/609: dread d5/dc/d12/f16 [0,4194304] 0 2024-03-24T10:46:16.583 INFO:tasks.workunit.client.1.smithi096.stdout:2/610: readlink d5/dc/d12/laa 0 2024-03-24T10:46:16.585 INFO:tasks.workunit.client.1.smithi096.stdout:7/750: mknod d0/d13/d32/d99/cfb 0 2024-03-24T10:46:16.585 INFO:tasks.workunit.client.1.smithi096.stdout:7/751: stat d0/d13/df0/d3c/d68/ceb 0 2024-03-24T10:46:16.586 INFO:tasks.workunit.client.1.smithi096.stdout:7/752: chown d0/f20 10978 1 2024-03-24T10:46:16.586 INFO:tasks.workunit.client.1.smithi096.stdout:7/753: chown d0/db/d26/d48/fd0 201476115 1 2024-03-24T10:46:16.590 INFO:tasks.workunit.client.1.smithi096.stdout:9/590: truncate d9/db/d6d/d4f/f54 2838299 0 2024-03-24T10:46:16.590 INFO:tasks.workunit.client.1.smithi096.stdout:9/591: fsync d9/db/fa4 0 2024-03-24T10:46:16.590 INFO:tasks.workunit.client.1.smithi096.stdout:9/592: creat d9/db/d6d/fc2 x:0 0 0 2024-03-24T10:46:16.590 INFO:tasks.workunit.client.1.smithi096.stdout:9/593: write d9/d15/f66 [4478701,22767] 0 2024-03-24T10:46:16.591 INFO:tasks.workunit.client.1.smithi096.stdout:7/754: unlink d0/db/d26/d48/fce 0 2024-03-24T10:46:16.592 INFO:tasks.workunit.client.1.smithi096.stdout:1/493: rmdir d3/d73 39 2024-03-24T10:46:16.592 INFO:tasks.workunit.client.1.smithi096.stdout:1/494: fsync d3/d23/d71/f8a 0 2024-03-24T10:46:16.593 INFO:tasks.workunit.client.1.smithi096.stdout:1/495: stat d3/d23/d8b/d79 0 2024-03-24T10:46:16.593 INFO:tasks.workunit.client.1.smithi096.stdout:1/496: write d3/d23/d8b/d5c/f92 [140491,100405] 0 2024-03-24T10:46:16.594 INFO:tasks.workunit.client.1.smithi096.stdout:9/594: chown d9/d79/l89 477596 1 2024-03-24T10:46:16.596 INFO:tasks.workunit.client.1.smithi096.stdout:7/755: unlink d0/db/l70 0 2024-03-24T10:46:16.599 INFO:tasks.workunit.client.1.smithi096.stdout:4/494: dwrite d1/d2b/d6b/d80/f9f [0,4194304] 0 2024-03-24T10:46:16.599 INFO:tasks.workunit.client.1.smithi096.stdout:4/495: fdatasync d1/d2b/d67/f52 0 2024-03-24T10:46:16.599 INFO:tasks.workunit.client.1.smithi096.stdout:4/496: truncate d1/d32/d42/f86 342249 0 2024-03-24T10:46:16.599 INFO:tasks.workunit.client.1.smithi096.stdout:9/595: symlink d9/d79/lc3 0 2024-03-24T10:46:16.599 INFO:tasks.workunit.client.1.smithi096.stdout:9/596: dread - d9/d15/d26/d29/d73/d43/d48/fae zero size 2024-03-24T10:46:16.599 INFO:tasks.workunit.client.1.smithi096.stdout:9/597: chown d9/db/f28 20 1 2024-03-24T10:46:16.599 INFO:tasks.workunit.client.1.smithi096.stdout:9/598: read d9/db/f53 [673426,9994] 0 2024-03-24T10:46:16.599 INFO:tasks.workunit.client.1.smithi096.stdout:9/599: write d9/db/fa4 [3188348,78400] 0 2024-03-24T10:46:16.599 INFO:tasks.workunit.client.1.smithi096.stdout:9/600: dread - d9/d15/d26/d29/d73/d49/d86/fb9 zero size 2024-03-24T10:46:16.600 INFO:tasks.workunit.client.1.smithi096.stdout:7/756: creat d0/d13/df0/d3c/d68/d4b/ffc x:0 0 0 2024-03-24T10:46:16.601 INFO:tasks.workunit.client.1.smithi096.stdout:6/604: dwrite d8/d10/d58/f72 [0,4194304] 0 2024-03-24T10:46:16.601 INFO:tasks.workunit.client.1.smithi096.stdout:2/611: dwrite d5/dc/d6b/d72/d91/fd0 [0,4194304] 0 2024-03-24T10:46:16.603 INFO:tasks.workunit.client.1.smithi096.stdout:1/497: getdents d3/d23/d71 0 2024-03-24T10:46:16.609 INFO:tasks.workunit.client.1.smithi096.stdout:6/605: read d8/d10/d19/f1b [934504,88185] 0 2024-03-24T10:46:16.616 INFO:tasks.workunit.client.1.smithi096.stdout:4/497: rename d1/d2b/c73 to d1/d32/d4c/ca4 0 2024-03-24T10:46:16.616 INFO:tasks.workunit.client.1.smithi096.stdout:4/498: creat d1/d2b/d40/fa5 x:0 0 0 2024-03-24T10:46:16.618 INFO:tasks.workunit.client.1.smithi096.stdout:2/612: dwrite d5/de/d2a/fd5 [0,4194304] 0 2024-03-24T10:46:16.625 INFO:tasks.workunit.client.1.smithi096.stdout:7/757: dwrite d0/db/d26/d48/f67 [0,4194304] 0 2024-03-24T10:46:16.633 INFO:tasks.workunit.client.1.smithi096.stdout:4/499: dwrite d1/d2b/d6b/d80/d9c/fa1 [0,4194304] 0 2024-03-24T10:46:16.634 INFO:tasks.workunit.client.1.smithi096.stdout:4/500: chown d1/d32/d42/c65 12468 1 2024-03-24T10:46:16.646 INFO:tasks.workunit.client.1.smithi096.stdout:4/501: dwrite d1/d2b/d59/d74/f57 [0,4194304] 0 2024-03-24T10:46:16.652 INFO:tasks.workunit.client.1.smithi096.stdout:4/502: dread d1/d11/f12 [0,4194304] 0 2024-03-24T10:46:16.662 INFO:tasks.workunit.client.1.smithi096.stdout:4/503: write d1/f35 [1148480,76391] 0 2024-03-24T10:46:16.662 INFO:tasks.workunit.client.1.smithi096.stdout:4/504: fsync d1/d11/f66 0 2024-03-24T10:46:16.663 INFO:tasks.workunit.client.1.smithi096.stdout:4/505: fdatasync d1/f1f 0 2024-03-24T10:46:16.666 INFO:tasks.workunit.client.1.smithi096.stdout:2/613: truncate d5/dc/f6c 466166 0 2024-03-24T10:46:16.671 INFO:tasks.workunit.client.1.smithi096.stdout:7/758: creat d0/d13/df0/d3c/d68/d4b/d62/ffd x:0 0 0 2024-03-24T10:46:16.678 INFO:tasks.workunit.client.1.smithi096.stdout:7/759: dread d0/d13/df0/d59/f79 [0,4194304] 0 2024-03-24T10:46:16.678 INFO:tasks.workunit.client.1.smithi096.stdout:7/760: write d0/f20 [1773643,112628] 0 2024-03-24T10:46:16.678 INFO:tasks.workunit.client.1.smithi096.stdout:7/761: stat d0/f9e 0 2024-03-24T10:46:16.681 INFO:tasks.workunit.client.1.smithi096.stdout:2/614: dwrite d5/f5f [0,4194304] 0 2024-03-24T10:46:16.721 INFO:tasks.workunit.client.1.smithi096.stdout:4/506: mkdir d1/d32/d4c/da6 0 2024-03-24T10:46:16.721 INFO:tasks.workunit.client.1.smithi096.stdout:4/507: truncate d1/d32/d42/f86 486770 0 2024-03-24T10:46:16.721 INFO:tasks.workunit.client.1.smithi096.stdout:4/508: creat d1/d2b/d40/fa7 x:0 0 0 2024-03-24T10:46:16.721 INFO:tasks.workunit.client.1.smithi096.stdout:4/509: chown d1/d11/c1a 440 1 2024-03-24T10:46:16.721 INFO:tasks.workunit.client.1.smithi096.stdout:4/510: readlink d1/d11/l6e 0 2024-03-24T10:46:16.721 INFO:tasks.workunit.client.1.smithi096.stdout:4/511: truncate d1/fe 1259990 0 2024-03-24T10:46:16.724 INFO:tasks.workunit.client.1.smithi096.stdout:2/615: symlink d5/d93/le1 0 2024-03-24T10:46:16.724 INFO:tasks.workunit.client.1.smithi096.stdout:2/616: readlink d5/dc/d12/laa 0 2024-03-24T10:46:16.742 INFO:tasks.workunit.client.1.smithi096.stdout:2/617: dwrite d5/de/d2a/d5e/f65 [0,4194304] 0 2024-03-24T10:46:16.743 INFO:tasks.workunit.client.1.smithi096.stdout:4/512: dwrite d1/d2b/d6b/f83 [4194304,4194304] 0 2024-03-24T10:46:16.746 INFO:tasks.workunit.client.1.smithi096.stdout:4/513: symlink d1/d2b/d6b/la8 0 2024-03-24T10:46:16.747 INFO:tasks.workunit.client.1.smithi096.stdout:2/618: link d5/d7c/f95 d5/de/fe2 0 2024-03-24T10:46:16.749 INFO:tasks.workunit.client.1.smithi096.stdout:4/514: mkdir d1/d2b/d67/d5f/da9 0 2024-03-24T10:46:16.749 INFO:tasks.workunit.client.1.smithi096.stdout:4/515: fsync d1/d2b/d40/f64 0 2024-03-24T10:46:16.749 INFO:tasks.workunit.client.1.smithi096.stdout:4/516: chown d1/d2b/l2d 9600 1 2024-03-24T10:46:16.749 INFO:tasks.workunit.client.1.smithi096.stdout:4/517: creat d1/d32/d4c/da6/faa x:0 0 0 2024-03-24T10:46:16.776 INFO:tasks.workunit.client.1.smithi096.stdout:7/762: creat d0/d13/df0/d59/ffe x:0 0 0 2024-03-24T10:46:16.787 INFO:tasks.workunit.client.1.smithi096.stdout:7/763: dread d0/d13/d32/d45/f6c [0,4194304] 0 2024-03-24T10:46:16.804 INFO:tasks.workunit.client.1.smithi096.stdout:7/764: dwrite d0/d13/df0/d3c/d68/f41 [0,4194304] 0 2024-03-24T10:46:16.805 INFO:tasks.workunit.client.1.smithi096.stdout:1/498: rmdir d3/d23/d8b 39 2024-03-24T10:46:16.805 INFO:tasks.workunit.client.1.smithi096.stdout:1/499: truncate d3/d23/d2f/f83 1256451 0 2024-03-24T10:46:16.808 INFO:tasks.workunit.client.1.smithi096.stdout:1/500: dread d3/d23/d71/f7d [0,4194304] 0 2024-03-24T10:46:16.812 INFO:tasks.workunit.client.1.smithi096.stdout:1/501: mknod d3/d23/ca7 0 2024-03-24T10:46:16.816 INFO:tasks.workunit.client.1.smithi096.stdout:1/502: unlink d3/f1f 0 2024-03-24T10:46:16.840 INFO:tasks.workunit.client.1.smithi096.stdout:9/601: rename d9/d15/d26/d29/d73/c46 to d9/db/d6d/cc4 0 2024-03-24T10:46:16.842 INFO:tasks.workunit.client.1.smithi096.stdout:4/518: unlink d1/d32/l33 0 2024-03-24T10:46:16.845 INFO:tasks.workunit.client.1.smithi096.stdout:9/602: getdents d9/d79 0 2024-03-24T10:46:16.850 INFO:tasks.workunit.client.1.smithi096.stdout:4/519: creat d1/d32/d4c/d7d/fab x:0 0 0 2024-03-24T10:46:16.852 INFO:tasks.workunit.client.1.smithi096.stdout:9/603: readlink d9/d15/d26/d29/d73/d49/l5f 0 2024-03-24T10:46:16.855 INFO:tasks.workunit.client.1.smithi096.stdout:4/520: dread d1/d11/f12 [0,4194304] 0 2024-03-24T10:46:16.855 INFO:tasks.workunit.client.1.smithi096.stdout:9/604: creat d9/d15/d26/d29/fc5 x:0 0 0 2024-03-24T10:46:16.857 INFO:tasks.workunit.client.1.smithi096.stdout:6/606: rename d8/d12/d61/d7e to d8/d12/d32/d66/da8/dbe 0 2024-03-24T10:46:16.857 INFO:tasks.workunit.client.1.smithi096.stdout:6/607: dread - d8/d12/d32/d66/da8/dbe/fbd zero size 2024-03-24T10:46:16.857 INFO:tasks.workunit.client.1.smithi096.stdout:6/608: stat d8/d10/d13/d63/c20 0 2024-03-24T10:46:16.857 INFO:tasks.workunit.client.1.smithi096.stdout:6/609: dread - d8/f81 zero size 2024-03-24T10:46:16.857 INFO:tasks.workunit.client.1.smithi096.stdout:6/610: write d8/d12/d32/d38/d59/f5a [4828984,124354] 0 2024-03-24T10:46:16.857 INFO:tasks.workunit.client.1.smithi096.stdout:6/611: chown d8/d12/d32/d66/fba 19673 1 2024-03-24T10:46:16.858 INFO:tasks.workunit.client.1.smithi096.stdout:6/612: readlink d8/d10/l35 0 2024-03-24T10:46:16.860 INFO:tasks.workunit.client.1.smithi096.stdout:7/765: rename d0/d13/d32/f3b to d0/db/d4f/fff 0 2024-03-24T10:46:16.863 INFO:tasks.workunit.client.1.smithi096.stdout:4/521: dread d1/d2b/d6b/f96 [0,4194304] 0 2024-03-24T10:46:16.866 INFO:tasks.workunit.client.1.smithi096.stdout:7/766: creat d0/d13/d32/d45/dcd/f100 x:0 0 0 2024-03-24T10:46:16.867 INFO:tasks.workunit.client.1.smithi096.stdout:6/613: dread d8/d10/d19/f48 [4194304,4194304] 0 2024-03-24T10:46:16.867 INFO:tasks.workunit.client.1.smithi096.stdout:1/503: rename d3/d23/d8b/d6a/fa2 to d3/d23/d8b/d79/fa8 0 2024-03-24T10:46:16.868 INFO:tasks.workunit.client.1.smithi096.stdout:1/504: chown d3/d23/d2f/l40 47623679 1 2024-03-24T10:46:16.870 INFO:tasks.workunit.client.1.smithi096.stdout:6/614: truncate d8/d10/d13/d63/f5e 4056894 0 2024-03-24T10:46:16.872 INFO:tasks.workunit.client.1.smithi096.stdout:9/605: rename d9/d15/d64 to d9/db/d6d/dc6 0 2024-03-24T10:46:16.873 INFO:tasks.workunit.client.1.smithi096.stdout:1/505: write d3/d23/d8b/d5c/f61 [1922275,114915] 0 2024-03-24T10:46:16.875 INFO:tasks.workunit.client.1.smithi096.stdout:9/606: creat d9/db/d6d/fc7 x:0 0 0 2024-03-24T10:46:16.875 INFO:tasks.workunit.client.1.smithi096.stdout:9/607: stat d9/db/d6d/c81 0 2024-03-24T10:46:16.881 INFO:tasks.workunit.client.1.smithi096.stdout:6/615: dread d8/d12/d28/d36/d65/f94 [0,4194304] 0 2024-03-24T10:46:16.903 INFO:tasks.workunit.client.1.smithi096.stdout:7/767: dwrite d0/d13/df0/d3c/d68/d4b/de8/ff1 [0,4194304] 0 2024-03-24T10:46:16.909 INFO:tasks.workunit.client.1.smithi096.stdout:8/657: sync 2024-03-24T10:46:16.909 INFO:tasks.workunit.client.1.smithi096.stdout:5/606: sync 2024-03-24T10:46:16.909 INFO:tasks.workunit.client.1.smithi096.stdout:3/616: sync 2024-03-24T10:46:16.909 INFO:tasks.workunit.client.1.smithi096.stdout:0/583: sync 2024-03-24T10:46:16.909 INFO:tasks.workunit.client.1.smithi096.stdout:5/607: write d2/d29/d32/d33/d3c/d3d/fb5 [839925,115636] 0 2024-03-24T10:46:16.909 INFO:tasks.workunit.client.1.smithi096.stdout:5/608: fdatasync d2/d29/d32/f40 0 2024-03-24T10:46:16.909 INFO:tasks.workunit.client.1.smithi096.stdout:5/609: write d2/d29/d32/db9/fb7 [172031,3503] 0 2024-03-24T10:46:16.910 INFO:tasks.workunit.client.1.smithi096.stdout:9/608: dwrite d9/db/d6d/d4f/d75/d8b/d8e/f95 [0,4194304] 0 2024-03-24T10:46:16.912 INFO:tasks.workunit.client.1.smithi096.stdout:3/617: rename d0/f44 to d0/d13/d41/fcd 0 2024-03-24T10:46:16.912 INFO:tasks.workunit.client.1.smithi096.stdout:3/618: dread - d0/d9b/fca zero size 2024-03-24T10:46:16.912 INFO:tasks.workunit.client.1.smithi096.stdout:6/616: dwrite d8/d10/d19/f30 [0,4194304] 0 2024-03-24T10:46:16.912 INFO:tasks.workunit.client.1.smithi096.stdout:1/506: dwrite d3/d36/f54 [0,4194304] 0 2024-03-24T10:46:16.912 INFO:tasks.workunit.client.1.smithi096.stdout:6/617: chown d8/d12/d28/d36/d65 248446 1 2024-03-24T10:46:16.913 INFO:tasks.workunit.client.1.smithi096.stdout:1/507: write d3/d23/d2f/f60 [456014,100026] 0 2024-03-24T10:46:16.913 INFO:tasks.workunit.client.1.smithi096.stdout:3/619: write d0/d13/d1e/f2a [2681144,5552] 0 2024-03-24T10:46:16.913 INFO:tasks.workunit.client.1.smithi096.stdout:3/620: write d0/d13/d45/d47/fb9 [585807,89134] 0 2024-03-24T10:46:16.913 INFO:tasks.workunit.client.1.smithi096.stdout:3/621: dread - d0/d9b/fca zero size 2024-03-24T10:46:16.914 INFO:tasks.workunit.client.1.smithi096.stdout:0/584: mknod d0/d11/d24/d2c/cc1 0 2024-03-24T10:46:16.914 INFO:tasks.workunit.client.1.smithi096.stdout:0/585: chown d0/d11/d20/d9e/d80/faf 1663855 1 2024-03-24T10:46:16.918 INFO:tasks.workunit.client.1.smithi096.stdout:5/610: creat d2/d29/d32/fcb x:0 0 0 2024-03-24T10:46:16.918 INFO:tasks.workunit.client.1.smithi096.stdout:5/611: fdatasync d2/d7/f8c 0 2024-03-24T10:46:16.921 INFO:tasks.workunit.client.1.smithi096.stdout:7/768: dwrite d0/db/d1c/f29 [0,4194304] 0 2024-03-24T10:46:16.923 INFO:tasks.workunit.client.1.smithi096.stdout:6/618: dread d8/d12/d28/f33 [0,4194304] 0 2024-03-24T10:46:16.923 INFO:tasks.workunit.client.1.smithi096.stdout:6/619: fdatasync d8/d12/d28/f53 0 2024-03-24T10:46:16.926 INFO:tasks.workunit.client.1.smithi096.stdout:1/508: read d3/f34 [3542500,80878] 0 2024-03-24T10:46:16.927 INFO:tasks.workunit.client.1.smithi096.stdout:1/509: dread d3/d23/d8b/f46 [0,4194304] 0 2024-03-24T10:46:16.928 INFO:tasks.workunit.client.1.smithi096.stdout:9/609: rmdir d9/d60 39 2024-03-24T10:46:16.932 INFO:tasks.workunit.client.1.smithi096.stdout:8/658: dwrite d8/d72/f76 [0,4194304] 0 2024-03-24T10:46:16.932 INFO:tasks.workunit.client.1.smithi096.stdout:3/622: symlink d0/d13/d1e/da4/lce 0 2024-03-24T10:46:16.934 INFO:tasks.workunit.client.1.smithi096.stdout:6/620: write d8/d12/d28/d36/f45 [6319346,130476] 0 2024-03-24T10:46:16.935 INFO:tasks.workunit.client.1.smithi096.stdout:8/659: dread d8/d11/d22/d4c/d65/fa2 [0,4194304] 0 2024-03-24T10:46:16.939 INFO:tasks.workunit.client.1.smithi096.stdout:0/586: rename d0/d11/d20/d9e/l90 to d0/d11/d24/d2c/d84/lc2 0 2024-03-24T10:46:16.942 INFO:tasks.workunit.client.1.smithi096.stdout:5/612: creat d2/d29/d32/d3f/d5f/dc0/fcc x:0 0 0 2024-03-24T10:46:16.944 INFO:tasks.workunit.client.1.smithi096.stdout:7/769: chown d0/d13/df0/l22 1074630490 1 2024-03-24T10:46:16.946 INFO:tasks.workunit.client.1.smithi096.stdout:3/623: symlink d0/d13/d45/d7d/d39/d48/d85/lcf 0 2024-03-24T10:46:16.946 INFO:tasks.workunit.client.1.smithi096.stdout:3/624: chown d0/d13/l87 12 1 2024-03-24T10:46:16.947 INFO:tasks.workunit.client.1.smithi096.stdout:8/660: truncate d8/d72/d96/f87 63928 0 2024-03-24T10:46:16.947 INFO:tasks.workunit.client.1.smithi096.stdout:8/661: truncate d8/d72/d96/fba 155788 0 2024-03-24T10:46:16.947 INFO:tasks.workunit.client.1.smithi096.stdout:8/662: fsync d8/d11/d22/d4c/d65/d37/f6e 0 2024-03-24T10:46:16.948 INFO:tasks.workunit.client.1.smithi096.stdout:8/663: fdatasync d8/d11/f27 0 2024-03-24T10:46:16.948 INFO:tasks.workunit.client.1.smithi096.stdout:8/664: write d8/d11/d22/d4c/d65/d1a/f80 [1236772,52092] 0 2024-03-24T10:46:16.948 INFO:tasks.workunit.client.1.smithi096.stdout:5/613: mknod d2/d29/d32/d3f/d84/ccd 0 2024-03-24T10:46:16.948 INFO:tasks.workunit.client.1.smithi096.stdout:5/614: chown d2/d29/d32/d33/d3c/d3d/c7d 494085840 1 2024-03-24T10:46:16.948 INFO:tasks.workunit.client.1.smithi096.stdout:9/610: dwrite d9/d15/d26/d29/d73/d43/f99 [0,4194304] 0 2024-03-24T10:46:16.951 INFO:tasks.workunit.client.1.smithi096.stdout:1/510: creat d3/d23/d8b/d5c/d9c/fa9 x:0 0 0 2024-03-24T10:46:16.951 INFO:tasks.workunit.client.1.smithi096.stdout:1/511: chown d3/d23/d8b/d6a/f70 627675984 1 2024-03-24T10:46:16.951 INFO:tasks.workunit.client.1.smithi096.stdout:1/512: stat d3/c10 0 2024-03-24T10:46:16.953 INFO:tasks.workunit.client.1.smithi096.stdout:6/621: dwrite d8/d12/d32/f47 [0,4194304] 0 2024-03-24T10:46:16.954 INFO:tasks.workunit.client.1.smithi096.stdout:0/587: dread d0/d11/d16/d21/d25/d47/f70 [4194304,4194304] 0 2024-03-24T10:46:16.954 INFO:tasks.workunit.client.1.smithi096.stdout:8/665: symlink d8/d11/d22/ld2 0 2024-03-24T10:46:16.957 INFO:tasks.workunit.client.1.smithi096.stdout:7/770: write d0/db/d1c/f29 [4126980,89017] 0 2024-03-24T10:46:16.960 INFO:tasks.workunit.client.1.smithi096.stdout:1/513: creat d3/d23/d71/d7b/faa x:0 0 0 2024-03-24T10:46:16.960 INFO:tasks.workunit.client.1.smithi096.stdout:1/514: read - d3/d23/d71/d7b/faa zero size 2024-03-24T10:46:16.963 INFO:tasks.workunit.client.1.smithi096.stdout:0/588: dread d0/d11/d20/d91/f96 [0,4194304] 0 2024-03-24T10:46:16.964 INFO:tasks.workunit.client.1.smithi096.stdout:7/771: dread d0/db/d26/dc3/f9d [0,4194304] 0 2024-03-24T10:46:16.965 INFO:tasks.workunit.client.1.smithi096.stdout:6/622: mkdir d8/d10/d13/d63/dbf 0 2024-03-24T10:46:16.968 INFO:tasks.workunit.client.1.smithi096.stdout:3/625: dwrite d0/d13/d45/d47/f89 [0,4194304] 0 2024-03-24T10:46:16.971 INFO:tasks.workunit.client.1.smithi096.stdout:8/666: creat d8/d11/d22/d4c/d65/d1a/da0/fd3 x:0 0 0 2024-03-24T10:46:16.971 INFO:tasks.workunit.client.1.smithi096.stdout:8/667: chown d8/d11/d22/d4c/d65/d1a/d46/lc1 12381207 1 2024-03-24T10:46:16.971 INFO:tasks.workunit.client.1.smithi096.stdout:8/668: chown d8/d11/d22/d4c/f6d 28158916 1 2024-03-24T10:46:16.977 INFO:tasks.workunit.client.1.smithi096.stdout:9/611: dwrite d9/f3a [0,4194304] 0 2024-03-24T10:46:16.984 INFO:tasks.workunit.client.1.smithi096.stdout:0/589: mkdir d0/d11/dc3 0 2024-03-24T10:46:16.987 INFO:tasks.workunit.client.1.smithi096.stdout:3/626: rename d0/d13/c7c to d0/d92/cd0 0 2024-03-24T10:46:16.990 INFO:tasks.workunit.client.1.smithi096.stdout:9/612: mkdir d9/d15/d26/d29/d73/d49/d86/dc8 0 2024-03-24T10:46:16.990 INFO:tasks.workunit.client.1.smithi096.stdout:9/613: write d9/db/f53 [1256791,93396] 0 2024-03-24T10:46:16.996 INFO:tasks.workunit.client.1.smithi096.stdout:0/590: symlink d0/d11/d24/d2c/lc4 0 2024-03-24T10:46:16.996 INFO:tasks.workunit.client.1.smithi096.stdout:0/591: stat d0/l4f 0 2024-03-24T10:46:16.996 INFO:tasks.workunit.client.1.smithi096.stdout:0/592: creat d0/fc5 x:0 0 0 2024-03-24T10:46:16.996 INFO:tasks.workunit.client.1.smithi096.stdout:0/593: read d0/d11/d16/d21/d81/d7f/f64 [2244692,501] 0 2024-03-24T10:46:16.998 INFO:tasks.workunit.client.1.smithi096.stdout:6/623: dwrite d8/d12/d28/d36/f45 [0,4194304] 0 2024-03-24T10:46:16.998 INFO:tasks.workunit.client.1.smithi096.stdout:1/515: rename d3/d23/c53 to d3/d23/d8b/d5c/d6d/cab 0 2024-03-24T10:46:16.999 INFO:tasks.workunit.client.1.smithi096.stdout:8/669: dwrite d8/f16 [0,4194304] 0 2024-03-24T10:46:16.999 INFO:tasks.workunit.client.1.smithi096.stdout:7/772: dwrite d0/db/d1c/d77/dab/fe9 [0,4194304] 0 2024-03-24T10:46:17.002 INFO:tasks.workunit.client.1.smithi096.stdout:9/614: symlink d9/db/d6d/d4f/d75/d8b/lc9 0 2024-03-24T10:46:17.003 INFO:tasks.workunit.client.1.smithi096.stdout:9/615: read - d9/db/d6d/d7f/fb6 zero size 2024-03-24T10:46:17.004 INFO:tasks.workunit.client.1.smithi096.stdout:0/594: dread d0/d11/d24/f2d [0,4194304] 0 2024-03-24T10:46:17.008 INFO:tasks.workunit.client.1.smithi096.stdout:6/624: link d8/d12/d32/d66/l9c d8/d12/d28/d36/d65/lc0 0 2024-03-24T10:46:17.008 INFO:tasks.workunit.client.1.smithi096.stdout:6/625: dread - d8/d12/d32/d38/d59/db8/fb1 zero size 2024-03-24T10:46:17.009 INFO:tasks.workunit.client.1.smithi096.stdout:1/516: rename d3/f9 to d3/d23/d8b/d5c/d6d/fac 0 2024-03-24T10:46:17.009 INFO:tasks.workunit.client.1.smithi096.stdout:1/517: creat d3/d23/d8b/d6a/fad x:0 0 0 2024-03-24T10:46:17.011 INFO:tasks.workunit.client.1.smithi096.stdout:1/518: dread d3/d23/d8b/f35 [0,4194304] 0 2024-03-24T10:46:17.011 INFO:tasks.workunit.client.1.smithi096.stdout:1/519: write d3/d23/d8b/f4c [741995,10345] 0 2024-03-24T10:46:17.011 INFO:tasks.workunit.client.1.smithi096.stdout:1/520: truncate d3/d23/f68 4749392 0 2024-03-24T10:46:17.012 INFO:tasks.workunit.client.1.smithi096.stdout:8/670: mkdir d8/d11/d22/d4c/d82/da1/dd4 0 2024-03-24T10:46:17.015 INFO:tasks.workunit.client.1.smithi096.stdout:3/627: rmdir d0/d13/d45/d47 39 2024-03-24T10:46:17.015 INFO:tasks.workunit.client.1.smithi096.stdout:3/628: chown d0/d9/l23 126193924 1 2024-03-24T10:46:17.018 INFO:tasks.workunit.client.1.smithi096.stdout:9/616: mkdir d9/d15/d26/d29/d73/d49/db7/dca 0 2024-03-24T10:46:17.018 INFO:tasks.workunit.client.1.smithi096.stdout:9/617: getdents d9/d79 0 2024-03-24T10:46:17.018 INFO:tasks.workunit.client.1.smithi096.stdout:9/618: chown d9/d15/d26/d29/caf 2620 1 2024-03-24T10:46:17.020 INFO:tasks.workunit.client.1.smithi096.stdout:0/595: stat d0/d11/d43/d75/l85 0 2024-03-24T10:46:17.020 INFO:tasks.workunit.client.1.smithi096.stdout:0/596: chown d0/cc 39 1 2024-03-24T10:46:17.021 INFO:tasks.workunit.client.1.smithi096.stdout:6/626: unlink d8/d12/d28/f4e 0 2024-03-24T10:46:17.021 INFO:tasks.workunit.client.1.smithi096.stdout:6/627: truncate d8/d12/d32/d38/d59/f98 445598 0 2024-03-24T10:46:17.021 INFO:tasks.workunit.client.1.smithi096.stdout:6/628: write d8/d10/d76/f8d [2159215,38655] 0 2024-03-24T10:46:17.025 INFO:tasks.workunit.client.1.smithi096.stdout:1/521: symlink d3/d84/d9a/lae 0 2024-03-24T10:46:17.028 INFO:tasks.workunit.client.1.smithi096.stdout:3/629: symlink d0/d13/d41/d7f/ld1 0 2024-03-24T10:46:17.030 INFO:tasks.workunit.client.1.smithi096.stdout:9/619: creat d9/d15/d26/d29/d73/d43/fcb x:0 0 0 2024-03-24T10:46:17.031 INFO:tasks.workunit.client.1.smithi096.stdout:0/597: getdents d0/d11/d16/d21 0 2024-03-24T10:46:17.033 INFO:tasks.workunit.client.1.smithi096.stdout:6/629: creat d8/d12/d28/d36/d65/fc1 x:0 0 0 2024-03-24T10:46:17.035 INFO:tasks.workunit.client.1.smithi096.stdout:1/522: link d3/d23/d8b/l29 d3/d23/d71/laf 0 2024-03-24T10:46:17.039 INFO:tasks.workunit.client.1.smithi096.stdout:1/523: dread d3/d73/f7a [0,4194304] 0 2024-03-24T10:46:17.040 INFO:tasks.workunit.client.1.smithi096.stdout:1/524: chown d3/d23/d8b/l48 34246209 1 2024-03-24T10:46:17.040 INFO:tasks.workunit.client.1.smithi096.stdout:1/525: fdatasync d3/d73/f7a 0 2024-03-24T10:46:17.040 INFO:tasks.workunit.client.1.smithi096.stdout:3/630: creat d0/d13/d45/d7d/d39/d48/dbd/dbc/dcc/fd2 x:0 0 0 2024-03-24T10:46:17.040 INFO:tasks.workunit.client.1.smithi096.stdout:3/631: chown d0/d13/d45/d47 7324 1 2024-03-24T10:46:17.041 INFO:tasks.workunit.client.1.smithi096.stdout:9/620: mkdir d9/db/d6d/d4f/d75/dcc 0 2024-03-24T10:46:17.041 INFO:tasks.workunit.client.1.smithi096.stdout:9/621: dread - d9/d7c/f9c zero size 2024-03-24T10:46:17.044 INFO:tasks.workunit.client.1.smithi096.stdout:1/526: mknod d3/d23/d8b/d6a/cb0 0 2024-03-24T10:46:17.051 INFO:tasks.workunit.client.1.smithi096.stdout:1/527: mknod d3/d23/d8b/d5c/d9c/cb1 0 2024-03-24T10:46:17.053 INFO:tasks.workunit.client.1.smithi096.stdout:1/528: mkdir d3/db2 0 2024-03-24T10:46:17.055 INFO:tasks.workunit.client.1.smithi096.stdout:0/598: dwrite d0/d11/f6d [0,4194304] 0 2024-03-24T10:46:17.055 INFO:tasks.workunit.client.1.smithi096.stdout:1/529: truncate d3/d23/f62 1524677 0 2024-03-24T10:46:17.056 INFO:tasks.workunit.client.1.smithi096.stdout:6/630: dwrite d8/d12/d28/f33 [4194304,4194304] 0 2024-03-24T10:46:17.057 INFO:tasks.workunit.client.1.smithi096.stdout:0/599: creat d0/d11/d16/d21/d81/d8f/fc6 x:0 0 0 2024-03-24T10:46:17.057 INFO:tasks.workunit.client.1.smithi096.stdout:0/600: write d0/d11/d20/d2b/fb4 [247593,54161] 0 2024-03-24T10:46:17.057 INFO:tasks.workunit.client.1.smithi096.stdout:3/632: rmdir d0/d13/d45/d7d/d39/d48/dbd/dbc/dcc 39 2024-03-24T10:46:17.059 INFO:tasks.workunit.client.1.smithi096.stdout:1/530: truncate d3/d23/d71/f8a 152276 0 2024-03-24T10:46:17.064 INFO:tasks.workunit.client.1.smithi096.stdout:6/631: creat d8/d12/d28/d36/d65/d6b/fc2 x:0 0 0 2024-03-24T10:46:17.066 INFO:tasks.workunit.client.1.smithi096.stdout:3/633: symlink d0/d13/d45/d7d/d39/d48/dbd/ld3 0 2024-03-24T10:46:17.066 INFO:tasks.workunit.client.1.smithi096.stdout:3/634: write d0/d13/d45/d52/f5b [3856912,65188] 0 2024-03-24T10:46:17.069 INFO:tasks.workunit.client.1.smithi096.stdout:1/531: mkdir d3/d36/db3 0 2024-03-24T10:46:17.074 INFO:tasks.workunit.client.1.smithi096.stdout:6/632: creat d8/d10/d76/fc3 x:0 0 0 2024-03-24T10:46:17.075 INFO:tasks.workunit.client.1.smithi096.stdout:9/622: dwrite d9/db/d6d/d7f/f59 [0,4194304] 0 2024-03-24T10:46:17.076 INFO:tasks.workunit.client.1.smithi096.stdout:9/623: write d9/db/d6d/f56 [4433064,125037] 0 2024-03-24T10:46:17.076 INFO:tasks.workunit.client.1.smithi096.stdout:0/601: dwrite d0/d11/d16/d21/d81/d7f/f5e [0,4194304] 0 2024-03-24T10:46:17.077 INFO:tasks.workunit.client.1.smithi096.stdout:1/532: creat d3/d23/d2f/d65/fb4 x:0 0 0 2024-03-24T10:46:17.086 INFO:tasks.workunit.client.1.smithi096.stdout:6/633: dread d8/d10/d19/f30 [0,4194304] 0 2024-03-24T10:46:17.086 INFO:tasks.workunit.client.1.smithi096.stdout:0/602: mknod d0/d11/d20/d2b/cc7 0 2024-03-24T10:46:17.086 INFO:tasks.workunit.client.1.smithi096.stdout:0/603: fdatasync d0/d11/d16/d21/d81/d8f/d9a/fbd 0 2024-03-24T10:46:17.088 INFO:tasks.workunit.client.1.smithi096.stdout:1/533: rename d3/d23/l25 to d3/d23/lb5 0 2024-03-24T10:46:17.090 INFO:tasks.workunit.client.1.smithi096.stdout:6/634: creat d8/d12/d32/fc4 x:0 0 0 2024-03-24T10:46:17.092 INFO:tasks.workunit.client.1.smithi096.stdout:0/604: stat d0/c6 0 2024-03-24T10:46:17.094 INFO:tasks.workunit.client.1.smithi096.stdout:1/534: dread d3/d23/f56 [0,4194304] 0 2024-03-24T10:46:17.095 INFO:tasks.workunit.client.1.smithi096.stdout:3/635: dwrite d0/d9/f49 [4194304,4194304] 0 2024-03-24T10:46:17.095 INFO:tasks.workunit.client.1.smithi096.stdout:3/636: chown d0/d9/d22 24317262 1 2024-03-24T10:46:17.095 INFO:tasks.workunit.client.1.smithi096.stdout:9/624: dwrite d9/db/d6d/fb0 [0,4194304] 0 2024-03-24T10:46:17.095 INFO:tasks.workunit.client.1.smithi096.stdout:9/625: write d9/d15/d26/d29/d73/d43/d48/d51/f72 [384205,5733] 0 2024-03-24T10:46:17.095 INFO:tasks.workunit.client.1.smithi096.stdout:9/626: write d9/d15/d26/d29/d73/f4b [2718226,123218] 0 2024-03-24T10:46:17.096 INFO:tasks.workunit.client.1.smithi096.stdout:1/535: unlink d3/c31 0 2024-03-24T10:46:17.096 INFO:tasks.workunit.client.1.smithi096.stdout:1/536: chown d3/d23/d8b/d5c/d6d/l76 0 1 2024-03-24T10:46:17.097 INFO:tasks.workunit.client.1.smithi096.stdout:1/537: chown d3/d9b/fa4 580147635 1 2024-03-24T10:46:17.097 INFO:tasks.workunit.client.1.smithi096.stdout:3/637: dread d0/d13/d45/d47/f76 [0,4194304] 0 2024-03-24T10:46:17.097 INFO:tasks.workunit.client.1.smithi096.stdout:3/638: write d0/d13/f21 [7515714,101081] 0 2024-03-24T10:46:17.102 INFO:tasks.workunit.client.1.smithi096.stdout:1/538: symlink d3/d23/d8b/d99/lb6 0 2024-03-24T10:46:17.112 INFO:tasks.workunit.client.1.smithi096.stdout:1/539: unlink d3/d23/l2c 0 2024-03-24T10:46:17.112 INFO:tasks.workunit.client.1.smithi096.stdout:1/540: dread - d3/d23/d8b/d6a/fad zero size 2024-03-24T10:46:17.112 INFO:tasks.workunit.client.1.smithi096.stdout:1/541: write d3/d23/d2f/f7e [48836,120336] 0 2024-03-24T10:46:17.125 INFO:tasks.workunit.client.1.smithi096.stdout:9/627: dwrite d9/d15/d26/d29/d73/d43/d48/d51/f72 [0,4194304] 0 2024-03-24T10:46:17.125 INFO:tasks.workunit.client.1.smithi096.stdout:9/628: truncate d9/d15/d26/d29/d73/d43/fcb 380462 0 2024-03-24T10:46:17.126 INFO:tasks.workunit.client.1.smithi096.stdout:3/639: dwrite d0/d13/d45/d47/fb9 [0,4194304] 0 2024-03-24T10:46:17.126 INFO:tasks.workunit.client.1.smithi096.stdout:0/605: dwrite d0/f73 [0,4194304] 0 2024-03-24T10:46:17.126 INFO:tasks.workunit.client.1.smithi096.stdout:0/606: chown d0/d11/d20/d56/fb2 499 1 2024-03-24T10:46:17.128 INFO:tasks.workunit.client.1.smithi096.stdout:9/629: mkdir d9/d15/d26/d29/dcd 0 2024-03-24T10:46:17.129 INFO:tasks.workunit.client.1.smithi096.stdout:1/542: dwrite d3/d23/d2f/f3d [0,4194304] 0 2024-03-24T10:46:17.136 INFO:tasks.workunit.client.1.smithi096.stdout:0/607: mknod d0/d11/d16/d21/d25/cc8 0 2024-03-24T10:46:17.136 INFO:tasks.workunit.client.1.smithi096.stdout:0/608: readlink d0/d11/d24/lb7 0 2024-03-24T10:46:17.136 INFO:tasks.workunit.client.1.smithi096.stdout:3/640: dread d0/d13/f21 [0,4194304] 0 2024-03-24T10:46:17.137 INFO:tasks.workunit.client.1.smithi096.stdout:3/641: write d0/f72 [1559570,69613] 0 2024-03-24T10:46:17.137 INFO:tasks.workunit.client.1.smithi096.stdout:3/642: fsync d0/d9/d22/f3d 0 2024-03-24T10:46:17.141 INFO:tasks.workunit.client.1.smithi096.stdout:1/543: rename d3/d23/d8b/f35 to d3/d23/fb7 0 2024-03-24T10:46:17.141 INFO:tasks.workunit.client.1.smithi096.stdout:1/544: chown d3/d36/c74 323 1 2024-03-24T10:46:17.141 INFO:tasks.workunit.client.1.smithi096.stdout:2/619: sync 2024-03-24T10:46:17.142 INFO:tasks.workunit.client.1.smithi096.stdout:9/630: dwrite d9/db/d6d/d4f/d75/d8b/d8e/fa1 [0,4194304] 0 2024-03-24T10:46:17.143 INFO:tasks.workunit.client.1.smithi096.stdout:9/631: fdatasync d9/d7c/f82 0 2024-03-24T10:46:17.143 INFO:tasks.workunit.client.1.smithi096.stdout:0/609: creat d0/d11/dc3/fc9 x:0 0 0 2024-03-24T10:46:17.143 INFO:tasks.workunit.client.1.smithi096.stdout:0/610: truncate d0/d11/f66 423630 0 2024-03-24T10:46:17.144 INFO:tasks.workunit.client.1.smithi096.stdout:3/643: creat d0/d13/d45/d7d/d39/d48/dbd/dbc/dcc/fd4 x:0 0 0 2024-03-24T10:46:17.144 INFO:tasks.workunit.client.1.smithi096.stdout:3/644: chown d0/d9/f32 454522805 1 2024-03-24T10:46:17.147 INFO:tasks.workunit.client.1.smithi096.stdout:2/620: creat d5/de/d2a/d5e/dd6/fe3 x:0 0 0 2024-03-24T10:46:17.148 INFO:tasks.workunit.client.1.smithi096.stdout:2/621: fdatasync d5/d38/f9b 0 2024-03-24T10:46:17.158 INFO:tasks.workunit.client.1.smithi096.stdout:9/632: dwrite d9/d15/d26/d29/d73/f91 [0,4194304] 0 2024-03-24T10:46:17.159 INFO:tasks.workunit.client.1.smithi096.stdout:0/611: mknod d0/d11/d20/d65/d8a/cca 0 2024-03-24T10:46:17.159 INFO:tasks.workunit.client.1.smithi096.stdout:3/645: creat d0/d13/d45/d7d/d39/d48/dbd/d9c/da9/fd5 x:0 0 0 2024-03-24T10:46:17.160 INFO:tasks.workunit.client.1.smithi096.stdout:3/646: dread - d0/d13/d45/d7d/d39/d48/dbd/d9c/da9/fd5 zero size 2024-03-24T10:46:17.160 INFO:tasks.workunit.client.1.smithi096.stdout:1/545: getdents d3/d23/d2f 0 2024-03-24T10:46:17.165 INFO:tasks.workunit.client.1.smithi096.stdout:0/612: mknod d0/d11/d20/d9e/d80/ccb 0 2024-03-24T10:46:17.167 INFO:tasks.workunit.client.1.smithi096.stdout:0/613: dread d0/d11/d24/d9d/fa7 [0,4194304] 0 2024-03-24T10:46:17.169 INFO:tasks.workunit.client.1.smithi096.stdout:9/633: write d9/db/d6d/fb0 [2270868,90856] 0 2024-03-24T10:46:17.172 INFO:tasks.workunit.client.1.smithi096.stdout:3/647: write d0/d13/d45/d52/f53 [1741304,123859] 0 2024-03-24T10:46:17.175 INFO:tasks.workunit.client.1.smithi096.stdout:9/634: unlink d9/d15/d26/d29/d73/d49/f7d 0 2024-03-24T10:46:17.176 INFO:tasks.workunit.client.1.smithi096.stdout:3/648: truncate d0/d13/d45/d52/fc1 296789 0 2024-03-24T10:46:17.177 INFO:tasks.workunit.client.1.smithi096.stdout:3/649: stat d0/d9/d22/f3d 0 2024-03-24T10:46:17.177 INFO:tasks.workunit.client.1.smithi096.stdout:3/650: chown d0/d13/f57 0 1 2024-03-24T10:46:17.177 INFO:tasks.workunit.client.1.smithi096.stdout:3/651: creat d0/d13/d45/d7d/d39/fd6 x:0 0 0 2024-03-24T10:46:17.177 INFO:tasks.workunit.client.1.smithi096.stdout:9/635: write d9/d15/d26/d29/d73/f4b [2582739,82574] 0 2024-03-24T10:46:17.177 INFO:tasks.workunit.client.1.smithi096.stdout:9/636: chown d9/d15/l4d 308674678 1 2024-03-24T10:46:17.181 INFO:tasks.workunit.client.1.smithi096.stdout:9/637: unlink d9/db/d6d/d7f/fb6 0 2024-03-24T10:46:17.184 INFO:tasks.workunit.client.1.smithi096.stdout:9/638: mknod d9/d15/cce 0 2024-03-24T10:46:17.184 INFO:tasks.workunit.client.1.smithi096.stdout:9/639: chown d9/d15/d26/d29/d73/d49/d86/dc8 2968622 1 2024-03-24T10:46:17.184 INFO:tasks.workunit.client.1.smithi096.stdout:9/640: chown d9/d15/d26/d29/d73/d49/d8a/cb8 218 1 2024-03-24T10:46:17.184 INFO:tasks.workunit.client.1.smithi096.stdout:0/614: dwrite d0/d11/f66 [0,4194304] 0 2024-03-24T10:46:17.185 INFO:tasks.workunit.client.1.smithi096.stdout:0/615: stat d0/c9b 0 2024-03-24T10:46:17.187 INFO:tasks.workunit.client.1.smithi096.stdout:2/622: dwrite d5/d38/f6f [0,4194304] 0 2024-03-24T10:46:17.193 INFO:tasks.workunit.client.1.smithi096.stdout:9/641: write d9/db/f80 [3136973,86865] 0 2024-03-24T10:46:17.194 INFO:tasks.workunit.client.1.smithi096.stdout:2/623: getdents d5 0 2024-03-24T10:46:17.197 INFO:tasks.workunit.client.1.smithi096.stdout:2/624: creat d5/de/d2a/d5e/fe4 x:0 0 0 2024-03-24T10:46:17.204 INFO:tasks.workunit.client.1.smithi096.stdout:9/642: dread d9/f3a [0,4194304] 0 2024-03-24T10:46:17.208 INFO:tasks.workunit.client.1.smithi096.stdout:2/625: write d5/dc/d6b/d72/d91/fdc [2122557,55335] 0 2024-03-24T10:46:17.208 INFO:tasks.workunit.client.1.smithi096.stdout:0/616: dwrite d0/d11/d16/d21/d81/faa [0,4194304] 0 2024-03-24T10:46:17.208 INFO:tasks.workunit.client.1.smithi096.stdout:0/617: readlink d0/d11/d43/lb5 0 2024-03-24T10:46:17.211 INFO:tasks.workunit.client.1.smithi096.stdout:9/643: dread d9/f3a [0,4194304] 0 2024-03-24T10:46:17.211 INFO:tasks.workunit.client.1.smithi096.stdout:0/618: mkdir d0/d11/d24/dcc 0 2024-03-24T10:46:17.213 INFO:tasks.workunit.client.1.smithi096.stdout:9/644: rename d9/d15/d26/d29/d73/d49/db7/fba to d9/d15/d26/d29/d73/d49/d86/fcf 0 2024-03-24T10:46:17.215 INFO:tasks.workunit.client.1.smithi096.stdout:0/619: mkdir d0/d11/d16/d21/d81/d9c/dcd 0 2024-03-24T10:46:17.215 INFO:tasks.workunit.client.1.smithi096.stdout:0/620: fdatasync d0/d11/d16/d21/d25/d47/f7c 0 2024-03-24T10:46:17.216 INFO:tasks.workunit.client.1.smithi096.stdout:2/626: dread d5/d7c/f8c [0,4194304] 0 2024-03-24T10:46:17.216 INFO:tasks.workunit.client.1.smithi096.stdout:9/645: truncate d9/d15/d26/d29/f36 3731919 0 2024-03-24T10:46:17.217 INFO:tasks.workunit.client.1.smithi096.stdout:0/621: dread d0/f8e [0,4194304] 0 2024-03-24T10:46:17.218 INFO:tasks.workunit.client.1.smithi096.stdout:0/622: unlink d0/d11/d16/d21/d81/d7f/cf 0 2024-03-24T10:46:17.218 INFO:tasks.workunit.client.1.smithi096.stdout:0/623: write d0/d11/d16/d21/d81/d7f/f64 [3825315,82012] 0 2024-03-24T10:46:17.221 INFO:tasks.workunit.client.1.smithi096.stdout:9/646: dread d9/d15/f66 [0,4194304] 0 2024-03-24T10:46:17.224 INFO:tasks.workunit.client.1.smithi096.stdout:9/647: mknod d9/d15/d26/d29/d73/d49/db7/dc1/cd0 0 2024-03-24T10:46:17.225 INFO:tasks.workunit.client.1.smithi096.stdout:2/627: dread d5/d38/f3a [0,4194304] 0 2024-03-24T10:46:17.225 INFO:tasks.workunit.client.1.smithi096.stdout:2/628: dread - d5/d7c/d88/fd7 zero size 2024-03-24T10:46:17.225 INFO:tasks.workunit.client.1.smithi096.stdout:2/629: chown d5/dc/d12/d23/f7a 16 1 2024-03-24T10:46:17.227 INFO:tasks.workunit.client.1.smithi096.stdout:9/648: creat d9/d79/fd1 x:0 0 0 2024-03-24T10:46:17.227 INFO:tasks.workunit.client.1.smithi096.stdout:9/649: chown d9/db/f28 42 1 2024-03-24T10:46:17.229 INFO:tasks.workunit.client.1.smithi096.stdout:0/624: dread d0/d11/d16/d21/d25/d47/f6b [0,4194304] 0 2024-03-24T10:46:17.229 INFO:tasks.workunit.client.1.smithi096.stdout:0/625: truncate d0/d11/d20/d9e/d80/f5d 4391924 0 2024-03-24T10:46:17.229 INFO:tasks.workunit.client.1.smithi096.stdout:0/626: readlink d0/d11/d16/d21/d81/d7f/l29 0 2024-03-24T10:46:17.229 INFO:tasks.workunit.client.1.smithi096.stdout:2/630: symlink d5/d93/le5 0 2024-03-24T10:46:17.229 INFO:tasks.workunit.client.1.smithi096.stdout:2/631: creat d5/d79/d9f/fe6 x:0 0 0 2024-03-24T10:46:17.231 INFO:tasks.workunit.client.1.smithi096.stdout:9/650: unlink d9/db/d6d/d4f/d75/f76 0 2024-03-24T10:46:17.231 INFO:tasks.workunit.client.1.smithi096.stdout:9/651: write d9/d15/d26/d29/d73/d43/d48/f9d [727626,36908] 0 2024-03-24T10:46:17.234 INFO:tasks.workunit.client.1.smithi096.stdout:0/627: mknod d0/d11/d16/d21/d78/d82/cce 0 2024-03-24T10:46:17.239 INFO:tasks.workunit.client.1.smithi096.stdout:2/632: rename d5/de/d2a/d5e/f8d to d5/d79/d9f/dad/fe7 0 2024-03-24T10:46:17.241 INFO:tasks.workunit.client.1.smithi096.stdout:0/628: mknod d0/d11/d16/d21/d78/ccf 0 2024-03-24T10:46:17.244 INFO:tasks.workunit.client.1.smithi096.stdout:2/633: mkdir d5/d79/de8 0 2024-03-24T10:46:17.245 INFO:tasks.workunit.client.1.smithi096.stdout:0/629: truncate d0/d11/d16/d21/d25/d47/f7c 4003142 0 2024-03-24T10:46:17.245 INFO:tasks.workunit.client.1.smithi096.stdout:0/630: readlink d0/d11/d16/d21/d81/d7f/l59 0 2024-03-24T10:46:17.245 INFO:tasks.workunit.client.1.smithi096.stdout:0/631: stat d0/d11/d43/lb5 0 2024-03-24T10:46:17.246 INFO:tasks.workunit.client.1.smithi096.stdout:9/652: dwrite d9/d15/d26/d29/d73/f4b [0,4194304] 0 2024-03-24T10:46:17.248 INFO:tasks.workunit.client.1.smithi096.stdout:2/634: symlink d5/le9 0 2024-03-24T10:46:17.249 INFO:tasks.workunit.client.1.smithi096.stdout:2/635: dread d5/de/f3d [0,4194304] 0 2024-03-24T10:46:17.252 INFO:tasks.workunit.client.1.smithi096.stdout:0/632: rename d0/d11/f66 to d0/d11/d20/d56/fd0 0 2024-03-24T10:46:17.255 INFO:tasks.workunit.client.1.smithi096.stdout:9/653: rename d9/d15/f27 to d9/db/d6d/d4f/d75/dcc/fd2 0 2024-03-24T10:46:17.255 INFO:tasks.workunit.client.1.smithi096.stdout:9/654: chown d9/d15/d26/d29/d73/d49/d86/fb9 0 1 2024-03-24T10:46:17.255 INFO:tasks.workunit.client.1.smithi096.stdout:9/655: chown d9/d15/d26/d29/c83 49659 1 2024-03-24T10:46:17.258 INFO:tasks.workunit.client.1.smithi096.stdout:2/636: mkdir d5/d38/dea 0 2024-03-24T10:46:17.258 INFO:tasks.workunit.client.1.smithi096.stdout:2/637: creat d5/d79/db5/feb x:0 0 0 2024-03-24T10:46:17.261 INFO:tasks.workunit.client.1.smithi096.stdout:9/656: dread d9/db/d6d/d4f/f54 [0,4194304] 0 2024-03-24T10:46:17.261 INFO:tasks.workunit.client.1.smithi096.stdout:9/657: truncate d9/d15/d26/d29/d73/d49/d86/fb9 137363 0 2024-03-24T10:46:17.261 INFO:tasks.workunit.client.1.smithi096.stdout:9/658: fsync d9/f2f 0 2024-03-24T10:46:17.261 INFO:tasks.workunit.client.1.smithi096.stdout:9/659: readlink d9/d79/lc3 0 2024-03-24T10:46:17.261 INFO:tasks.workunit.client.1.smithi096.stdout:9/660: write d9/d15/d26/d29/d73/d43/d48/f9d [1085988,11680] 0 2024-03-24T10:46:17.262 INFO:tasks.workunit.client.1.smithi096.stdout:2/638: symlink d5/lec 0 2024-03-24T10:46:17.262 INFO:tasks.workunit.client.1.smithi096.stdout:2/639: write d5/de/d2a/f35 [61853,68345] 0 2024-03-24T10:46:17.262 INFO:tasks.workunit.client.1.smithi096.stdout:9/661: rmdir d9/d15/d26/d29/d73 39 2024-03-24T10:46:17.264 INFO:tasks.workunit.client.1.smithi096.stdout:2/640: symlink d5/dc/d12/led 0 2024-03-24T10:46:17.264 INFO:tasks.workunit.client.1.smithi096.stdout:2/641: readlink d5/dc/d12/l4d 0 2024-03-24T10:46:17.264 INFO:tasks.workunit.client.1.smithi096.stdout:2/642: creat d5/d79/db5/fee x:0 0 0 2024-03-24T10:46:17.264 INFO:tasks.workunit.client.1.smithi096.stdout:2/643: readlink d5/d79/d9f/dad/lc3 0 2024-03-24T10:46:17.265 INFO:tasks.workunit.client.1.smithi096.stdout:9/662: mknod d9/d15/d26/d29/d73/cd3 0 2024-03-24T10:46:17.265 INFO:tasks.workunit.client.1.smithi096.stdout:9/663: chown d9/d15/d26/d29/d73/d43/f85 38 1 2024-03-24T10:46:17.268 INFO:tasks.workunit.client.1.smithi096.stdout:2/644: mkdir d5/dc/d6b/def 0 2024-03-24T10:46:17.268 INFO:tasks.workunit.client.1.smithi096.stdout:0/633: dwrite d0/d11/d16/d21/d81/d7f/f64 [0,4194304] 0 2024-03-24T10:46:17.270 INFO:tasks.workunit.client.1.smithi096.stdout:9/664: read d9/d15/d26/d29/d73/d49/f50 [1786793,64804] 0 2024-03-24T10:46:17.272 INFO:tasks.workunit.client.1.smithi096.stdout:0/634: rename d0/d11/d20/d56/fd0 to d0/d11/d16/d21/d81/d9c/dcd/fd1 0 2024-03-24T10:46:17.276 INFO:tasks.workunit.client.1.smithi096.stdout:9/665: truncate d9/f13 1273608 0 2024-03-24T10:46:17.278 INFO:tasks.workunit.client.1.smithi096.stdout:0/635: rename d0/d11/d16/d21/d81/d8f/d9a/fbd to d0/d11/d16/d21/d81/d7f/fd2 0 2024-03-24T10:46:17.280 INFO:tasks.workunit.client.1.smithi096.stdout:9/666: mknod d9/d60/cd4 0 2024-03-24T10:46:17.282 INFO:tasks.workunit.client.1.smithi096.stdout:4/522: sync 2024-03-24T10:46:17.282 INFO:tasks.workunit.client.1.smithi096.stdout:4/523: chown d1/d2b/l90 69138 1 2024-03-24T10:46:17.282 INFO:tasks.workunit.client.1.smithi096.stdout:0/636: rename d0/d11/d20/d2b/f89 to d0/d11/d24/dcc/fd3 0 2024-03-24T10:46:17.289 INFO:tasks.workunit.client.1.smithi096.stdout:2/645: dwrite d5/dc/f47 [0,4194304] 0 2024-03-24T10:46:17.291 INFO:tasks.workunit.client.1.smithi096.stdout:4/524: rmdir d1/d32 39 2024-03-24T10:46:17.291 INFO:tasks.workunit.client.1.smithi096.stdout:4/525: read - d1/d2b/d40/f64 zero size 2024-03-24T10:46:17.296 INFO:tasks.workunit.client.1.smithi096.stdout:2/646: write d5/dc/d6b/fae [851358,10294] 0 2024-03-24T10:46:17.300 INFO:tasks.workunit.client.1.smithi096.stdout:9/667: dwrite d9/d65/f9f [0,4194304] 0 2024-03-24T10:46:17.302 INFO:tasks.workunit.client.1.smithi096.stdout:2/647: rmdir d5/dc/d6b 39 2024-03-24T10:46:17.304 INFO:tasks.workunit.client.1.smithi096.stdout:2/648: read d5/de/d2a/d5e/fb6 [581305,49480] 0 2024-03-24T10:46:17.304 INFO:tasks.workunit.client.1.smithi096.stdout:2/649: read - d5/d7c/d88/fd7 zero size 2024-03-24T10:46:17.305 INFO:tasks.workunit.client.1.smithi096.stdout:9/668: creat d9/d15/d26/d29/d73/d49/db7/dc1/fd5 x:0 0 0 2024-03-24T10:46:17.308 INFO:tasks.workunit.client.1.smithi096.stdout:2/650: rename d5/dc/d12/d23/cb4 to d5/d79/cf0 0 2024-03-24T10:46:17.309 INFO:tasks.workunit.client.1.smithi096.stdout:9/669: getdents d9/d15/d26/d29/d73/d43/d48 0 2024-03-24T10:46:17.309 INFO:tasks.workunit.client.1.smithi096.stdout:0/637: dwrite d0/d11/d16/d21/d25/f4d [0,4194304] 0 2024-03-24T10:46:17.310 INFO:tasks.workunit.client.1.smithi096.stdout:4/526: dwrite d1/f6f [0,4194304] 0 2024-03-24T10:46:17.310 INFO:tasks.workunit.client.1.smithi096.stdout:4/527: readlink d1/d11/l2c 0 2024-03-24T10:46:17.313 INFO:tasks.workunit.client.1.smithi096.stdout:9/670: rename d9/db/d6d/d4f/d75/d8b/lbe to d9/d15/d26/d29/d73/d43/d48/ld6 0 2024-03-24T10:46:17.315 INFO:tasks.workunit.client.1.smithi096.stdout:0/638: rmdir d0/d11/d24/dcc 39 2024-03-24T10:46:17.324 INFO:tasks.workunit.client.1.smithi096.stdout:2/651: getdents d5/dc/d12/d23 0 2024-03-24T10:46:17.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:17 smithi096 ceph-mon[38097]: mgrmap e20: smithi094.yaethu(active, since 7m), standbys: smithi096.gfonnu 2024-03-24T10:46:17.326 INFO:tasks.workunit.client.1.smithi096.stdout:9/671: dwrite d9/db/fe [0,4194304] 0 2024-03-24T10:46:17.326 INFO:tasks.workunit.client.1.smithi096.stdout:9/672: write d9/db/d6d/d4f/d75/dcc/fd2 [3485739,5966] 0 2024-03-24T10:46:17.328 INFO:tasks.workunit.client.1.smithi096.stdout:4/528: unlink d1/d11/f19 0 2024-03-24T10:46:17.331 INFO:tasks.workunit.client.1.smithi096.stdout:0/639: symlink d0/d11/d20/d2b/ld4 0 2024-03-24T10:46:17.344 INFO:tasks.workunit.client.1.smithi096.stdout:4/529: dwrite d1/f1c [0,4194304] 0 2024-03-24T10:46:17.422 INFO:tasks.workunit.client.1.smithi096.stdout:2/652: rmdir d5/dc/d6b 39 2024-03-24T10:46:17.462 INFO:tasks.workunit.client.1.smithi096.stdout:9/673: fdatasync d9/d15/d26/d29/d73/d43/d48/f98 0 2024-03-24T10:46:17.462 INFO:tasks.workunit.client.1.smithi096.stdout:9/674: dread - d9/db/d6d/d7f/f88 zero size 2024-03-24T10:46:17.464 INFO:tasks.workunit.client.1.smithi096.stdout:0/640: creat d0/d11/d20/fd5 x:0 0 0 2024-03-24T10:46:17.464 INFO:tasks.workunit.client.1.smithi096.stdout:0/641: readlink d0/d11/d43/la1 0 2024-03-24T10:46:17.465 INFO:tasks.workunit.client.1.smithi096.stdout:4/530: truncate d1/ff 5417957 0 2024-03-24T10:46:17.465 INFO:tasks.workunit.client.1.smithi096.stdout:4/531: chown d1/d32/d42/c4a 1 1 2024-03-24T10:46:17.471 INFO:tasks.workunit.client.1.smithi096.stdout:9/675: write d9/d15/d26/d29/d73/f4b [3682260,41340] 0 2024-03-24T10:46:17.474 INFO:tasks.workunit.client.1.smithi096.stdout:2/653: stat d5/d38/c8e 0 2024-03-24T10:46:17.475 INFO:tasks.workunit.client.1.smithi096.stdout:0/642: unlink d0/d11/d16/d21/d81/d9c/dcd/fd1 0 2024-03-24T10:46:17.475 INFO:tasks.workunit.client.1.smithi096.stdout:0/643: creat d0/d11/d20/d65/fd6 x:0 0 0 2024-03-24T10:46:17.477 INFO:tasks.workunit.client.1.smithi096.stdout:9/676: unlink d9/db/f14 0 2024-03-24T10:46:17.477 INFO:tasks.workunit.client.1.smithi096.stdout:9/677: creat d9/d79/fd7 x:0 0 0 2024-03-24T10:46:17.483 INFO:tasks.workunit.client.1.smithi096.stdout:4/532: dwrite d1/d32/d4c/d7e/f99 [0,4194304] 0 2024-03-24T10:46:17.483 INFO:tasks.workunit.client.1.smithi096.stdout:2/654: write d5/f5f [660036,84087] 0 2024-03-24T10:46:17.485 INFO:tasks.workunit.client.1.smithi096.stdout:9/678: rmdir d9/db/d6d/d4f 39 2024-03-24T10:46:17.492 INFO:tasks.workunit.client.1.smithi096.stdout:0/644: dwrite d0/d11/d16/d21/d25/d47/f7c [4194304,4194304] 0 2024-03-24T10:46:17.493 INFO:tasks.workunit.client.1.smithi096.stdout:2/655: mkdir d5/d79/d9f/df1 0 2024-03-24T10:46:17.494 INFO:tasks.workunit.client.1.smithi096.stdout:4/533: dread d1/d2b/d40/f46 [0,4194304] 0 2024-03-24T10:46:17.496 INFO:tasks.workunit.client.1.smithi096.stdout:5/615: sync 2024-03-24T10:46:17.497 INFO:tasks.workunit.client.1.smithi096.stdout:5/616: write d2/d7/d8/d18/f93 [604680,88760] 0 2024-03-24T10:46:17.497 INFO:tasks.workunit.client.1.smithi096.stdout:5/617: getdents d2/d29/d32/d33/d3c/dc3 0 2024-03-24T10:46:17.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:17 smithi094 ceph-mon[29446]: mgrmap e20: smithi094.yaethu(active, since 7m), standbys: smithi096.gfonnu 2024-03-24T10:46:17.500 INFO:tasks.workunit.client.1.smithi096.stdout:0/645: link d0/d11/d24/lb7 d0/d11/d24/d2c/d84/ld7 0 2024-03-24T10:46:17.500 INFO:tasks.workunit.client.1.smithi096.stdout:0/646: chown d0/d11/d24/d2c/l77 512579 1 2024-03-24T10:46:17.501 INFO:tasks.workunit.client.1.smithi096.stdout:0/647: read d0/d11/d20/f26 [4458,11319] 0 2024-03-24T10:46:17.502 INFO:tasks.workunit.client.1.smithi096.stdout:2/656: rename d5/fa4 to d5/de/d2a/d5e/dca/ff2 0 2024-03-24T10:46:17.509 INFO:tasks.workunit.client.1.smithi096.stdout:4/534: dwrite d1/f1f [0,4194304] 0 2024-03-24T10:46:17.510 INFO:tasks.workunit.client.1.smithi096.stdout:2/657: dread f2 [0,4194304] 0 2024-03-24T10:46:17.512 INFO:tasks.workunit.client.1.smithi096.stdout:9/679: mkdir d9/d15/d26/d29/d73/d43/d48/dc0/dd8 0 2024-03-24T10:46:17.512 INFO:tasks.workunit.client.1.smithi096.stdout:9/680: write d9/db/d6d/d7f/f3f [651326,69130] 0 2024-03-24T10:46:17.513 INFO:tasks.workunit.client.1.smithi096.stdout:5/618: mkdir d2/d7/d69/dce 0 2024-03-24T10:46:17.513 INFO:tasks.workunit.client.1.smithi096.stdout:5/619: truncate d2/d29/d32/d33/d8d/f9e 334923 0 2024-03-24T10:46:17.519 INFO:tasks.workunit.client.1.smithi096.stdout:0/648: rename d0/d11/d24/d2c/cc1 to d0/d11/d24/d2c/d84/cd8 0 2024-03-24T10:46:17.521 INFO:tasks.workunit.client.1.smithi096.stdout:2/658: write d5/dc/d12/f26 [4076870,45967] 0 2024-03-24T10:46:17.521 INFO:tasks.workunit.client.1.smithi096.stdout:2/659: readlink d5/dc/d12/laa 0 2024-03-24T10:46:17.523 INFO:tasks.workunit.client.1.smithi096.stdout:0/649: creat d0/d11/d20/d9e/fd9 x:0 0 0 2024-03-24T10:46:17.524 INFO:tasks.workunit.client.1.smithi096.stdout:9/681: write d9/db/fab [1913222,98296] 0 2024-03-24T10:46:17.524 INFO:tasks.workunit.client.1.smithi096.stdout:4/535: rmdir d1/d11 39 2024-03-24T10:46:17.526 INFO:tasks.workunit.client.1.smithi096.stdout:5/620: dwrite d2/d29/d32/d33/d8d/fa2 [0,4194304] 0 2024-03-24T10:46:17.526 INFO:tasks.workunit.client.1.smithi096.stdout:0/650: mkdir d0/d11/d16/d21/d78/db6/dda 0 2024-03-24T10:46:17.530 INFO:tasks.workunit.client.1.smithi096.stdout:4/536: mknod d1/d2b/d67/d5f/da9/cac 0 2024-03-24T10:46:17.530 INFO:tasks.workunit.client.1.smithi096.stdout:4/537: dread - d1/d2b/d40/fa5 zero size 2024-03-24T10:46:17.530 INFO:tasks.workunit.client.1.smithi096.stdout:4/538: stat d1/f1f 0 2024-03-24T10:46:17.530 INFO:tasks.workunit.client.1.smithi096.stdout:4/539: fdatasync d1/d32/f75 0 2024-03-24T10:46:17.532 INFO:tasks.workunit.client.1.smithi096.stdout:5/621: creat d2/d29/fcf x:0 0 0 2024-03-24T10:46:17.532 INFO:tasks.workunit.client.1.smithi096.stdout:7/773: sync 2024-03-24T10:46:17.532 INFO:tasks.workunit.client.1.smithi096.stdout:6/635: sync 2024-03-24T10:46:17.533 INFO:tasks.workunit.client.1.smithi096.stdout:8/671: sync 2024-03-24T10:46:17.536 INFO:tasks.workunit.client.1.smithi096.stdout:0/651: link d0/d11/d16/d21/d81/d8f/fc6 d0/d11/d20/fdb 0 2024-03-24T10:46:17.536 INFO:tasks.workunit.client.1.smithi096.stdout:0/652: chown d0/d11/d16/d21/d78/fa8 4032942 1 2024-03-24T10:46:17.536 INFO:tasks.workunit.client.1.smithi096.stdout:0/653: chown d0/d11/d16/d21/d25/f46 146 1 2024-03-24T10:46:17.536 INFO:tasks.workunit.client.1.smithi096.stdout:0/654: truncate d0/d11/d20/fdb 777731 0 2024-03-24T10:46:17.537 INFO:tasks.workunit.client.1.smithi096.stdout:9/682: dread d9/d15/d26/d29/d73/d43/f45 [4194304,4194304] 0 2024-03-24T10:46:17.537 INFO:tasks.workunit.client.1.smithi096.stdout:9/683: dread - d9/d7c/f9c zero size 2024-03-24T10:46:17.537 INFO:tasks.workunit.client.1.smithi096.stdout:4/540: symlink d1/d32/lad 0 2024-03-24T10:46:17.540 INFO:tasks.workunit.client.1.smithi096.stdout:2/660: dwrite d5/dc/f1c [0,4194304] 0 2024-03-24T10:46:17.542 INFO:tasks.workunit.client.1.smithi096.stdout:5/622: dread d2/d7/f3e [0,4194304] 0 2024-03-24T10:46:17.542 INFO:tasks.workunit.client.1.smithi096.stdout:7/774: mkdir d0/d13/df0/d3c/d68/d101 0 2024-03-24T10:46:17.543 INFO:tasks.workunit.client.1.smithi096.stdout:8/672: symlink d8/d11/d44/ld5 0 2024-03-24T10:46:17.543 INFO:tasks.workunit.client.1.smithi096.stdout:8/673: fsync d8/d11/d22/d4c/d65/d1d/f30 0 2024-03-24T10:46:17.543 INFO:tasks.workunit.client.1.smithi096.stdout:8/674: creat d8/fd6 x:0 0 0 2024-03-24T10:46:17.545 INFO:tasks.workunit.client.1.smithi096.stdout:0/655: link d0/d11/l35 d0/d11/d16/d21/d78/d82/ldc 0 2024-03-24T10:46:17.545 INFO:tasks.workunit.client.1.smithi096.stdout:0/656: dread - d0/d11/d16/d21/d81/d7f/fb9 zero size 2024-03-24T10:46:17.545 INFO:tasks.workunit.client.1.smithi096.stdout:0/657: chown d0/d11/d43/d75 8909 1 2024-03-24T10:46:17.545 INFO:tasks.workunit.client.1.smithi096.stdout:9/684: mkdir d9/d15/d26/d29/d73/d49/d8a/dd9 0 2024-03-24T10:46:17.547 INFO:tasks.workunit.client.1.smithi096.stdout:4/541: rmdir d1/d32/d4c/d7d/d9b 0 2024-03-24T10:46:17.549 INFO:tasks.workunit.client.1.smithi096.stdout:2/661: mkdir d5/de/d2a/df3 0 2024-03-24T10:46:17.550 INFO:tasks.workunit.client.1.smithi096.stdout:7/775: dread d0/d13/df0/f8f [0,4194304] 0 2024-03-24T10:46:17.551 INFO:tasks.workunit.client.1.smithi096.stdout:7/776: getdents d0/db/d26/d48/dd4 0 2024-03-24T10:46:17.551 INFO:tasks.workunit.client.1.smithi096.stdout:8/675: rmdir d8/d11/d22/d4c/d65/d1d/d3d 39 2024-03-24T10:46:17.554 INFO:tasks.workunit.client.1.smithi096.stdout:9/685: fsync d9/db/d6d/d7f/f3f 0 2024-03-24T10:46:17.554 INFO:tasks.workunit.client.1.smithi096.stdout:6/636: dwrite d8/d12/d32/d38/d59/db8/fbb [0,4194304] 0 2024-03-24T10:46:17.554 INFO:tasks.workunit.client.1.smithi096.stdout:5/623: write d2/d7/d8/df/d14/f31 [350965,99279] 0 2024-03-24T10:46:17.554 INFO:tasks.workunit.client.1.smithi096.stdout:5/624: chown d2/d29/d32/d3f/d84/faf 1687096 1 2024-03-24T10:46:17.555 INFO:tasks.workunit.client.1.smithi096.stdout:0/658: unlink d0/d11/c97 0 2024-03-24T10:46:17.557 INFO:tasks.workunit.client.1.smithi096.stdout:4/542: creat d1/d11/fae x:0 0 0 2024-03-24T10:46:17.559 INFO:tasks.workunit.client.1.smithi096.stdout:2/662: creat d5/dc/d12/ff4 x:0 0 0 2024-03-24T10:46:17.560 INFO:tasks.workunit.client.1.smithi096.stdout:7/777: truncate d0/db/d4f/fff 5580158 0 2024-03-24T10:46:17.560 INFO:tasks.workunit.client.1.smithi096.stdout:7/778: chown d0/d13/df0/d3c/faf 6 1 2024-03-24T10:46:17.561 INFO:tasks.workunit.client.1.smithi096.stdout:8/676: mkdir d8/d11/d22/d4c/db0/dd7 0 2024-03-24T10:46:17.561 INFO:tasks.workunit.client.1.smithi096.stdout:6/637: dread d8/d12/f5c [0,4194304] 0 2024-03-24T10:46:17.566 INFO:tasks.workunit.client.1.smithi096.stdout:9/686: dwrite d9/db/d6d/d7f/f3f [0,4194304] 0 2024-03-24T10:46:17.567 INFO:tasks.workunit.client.1.smithi096.stdout:5/625: write d2/d7/f10 [5183722,114977] 0 2024-03-24T10:46:17.570 INFO:tasks.workunit.client.1.smithi096.stdout:0/659: getdents d0/d11/d16 0 2024-03-24T10:46:17.572 INFO:tasks.workunit.client.1.smithi096.stdout:4/543: creat d1/d2b/d67/d5f/da9/faf x:0 0 0 2024-03-24T10:46:17.575 INFO:tasks.workunit.client.1.smithi096.stdout:2/663: link d5/d7c/d8f/fc9 d5/dc/d12/ff5 0 2024-03-24T10:46:17.577 INFO:tasks.workunit.client.1.smithi096.stdout:7/779: mkdir d0/da9/ddb/d102 0 2024-03-24T10:46:17.578 INFO:tasks.workunit.client.1.smithi096.stdout:6/638: truncate d8/d12/d28/f53 960882 0 2024-03-24T10:46:17.579 INFO:tasks.workunit.client.1.smithi096.stdout:9/687: rmdir d9/db/d6d/d4f/d75/d8b/d8e 39 2024-03-24T10:46:17.581 INFO:tasks.workunit.client.1.smithi096.stdout:5/626: mknod d2/d29/d32/dbb/cd0 0 2024-03-24T10:46:17.582 INFO:tasks.workunit.client.1.smithi096.stdout:0/660: mkdir d0/d11/d16/d21/d81/d8f/d9a/ddd 0 2024-03-24T10:46:17.585 INFO:tasks.workunit.client.1.smithi096.stdout:4/544: creat d1/d32/d42/fb0 x:0 0 0 2024-03-24T10:46:17.586 INFO:tasks.workunit.client.1.smithi096.stdout:2/664: mkdir d5/d79/d9f/df6 0 2024-03-24T10:46:17.588 INFO:tasks.workunit.client.1.smithi096.stdout:7/780: truncate d0/d13/d32/faa 829589 0 2024-03-24T10:46:17.589 INFO:tasks.workunit.client.1.smithi096.stdout:6/639: rename d8/d10/d13/f55 to d8/d10/d13/fc5 0 2024-03-24T10:46:17.594 INFO:tasks.workunit.client.1.smithi096.stdout:8/677: dwrite d8/f69 [0,4194304] 0 2024-03-24T10:46:17.594 INFO:tasks.workunit.client.1.smithi096.stdout:4/545: mknod d1/d2b/d67/d5f/da9/cb1 0 2024-03-24T10:46:17.595 INFO:tasks.workunit.client.1.smithi096.stdout:9/688: dread d9/db/d6d/d7f/f3f [0,4194304] 0 2024-03-24T10:46:17.596 INFO:tasks.workunit.client.1.smithi096.stdout:9/689: dread d9/d15/d26/d29/d73/d43/fcb [0,4194304] 0 2024-03-24T10:46:17.596 INFO:tasks.workunit.client.1.smithi096.stdout:9/690: chown d9/db/d6d/d4f/f7b 804399 1 2024-03-24T10:46:17.596 INFO:tasks.workunit.client.1.smithi096.stdout:9/691: chown d9/d60/c7a 65 1 2024-03-24T10:46:17.598 INFO:tasks.workunit.client.1.smithi096.stdout:6/640: mknod d8/d10/d13/d63/cc6 0 2024-03-24T10:46:17.598 INFO:tasks.workunit.client.1.smithi096.stdout:0/661: write d0/d11/d24/d2c/f32 [3086581,89218] 0 2024-03-24T10:46:17.610 INFO:tasks.workunit.client.1.smithi096.stdout:2/665: write d5/dc/d6b/f8a [2604981,11679] 0 2024-03-24T10:46:17.610 INFO:tasks.workunit.client.1.smithi096.stdout:2/666: fsync d5/d7c/f99 0 2024-03-24T10:46:17.613 INFO:tasks.workunit.client.1.smithi096.stdout:4/546: link d1/d2b/d6b/d80/la2 d1/d2b/d6b/d80/d9c/lb2 0 2024-03-24T10:46:17.615 INFO:tasks.workunit.client.1.smithi096.stdout:9/692: mknod d9/d15/d26/d29/d73/d49/db7/dc1/cda 0 2024-03-24T10:46:17.623 INFO:tasks.workunit.client.1.smithi096.stdout:1/546: sync 2024-03-24T10:46:17.623 INFO:tasks.workunit.client.1.smithi096.stdout:3/652: sync 2024-03-24T10:46:17.623 INFO:tasks.workunit.client.1.smithi096.stdout:7/781: rmdir d0/db/d4f 39 2024-03-24T10:46:17.623 INFO:tasks.workunit.client.1.smithi096.stdout:7/782: truncate d0/db/d1c/f29 4891721 0 2024-03-24T10:46:17.627 INFO:tasks.workunit.client.1.smithi096.stdout:0/662: mknod d0/d11/d16/d21/d78/cde 0 2024-03-24T10:46:17.630 INFO:tasks.workunit.client.1.smithi096.stdout:2/667: symlink d5/lf7 0 2024-03-24T10:46:17.632 INFO:tasks.workunit.client.1.smithi096.stdout:4/547: mknod d1/d2b/d67/d5f/cb3 0 2024-03-24T10:46:17.632 INFO:tasks.workunit.client.1.smithi096.stdout:4/548: getdents d1/d32/d4c/d9d 0 2024-03-24T10:46:17.642 INFO:tasks.workunit.client.1.smithi096.stdout:0/663: dread d0/d11/d16/d21/d81/d7f/f4a [0,4194304] 0 2024-03-24T10:46:17.642 INFO:tasks.workunit.client.1.smithi096.stdout:3/653: dread d0/d13/f46 [0,4194304] 0 2024-03-24T10:46:17.662 INFO:tasks.workunit.client.1.smithi096.stdout:1/547: dwrite d3/d23/d8b/d5c/d6d/fac [0,4194304] 0 2024-03-24T10:46:17.662 INFO:tasks.workunit.client.1.smithi096.stdout:3/654: dwrite d0/d9/d22/f4f [0,4194304] 0 2024-03-24T10:46:17.695 INFO:tasks.workunit.client.1.smithi096.stdout:7/783: creat d0/d13/df0/d59/d87/da0/f103 x:0 0 0 2024-03-24T10:46:17.695 INFO:tasks.workunit.client.1.smithi096.stdout:7/784: read d0/d13/f3d [2900575,89124] 0 2024-03-24T10:46:17.696 INFO:tasks.workunit.client.1.smithi096.stdout:6/641: mknod d8/d10/d76/cc7 0 2024-03-24T10:46:17.696 INFO:tasks.workunit.client.1.smithi096.stdout:6/642: fdatasync d8/d10/d19/f96 0 2024-03-24T10:46:17.699 INFO:tasks.workunit.client.1.smithi096.stdout:4/549: creat d1/d2b/d59/d74/fb4 x:0 0 0 2024-03-24T10:46:17.700 INFO:tasks.workunit.client.1.smithi096.stdout:0/664: unlink d0/d11/d43/f51 0 2024-03-24T10:46:17.704 INFO:tasks.workunit.client.1.smithi096.stdout:3/655: symlink d0/d13/d45/d7d/d71/ld7 0 2024-03-24T10:46:17.704 INFO:tasks.workunit.client.1.smithi096.stdout:3/656: stat d0/d13/d45/d47/fc7 0 2024-03-24T10:46:17.705 INFO:tasks.workunit.client.1.smithi096.stdout:7/785: unlink d0/db/d4f/fff 0 2024-03-24T10:46:17.705 INFO:tasks.workunit.client.1.smithi096.stdout:7/786: write d0/d13/df0/d3c/f71 [1624954,124814] 0 2024-03-24T10:46:17.706 INFO:tasks.workunit.client.1.smithi096.stdout:2/668: rmdir d5/d79/d9f/dad 39 2024-03-24T10:46:17.707 INFO:tasks.workunit.client.1.smithi096.stdout:0/665: truncate d0/d11/d16/d21/d25/f4d 2973044 0 2024-03-24T10:46:17.710 INFO:tasks.workunit.client.1.smithi096.stdout:2/669: rmdir d5/d79/dd4 0 2024-03-24T10:46:17.712 INFO:tasks.workunit.client.1.smithi096.stdout:0/666: mkdir d0/d11/d20/d2b/ddf 0 2024-03-24T10:46:17.712 INFO:tasks.workunit.client.1.smithi096.stdout:0/667: write d0/d11/d43/fa6 [156261,97586] 0 2024-03-24T10:46:17.712 INFO:tasks.workunit.client.1.smithi096.stdout:0/668: chown d0/d11/d24/f30 234722581 1 2024-03-24T10:46:17.712 INFO:tasks.workunit.client.1.smithi096.stdout:0/669: chown d0/d11/d24/d2c/d84/lc2 118203332 1 2024-03-24T10:46:17.714 INFO:tasks.workunit.client.1.smithi096.stdout:3/657: getdents d0/d9 0 2024-03-24T10:46:17.717 INFO:tasks.workunit.client.1.smithi096.stdout:4/550: dwrite d1/ff [0,4194304] 0 2024-03-24T10:46:17.717 INFO:tasks.workunit.client.1.smithi096.stdout:7/787: link d0/db/d26/dc3/c7f d0/db/d1c/d39/c104 0 2024-03-24T10:46:17.719 INFO:tasks.workunit.client.1.smithi096.stdout:2/670: rename d5/dc/f21 to d5/d93/ff8 0 2024-03-24T10:46:17.719 INFO:tasks.workunit.client.1.smithi096.stdout:2/671: stat d5/d79/d9f/df1 0 2024-03-24T10:46:17.722 INFO:tasks.workunit.client.1.smithi096.stdout:7/788: mknod d0/db/d26/d48/da8/db1/c105 0 2024-03-24T10:46:17.722 INFO:tasks.workunit.client.1.smithi096.stdout:7/789: fsync d0/d13/df0/d3c/d68/d7e/ff4 0 2024-03-24T10:46:17.723 INFO:tasks.workunit.client.1.smithi096.stdout:3/658: rename d0/d13/d45/d7d/d71/ca1 to d0/cd8 0 2024-03-24T10:46:17.724 INFO:tasks.workunit.client.1.smithi096.stdout:3/659: write d0/d13/d45/d7d/d71/db6/fc0 [178572,94859] 0 2024-03-24T10:46:17.725 INFO:tasks.workunit.client.1.smithi096.stdout:2/672: truncate d5/dc/d12/fd1 6159 0 2024-03-24T10:46:17.727 INFO:tasks.workunit.client.1.smithi096.stdout:3/660: creat d0/d13/d1e/da4/fd9 x:0 0 0 2024-03-24T10:46:17.733 INFO:tasks.workunit.client.1.smithi096.stdout:2/673: symlink d5/d93/lf9 0 2024-03-24T10:46:17.739 INFO:tasks.workunit.client.1.smithi096.stdout:2/674: dread d5/f5f [0,4194304] 0 2024-03-24T10:46:17.740 INFO:tasks.workunit.client.1.smithi096.stdout:2/675: chown d5/d79/d9f/dad/lc3 974 1 2024-03-24T10:46:17.740 INFO:tasks.workunit.client.1.smithi096.stdout:2/676: creat d5/d7c/ffa x:0 0 0 2024-03-24T10:46:17.753 INFO:tasks.workunit.client.1.smithi096.stdout:0/670: dwrite d0/f73 [0,4194304] 0 2024-03-24T10:46:17.753 INFO:tasks.workunit.client.1.smithi096.stdout:0/671: creat d0/d11/d20/d9e/fe0 x:0 0 0 2024-03-24T10:46:17.753 INFO:tasks.workunit.client.1.smithi096.stdout:0/672: chown d0/d11/d24/d2c/c3e 698 1 2024-03-24T10:46:17.753 INFO:tasks.workunit.client.1.smithi096.stdout:3/661: dwrite d0/d13/d45/d47/f8e [0,4194304] 0 2024-03-24T10:46:17.753 INFO:tasks.workunit.client.1.smithi096.stdout:4/551: dwrite d1/d32/d4c/d7e/f99 [0,4194304] 0 2024-03-24T10:46:17.753 INFO:tasks.workunit.client.1.smithi096.stdout:4/552: chown d1/d11/f41 67366203 1 2024-03-24T10:46:17.753 INFO:tasks.workunit.client.1.smithi096.stdout:4/553: truncate d1/f3e 1119029 0 2024-03-24T10:46:17.756 INFO:tasks.workunit.client.1.smithi096.stdout:7/790: dwrite d0/d13/f35 [0,4194304] 0 2024-03-24T10:46:17.757 INFO:tasks.workunit.client.1.smithi096.stdout:7/791: stat d0/d13/df0/d59/d87/cc4 0 2024-03-24T10:46:17.757 INFO:tasks.workunit.client.1.smithi096.stdout:7/792: write d0/d13/df0/d3c/d68/d4b/fc0 [629637,24355] 0 2024-03-24T10:46:17.757 INFO:tasks.workunit.client.1.smithi096.stdout:7/793: truncate d0/f20 4313529 0 2024-03-24T10:46:17.757 INFO:tasks.workunit.client.1.smithi096.stdout:7/794: chown d0/l95 825 1 2024-03-24T10:46:17.757 INFO:tasks.workunit.client.1.smithi096.stdout:7/795: chown d0/d13/df0/d59/d87/da0/la1 6311 1 2024-03-24T10:46:17.764 INFO:tasks.workunit.client.1.smithi096.stdout:0/673: write d0/d11/d16/d21/d81/faa [2236656,107467] 0 2024-03-24T10:46:17.765 INFO:tasks.workunit.client.1.smithi096.stdout:4/554: mkdir d1/d2b/d67/d5f/da9/db5 0 2024-03-24T10:46:17.767 INFO:tasks.workunit.client.1.smithi096.stdout:2/677: dwrite d5/d93/ff8 [0,4194304] 0 2024-03-24T10:46:17.768 INFO:tasks.workunit.client.1.smithi096.stdout:4/555: dread d1/d2b/d59/d74/f62 [0,4194304] 0 2024-03-24T10:46:17.768 INFO:tasks.workunit.client.1.smithi096.stdout:4/556: chown d1/d2b/d40/l54 418910767 1 2024-03-24T10:46:17.772 INFO:tasks.workunit.client.1.smithi096.stdout:2/678: creat d5/dc/d6b/d72/ffb x:0 0 0 2024-03-24T10:46:17.774 INFO:tasks.workunit.client.1.smithi096.stdout:7/796: rmdir d0/d13/df0/d59/d87/da0 39 2024-03-24T10:46:17.776 INFO:tasks.workunit.client.1.smithi096.stdout:2/679: link d5/dc/d6b/cd2 d5/dc/d6b/d72/de0/cfc 0 2024-03-24T10:46:17.777 INFO:tasks.workunit.client.1.smithi096.stdout:7/797: symlink d0/d13/df0/d59/l106 0 2024-03-24T10:46:17.777 INFO:tasks.workunit.client.1.smithi096.stdout:7/798: dread - d0/da9/fbf zero size 2024-03-24T10:46:17.778 INFO:tasks.workunit.client.1.smithi096.stdout:4/557: getdents d1/d32/d42 0 2024-03-24T10:46:17.779 INFO:tasks.workunit.client.1.smithi096.stdout:2/680: mkdir d5/d7c/dfd 0 2024-03-24T10:46:17.781 INFO:tasks.workunit.client.1.smithi096.stdout:4/558: rename d1/d2b/d67/d5f/da9/faf to d1/d2b/d6b/d80/fb6 0 2024-03-24T10:46:17.782 INFO:tasks.workunit.client.1.smithi096.stdout:2/681: mknod d5/d38/dbd/cfe 0 2024-03-24T10:46:17.783 INFO:tasks.workunit.client.1.smithi096.stdout:7/799: rmdir d0/d13/df0/d3c/d80 39 2024-03-24T10:46:17.783 INFO:tasks.workunit.client.1.smithi096.stdout:7/800: chown d0/d13/df0/d3c/d68/d4b/de8 55988 1 2024-03-24T10:46:17.785 INFO:tasks.workunit.client.1.smithi096.stdout:3/662: dwrite d0/d9/f5e [0,4194304] 0 2024-03-24T10:46:17.785 INFO:tasks.workunit.client.1.smithi096.stdout:0/674: dwrite d0/d11/d16/d21/d78/db6/fba [0,4194304] 0 2024-03-24T10:46:17.788 INFO:tasks.workunit.client.1.smithi096.stdout:3/663: dread d0/d13/d45/d7d/d71/db6/fc0 [0,4194304] 0 2024-03-24T10:46:17.791 INFO:tasks.workunit.client.1.smithi096.stdout:4/559: read d1/f6f [3889944,25189] 0 2024-03-24T10:46:17.794 INFO:tasks.workunit.client.1.smithi096.stdout:0/675: write d0/d11/d24/f4c [61677,39903] 0 2024-03-24T10:46:17.794 INFO:tasks.workunit.client.1.smithi096.stdout:2/682: dread d5/de/d2a/f4c [0,4194304] 0 2024-03-24T10:46:17.794 INFO:tasks.workunit.client.1.smithi096.stdout:2/683: fsync d5/de/d2a/d5e/dca/ff2 0 2024-03-24T10:46:17.799 INFO:tasks.workunit.client.1.smithi096.stdout:3/664: getdents d0/d13/d45/d52 0 2024-03-24T10:46:17.802 INFO:tasks.workunit.client.1.smithi096.stdout:0/676: link d0/d11/d20/d9e/d80/f44 d0/d11/d16/d21/d78/d92/fe1 0 2024-03-24T10:46:17.804 INFO:tasks.workunit.client.1.smithi096.stdout:2/684: rename d5/c8 to d5/d38/dea/cff 0 2024-03-24T10:46:17.804 INFO:tasks.workunit.client.1.smithi096.stdout:2/685: fdatasync d5/d79/d9f/dad/fdf 0 2024-03-24T10:46:17.805 INFO:tasks.workunit.client.1.smithi096.stdout:3/665: symlink d0/d13/d45/d7d/d39/d48/dbd/lda 0 2024-03-24T10:46:17.807 INFO:tasks.workunit.client.1.smithi096.stdout:0/677: rmdir d0/d11/d16/d21/d78/db6/dda 0 2024-03-24T10:46:17.808 INFO:tasks.workunit.client.1.smithi096.stdout:7/801: dwrite d0/db/d4f/f58 [0,4194304] 0 2024-03-24T10:46:17.809 INFO:tasks.workunit.client.1.smithi096.stdout:2/686: rename d5/d38/f9b to d5/d79/de8/f100 0 2024-03-24T10:46:17.810 INFO:tasks.workunit.client.1.smithi096.stdout:0/678: mknod d0/d11/d16/d21/d25/d47/ce2 0 2024-03-24T10:46:17.812 INFO:tasks.workunit.client.1.smithi096.stdout:7/802: creat d0/d13/d32/f107 x:0 0 0 2024-03-24T10:46:17.814 INFO:tasks.workunit.client.1.smithi096.stdout:2/687: symlink d5/dc/l101 0 2024-03-24T10:46:17.816 INFO:tasks.workunit.client.1.smithi096.stdout:0/679: link d0/d11/d16/d21/d25/f4d d0/d11/d20/d65/fe3 0 2024-03-24T10:46:17.816 INFO:tasks.workunit.client.1.smithi096.stdout:0/680: readlink d0/d11/d24/lb7 0 2024-03-24T10:46:17.821 INFO:tasks.workunit.client.1.smithi096.stdout:7/803: read d0/db/d4f/f58 [1820276,59768] 0 2024-03-24T10:46:17.825 INFO:tasks.workunit.client.1.smithi096.stdout:7/804: unlink d0/d13/df0/l22 0 2024-03-24T10:46:17.828 INFO:tasks.workunit.client.1.smithi096.stdout:7/805: truncate d0/db/f8c 3378117 0 2024-03-24T10:46:17.836 INFO:tasks.workunit.client.1.smithi096.stdout:2/688: dwrite d5/d38/f76 [0,4194304] 0 2024-03-24T10:46:17.842 INFO:tasks.workunit.client.1.smithi096.stdout:0/681: dwrite d0/d11/d16/d21/d81/d7f/fbc [0,4194304] 0 2024-03-24T10:46:17.859 INFO:tasks.workunit.client.1.smithi096.stdout:2/689: dwrite d5/dc/d12/ff4 [0,4194304] 0 2024-03-24T10:46:17.865 INFO:tasks.workunit.client.1.smithi096.stdout:2/690: mkdir d5/d38/dbd/ddb/d102 0 2024-03-24T10:46:17.865 INFO:tasks.workunit.client.1.smithi096.stdout:2/691: stat f3 0 2024-03-24T10:46:17.874 INFO:tasks.workunit.client.1.smithi096.stdout:2/692: write d5/d79/de8/f100 [342071,42567] 0 2024-03-24T10:46:17.874 INFO:tasks.workunit.client.1.smithi096.stdout:2/693: write d5/dc/d12/f20 [4129765,81593] 0 2024-03-24T10:46:17.882 INFO:tasks.workunit.client.1.smithi096.stdout:2/694: creat d5/de/f103 x:0 0 0 2024-03-24T10:46:17.886 INFO:tasks.workunit.client.1.smithi096.stdout:2/695: creat d5/de/d2a/d5e/dca/f104 x:0 0 0 2024-03-24T10:46:17.886 INFO:tasks.workunit.client.1.smithi096.stdout:2/696: truncate d5/d7c/ffa 986323 0 2024-03-24T10:46:17.886 INFO:tasks.workunit.client.1.smithi096.stdout:2/697: fdatasync d5/dc/f58 0 2024-03-24T10:46:17.886 INFO:tasks.workunit.client.1.smithi096.stdout:2/698: truncate d5/dc/f83 1091712 0 2024-03-24T10:46:17.889 INFO:tasks.workunit.client.1.smithi096.stdout:0/682: truncate d0/d11/d24/f4c 2249715 0 2024-03-24T10:46:17.889 INFO:tasks.workunit.client.1.smithi096.stdout:0/683: truncate d0/d11/d20/d9e/d80/f41 988362 0 2024-03-24T10:46:17.896 INFO:tasks.workunit.client.1.smithi096.stdout:2/699: write d5/d7c/d88/d90/fb1 [3021413,14305] 0 2024-03-24T10:46:17.900 INFO:tasks.workunit.client.1.smithi096.stdout:2/700: stat d5/d7c/cd8 0 2024-03-24T10:46:17.906 INFO:tasks.workunit.client.1.smithi096.stdout:2/701: mkdir d5/dc/d12/d23/d105 0 2024-03-24T10:46:17.908 INFO:tasks.workunit.client.1.smithi096.stdout:2/702: read d5/dc/fb7 [133698,44819] 0 2024-03-24T10:46:17.908 INFO:tasks.workunit.client.1.smithi096.stdout:2/703: dread - d5/de/f103 zero size 2024-03-24T10:46:17.908 INFO:tasks.workunit.client.1.smithi096.stdout:2/704: creat d5/d79/d9f/f106 x:0 0 0 2024-03-24T10:46:17.908 INFO:tasks.workunit.client.1.smithi096.stdout:2/705: chown d5/d7c/ca3 514799 1 2024-03-24T10:46:17.910 INFO:tasks.workunit.client.1.smithi096.stdout:0/684: dwrite d0/d11/d20/d91/fa0 [0,4194304] 0 2024-03-24T10:46:17.912 INFO:tasks.workunit.client.1.smithi096.stdout:2/706: dread d5/dc/d12/f16 [0,4194304] 0 2024-03-24T10:46:17.922 INFO:tasks.workunit.client.1.smithi096.stdout:0/685: symlink d0/d11/d16/d21/d81/d7f/le4 0 2024-03-24T10:46:17.922 INFO:tasks.workunit.client.1.smithi096.stdout:0/686: chown d0/f72 1 1 2024-03-24T10:46:17.925 INFO:tasks.workunit.client.1.smithi096.stdout:0/687: creat d0/d11/d20/d91/fe5 x:0 0 0 2024-03-24T10:46:17.931 INFO:tasks.workunit.client.1.smithi096.stdout:2/707: dwrite d5/d7c/d88/d90/fb1 [0,4194304] 0 2024-03-24T10:46:17.944 INFO:tasks.workunit.client.1.smithi096.stdout:2/708: write d5/d7c/f95 [367425,67122] 0 2024-03-24T10:46:17.945 INFO:tasks.workunit.client.1.smithi096.stdout:2/709: stat d5/de/d2a/d5e/c60 0 2024-03-24T10:46:17.945 INFO:tasks.workunit.client.1.smithi096.stdout:2/710: stat d5/dc/f5b 0 2024-03-24T10:46:17.945 INFO:tasks.workunit.client.1.smithi096.stdout:2/711: fdatasync d5/de/d2a/f2b 0 2024-03-24T10:46:17.958 INFO:tasks.workunit.client.1.smithi096.stdout:2/712: dwrite d5/dc/d12/f31 [4194304,4194304] 0 2024-03-24T10:46:17.963 INFO:tasks.workunit.client.1.smithi096.stdout:0/688: rmdir d0/d11/d16/d21/d81 39 2024-03-24T10:46:17.964 INFO:tasks.workunit.client.1.smithi096.stdout:2/713: dread d5/d7c/d88/f69 [0,4194304] 0 2024-03-24T10:46:17.965 INFO:tasks.workunit.client.1.smithi096.stdout:0/689: link d0/d11/d16/d21/d81/d8f/d9a/ca9 d0/d11/d16/d21/ce6 0 2024-03-24T10:46:17.968 INFO:tasks.workunit.client.1.smithi096.stdout:2/714: symlink d5/d79/d9f/dad/l107 0 2024-03-24T10:46:17.968 INFO:tasks.workunit.client.1.smithi096.stdout:2/715: chown d5/dc/d12/ff5 79730 1 2024-03-24T10:46:17.970 INFO:tasks.workunit.client.1.smithi096.stdout:0/690: truncate d0/d11/d20/f1f 3868405 0 2024-03-24T10:46:17.970 INFO:tasks.workunit.client.1.smithi096.stdout:0/691: chown d0/d11/d16/d21/d81/d8f/fc6 0 1 2024-03-24T10:46:17.972 INFO:tasks.workunit.client.1.smithi096.stdout:2/716: mkdir d5/de/d2a/d5e/dd6/d108 0 2024-03-24T10:46:17.974 INFO:tasks.workunit.client.1.smithi096.stdout:0/692: link d0/d11/d16/d21/d81/faa d0/d11/d20/d56/fe7 0 2024-03-24T10:46:17.974 INFO:tasks.workunit.client.1.smithi096.stdout:5/627: sync 2024-03-24T10:46:17.974 INFO:tasks.workunit.client.1.smithi096.stdout:8/678: sync 2024-03-24T10:46:17.974 INFO:tasks.workunit.client.1.smithi096.stdout:5/628: chown d2/d7/d8/df/d14/f5c 843 1 2024-03-24T10:46:17.975 INFO:tasks.workunit.client.1.smithi096.stdout:2/717: creat d5/dc/d12/d23/d105/f109 x:0 0 0 2024-03-24T10:46:17.975 INFO:tasks.workunit.client.1.smithi096.stdout:2/718: creat d5/de/d2a/d5e/dca/f10a x:0 0 0 2024-03-24T10:46:17.975 INFO:tasks.workunit.client.1.smithi096.stdout:2/719: stat d5/dc/d12/d23/c73 0 2024-03-24T10:46:17.976 INFO:tasks.workunit.client.1.smithi096.stdout:2/720: chown d5/d79/d9f/dad/l107 32795529 1 2024-03-24T10:46:17.976 INFO:tasks.workunit.client.1.smithi096.stdout:2/721: readlink d5/dc/l30 0 2024-03-24T10:46:17.976 INFO:tasks.workunit.client.1.smithi096.stdout:8/679: dread d8/d11/d22/d4c/d65/d1a/f26 [0,4194304] 0 2024-03-24T10:46:17.976 INFO:tasks.workunit.client.1.smithi096.stdout:0/693: rmdir d0/d11/d16/d21/d78 39 2024-03-24T10:46:17.982 INFO:tasks.workunit.client.1.smithi096.stdout:8/680: creat d8/d72/d96/d91/fd8 x:0 0 0 2024-03-24T10:46:17.982 INFO:tasks.workunit.client.1.smithi096.stdout:5/629: write d2/d29/f54 [3930979,85576] 0 2024-03-24T10:46:17.983 INFO:tasks.workunit.client.1.smithi096.stdout:5/630: stat d2/d29/d32/d3f/f9b 0 2024-03-24T10:46:17.986 INFO:tasks.workunit.client.1.smithi096.stdout:8/681: mkdir d8/d11/d22/d4c/d65/d52/dc0/dd9 0 2024-03-24T10:46:17.990 INFO:tasks.workunit.client.1.smithi096.stdout:2/722: dwrite d5/d79/db5/feb [0,4194304] 0 2024-03-24T10:46:17.990 INFO:tasks.workunit.client.1.smithi096.stdout:2/723: creat d5/d7c/f10b x:0 0 0 2024-03-24T10:46:17.990 INFO:tasks.workunit.client.1.smithi096.stdout:2/724: truncate d5/de/d2a/d5e/dca/f10a 359197 0 2024-03-24T10:46:17.990 INFO:tasks.workunit.client.1.smithi096.stdout:2/725: readlink d5/lec 0 2024-03-24T10:46:17.991 INFO:tasks.workunit.client.1.smithi096.stdout:2/726: fdatasync d5/fb0 0 2024-03-24T10:46:18.000 INFO:tasks.workunit.client.1.smithi096.stdout:0/694: write d0/d11/d20/f1f [1227589,126714] 0 2024-03-24T10:46:18.005 INFO:tasks.workunit.client.1.smithi096.stdout:0/695: rmdir d0/d11/d43/d75 39 2024-03-24T10:46:18.008 INFO:tasks.workunit.client.1.smithi096.stdout:8/682: dwrite d8/d11/d22/d4c/d65/d1a/da0/da6/fbc [4194304,4194304] 0 2024-03-24T10:46:18.008 INFO:tasks.workunit.client.1.smithi096.stdout:8/683: readlink d8/d11/d22/d4c/d65/d1d/l9e 0 2024-03-24T10:46:18.015 INFO:tasks.workunit.client.1.smithi096.stdout:0/696: write d0/d11/d20/d56/f5c [740034,90354] 0 2024-03-24T10:46:18.015 INFO:tasks.workunit.client.1.smithi096.stdout:0/697: chown d0/d11/d43/d75 5 1 2024-03-24T10:46:18.015 INFO:tasks.workunit.client.1.smithi096.stdout:0/698: write d0/d11/d20/d91/fe5 [375759,23490] 0 2024-03-24T10:46:18.017 INFO:tasks.workunit.client.1.smithi096.stdout:2/727: dwrite d5/dc/d12/f48 [0,4194304] 0 2024-03-24T10:46:18.017 INFO:tasks.workunit.client.1.smithi096.stdout:2/728: chown d5/d93/la2 15236 1 2024-03-24T10:46:18.017 INFO:tasks.workunit.client.1.smithi096.stdout:2/729: chown d5/d7c/cd8 50409 1 2024-03-24T10:46:18.017 INFO:tasks.workunit.client.1.smithi096.stdout:2/730: creat d5/de/d2a/d5e/f10c x:0 0 0 2024-03-24T10:46:18.017 INFO:tasks.workunit.client.1.smithi096.stdout:2/731: stat d5/de/d2a/d5e/ld3 0 2024-03-24T10:46:18.017 INFO:tasks.workunit.client.1.smithi096.stdout:2/732: fsync d5/d7c/d8f/fc9 0 2024-03-24T10:46:18.017 INFO:tasks.workunit.client.1.smithi096.stdout:2/733: creat d5/dc/d6b/d72/f10d x:0 0 0 2024-03-24T10:46:18.017 INFO:tasks.workunit.client.1.smithi096.stdout:2/734: fdatasync d5/d93/ff8 0 2024-03-24T10:46:18.017 INFO:tasks.workunit.client.1.smithi096.stdout:0/699: unlink d0/d11/d16/d21/d78/d82/ldc 0 2024-03-24T10:46:18.020 INFO:tasks.workunit.client.1.smithi096.stdout:2/735: mknod d5/d79/de8/c10e 0 2024-03-24T10:46:18.020 INFO:tasks.workunit.client.1.smithi096.stdout:0/700: getdents d0/d11/d24 0 2024-03-24T10:46:18.023 INFO:tasks.workunit.client.1.smithi096.stdout:2/736: rename d5/d79/d9f/dad/fe7 to d5/de/d2a/f10f 0 2024-03-24T10:46:18.024 INFO:tasks.workunit.client.1.smithi096.stdout:5/631: dwrite d2/d7/f7f [0,4194304] 0 2024-03-24T10:46:18.025 INFO:tasks.workunit.client.1.smithi096.stdout:0/701: dread d0/d11/d20/f26 [0,4194304] 0 2024-03-24T10:46:18.025 INFO:tasks.workunit.client.1.smithi096.stdout:0/702: chown d0/d11/d20/d9e/d80/f44 294458867 1 2024-03-24T10:46:18.026 INFO:tasks.workunit.client.1.smithi096.stdout:2/737: mknod d5/dc/c110 0 2024-03-24T10:46:18.026 INFO:tasks.workunit.client.1.smithi096.stdout:2/738: dread - d5/d38/fc4 zero size 2024-03-24T10:46:18.026 INFO:tasks.workunit.client.1.smithi096.stdout:2/739: readlink d5/de/d2a/d5e/ld3 0 2024-03-24T10:46:18.026 INFO:tasks.workunit.client.1.smithi096.stdout:8/684: dwrite d8/d11/d22/d4c/d82/fb2 [0,4194304] 0 2024-03-24T10:46:18.030 INFO:tasks.workunit.client.1.smithi096.stdout:0/703: mkdir d0/d11/d24/d2c/d84/de8 0 2024-03-24T10:46:18.031 INFO:tasks.workunit.client.1.smithi096.stdout:0/704: readlink d0/d11/d16/l9f 0 2024-03-24T10:46:18.031 INFO:tasks.workunit.client.1.smithi096.stdout:0/705: stat d0/c9b 0 2024-03-24T10:46:18.031 INFO:tasks.workunit.client.1.smithi096.stdout:0/706: truncate d0/d11/d20/f1f 4009629 0 2024-03-24T10:46:18.038 INFO:tasks.workunit.client.1.smithi096.stdout:5/632: dread d2/d29/d32/f40 [0,4194304] 0 2024-03-24T10:46:18.045 INFO:tasks.workunit.client.1.smithi096.stdout:2/740: dwrite d5/dc/d12/fd1 [0,4194304] 0 2024-03-24T10:46:18.048 INFO:tasks.workunit.client.1.smithi096.stdout:8/685: rename d8/d11/d22/d4c/d65/d1d/f30 to d8/d11/d85/fda 0 2024-03-24T10:46:18.048 INFO:tasks.workunit.client.1.smithi096.stdout:8/686: write d8/d72/d96/d91/fd8 [1047659,120252] 0 2024-03-24T10:46:18.048 INFO:tasks.workunit.client.1.smithi096.stdout:8/687: stat d8/d11/f19 0 2024-03-24T10:46:18.048 INFO:tasks.workunit.client.1.smithi096.stdout:8/688: fdatasync d8/d72/f76 0 2024-03-24T10:46:18.049 INFO:tasks.workunit.client.1.smithi096.stdout:0/707: creat d0/d11/d16/d21/d81/d8f/d9a/ddd/fe9 x:0 0 0 2024-03-24T10:46:18.049 INFO:tasks.workunit.client.1.smithi096.stdout:0/708: chown d0/d11/d24/d2c/lc4 14849938 1 2024-03-24T10:46:18.049 INFO:tasks.workunit.client.1.smithi096.stdout:0/709: write d0/d11/d24/f4c [524959,9749] 0 2024-03-24T10:46:18.051 INFO:tasks.workunit.client.1.smithi096.stdout:5/633: fsync d2/d7/f52 0 2024-03-24T10:46:18.057 INFO:tasks.workunit.client.1.smithi096.stdout:5/634: dread d2/d29/f54 [0,4194304] 0 2024-03-24T10:46:18.058 INFO:tasks.workunit.client.1.smithi096.stdout:1/548: sync 2024-03-24T10:46:18.058 INFO:tasks.workunit.client.1.smithi096.stdout:6/643: sync 2024-03-24T10:46:18.059 INFO:tasks.workunit.client.1.smithi096.stdout:6/644: chown d8/d10/d13/d63/fb4 135107 1 2024-03-24T10:46:18.059 INFO:tasks.workunit.client.1.smithi096.stdout:9/693: sync 2024-03-24T10:46:18.063 INFO:tasks.workunit.client.1.smithi096.stdout:2/741: rename d5/dc/d12/c29 to d5/dc/d12/d23/d105/c111 0 2024-03-24T10:46:18.063 INFO:tasks.workunit.client.1.smithi096.stdout:2/742: fdatasync d5/de/d2a/d5e/fb6 0 2024-03-24T10:46:18.063 INFO:tasks.workunit.client.1.smithi096.stdout:2/743: dread - d5/d7c/f10b zero size 2024-03-24T10:46:18.063 INFO:tasks.workunit.client.1.smithi096.stdout:4/560: sync 2024-03-24T10:46:18.063 INFO:tasks.workunit.client.1.smithi096.stdout:4/561: write d1/fe [754905,94368] 0 2024-03-24T10:46:18.063 INFO:tasks.workunit.client.1.smithi096.stdout:3/666: sync 2024-03-24T10:46:18.063 INFO:tasks.workunit.client.1.smithi096.stdout:7/806: sync 2024-03-24T10:46:18.063 INFO:tasks.workunit.client.1.smithi096.stdout:7/807: dread - d0/fd2 zero size 2024-03-24T10:46:18.066 INFO:tasks.workunit.client.1.smithi096.stdout:0/710: mknod d0/d11/d16/d21/d78/d92/cea 0 2024-03-24T10:46:18.067 INFO:tasks.workunit.client.1.smithi096.stdout:3/667: dread d0/d13/f21 [0,4194304] 0 2024-03-24T10:46:18.067 INFO:tasks.workunit.client.1.smithi096.stdout:6/645: truncate d8/d10/d13/fc5 656866 0 2024-03-24T10:46:18.069 INFO:tasks.workunit.client.1.smithi096.stdout:1/549: creat d3/d23/d71/fb8 x:0 0 0 2024-03-24T10:46:18.071 INFO:tasks.workunit.client.1.smithi096.stdout:8/689: rename d8/d11/d22/d4c/f6d to d8/d11/fdb 0 2024-03-24T10:46:18.072 INFO:tasks.workunit.client.1.smithi096.stdout:4/562: mkdir d1/d32/d4c/d9d/db7 0 2024-03-24T10:46:18.072 INFO:tasks.workunit.client.1.smithi096.stdout:4/563: chown d1/d32/d4c/d7e/f9e 827899057 1 2024-03-24T10:46:18.073 INFO:tasks.workunit.client.1.smithi096.stdout:3/668: write d0/d13/d45/d47/f8e [2494124,102851] 0 2024-03-24T10:46:18.075 INFO:tasks.workunit.client.1.smithi096.stdout:7/808: mkdir d0/d13/df0/d3c/d68/d4b/d62/d108 0 2024-03-24T10:46:18.080 INFO:tasks.workunit.client.1.smithi096.stdout:0/711: mkdir d0/d11/d16/d21/d78/deb 0 2024-03-24T10:46:18.080 INFO:tasks.workunit.client.1.smithi096.stdout:0/712: fdatasync d0/d11/d20/d56/f5c 0 2024-03-24T10:46:18.082 INFO:tasks.workunit.client.1.smithi096.stdout:6/646: truncate d8/d12/fb7 4190211 0 2024-03-24T10:46:18.083 INFO:tasks.workunit.client.1.smithi096.stdout:1/550: stat d3/d23/d2f/c2e 0 2024-03-24T10:46:18.087 INFO:tasks.workunit.client.1.smithi096.stdout:5/635: rename d2/d29/d32/d89/f8e to d2/d7/d8/d18/db6/fd1 0 2024-03-24T10:46:18.087 INFO:tasks.workunit.client.1.smithi096.stdout:5/636: fsync d2/d29/d32/d33/d37/dab/fb1 0 2024-03-24T10:46:18.087 INFO:tasks.workunit.client.1.smithi096.stdout:5/637: write d2/d7/d8/d18/db6/fd1 [1775307,15995] 0 2024-03-24T10:46:18.088 INFO:tasks.workunit.client.1.smithi096.stdout:8/690: symlink d8/d11/ldc 0 2024-03-24T10:46:18.093 INFO:tasks.workunit.client.1.smithi096.stdout:4/564: stat d1/d32/c3d 0 2024-03-24T10:46:18.093 INFO:tasks.workunit.client.1.smithi096.stdout:7/809: dwrite d0/d13/df0/d59/d87/fe0 [0,4194304] 0 2024-03-24T10:46:18.093 INFO:tasks.workunit.client.1.smithi096.stdout:7/810: write d0/db/d1c/d77/f9c [728980,110410] 0 2024-03-24T10:46:18.094 INFO:tasks.workunit.client.1.smithi096.stdout:7/811: stat d0/d13/df0/d3c 0 2024-03-24T10:46:18.099 INFO:tasks.workunit.client.1.smithi096.stdout:0/713: truncate d0/d11/d24/f30 2424347 0 2024-03-24T10:46:18.106 INFO:tasks.workunit.client.1.smithi096.stdout:9/694: rename d9/db/l9b to d9/db/d6d/d4f/ldb 0 2024-03-24T10:46:18.106 INFO:tasks.workunit.client.1.smithi096.stdout:9/695: fsync d9/db/d6d/d4f/d75/d8b/d8e/f8f 0 2024-03-24T10:46:18.106 INFO:tasks.workunit.client.1.smithi096.stdout:9/696: chown d9/d65/f9f 21861 1 2024-03-24T10:46:18.107 INFO:tasks.workunit.client.1.smithi096.stdout:1/551: dwrite d3/d23/d8b/f3e [0,4194304] 0 2024-03-24T10:46:18.108 INFO:tasks.workunit.client.1.smithi096.stdout:7/812: dwrite d0/d13/d32/f107 [0,4194304] 0 2024-03-24T10:46:18.108 INFO:tasks.workunit.client.1.smithi096.stdout:7/813: readlink d0/d13/d32/l34 0 2024-03-24T10:46:18.109 INFO:tasks.workunit.client.1.smithi096.stdout:5/638: unlink d2/d49/f5d 0 2024-03-24T10:46:18.111 INFO:tasks.workunit.client.1.smithi096.stdout:8/691: symlink d8/d11/d22/d4c/d65/ldd 0 2024-03-24T10:46:18.111 INFO:tasks.workunit.client.1.smithi096.stdout:8/692: write d8/d11/d22/d4c/d65/d52/f57 [554381,527] 0 2024-03-24T10:46:18.114 INFO:tasks.workunit.client.1.smithi096.stdout:4/565: truncate d1/d2b/d67/f8a 121383 0 2024-03-24T10:46:18.118 INFO:tasks.workunit.client.1.smithi096.stdout:0/714: mkdir d0/d11/d20/d2b/ddf/dec 0 2024-03-24T10:46:18.119 INFO:tasks.workunit.client.1.smithi096.stdout:0/715: creat d0/d11/d20/d9e/d80/fed x:0 0 0 2024-03-24T10:46:18.119 INFO:tasks.workunit.client.1.smithi096.stdout:0/716: chown d0/d11/d20/d2b/fb4 2509588 1 2024-03-24T10:46:18.119 INFO:tasks.workunit.client.1.smithi096.stdout:0/717: write d0/d11/d16/d21/d81/d7f/f5e [4246076,29208] 0 2024-03-24T10:46:18.119 INFO:tasks.workunit.client.1.smithi096.stdout:6/647: getdents d8/d7b 0 2024-03-24T10:46:18.125 INFO:tasks.workunit.client.1.smithi096.stdout:3/669: rename d0/f72 to d0/d13/d45/d47/d8f/fdb 0 2024-03-24T10:46:18.125 INFO:tasks.workunit.client.1.smithi096.stdout:3/670: stat d0/f5c 0 2024-03-24T10:46:18.130 INFO:tasks.workunit.client.1.smithi096.stdout:9/697: dread d9/f13 [0,4194304] 0 2024-03-24T10:46:18.134 INFO:tasks.workunit.client.1.smithi096.stdout:7/814: dwrite d0/d13/df0/d59/fe2 [0,4194304] 0 2024-03-24T10:46:18.134 INFO:tasks.workunit.client.1.smithi096.stdout:7/815: chown d0/db/l3e 0 1 2024-03-24T10:46:18.248 INFO:tasks.workunit.client.1.smithi096.stdout:8/693: symlink d8/d72/lde 0 2024-03-24T10:46:18.253 INFO:tasks.workunit.client.1.smithi096.stdout:4/566: rmdir d1/d32/d4c 39 2024-03-24T10:46:18.260 INFO:tasks.workunit.client.1.smithi096.stdout:0/718: symlink d0/d11/d16/d21/d78/lee 0 2024-03-24T10:46:18.261 INFO:tasks.workunit.client.1.smithi096.stdout:6/648: getdents d8/d12/d32/d38/d59 0 2024-03-24T10:46:18.261 INFO:tasks.workunit.client.1.smithi096.stdout:6/649: chown d8/d10/d19/f96 3011 1 2024-03-24T10:46:18.263 INFO:tasks.workunit.client.1.smithi096.stdout:1/552: rename d3/d23/l63 to d3/d23/d8b/d5c/d9c/lb9 0 2024-03-24T10:46:18.263 INFO:tasks.workunit.client.1.smithi096.stdout:3/671: mkdir d0/d92/ddc 0 2024-03-24T10:46:18.265 INFO:tasks.workunit.client.1.smithi096.stdout:9/698: creat d9/d15/d26/d29/d73/fdc x:0 0 0 2024-03-24T10:46:18.267 INFO:tasks.workunit.client.1.smithi096.stdout:7/816: creat d0/db/d1c/d39/da2/f109 x:0 0 0 2024-03-24T10:46:18.270 INFO:tasks.workunit.client.1.smithi096.stdout:4/567: mknod d1/d32/d4c/da6/cb8 0 2024-03-24T10:46:18.270 INFO:tasks.workunit.client.1.smithi096.stdout:4/568: write d1/d32/d42/f86 [362325,120584] 0 2024-03-24T10:46:18.273 INFO:tasks.workunit.client.1.smithi096.stdout:0/719: getdents d0/d11/d16/d21/d81/d7f 0 2024-03-24T10:46:18.278 INFO:tasks.workunit.client.1.smithi096.stdout:1/553: mknod d3/d23/cba 0 2024-03-24T10:46:18.280 INFO:tasks.workunit.client.1.smithi096.stdout:3/672: truncate d0/d13/d45/d52/f53 3023586 0 2024-03-24T10:46:18.280 INFO:tasks.workunit.client.1.smithi096.stdout:3/673: read d0/d13/d45/d47/f8e [1864213,30258] 0 2024-03-24T10:46:18.280 INFO:tasks.workunit.client.1.smithi096.stdout:3/674: dread - d0/d13/d77/fb4 zero size 2024-03-24T10:46:18.280 INFO:tasks.workunit.client.1.smithi096.stdout:3/675: chown d0/f4 144809 1 2024-03-24T10:46:18.280 INFO:tasks.workunit.client.1.smithi096.stdout:3/676: fdatasync d0/d9/d22/f4f 0 2024-03-24T10:46:18.282 INFO:tasks.workunit.client.1.smithi096.stdout:9/699: chown d9/c10 7 1 2024-03-24T10:46:18.283 INFO:tasks.workunit.client.1.smithi096.stdout:9/700: readlink d9/db/la8 0 2024-03-24T10:46:18.284 INFO:tasks.workunit.client.1.smithi096.stdout:8/694: mknod d8/d11/d22/d4c/d65/d37/cdf 0 2024-03-24T10:46:18.284 INFO:tasks.workunit.client.1.smithi096.stdout:8/695: chown d8/d11/l7c 15029256 1 2024-03-24T10:46:18.285 INFO:tasks.workunit.client.1.smithi096.stdout:3/677: write d0/f5c [494189,55765] 0 2024-03-24T10:46:18.287 INFO:tasks.workunit.client.1.smithi096.stdout:4/569: link d1/d2b/d40/c93 d1/d2b/d6b/cb9 0 2024-03-24T10:46:18.288 INFO:tasks.workunit.client.1.smithi096.stdout:4/570: write d1/d2b/d67/f6a [1291230,69372] 0 2024-03-24T10:46:18.288 INFO:tasks.workunit.client.1.smithi096.stdout:4/571: truncate d1/d32/d4c/d7d/fab 298904 0 2024-03-24T10:46:18.289 INFO:tasks.workunit.client.1.smithi096.stdout:0/720: creat d0/d11/d16/d21/fef x:0 0 0 2024-03-24T10:46:18.291 INFO:tasks.workunit.client.1.smithi096.stdout:7/817: dwrite d0/d13/d32/f107 [4194304,4194304] 0 2024-03-24T10:46:18.293 INFO:tasks.workunit.client.1.smithi096.stdout:6/650: mknod d8/d10/d76/d91/cc8 0 2024-03-24T10:46:18.299 INFO:tasks.workunit.client.1.smithi096.stdout:8/696: mknod d8/d11/d22/d4c/d67/ce0 0 2024-03-24T10:46:18.299 INFO:tasks.workunit.client.1.smithi096.stdout:8/697: stat d8/d11/d22/d4c/d65/d1a/d46/lc5 0 2024-03-24T10:46:18.299 INFO:tasks.workunit.client.1.smithi096.stdout:8/698: stat d8/d11/d22/d4c/d67/f7a 0 2024-03-24T10:46:18.299 INFO:tasks.workunit.client.1.smithi096.stdout:8/699: truncate d8/d11/d22/d4c/fca 825455 0 2024-03-24T10:46:18.303 INFO:tasks.workunit.client.1.smithi096.stdout:3/678: creat d0/d13/d1e/fdd x:0 0 0 2024-03-24T10:46:18.303 INFO:tasks.workunit.client.1.smithi096.stdout:3/679: chown d0/d13/f57 29159284 1 2024-03-24T10:46:18.303 INFO:tasks.workunit.client.1.smithi096.stdout:3/680: creat d0/d13/d77/fde x:0 0 0 2024-03-24T10:46:18.303 INFO:tasks.workunit.client.1.smithi096.stdout:3/681: write d0/d13/d45/d52/f53 [1226787,1557] 0 2024-03-24T10:46:18.304 INFO:tasks.workunit.client.1.smithi096.stdout:4/572: mkdir d1/d32/d42/dba 0 2024-03-24T10:46:18.304 INFO:tasks.workunit.client.1.smithi096.stdout:4/573: dread - d1/d32/d42/fb0 zero size 2024-03-24T10:46:18.307 INFO:tasks.workunit.client.1.smithi096.stdout:0/721: rename d0/d11/d16/d21/d78/d92/cea to d0/d11/d20/d2b/cf0 0 2024-03-24T10:46:18.309 INFO:tasks.workunit.client.1.smithi096.stdout:1/554: mknod d3/d23/d8b/d5c/cbb 0 2024-03-24T10:46:18.310 INFO:tasks.workunit.client.1.smithi096.stdout:9/701: symlink d9/d15/d26/d29/d73/d43/d48/ldd 0 2024-03-24T10:46:18.311 INFO:tasks.workunit.client.1.smithi096.stdout:8/700: dread d8/d11/f7f [0,4194304] 0 2024-03-24T10:46:18.313 INFO:tasks.workunit.client.1.smithi096.stdout:4/574: mknod d1/d2b/d59/d74/cbb 0 2024-03-24T10:46:18.313 INFO:tasks.workunit.client.1.smithi096.stdout:4/575: chown d1/d2b/l90 4870 1 2024-03-24T10:46:18.313 INFO:tasks.workunit.client.1.smithi096.stdout:4/576: write d1/d2b/d40/fa7 [830934,115665] 0 2024-03-24T10:46:18.313 INFO:tasks.workunit.client.1.smithi096.stdout:4/577: write d1/d2b/d67/f8a [721829,42354] 0 2024-03-24T10:46:18.313 INFO:tasks.workunit.client.1.smithi096.stdout:4/578: write d1/ff [6367701,31463] 0 2024-03-24T10:46:18.317 INFO:tasks.workunit.client.1.smithi096.stdout:6/651: dwrite d8/d7b/fab [0,4194304] 0 2024-03-24T10:46:18.317 INFO:tasks.workunit.client.1.smithi096.stdout:0/722: unlink d0/d11/d16/d21/d81/d7f/c4 0 2024-03-24T10:46:18.317 INFO:tasks.workunit.client.1.smithi096.stdout:7/818: dwrite d0/db/d26/d48/f8d [4194304,4194304] 0 2024-03-24T10:46:18.320 INFO:tasks.workunit.client.1.smithi096.stdout:1/555: write d3/d23/d8b/d5c/f92 [337563,24616] 0 2024-03-24T10:46:18.322 INFO:tasks.workunit.client.1.smithi096.stdout:9/702: creat d9/d15/d26/d29/d73/d49/d8a/fde x:0 0 0 2024-03-24T10:46:18.323 INFO:tasks.workunit.client.1.smithi096.stdout:8/701: creat d8/d11/d22/fe1 x:0 0 0 2024-03-24T10:46:18.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:18 smithi096 ceph-mon[38097]: pgmap v252: 65 pgs: 65 active+clean; 2.7 GiB data, 9.6 GiB used, 527 GiB / 536 GiB avail; 190 MiB/s rd, 236 MiB/s wr, 416 op/s 2024-03-24T10:46:18.327 INFO:tasks.workunit.client.1.smithi096.stdout:0/723: unlink d0/c8 0 2024-03-24T10:46:18.328 INFO:tasks.workunit.client.1.smithi096.stdout:7/819: creat d0/db/d1c/f10a x:0 0 0 2024-03-24T10:46:18.328 INFO:tasks.workunit.client.1.smithi096.stdout:7/820: readlink d0/db/d26/ldc 0 2024-03-24T10:46:18.329 INFO:tasks.workunit.client.1.smithi096.stdout:4/579: dwrite d1/d32/d4c/d7e/f9e [0,4194304] 0 2024-03-24T10:46:18.331 INFO:tasks.workunit.client.1.smithi096.stdout:1/556: mkdir d3/dbc 0 2024-03-24T10:46:18.334 INFO:tasks.workunit.client.1.smithi096.stdout:8/702: mkdir d8/d11/d22/d4c/d65/d1d/d3d/d5f/de2 0 2024-03-24T10:46:18.338 INFO:tasks.workunit.client.1.smithi096.stdout:3/682: rename d0/d13/d45/d7d/d39/d48/dbd/dbc/dcc to d0/d13/d41/ddf 0 2024-03-24T10:46:18.340 INFO:tasks.workunit.client.1.smithi096.stdout:0/724: creat d0/d11/d16/d21/d81/ff1 x:0 0 0 2024-03-24T10:46:18.341 INFO:tasks.workunit.client.1.smithi096.stdout:3/683: fdatasync d0/f5c 0 2024-03-24T10:46:18.341 INFO:tasks.workunit.client.1.smithi096.stdout:3/684: creat d0/d13/d45/d7d/d39/d48/d8d/fe0 x:0 0 0 2024-03-24T10:46:18.342 INFO:tasks.workunit.client.1.smithi096.stdout:4/580: mkdir d1/d32/d4c/da6/dbc 0 2024-03-24T10:46:18.343 INFO:tasks.workunit.client.1.smithi096.stdout:1/557: creat d3/d23/d8b/fbd x:0 0 0 2024-03-24T10:46:18.347 INFO:tasks.workunit.client.1.smithi096.stdout:9/703: getdents d9/d15/d26/d29/d73/d49/db7 0 2024-03-24T10:46:18.348 INFO:tasks.workunit.client.1.smithi096.stdout:8/703: dwrite d8/d11/d85/fda [0,4194304] 0 2024-03-24T10:46:18.348 INFO:tasks.workunit.client.1.smithi096.stdout:8/704: fsync d8/d11/d44/d45/f4d 0 2024-03-24T10:46:18.350 INFO:tasks.workunit.client.1.smithi096.stdout:3/685: dread d0/d13/d45/d7d/f63 [0,4194304] 0 2024-03-24T10:46:18.350 INFO:tasks.workunit.client.1.smithi096.stdout:3/686: dread - d0/d13/d77/fde zero size 2024-03-24T10:46:18.350 INFO:tasks.workunit.client.1.smithi096.stdout:6/652: rename f6 to d8/d12/d32/d68/fc9 0 2024-03-24T10:46:18.351 INFO:tasks.workunit.client.1.smithi096.stdout:0/725: readlink d0/d11/d24/d2c/lc4 0 2024-03-24T10:46:18.357 INFO:tasks.workunit.client.1.smithi096.stdout:7/821: rmdir d0/db/d1c/d77 39 2024-03-24T10:46:18.359 INFO:tasks.workunit.client.1.smithi096.stdout:4/581: mkdir d1/d2b/d40/dbd 0 2024-03-24T10:46:18.360 INFO:tasks.workunit.client.1.smithi096.stdout:1/558: mkdir d3/d23/d71/dbe 0 2024-03-24T10:46:18.361 INFO:tasks.workunit.client.1.smithi096.stdout:9/704: creat d9/d15/d26/d29/d73/d49/db7/fdf x:0 0 0 2024-03-24T10:46:18.361 INFO:tasks.workunit.client.1.smithi096.stdout:9/705: write d9/d15/d26/d29/d73/d49/f50 [2805591,35294] 0 2024-03-24T10:46:18.361 INFO:tasks.workunit.client.1.smithi096.stdout:8/705: getdents d8/d11/d22 0 2024-03-24T10:46:18.362 INFO:tasks.workunit.client.1.smithi096.stdout:6/653: dwrite d8/d12/d28/d36/f7d [0,4194304] 0 2024-03-24T10:46:18.362 INFO:tasks.workunit.client.1.smithi096.stdout:3/687: symlink d0/d13/d45/d7d/d39/le1 0 2024-03-24T10:46:18.364 INFO:tasks.workunit.client.1.smithi096.stdout:0/726: creat d0/d11/d43/d75/ff2 x:0 0 0 2024-03-24T10:46:18.366 INFO:tasks.workunit.client.1.smithi096.stdout:7/822: creat d0/db/d26/d48/f10b x:0 0 0 2024-03-24T10:46:18.366 INFO:tasks.workunit.client.1.smithi096.stdout:7/823: write d0/d13/df0/d3c/faf [250539,124447] 0 2024-03-24T10:46:18.369 INFO:tasks.workunit.client.1.smithi096.stdout:8/706: dread d8/d11/fdb [0,4194304] 0 2024-03-24T10:46:18.370 INFO:tasks.workunit.client.1.smithi096.stdout:4/582: mkdir d1/d32/d42/dbe 0 2024-03-24T10:46:18.370 INFO:tasks.workunit.client.1.smithi096.stdout:4/583: fdatasync d1/d2b/d6b/d80/d9c/fa1 0 2024-03-24T10:46:18.371 INFO:tasks.workunit.client.1.smithi096.stdout:1/559: mknod d3/d23/d71/dbe/cbf 0 2024-03-24T10:46:18.375 INFO:tasks.workunit.client.1.smithi096.stdout:9/706: creat d9/db/d6d/dc6/fe0 x:0 0 0 2024-03-24T10:46:18.381 INFO:tasks.workunit.client.1.smithi096.stdout:3/688: dwrite d0/d13/f1f [0,4194304] 0 2024-03-24T10:46:18.385 INFO:tasks.workunit.client.1.smithi096.stdout:9/707: dread d9/db/fe [0,4194304] 0 2024-03-24T10:46:18.391 INFO:tasks.workunit.client.1.smithi096.stdout:3/689: write d0/d9/d22/fc2 [3678695,26001] 0 2024-03-24T10:46:18.397 INFO:tasks.workunit.client.1.smithi096.stdout:6/654: chown d8/d10/d76/f97 8090 1 2024-03-24T10:46:18.401 INFO:tasks.workunit.client.1.smithi096.stdout:0/727: rename d0/d11/d20/d56/c6c to d0/d11/d16/d21/d78/d82/cf3 0 2024-03-24T10:46:18.401 INFO:tasks.workunit.client.1.smithi096.stdout:0/728: fdatasync d0/d11/d16/d21/d81/d8f/fc6 0 2024-03-24T10:46:18.403 INFO:tasks.workunit.client.1.smithi096.stdout:7/824: dwrite d0/d13/df0/d3c/d68/d4b/de8/ff1 [0,4194304] 0 2024-03-24T10:46:18.403 INFO:tasks.workunit.client.1.smithi096.stdout:8/707: dwrite d8/d11/d22/d4c/d65/d52/f57 [0,4194304] 0 2024-03-24T10:46:18.404 INFO:tasks.workunit.client.1.smithi096.stdout:8/708: chown d8/d11/d22/d4c/d65/d37/l50 207898 1 2024-03-24T10:46:18.404 INFO:tasks.workunit.client.1.smithi096.stdout:8/709: chown l6 3962 1 2024-03-24T10:46:18.404 INFO:tasks.workunit.client.1.smithi096.stdout:1/560: creat d3/db2/fc0 x:0 0 0 2024-03-24T10:46:18.404 INFO:tasks.workunit.client.1.smithi096.stdout:1/561: stat d3/d23/d8b/d8c/l9e 0 2024-03-24T10:46:18.406 INFO:tasks.workunit.client.1.smithi096.stdout:4/584: dwrite d1/d2b/d6b/f83 [0,4194304] 0 2024-03-24T10:46:18.411 INFO:tasks.workunit.client.1.smithi096.stdout:2/744: sync 2024-03-24T10:46:18.411 INFO:tasks.workunit.client.1.smithi096.stdout:9/708: mknod d9/d79/ce1 0 2024-03-24T10:46:18.412 INFO:tasks.workunit.client.1.smithi096.stdout:5/639: sync 2024-03-24T10:46:18.412 INFO:tasks.workunit.client.1.smithi096.stdout:5/640: readlink d2/d29/d32/d33/d8d/la3 0 2024-03-24T10:46:18.412 INFO:tasks.workunit.client.1.smithi096.stdout:2/745: dread d5/de/d2a/f2b [0,4194304] 0 2024-03-24T10:46:18.416 INFO:tasks.workunit.client.1.smithi096.stdout:4/585: dread d1/d2b/d67/f52 [0,4194304] 0 2024-03-24T10:46:18.416 INFO:tasks.workunit.client.1.smithi096.stdout:6/655: creat d8/d7b/fca x:0 0 0 2024-03-24T10:46:18.416 INFO:tasks.workunit.client.1.smithi096.stdout:6/656: write d8/f31 [925210,13576] 0 2024-03-24T10:46:18.419 INFO:tasks.workunit.client.1.smithi096.stdout:3/690: rename d0/d13/d45/d47/d8f/la0 to d0/d92/le2 0 2024-03-24T10:46:18.420 INFO:tasks.workunit.client.1.smithi096.stdout:3/691: fdatasync d0/d13/d45/d47/d8f/fdb 0 2024-03-24T10:46:18.424 INFO:tasks.workunit.client.1.smithi096.stdout:9/709: write d9/d15/d26/d29/d73/d43/f99 [571114,92951] 0 2024-03-24T10:46:18.424 INFO:tasks.workunit.client.1.smithi096.stdout:3/692: dread d0/d9/f1a [4194304,4194304] 0 2024-03-24T10:46:18.428 INFO:tasks.workunit.client.1.smithi096.stdout:8/710: unlink d8/d72/lde 0 2024-03-24T10:46:18.428 INFO:tasks.workunit.client.1.smithi096.stdout:8/711: fsync d8/d11/d22/d4c/faf 0 2024-03-24T10:46:18.432 INFO:tasks.workunit.client.1.smithi096.stdout:2/746: mknod d5/d38/dbd/ddb/c112 0 2024-03-24T10:46:18.432 INFO:tasks.workunit.client.1.smithi096.stdout:4/586: dwrite d1/f6f [4194304,4194304] 0 2024-03-24T10:46:18.433 INFO:tasks.workunit.client.1.smithi096.stdout:1/562: rename d3/d23/l4b to d3/d23/d8b/d5c/d6d/lc1 0 2024-03-24T10:46:18.434 INFO:tasks.workunit.client.1.smithi096.stdout:0/729: mknod d0/cf4 0 2024-03-24T10:46:18.434 INFO:tasks.workunit.client.1.smithi096.stdout:0/730: fdatasync d0/d11/d20/d65/fd6 0 2024-03-24T10:46:18.436 INFO:tasks.workunit.client.1.smithi096.stdout:9/710: rmdir d9/d15/d26/dbd 0 2024-03-24T10:46:18.436 INFO:tasks.workunit.client.1.smithi096.stdout:9/711: fsync d9/d15/d26/d29/d73/f5a 0 2024-03-24T10:46:18.437 INFO:tasks.workunit.client.1.smithi096.stdout:6/657: dwrite d8/d10/d58/f72 [0,4194304] 0 2024-03-24T10:46:18.437 INFO:tasks.workunit.client.1.smithi096.stdout:6/658: dread - d8/d12/d28/f9b zero size 2024-03-24T10:46:18.438 INFO:tasks.workunit.client.1.smithi096.stdout:3/693: unlink d0/d13/l87 0 2024-03-24T10:46:18.438 INFO:tasks.workunit.client.1.smithi096.stdout:3/694: write d0/d13/d41/fcd [2150985,63971] 0 2024-03-24T10:46:18.440 INFO:tasks.workunit.client.1.smithi096.stdout:8/712: symlink d8/d72/d96/le3 0 2024-03-24T10:46:18.440 INFO:tasks.workunit.client.1.smithi096.stdout:2/747: symlink d5/l113 0 2024-03-24T10:46:18.448 INFO:tasks.workunit.client.1.smithi096.stdout:1/563: rmdir d3/d23/d8b/d8c 39 2024-03-24T10:46:18.454 INFO:tasks.workunit.client.1.smithi096.stdout:2/748: dwrite d5/d7c/f10b [0,4194304] 0 2024-03-24T10:46:18.454 INFO:tasks.workunit.client.1.smithi096.stdout:2/749: write d5/d7c/f7e [3666749,89297] 0 2024-03-24T10:46:18.455 INFO:tasks.workunit.client.1.smithi096.stdout:2/750: chown d5/d7c/ca3 104 1 2024-03-24T10:46:18.456 INFO:tasks.workunit.client.1.smithi096.stdout:6/659: truncate d8/d12/d32/d38/d59/db8/fbb 2566699 0 2024-03-24T10:46:18.456 INFO:tasks.workunit.client.1.smithi096.stdout:6/660: fdatasync d8/d12/d28/d36/d65/d6b/f75 0 2024-03-24T10:46:18.456 INFO:tasks.workunit.client.1.smithi096.stdout:6/661: truncate d8/d12/d32/d38/d59/db8/fb1 816595 0 2024-03-24T10:46:18.456 INFO:tasks.workunit.client.1.smithi096.stdout:6/662: readlink d8/d12/l43 0 2024-03-24T10:46:18.457 INFO:tasks.workunit.client.1.smithi096.stdout:6/663: getdents d8/d12/d28/d36/d65/da4 0 2024-03-24T10:46:18.457 INFO:tasks.workunit.client.1.smithi096.stdout:8/713: mknod d8/d11/d22/d4c/d65/d1a/d49/ce4 0 2024-03-24T10:46:18.457 INFO:tasks.workunit.client.1.smithi096.stdout:8/714: readlink d8/d11/d44/d45/l79 0 2024-03-24T10:46:18.460 INFO:tasks.workunit.client.1.smithi096.stdout:4/587: rmdir d1/d2b/d89 39 2024-03-24T10:46:18.465 INFO:tasks.workunit.client.1.smithi096.stdout:1/564: link d3/d23/d2f/d65/fb4 d3/d36/fc2 0 2024-03-24T10:46:18.465 INFO:tasks.workunit.client.1.smithi096.stdout:1/565: chown d3/d23/d71/d85/c87 1733918 1 2024-03-24T10:46:18.466 INFO:tasks.workunit.client.1.smithi096.stdout:0/731: getdents d0/d11/d16/d21/d81/d8f/d9a/ddd 0 2024-03-24T10:46:18.466 INFO:tasks.workunit.client.1.smithi096.stdout:0/732: chown d0/d11/d20/d2b/ld4 9221 1 2024-03-24T10:46:18.468 INFO:tasks.workunit.client.1.smithi096.stdout:9/712: rmdir d9/d15/d26/d29/d73/d43 39 2024-03-24T10:46:18.470 INFO:tasks.workunit.client.1.smithi096.stdout:2/751: mkdir d5/d7c/d88/d114 0 2024-03-24T10:46:18.474 INFO:tasks.workunit.client.1.smithi096.stdout:6/664: creat d8/d12/d32/d68/fcb x:0 0 0 2024-03-24T10:46:18.475 INFO:tasks.workunit.client.1.smithi096.stdout:8/715: creat d8/d11/d22/d4c/d65/d52/dc0/dd9/fe5 x:0 0 0 2024-03-24T10:46:18.475 INFO:tasks.workunit.client.1.smithi096.stdout:8/716: truncate d8/d11/d22/d4c/d82/da1/fd1 380523 0 2024-03-24T10:46:18.476 INFO:tasks.workunit.client.1.smithi096.stdout:8/717: getdents d8/d11/d22/d4c/d82/da1/dd4 0 2024-03-24T10:46:18.477 INFO:tasks.workunit.client.1.smithi096.stdout:4/588: mkdir d1/d2b/d89/dbf 0 2024-03-24T10:46:18.477 INFO:tasks.workunit.client.1.smithi096.stdout:4/589: fdatasync d1/d2b/d67/f8a 0 2024-03-24T10:46:18.477 INFO:tasks.workunit.client.1.smithi096.stdout:4/590: truncate d1/d32/d4c/d7e/f99 4904981 0 2024-03-24T10:46:18.478 INFO:tasks.workunit.client.1.smithi096.stdout:5/641: rename d2/d7/d8/d18/d4d to d2/d29/d32/d33/d3c/dd2 0 2024-03-24T10:46:18.479 INFO:tasks.workunit.client.1.smithi096.stdout:1/566: creat d3/d23/d8b/d5c/fc3 x:0 0 0 2024-03-24T10:46:18.480 INFO:tasks.workunit.client.1.smithi096.stdout:0/733: mkdir d0/d11/d24/d9d/df5 0 2024-03-24T10:46:18.482 INFO:tasks.workunit.client.1.smithi096.stdout:2/752: mkdir d5/d93/d115 0 2024-03-24T10:46:18.483 INFO:tasks.workunit.client.1.smithi096.stdout:8/718: mknod d8/d11/d22/d4c/d65/d1d/d3d/d5f/de2/ce6 0 2024-03-24T10:46:18.485 INFO:tasks.workunit.client.1.smithi096.stdout:0/734: dread d0/d11/d16/d21/d25/f36 [0,4194304] 0 2024-03-24T10:46:18.485 INFO:tasks.workunit.client.1.smithi096.stdout:0/735: creat d0/d11/d24/d2c/d84/ff6 x:0 0 0 2024-03-24T10:46:18.486 INFO:tasks.workunit.client.1.smithi096.stdout:6/665: rename d8/d10/d9e to d8/d10/d13/dcc 0 2024-03-24T10:46:18.490 INFO:tasks.workunit.client.1.smithi096.stdout:9/713: dwrite d9/d15/d26/d29/f36 [0,4194304] 0 2024-03-24T10:46:18.490 INFO:tasks.workunit.client.1.smithi096.stdout:9/714: chown d9/db/l55 4892738 1 2024-03-24T10:46:18.490 INFO:tasks.workunit.client.1.smithi096.stdout:5/642: unlink d2/d29/d32/d33/d3c/d3d/c58 0 2024-03-24T10:46:18.491 INFO:tasks.workunit.client.1.smithi096.stdout:2/753: creat d5/f116 x:0 0 0 2024-03-24T10:46:18.492 INFO:tasks.workunit.client.1.smithi096.stdout:4/591: dwrite d1/d2b/d6b/d80/f9f [0,4194304] 0 2024-03-24T10:46:18.494 INFO:tasks.workunit.client.1.smithi096.stdout:1/567: dread d3/d23/d2f/f59 [0,4194304] 0 2024-03-24T10:46:18.494 INFO:tasks.workunit.client.1.smithi096.stdout:1/568: write d3/d23/d8b/d6a/f77 [941934,104850] 0 2024-03-24T10:46:18.497 INFO:tasks.workunit.client.1.smithi096.stdout:0/736: unlink d0/d11/d16/d21/d81/d7f/fd2 0 2024-03-24T10:46:18.497 INFO:tasks.workunit.client.1.smithi096.stdout:0/737: write d0/d11/d16/d21/fef [561936,16066] 0 2024-03-24T10:46:18.497 INFO:tasks.workunit.client.1.smithi096.stdout:0/738: creat d0/d11/d16/d21/d78/db6/ff7 x:0 0 0 2024-03-24T10:46:18.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:18 smithi094 ceph-mon[29446]: pgmap v252: 65 pgs: 65 active+clean; 2.7 GiB data, 9.6 GiB used, 527 GiB / 536 GiB avail; 190 MiB/s rd, 236 MiB/s wr, 416 op/s 2024-03-24T10:46:18.498 INFO:tasks.workunit.client.1.smithi096.stdout:6/666: mknod d8/d12/d32/d38/d59/db8/ccd 0 2024-03-24T10:46:18.499 INFO:tasks.workunit.client.1.smithi096.stdout:8/719: dwrite d8/f58 [4194304,4194304] 0 2024-03-24T10:46:18.502 INFO:tasks.workunit.client.1.smithi096.stdout:5/643: getdents d2/d29/d32/d33 0 2024-03-24T10:46:18.503 INFO:tasks.workunit.client.1.smithi096.stdout:2/754: creat d5/d38/dea/f117 x:0 0 0 2024-03-24T10:46:18.509 INFO:tasks.workunit.client.1.smithi096.stdout:6/667: dread d8/f9 [0,4194304] 0 2024-03-24T10:46:18.510 INFO:tasks.workunit.client.1.smithi096.stdout:1/569: creat d3/d23/d2f/fc4 x:0 0 0 2024-03-24T10:46:18.514 INFO:tasks.workunit.client.1.smithi096.stdout:0/739: mkdir d0/d11/d16/d21/d25/df8 0 2024-03-24T10:46:18.518 INFO:tasks.workunit.client.1.smithi096.stdout:8/720: mkdir d8/d11/d22/d4c/d82/da1/dd4/de7 0 2024-03-24T10:46:18.518 INFO:tasks.workunit.client.1.smithi096.stdout:8/721: write d8/d11/d22/d4c/d67/f7a [529589,54044] 0 2024-03-24T10:46:18.519 INFO:tasks.workunit.client.1.smithi096.stdout:2/755: creat d5/de/d2a/df3/f118 x:0 0 0 2024-03-24T10:46:18.520 INFO:tasks.workunit.client.1.smithi096.stdout:2/756: read d5/dc/f58 [127122,110787] 0 2024-03-24T10:46:18.521 INFO:tasks.workunit.client.1.smithi096.stdout:2/757: dread d5/de/d2a/f10f [0,4194304] 0 2024-03-24T10:46:18.526 INFO:tasks.workunit.client.1.smithi096.stdout:4/592: dwrite d1/d32/d42/f5e [4194304,4194304] 0 2024-03-24T10:46:18.527 INFO:tasks.workunit.client.1.smithi096.stdout:1/570: unlink d3/d23/d8b/d79/ca1 0 2024-03-24T10:46:18.529 INFO:tasks.workunit.client.1.smithi096.stdout:0/740: mknod d0/d11/dc3/cf9 0 2024-03-24T10:46:18.529 INFO:tasks.workunit.client.1.smithi096.stdout:0/741: read - d0/d11/d16/d21/d81/d7f/f71 zero size 2024-03-24T10:46:18.532 INFO:tasks.workunit.client.1.smithi096.stdout:5/644: dwrite d2/d7/d8/d18/db6/fd1 [0,4194304] 0 2024-03-24T10:46:18.536 INFO:tasks.workunit.client.1.smithi096.stdout:4/593: rename d1/d2b/d59/d74/f62 to d1/d2b/d67/d5f/fc0 0 2024-03-24T10:46:18.539 INFO:tasks.workunit.client.1.smithi096.stdout:2/758: link d5/d38/f45 d5/d79/d9f/f119 0 2024-03-24T10:46:18.543 INFO:tasks.workunit.client.1.smithi096.stdout:6/668: dwrite d8/d12/d32/d66/fba [0,4194304] 0 2024-03-24T10:46:18.543 INFO:tasks.workunit.client.1.smithi096.stdout:6/669: dread - d8/d12/d28/d36/d65/fc1 zero size 2024-03-24T10:46:18.543 INFO:tasks.workunit.client.1.smithi096.stdout:6/670: creat d8/d12/d28/d36/d65/d6b/fce x:0 0 0 2024-03-24T10:46:18.545 INFO:tasks.workunit.client.1.smithi096.stdout:1/571: rename d3/d23/d8b/d5c/fc3 to d3/d23/d71/d7b/d93/fc5 0 2024-03-24T10:46:18.545 INFO:tasks.workunit.client.1.smithi096.stdout:1/572: chown d3/d23/d8b/d5c/d6d 406162 1 2024-03-24T10:46:18.545 INFO:tasks.workunit.client.1.smithi096.stdout:5/645: dread d2/d29/d32/d3f/d5f/f6c [0,4194304] 0 2024-03-24T10:46:18.546 INFO:tasks.workunit.client.1.smithi096.stdout:8/722: dwrite d8/f15 [0,4194304] 0 2024-03-24T10:46:18.546 INFO:tasks.workunit.client.1.smithi096.stdout:4/594: mkdir d1/d32/d4c/dc1 0 2024-03-24T10:46:18.551 INFO:tasks.workunit.client.1.smithi096.stdout:0/742: dwrite d0/d11/d43/d75/ff2 [0,4194304] 0 2024-03-24T10:46:18.551 INFO:tasks.workunit.client.1.smithi096.stdout:0/743: write d0/d11/d16/fa5 [693518,79958] 0 2024-03-24T10:46:18.551 INFO:tasks.workunit.client.1.smithi096.stdout:0/744: truncate d0/d11/d20/d9e/fe0 292125 0 2024-03-24T10:46:18.553 INFO:tasks.workunit.client.1.smithi096.stdout:2/759: unlink d5/c6a 0 2024-03-24T10:46:18.555 INFO:tasks.workunit.client.1.smithi096.stdout:1/573: write d3/d23/f56 [4065245,116075] 0 2024-03-24T10:46:18.555 INFO:tasks.workunit.client.1.smithi096.stdout:1/574: readlink d3/d23/lb5 0 2024-03-24T10:46:18.555 INFO:tasks.workunit.client.1.smithi096.stdout:1/575: truncate d3/d23/d2f/fc4 122310 0 2024-03-24T10:46:18.558 INFO:tasks.workunit.client.1.smithi096.stdout:5/646: creat d2/d29/d32/d33/d3c/dd2/fd3 x:0 0 0 2024-03-24T10:46:18.560 INFO:tasks.workunit.client.1.smithi096.stdout:8/723: mknod d8/d11/d22/d4c/d82/da1/dd4/de7/ce8 0 2024-03-24T10:46:18.565 INFO:tasks.workunit.client.1.smithi096.stdout:5/647: dread d2/d29/d32/d33/d3c/d3d/fb5 [0,4194304] 0 2024-03-24T10:46:18.566 INFO:tasks.workunit.client.1.smithi096.stdout:1/576: dread d3/d23/d8b/d6a/f77 [0,4194304] 0 2024-03-24T10:46:18.566 INFO:tasks.workunit.client.1.smithi096.stdout:1/577: creat d3/d23/d2f/d65/fc6 x:0 0 0 2024-03-24T10:46:18.568 INFO:tasks.workunit.client.1.smithi096.stdout:6/671: dwrite d8/d10/d19/f96 [0,4194304] 0 2024-03-24T10:46:18.568 INFO:tasks.workunit.client.1.smithi096.stdout:9/715: dwrite d9/d15/d26/d29/f4e [0,4194304] 0 2024-03-24T10:46:18.568 INFO:tasks.workunit.client.1.smithi096.stdout:0/745: creat d0/d11/d16/d21/d78/deb/ffa x:0 0 0 2024-03-24T10:46:18.569 INFO:tasks.workunit.client.1.smithi096.stdout:8/724: write d8/d11/d22/d4c/d65/d1a/da0/fbd [3488659,47404] 0 2024-03-24T10:46:18.569 INFO:tasks.workunit.client.1.smithi096.stdout:2/760: mkdir d5/dc/d6b/d72/d11a 0 2024-03-24T10:46:18.569 INFO:tasks.workunit.client.1.smithi096.stdout:2/761: write d5/fb0 [4267334,59320] 0 2024-03-24T10:46:18.574 INFO:tasks.workunit.client.1.smithi096.stdout:5/648: creat d2/d7/fd4 x:0 0 0 2024-03-24T10:46:18.577 INFO:tasks.workunit.client.1.smithi096.stdout:1/578: mknod d3/d23/d8b/cc7 0 2024-03-24T10:46:18.577 INFO:tasks.workunit.client.1.smithi096.stdout:1/579: readlink d3/d23/d2f/d65/d7c/l90 0 2024-03-24T10:46:18.580 INFO:tasks.workunit.client.1.smithi096.stdout:1/580: dread d3/d23/f62 [0,4194304] 0 2024-03-24T10:46:18.683 INFO:tasks.workunit.client.1.smithi096.stdout:6/672: getdents d8/d12/d32/d38/d59/db8 0 2024-03-24T10:46:18.685 INFO:tasks.workunit.client.1.smithi096.stdout:9/716: truncate d9/d15/d26/d29/d73/f91 4143019 0 2024-03-24T10:46:18.687 INFO:tasks.workunit.client.1.smithi096.stdout:0/746: link d0/d11/d16/d21/d78/d82/cf3 d0/d11/d20/d65/d8a/cfb 0 2024-03-24T10:46:18.688 INFO:tasks.workunit.client.1.smithi096.stdout:8/725: truncate d8/d11/d22/d4c/d65/d37/f6e 2231207 0 2024-03-24T10:46:18.689 INFO:tasks.workunit.client.1.smithi096.stdout:2/762: rename d5/dc/l28 to d5/d79/d9f/df1/l11b 0 2024-03-24T10:46:18.690 INFO:tasks.workunit.client.1.smithi096.stdout:5/649: symlink d2/d29/d32/d33/d3c/dd2/ld5 0 2024-03-24T10:46:18.690 INFO:tasks.workunit.client.1.smithi096.stdout:5/650: fdatasync d2/d29/d32/d89/f9c 0 2024-03-24T10:46:18.704 INFO:tasks.workunit.client.1.smithi096.stdout:6/673: fsync d8/d10/d76/fa9 0 2024-03-24T10:46:18.707 INFO:tasks.workunit.client.1.smithi096.stdout:8/726: rmdir d8/d11/d22/d4c/db0/dd7 0 2024-03-24T10:46:18.710 INFO:tasks.workunit.client.1.smithi096.stdout:1/581: rename d3/f3f to d3/d23/d8b/d99/fc8 0 2024-03-24T10:46:18.712 INFO:tasks.workunit.client.1.smithi096.stdout:2/763: link d5/cd d5/de/d2a/d5e/dd6/d108/c11c 0 2024-03-24T10:46:18.713 INFO:tasks.workunit.client.1.smithi096.stdout:5/651: mknod d2/d29/d32/d33/d3c/cd6 0 2024-03-24T10:46:18.715 INFO:tasks.workunit.client.1.smithi096.stdout:6/674: creat d8/d12/d32/d38/d59/fcf x:0 0 0 2024-03-24T10:46:18.716 INFO:tasks.workunit.client.1.smithi096.stdout:6/675: write d8/d12/d61/f93 [938696,129050] 0 2024-03-24T10:46:18.716 INFO:tasks.workunit.client.1.smithi096.stdout:6/676: write d8/d12/d32/fc4 [658567,67104] 0 2024-03-24T10:46:18.716 INFO:tasks.workunit.client.1.smithi096.stdout:6/677: chown d8/f4a 289 1 2024-03-24T10:46:18.716 INFO:tasks.workunit.client.1.smithi096.stdout:9/717: rmdir d9/d15/d26/d29/d73/d43/d48/d51 39 2024-03-24T10:46:18.717 INFO:tasks.workunit.client.1.smithi096.stdout:0/747: rmdir d0/d11/d20/d91 39 2024-03-24T10:46:18.717 INFO:tasks.workunit.client.1.smithi096.stdout:2/764: dread d5/de/f13 [0,4194304] 0 2024-03-24T10:46:18.718 INFO:tasks.workunit.client.1.smithi096.stdout:8/727: getdents d8/d11/d22/d4c/d65/d1d 0 2024-03-24T10:46:18.718 INFO:tasks.workunit.client.1.smithi096.stdout:8/728: write d8/d11/d85/fda [4485951,14681] 0 2024-03-24T10:46:18.721 INFO:tasks.workunit.client.1.smithi096.stdout:1/582: creat d3/d23/d71/dbe/fc9 x:0 0 0 2024-03-24T10:46:18.721 INFO:tasks.workunit.client.1.smithi096.stdout:5/652: read d2/d7/d8/d18/db6/fd1 [2057489,31883] 0 2024-03-24T10:46:18.724 INFO:tasks.workunit.client.1.smithi096.stdout:5/653: dread d2/d29/d32/d3f/f66 [0,4194304] 0 2024-03-24T10:46:18.727 INFO:tasks.workunit.client.1.smithi096.stdout:9/718: rename d9/f2e to d9/db/d6d/fe2 0 2024-03-24T10:46:18.727 INFO:tasks.workunit.client.1.smithi096.stdout:9/719: fdatasync d9/db/f8c 0 2024-03-24T10:46:18.731 INFO:tasks.workunit.client.1.smithi096.stdout:8/729: creat d8/d11/fe9 x:0 0 0 2024-03-24T10:46:18.733 INFO:tasks.workunit.client.1.smithi096.stdout:6/678: getdents d8/d12/d32/d66 0 2024-03-24T10:46:18.733 INFO:tasks.workunit.client.1.smithi096.stdout:6/679: fdatasync d8/f29 0 2024-03-24T10:46:18.733 INFO:tasks.workunit.client.1.smithi096.stdout:6/680: dread - d8/d12/d61/faa zero size 2024-03-24T10:46:18.733 INFO:tasks.workunit.client.1.smithi096.stdout:8/730: dread d8/d11/d22/d4c/d65/d1d/d3d/f56 [0,4194304] 0 2024-03-24T10:46:18.733 INFO:tasks.workunit.client.1.smithi096.stdout:8/731: write d8/d11/d22/d4c/d65/d1d/f5c [4589994,119339] 0 2024-03-24T10:46:18.733 INFO:tasks.workunit.client.1.smithi096.stdout:8/732: dread d8/d11/d22/d4c/d82/da1/fd1 [0,4194304] 0 2024-03-24T10:46:18.734 INFO:tasks.workunit.client.1.smithi096.stdout:8/733: fdatasync d8/d72/f83 0 2024-03-24T10:46:18.734 INFO:tasks.workunit.client.1.smithi096.stdout:5/654: mkdir d2/dd7 0 2024-03-24T10:46:18.735 INFO:tasks.workunit.client.1.smithi096.stdout:2/765: rmdir d5/d38/dea 39 2024-03-24T10:46:18.742 INFO:tasks.workunit.client.1.smithi096.stdout:5/655: unlink d2/d29/d32/d3f/d84/fa5 0 2024-03-24T10:46:18.742 INFO:tasks.workunit.client.1.smithi096.stdout:0/748: dwrite d0/d11/d16/d21/d78/fa8 [0,4194304] 0 2024-03-24T10:46:18.742 INFO:tasks.workunit.client.1.smithi096.stdout:0/749: chown d0/d11/d16/d21/d81/d7f/fbc 115490797 1 2024-03-24T10:46:18.742 INFO:tasks.workunit.client.1.smithi096.stdout:0/750: chown d0/d11/d24/l86 2622098 1 2024-03-24T10:46:18.742 INFO:tasks.workunit.client.1.smithi096.stdout:1/583: dwrite d3/d9b/fa4 [0,4194304] 0 2024-03-24T10:46:18.742 INFO:tasks.workunit.client.1.smithi096.stdout:9/720: mkdir d9/d15/d26/d29/d73/d49/db7/dca/de3 0 2024-03-24T10:46:18.742 INFO:tasks.workunit.client.1.smithi096.stdout:9/721: write d9/d15/d26/d29/d73/f4b [170929,89908] 0 2024-03-24T10:46:18.744 INFO:tasks.workunit.client.1.smithi096.stdout:2/766: dread d5/d7c/f99 [0,4194304] 0 2024-03-24T10:46:18.744 INFO:tasks.workunit.client.1.smithi096.stdout:6/681: rename d8/d12/d32/d66 to d8/d12/d28/d36/d65/d71/dd0 0 2024-03-24T10:46:18.757 INFO:tasks.workunit.client.1.smithi096.stdout:8/734: rmdir d8/d11 39 2024-03-24T10:46:18.759 INFO:tasks.workunit.client.1.smithi096.stdout:5/656: unlink d2/d29/d32/d33/l48 0 2024-03-24T10:46:18.762 INFO:tasks.workunit.client.1.smithi096.stdout:0/751: mknod d0/d11/d16/d21/d78/deb/cfc 0 2024-03-24T10:46:18.762 INFO:tasks.workunit.client.1.smithi096.stdout:0/752: fdatasync d0/d11/d20/d65/f6e 0 2024-03-24T10:46:18.762 INFO:tasks.workunit.client.1.smithi096.stdout:0/753: readlink d0/d11/d24/d2c/d84/lc2 0 2024-03-24T10:46:18.762 INFO:tasks.workunit.client.1.smithi096.stdout:0/754: dread - d0/d11/d20/d65/fd6 zero size 2024-03-24T10:46:18.786 INFO:tasks.workunit.client.1.smithi096.stdout:4/595: sync 2024-03-24T10:46:18.786 INFO:tasks.workunit.client.1.smithi096.stdout:3/695: sync 2024-03-24T10:46:18.787 INFO:tasks.workunit.client.1.smithi096.stdout:7/825: sync 2024-03-24T10:46:18.787 INFO:tasks.workunit.client.1.smithi096.stdout:4/596: write d1/d2b/d67/f8a [1770533,124554] 0 2024-03-24T10:46:18.788 INFO:tasks.workunit.client.1.smithi096.stdout:9/722: mkdir d9/d15/d26/d29/d73/d49/d86/dc8/de4 0 2024-03-24T10:46:18.788 INFO:tasks.workunit.client.1.smithi096.stdout:9/723: chown d9/db/d6d/fc7 1225533354 1 2024-03-24T10:46:18.789 INFO:tasks.workunit.client.1.smithi096.stdout:6/682: creat d8/d12/d32/d38/fd1 x:0 0 0 2024-03-24T10:46:18.790 INFO:tasks.workunit.client.1.smithi096.stdout:4/597: dread d1/fe [0,4194304] 0 2024-03-24T10:46:18.791 INFO:tasks.workunit.client.1.smithi096.stdout:9/724: dread d9/db/d6d/d4f/f63 [0,4194304] 0 2024-03-24T10:46:18.792 INFO:tasks.workunit.client.1.smithi096.stdout:2/767: creat d5/dc/d6b/dcf/f11d x:0 0 0 2024-03-24T10:46:18.792 INFO:tasks.workunit.client.1.smithi096.stdout:2/768: chown d5/d7c/d88/f61 42 1 2024-03-24T10:46:18.793 INFO:tasks.workunit.client.1.smithi096.stdout:8/735: rename d8/d11/d22/d4c/d65/cbb to d8/d11/d22/d4c/d65/d1d/d3d/cea 0 2024-03-24T10:46:18.799 INFO:tasks.workunit.client.1.smithi096.stdout:1/584: symlink d3/d23/d8b/d5c/lca 0 2024-03-24T10:46:18.801 INFO:tasks.workunit.client.1.smithi096.stdout:6/683: symlink d8/d12/d28/d36/d65/d71/ld2 0 2024-03-24T10:46:18.803 INFO:tasks.workunit.client.1.smithi096.stdout:1/585: write d3/d73/f7a [1119378,109605] 0 2024-03-24T10:46:18.803 INFO:tasks.workunit.client.1.smithi096.stdout:1/586: chown d3/d23/d8b/fbd 9397014 1 2024-03-24T10:46:18.803 INFO:tasks.workunit.client.1.smithi096.stdout:4/598: mkdir d1/d32/d4c/d7d/dc2 0 2024-03-24T10:46:18.804 INFO:tasks.workunit.client.1.smithi096.stdout:8/736: truncate d8/f14 935956 0 2024-03-24T10:46:18.804 INFO:tasks.workunit.client.1.smithi096.stdout:8/737: chown d8/d11/d22/d4c/d65/d1a/da0/fd3 536307 1 2024-03-24T10:46:18.807 INFO:tasks.workunit.client.1.smithi096.stdout:0/755: link d0/cd d0/d11/d16/d21/d81/d9c/dcd/cfd 0 2024-03-24T10:46:18.810 INFO:tasks.workunit.client.1.smithi096.stdout:6/684: rename d8/d10/d13/d63/d51/laf to d8/d12/d32/d68/ld3 0 2024-03-24T10:46:18.813 INFO:tasks.workunit.client.1.smithi096.stdout:9/725: symlink d9/db/le5 0 2024-03-24T10:46:18.814 INFO:tasks.workunit.client.1.smithi096.stdout:8/738: creat d8/d11/d22/d4c/d65/d1a/da0/feb x:0 0 0 2024-03-24T10:46:18.816 INFO:tasks.workunit.client.1.smithi096.stdout:0/756: mknod d0/d11/d24/dcc/cfe 0 2024-03-24T10:46:18.816 INFO:tasks.workunit.client.1.smithi096.stdout:0/757: dread - d0/d11/d16/d21/d81/d9c/fa2 zero size 2024-03-24T10:46:18.819 INFO:tasks.workunit.client.1.smithi096.stdout:1/587: rename d3/d23/d2f/f4a to d3/d23/d8b/d5c/d9c/fcb 0 2024-03-24T10:46:18.821 INFO:tasks.workunit.client.1.smithi096.stdout:6/685: mknod d8/d12/d28/d36/cd4 0 2024-03-24T10:46:18.821 INFO:tasks.workunit.client.1.smithi096.stdout:9/726: write d9/d15/d26/d29/d73/d43/d48/fae [1035038,118413] 0 2024-03-24T10:46:18.829 INFO:tasks.workunit.client.1.smithi096.stdout:8/739: rename d8/d72/d96/d91/cc9 to d8/d72/d98/cec 0 2024-03-24T10:46:18.832 INFO:tasks.workunit.client.1.smithi096.stdout:7/826: dwrite d0/f14 [0,4194304] 0 2024-03-24T10:46:18.833 INFO:tasks.workunit.client.1.smithi096.stdout:1/588: rmdir d3/d23/d71/d7b/d93 39 2024-03-24T10:46:18.836 INFO:tasks.workunit.client.1.smithi096.stdout:3/696: dwrite d0/d13/f46 [0,4194304] 0 2024-03-24T10:46:18.837 INFO:tasks.workunit.client.1.smithi096.stdout:6/686: mkdir d8/d10/d76/d91/dd5 0 2024-03-24T10:46:18.838 INFO:tasks.workunit.client.1.smithi096.stdout:9/727: getdents d9/db/d6d/d4f 0 2024-03-24T10:46:18.840 INFO:tasks.workunit.client.1.smithi096.stdout:9/728: dread d9/fb3 [0,4194304] 0 2024-03-24T10:46:18.840 INFO:tasks.workunit.client.1.smithi096.stdout:8/740: mkdir d8/d11/d22/ded 0 2024-03-24T10:46:18.842 INFO:tasks.workunit.client.1.smithi096.stdout:2/769: dwrite d5/dc/d12/ff4 [0,4194304] 0 2024-03-24T10:46:18.842 INFO:tasks.workunit.client.1.smithi096.stdout:0/758: dwrite d0/fb0 [0,4194304] 0 2024-03-24T10:46:18.843 INFO:tasks.workunit.client.1.smithi096.stdout:4/599: dwrite d1/d2b/d6b/f96 [0,4194304] 0 2024-03-24T10:46:18.843 INFO:tasks.workunit.client.1.smithi096.stdout:2/770: write d5/d7c/d88/f69 [2592292,20395] 0 2024-03-24T10:46:18.843 INFO:tasks.workunit.client.1.smithi096.stdout:2/771: dread - d5/f84 zero size 2024-03-24T10:46:18.843 INFO:tasks.workunit.client.1.smithi096.stdout:2/772: chown d5/de/d2a/f62 90003842 1 2024-03-24T10:46:18.843 INFO:tasks.workunit.client.1.smithi096.stdout:2/773: readlink d5/dc/l30 0 2024-03-24T10:46:18.843 INFO:tasks.workunit.client.1.smithi096.stdout:7/827: creat d0/d13/df0/d3c/d68/d101/f10c x:0 0 0 2024-03-24T10:46:18.843 INFO:tasks.workunit.client.1.smithi096.stdout:7/828: truncate d0/d13/df0/d3c/d68/fa7 645336 0 2024-03-24T10:46:18.843 INFO:tasks.workunit.client.1.smithi096.stdout:7/829: readlink d0/db/d26/ldc 0 2024-03-24T10:46:18.844 INFO:tasks.workunit.client.1.smithi096.stdout:1/589: creat d3/d23/d8b/d8c/fcc x:0 0 0 2024-03-24T10:46:18.844 INFO:tasks.workunit.client.1.smithi096.stdout:1/590: chown d3/c10 25 1 2024-03-24T10:46:18.845 INFO:tasks.workunit.client.1.smithi096.stdout:1/591: fdatasync d3/d23/d71/dbe/fc9 0 2024-03-24T10:46:18.845 INFO:tasks.workunit.client.1.smithi096.stdout:1/592: creat d3/d23/d8b/fcd x:0 0 0 2024-03-24T10:46:18.845 INFO:tasks.workunit.client.1.smithi096.stdout:8/741: dread d8/d11/d22/d4c/d65/d1a/da0/da6/fbc [0,4194304] 0 2024-03-24T10:46:18.845 INFO:tasks.workunit.client.1.smithi096.stdout:8/742: dread - d8/d11/d22/d4c/d65/d1d/fb6 zero size 2024-03-24T10:46:18.845 INFO:tasks.workunit.client.1.smithi096.stdout:1/593: write d3/f34 [2047489,124618] 0 2024-03-24T10:46:18.847 INFO:tasks.workunit.client.1.smithi096.stdout:3/697: symlink d0/d13/d45/d7d/d39/d48/d8d/le3 0 2024-03-24T10:46:18.847 INFO:tasks.workunit.client.1.smithi096.stdout:3/698: readlink d0/d13/d1e/l58 0 2024-03-24T10:46:18.853 INFO:tasks.workunit.client.1.smithi096.stdout:0/759: symlink d0/d11/d24/d2c/d84/de8/lff 0 2024-03-24T10:46:18.854 INFO:tasks.workunit.client.1.smithi096.stdout:8/743: write d8/d11/d22/d4c/d65/d1d/f2b [1606553,113327] 0 2024-03-24T10:46:18.854 INFO:tasks.workunit.client.1.smithi096.stdout:8/744: write d8/d11/d22/d4c/d65/d1a/f26 [2337177,112305] 0 2024-03-24T10:46:18.854 INFO:tasks.workunit.client.1.smithi096.stdout:8/745: stat d8/d11/d22/d4c 0 2024-03-24T10:46:18.855 INFO:tasks.workunit.client.1.smithi096.stdout:4/600: mknod d1/d32/d42/dbe/cc3 0 2024-03-24T10:46:18.856 INFO:tasks.workunit.client.1.smithi096.stdout:2/774: symlink d5/d7c/d88/d90/l11e 0 2024-03-24T10:46:18.865 INFO:tasks.workunit.client.1.smithi096.stdout:4/601: dread d1/f1c [4194304,4194304] 0 2024-03-24T10:46:18.865 INFO:tasks.workunit.client.1.smithi096.stdout:4/602: chown d1/d2b/d59/d74/cbb 58 1 2024-03-24T10:46:18.866 INFO:tasks.workunit.client.1.smithi096.stdout:4/603: fdatasync d1/f35 0 2024-03-24T10:46:18.866 INFO:tasks.workunit.client.1.smithi096.stdout:4/604: fdatasync d1/d2b/d89/fa0 0 2024-03-24T10:46:18.867 INFO:tasks.workunit.client.1.smithi096.stdout:7/830: unlink d0/db/d26/f5b 0 2024-03-24T10:46:18.869 INFO:tasks.workunit.client.1.smithi096.stdout:4/605: dread d1/d2b/d40/fa7 [0,4194304] 0 2024-03-24T10:46:18.869 INFO:tasks.workunit.client.1.smithi096.stdout:4/606: chown d1/f8 278 1 2024-03-24T10:46:18.871 INFO:tasks.workunit.client.1.smithi096.stdout:3/699: mknod d0/ce4 0 2024-03-24T10:46:18.872 INFO:tasks.workunit.client.1.smithi096.stdout:0/760: mkdir d0/d11/d24/d9d/df5/d100 0 2024-03-24T10:46:18.878 INFO:tasks.workunit.client.1.smithi096.stdout:7/831: mkdir d0/d13/df0/d3c/d68/d85/d10d 0 2024-03-24T10:46:18.878 INFO:tasks.workunit.client.1.smithi096.stdout:7/832: write d0/fd2 [253487,33952] 0 2024-03-24T10:46:18.878 INFO:tasks.workunit.client.1.smithi096.stdout:7/833: dread - d0/d13/df0/d3c/d68/d4b/d62/ffd zero size 2024-03-24T10:46:18.878 INFO:tasks.workunit.client.1.smithi096.stdout:7/834: fsync d0/db/ff 0 2024-03-24T10:46:18.879 INFO:tasks.workunit.client.1.smithi096.stdout:7/835: readlink d0/d13/df0/d3c/d68/d85/l98 0 2024-03-24T10:46:18.880 INFO:tasks.workunit.client.1.smithi096.stdout:3/700: mkdir d0/d13/d45/d7d/d39/d48/dbd/d9c/da9/de5 0 2024-03-24T10:46:18.882 INFO:tasks.workunit.client.1.smithi096.stdout:4/607: dread d1/fd [0,4194304] 0 2024-03-24T10:46:18.885 INFO:tasks.workunit.client.1.smithi096.stdout:0/761: dread d0/d11/d24/f68 [0,4194304] 0 2024-03-24T10:46:18.886 INFO:tasks.workunit.client.1.smithi096.stdout:9/729: dwrite d9/d15/d26/d29/d73/d49/db7/dc1/fd5 [0,4194304] 0 2024-03-24T10:46:18.887 INFO:tasks.workunit.client.1.smithi096.stdout:2/775: dwrite d5/dc/f58 [0,4194304] 0 2024-03-24T10:46:18.888 INFO:tasks.workunit.client.1.smithi096.stdout:0/762: creat d0/d11/d16/d21/d25/f101 x:0 0 0 2024-03-24T10:46:18.892 INFO:tasks.workunit.client.1.smithi096.stdout:3/701: dread d0/d13/d1e/f2a [0,4194304] 0 2024-03-24T10:46:18.895 INFO:tasks.workunit.client.1.smithi096.stdout:3/702: dread d0/d13/d45/d52/f66 [0,4194304] 0 2024-03-24T10:46:18.900 INFO:tasks.workunit.client.1.smithi096.stdout:2/776: mkdir d5/dc/d11f 0 2024-03-24T10:46:18.903 INFO:tasks.workunit.client.1.smithi096.stdout:3/703: dread d0/d13/f21 [0,4194304] 0 2024-03-24T10:46:18.907 INFO:tasks.workunit.client.1.smithi096.stdout:0/763: rename d0/d11/d20/d65/d8a to d0/d11/d24/d102 0 2024-03-24T10:46:18.908 INFO:tasks.workunit.client.1.smithi096.stdout:4/608: dwrite d1/d32/f75 [0,4194304] 0 2024-03-24T10:46:18.908 INFO:tasks.workunit.client.1.smithi096.stdout:4/609: read - d1/d11/fae zero size 2024-03-24T10:46:18.908 INFO:tasks.workunit.client.1.smithi096.stdout:4/610: chown d1/d2b/d59/d74/fb4 8256 1 2024-03-24T10:46:18.908 INFO:tasks.workunit.client.1.smithi096.stdout:4/611: truncate d1/d2b/d89/fa0 361212 0 2024-03-24T10:46:18.909 INFO:tasks.workunit.client.1.smithi096.stdout:4/612: fsync d1/d32/d4c/d7e/f99 0 2024-03-24T10:46:18.909 INFO:tasks.workunit.client.1.smithi096.stdout:4/613: read - d1/d2b/d67/f82 zero size 2024-03-24T10:46:18.909 INFO:tasks.workunit.client.1.smithi096.stdout:4/614: stat d1/d2b/d40 0 2024-03-24T10:46:18.909 INFO:tasks.workunit.client.1.smithi096.stdout:9/730: dwrite d9/d15/fa6 [0,4194304] 0 2024-03-24T10:46:18.912 INFO:tasks.workunit.client.1.smithi096.stdout:3/704: rmdir d0/d13/d45/d7d/d71/db6 39 2024-03-24T10:46:18.916 INFO:tasks.workunit.client.1.smithi096.stdout:2/777: mkdir d5/de/d120 0 2024-03-24T10:46:18.916 INFO:tasks.workunit.client.1.smithi096.stdout:0/764: dread d0/d11/d20/d56/fe7 [0,4194304] 0 2024-03-24T10:46:18.917 INFO:tasks.workunit.client.1.smithi096.stdout:0/765: write d0/d11/d20/d9e/d80/f44 [4151123,129831] 0 2024-03-24T10:46:18.920 INFO:tasks.workunit.client.1.smithi096.stdout:4/615: mknod d1/d32/d4c/d7e/cc4 0 2024-03-24T10:46:18.920 INFO:tasks.workunit.client.1.smithi096.stdout:4/616: creat d1/d2b/d40/fc5 x:0 0 0 2024-03-24T10:46:18.921 INFO:tasks.workunit.client.1.smithi096.stdout:9/731: truncate d9/db/f28 240658 0 2024-03-24T10:46:18.922 INFO:tasks.workunit.client.1.smithi096.stdout:4/617: dread d1/d2b/d6b/f76 [0,4194304] 0 2024-03-24T10:46:18.923 INFO:tasks.workunit.client.1.smithi096.stdout:4/618: chown d1/l15 54868534 1 2024-03-24T10:46:18.923 INFO:tasks.workunit.client.1.smithi096.stdout:3/705: creat d0/d13/d45/d7d/d71/db6/fe6 x:0 0 0 2024-03-24T10:46:18.924 INFO:tasks.workunit.client.1.smithi096.stdout:2/778: rename d5/d93/cbb to d5/d38/dbd/ddb/c121 0 2024-03-24T10:46:18.924 INFO:tasks.workunit.client.1.smithi096.stdout:0/766: getdents d0/d11/d16/d21/d81/d9c 0 2024-03-24T10:46:18.927 INFO:tasks.workunit.client.1.smithi096.stdout:8/746: dwrite d8/d11/d22/d4c/d82/f2c [0,4194304] 0 2024-03-24T10:46:18.927 INFO:tasks.workunit.client.1.smithi096.stdout:8/747: write d8/d11/d22/d4c/d65/d1d/fcf [174223,92817] 0 2024-03-24T10:46:18.927 INFO:tasks.workunit.client.1.smithi096.stdout:8/748: creat d8/d72/d96/fee x:0 0 0 2024-03-24T10:46:18.928 INFO:tasks.workunit.client.1.smithi096.stdout:9/732: creat d9/db/d6d/d4f/d75/dcc/fe6 x:0 0 0 2024-03-24T10:46:18.928 INFO:tasks.workunit.client.1.smithi096.stdout:9/733: readlink d9/db/d6d/d4f/d75/d8b/lc9 0 2024-03-24T10:46:18.928 INFO:tasks.workunit.client.1.smithi096.stdout:9/734: write d9/db/fab [4567227,22685] 0 2024-03-24T10:46:18.929 INFO:tasks.workunit.client.1.smithi096.stdout:4/619: creat d1/d32/d4c/d7e/fc6 x:0 0 0 2024-03-24T10:46:18.929 INFO:tasks.workunit.client.1.smithi096.stdout:4/620: chown d1/d32/c48 63426 1 2024-03-24T10:46:18.929 INFO:tasks.workunit.client.1.smithi096.stdout:3/706: truncate d0/d13/d1e/f2a 3993959 0 2024-03-24T10:46:18.930 INFO:tasks.workunit.client.1.smithi096.stdout:3/707: write d0/d13/d45/d7d/d39/d48/f6a [4103589,64633] 0 2024-03-24T10:46:18.935 INFO:tasks.workunit.client.1.smithi096.stdout:2/779: link d5/dc/d12/f48 d5/d38/dbd/f122 0 2024-03-24T10:46:18.936 INFO:tasks.workunit.client.1.smithi096.stdout:2/780: creat d5/d79/d9f/f123 x:0 0 0 2024-03-24T10:46:18.936 INFO:tasks.workunit.client.1.smithi096.stdout:2/781: truncate d5/de/d2a/f62 803212 0 2024-03-24T10:46:18.936 INFO:tasks.workunit.client.1.smithi096.stdout:2/782: fsync d5/dc/d12/f41 0 2024-03-24T10:46:18.936 INFO:tasks.workunit.client.1.smithi096.stdout:2/783: dread - d5/d79/db5/fee zero size 2024-03-24T10:46:18.936 INFO:tasks.workunit.client.1.smithi096.stdout:9/735: read d9/d15/f3c [3162457,13894] 0 2024-03-24T10:46:18.936 INFO:tasks.workunit.client.1.smithi096.stdout:9/736: read d9/d15/d26/f33 [2864290,44748] 0 2024-03-24T10:46:18.937 INFO:tasks.workunit.client.1.smithi096.stdout:0/767: rename d0/d11/f1c to d0/d11/dc3/f103 0 2024-03-24T10:46:18.946 INFO:tasks.workunit.client.1.smithi096.stdout:8/749: mknod d8/d11/d22/d4c/d65/d1d/d3d/cef 0 2024-03-24T10:46:18.947 INFO:tasks.workunit.client.1.smithi096.stdout:4/621: mknod d1/d32/d42/cc7 0 2024-03-24T10:46:18.948 INFO:tasks.workunit.client.1.smithi096.stdout:3/708: dwrite d0/f1 [0,4194304] 0 2024-03-24T10:46:18.948 INFO:tasks.workunit.client.1.smithi096.stdout:3/709: stat d0/d13/d45/d7d/d39/d48/dbd/d9c 0 2024-03-24T10:46:18.950 INFO:tasks.workunit.client.1.smithi096.stdout:2/784: mkdir d5/d7c/d124 0 2024-03-24T10:46:18.950 INFO:tasks.workunit.client.1.smithi096.stdout:4/622: dread d1/fd [0,4194304] 0 2024-03-24T10:46:18.953 INFO:tasks.workunit.client.1.smithi096.stdout:0/768: rename d0/d11/d24/d9d/df5/d100 to d0/d11/d43/d75/d104 0 2024-03-24T10:46:18.953 INFO:tasks.workunit.client.1.smithi096.stdout:0/769: creat d0/d11/d20/d9e/d80/f105 x:0 0 0 2024-03-24T10:46:18.953 INFO:tasks.workunit.client.1.smithi096.stdout:0/770: creat d0/d11/d16/d21/d25/f106 x:0 0 0 2024-03-24T10:46:18.953 INFO:tasks.workunit.client.1.smithi096.stdout:0/771: write d0/d11/d16/d21/d81/faa [4411166,82438] 0 2024-03-24T10:46:18.953 INFO:tasks.workunit.client.1.smithi096.stdout:0/772: read - d0/d11/d24/d2c/d84/fb8 zero size 2024-03-24T10:46:18.953 INFO:tasks.workunit.client.1.smithi096.stdout:0/773: write d0/d11/d20/d9e/d80/fed [560147,105544] 0 2024-03-24T10:46:18.953 INFO:tasks.workunit.client.1.smithi096.stdout:0/774: truncate d0/d11/d24/d2c/fb3 163546 0 2024-03-24T10:46:18.954 INFO:tasks.workunit.client.1.smithi096.stdout:9/737: dwrite d9/d15/d26/d29/d73/f91 [0,4194304] 0 2024-03-24T10:46:18.954 INFO:tasks.workunit.client.1.smithi096.stdout:9/738: creat d9/db/d6d/d4f/d75/dcc/fe7 x:0 0 0 2024-03-24T10:46:18.956 INFO:tasks.workunit.client.1.smithi096.stdout:3/710: mkdir d0/d13/d45/d7d/de7 0 2024-03-24T10:46:18.956 INFO:tasks.workunit.client.1.smithi096.stdout:3/711: creat d0/d13/d1e/fe8 x:0 0 0 2024-03-24T10:46:18.956 INFO:tasks.workunit.client.1.smithi096.stdout:3/712: stat d0/d13/d41/c6c 0 2024-03-24T10:46:18.957 INFO:tasks.workunit.client.1.smithi096.stdout:2/785: creat d5/dc/d6b/d72/d11a/f125 x:0 0 0 2024-03-24T10:46:18.957 INFO:tasks.workunit.client.1.smithi096.stdout:2/786: readlink d5/dc/d12/l4d 0 2024-03-24T10:46:18.957 INFO:tasks.workunit.client.1.smithi096.stdout:2/787: chown d5/dc/d6b/d72 241022074 1 2024-03-24T10:46:18.957 INFO:tasks.workunit.client.1.smithi096.stdout:2/788: write d5/de/f103 [24807,59477] 0 2024-03-24T10:46:18.958 INFO:tasks.workunit.client.1.smithi096.stdout:4/623: rmdir d1/d2b/d6b/d80/d9c 39 2024-03-24T10:46:18.965 INFO:tasks.workunit.client.1.smithi096.stdout:0/775: creat d0/d11/d20/d2b/ddf/f107 x:0 0 0 2024-03-24T10:46:18.968 INFO:tasks.workunit.client.1.smithi096.stdout:9/739: mkdir d9/db/d6d/d4f/de8 0 2024-03-24T10:46:18.971 INFO:tasks.workunit.client.1.smithi096.stdout:3/713: rmdir d0/d92 39 2024-03-24T10:46:18.971 INFO:tasks.workunit.client.1.smithi096.stdout:3/714: chown d0/d13/d41/d7f/fb7 5197024 1 2024-03-24T10:46:18.971 INFO:tasks.workunit.client.1.smithi096.stdout:3/715: readlink d0/d13/d45/d7d/d39/d48/d85/lc4 0 2024-03-24T10:46:18.973 INFO:tasks.workunit.client.1.smithi096.stdout:2/789: symlink d5/d7c/d8f/l126 0 2024-03-24T10:46:18.974 INFO:tasks.workunit.client.1.smithi096.stdout:2/790: write d5/d38/fa8 [3312227,78332] 0 2024-03-24T10:46:18.974 INFO:tasks.workunit.client.1.smithi096.stdout:2/791: dread d5/d7c/ffa [0,4194304] 0 2024-03-24T10:46:18.977 INFO:tasks.workunit.client.1.smithi096.stdout:4/624: creat d1/d2b/d6b/d80/d9c/fc8 x:0 0 0 2024-03-24T10:46:18.979 INFO:tasks.workunit.client.1.smithi096.stdout:0/776: mkdir d0/d11/d16/d21/d81/d8f/d9a/d108 0 2024-03-24T10:46:18.979 INFO:tasks.workunit.client.1.smithi096.stdout:0/777: truncate d0/d11/d16/d21/f33 4654888 0 2024-03-24T10:46:18.979 INFO:tasks.workunit.client.1.smithi096.stdout:0/778: dread - d0/d11/d20/d2b/ddf/f107 zero size 2024-03-24T10:46:18.981 INFO:tasks.workunit.client.1.smithi096.stdout:9/740: creat d9/d15/d26/d29/d73/d43/d48/dc0/dd8/fe9 x:0 0 0 2024-03-24T10:46:18.981 INFO:tasks.workunit.client.1.smithi096.stdout:9/741: truncate d9/d15/d26/d29/d73/fdc 507935 0 2024-03-24T10:46:18.981 INFO:tasks.workunit.client.1.smithi096.stdout:9/742: fsync d9/db/f53 0 2024-03-24T10:46:18.981 INFO:tasks.workunit.client.1.smithi096.stdout:3/716: rename d0/fad to d0/fe9 0 2024-03-24T10:46:18.982 INFO:tasks.workunit.client.1.smithi096.stdout:0/779: mkdir d0/d11/d20/d56/d109 0 2024-03-24T10:46:18.984 INFO:tasks.workunit.client.1.smithi096.stdout:9/743: rename d9/db/fab to d9/d15/d26/d29/d73/d49/db7/fea 0 2024-03-24T10:46:18.986 INFO:tasks.workunit.client.1.smithi096.stdout:0/780: rename d0/d11/d16/d21/d78/db6 to d0/d11/d16/d21/d81/d9c/dcd/d10a 0 2024-03-24T10:46:18.986 INFO:tasks.workunit.client.1.smithi096.stdout:0/781: readlink d0/d11/l35 0 2024-03-24T10:46:18.986 INFO:tasks.workunit.client.1.smithi096.stdout:0/782: write d0/fc5 [706721,43606] 0 2024-03-24T10:46:18.986 INFO:tasks.workunit.client.1.smithi096.stdout:0/783: chown d0/d11/d16/d21/d81/d8f/d9a/ca9 42972 1 2024-03-24T10:46:18.986 INFO:tasks.workunit.client.1.smithi096.stdout:0/784: write d0/d11/d20/d9e/d80/f105 [1017115,33336] 0 2024-03-24T10:46:18.987 INFO:tasks.workunit.client.1.smithi096.stdout:0/785: truncate d0/d11/d16/fa5 50127 0 2024-03-24T10:46:18.987 INFO:tasks.workunit.client.1.smithi096.stdout:0/786: symlink d0/d11/d20/l10b 0 2024-03-24T10:46:18.987 INFO:tasks.workunit.client.1.smithi096.stdout:0/787: truncate d0/d11/d16/d21/d78/d92/f99 678663 0 2024-03-24T10:46:18.997 INFO:tasks.workunit.client.1.smithi096.stdout:2/792: dwrite d5/d7c/f7e [0,4194304] 0 2024-03-24T10:46:19.003 INFO:tasks.workunit.client.1.smithi096.stdout:9/744: write d9/d15/d26/d29/d73/f5a [1320900,13919] 0 2024-03-24T10:46:19.005 INFO:tasks.workunit.client.1.smithi096.stdout:3/717: dread d0/d9/d22/f4f [0,4194304] 0 2024-03-24T10:46:19.007 INFO:tasks.workunit.client.1.smithi096.stdout:3/718: creat d0/d13/d45/d7d/fea x:0 0 0 2024-03-24T10:46:19.007 INFO:tasks.workunit.client.1.smithi096.stdout:3/719: chown d0/d13/d45/d52/fc1 51852 1 2024-03-24T10:46:19.007 INFO:tasks.workunit.client.1.smithi096.stdout:3/720: stat d0/c12 0 2024-03-24T10:46:19.009 INFO:tasks.workunit.client.1.smithi096.stdout:2/793: read d5/d7c/d88/f70 [3620809,52893] 0 2024-03-24T10:46:19.012 INFO:tasks.workunit.client.1.smithi096.stdout:0/788: dwrite d0/d11/d24/dcc/fd3 [0,4194304] 0 2024-03-24T10:46:19.012 INFO:tasks.workunit.client.1.smithi096.stdout:0/789: chown d0/d11/d24/d2c/c61 3162 1 2024-03-24T10:46:19.012 INFO:tasks.workunit.client.1.smithi096.stdout:8/750: dwrite d8/d72/d96/f87 [0,4194304] 0 2024-03-24T10:46:19.012 INFO:tasks.workunit.client.1.smithi096.stdout:8/751: dread - d8/d11/d22/d4c/d65/d1d/fb6 zero size 2024-03-24T10:46:19.012 INFO:tasks.workunit.client.1.smithi096.stdout:4/625: dwrite d1/f51 [4194304,4194304] 0 2024-03-24T10:46:19.014 INFO:tasks.workunit.client.1.smithi096.stdout:2/794: rename d5/dc/f47 to d5/d7c/d88/d114/f127 0 2024-03-24T10:46:19.022 INFO:tasks.workunit.client.1.smithi096.stdout:8/752: mkdir d8/d11/d22/d4c/d65/df0 0 2024-03-24T10:46:19.022 INFO:tasks.workunit.client.1.smithi096.stdout:2/795: dread f2 [0,4194304] 0 2024-03-24T10:46:19.026 INFO:tasks.workunit.client.1.smithi096.stdout:4/626: mknod d1/d32/cc9 0 2024-03-24T10:46:19.027 INFO:tasks.workunit.client.1.smithi096.stdout:9/745: dwrite d9/d15/d26/d29/f6a [0,4194304] 0 2024-03-24T10:46:19.041 INFO:tasks.workunit.client.1.smithi096.stdout:8/753: rmdir d8/d11/d22/d4c/d65/d1d/d3d/d5f 39 2024-03-24T10:46:19.043 INFO:tasks.workunit.client.1.smithi096.stdout:2/796: mkdir d5/d7c/d8f/d128 0 2024-03-24T10:46:19.044 INFO:tasks.workunit.client.1.smithi096.stdout:2/797: chown d5/d79/de8/f100 64951 1 2024-03-24T10:46:19.045 INFO:tasks.workunit.client.1.smithi096.stdout:4/627: creat d1/d32/d42/dbe/fca x:0 0 0 2024-03-24T10:46:19.047 INFO:tasks.workunit.client.1.smithi096.stdout:9/746: dwrite d9/db/d6d/f56 [0,4194304] 0 2024-03-24T10:46:19.052 INFO:tasks.workunit.client.1.smithi096.stdout:2/798: dread d5/d38/f66 [0,4194304] 0 2024-03-24T10:46:19.053 INFO:tasks.workunit.client.1.smithi096.stdout:8/754: rename d8/d11/d22/d4c/d65/d37/l47 to d8/d11/d22/lf1 0 2024-03-24T10:46:19.055 INFO:tasks.workunit.client.1.smithi096.stdout:2/799: symlink d5/d38/dea/l129 0 2024-03-24T10:46:19.058 INFO:tasks.workunit.client.1.smithi096.stdout:2/800: symlink d5/dc/d6b/def/l12a 0 2024-03-24T10:46:19.060 INFO:tasks.workunit.client.1.smithi096.stdout:8/755: mkdir d8/d11/d22/d4c/d65/d37/df2 0 2024-03-24T10:46:19.060 INFO:tasks.workunit.client.1.smithi096.stdout:2/801: mknod d5/dc/d6b/def/c12b 0 2024-03-24T10:46:19.065 INFO:tasks.workunit.client.1.smithi096.stdout:8/756: rename d8/d11/d22/d4c/d65/d1a/d46/lc1 to d8/d11/d22/d4c/d65/d52/lf3 0 2024-03-24T10:46:19.067 INFO:tasks.workunit.client.1.smithi096.stdout:8/757: link d8/d11/f43 d8/d11/d44/ff4 0 2024-03-24T10:46:19.067 INFO:tasks.workunit.client.1.smithi096.stdout:8/758: creat d8/d72/d96/ff5 x:0 0 0 2024-03-24T10:46:19.086 INFO:tasks.workunit.client.1.smithi096.stdout:9/747: dwrite d9/f13 [0,4194304] 0 2024-03-24T10:46:19.087 INFO:tasks.workunit.client.1.smithi096.stdout:9/748: write d9/d7c/f9a [68940,72611] 0 2024-03-24T10:46:19.087 INFO:tasks.workunit.client.1.smithi096.stdout:2/802: dwrite d5/d38/f6f [4194304,4194304] 0 2024-03-24T10:46:19.087 INFO:tasks.workunit.client.1.smithi096.stdout:8/759: dwrite d8/d11/d44/fa7 [0,4194304] 0 2024-03-24T10:46:19.087 INFO:tasks.workunit.client.1.smithi096.stdout:8/760: fsync d8/fb3 0 2024-03-24T10:46:19.088 INFO:tasks.workunit.client.1.smithi096.stdout:8/761: write d8/d11/d22/d4c/d65/d1a/f80 [1296399,58636] 0 2024-03-24T10:46:19.088 INFO:tasks.workunit.client.1.smithi096.stdout:2/803: write d5/de/f53 [65319,80069] 0 2024-03-24T10:46:19.093 INFO:tasks.workunit.client.1.smithi096.stdout:9/749: symlink d9/d7c/leb 0 2024-03-24T10:46:19.097 INFO:tasks.workunit.client.1.smithi096.stdout:8/762: mkdir d8/d11/d22/d4c/d65/d1a/da0/df6 0 2024-03-24T10:46:19.100 INFO:tasks.workunit.client.1.smithi096.stdout:2/804: symlink d5/dc/d12/d23/l12c 0 2024-03-24T10:46:19.100 INFO:tasks.workunit.client.1.smithi096.stdout:2/805: chown d5/dc/d12/l4d 12 1 2024-03-24T10:46:19.102 INFO:tasks.workunit.client.1.smithi096.stdout:9/750: creat d9/d15/fec x:0 0 0 2024-03-24T10:46:19.102 INFO:tasks.workunit.client.1.smithi096.stdout:9/751: truncate d9/d15/d26/d29/d73/d43/f85 4653302 0 2024-03-24T10:46:19.105 INFO:tasks.workunit.client.1.smithi096.stdout:8/763: rename d8/d11/d22/ld2 to d8/d11/d85/lf7 0 2024-03-24T10:46:19.108 INFO:tasks.workunit.client.1.smithi096.stdout:2/806: symlink d5/dc/d6b/d72/l12d 0 2024-03-24T10:46:19.109 INFO:tasks.workunit.client.1.smithi096.stdout:9/752: creat d9/d15/d26/d29/d73/d43/d48/fed x:0 0 0 2024-03-24T10:46:19.114 INFO:tasks.workunit.client.1.smithi096.stdout:2/807: creat d5/d93/f12e x:0 0 0 2024-03-24T10:46:19.115 INFO:tasks.workunit.client.1.smithi096.stdout:9/753: symlink d9/db/d6d/d4f/d75/d8b/d8e/lee 0 2024-03-24T10:46:19.117 INFO:tasks.workunit.client.1.smithi096.stdout:2/808: unlink d5/dc/d6b/d72/f10d 0 2024-03-24T10:46:19.118 INFO:tasks.workunit.client.1.smithi096.stdout:9/754: rmdir d9/d65 39 2024-03-24T10:46:19.122 INFO:tasks.workunit.client.1.smithi096.stdout:9/755: creat d9/d15/d26/d29/d73/d49/d86/dc8/fef x:0 0 0 2024-03-24T10:46:19.126 INFO:tasks.workunit.client.1.smithi096.stdout:2/809: rmdir d5/de/d2a/d5e 39 2024-03-24T10:46:19.126 INFO:tasks.workunit.client.1.smithi096.stdout:2/810: fdatasync d5/d7c/f8c 0 2024-03-24T10:46:19.126 INFO:tasks.workunit.client.1.smithi096.stdout:2/811: fsync d5/de/d2a/f62 0 2024-03-24T10:46:19.126 INFO:tasks.workunit.client.1.smithi096.stdout:2/812: chown d5/dc/d12/d23/d105/c111 153084683 1 2024-03-24T10:46:19.131 INFO:tasks.workunit.client.1.smithi096.stdout:2/813: getdents d5/d7c/d88/d90 0 2024-03-24T10:46:19.139 INFO:tasks.workunit.client.1.smithi096.stdout:9/756: rename d9/d15/d26/d29/d73/d49/d86/dc8 to d9/db/d6d/dc6/df0 0 2024-03-24T10:46:19.139 INFO:tasks.workunit.client.1.smithi096.stdout:9/757: write d9/db/d6d/d4f/f54 [3639086,121002] 0 2024-03-24T10:46:19.139 INFO:tasks.workunit.client.1.smithi096.stdout:9/758: write d9/db/d6d/f35 [4624863,105734] 0 2024-03-24T10:46:19.140 INFO:tasks.workunit.client.1.smithi096.stdout:9/759: dread - d9/db/d6d/d4f/fb2 zero size 2024-03-24T10:46:19.140 INFO:tasks.workunit.client.1.smithi096.stdout:2/814: rename d5/cd to d5/dc/d12/d23/c12f 0 2024-03-24T10:46:19.156 INFO:tasks.workunit.client.1.smithi096.stdout:2/815: dwrite d5/d79/d9f/dad/fdf [0,4194304] 0 2024-03-24T10:46:19.186 INFO:tasks.workunit.client.1.smithi096.stdout:5/657: sync 2024-03-24T10:46:19.186 INFO:tasks.workunit.client.1.smithi096.stdout:5/658: truncate d2/d29/d32/d3f/f55 4525135 0 2024-03-24T10:46:19.210 INFO:tasks.workunit.client.1.smithi096.stdout:2/816: symlink d5/d38/dea/l130 0 2024-03-24T10:46:19.210 INFO:tasks.workunit.client.1.smithi096.stdout:2/817: write d5/d7c/d88/f69 [913122,18157] 0 2024-03-24T10:46:19.211 INFO:tasks.workunit.client.1.smithi096.stdout:5/659: rmdir d2/d29/d32/d33/d3c 39 2024-03-24T10:46:19.220 INFO:tasks.workunit.client.1.smithi096.stdout:2/818: mkdir d5/de/d120/d131 0 2024-03-24T10:46:19.224 INFO:tasks.workunit.client.1.smithi096.stdout:2/819: mknod d5/d38/dbd/ddb/d102/c132 0 2024-03-24T10:46:19.225 INFO:tasks.workunit.client.1.smithi096.stdout:5/660: truncate d2/d29/d32/d3f/f55 1266878 0 2024-03-24T10:46:19.226 INFO:tasks.workunit.client.1.smithi096.stdout:5/661: creat d2/d29/d32/d33/d8d/d94/fd8 x:0 0 0 2024-03-24T10:46:19.226 INFO:tasks.workunit.client.1.smithi096.stdout:5/662: write d2/d7/f4a [1929299,1684] 0 2024-03-24T10:46:19.226 INFO:tasks.workunit.client.1.smithi096.stdout:5/663: write d2/d29/d32/f40 [4737702,49920] 0 2024-03-24T10:46:19.226 INFO:tasks.workunit.client.1.smithi096.stdout:5/664: chown d2/d29/d32/d89/f9c 277559 1 2024-03-24T10:46:19.228 INFO:tasks.workunit.client.1.smithi096.stdout:9/760: rename d9/d15/d26/d29/d73/d49/d8a to d9/df1 0 2024-03-24T10:46:19.228 INFO:tasks.workunit.client.1.smithi096.stdout:9/761: dread - d9/d15/d26/d29/d73/d49/db7/fdf zero size 2024-03-24T10:46:19.229 INFO:tasks.workunit.client.1.smithi096.stdout:2/820: creat d5/d7c/dfd/f133 x:0 0 0 2024-03-24T10:46:19.230 INFO:tasks.workunit.client.1.smithi096.stdout:2/821: dread d5/de/d2a/f10f [0,4194304] 0 2024-03-24T10:46:19.232 INFO:tasks.workunit.client.1.smithi096.stdout:2/822: creat d5/de/d2a/d5e/dd6/f134 x:0 0 0 2024-03-24T10:46:19.234 INFO:tasks.workunit.client.1.smithi096.stdout:5/665: rename d2/d29/d32/d33/d3c/d3d/f75 to d2/d29/d32/d33/d8d/d94/fd9 0 2024-03-24T10:46:19.234 INFO:tasks.workunit.client.1.smithi096.stdout:5/666: creat d2/d29/d32/d33/d3c/dd2/fda x:0 0 0 2024-03-24T10:46:19.236 INFO:tasks.workunit.client.1.smithi096.stdout:5/667: dread d2/d7/d8/df/f25 [0,4194304] 0 2024-03-24T10:46:19.236 INFO:tasks.workunit.client.1.smithi096.stdout:5/668: write d2/d7/f4a [266118,85715] 0 2024-03-24T10:46:19.241 INFO:tasks.workunit.client.1.smithi096.stdout:2/823: dread d5/fb0 [0,4194304] 0 2024-03-24T10:46:19.245 INFO:tasks.workunit.client.1.smithi096.stdout:9/762: rename d9/c37 to d9/db/d6d/d4f/d75/cf2 0 2024-03-24T10:46:19.249 INFO:tasks.workunit.client.1.smithi096.stdout:2/824: rmdir d5/d79/d9f/df6 0 2024-03-24T10:46:19.251 INFO:tasks.workunit.client.1.smithi096.stdout:2/825: dread d5/de/d2a/d5e/fb6 [0,4194304] 0 2024-03-24T10:46:19.251 INFO:tasks.workunit.client.1.smithi096.stdout:9/763: symlink d9/d79/lf3 0 2024-03-24T10:46:19.253 INFO:tasks.workunit.client.1.smithi096.stdout:5/669: dwrite d2/d7/d8/d18/db6/fd1 [0,4194304] 0 2024-03-24T10:46:19.254 INFO:tasks.workunit.client.1.smithi096.stdout:2/826: truncate d5/de/f51 1190686 0 2024-03-24T10:46:19.256 INFO:tasks.workunit.client.1.smithi096.stdout:9/764: mkdir d9/d15/d26/d29/d73/df4 0 2024-03-24T10:46:19.263 INFO:tasks.workunit.client.1.smithi096.stdout:9/765: mknod d9/db/d6d/d4f/de8/cf5 0 2024-03-24T10:46:19.268 INFO:tasks.workunit.client.1.smithi096.stdout:9/766: mkdir d9/df6 0 2024-03-24T10:46:19.268 INFO:tasks.workunit.client.1.smithi096.stdout:9/767: chown d9/d15/d26/f97 18 1 2024-03-24T10:46:19.268 INFO:tasks.workunit.client.1.smithi096.stdout:9/768: rmdir d9/d15/d26/d29/d73/d49/db7/dca 39 2024-03-24T10:46:19.269 INFO:tasks.workunit.client.1.smithi096.stdout:9/769: creat d9/df1/dd9/ff7 x:0 0 0 2024-03-24T10:46:19.270 INFO:tasks.workunit.client.1.smithi096.stdout:9/770: mkdir d9/df1/df8 0 2024-03-24T10:46:19.270 INFO:tasks.workunit.client.1.smithi096.stdout:9/771: write d9/d15/d26/d29/d73/d49/d86/fb9 [556731,47327] 0 2024-03-24T10:46:19.271 INFO:tasks.workunit.client.1.smithi096.stdout:9/772: mknod d9/db/d6d/cf9 0 2024-03-24T10:46:19.271 INFO:tasks.workunit.client.1.smithi096.stdout:9/773: creat d9/d15/d26/d29/d73/d49/d86/ffa x:0 0 0 2024-03-24T10:46:19.274 INFO:tasks.workunit.client.1.smithi096.stdout:5/670: dwrite d2/d7/d69/f91 [4194304,4194304] 0 2024-03-24T10:46:19.274 INFO:tasks.workunit.client.1.smithi096.stdout:2/827: dwrite d5/de/d2a/d5e/dca/f10a [0,4194304] 0 2024-03-24T10:46:19.276 INFO:tasks.workunit.client.1.smithi096.stdout:5/671: mknod d2/d29/d32/d33/d3c/dd2/cdb 0 2024-03-24T10:46:19.282 INFO:tasks.workunit.client.1.smithi096.stdout:2/828: mknod d5/dc/d6b/c135 0 2024-03-24T10:46:19.283 INFO:tasks.workunit.client.1.smithi096.stdout:5/672: creat d2/d7/fdc x:0 0 0 2024-03-24T10:46:19.284 INFO:tasks.workunit.client.1.smithi096.stdout:2/829: rmdir d5/de/d2a/d5e 39 2024-03-24T10:46:19.284 INFO:tasks.workunit.client.1.smithi096.stdout:2/830: creat d5/d7c/d8f/f136 x:0 0 0 2024-03-24T10:46:19.286 INFO:tasks.workunit.client.1.smithi096.stdout:5/673: rename d2/dbd to d2/d7/d8/d18/db6/ddd 0 2024-03-24T10:46:19.294 INFO:tasks.workunit.client.1.smithi096.stdout:5/674: write d2/d7/fa7 [696841,77466] 0 2024-03-24T10:46:19.308 INFO:tasks.workunit.client.1.smithi096.stdout:2/831: rmdir d5 39 2024-03-24T10:46:19.311 INFO:tasks.workunit.client.1.smithi096.stdout:2/832: rename d5/dc/d6b/def/c12b to d5/de/d2a/c137 0 2024-03-24T10:46:19.312 INFO:tasks.workunit.client.1.smithi096.stdout:2/833: symlink d5/d38/dbd/ddb/d102/l138 0 2024-03-24T10:46:19.312 INFO:tasks.workunit.client.1.smithi096.stdout:5/675: dwrite d2/d7/f8c [0,4194304] 0 2024-03-24T10:46:19.313 INFO:tasks.workunit.client.1.smithi096.stdout:2/834: creat d5/d7c/d124/f139 x:0 0 0 2024-03-24T10:46:19.313 INFO:tasks.workunit.client.1.smithi096.stdout:2/835: dread - d5/de/d2a/d5e/fcd zero size 2024-03-24T10:46:19.314 INFO:tasks.workunit.client.1.smithi096.stdout:2/836: mkdir d5/de/d2a/df3/d13a 0 2024-03-24T10:46:19.315 INFO:tasks.workunit.client.1.smithi096.stdout:2/837: stat d5/d79 0 2024-03-24T10:46:19.318 INFO:tasks.workunit.client.1.smithi096.stdout:2/838: creat d5/dc/d6b/def/f13b x:0 0 0 2024-03-24T10:46:19.318 INFO:tasks.workunit.client.1.smithi096.stdout:2/839: fsync d5/d79/d9f/dad/fdf 0 2024-03-24T10:46:19.319 INFO:tasks.workunit.client.1.smithi096.stdout:2/840: creat d5/d7c/d88/d114/f13c x:0 0 0 2024-03-24T10:46:19.321 INFO:tasks.workunit.client.1.smithi096.stdout:2/841: creat d5/dc/f13d x:0 0 0 2024-03-24T10:46:19.321 INFO:tasks.workunit.client.1.smithi096.stdout:2/842: dread - d5/dc/d12/d23/f77 zero size 2024-03-24T10:46:19.326 INFO:tasks.workunit.client.1.smithi096.stdout:5/676: dwrite d2/d29/d32/d33/d37/dab/fb1 [0,4194304] 0 2024-03-24T10:46:19.326 INFO:tasks.workunit.client.1.smithi096.stdout:5/677: fsync d2/d7/d8/d18/f72 0 2024-03-24T10:46:19.327 INFO:tasks.workunit.client.1.smithi096.stdout:5/678: chown d2/d7/d8/df/f6a 3097558 1 2024-03-24T10:46:19.327 INFO:tasks.workunit.client.1.smithi096.stdout:5/679: chown d2/d29/d32/d33/d37/dab/lbf 372576 1 2024-03-24T10:46:19.329 INFO:tasks.workunit.client.1.smithi096.stdout:5/680: mknod d2/d7/d8/df/cde 0 2024-03-24T10:46:19.330 INFO:tasks.workunit.client.1.smithi096.stdout:5/681: creat d2/d29/d32/d33/d3c/dc3/fdf x:0 0 0 2024-03-24T10:46:19.333 INFO:tasks.workunit.client.1.smithi096.stdout:5/682: getdents d2 0 2024-03-24T10:46:19.335 INFO:tasks.workunit.client.1.smithi096.stdout:5/683: mknod d2/d29/d32/d3f/d84/db8/ce0 0 2024-03-24T10:46:19.335 INFO:tasks.workunit.client.1.smithi096.stdout:5/684: dread - d2/d29/d32/d3f/d5f/dc0/fcc zero size 2024-03-24T10:46:19.337 INFO:tasks.workunit.client.1.smithi096.stdout:5/685: creat d2/d7/d69/fe1 x:0 0 0 2024-03-24T10:46:19.337 INFO:tasks.workunit.client.1.smithi096.stdout:5/686: creat d2/d29/d32/d3f/d5f/dc0/fe2 x:0 0 0 2024-03-24T10:46:19.337 INFO:tasks.workunit.client.1.smithi096.stdout:5/687: creat d2/d29/d32/d33/d8d/d94/fe3 x:0 0 0 2024-03-24T10:46:19.337 INFO:tasks.workunit.client.1.smithi096.stdout:6/687: sync 2024-03-24T10:46:19.341 INFO:tasks.workunit.client.1.smithi096.stdout:5/688: unlink d2/d7/d8/df/f6a 0 2024-03-24T10:46:19.342 INFO:tasks.workunit.client.1.smithi096.stdout:5/689: rmdir d2/d29/d32/d3f/d84/db8 39 2024-03-24T10:46:19.345 INFO:tasks.workunit.client.1.smithi096.stdout:2/843: dwrite d5/dc/d6b/dcf/f11d [0,4194304] 0 2024-03-24T10:46:19.350 INFO:tasks.workunit.client.1.smithi096.stdout:5/690: getdents d2/d29/d32/d3f/d84/db8 0 2024-03-24T10:46:19.351 INFO:tasks.workunit.client.1.smithi096.stdout:2/844: truncate d5/d38/f3a 364743 0 2024-03-24T10:46:19.351 INFO:tasks.workunit.client.1.smithi096.stdout:2/845: readlink d5/d7c/d88/d90/l11e 0 2024-03-24T10:46:19.353 INFO:tasks.workunit.client.1.smithi096.stdout:6/688: dwrite d8/d12/d32/d38/d59/f79 [0,4194304] 0 2024-03-24T10:46:19.358 INFO:tasks.workunit.client.1.smithi096.stdout:5/691: mkdir d2/d7/d69/de4 0 2024-03-24T10:46:19.364 INFO:tasks.workunit.client.1.smithi096.stdout:5/692: truncate d2/d29/d32/d33/d8d/fa2 1148602 0 2024-03-24T10:46:19.364 INFO:tasks.workunit.client.1.smithi096.stdout:5/693: chown d2/fa6 8892141 1 2024-03-24T10:46:19.365 INFO:tasks.workunit.client.1.smithi096.stdout:2/846: rmdir d5/d7c/d124 39 2024-03-24T10:46:19.370 INFO:tasks.workunit.client.1.smithi096.stdout:6/689: mkdir d8/d10/d76/dd6 0 2024-03-24T10:46:19.371 INFO:tasks.workunit.client.1.smithi096.stdout:6/690: dread d8/d10/d58/f85 [0,4194304] 0 2024-03-24T10:46:19.373 INFO:tasks.workunit.client.1.smithi096.stdout:5/694: unlink d2/l92 0 2024-03-24T10:46:19.373 INFO:tasks.workunit.client.1.smithi096.stdout:6/691: dread d8/d10/d19/f1b [4194304,4194304] 0 2024-03-24T10:46:19.373 INFO:tasks.workunit.client.1.smithi096.stdout:5/695: dread - d2/d29/d32/d3f/d5f/dc0/fcc zero size 2024-03-24T10:46:19.375 INFO:tasks.workunit.client.1.smithi096.stdout:6/692: mknod d8/d12/d32/d38/d59/cd7 0 2024-03-24T10:46:19.376 INFO:tasks.workunit.client.1.smithi096.stdout:5/696: rename d2/d29/d32/db9/fbc to d2/d29/d32/fe5 0 2024-03-24T10:46:19.376 INFO:tasks.workunit.client.1.smithi096.stdout:5/697: readlink d2/d29/d32/d3f/d5f/l67 0 2024-03-24T10:46:19.379 INFO:tasks.workunit.client.1.smithi096.stdout:6/693: unlink d8/d10/d13/d63/f21 0 2024-03-24T10:46:19.379 INFO:tasks.workunit.client.1.smithi096.stdout:2/847: dwrite d5/dc/d6b/d72/d91/fa0 [0,4194304] 0 2024-03-24T10:46:19.379 INFO:tasks.workunit.client.1.smithi096.stdout:2/848: readlink d5/l113 0 2024-03-24T10:46:19.380 INFO:tasks.workunit.client.1.smithi096.stdout:5/698: creat d2/d29/d32/d3f/d84/db8/fe6 x:0 0 0 2024-03-24T10:46:19.380 INFO:tasks.workunit.client.1.smithi096.stdout:5/699: write d2/d7/fd4 [491012,87368] 0 2024-03-24T10:46:19.380 INFO:tasks.workunit.client.1.smithi096.stdout:5/700: write d2/d29/d32/d33/d8d/d94/fd8 [241653,88807] 0 2024-03-24T10:46:19.381 INFO:tasks.workunit.client.1.smithi096.stdout:6/694: mkdir d8/d10/d13/dd8 0 2024-03-24T10:46:19.382 INFO:tasks.workunit.client.1.smithi096.stdout:6/695: getdents d8/d10/d76/dd6 0 2024-03-24T10:46:19.384 INFO:tasks.workunit.client.1.smithi096.stdout:2/849: rename d5/d79/d9f/df1 to d5/d93/d115/d13e 0 2024-03-24T10:46:19.385 INFO:tasks.workunit.client.1.smithi096.stdout:2/850: truncate d5/dc/d12/fc5 336966 0 2024-03-24T10:46:19.385 INFO:tasks.workunit.client.1.smithi096.stdout:2/851: chown d5/dc/d12/l4d 1471732 1 2024-03-24T10:46:19.386 INFO:tasks.workunit.client.1.smithi096.stdout:5/701: unlink d2/fa6 0 2024-03-24T10:46:19.386 INFO:tasks.workunit.client.1.smithi096.stdout:5/702: write d2/d7/fa7 [4140007,1873] 0 2024-03-24T10:46:19.387 INFO:tasks.workunit.client.1.smithi096.stdout:2/852: dread d5/de/d2a/d5e/fb6 [0,4194304] 0 2024-03-24T10:46:19.387 INFO:tasks.workunit.client.1.smithi096.stdout:2/853: dread - d5/d38/dea/f117 zero size 2024-03-24T10:46:19.390 INFO:tasks.workunit.client.1.smithi096.stdout:5/703: creat d2/d29/d32/dbb/fe7 x:0 0 0 2024-03-24T10:46:19.391 INFO:tasks.workunit.client.1.smithi096.stdout:2/854: getdents d5/d38/dea 0 2024-03-24T10:46:19.394 INFO:tasks.workunit.client.1.smithi096.stdout:2/855: mknod d5/d7c/c13f 0 2024-03-24T10:46:19.403 INFO:tasks.workunit.client.1.smithi096.stdout:5/704: rmdir d2/d29/d32/d33 39 2024-03-24T10:46:19.403 INFO:tasks.workunit.client.1.smithi096.stdout:2/856: truncate d5/d38/f7b 1026577 0 2024-03-24T10:46:19.412 INFO:tasks.workunit.client.1.smithi096.stdout:2/857: rmdir d5/d93 39 2024-03-24T10:46:19.414 INFO:tasks.workunit.client.1.smithi096.stdout:5/705: getdents d2/d29/d32/d3f/d84 0 2024-03-24T10:46:19.414 INFO:tasks.workunit.client.1.smithi096.stdout:5/706: chown d2/d29/d32/d33/d3c/c9f 1 1 2024-03-24T10:46:19.414 INFO:tasks.workunit.client.1.smithi096.stdout:5/707: fsync d2/d29/d32/d33/f63 0 2024-03-24T10:46:19.414 INFO:tasks.workunit.client.1.smithi096.stdout:5/708: read d2/d29/d32/d89/fa1 [611917,83685] 0 2024-03-24T10:46:19.415 INFO:tasks.workunit.client.1.smithi096.stdout:2/858: mkdir d5/d7c/dfd/d140 0 2024-03-24T10:46:19.415 INFO:tasks.workunit.client.1.smithi096.stdout:2/859: chown d5/fb0 52 1 2024-03-24T10:46:19.415 INFO:tasks.workunit.client.1.smithi096.stdout:2/860: stat d5/dc/d6b/d72/de0 0 2024-03-24T10:46:19.416 INFO:tasks.workunit.client.1.smithi096.stdout:5/709: link d2/d7/fa7 d2/d29/d32/d33/d37/fe8 0 2024-03-24T10:46:19.418 INFO:tasks.workunit.client.1.smithi096.stdout:5/710: symlink d2/d29/d32/d33/d3c/le9 0 2024-03-24T10:46:19.419 INFO:tasks.workunit.client.1.smithi096.stdout:5/711: unlink d2/d29/d32/d33/d3c/dc3/fdf 0 2024-03-24T10:46:19.419 INFO:tasks.workunit.client.1.smithi096.stdout:5/712: readlink d2/lc7 0 2024-03-24T10:46:19.421 INFO:tasks.workunit.client.1.smithi096.stdout:2/861: write d5/de/d2a/d5e/f65 [2352080,69978] 0 2024-03-24T10:46:19.422 INFO:tasks.workunit.client.1.smithi096.stdout:2/862: unlink d5/dc/d6b/cd2 0 2024-03-24T10:46:19.425 INFO:tasks.workunit.client.1.smithi096.stdout:2/863: rename d5/dc/d6b/d72/de0 to d5/d7c/d8f/d128/d141 0 2024-03-24T10:46:19.435 INFO:tasks.workunit.client.1.smithi096.stdout:5/713: dwrite d2/d29/d32/db9/fb7 [0,4194304] 0 2024-03-24T10:46:19.439 INFO:tasks.workunit.client.1.smithi096.stdout:5/714: mkdir d2/d7/d8/df/d14/dea 0 2024-03-24T10:46:19.439 INFO:tasks.workunit.client.1.smithi096.stdout:5/715: chown d2/d29/d32/d3f/d84/db8/ce0 911 1 2024-03-24T10:46:19.442 INFO:tasks.workunit.client.1.smithi096.stdout:5/716: mkdir d2/d7/d69/de4/deb 0 2024-03-24T10:46:19.462 INFO:tasks.workunit.client.1.smithi096.stdout:2/864: dwrite d5/dc/d12/fc5 [0,4194304] 0 2024-03-24T10:46:19.462 INFO:tasks.workunit.client.1.smithi096.stdout:5/717: dwrite d2/d29/d32/d33/d37/f6d [0,4194304] 0 2024-03-24T10:46:19.467 INFO:tasks.workunit.client.1.smithi096.stdout:5/718: mknod d2/d7/d8/d18/db6/ddd/cec 0 2024-03-24T10:46:19.475 INFO:tasks.workunit.client.1.smithi096.stdout:5/719: write d2/d7/f7f [1643655,123252] 0 2024-03-24T10:46:19.475 INFO:tasks.workunit.client.1.smithi096.stdout:5/720: chown d2/d7/d69/de4 0 1 2024-03-24T10:46:19.477 INFO:tasks.workunit.client.1.smithi096.stdout:5/721: creat d2/d29/d32/d33/d37/dab/fed x:0 0 0 2024-03-24T10:46:19.477 INFO:tasks.workunit.client.1.smithi096.stdout:5/722: write d2/d7/d8/d18/f93 [1577625,65531] 0 2024-03-24T10:46:19.477 INFO:tasks.workunit.client.1.smithi096.stdout:5/723: write d2/d29/d32/d33/f63 [473749,61703] 0 2024-03-24T10:46:19.482 INFO:tasks.workunit.client.1.smithi096.stdout:5/724: dread d2/d7/d69/f91 [0,4194304] 0 2024-03-24T10:46:19.482 INFO:tasks.workunit.client.1.smithi096.stdout:5/725: fdatasync d2/d29/d32/d33/d8d/f9e 0 2024-03-24T10:46:19.484 INFO:tasks.workunit.client.1.smithi096.stdout:5/726: link d2/d29/d32/d33/d3c/dd2/cc4 d2/d29/d32/dbb/cee 0 2024-03-24T10:46:19.484 INFO:tasks.workunit.client.1.smithi096.stdout:5/727: fsync d2/d7/d8/df/f25 0 2024-03-24T10:46:19.484 INFO:tasks.workunit.client.1.smithi096.stdout:5/728: fsync d2/d29/d32/d3f/d5f/f8f 0 2024-03-24T10:46:19.485 INFO:tasks.workunit.client.1.smithi096.stdout:5/729: mknod d2/d29/cef 0 2024-03-24T10:46:19.486 INFO:tasks.workunit.client.1.smithi096.stdout:5/730: mkdir d2/d7/d69/dce/df0 0 2024-03-24T10:46:19.487 INFO:tasks.workunit.client.1.smithi096.stdout:5/731: truncate d2/d7/d8/df/d14/f5c 4062146 0 2024-03-24T10:46:19.489 INFO:tasks.workunit.client.1.smithi096.stdout:5/732: symlink d2/d7/d8/df/d14/d96/lf1 0 2024-03-24T10:46:19.489 INFO:tasks.workunit.client.1.smithi096.stdout:5/733: creat d2/d29/d32/dbb/ff2 x:0 0 0 2024-03-24T10:46:19.489 INFO:tasks.workunit.client.1.smithi096.stdout:1/594: sync 2024-03-24T10:46:19.492 INFO:tasks.workunit.client.1.smithi096.stdout:5/734: mkdir d2/d29/d32/d33/d8d/df3 0 2024-03-24T10:46:19.492 INFO:tasks.workunit.client.1.smithi096.stdout:5/735: chown d2/d7/d69/f91 5 1 2024-03-24T10:46:19.492 INFO:tasks.workunit.client.1.smithi096.stdout:5/736: read - d2/d29/d32/fa8 zero size 2024-03-24T10:46:19.492 INFO:tasks.workunit.client.1.smithi096.stdout:5/737: read - d2/d29/d32/d3f/d84/faf zero size 2024-03-24T10:46:19.492 INFO:tasks.workunit.client.1.smithi096.stdout:5/738: chown d2/d29/d32/d33/d3c/c7b 86 1 2024-03-24T10:46:19.492 INFO:tasks.workunit.client.1.smithi096.stdout:5/739: dread - d2/d29/d32/dbb/ff2 zero size 2024-03-24T10:46:19.492 INFO:tasks.workunit.client.1.smithi096.stdout:5/740: write d2/d29/d32/d89/fa1 [1243558,53664] 0 2024-03-24T10:46:19.492 INFO:tasks.workunit.client.1.smithi096.stdout:5/741: fdatasync d2/d29/d32/d33/d37/dab/fed 0 2024-03-24T10:46:19.493 INFO:tasks.workunit.client.1.smithi096.stdout:5/742: symlink d2/d7/dba/lf4 0 2024-03-24T10:46:19.493 INFO:tasks.workunit.client.1.smithi096.stdout:5/743: dread - d2/d29/d32/fe5 zero size 2024-03-24T10:46:19.494 INFO:tasks.workunit.client.1.smithi096.stdout:5/744: mknod d2/d7/d8/d18/db6/cf5 0 2024-03-24T10:46:19.497 INFO:tasks.workunit.client.1.smithi096.stdout:5/745: rename d2/d29/d32/dbb/cee to d2/d29/d32/d33/d3c/dd2/cf6 0 2024-03-24T10:46:19.504 INFO:tasks.workunit.client.1.smithi096.stdout:1/595: dwrite d3/d23/f62 [0,4194304] 0 2024-03-24T10:46:19.504 INFO:tasks.workunit.client.1.smithi096.stdout:1/596: fsync d3/d36/fc2 0 2024-03-24T10:46:19.505 INFO:tasks.workunit.client.1.smithi096.stdout:5/746: rename d2/d29/d32/d3f/d5f/db3 to d2/d29/df7 0 2024-03-24T10:46:19.505 INFO:tasks.workunit.client.1.smithi096.stdout:5/747: dread - d2/d29/d32/d3f/f9b zero size 2024-03-24T10:46:19.510 INFO:tasks.workunit.client.1.smithi096.stdout:1/597: mkdir d3/d23/d71/d7b/dce 0 2024-03-24T10:46:19.510 INFO:tasks.workunit.client.1.smithi096.stdout:1/598: write d3/d23/d71/f8a [347725,20511] 0 2024-03-24T10:46:19.510 INFO:tasks.workunit.client.1.smithi096.stdout:1/599: chown d3/d23/d2f 14927 1 2024-03-24T10:46:19.516 INFO:tasks.workunit.client.1.smithi096.stdout:5/748: creat d2/d29/d32/d33/d8d/d94/ff8 x:0 0 0 2024-03-24T10:46:19.519 INFO:tasks.workunit.client.1.smithi096.stdout:1/600: unlink d3/d36/c74 0 2024-03-24T10:46:19.535 INFO:tasks.workunit.client.1.smithi096.stdout:1/601: dwrite d3/d23/d2f/f3d [0,4194304] 0 2024-03-24T10:46:19.537 INFO:tasks.workunit.client.1.smithi096.stdout:1/602: dread d3/d23/d8b/f4c [0,4194304] 0 2024-03-24T10:46:19.538 INFO:tasks.workunit.client.1.smithi096.stdout:1/603: creat d3/d73/fcf x:0 0 0 2024-03-24T10:46:19.558 INFO:tasks.workunit.client.1.smithi096.stdout:5/749: dwrite d2/d29/d32/d3f/f55 [0,4194304] 0 2024-03-24T10:46:19.558 INFO:tasks.workunit.client.1.smithi096.stdout:1/604: dwrite d3/d23/d2f/f3b [0,4194304] 0 2024-03-24T10:46:19.561 INFO:tasks.workunit.client.1.smithi096.stdout:5/750: mkdir d2/d29/d32/d33/d8d/d94/df9 0 2024-03-24T10:46:19.561 INFO:tasks.workunit.client.1.smithi096.stdout:5/751: chown d2/d7/d69/f91 6387 1 2024-03-24T10:46:19.561 INFO:tasks.workunit.client.1.smithi096.stdout:5/752: getdents d2/d29/d32/d3f/d84 0 2024-03-24T10:46:19.566 INFO:tasks.workunit.client.1.smithi096.stdout:1/605: dread d3/d23/f62 [0,4194304] 0 2024-03-24T10:46:19.567 INFO:tasks.workunit.client.1.smithi096.stdout:3/721: sync 2024-03-24T10:46:19.568 INFO:tasks.workunit.client.1.smithi096.stdout:7/836: sync 2024-03-24T10:46:19.570 INFO:tasks.workunit.client.1.smithi096.stdout:5/753: dread d2/d29/d32/d33/d37/dab/fb1 [0,4194304] 0 2024-03-24T10:46:19.570 INFO:tasks.workunit.client.1.smithi096.stdout:5/754: dread - d2/d29/d32/d3f/d84/db8/fe6 zero size 2024-03-24T10:46:19.570 INFO:tasks.workunit.client.1.smithi096.stdout:7/837: unlink d0/db/l90 0 2024-03-24T10:46:19.574 INFO:tasks.workunit.client.1.smithi096.stdout:3/722: mknod d0/d13/d45/d7d/d39/d48/dbd/dbc/ceb 0 2024-03-24T10:46:19.574 INFO:tasks.workunit.client.1.smithi096.stdout:3/723: getdents d0/d9 0 2024-03-24T10:46:19.575 INFO:tasks.workunit.client.1.smithi096.stdout:1/606: dread d3/d23/d8b/d5c/d9c/fcb [0,4194304] 0 2024-03-24T10:46:19.575 INFO:tasks.workunit.client.1.smithi096.stdout:1/607: dread - d3/d23/d8b/fcd zero size 2024-03-24T10:46:19.581 INFO:tasks.workunit.client.1.smithi096.stdout:1/608: link d3/d23/d8b/d5c/d6d/cab d3/d23/d2f/d65/d7c/cd0 0 2024-03-24T10:46:19.581 INFO:tasks.workunit.client.1.smithi096.stdout:5/755: getdents d2/d29/d32/d3f/d5f/dc0 0 2024-03-24T10:46:19.582 INFO:tasks.workunit.client.1.smithi096.stdout:7/838: rmdir d0/db/d1c/d39 39 2024-03-24T10:46:19.582 INFO:tasks.workunit.client.1.smithi096.stdout:7/839: write d0/d13/df0/d3c/d68/d101/f10c [575452,79964] 0 2024-03-24T10:46:19.583 INFO:tasks.workunit.client.1.smithi096.stdout:7/840: truncate d0/d13/df0/d59/f83 767634 0 2024-03-24T10:46:19.583 INFO:tasks.workunit.client.1.smithi096.stdout:7/841: creat d0/d13/df0/d3c/d68/d4b/de8/f10e x:0 0 0 2024-03-24T10:46:19.584 INFO:tasks.workunit.client.1.smithi096.stdout:1/609: rename d3/d23/d2f/f3b to d3/d23/d8b/d79/fd1 0 2024-03-24T10:46:19.585 INFO:tasks.workunit.client.1.smithi096.stdout:5/756: truncate d2/d29/d32/d89/f9c 539614 0 2024-03-24T10:46:19.589 INFO:tasks.workunit.client.1.smithi096.stdout:1/610: mkdir d3/d84/d9a/dd2 0 2024-03-24T10:46:19.592 INFO:tasks.workunit.client.1.smithi096.stdout:7/842: dread d0/db/ff [4194304,4194304] 0 2024-03-24T10:46:19.596 INFO:tasks.workunit.client.1.smithi096.stdout:7/843: truncate d0/db/f6d 3460477 0 2024-03-24T10:46:19.596 INFO:tasks.workunit.client.1.smithi096.stdout:7/844: stat d0/da9/ddb/lee 0 2024-03-24T10:46:19.596 INFO:tasks.workunit.client.1.smithi096.stdout:7/845: stat d0/d13/d32/d45/dcd 0 2024-03-24T10:46:19.599 INFO:tasks.workunit.client.1.smithi096.stdout:1/611: rename d3/d84/c91 to d3/d23/d8b/d5c/cd3 0 2024-03-24T10:46:19.599 INFO:tasks.workunit.client.1.smithi096.stdout:1/612: chown d3/d36/l5b 7820 1 2024-03-24T10:46:19.599 INFO:tasks.workunit.client.1.smithi096.stdout:1/613: chown d3/d23/d8b/d79/fd1 178 1 2024-03-24T10:46:19.600 INFO:tasks.workunit.client.1.smithi096.stdout:1/614: mkdir d3/d36/dd4 0 2024-03-24T10:46:19.606 INFO:tasks.workunit.client.1.smithi096.stdout:5/757: dwrite d2/d7/f8c [0,4194304] 0 2024-03-24T10:46:19.609 INFO:tasks.workunit.client.1.smithi096.stdout:5/758: creat d2/d7/d8/d18/ffa x:0 0 0 2024-03-24T10:46:19.609 INFO:tasks.workunit.client.1.smithi096.stdout:5/759: chown d2/d29/d32/fe5 521432381 1 2024-03-24T10:46:19.617 INFO:tasks.workunit.client.1.smithi096.stdout:7/846: dwrite d0/d13/df0/d3c/faf [0,4194304] 0 2024-03-24T10:46:19.617 INFO:tasks.workunit.client.1.smithi096.stdout:7/847: readlink d0/db/d4f/l7b 0 2024-03-24T10:46:19.617 INFO:tasks.workunit.client.1.smithi096.stdout:7/848: chown d0/d13/ffa 361098 1 2024-03-24T10:46:19.618 INFO:tasks.workunit.client.1.smithi096.stdout:7/849: creat d0/f10f x:0 0 0 2024-03-24T10:46:19.621 INFO:tasks.workunit.client.1.smithi096.stdout:5/760: dread d2/d7/d8/d18/db6/fd1 [0,4194304] 0 2024-03-24T10:46:19.626 INFO:tasks.workunit.client.1.smithi096.stdout:7/850: unlink d0/db/d1c/d77/fd8 0 2024-03-24T10:46:19.627 INFO:tasks.workunit.client.1.smithi096.stdout:7/851: dread d0/d13/d32/d45/f63 [0,4194304] 0 2024-03-24T10:46:19.628 INFO:tasks.workunit.client.1.smithi096.stdout:5/761: symlink d2/d7/d8/df/d14/dea/lfb 0 2024-03-24T10:46:19.637 INFO:tasks.workunit.client.1.smithi096.stdout:7/852: getdents d0/db/d1c 0 2024-03-24T10:46:19.637 INFO:tasks.workunit.client.1.smithi096.stdout:7/853: creat d0/db/d26/d48/f110 x:0 0 0 2024-03-24T10:46:19.638 INFO:tasks.workunit.client.1.smithi096.stdout:5/762: rename d2/d7/d8/df/c68 to d2/d7/d8/cfc 0 2024-03-24T10:46:19.643 INFO:tasks.workunit.client.1.smithi096.stdout:7/854: mkdir d0/d111 0 2024-03-24T10:46:19.643 INFO:tasks.workunit.client.1.smithi096.stdout:7/855: chown d0/db/l3e 3059330 1 2024-03-24T10:46:19.643 INFO:tasks.workunit.client.1.smithi096.stdout:7/856: chown d0/d13/df0/d3c/d68/d4b/ffc 103 1 2024-03-24T10:46:19.643 INFO:tasks.workunit.client.1.smithi096.stdout:5/763: symlink d2/d29/d32/d3f/d5f/lfd 0 2024-03-24T10:46:19.649 INFO:tasks.workunit.client.1.smithi096.stdout:5/764: dread d2/d29/d32/d33/d3c/d3d/fb5 [0,4194304] 0 2024-03-24T10:46:19.657 INFO:tasks.workunit.client.1.smithi096.stdout:5/765: read d2/d7/f10 [3137154,61336] 0 2024-03-24T10:46:19.662 INFO:tasks.workunit.client.1.smithi096.stdout:5/766: getdents d2/d7/d69 0 2024-03-24T10:46:19.671 INFO:tasks.workunit.client.1.smithi096.stdout:5/767: dread d2/d29/d32/d33/d8d/d94/fd9 [0,4194304] 0 2024-03-24T10:46:19.671 INFO:tasks.workunit.client.1.smithi096.stdout:5/768: rmdir d2/d7/d8 39 2024-03-24T10:46:19.672 INFO:tasks.workunit.client.1.smithi096.stdout:7/857: dwrite d0/d13/df0/d3c/d68/fa7 [0,4194304] 0 2024-03-24T10:46:19.672 INFO:tasks.workunit.client.1.smithi096.stdout:5/769: creat d2/d29/d32/d33/d3c/d3d/ffe x:0 0 0 2024-03-24T10:46:19.676 INFO:tasks.workunit.client.1.smithi096.stdout:5/770: fsync d2/d7/d8/df/d14/f5c 0 2024-03-24T10:46:19.678 INFO:tasks.workunit.client.1.smithi096.stdout:9/774: sync 2024-03-24T10:46:19.678 INFO:tasks.workunit.client.1.smithi096.stdout:2/865: sync 2024-03-24T10:46:19.678 INFO:tasks.workunit.client.1.smithi096.stdout:6/696: sync 2024-03-24T10:46:19.678 INFO:tasks.workunit.client.1.smithi096.stdout:0/790: sync 2024-03-24T10:46:19.679 INFO:tasks.workunit.client.1.smithi096.stdout:8/764: sync 2024-03-24T10:46:19.679 INFO:tasks.workunit.client.1.smithi096.stdout:4/628: sync 2024-03-24T10:46:19.679 INFO:tasks.workunit.client.1.smithi096.stdout:8/765: fsync d8/d11/d22/d4c/d65/d1a/fb5 0 2024-03-24T10:46:19.679 INFO:tasks.workunit.client.1.smithi096.stdout:4/629: truncate d1/d32/d4c/d7e/f99 5467353 0 2024-03-24T10:46:19.679 INFO:tasks.workunit.client.1.smithi096.stdout:4/630: stat d1/d32/d4c/dc1 0 2024-03-24T10:46:19.679 INFO:tasks.workunit.client.1.smithi096.stdout:5/771: creat d2/d7/d8/d18/db6/fff x:0 0 0 2024-03-24T10:46:19.686 INFO:tasks.workunit.client.1.smithi096.stdout:8/766: dread d8/d72/d96/f87 [0,4194304] 0 2024-03-24T10:46:19.690 INFO:tasks.workunit.client.1.smithi096.stdout:6/697: write d8/d12/d28/d36/d65/d71/dd0/da8/dbe/fbd [257370,36940] 0 2024-03-24T10:46:19.691 INFO:tasks.workunit.client.1.smithi096.stdout:6/698: write d8/f31 [3953205,130021] 0 2024-03-24T10:46:19.691 INFO:tasks.workunit.client.1.smithi096.stdout:7/858: dwrite d0/d13/df0/f8f [0,4194304] 0 2024-03-24T10:46:19.691 INFO:tasks.workunit.client.1.smithi096.stdout:7/859: chown d0/d13/df0/d59/fe2 7 1 2024-03-24T10:46:19.691 INFO:tasks.workunit.client.1.smithi096.stdout:7/860: readlink d0/d13/d32/l34 0 2024-03-24T10:46:19.692 INFO:tasks.workunit.client.1.smithi096.stdout:2/866: creat d5/dc/d12/f142 x:0 0 0 2024-03-24T10:46:19.694 INFO:tasks.workunit.client.1.smithi096.stdout:9/775: mkdir d9/db/d6d/d4f/d75/d8b/d8e/dfb 0 2024-03-24T10:46:19.696 INFO:tasks.workunit.client.1.smithi096.stdout:4/631: mkdir d1/d2b/d6b/d80/d9c/dcb 0 2024-03-24T10:46:19.696 INFO:tasks.workunit.client.1.smithi096.stdout:5/772: mkdir d2/d29/d32/d33/d37/dab/d100 0 2024-03-24T10:46:19.698 INFO:tasks.workunit.client.1.smithi096.stdout:8/767: rmdir d8/d11/d22/d4c/d65/d1a 39 2024-03-24T10:46:19.699 INFO:tasks.workunit.client.1.smithi096.stdout:6/699: getdents d8/d12/d28/d36/d65/d71 0 2024-03-24T10:46:19.700 INFO:tasks.workunit.client.1.smithi096.stdout:2/867: write d5/d38/fa1 [3428273,67859] 0 2024-03-24T10:46:19.700 INFO:tasks.workunit.client.1.smithi096.stdout:2/868: write d5/dc/d12/f26 [4034241,73254] 0 2024-03-24T10:46:19.706 INFO:tasks.workunit.client.1.smithi096.stdout:9/776: creat d9/db/d6d/dc6/df0/ffc x:0 0 0 2024-03-24T10:46:19.706 INFO:tasks.workunit.client.1.smithi096.stdout:5/773: symlink d2/d7/d69/l101 0 2024-03-24T10:46:19.707 INFO:tasks.workunit.client.1.smithi096.stdout:8/768: rmdir d8/d11/d22/d4c/d65/d1d/d3d/d5f/de2 39 2024-03-24T10:46:19.708 INFO:tasks.workunit.client.1.smithi096.stdout:6/700: symlink d8/d12/d32/d38/ld9 0 2024-03-24T10:46:19.709 INFO:tasks.workunit.client.1.smithi096.stdout:2/869: creat d5/dc/d12/d23/f143 x:0 0 0 2024-03-24T10:46:19.709 INFO:tasks.workunit.client.1.smithi096.stdout:2/870: write d5/d38/fc4 [15284,86156] 0 2024-03-24T10:46:19.710 INFO:tasks.workunit.client.1.smithi096.stdout:7/861: mknod d0/db/d26/dc3/c112 0 2024-03-24T10:46:19.711 INFO:tasks.workunit.client.1.smithi096.stdout:4/632: dwrite d1/d11/f66 [0,4194304] 0 2024-03-24T10:46:19.711 INFO:tasks.workunit.client.1.smithi096.stdout:9/777: symlink d9/d15/d26/d29/dcd/lfd 0 2024-03-24T10:46:19.711 INFO:tasks.workunit.client.1.smithi096.stdout:9/778: chown d9/db/d6d/d7f/f3f 4749826 1 2024-03-24T10:46:19.713 INFO:tasks.workunit.client.1.smithi096.stdout:8/769: truncate d8/d11/d22/d4c/d65/d37/f6e 725423 0 2024-03-24T10:46:19.714 INFO:tasks.workunit.client.1.smithi096.stdout:6/701: link f7 d8/d10/d13/dcc/fda 0 2024-03-24T10:46:19.716 INFO:tasks.workunit.client.1.smithi096.stdout:7/862: mknod d0/db/d1c/d39/da2/c113 0 2024-03-24T10:46:19.716 INFO:tasks.workunit.client.1.smithi096.stdout:7/863: chown d0/d13/df0/d3c/d68/d4b/d62/d108 52394 1 2024-03-24T10:46:19.716 INFO:tasks.workunit.client.1.smithi096.stdout:5/774: write d2/d29/f38 [1133592,14701] 0 2024-03-24T10:46:19.720 INFO:tasks.workunit.client.1.smithi096.stdout:4/633: rename d1/d2b/d40/fa5 to d1/d2b/d6b/d80/d9c/dcb/fcc 0 2024-03-24T10:46:19.720 INFO:tasks.workunit.client.1.smithi096.stdout:4/634: chown d1/d11/l8e 56238 1 2024-03-24T10:46:19.720 INFO:tasks.workunit.client.1.smithi096.stdout:4/635: chown d1/d2b/d59/c91 15 1 2024-03-24T10:46:19.720 INFO:tasks.workunit.client.1.smithi096.stdout:4/636: creat d1/d2b/d59/d74/fcd x:0 0 0 2024-03-24T10:46:19.720 INFO:tasks.workunit.client.1.smithi096.stdout:4/637: truncate d1/d2b/d6b/f76 1056094 0 2024-03-24T10:46:19.720 INFO:tasks.workunit.client.1.smithi096.stdout:4/638: chown d1/d32/d4c/d7e 1 1 2024-03-24T10:46:19.720 INFO:tasks.workunit.client.1.smithi096.stdout:4/639: chown d1/d32/cc9 333 1 2024-03-24T10:46:19.721 INFO:tasks.workunit.client.1.smithi096.stdout:9/779: truncate d9/db/d6d/f35 4162754 0 2024-03-24T10:46:19.723 INFO:tasks.workunit.client.1.smithi096.stdout:8/770: write d8/d11/d85/fda [3193442,21259] 0 2024-03-24T10:46:19.726 INFO:tasks.workunit.client.1.smithi096.stdout:6/702: unlink d8/d12/d32/d38/l4c 0 2024-03-24T10:46:19.726 INFO:tasks.workunit.client.1.smithi096.stdout:6/703: dread - d8/d12/d61/faa zero size 2024-03-24T10:46:19.727 INFO:tasks.workunit.client.1.smithi096.stdout:5/775: mknod d2/c102 0 2024-03-24T10:46:19.728 INFO:tasks.workunit.client.1.smithi096.stdout:8/771: dread d8/d11/d22/d4c/d82/f70 [0,4194304] 0 2024-03-24T10:46:19.729 INFO:tasks.workunit.client.1.smithi096.stdout:2/871: dwrite d5/dc/fb7 [0,4194304] 0 2024-03-24T10:46:19.729 INFO:tasks.workunit.client.1.smithi096.stdout:2/872: fsync d5/dc/d6b/d72/d91/fdc 0 2024-03-24T10:46:19.731 INFO:tasks.workunit.client.1.smithi096.stdout:7/864: rename d0/db/c2b to d0/c114 0 2024-03-24T10:46:19.731 INFO:tasks.workunit.client.1.smithi096.stdout:7/865: fdatasync d0/db/d26/d48/f8d 0 2024-03-24T10:46:19.731 INFO:tasks.workunit.client.1.smithi096.stdout:7/866: chown d0/d13/d32/d45/l92 2 1 2024-03-24T10:46:19.737 INFO:tasks.workunit.client.1.smithi096.stdout:6/704: creat d8/d12/d61/fdb x:0 0 0 2024-03-24T10:46:19.738 INFO:tasks.workunit.client.1.smithi096.stdout:4/640: dwrite d1/d2b/d6b/d80/fb6 [0,4194304] 0 2024-03-24T10:46:19.740 INFO:tasks.workunit.client.1.smithi096.stdout:8/772: write d8/d11/d85/f8a [3068520,124855] 0 2024-03-24T10:46:19.746 INFO:tasks.workunit.client.1.smithi096.stdout:2/873: unlink d5/de/f103 0 2024-03-24T10:46:19.746 INFO:tasks.workunit.client.1.smithi096.stdout:2/874: read d5/de/d2a/f10f [50025,70253] 0 2024-03-24T10:46:19.750 INFO:tasks.workunit.client.1.smithi096.stdout:6/705: getdents d8/d12/d28/d36 0 2024-03-24T10:46:19.754 INFO:tasks.workunit.client.1.smithi096.stdout:5/776: dwrite d2/d7/d8/d18/db6/fff [0,4194304] 0 2024-03-24T10:46:19.754 INFO:tasks.workunit.client.1.smithi096.stdout:7/867: dwrite d0/d13/df0/d3c/d80/fc1 [0,4194304] 0 2024-03-24T10:46:19.754 INFO:tasks.workunit.client.1.smithi096.stdout:7/868: read - d0/d13/df0/d3c/d68/d4b/de8/f10e zero size 2024-03-24T10:46:19.755 INFO:tasks.workunit.client.1.smithi096.stdout:2/875: dread d5/d7c/f7e [0,4194304] 0 2024-03-24T10:46:19.761 INFO:tasks.workunit.client.1.smithi096.stdout:4/641: symlink d1/d2b/d67/lce 0 2024-03-24T10:46:19.765 INFO:tasks.workunit.client.1.smithi096.stdout:8/773: creat d8/d11/d22/d4c/d82/da1/ff8 x:0 0 0 2024-03-24T10:46:19.772 INFO:tasks.workunit.client.1.smithi096.stdout:6/706: dwrite d8/d12/d32/d38/d59/db8/fb1 [0,4194304] 0 2024-03-24T10:46:19.773 INFO:tasks.workunit.client.1.smithi096.stdout:2/876: dwrite d5/dc/f13d [0,4194304] 0 2024-03-24T10:46:19.780 INFO:tasks.workunit.client.1.smithi096.stdout:4/642: dwrite d1/d11/f12 [0,4194304] 0 2024-03-24T10:46:19.780 INFO:tasks.workunit.client.1.smithi096.stdout:4/643: chown d1/d2b/d6b/d80/d9c/dcb/fcc 55504 1 2024-03-24T10:46:19.780 INFO:tasks.workunit.client.1.smithi096.stdout:4/644: creat d1/d32/d4c/d7e/fcf x:0 0 0 2024-03-24T10:46:19.780 INFO:tasks.workunit.client.1.smithi096.stdout:4/645: truncate d1/d11/fae 811333 0 2024-03-24T10:46:19.780 INFO:tasks.workunit.client.1.smithi096.stdout:4/646: creat d1/d32/d4c/d7e/fd0 x:0 0 0 2024-03-24T10:46:19.780 INFO:tasks.workunit.client.1.smithi096.stdout:4/647: readlink d1/d11/l2c 0 2024-03-24T10:46:19.780 INFO:tasks.workunit.client.1.smithi096.stdout:4/648: chown d1/d2b/d6b/d80/d9c/dcb 2865872 1 2024-03-24T10:46:19.780 INFO:tasks.workunit.client.1.smithi096.stdout:4/649: creat d1/d2b/d6b/d80/fd1 x:0 0 0 2024-03-24T10:46:19.780 INFO:tasks.workunit.client.1.smithi096.stdout:5/777: mkdir d2/d7/d8/d18/db6/ddd/d103 0 2024-03-24T10:46:19.780 INFO:tasks.workunit.client.1.smithi096.stdout:4/650: write d1/d32/d4c/d7d/fab [58596,110683] 0 2024-03-24T10:46:19.782 INFO:tasks.workunit.client.1.smithi096.stdout:7/869: getdents d0/db/d26/dc3 0 2024-03-24T10:46:19.784 INFO:tasks.workunit.client.1.smithi096.stdout:8/774: truncate d8/d11/f43 1772995 0 2024-03-24T10:46:19.790 INFO:tasks.workunit.client.1.smithi096.stdout:7/870: write d0/d13/df0/d59/f79 [1639674,33730] 0 2024-03-24T10:46:19.791 INFO:tasks.workunit.client.1.smithi096.stdout:7/871: fdatasync d0/d13/ffa 0 2024-03-24T10:46:19.791 INFO:tasks.workunit.client.1.smithi096.stdout:7/872: stat d0/db/d1c/f91 0 2024-03-24T10:46:19.793 INFO:tasks.workunit.client.1.smithi096.stdout:2/877: mkdir d5/dc/d144 0 2024-03-24T10:46:19.794 INFO:tasks.workunit.client.1.smithi096.stdout:4/651: dwrite d1/d2b/d59/d74/fcd [0,4194304] 0 2024-03-24T10:46:19.794 INFO:tasks.workunit.client.1.smithi096.stdout:4/652: readlink d1/d2b/l2d 0 2024-03-24T10:46:19.797 INFO:tasks.workunit.client.1.smithi096.stdout:5/778: rename d2/d29/d32/dbb/cd0 to d2/d7/d8/d18/db6/ddd/c104 0 2024-03-24T10:46:19.798 INFO:tasks.workunit.client.1.smithi096.stdout:8/775: truncate d8/d11/f19 716353 0 2024-03-24T10:46:19.799 INFO:tasks.workunit.client.1.smithi096.stdout:8/776: chown d8/d11/d22/d4c/d65 7 1 2024-03-24T10:46:19.800 INFO:tasks.workunit.client.1.smithi096.stdout:6/707: rmdir d8/d12/d28/d36/d65 39 2024-03-24T10:46:19.800 INFO:tasks.workunit.client.1.smithi096.stdout:6/708: stat d8/d12/d28/d36/d65/d6b/da6/db2/cb9 0 2024-03-24T10:46:19.801 INFO:tasks.workunit.client.1.smithi096.stdout:7/873: creat d0/db/d1c/d39/da2/f115 x:0 0 0 2024-03-24T10:46:19.801 INFO:tasks.workunit.client.1.smithi096.stdout:7/874: dread - d0/d13/d32/d45/fde zero size 2024-03-24T10:46:19.801 INFO:tasks.workunit.client.1.smithi096.stdout:7/875: creat d0/db/d26/d48/da8/f116 x:0 0 0 2024-03-24T10:46:19.802 INFO:tasks.workunit.client.1.smithi096.stdout:2/878: link d5/d7c/d88/d90/fb1 d5/dc/d6b/d72/d91/f145 0 2024-03-24T10:46:19.810 INFO:tasks.workunit.client.1.smithi096.stdout:4/653: unlink d1/d2b/d6b/d80/c94 0 2024-03-24T10:46:19.810 INFO:tasks.workunit.client.1.smithi096.stdout:7/876: dread d0/d13/df0/d3c/d68/f41 [0,4194304] 0 2024-03-24T10:46:19.810 INFO:tasks.workunit.client.1.smithi096.stdout:7/877: readlink d0/db/d26/l5e 0 2024-03-24T10:46:19.810 INFO:tasks.workunit.client.1.smithi096.stdout:5/779: creat d2/d29/d32/d33/d3c/dc3/f105 x:0 0 0 2024-03-24T10:46:19.812 INFO:tasks.workunit.client.1.smithi096.stdout:8/777: creat d8/d11/d22/d4c/d65/d1d/d3d/ff9 x:0 0 0 2024-03-24T10:46:19.813 INFO:tasks.workunit.client.1.smithi096.stdout:6/709: creat d8/fdc x:0 0 0 2024-03-24T10:46:19.813 INFO:tasks.workunit.client.1.smithi096.stdout:2/879: dread d5/fa7 [0,4194304] 0 2024-03-24T10:46:19.815 INFO:tasks.workunit.client.1.smithi096.stdout:4/654: creat d1/d2b/d67/d5f/da9/db5/fd2 x:0 0 0 2024-03-24T10:46:19.815 INFO:tasks.workunit.client.1.smithi096.stdout:8/778: rmdir d8/d11/d22/d4c/d65/d1d 39 2024-03-24T10:46:19.818 INFO:tasks.workunit.client.1.smithi096.stdout:6/710: rename d8/d12/d32/c54 to d8/d12/d28/cdd 0 2024-03-24T10:46:19.820 INFO:tasks.workunit.client.1.smithi096.stdout:2/880: rename d5/dc/d12/d23/d105/c111 to d5/d38/dbd/c146 0 2024-03-24T10:46:19.820 INFO:tasks.workunit.client.1.smithi096.stdout:2/881: chown d5/d79/db5/cc1 1 1 2024-03-24T10:46:19.820 INFO:tasks.workunit.client.1.smithi096.stdout:6/711: stat d8/d12/fb7 0 2024-03-24T10:46:19.821 INFO:tasks.workunit.client.1.smithi096.stdout:4/655: read d1/d2b/d40/f60 [887829,10669] 0 2024-03-24T10:46:19.822 INFO:tasks.workunit.client.1.smithi096.stdout:8/779: rename d8/d11/d22/d4c/d65/d1a/d46/c40 to d8/d11/d22/d4c/db0/cfa 0 2024-03-24T10:46:19.822 INFO:tasks.workunit.client.1.smithi096.stdout:8/780: readlink d8/d11/d22/d4c/db0/dcc/lcd 0 2024-03-24T10:46:19.824 INFO:tasks.workunit.client.1.smithi096.stdout:2/882: creat d5/de/d2a/d5e/dd6/f147 x:0 0 0 2024-03-24T10:46:19.826 INFO:tasks.workunit.client.1.smithi096.stdout:6/712: mkdir d8/d10/d13/dd8/dde 0 2024-03-24T10:46:19.826 INFO:tasks.workunit.client.1.smithi096.stdout:6/713: stat d8/d12/d28/d36/d65/d71/l87 0 2024-03-24T10:46:19.830 INFO:tasks.workunit.client.1.smithi096.stdout:4/656: rename d1/d2b/d6b/d80/la2 to d1/ld3 0 2024-03-24T10:46:19.834 INFO:tasks.workunit.client.1.smithi096.stdout:6/714: write d8/d12/d28/d36/d65/d71/dd0/fba [371923,36900] 0 2024-03-24T10:46:19.835 INFO:tasks.workunit.client.1.smithi096.stdout:6/715: write d8/d10/d19/f60 [2335992,31481] 0 2024-03-24T10:46:19.835 INFO:tasks.workunit.client.1.smithi096.stdout:2/883: mkdir d5/de/d148 0 2024-03-24T10:46:19.838 INFO:tasks.workunit.client.1.smithi096.stdout:2/884: truncate d5/d79/d9f/dad/fdf 859189 0 2024-03-24T10:46:19.841 INFO:tasks.workunit.client.1.smithi096.stdout:2/885: unlink d5/de/d2a/c137 0 2024-03-24T10:46:19.841 INFO:tasks.workunit.client.1.smithi096.stdout:2/886: creat d5/dc/d6b/def/f149 x:0 0 0 2024-03-24T10:46:19.845 INFO:tasks.workunit.client.1.smithi096.stdout:2/887: rename d5/d38/dbd/cfe to d5/de/d148/c14a 0 2024-03-24T10:46:19.847 INFO:tasks.workunit.client.1.smithi096.stdout:2/888: link d5/d7c/d88/f61 d5/d7c/d88/f14b 0 2024-03-24T10:46:19.848 INFO:tasks.workunit.client.1.smithi096.stdout:2/889: mkdir d5/dc/d6b/d72/d14c 0 2024-03-24T10:46:19.849 INFO:tasks.workunit.client.1.smithi096.stdout:2/890: getdents d5/d79/d9f 0 2024-03-24T10:46:19.850 INFO:tasks.workunit.client.1.smithi096.stdout:7/878: dwrite d0/db/d4f/f58 [0,4194304] 0 2024-03-24T10:46:19.851 INFO:tasks.workunit.client.1.smithi096.stdout:7/879: dread - d0/d13/d32/d45/dcd/f100 zero size 2024-03-24T10:46:19.851 INFO:tasks.workunit.client.1.smithi096.stdout:2/891: symlink d5/dc/d12/d23/d105/l14d 0 2024-03-24T10:46:19.852 INFO:tasks.workunit.client.1.smithi096.stdout:7/880: creat d0/d13/df0/d3c/f117 x:0 0 0 2024-03-24T10:46:19.852 INFO:tasks.workunit.client.1.smithi096.stdout:2/892: mknod d5/dc/d11f/c14e 0 2024-03-24T10:46:19.858 INFO:tasks.workunit.client.1.smithi096.stdout:8/781: dwrite d8/d11/d22/d4c/d65/d1d/fb6 [0,4194304] 0 2024-03-24T10:46:19.858 INFO:tasks.workunit.client.1.smithi096.stdout:6/716: dwrite d8/d12/f1f [4194304,4194304] 0 2024-03-24T10:46:19.860 INFO:tasks.workunit.client.1.smithi096.stdout:7/881: write d0/db/d1c/f91 [5326520,122541] 0 2024-03-24T10:46:19.863 INFO:tasks.workunit.client.1.smithi096.stdout:8/782: stat d8/d11/d22/d4c/db0/cfa 0 2024-03-24T10:46:19.863 INFO:tasks.workunit.client.1.smithi096.stdout:8/783: creat d8/d72/d96/ffb x:0 0 0 2024-03-24T10:46:19.866 INFO:tasks.workunit.client.1.smithi096.stdout:6/717: symlink d8/d12/ldf 0 2024-03-24T10:46:19.867 INFO:tasks.workunit.client.1.smithi096.stdout:7/882: symlink d0/d13/df0/d3c/d68/d85/d10d/l118 0 2024-03-24T10:46:19.870 INFO:tasks.workunit.client.1.smithi096.stdout:6/718: mkdir d8/d10/d76/d91/de0 0 2024-03-24T10:46:19.872 INFO:tasks.workunit.client.1.smithi096.stdout:2/893: dwrite d5/d79/d9f/fba [0,4194304] 0 2024-03-24T10:46:19.872 INFO:tasks.workunit.client.1.smithi096.stdout:2/894: dread - d5/d93/f12e zero size 2024-03-24T10:46:19.872 INFO:tasks.workunit.client.1.smithi096.stdout:7/883: mknod d0/d13/d32/d45/dcd/c119 0 2024-03-24T10:46:19.872 INFO:tasks.workunit.client.1.smithi096.stdout:7/884: write d0/db/d1c/f91 [7792097,8862] 0 2024-03-24T10:46:19.875 INFO:tasks.workunit.client.1.smithi096.stdout:8/784: write d8/fe [3370639,77950] 0 2024-03-24T10:46:19.878 INFO:tasks.workunit.client.1.smithi096.stdout:6/719: read d8/f9 [3256677,129350] 0 2024-03-24T10:46:19.880 INFO:tasks.workunit.client.1.smithi096.stdout:2/895: symlink d5/d38/dea/l14f 0 2024-03-24T10:46:19.880 INFO:tasks.workunit.client.1.smithi096.stdout:2/896: readlink d5/lec 0 2024-03-24T10:46:19.880 INFO:tasks.workunit.client.1.smithi096.stdout:2/897: chown d5/dc/f1c 2570527 1 2024-03-24T10:46:19.880 INFO:tasks.workunit.client.1.smithi096.stdout:2/898: dread - d5/d93/f12e zero size 2024-03-24T10:46:19.890 INFO:tasks.workunit.client.1.smithi096.stdout:8/785: dread d8/d11/d22/d4c/d82/fb2 [0,4194304] 0 2024-03-24T10:46:19.891 INFO:tasks.workunit.client.1.smithi096.stdout:8/786: write d8/d72/f83 [382328,125754] 0 2024-03-24T10:46:19.892 INFO:tasks.workunit.client.1.smithi096.stdout:7/885: dwrite d0/db/d4f/fb7 [0,4194304] 0 2024-03-24T10:46:19.898 INFO:tasks.workunit.client.1.smithi096.stdout:7/886: mknod d0/db/d26/d48/da8/c11a 0 2024-03-24T10:46:19.904 INFO:tasks.workunit.client.1.smithi096.stdout:8/787: dread d8/fc [0,4194304] 0 2024-03-24T10:46:19.905 INFO:tasks.workunit.client.1.smithi096.stdout:6/720: dwrite d8/d12/d32/d38/d59/db8/fb1 [0,4194304] 0 2024-03-24T10:46:19.905 INFO:tasks.workunit.client.1.smithi096.stdout:2/899: dwrite d5/de/d2a/d5e/dd6/f147 [0,4194304] 0 2024-03-24T10:46:19.908 INFO:tasks.workunit.client.1.smithi096.stdout:8/788: mkdir d8/d72/d96/d91/dfc 0 2024-03-24T10:46:19.908 INFO:tasks.workunit.client.1.smithi096.stdout:8/789: chown d8 86 1 2024-03-24T10:46:19.912 INFO:tasks.workunit.client.1.smithi096.stdout:6/721: link d8/d12/d28/d36/d65/f94 d8/d10/d13/dcc/dad/fe1 0 2024-03-24T10:46:19.912 INFO:tasks.workunit.client.1.smithi096.stdout:6/722: creat d8/fe2 x:0 0 0 2024-03-24T10:46:19.913 INFO:tasks.workunit.client.1.smithi096.stdout:6/723: write d8/d12/d32/d68/fcb [226942,27776] 0 2024-03-24T10:46:19.917 INFO:tasks.workunit.client.1.smithi096.stdout:2/900: symlink d5/de/d2a/df3/l150 0 2024-03-24T10:46:19.918 INFO:tasks.workunit.client.1.smithi096.stdout:3/724: sync 2024-03-24T10:46:19.918 INFO:tasks.workunit.client.1.smithi096.stdout:3/725: creat d0/d9b/fec x:0 0 0 2024-03-24T10:46:19.918 INFO:tasks.workunit.client.1.smithi096.stdout:3/726: chown d0/d13/f21 300 1 2024-03-24T10:46:19.918 INFO:tasks.workunit.client.1.smithi096.stdout:3/727: readlink d0/d13/d41/lcb 0 2024-03-24T10:46:19.918 INFO:tasks.workunit.client.1.smithi096.stdout:8/790: creat d8/d11/d22/d4c/d82/da1/dd4/de7/ffd x:0 0 0 2024-03-24T10:46:19.919 INFO:tasks.workunit.client.1.smithi096.stdout:7/887: dwrite d0/da9/ddb/fe7 [0,4194304] 0 2024-03-24T10:46:19.922 INFO:tasks.workunit.client.1.smithi096.stdout:6/724: mknod d8/d10/d76/d91/dd5/ce3 0 2024-03-24T10:46:19.924 INFO:tasks.workunit.client.1.smithi096.stdout:2/901: mknod d5/dc/d12/d23/d105/c151 0 2024-03-24T10:46:19.927 INFO:tasks.workunit.client.1.smithi096.stdout:7/888: dread d0/d13/f35 [0,4194304] 0 2024-03-24T10:46:19.928 INFO:tasks.workunit.client.1.smithi096.stdout:3/728: truncate d0/d13/d45/d7d/f94 2704997 0 2024-03-24T10:46:19.928 INFO:tasks.workunit.client.1.smithi096.stdout:3/729: read - d0/d13/d41/ddf/fd4 zero size 2024-03-24T10:46:19.929 INFO:tasks.workunit.client.1.smithi096.stdout:3/730: write d0/fe9 [56799,36355] 0 2024-03-24T10:46:19.929 INFO:tasks.workunit.client.1.smithi096.stdout:8/791: stat d8/d11/d22/d4c/l4f 0 2024-03-24T10:46:19.929 INFO:tasks.workunit.client.1.smithi096.stdout:8/792: write d8/fb3 [4606755,114982] 0 2024-03-24T10:46:19.929 INFO:tasks.workunit.client.1.smithi096.stdout:8/793: stat d8/d11/d22/c99 0 2024-03-24T10:46:19.934 INFO:tasks.workunit.client.1.smithi096.stdout:6/725: symlink d8/d12/d28/d36/d65/da4/le4 0 2024-03-24T10:46:19.936 INFO:tasks.workunit.client.1.smithi096.stdout:7/889: mkdir d0/db/d26/d11b 0 2024-03-24T10:46:19.940 INFO:tasks.workunit.client.1.smithi096.stdout:6/726: mkdir d8/d10/d76/d91/dd5/de5 0 2024-03-24T10:46:19.942 INFO:tasks.workunit.client.1.smithi096.stdout:7/890: mkdir d0/d13/df0/d3c/d68/d11c 0 2024-03-24T10:46:19.944 INFO:tasks.workunit.client.1.smithi096.stdout:7/891: symlink d0/d13/l11d 0 2024-03-24T10:46:19.944 INFO:tasks.workunit.client.1.smithi096.stdout:7/892: chown d0/db/d26 37 1 2024-03-24T10:46:19.947 INFO:tasks.workunit.client.1.smithi096.stdout:7/893: creat d0/d13/df0/d59/d87/f11e x:0 0 0 2024-03-24T10:46:19.955 INFO:tasks.workunit.client.1.smithi096.stdout:6/727: write d8/d12/d32/f3c [2886002,96529] 0 2024-03-24T10:46:19.955 INFO:tasks.workunit.client.1.smithi096.stdout:8/794: dwrite d8/d11/d22/d4c/d65/d1d/d3d/f56 [0,4194304] 0 2024-03-24T10:46:19.960 INFO:tasks.workunit.client.1.smithi096.stdout:6/728: creat d8/d10/d76/d91/de0/fe6 x:0 0 0 2024-03-24T10:46:19.964 INFO:tasks.workunit.client.1.smithi096.stdout:8/795: rename d8/d11/d22/d4c/fca to d8/d11/d85/d93/ffe 0 2024-03-24T10:46:19.965 INFO:tasks.workunit.client.1.smithi096.stdout:7/894: dwrite d0/d13/f5 [0,4194304] 0 2024-03-24T10:46:19.967 INFO:tasks.workunit.client.1.smithi096.stdout:8/796: truncate d8/f58 875792 0 2024-03-24T10:46:19.967 INFO:tasks.workunit.client.1.smithi096.stdout:8/797: write d8/d11/d22/d4c/d65/d1a/f26 [1097644,67769] 0 2024-03-24T10:46:19.967 INFO:tasks.workunit.client.1.smithi096.stdout:8/798: creat d8/fff x:0 0 0 2024-03-24T10:46:19.971 INFO:tasks.workunit.client.1.smithi096.stdout:7/895: read d0/d13/f35 [1583945,19451] 0 2024-03-24T10:46:19.971 INFO:tasks.workunit.client.1.smithi096.stdout:7/896: fdatasync d0/db/d4f/f58 0 2024-03-24T10:46:19.971 INFO:tasks.workunit.client.1.smithi096.stdout:7/897: chown d0/d13/df0/d59/fd6 1250882 1 2024-03-24T10:46:19.981 INFO:tasks.workunit.client.1.smithi096.stdout:7/898: write d0/db/d26/d48/f8d [5555997,94211] 0 2024-03-24T10:46:19.985 INFO:tasks.workunit.client.1.smithi096.stdout:7/899: dread d0/db/d1c/d77/dab/fba [0,4194304] 0 2024-03-24T10:46:19.985 INFO:tasks.workunit.client.1.smithi096.stdout:7/900: write d0/db/d26/dc3/f93 [53499,31149] 0 2024-03-24T10:46:19.986 INFO:tasks.workunit.client.1.smithi096.stdout:8/799: dwrite d8/d11/d22/d4c/d65/d1a/da0/feb [0,4194304] 0 2024-03-24T10:46:19.989 INFO:tasks.workunit.client.1.smithi096.stdout:6/729: dwrite d8/d12/d32/f3c [0,4194304] 0 2024-03-24T10:46:19.989 INFO:tasks.workunit.client.1.smithi096.stdout:6/730: write d8/d10/d76/fc3 [566454,101179] 0 2024-03-24T10:46:19.990 INFO:tasks.workunit.client.1.smithi096.stdout:7/901: link d0/d13/d32/d99/fdf d0/d13/d32/d45/f11f 0 2024-03-24T10:46:19.991 INFO:tasks.workunit.client.1.smithi096.stdout:8/800: symlink d8/d11/d22/d4c/d82/da1/dd4/l100 0 2024-03-24T10:46:19.993 INFO:tasks.workunit.client.1.smithi096.stdout:6/731: symlink d8/d12/d61/le7 0 2024-03-24T10:46:19.995 INFO:tasks.workunit.client.1.smithi096.stdout:7/902: mkdir d0/d13/d32/d120 0 2024-03-24T10:46:19.999 INFO:tasks.workunit.client.1.smithi096.stdout:8/801: symlink d8/d11/d22/ded/l101 0 2024-03-24T10:46:19.999 INFO:tasks.workunit.client.1.smithi096.stdout:8/802: write d8/d11/d22/d4c/d65/d1a/da0/fd3 [756209,118093] 0 2024-03-24T10:46:20.000 INFO:tasks.workunit.client.1.smithi096.stdout:6/732: unlink d8/d12/d61/le7 0 2024-03-24T10:46:20.002 INFO:tasks.workunit.client.1.smithi096.stdout:8/803: creat d8/d11/d44/d45/f102 x:0 0 0 2024-03-24T10:46:20.004 INFO:tasks.workunit.client.1.smithi096.stdout:8/804: truncate d8/d72/d96/d91/fd8 810510 0 2024-03-24T10:46:20.004 INFO:tasks.workunit.client.1.smithi096.stdout:8/805: write d8/d11/f27 [3854781,119759] 0 2024-03-24T10:46:20.004 INFO:tasks.workunit.client.1.smithi096.stdout:8/806: fdatasync d8/d11/d22/d4c/faf 0 2024-03-24T10:46:20.004 INFO:tasks.workunit.client.1.smithi096.stdout:8/807: stat d8/d11/d22/d4c/d82/f70 0 2024-03-24T10:46:20.006 INFO:tasks.workunit.client.1.smithi096.stdout:8/808: mknod d8/d11/d85/c103 0 2024-03-24T10:46:20.012 INFO:tasks.workunit.client.1.smithi096.stdout:6/733: dread d8/d7b/fab [0,4194304] 0 2024-03-24T10:46:20.014 INFO:tasks.workunit.client.1.smithi096.stdout:6/734: creat d8/d10/d58/fe8 x:0 0 0 2024-03-24T10:46:20.016 INFO:tasks.workunit.client.1.smithi096.stdout:6/735: rmdir d8/d12/d28/d36/d65/d6b 39 2024-03-24T10:46:20.019 INFO:tasks.workunit.client.1.smithi096.stdout:6/736: unlink d8/d12/d28/d36/d65/d71/dd0/da8/dbe/fbd 0 2024-03-24T10:46:20.019 INFO:tasks.workunit.client.1.smithi096.stdout:6/737: truncate d8/d12/d61/faa 7762 0 2024-03-24T10:46:20.023 INFO:tasks.workunit.client.1.smithi096.stdout:7/903: dwrite d0/d13/d32/d99/fdf [0,4194304] 0 2024-03-24T10:46:20.023 INFO:tasks.workunit.client.1.smithi096.stdout:7/904: readlink d0/d13/df0/d59/l106 0 2024-03-24T10:46:20.025 INFO:tasks.workunit.client.1.smithi096.stdout:6/738: unlink d8/d10/d58/f72 0 2024-03-24T10:46:20.025 INFO:tasks.workunit.client.1.smithi096.stdout:6/739: dread - d8/f81 zero size 2024-03-24T10:46:20.036 INFO:tasks.workunit.client.1.smithi096.stdout:8/809: dwrite d8/d11/d22/d4c/d65/d52/dc0/dd9/fe5 [0,4194304] 0 2024-03-24T10:46:20.043 INFO:tasks.workunit.client.1.smithi096.stdout:8/810: mkdir d8/d11/d22/d4c/d65/d1a/d46/d2e/d104 0 2024-03-24T10:46:20.046 INFO:tasks.workunit.client.1.smithi096.stdout:7/905: dwrite d0/d13/df0/d59/d87/f11e [0,4194304] 0 2024-03-24T10:46:20.047 INFO:tasks.workunit.client.1.smithi096.stdout:8/811: unlink d8/d11/d44/d45/l79 0 2024-03-24T10:46:20.049 INFO:tasks.workunit.client.1.smithi096.stdout:1/615: sync 2024-03-24T10:46:20.051 INFO:tasks.workunit.client.1.smithi096.stdout:6/740: dwrite d8/d10/f8a [0,4194304] 0 2024-03-24T10:46:20.054 INFO:tasks.workunit.client.1.smithi096.stdout:7/906: getdents d0/db/d1c/d77/dab 0 2024-03-24T10:46:20.054 INFO:tasks.workunit.client.1.smithi096.stdout:7/907: chown d0/f38 2396839 1 2024-03-24T10:46:20.054 INFO:tasks.workunit.client.1.smithi096.stdout:8/812: dread d8/d11/d22/d4c/d65/d1a/da0/fbd [0,4194304] 0 2024-03-24T10:46:20.056 INFO:tasks.workunit.client.1.smithi096.stdout:8/813: write d8/f14 [516580,72461] 0 2024-03-24T10:46:20.056 INFO:tasks.workunit.client.1.smithi096.stdout:8/814: write d8/d72/d96/ff5 [186404,35527] 0 2024-03-24T10:46:20.056 INFO:tasks.workunit.client.1.smithi096.stdout:8/815: chown d8/d11/d44/d45/cc3 25916 1 2024-03-24T10:46:20.056 INFO:tasks.workunit.client.1.smithi096.stdout:8/816: fsync d8/d11/d22/d4c/d65/d37/f6b 0 2024-03-24T10:46:20.056 INFO:tasks.workunit.client.1.smithi096.stdout:8/817: readlink l6 0 2024-03-24T10:46:20.059 INFO:tasks.workunit.client.1.smithi096.stdout:6/741: rename d8/d12/d28/d36/d65/l9d to d8/d12/d32/d38/d59/db8/le9 0 2024-03-24T10:46:20.060 INFO:tasks.workunit.client.1.smithi096.stdout:8/818: dread d8/d11/d22/d4c/d67/f7a [0,4194304] 0 2024-03-24T10:46:20.060 INFO:tasks.workunit.client.1.smithi096.stdout:8/819: fsync d8/d11/d22/d4c/d65/d1d/f3b 0 2024-03-24T10:46:20.061 INFO:tasks.workunit.client.1.smithi096.stdout:7/908: symlink d0/db/d26/d48/da8/db1/l121 0 2024-03-24T10:46:20.067 INFO:tasks.workunit.client.1.smithi096.stdout:6/742: mknod d8/d10/d76/d91/cea 0 2024-03-24T10:46:20.067 INFO:tasks.workunit.client.1.smithi096.stdout:6/743: creat d8/d10/d58/feb x:0 0 0 2024-03-24T10:46:20.067 INFO:tasks.workunit.client.1.smithi096.stdout:6/744: write d8/d12/d28/d36/d65/d71/dd0/fba [4680643,114872] 0 2024-03-24T10:46:20.072 INFO:tasks.workunit.client.1.smithi096.stdout:1/616: dwrite d3/db2/fc0 [0,4194304] 0 2024-03-24T10:46:20.081 INFO:tasks.workunit.client.1.smithi096.stdout:6/745: mkdir d8/d12/d28/d36/d65/d6b/dec 0 2024-03-24T10:46:20.091 INFO:tasks.workunit.client.1.smithi096.stdout:1/617: dread d3/d23/d2f/f59 [0,4194304] 0 2024-03-24T10:46:20.091 INFO:tasks.workunit.client.1.smithi096.stdout:1/618: dread - d3/d23/d8b/d8c/fcc zero size 2024-03-24T10:46:20.091 INFO:tasks.workunit.client.1.smithi096.stdout:1/619: truncate d3/d23/d8b/d8c/fcc 541147 0 2024-03-24T10:46:20.091 INFO:tasks.workunit.client.1.smithi096.stdout:1/620: creat d3/d36/fd5 x:0 0 0 2024-03-24T10:46:20.091 INFO:tasks.workunit.client.1.smithi096.stdout:1/621: chown d3/d23/d8b/d99 66823 1 2024-03-24T10:46:20.094 INFO:tasks.workunit.client.1.smithi096.stdout:6/746: dread f7 [0,4194304] 0 2024-03-24T10:46:20.096 INFO:tasks.workunit.client.1.smithi096.stdout:8/820: dwrite d8/d11/d22/d4c/d65/d1a/da0/fbd [0,4194304] 0 2024-03-24T10:46:20.100 INFO:tasks.workunit.client.1.smithi096.stdout:7/909: dwrite d0/d13/df0/f27 [0,4194304] 0 2024-03-24T10:46:20.101 INFO:tasks.workunit.client.1.smithi096.stdout:1/622: write d3/d23/d8b/d79/fd1 [1879351,52968] 0 2024-03-24T10:46:20.101 INFO:tasks.workunit.client.1.smithi096.stdout:1/623: chown d3/d23/d2f/d65/d7c/cd0 47416434 1 2024-03-24T10:46:20.108 INFO:tasks.workunit.client.1.smithi096.stdout:7/910: dread d0/db/f75 [0,4194304] 0 2024-03-24T10:46:20.108 INFO:tasks.workunit.client.1.smithi096.stdout:7/911: creat d0/f122 x:0 0 0 2024-03-24T10:46:20.116 INFO:tasks.workunit.client.1.smithi096.stdout:6/747: dwrite d8/fe2 [0,4194304] 0 2024-03-24T10:46:20.122 INFO:tasks.workunit.client.1.smithi096.stdout:1/624: stat d3/d23/d8b/d5c/f75 0 2024-03-24T10:46:20.122 INFO:tasks.workunit.client.1.smithi096.stdout:1/625: creat d3/d23/d8b/d6a/fd6 x:0 0 0 2024-03-24T10:46:20.122 INFO:tasks.workunit.client.1.smithi096.stdout:8/821: dwrite d8/d11/d22/d4c/d65/d1d/d3d/fce [0,4194304] 0 2024-03-24T10:46:20.123 INFO:tasks.workunit.client.1.smithi096.stdout:8/822: write d8/fe [3335149,127143] 0 2024-03-24T10:46:20.123 INFO:tasks.workunit.client.1.smithi096.stdout:8/823: write d8/d11/d85/fda [1152502,12980] 0 2024-03-24T10:46:20.123 INFO:tasks.workunit.client.1.smithi096.stdout:8/824: creat d8/d72/f105 x:0 0 0 2024-03-24T10:46:20.123 INFO:tasks.workunit.client.1.smithi096.stdout:8/825: chown d8/d11/d22/d4c/d65/d1a/d49/f6a 54893 1 2024-03-24T10:46:20.127 INFO:tasks.workunit.client.1.smithi096.stdout:7/912: creat d0/db/d26/d48/dd4/f123 x:0 0 0 2024-03-24T10:46:20.130 INFO:tasks.workunit.client.1.smithi096.stdout:6/748: creat d8/d12/d28/d36/d65/d6b/dec/fed x:0 0 0 2024-03-24T10:46:20.133 INFO:tasks.workunit.client.1.smithi096.stdout:1/626: creat d3/d84/fd7 x:0 0 0 2024-03-24T10:46:20.145 INFO:tasks.workunit.client.1.smithi096.stdout:8/826: link d8/d72/d96/la4 d8/d11/d22/d4c/d65/d52/dc0/dd9/l106 0 2024-03-24T10:46:20.146 INFO:tasks.workunit.client.1.smithi096.stdout:8/827: read d8/d11/d22/d4c/d65/d1a/f6c [469086,595] 0 2024-03-24T10:46:20.146 INFO:tasks.workunit.client.1.smithi096.stdout:8/828: dread - d8/d11/fe9 zero size 2024-03-24T10:46:20.147 INFO:tasks.workunit.client.1.smithi096.stdout:7/913: mknod d0/d13/df0/d3c/d68/d101/c124 0 2024-03-24T10:46:20.147 INFO:tasks.workunit.client.1.smithi096.stdout:6/749: dwrite d8/d12/d32/fc4 [0,4194304] 0 2024-03-24T10:46:20.147 INFO:tasks.workunit.client.1.smithi096.stdout:6/750: creat d8/d12/d32/fee x:0 0 0 2024-03-24T10:46:20.148 INFO:tasks.workunit.client.1.smithi096.stdout:1/627: link d3/d73/fcf d3/fd8 0 2024-03-24T10:46:20.148 INFO:tasks.workunit.client.1.smithi096.stdout:1/628: write d3/d23/d8b/d6a/fad [548690,95223] 0 2024-03-24T10:46:20.149 INFO:tasks.workunit.client.1.smithi096.stdout:1/629: dread d3/d23/fb7 [0,4194304] 0 2024-03-24T10:46:20.149 INFO:tasks.workunit.client.1.smithi096.stdout:1/630: chown d3/d23/d2f/f37 72260 1 2024-03-24T10:46:20.150 INFO:tasks.workunit.client.1.smithi096.stdout:8/829: rename d8/d11/d22/d4c/db0/lc6 to d8/d11/d22/d4c/d65/d37/df2/l107 0 2024-03-24T10:46:20.151 INFO:tasks.workunit.client.1.smithi096.stdout:8/830: creat d8/d11/d22/d4c/d82/da1/f108 x:0 0 0 2024-03-24T10:46:20.151 INFO:tasks.workunit.client.1.smithi096.stdout:8/831: creat d8/d72/d96/f109 x:0 0 0 2024-03-24T10:46:20.151 INFO:tasks.workunit.client.1.smithi096.stdout:7/914: mkdir d0/db/d4f/d125 0 2024-03-24T10:46:20.155 INFO:tasks.workunit.client.1.smithi096.stdout:6/751: mknod d8/d10/d76/dd6/cef 0 2024-03-24T10:46:20.155 INFO:tasks.workunit.client.1.smithi096.stdout:6/752: chown d8/d10/d13/dd8 50592951 1 2024-03-24T10:46:20.155 INFO:tasks.workunit.client.1.smithi096.stdout:0/791: sync 2024-03-24T10:46:20.156 INFO:tasks.workunit.client.1.smithi096.stdout:5/780: sync 2024-03-24T10:46:20.156 INFO:tasks.workunit.client.1.smithi096.stdout:9/780: sync 2024-03-24T10:46:20.156 INFO:tasks.workunit.client.1.smithi096.stdout:0/792: creat d0/d11/d43/d75/f10c x:0 0 0 2024-03-24T10:46:20.157 INFO:tasks.workunit.client.1.smithi096.stdout:1/631: mkdir d3/d23/d2f/d65/dd9 0 2024-03-24T10:46:20.158 INFO:tasks.workunit.client.1.smithi096.stdout:1/632: dread - d3/d23/d8b/fcd zero size 2024-03-24T10:46:20.158 INFO:tasks.workunit.client.1.smithi096.stdout:1/633: truncate d3/d23/d2f/f7e 1028996 0 2024-03-24T10:46:20.158 INFO:tasks.workunit.client.1.smithi096.stdout:1/634: creat d3/d23/d8b/d5c/d9c/fda x:0 0 0 2024-03-24T10:46:20.161 INFO:tasks.workunit.client.1.smithi096.stdout:7/915: dread d0/f14 [0,4194304] 0 2024-03-24T10:46:20.163 INFO:tasks.workunit.client.1.smithi096.stdout:8/832: truncate d8/d11/d22/d4c/d65/d1a/d46/d2e/f54 288669 0 2024-03-24T10:46:20.164 INFO:tasks.workunit.client.1.smithi096.stdout:3/731: sync 2024-03-24T10:46:20.164 INFO:tasks.workunit.client.1.smithi096.stdout:2/902: sync 2024-03-24T10:46:20.164 INFO:tasks.workunit.client.1.smithi096.stdout:4/657: sync 2024-03-24T10:46:20.166 INFO:tasks.workunit.client.1.smithi096.stdout:6/753: dread d8/d10/d13/d63/f5e [0,4194304] 0 2024-03-24T10:46:20.167 INFO:tasks.workunit.client.1.smithi096.stdout:0/793: mknod d0/d11/d20/d56/d109/c10d 0 2024-03-24T10:46:20.167 INFO:tasks.workunit.client.1.smithi096.stdout:0/794: write d0/d11/d24/f68 [468628,45981] 0 2024-03-24T10:46:20.170 INFO:tasks.workunit.client.1.smithi096.stdout:1/635: creat d3/d84/d9a/dd2/fdb x:0 0 0 2024-03-24T10:46:20.170 INFO:tasks.workunit.client.1.smithi096.stdout:1/636: truncate d3/d23/d2f/f60 834111 0 2024-03-24T10:46:20.176 INFO:tasks.workunit.client.1.smithi096.stdout:4/658: dread d1/d11/f41 [0,4194304] 0 2024-03-24T10:46:20.176 INFO:tasks.workunit.client.1.smithi096.stdout:4/659: readlink d1/l78 0 2024-03-24T10:46:20.178 INFO:tasks.workunit.client.1.smithi096.stdout:8/833: mkdir d8/d11/d22/d4c/d65/d52/dc0/dd9/d10a 0 2024-03-24T10:46:20.180 INFO:tasks.workunit.client.1.smithi096.stdout:9/781: dwrite d9/d15/d26/f70 [0,4194304] 0 2024-03-24T10:46:20.180 INFO:tasks.workunit.client.1.smithi096.stdout:9/782: creat d9/d15/d26/d29/d73/ffe x:0 0 0 2024-03-24T10:46:20.180 INFO:tasks.workunit.client.1.smithi096.stdout:3/732: dread d0/d9/f1a [0,4194304] 0 2024-03-24T10:46:20.180 INFO:tasks.workunit.client.1.smithi096.stdout:2/903: unlink d5/dc/d6b/d72/d11a/f125 0 2024-03-24T10:46:20.180 INFO:tasks.workunit.client.1.smithi096.stdout:2/904: creat d5/dc/d6b/def/f152 x:0 0 0 2024-03-24T10:46:20.181 INFO:tasks.workunit.client.1.smithi096.stdout:7/916: dwrite d0/d13/df0/d3c/f71 [0,4194304] 0 2024-03-24T10:46:20.188 INFO:tasks.workunit.client.1.smithi096.stdout:6/754: mkdir d8/d10/d13/dcc/dad/df0 0 2024-03-24T10:46:20.190 INFO:tasks.workunit.client.1.smithi096.stdout:3/733: dread d0/d13/d45/d47/f89 [0,4194304] 0 2024-03-24T10:46:20.190 INFO:tasks.workunit.client.1.smithi096.stdout:3/734: truncate d0/d9/f3b 622504 0 2024-03-24T10:46:20.190 INFO:tasks.workunit.client.1.smithi096.stdout:3/735: readlink d0/d13/d45/d7d/d39/d48/d85/lcf 0 2024-03-24T10:46:20.190 INFO:tasks.workunit.client.1.smithi096.stdout:3/736: write d0/d13/d45/d7d/d39/d48/d8d/fe0 [56112,93912] 0 2024-03-24T10:46:20.191 INFO:tasks.workunit.client.1.smithi096.stdout:3/737: dread d0/d13/d45/d52/fc1 [0,4194304] 0 2024-03-24T10:46:20.191 INFO:tasks.workunit.client.1.smithi096.stdout:3/738: chown d0/d13/d45/d52/f5b 96437 1 2024-03-24T10:46:20.192 INFO:tasks.workunit.client.1.smithi096.stdout:1/637: dwrite d3/d23/d2f/f59 [0,4194304] 0 2024-03-24T10:46:20.193 INFO:tasks.workunit.client.1.smithi096.stdout:4/660: link d1/d32/d42/fb0 d1/d32/d4c/d7e/fd4 0 2024-03-24T10:46:20.196 INFO:tasks.workunit.client.1.smithi096.stdout:9/783: creat d9/d15/d26/d29/dcd/fff x:0 0 0 2024-03-24T10:46:20.198 INFO:tasks.workunit.client.1.smithi096.stdout:6/755: write d8/d12/f5c [2784398,27092] 0 2024-03-24T10:46:20.202 INFO:tasks.workunit.client.1.smithi096.stdout:0/795: dread d0/f73 [0,4194304] 0 2024-03-24T10:46:20.202 INFO:tasks.workunit.client.1.smithi096.stdout:6/756: dread d8/d12/d61/f93 [0,4194304] 0 2024-03-24T10:46:20.202 INFO:tasks.workunit.client.1.smithi096.stdout:6/757: write d8/d12/d32/d38/d59/f79 [4894834,7560] 0 2024-03-24T10:46:20.203 INFO:tasks.workunit.client.1.smithi096.stdout:3/739: rename d0/d13/d41/c42 to d0/d13/d45/d7d/d39/d48/dbd/d9c/ced 0 2024-03-24T10:46:20.203 INFO:tasks.workunit.client.1.smithi096.stdout:3/740: write d0/d13/d45/d7d/d39/fd6 [467338,79863] 0 2024-03-24T10:46:20.203 INFO:tasks.workunit.client.1.smithi096.stdout:6/758: dread d8/d10/d58/f85 [0,4194304] 0 2024-03-24T10:46:20.204 INFO:tasks.workunit.client.1.smithi096.stdout:8/834: dwrite d8/d11/d22/d4c/d82/f8c [0,4194304] 0 2024-03-24T10:46:20.204 INFO:tasks.workunit.client.1.smithi096.stdout:2/905: dwrite d5/de/fe2 [0,4194304] 0 2024-03-24T10:46:20.204 INFO:tasks.workunit.client.1.smithi096.stdout:2/906: chown d5/de/d2a/d5e/fb6 4218 1 2024-03-24T10:46:20.205 INFO:tasks.workunit.client.1.smithi096.stdout:2/907: write d5/d38/f7b [234988,21645] 0 2024-03-24T10:46:20.205 INFO:tasks.workunit.client.1.smithi096.stdout:2/908: dread - d5/de/d2a/df3/f118 zero size 2024-03-24T10:46:20.205 INFO:tasks.workunit.client.1.smithi096.stdout:1/638: mkdir d3/d23/d8b/d8c/ddc 0 2024-03-24T10:46:20.206 INFO:tasks.workunit.client.1.smithi096.stdout:1/639: chown d3/d73/fcf 438 1 2024-03-24T10:46:20.208 INFO:tasks.workunit.client.1.smithi096.stdout:9/784: truncate d9/d15/d26/d29/d73/f4b 3864440 0 2024-03-24T10:46:20.210 INFO:tasks.workunit.client.1.smithi096.stdout:1/640: dread d3/d23/d8b/d79/fd1 [0,4194304] 0 2024-03-24T10:46:20.212 INFO:tasks.workunit.client.1.smithi096.stdout:0/796: symlink d0/d11/d20/d2b/l10e 0 2024-03-24T10:46:20.215 INFO:tasks.workunit.client.1.smithi096.stdout:6/759: write d8/d12/d32/fc4 [880658,22835] 0 2024-03-24T10:46:20.215 INFO:tasks.workunit.client.1.smithi096.stdout:6/760: readlink d8/d12/d28/d36/d65/d71/l7a 0 2024-03-24T10:46:20.216 INFO:tasks.workunit.client.1.smithi096.stdout:6/761: dread - d8/fdc zero size 2024-03-24T10:46:20.216 INFO:tasks.workunit.client.1.smithi096.stdout:6/762: truncate d8/d12/d32/d38/d59/f79 5852261 0 2024-03-24T10:46:20.216 INFO:tasks.workunit.client.1.smithi096.stdout:3/741: symlink d0/d13/d45/d7d/d39/d48/lee 0 2024-03-24T10:46:20.216 INFO:tasks.workunit.client.1.smithi096.stdout:3/742: readlink d0/d13/d45/d7d/d39/d48/dbd/lb8 0 2024-03-24T10:46:20.220 INFO:tasks.workunit.client.1.smithi096.stdout:8/835: creat d8/d11/d22/d4c/d65/d1a/d46/f10b x:0 0 0 2024-03-24T10:46:20.220 INFO:tasks.workunit.client.1.smithi096.stdout:8/836: chown d8/d11/f27 53 1 2024-03-24T10:46:20.223 INFO:tasks.workunit.client.1.smithi096.stdout:2/909: creat d5/de/f153 x:0 0 0 2024-03-24T10:46:20.223 INFO:tasks.workunit.client.1.smithi096.stdout:2/910: getdents d5/dc/d144 0 2024-03-24T10:46:20.225 INFO:tasks.workunit.client.1.smithi096.stdout:0/797: creat d0/d11/d24/f10f x:0 0 0 2024-03-24T10:46:20.228 INFO:tasks.workunit.client.1.smithi096.stdout:6/763: link d8/d12/d32/d68/fcb d8/d10/ff1 0 2024-03-24T10:46:20.231 INFO:tasks.workunit.client.1.smithi096.stdout:2/911: getdents d5/d93/d115/d13e 0 2024-03-24T10:46:20.231 INFO:tasks.workunit.client.1.smithi096.stdout:2/912: chown d5/d38/dea/l130 8465373 1 2024-03-24T10:46:20.232 INFO:tasks.workunit.client.1.smithi096.stdout:9/785: getdents d9/d15/d26/d29/d73/d49/d86 0 2024-03-24T10:46:20.232 INFO:tasks.workunit.client.1.smithi096.stdout:9/786: write d9/d15/d26/d29/d73/f5a [684468,100789] 0 2024-03-24T10:46:20.232 INFO:tasks.workunit.client.1.smithi096.stdout:9/787: truncate d9/d15/d26/d29/dcd/fff 356528 0 2024-03-24T10:46:20.234 INFO:tasks.workunit.client.1.smithi096.stdout:0/798: mkdir d0/d11/d20/d9e/d80/d110 0 2024-03-24T10:46:20.235 INFO:tasks.workunit.client.1.smithi096.stdout:6/764: rmdir d8/d12/d28 39 2024-03-24T10:46:20.236 INFO:tasks.workunit.client.1.smithi096.stdout:6/765: chown d8/d12/d28/d36/d65/d71/dd0/da8 1034107391 1 2024-03-24T10:46:20.238 INFO:tasks.workunit.client.1.smithi096.stdout:0/799: rename d0/d11/d20/d9e/fe0 to d0/d11/d20/f111 0 2024-03-24T10:46:20.239 INFO:tasks.workunit.client.1.smithi096.stdout:0/800: creat d0/d11/d16/d21/d78/d92/f112 x:0 0 0 2024-03-24T10:46:20.241 INFO:tasks.workunit.client.1.smithi096.stdout:6/766: unlink d8/d10/d13/d63/f5e 0 2024-03-24T10:46:20.242 INFO:tasks.workunit.client.1.smithi096.stdout:2/913: mknod d5/d38/c154 0 2024-03-24T10:46:20.245 INFO:tasks.workunit.client.1.smithi096.stdout:0/801: mknod d0/d11/d43/d75/d104/c113 0 2024-03-24T10:46:20.246 INFO:tasks.workunit.client.1.smithi096.stdout:6/767: truncate d8/f62 5021437 0 2024-03-24T10:46:20.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:20 smithi094 ceph-mon[29446]: pgmap v253: 65 pgs: 65 active+clean; 2.7 GiB data, 9.8 GiB used, 527 GiB / 536 GiB avail; 165 MiB/s rd, 198 MiB/s wr, 373 op/s 2024-03-24T10:46:20.249 INFO:tasks.workunit.client.1.smithi096.stdout:2/914: rmdir d5/d38/dbd/ddb/d102 39 2024-03-24T10:46:20.249 INFO:tasks.workunit.client.1.smithi096.stdout:2/915: dread - d5/f116 zero size 2024-03-24T10:46:20.249 INFO:tasks.workunit.client.1.smithi096.stdout:2/916: read d5/dc/d12/f4a [2325977,36562] 0 2024-03-24T10:46:20.251 INFO:tasks.workunit.client.1.smithi096.stdout:0/802: symlink d0/d11/d16/d21/d81/d9c/l114 0 2024-03-24T10:46:20.251 INFO:tasks.workunit.client.1.smithi096.stdout:0/803: truncate d0/d11/d43/d75/f10c 171235 0 2024-03-24T10:46:20.251 INFO:tasks.workunit.client.1.smithi096.stdout:0/804: dread - d0/d11/d24/f10f zero size 2024-03-24T10:46:20.253 INFO:tasks.workunit.client.1.smithi096.stdout:0/805: unlink d0/d11/d20/d91/fa0 0 2024-03-24T10:46:20.262 INFO:tasks.workunit.client.1.smithi096.stdout:6/768: dwrite d8/d10/f3e [0,4194304] 0 2024-03-24T10:46:20.265 INFO:tasks.workunit.client.1.smithi096.stdout:6/769: mkdir d8/d12/d28/d36/d65/d6b/da6/db2/df2 0 2024-03-24T10:46:20.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:20 smithi096 ceph-mon[38097]: pgmap v253: 65 pgs: 65 active+clean; 2.7 GiB data, 9.8 GiB used, 527 GiB / 536 GiB avail; 165 MiB/s rd, 198 MiB/s wr, 373 op/s 2024-03-24T10:46:20.335 INFO:tasks.workunit.client.1.smithi096.stdout:8/837: write d8/d11/d22/d4c/db0/fc4 [278828,118779] 0 2024-03-24T10:46:20.335 INFO:tasks.workunit.client.1.smithi096.stdout:1/641: dwrite d3/d23/d8b/f46 [0,4194304] 0 2024-03-24T10:46:20.335 INFO:tasks.workunit.client.1.smithi096.stdout:1/642: chown d3/d36/c43 29254468 1 2024-03-24T10:46:20.336 INFO:tasks.workunit.client.1.smithi096.stdout:1/643: chown d3/d23/d2f 26982 1 2024-03-24T10:46:20.336 INFO:tasks.workunit.client.1.smithi096.stdout:1/644: creat d3/d23/d71/dbe/fdd x:0 0 0 2024-03-24T10:46:20.343 INFO:tasks.workunit.client.1.smithi096.stdout:1/645: creat d3/d23/d8b/d8c/ddc/fde x:0 0 0 2024-03-24T10:46:20.343 INFO:tasks.workunit.client.1.smithi096.stdout:1/646: read d3/f66 [192540,86399] 0 2024-03-24T10:46:20.344 INFO:tasks.workunit.client.1.smithi096.stdout:1/647: mkdir d3/d23/d2f/d65/d7c/ddf 0 2024-03-24T10:46:20.346 INFO:tasks.workunit.client.1.smithi096.stdout:1/648: creat d3/d36/dd4/fe0 x:0 0 0 2024-03-24T10:46:20.348 INFO:tasks.workunit.client.1.smithi096.stdout:1/649: getdents d3/d84/d9a 0 2024-03-24T10:46:20.350 INFO:tasks.workunit.client.1.smithi096.stdout:1/650: mkdir d3/d23/d71/d7b/dce/de1 0 2024-03-24T10:46:20.363 INFO:tasks.workunit.client.1.smithi096.stdout:0/806: dwrite d0/d11/d20/d56/fb2 [0,4194304] 0 2024-03-24T10:46:20.363 INFO:tasks.workunit.client.1.smithi096.stdout:6/770: dwrite d8/d12/d28/d36/f7d [0,4194304] 0 2024-03-24T10:46:20.367 INFO:tasks.workunit.client.1.smithi096.stdout:0/807: dread d0/d11/d20/d9e/d80/f41 [0,4194304] 0 2024-03-24T10:46:20.367 INFO:tasks.workunit.client.1.smithi096.stdout:0/808: creat d0/d11/d20/d2b/f115 x:0 0 0 2024-03-24T10:46:20.367 INFO:tasks.workunit.client.1.smithi096.stdout:1/651: write d3/d9b/fa4 [3523814,581] 0 2024-03-24T10:46:20.368 INFO:tasks.workunit.client.1.smithi096.stdout:1/652: chown d3/d23/d8b/d5c/d6d/fac 3 1 2024-03-24T10:46:20.368 INFO:tasks.workunit.client.1.smithi096.stdout:9/788: dwrite d9/d15/d26/d29/d73/d43/d48/f98 [0,4194304] 0 2024-03-24T10:46:20.368 INFO:tasks.workunit.client.1.smithi096.stdout:6/771: truncate d8/d12/d28/d36/d65/d71/dd0/fba 2167574 0 2024-03-24T10:46:20.371 INFO:tasks.workunit.client.1.smithi096.stdout:0/809: mknod d0/d11/d20/d9e/c116 0 2024-03-24T10:46:20.371 INFO:tasks.workunit.client.1.smithi096.stdout:0/810: fdatasync d0/d11/d24/f87 0 2024-03-24T10:46:20.374 INFO:tasks.workunit.client.1.smithi096.stdout:1/653: truncate d3/d23/f56 1618485 0 2024-03-24T10:46:20.375 INFO:tasks.workunit.client.1.smithi096.stdout:1/654: write d3/d23/d8b/d8c/fcc [212576,26725] 0 2024-03-24T10:46:20.377 INFO:tasks.workunit.client.1.smithi096.stdout:6/772: dread f2 [0,4194304] 0 2024-03-24T10:46:20.380 INFO:tasks.workunit.client.1.smithi096.stdout:0/811: rename d0/d11/d24/d2c/d84/ld7 to d0/d11/d16/d21/d81/d8f/d9a/l117 0 2024-03-24T10:46:20.382 INFO:tasks.workunit.client.1.smithi096.stdout:6/773: mkdir d8/d10/d13/df3 0 2024-03-24T10:46:20.385 INFO:tasks.workunit.client.1.smithi096.stdout:9/789: mknod d9/d15/d26/d29/d73/d43/c100 0 2024-03-24T10:46:20.385 INFO:tasks.workunit.client.1.smithi096.stdout:9/790: creat d9/d15/d26/d29/d73/f101 x:0 0 0 2024-03-24T10:46:20.394 INFO:tasks.workunit.client.1.smithi096.stdout:1/655: dwrite d3/d23/d8b/d6a/f96 [0,4194304] 0 2024-03-24T10:46:20.399 INFO:tasks.workunit.client.1.smithi096.stdout:9/791: read d9/d15/d26/d29/d73/d49/db7/dc1/fd5 [3271497,6601] 0 2024-03-24T10:46:20.399 INFO:tasks.workunit.client.1.smithi096.stdout:9/792: chown d9/d15/f66 1385640 1 2024-03-24T10:46:20.402 INFO:tasks.workunit.client.1.smithi096.stdout:9/793: dread d9/d15/d26/d29/d73/fdc [0,4194304] 0 2024-03-24T10:46:20.402 INFO:tasks.workunit.client.1.smithi096.stdout:9/794: fdatasync d9/d15/d26/d29/d73/d43/f45 0 2024-03-24T10:46:20.402 INFO:tasks.workunit.client.1.smithi096.stdout:1/656: dread d3/d23/d8b/f46 [0,4194304] 0 2024-03-24T10:46:20.403 INFO:tasks.workunit.client.1.smithi096.stdout:1/657: read d3/d23/f42 [597308,59483] 0 2024-03-24T10:46:20.403 INFO:tasks.workunit.client.1.smithi096.stdout:1/658: truncate d3/d23/d2f/f78 813084 0 2024-03-24T10:46:20.403 INFO:tasks.workunit.client.1.smithi096.stdout:1/659: dread - d3/d36/fc2 zero size 2024-03-24T10:46:20.403 INFO:tasks.workunit.client.1.smithi096.stdout:1/660: write d3/d36/fc2 [1041141,19139] 0 2024-03-24T10:46:20.403 INFO:tasks.workunit.client.1.smithi096.stdout:1/661: creat d3/d23/d8b/d8c/fe2 x:0 0 0 2024-03-24T10:46:20.403 INFO:tasks.workunit.client.1.smithi096.stdout:1/662: chown d3/d23/d8b/d5c/d6d/cab 2585163 1 2024-03-24T10:46:20.404 INFO:tasks.workunit.client.1.smithi096.stdout:1/663: fsync d3/d23/d8b/f3e 0 2024-03-24T10:46:20.409 INFO:tasks.workunit.client.1.smithi096.stdout:0/812: dwrite d0/d11/d20/f111 [0,4194304] 0 2024-03-24T10:46:20.409 INFO:tasks.workunit.client.1.smithi096.stdout:0/813: chown d0/d11/d20/d65/fbb 15663994 1 2024-03-24T10:46:20.412 INFO:tasks.workunit.client.1.smithi096.stdout:9/795: rmdir d9/d15/d26/d29 39 2024-03-24T10:46:20.412 INFO:tasks.workunit.client.1.smithi096.stdout:9/796: truncate d9/d15/d26/d29/d73/d43/d48/f98 4703576 0 2024-03-24T10:46:20.415 INFO:tasks.workunit.client.1.smithi096.stdout:1/664: dread d3/d36/f5f [0,4194304] 0 2024-03-24T10:46:20.425 INFO:tasks.workunit.client.1.smithi096.stdout:9/797: dread d9/d15/fa0 [4194304,4194304] 0 2024-03-24T10:46:20.428 INFO:tasks.workunit.client.1.smithi096.stdout:0/814: dwrite d0/d11/d16/d21/d81/d7f/f64 [0,4194304] 0 2024-03-24T10:46:20.429 INFO:tasks.workunit.client.1.smithi096.stdout:9/798: mkdir d9/d102 0 2024-03-24T10:46:20.429 INFO:tasks.workunit.client.1.smithi096.stdout:9/799: chown d9/d15/d26/f97 849402216 1 2024-03-24T10:46:20.429 INFO:tasks.workunit.client.1.smithi096.stdout:9/800: readlink d9/d15/d26/d29/d73/d43/d48/ld6 0 2024-03-24T10:46:20.431 INFO:tasks.workunit.client.1.smithi096.stdout:9/801: rename d9/cb4 to d9/d15/d26/d29/dcd/c103 0 2024-03-24T10:46:20.431 INFO:tasks.workunit.client.1.smithi096.stdout:9/802: chown d9/d15/d26/d29/d73/c8d 1905 1 2024-03-24T10:46:20.434 INFO:tasks.workunit.client.1.smithi096.stdout:9/803: dread d9/db/d6d/d4f/f63 [0,4194304] 0 2024-03-24T10:46:20.434 INFO:tasks.workunit.client.1.smithi096.stdout:9/804: fsync d9/df1/fde 0 2024-03-24T10:46:20.434 INFO:tasks.workunit.client.1.smithi096.stdout:9/805: fdatasync d9/d15/f3c 0 2024-03-24T10:46:20.434 INFO:tasks.workunit.client.1.smithi096.stdout:0/815: link d0/cc d0/d11/d24/d102/c118 0 2024-03-24T10:46:20.435 INFO:tasks.workunit.client.1.smithi096.stdout:0/816: rmdir d0/d11/d20/d91 39 2024-03-24T10:46:20.435 INFO:tasks.workunit.client.1.smithi096.stdout:0/817: chown d0/d11/d16/d21/d81/d8f/fc6 30705164 1 2024-03-24T10:46:20.444 INFO:tasks.workunit.client.1.smithi096.stdout:1/665: dwrite d3/d23/d8b/d8c/fe2 [0,4194304] 0 2024-03-24T10:46:20.448 INFO:tasks.workunit.client.1.smithi096.stdout:9/806: dread d9/d15/f4a [0,4194304] 0 2024-03-24T10:46:20.448 INFO:tasks.workunit.client.1.smithi096.stdout:0/818: dread d0/d11/d43/fa6 [0,4194304] 0 2024-03-24T10:46:20.459 INFO:tasks.workunit.client.1.smithi096.stdout:0/819: dread d0/f48 [0,4194304] 0 2024-03-24T10:46:20.461 INFO:tasks.workunit.client.1.smithi096.stdout:0/820: link d0/d11/d20/d91/f96 d0/d11/d43/f119 0 2024-03-24T10:46:20.464 INFO:tasks.workunit.client.1.smithi096.stdout:0/821: mkdir d0/d11/d20/d2b/d11a 0 2024-03-24T10:46:20.464 INFO:tasks.workunit.client.1.smithi096.stdout:0/822: readlink d0/d11/d24/d2c/lc4 0 2024-03-24T10:46:20.464 INFO:tasks.workunit.client.1.smithi096.stdout:0/823: chown d0/d11/d24/f55 129675 1 2024-03-24T10:46:20.465 INFO:tasks.workunit.client.1.smithi096.stdout:1/666: dwrite d3/d23/d71/f7d [0,4194304] 0 2024-03-24T10:46:20.469 INFO:tasks.workunit.client.1.smithi096.stdout:1/667: unlink d3/d23/d8b/l9d 0 2024-03-24T10:46:20.487 INFO:tasks.workunit.client.1.smithi096.stdout:1/668: creat d3/d84/fe3 x:0 0 0 2024-03-24T10:46:20.505 INFO:tasks.workunit.client.1.smithi096.stdout:9/807: dwrite d9/d15/d26/d29/f6a [0,4194304] 0 2024-03-24T10:46:20.511 INFO:tasks.workunit.client.1.smithi096.stdout:1/669: dwrite d3/d36/f4d [0,4194304] 0 2024-03-24T10:46:20.511 INFO:tasks.workunit.client.1.smithi096.stdout:1/670: fdatasync d3/d23/d8b/f46 0 2024-03-24T10:46:20.511 INFO:tasks.workunit.client.1.smithi096.stdout:0/824: dwrite d0/d11/d24/d2c/fb3 [0,4194304] 0 2024-03-24T10:46:20.528 INFO:tasks.workunit.client.1.smithi096.stdout:1/671: dwrite d3/d23/d8b/f57 [0,4194304] 0 2024-03-24T10:46:20.528 INFO:tasks.workunit.client.1.smithi096.stdout:1/672: chown d3/d23/d8b/d8c/fe2 28685 1 2024-03-24T10:46:20.529 INFO:tasks.workunit.client.1.smithi096.stdout:1/673: creat d3/d73/fe4 x:0 0 0 2024-03-24T10:46:20.533 INFO:tasks.workunit.client.1.smithi096.stdout:0/825: dwrite d0/d11/d16/d21/d25/f36 [0,4194304] 0 2024-03-24T10:46:20.534 INFO:tasks.workunit.client.1.smithi096.stdout:1/674: link d3/d73/f9f d3/d23/d8b/d5c/fe5 0 2024-03-24T10:46:20.540 INFO:tasks.workunit.client.1.smithi096.stdout:1/675: unlink d3/d84/d9a/dd2/fdb 0 2024-03-24T10:46:20.540 INFO:tasks.workunit.client.1.smithi096.stdout:1/676: creat d3/d23/d71/fe6 x:0 0 0 2024-03-24T10:46:20.540 INFO:tasks.workunit.client.1.smithi096.stdout:1/677: readlink d3/d23/d2f/l5e 0 2024-03-24T10:46:20.540 INFO:tasks.workunit.client.1.smithi096.stdout:1/678: stat d3/dbc 0 2024-03-24T10:46:20.540 INFO:tasks.workunit.client.1.smithi096.stdout:1/679: fdatasync d3/d23/fb7 0 2024-03-24T10:46:20.544 INFO:tasks.workunit.client.1.smithi096.stdout:1/680: unlink d3/d23/d8b/d5c/d9c/lb9 0 2024-03-24T10:46:20.547 INFO:tasks.workunit.client.1.smithi096.stdout:1/681: rename d3/d23/l47 to d3/dbc/le7 0 2024-03-24T10:46:20.548 INFO:tasks.workunit.client.1.smithi096.stdout:0/826: dwrite d0/d11/d43/d75/f10c [0,4194304] 0 2024-03-24T10:46:20.548 INFO:tasks.workunit.client.1.smithi096.stdout:1/682: symlink d3/le8 0 2024-03-24T10:46:20.570 INFO:tasks.workunit.client.1.smithi096.stdout:0/827: dwrite d0/d11/d24/f50 [0,4194304] 0 2024-03-24T10:46:20.573 INFO:tasks.workunit.client.1.smithi096.stdout:1/683: dwrite d3/d23/fb7 [0,4194304] 0 2024-03-24T10:46:20.573 INFO:tasks.workunit.client.1.smithi096.stdout:1/684: fsync d3/ff 0 2024-03-24T10:46:20.586 INFO:tasks.workunit.client.1.smithi096.stdout:1/685: dwrite d3/fd8 [0,4194304] 0 2024-03-24T10:46:20.594 INFO:tasks.workunit.client.1.smithi096.stdout:1/686: dread d3/d23/d2f/f7e [0,4194304] 0 2024-03-24T10:46:20.594 INFO:tasks.workunit.client.1.smithi096.stdout:1/687: read - d3/d84/fe3 zero size 2024-03-24T10:46:20.594 INFO:tasks.workunit.client.1.smithi096.stdout:0/828: dwrite d0/d11/d16/d21/d81/d9c/fa2 [0,4194304] 0 2024-03-24T10:46:20.600 INFO:tasks.workunit.client.1.smithi096.stdout:1/688: rename d3/d23/d2f/c2e to d3/d23/d71/d7b/dce/de1/ce9 0 2024-03-24T10:46:20.600 INFO:tasks.workunit.client.1.smithi096.stdout:1/689: write d3/d36/fd5 [1006726,78639] 0 2024-03-24T10:46:20.601 INFO:tasks.workunit.client.1.smithi096.stdout:1/690: truncate d3/d23/d71/d7b/faa 476459 0 2024-03-24T10:46:20.603 INFO:tasks.workunit.client.1.smithi096.stdout:0/829: mknod d0/d11/d24/c11b 0 2024-03-24T10:46:20.612 INFO:tasks.workunit.client.1.smithi096.stdout:1/691: creat d3/d23/d8b/d99/fea x:0 0 0 2024-03-24T10:46:20.620 INFO:tasks.workunit.client.1.smithi096.stdout:0/830: dwrite d0/d11/d16/d21/d25/f36 [0,4194304] 0 2024-03-24T10:46:20.622 INFO:tasks.workunit.client.1.smithi096.stdout:1/692: dread d3/d23/d8b/d99/fa6 [0,4194304] 0 2024-03-24T10:46:20.626 INFO:tasks.workunit.client.1.smithi096.stdout:0/831: link d0/d11/d43/lb5 d0/d11/d24/d2c/d84/de8/l11c 0 2024-03-24T10:46:20.627 INFO:tasks.workunit.client.1.smithi096.stdout:1/693: mknod d3/d84/d9a/ceb 0 2024-03-24T10:46:20.627 INFO:tasks.workunit.client.1.smithi096.stdout:1/694: fdatasync d3/d23/d71/dbe/fc9 0 2024-03-24T10:46:20.627 INFO:tasks.workunit.client.1.smithi096.stdout:1/695: creat d3/d23/d8b/d5c/d9c/fec x:0 0 0 2024-03-24T10:46:20.629 INFO:tasks.workunit.client.1.smithi096.stdout:0/832: link d0/d11/d24/f95 d0/d11/d24/d2c/d84/f11d 0 2024-03-24T10:46:20.632 INFO:tasks.workunit.client.1.smithi096.stdout:1/696: mkdir d3/d73/ded 0 2024-03-24T10:46:20.632 INFO:tasks.workunit.client.1.smithi096.stdout:1/697: write d3/d23/d8b/fbd [124856,52980] 0 2024-03-24T10:46:20.634 INFO:tasks.workunit.client.1.smithi096.stdout:0/833: dread d0/d11/d20/d65/fe3 [0,4194304] 0 2024-03-24T10:46:20.634 INFO:tasks.workunit.client.1.smithi096.stdout:0/834: chown d0/d11/d16/cad 115 1 2024-03-24T10:46:20.637 INFO:tasks.workunit.client.1.smithi096.stdout:1/698: unlink d3/d23/d8b/d5c/d9c/cb1 0 2024-03-24T10:46:20.639 INFO:tasks.workunit.client.1.smithi096.stdout:0/835: rename d0/d11/d20/d2b/f3d to d0/d11/d16/d21/d81/d9c/dcd/d10a/f11e 0 2024-03-24T10:46:20.641 INFO:tasks.workunit.client.1.smithi096.stdout:1/699: creat d3/d23/d2f/d65/dd9/fee x:0 0 0 2024-03-24T10:46:20.644 INFO:tasks.workunit.client.1.smithi096.stdout:0/836: truncate d0/d11/d20/d65/f6e 1945038 0 2024-03-24T10:46:20.644 INFO:tasks.workunit.client.1.smithi096.stdout:0/837: chown d0/d11/d16/d21/d81/d7f/l59 463616885 1 2024-03-24T10:46:20.644 INFO:tasks.workunit.client.1.smithi096.stdout:0/838: chown d0/d11/d20/d2b/ddf/f107 334382 1 2024-03-24T10:46:20.645 INFO:tasks.workunit.client.1.smithi096.stdout:1/700: rmdir d3/d84/d9a 39 2024-03-24T10:46:20.646 INFO:tasks.workunit.client.1.smithi096.stdout:1/701: chown d3/d23/d2f/d65/d7c 412326881 1 2024-03-24T10:46:20.646 INFO:tasks.workunit.client.1.smithi096.stdout:1/702: write d3/d23/f38 [3739243,39150] 0 2024-03-24T10:46:20.647 INFO:tasks.workunit.client.1.smithi096.stdout:0/839: mkdir d0/d11/d24/d11f 0 2024-03-24T10:46:20.650 INFO:tasks.workunit.client.1.smithi096.stdout:0/840: write d0/d11/d20/d65/fe3 [1683964,18084] 0 2024-03-24T10:46:20.650 INFO:tasks.workunit.client.1.smithi096.stdout:0/841: write d0/d11/d16/fbe [107191,17040] 0 2024-03-24T10:46:20.651 INFO:tasks.workunit.client.1.smithi096.stdout:1/703: symlink d3/d23/d8b/d6a/lef 0 2024-03-24T10:46:20.651 INFO:tasks.workunit.client.1.smithi096.stdout:1/704: chown d3/d23/d8b/fcd 1155 1 2024-03-24T10:46:20.651 INFO:tasks.workunit.client.1.smithi096.stdout:1/705: fsync d3/d36/fc2 0 2024-03-24T10:46:20.653 INFO:tasks.workunit.client.1.smithi096.stdout:0/842: symlink d0/d11/d16/d21/d81/d8f/l120 0 2024-03-24T10:46:20.680 INFO:tasks.workunit.client.1.smithi096.stdout:1/706: dwrite d3/d23/d8b/d5c/fa5 [4194304,4194304] 0 2024-03-24T10:46:20.694 INFO:tasks.workunit.client.1.smithi096.stdout:1/707: dwrite d3/d23/d2f/f83 [0,4194304] 0 2024-03-24T10:46:20.698 INFO:tasks.workunit.client.1.smithi096.stdout:1/708: link d3/d23/d8b/d99/lb6 d3/d23/d71/d7b/d93/lf0 0 2024-03-24T10:46:20.701 INFO:tasks.workunit.client.1.smithi096.stdout:1/709: mkdir d3/d36/df1 0 2024-03-24T10:46:20.701 INFO:tasks.workunit.client.1.smithi096.stdout:1/710: dread - d3/d84/fd7 zero size 2024-03-24T10:46:20.701 INFO:tasks.workunit.client.1.smithi096.stdout:1/711: readlink d3/d23/d2f/d65/d7c/l90 0 2024-03-24T10:46:20.703 INFO:tasks.workunit.client.1.smithi096.stdout:1/712: truncate d3/d36/f4d 1811462 0 2024-03-24T10:46:20.707 INFO:tasks.workunit.client.1.smithi096.stdout:1/713: mkdir d3/d23/d2f/d65/d7c/ddf/df2 0 2024-03-24T10:46:20.707 INFO:tasks.workunit.client.1.smithi096.stdout:1/714: fdatasync d3/d23/d2f/f37 0 2024-03-24T10:46:20.710 INFO:tasks.workunit.client.1.smithi096.stdout:1/715: symlink d3/d23/d8b/lf3 0 2024-03-24T10:46:20.710 INFO:tasks.workunit.client.1.smithi096.stdout:1/716: dread - d3/d23/d71/d7b/d93/fc5 zero size 2024-03-24T10:46:20.710 INFO:tasks.workunit.client.1.smithi096.stdout:1/717: fsync d3/d23/d8b/d5c/d9c/fda 0 2024-03-24T10:46:20.712 INFO:tasks.workunit.client.1.smithi096.stdout:1/718: creat d3/d23/d2f/d65/d7c/ff4 x:0 0 0 2024-03-24T10:46:20.756 INFO:tasks.workunit.client.1.smithi096.stdout:7/917: sync 2024-03-24T10:46:20.757 INFO:tasks.workunit.client.1.smithi096.stdout:5/781: sync 2024-03-24T10:46:20.757 INFO:tasks.workunit.client.1.smithi096.stdout:5/782: chown d2/d7/d69/dce/df0 0 1 2024-03-24T10:46:20.760 INFO:tasks.workunit.client.1.smithi096.stdout:7/918: truncate d0/d13/df0/d3c/d68/d4b/fa4 3269766 0 2024-03-24T10:46:20.760 INFO:tasks.workunit.client.1.smithi096.stdout:7/919: write d0/d13/df0/d3c/d68/d4b/ffc [901865,106774] 0 2024-03-24T10:46:20.762 INFO:tasks.workunit.client.1.smithi096.stdout:5/783: creat d2/d29/d32/d33/d3c/f106 x:0 0 0 2024-03-24T10:46:20.782 INFO:tasks.workunit.client.1.smithi096.stdout:5/784: dwrite d2/d7/fdc [0,4194304] 0 2024-03-24T10:46:20.782 INFO:tasks.workunit.client.1.smithi096.stdout:7/920: dwrite d0/d13/d32/f107 [0,4194304] 0 2024-03-24T10:46:20.782 INFO:tasks.workunit.client.1.smithi096.stdout:7/921: dread - d0/f122 zero size 2024-03-24T10:46:20.788 INFO:tasks.workunit.client.1.smithi096.stdout:5/785: truncate d2/d7/d8/d18/db6/fff 1529107 0 2024-03-24T10:46:20.792 INFO:tasks.workunit.client.1.smithi096.stdout:5/786: mknod d2/d29/d32/d3f/c107 0 2024-03-24T10:46:20.796 INFO:tasks.workunit.client.1.smithi096.stdout:5/787: unlink d2/d7/d8/f12 0 2024-03-24T10:46:20.797 INFO:tasks.workunit.client.1.smithi096.stdout:5/788: symlink d2/d29/d32/dbb/l108 0 2024-03-24T10:46:20.797 INFO:tasks.workunit.client.1.smithi096.stdout:5/789: fsync d2/d7/d8/df/f25 0 2024-03-24T10:46:20.801 INFO:tasks.workunit.client.1.smithi096.stdout:7/922: dwrite d0/d13/df0/d3c/f71 [0,4194304] 0 2024-03-24T10:46:20.801 INFO:tasks.workunit.client.1.smithi096.stdout:7/923: chown d0/d13/d32/la3 0 1 2024-03-24T10:46:20.803 INFO:tasks.workunit.client.1.smithi096.stdout:7/924: mkdir d0/db/d1c/d77/d126 0 2024-03-24T10:46:20.805 INFO:tasks.workunit.client.1.smithi096.stdout:7/925: dread d0/d13/df0/d3c/f82 [0,4194304] 0 2024-03-24T10:46:20.807 INFO:tasks.workunit.client.1.smithi096.stdout:7/926: rename d0/d13/df0/d3c/d68/d4b/d62/c69 to d0/d13/df0/d3c/d68/d85/d10d/c127 0 2024-03-24T10:46:20.814 INFO:tasks.workunit.client.1.smithi096.stdout:5/790: dwrite d2/d29/d32/d33/f83 [0,4194304] 0 2024-03-24T10:46:20.820 INFO:tasks.workunit.client.1.smithi096.stdout:5/791: rename d2/d7/d8/f62 to d2/d7/d8/df/d14/f109 0 2024-03-24T10:46:20.820 INFO:tasks.workunit.client.1.smithi096.stdout:5/792: chown d2/d7/d8/df/d14/c24 7408 1 2024-03-24T10:46:20.826 INFO:tasks.workunit.client.1.smithi096.stdout:7/927: dwrite d0/d13/df0/d3c/d80/ff8 [0,4194304] 0 2024-03-24T10:46:20.829 INFO:tasks.workunit.client.1.smithi096.stdout:7/928: symlink d0/da9/l128 0 2024-03-24T10:46:20.830 INFO:tasks.workunit.client.1.smithi096.stdout:7/929: creat d0/d13/df0/d3c/d68/d11c/f129 x:0 0 0 2024-03-24T10:46:20.830 INFO:tasks.workunit.client.1.smithi096.stdout:7/930: chown d0/da9 1452660 1 2024-03-24T10:46:20.830 INFO:tasks.workunit.client.1.smithi096.stdout:5/793: dread d2/d7/d8/d18/f72 [0,4194304] 0 2024-03-24T10:46:20.836 INFO:tasks.workunit.client.1.smithi096.stdout:5/794: getdents d2/d7/d8 0 2024-03-24T10:46:20.839 INFO:tasks.workunit.client.1.smithi096.stdout:5/795: mkdir d2/d7/d8/df/d8a/d10a 0 2024-03-24T10:46:20.845 INFO:tasks.workunit.client.1.smithi096.stdout:7/931: dwrite d0/d13/df0/d3c/d68/d85/fdd [0,4194304] 0 2024-03-24T10:46:20.845 INFO:tasks.workunit.client.1.smithi096.stdout:7/932: fsync d0/d13/df0/d3c/f82 0 2024-03-24T10:46:20.848 INFO:tasks.workunit.client.1.smithi096.stdout:3/743: sync 2024-03-24T10:46:20.848 INFO:tasks.workunit.client.1.smithi096.stdout:0/843: sync 2024-03-24T10:46:20.849 INFO:tasks.workunit.client.1.smithi096.stdout:0/844: chown d0/d11/d24/d2c/d84/ff6 824277 1 2024-03-24T10:46:20.849 INFO:tasks.workunit.client.1.smithi096.stdout:3/744: read - d0/d13/d45/d7d/fea zero size 2024-03-24T10:46:20.849 INFO:tasks.workunit.client.1.smithi096.stdout:9/808: sync 2024-03-24T10:46:20.849 INFO:tasks.workunit.client.1.smithi096.stdout:2/917: sync 2024-03-24T10:46:20.849 INFO:tasks.workunit.client.1.smithi096.stdout:8/838: sync 2024-03-24T10:46:20.849 INFO:tasks.workunit.client.1.smithi096.stdout:6/774: sync 2024-03-24T10:46:20.849 INFO:tasks.workunit.client.1.smithi096.stdout:4/661: sync 2024-03-24T10:46:20.855 INFO:tasks.workunit.client.1.smithi096.stdout:0/845: unlink d0/d11/d20/fdb 0 2024-03-24T10:46:20.857 INFO:tasks.workunit.client.1.smithi096.stdout:7/933: dread d0/d13/df0/d3c/d80/ff8 [0,4194304] 0 2024-03-24T10:46:20.858 INFO:tasks.workunit.client.1.smithi096.stdout:4/662: dread d1/f51 [0,4194304] 0 2024-03-24T10:46:20.858 INFO:tasks.workunit.client.1.smithi096.stdout:4/663: write d1/d32/d42/fb0 [493426,106555] 0 2024-03-24T10:46:20.858 INFO:tasks.workunit.client.1.smithi096.stdout:4/664: chown d1/d32 939732047 1 2024-03-24T10:46:20.859 INFO:tasks.workunit.client.1.smithi096.stdout:2/918: mknod d5/d7c/c155 0 2024-03-24T10:46:20.862 INFO:tasks.workunit.client.1.smithi096.stdout:6/775: link d8/d12/d32/d38/d59/l9f d8/d10/d76/d91/de0/lf4 0 2024-03-24T10:46:20.865 INFO:tasks.workunit.client.1.smithi096.stdout:3/745: write d0/d13/d45/d47/fb9 [1536343,73096] 0 2024-03-24T10:46:20.865 INFO:tasks.workunit.client.1.smithi096.stdout:3/746: fsync d0/fe9 0 2024-03-24T10:46:20.865 INFO:tasks.workunit.client.1.smithi096.stdout:0/846: mkdir d0/d11/d16/d21/d81/d8f/d9a/d108/d121 0 2024-03-24T10:46:20.867 INFO:tasks.workunit.client.1.smithi096.stdout:9/809: mkdir d9/d15/d26/d29/d73/d43/d48/d104 0 2024-03-24T10:46:20.870 INFO:tasks.workunit.client.1.smithi096.stdout:4/665: truncate d1/d2b/d89/fa0 358472 0 2024-03-24T10:46:20.870 INFO:tasks.workunit.client.1.smithi096.stdout:1/719: sync 2024-03-24T10:46:20.871 INFO:tasks.workunit.client.1.smithi096.stdout:8/839: dwrite d8/d11/d22/d4c/d65/d37/f39 [0,4194304] 0 2024-03-24T10:46:20.872 INFO:tasks.workunit.client.1.smithi096.stdout:2/919: read d5/d38/f6f [428578,55172] 0 2024-03-24T10:46:20.872 INFO:tasks.workunit.client.1.smithi096.stdout:2/920: dread - d5/d79/fde zero size 2024-03-24T10:46:20.872 INFO:tasks.workunit.client.1.smithi096.stdout:3/747: creat d0/d13/d45/d47/d8f/fef x:0 0 0 2024-03-24T10:46:20.872 INFO:tasks.workunit.client.1.smithi096.stdout:3/748: readlink d0/d13/d41/lcb 0 2024-03-24T10:46:20.872 INFO:tasks.workunit.client.1.smithi096.stdout:3/749: write d0/d13/d45/d47/f8e [2973692,44822] 0 2024-03-24T10:46:20.872 INFO:tasks.workunit.client.1.smithi096.stdout:3/750: chown d0/d13/d1e/da4 1790 1 2024-03-24T10:46:20.874 INFO:tasks.workunit.client.1.smithi096.stdout:1/720: dread d3/d23/d8b/d8c/fcc [0,4194304] 0 2024-03-24T10:46:20.874 INFO:tasks.workunit.client.1.smithi096.stdout:0/847: mkdir d0/d11/d20/d9e/d122 0 2024-03-24T10:46:20.876 INFO:tasks.workunit.client.1.smithi096.stdout:6/776: write d8/d10/f3d [8133607,5505] 0 2024-03-24T10:46:20.876 INFO:tasks.workunit.client.1.smithi096.stdout:6/777: dread - d8/d12/d32/d38/fd1 zero size 2024-03-24T10:46:20.877 INFO:tasks.workunit.client.1.smithi096.stdout:9/810: symlink d9/df6/l105 0 2024-03-24T10:46:20.880 INFO:tasks.workunit.client.1.smithi096.stdout:4/666: mknod d1/d2b/d67/cd5 0 2024-03-24T10:46:20.883 INFO:tasks.workunit.client.1.smithi096.stdout:8/840: symlink d8/d11/d22/d4c/d82/da1/dd4/de7/l10c 0 2024-03-24T10:46:20.884 INFO:tasks.workunit.client.1.smithi096.stdout:2/921: rmdir d5/de/d148 39 2024-03-24T10:46:20.889 INFO:tasks.workunit.client.1.smithi096.stdout:3/751: creat d0/d13/d45/d7d/d39/d48/dbd/d9c/ff0 x:0 0 0 2024-03-24T10:46:20.893 INFO:tasks.workunit.client.1.smithi096.stdout:9/811: rename d9/d7c/f82 to d9/d15/f106 0 2024-03-24T10:46:20.893 INFO:tasks.workunit.client.1.smithi096.stdout:6/778: dread d8/d10/d19/f34 [0,4194304] 0 2024-03-24T10:46:20.893 INFO:tasks.workunit.client.1.smithi096.stdout:0/848: dwrite d0/d11/d20/d2b/ddf/f107 [0,4194304] 0 2024-03-24T10:46:20.893 INFO:tasks.workunit.client.1.smithi096.stdout:7/934: dwrite d0/db/d1c/f10a [0,4194304] 0 2024-03-24T10:46:20.893 INFO:tasks.workunit.client.1.smithi096.stdout:7/935: readlink d0/d13/df0/d59/d87/da0/l9a 0 2024-03-24T10:46:20.893 INFO:tasks.workunit.client.1.smithi096.stdout:7/936: creat d0/d13/df0/d3c/d68/d4b/d62/f12a x:0 0 0 2024-03-24T10:46:20.893 INFO:tasks.workunit.client.1.smithi096.stdout:7/937: chown d0/db/d1c/d39/c6f 2 1 2024-03-24T10:46:20.893 INFO:tasks.workunit.client.1.smithi096.stdout:7/938: fsync d0/db/ff9 0 2024-03-24T10:46:20.897 INFO:tasks.workunit.client.1.smithi096.stdout:4/667: creat d1/d2b/d40/dbd/fd6 x:0 0 0 2024-03-24T10:46:20.902 INFO:tasks.workunit.client.1.smithi096.stdout:2/922: symlink d5/de/d2a/d5e/dca/l156 0 2024-03-24T10:46:20.910 INFO:tasks.workunit.client.1.smithi096.stdout:3/752: dwrite d0/d13/d45/d7d/d39/fbf [0,4194304] 0 2024-03-24T10:46:20.911 INFO:tasks.workunit.client.1.smithi096.stdout:6/779: dwrite d8/d12/f2d [0,4194304] 0 2024-03-24T10:46:20.911 INFO:tasks.workunit.client.1.smithi096.stdout:6/780: stat d8/d10/d76/d91 0 2024-03-24T10:46:20.911 INFO:tasks.workunit.client.1.smithi096.stdout:0/849: truncate d0/d11/d20/d9e/d80/fed 172805 0 2024-03-24T10:46:20.914 INFO:tasks.workunit.client.1.smithi096.stdout:7/939: mknod d0/db/d26/dc3/c12b 0 2024-03-24T10:46:20.914 INFO:tasks.workunit.client.1.smithi096.stdout:7/940: creat d0/d13/df0/d3c/d68/d11c/f12c x:0 0 0 2024-03-24T10:46:20.915 INFO:tasks.workunit.client.1.smithi096.stdout:4/668: link d1/f51 d1/d2b/d67/d5f/fd7 0 2024-03-24T10:46:20.918 INFO:tasks.workunit.client.1.smithi096.stdout:4/669: dread d1/d11/f29 [0,4194304] 0 2024-03-24T10:46:20.918 INFO:tasks.workunit.client.1.smithi096.stdout:2/923: symlink d5/dc/l157 0 2024-03-24T10:46:20.919 INFO:tasks.workunit.client.1.smithi096.stdout:3/753: dread d0/d13/f57 [0,4194304] 0 2024-03-24T10:46:20.920 INFO:tasks.workunit.client.1.smithi096.stdout:7/941: dread d0/d13/d32/d45/f6c [0,4194304] 0 2024-03-24T10:46:20.926 INFO:tasks.workunit.client.1.smithi096.stdout:4/670: mknod d1/d32/d4c/cd8 0 2024-03-24T10:46:20.928 INFO:tasks.workunit.client.1.smithi096.stdout:3/754: mkdir d0/d13/df1 0 2024-03-24T10:46:20.928 INFO:tasks.workunit.client.1.smithi096.stdout:3/755: write d0/d9/d22/fc2 [418821,30566] 0 2024-03-24T10:46:20.933 INFO:tasks.workunit.client.1.smithi096.stdout:2/924: dwrite d5/de/f13 [0,4194304] 0 2024-03-24T10:46:20.933 INFO:tasks.workunit.client.1.smithi096.stdout:2/925: write d5/f1f [1903820,84537] 0 2024-03-24T10:46:20.933 INFO:tasks.workunit.client.1.smithi096.stdout:2/926: read - d5/f116 zero size 2024-03-24T10:46:20.933 INFO:tasks.workunit.client.1.smithi096.stdout:2/927: readlink d5/dc/l157 0 2024-03-24T10:46:20.933 INFO:tasks.workunit.client.1.smithi096.stdout:2/928: stat d5/d7c/f10b 0 2024-03-24T10:46:20.933 INFO:tasks.workunit.client.1.smithi096.stdout:2/929: write d5/de/d2a/f40 [1069145,12724] 0 2024-03-24T10:46:20.934 INFO:tasks.workunit.client.1.smithi096.stdout:2/930: fdatasync d5/de/d2a/d5e/f65 0 2024-03-24T10:46:20.934 INFO:tasks.workunit.client.1.smithi096.stdout:0/850: link d0/l4f d0/d11/d16/d21/d81/l123 0 2024-03-24T10:46:20.934 INFO:tasks.workunit.client.1.smithi096.stdout:4/671: rmdir d1/d32/d42 39 2024-03-24T10:46:20.938 INFO:tasks.workunit.client.1.smithi096.stdout:4/672: dread d1/ff [4194304,4194304] 0 2024-03-24T10:46:20.938 INFO:tasks.workunit.client.1.smithi096.stdout:4/673: truncate d1/d32/d42/dbe/fca 156921 0 2024-03-24T10:46:20.940 INFO:tasks.workunit.client.1.smithi096.stdout:7/942: getdents d0/db/d26/d48 0 2024-03-24T10:46:20.941 INFO:tasks.workunit.client.1.smithi096.stdout:7/943: read d0/d13/f35 [2936585,79880] 0 2024-03-24T10:46:20.952 INFO:tasks.workunit.client.1.smithi096.stdout:7/944: link d0/d13/d32/d45/c66 d0/db/d4f/c12d 0 2024-03-24T10:46:20.956 INFO:tasks.workunit.client.1.smithi096.stdout:0/851: dwrite d0/d11/d24/d2c/fb3 [0,4194304] 0 2024-03-24T10:46:20.956 INFO:tasks.workunit.client.1.smithi096.stdout:0/852: stat d0/fb0 0 2024-03-24T10:46:20.956 INFO:tasks.workunit.client.1.smithi096.stdout:4/674: dwrite d1/d11/f41 [0,4194304] 0 2024-03-24T10:46:20.957 INFO:tasks.workunit.client.1.smithi096.stdout:7/945: mknod d0/d13/d32/c12e 0 2024-03-24T10:46:20.958 INFO:tasks.workunit.client.1.smithi096.stdout:7/946: write d0/db/d26/d48/fb4 [444010,115102] 0 2024-03-24T10:46:20.962 INFO:tasks.workunit.client.1.smithi096.stdout:0/853: unlink d0/d11/d16/d21/d78/lee 0 2024-03-24T10:46:20.967 INFO:tasks.workunit.client.1.smithi096.stdout:4/675: rename d1/f6f to d1/d32/d42/dbe/fd9 0 2024-03-24T10:46:20.967 INFO:tasks.workunit.client.1.smithi096.stdout:7/947: read d0/db/d26/d48/f8d [2445634,57637] 0 2024-03-24T10:46:20.967 INFO:tasks.workunit.client.1.smithi096.stdout:7/948: chown d0/d13/d32/lac 33899 1 2024-03-24T10:46:20.969 INFO:tasks.workunit.client.1.smithi096.stdout:0/854: dread d0/d11/d24/f55 [0,4194304] 0 2024-03-24T10:46:20.969 INFO:tasks.workunit.client.1.smithi096.stdout:0/855: readlink d0/d11/d16/d21/d81/l123 0 2024-03-24T10:46:20.969 INFO:tasks.workunit.client.1.smithi096.stdout:0/856: dread - d0/d11/d16/f53 zero size 2024-03-24T10:46:20.969 INFO:tasks.workunit.client.1.smithi096.stdout:2/931: rmdir d5/dc/d6b/d72/d11a 0 2024-03-24T10:46:20.969 INFO:tasks.workunit.client.1.smithi096.stdout:2/932: chown d5/d38/dea 571295 1 2024-03-24T10:46:20.970 INFO:tasks.workunit.client.1.smithi096.stdout:0/857: dread d0/d11/d16/d21/d81/d9c/dcd/d10a/f11e [0,4194304] 0 2024-03-24T10:46:20.971 INFO:tasks.workunit.client.1.smithi096.stdout:3/756: dwrite d0/d13/d45/d7d/f94 [0,4194304] 0 2024-03-24T10:46:20.971 INFO:tasks.workunit.client.1.smithi096.stdout:3/757: write d0/d13/d77/fde [1022271,128821] 0 2024-03-24T10:46:20.971 INFO:tasks.workunit.client.1.smithi096.stdout:4/676: symlink d1/d11/lda 0 2024-03-24T10:46:20.971 INFO:tasks.workunit.client.1.smithi096.stdout:4/677: readlink d1/d2b/l68 0 2024-03-24T10:46:20.971 INFO:tasks.workunit.client.1.smithi096.stdout:4/678: truncate d1/d2b/d40/dbd/fd6 564523 0 2024-03-24T10:46:20.971 INFO:tasks.workunit.client.1.smithi096.stdout:4/679: dread - d1/d32/d4c/d7e/fcf zero size 2024-03-24T10:46:20.971 INFO:tasks.workunit.client.1.smithi096.stdout:4/680: write d1/d2b/d67/d5f/da9/db5/fd2 [741311,29962] 0 2024-03-24T10:46:20.971 INFO:tasks.workunit.client.1.smithi096.stdout:4/681: stat d1/d2b/d59/d74/fb4 0 2024-03-24T10:46:20.972 INFO:tasks.workunit.client.1.smithi096.stdout:7/949: creat d0/db/d1c/d77/f12f x:0 0 0 2024-03-24T10:46:20.973 INFO:tasks.workunit.client.1.smithi096.stdout:2/933: truncate d5/de/d2a/d5e/f65 129047 0 2024-03-24T10:46:20.973 INFO:tasks.workunit.client.1.smithi096.stdout:2/934: fsync f2 0 2024-03-24T10:46:20.973 INFO:tasks.workunit.client.1.smithi096.stdout:2/935: write d5/dc/f2f [4453483,69980] 0 2024-03-24T10:46:20.974 INFO:tasks.workunit.client.1.smithi096.stdout:0/858: mknod d0/d11/d16/d21/d25/c124 0 2024-03-24T10:46:20.974 INFO:tasks.workunit.client.1.smithi096.stdout:0/859: getdents d0/d11/d16/d21/d25/df8 0 2024-03-24T10:46:20.974 INFO:tasks.workunit.client.1.smithi096.stdout:0/860: write d0/d11/d16/d21/d81/d8f/d9a/ddd/fe9 [79235,94699] 0 2024-03-24T10:46:20.974 INFO:tasks.workunit.client.1.smithi096.stdout:3/758: symlink d0/d13/d45/d47/d8f/lf2 0 2024-03-24T10:46:20.976 INFO:tasks.workunit.client.1.smithi096.stdout:7/950: truncate d0/d13/ffa 3241690 0 2024-03-24T10:46:20.977 INFO:tasks.workunit.client.1.smithi096.stdout:7/951: dread d0/db/d4f/f74 [0,4194304] 0 2024-03-24T10:46:20.977 INFO:tasks.workunit.client.1.smithi096.stdout:2/936: creat d5/dc/d6b/d72/d91/f158 x:0 0 0 2024-03-24T10:46:20.981 INFO:tasks.workunit.client.1.smithi096.stdout:3/759: rename d0/d13/d45/d7d/d39/d48/d85/c98 to d0/d13/d45/d7d/d39/d48/d8d/cf3 0 2024-03-24T10:46:20.981 INFO:tasks.workunit.client.1.smithi096.stdout:3/760: creat d0/d13/d45/d7d/d39/d48/dbd/d9c/ff4 x:0 0 0 2024-03-24T10:46:20.981 INFO:tasks.workunit.client.1.smithi096.stdout:3/761: write d0/d9/f49 [3468251,1733] 0 2024-03-24T10:46:20.982 INFO:tasks.workunit.client.1.smithi096.stdout:7/952: mkdir d0/d13/df0/d3c/d68/d130 0 2024-03-24T10:46:20.983 INFO:tasks.workunit.client.1.smithi096.stdout:0/861: getdents d0/d11/d43 0 2024-03-24T10:46:20.986 INFO:tasks.workunit.client.1.smithi096.stdout:3/762: dread d0/d13/d77/fde [0,4194304] 0 2024-03-24T10:46:20.990 INFO:tasks.workunit.client.1.smithi096.stdout:7/953: mkdir d0/db/d26/d48/d131 0 2024-03-24T10:46:20.990 INFO:tasks.workunit.client.1.smithi096.stdout:7/954: creat d0/d13/df0/d3c/d68/d101/f132 x:0 0 0 2024-03-24T10:46:20.990 INFO:tasks.workunit.client.1.smithi096.stdout:7/955: truncate d0/db/d26/dc3/f86 1213175 0 2024-03-24T10:46:20.992 INFO:tasks.workunit.client.1.smithi096.stdout:4/682: link d1/d32/d42/dbe/fd9 d1/d2b/d89/fdb 0 2024-03-24T10:46:20.992 INFO:tasks.workunit.client.1.smithi096.stdout:4/683: chown d1/d2b/d67/d5f/da9 457519436 1 2024-03-24T10:46:20.992 INFO:tasks.workunit.client.1.smithi096.stdout:0/862: mknod d0/d11/d20/d9e/d122/c125 0 2024-03-24T10:46:20.996 INFO:tasks.workunit.client.1.smithi096.stdout:3/763: rename d0/d13/c18 to d0/d13/d45/d7d/de7/cf5 0 2024-03-24T10:46:20.996 INFO:tasks.workunit.client.1.smithi096.stdout:3/764: stat d0/d13/d45/d7d/d71/db6/fe6 0 2024-03-24T10:46:21.001 INFO:tasks.workunit.client.1.smithi096.stdout:2/937: dwrite d5/d79/de8/f100 [4194304,4194304] 0 2024-03-24T10:46:21.001 INFO:tasks.workunit.client.1.smithi096.stdout:0/863: dread d0/d11/d20/d56/f5c [0,4194304] 0 2024-03-24T10:46:21.001 INFO:tasks.workunit.client.1.smithi096.stdout:2/938: creat d5/dc/d12/d23/f159 x:0 0 0 2024-03-24T10:46:21.001 INFO:tasks.workunit.client.1.smithi096.stdout:2/939: write d5/de/d2a/d5e/fe4 [896935,5361] 0 2024-03-24T10:46:21.001 INFO:tasks.workunit.client.1.smithi096.stdout:7/956: creat d0/db/d26/d48/f133 x:0 0 0 2024-03-24T10:46:21.019 INFO:tasks.workunit.client.1.smithi096.stdout:2/940: dwrite d5/dc/d12/ff4 [0,4194304] 0 2024-03-24T10:46:21.026 INFO:tasks.workunit.client.1.smithi096.stdout:4/684: rename d1/d32/d4c/d7e/f9e to d1/d2b/d40/fdc 0 2024-03-24T10:46:21.031 INFO:tasks.workunit.client.1.smithi096.stdout:0/864: mkdir d0/d11/d16/d21/d78/d92/d126 0 2024-03-24T10:46:21.033 INFO:tasks.workunit.client.1.smithi096.stdout:7/957: unlink d0/d13/df0/d3c/d68/d7e/ff4 0 2024-03-24T10:46:21.033 INFO:tasks.workunit.client.1.smithi096.stdout:7/958: chown d0/db/d26/dc3/ld3 0 1 2024-03-24T10:46:21.039 INFO:tasks.workunit.client.1.smithi096.stdout:3/765: rename d0/d9/d22/fa8 to d0/d13/d45/d7d/d39/d48/dbd/d9c/da9/de5/ff6 0 2024-03-24T10:46:21.039 INFO:tasks.workunit.client.1.smithi096.stdout:3/766: readlink d0/d13/d45/d7d/d39/l80 0 2024-03-24T10:46:21.040 INFO:tasks.workunit.client.1.smithi096.stdout:7/959: mkdir d0/d13/d32/d45/d134 0 2024-03-24T10:46:21.042 INFO:tasks.workunit.client.1.smithi096.stdout:2/941: mkdir d5/d7c/d8f/d128/d141/d15a 0 2024-03-24T10:46:21.042 INFO:tasks.workunit.client.1.smithi096.stdout:2/942: chown d5/dc/d6b/def/l12a 2581993 1 2024-03-24T10:46:21.045 INFO:tasks.workunit.client.1.smithi096.stdout:3/767: symlink d0/d13/d45/d7d/d39/d48/dbd/d9c/da9/lf7 0 2024-03-24T10:46:21.046 INFO:tasks.workunit.client.1.smithi096.stdout:7/960: mknod d0/d111/c135 0 2024-03-24T10:46:21.047 INFO:tasks.workunit.client.1.smithi096.stdout:7/961: readlink d0/db/d1c/d77/dab/de6/lea 0 2024-03-24T10:46:21.048 INFO:tasks.workunit.client.1.smithi096.stdout:0/865: dwrite d0/d11/d16/d21/d25/f106 [0,4194304] 0 2024-03-24T10:46:21.048 INFO:tasks.workunit.client.1.smithi096.stdout:4/685: dwrite d1/d2b/d6b/d80/fb6 [4194304,4194304] 0 2024-03-24T10:46:21.049 INFO:tasks.workunit.client.1.smithi096.stdout:3/768: mknod d0/d13/d45/d47/cf8 0 2024-03-24T10:46:21.051 INFO:tasks.workunit.client.1.smithi096.stdout:7/962: link d0/db/d4f/f74 d0/d13/d32/d45/f136 0 2024-03-24T10:46:21.051 INFO:tasks.workunit.client.1.smithi096.stdout:7/963: write d0/d13/d32/d45/f136 [1849860,123718] 0 2024-03-24T10:46:21.051 INFO:tasks.workunit.client.1.smithi096.stdout:7/964: fdatasync d0/f10f 0 2024-03-24T10:46:21.051 INFO:tasks.workunit.client.1.smithi096.stdout:7/965: readlink d0/d13/df0/d3c/d68/d85/l98 0 2024-03-24T10:46:21.057 INFO:tasks.workunit.client.1.smithi096.stdout:0/866: write d0/d11/d20/d2b/ddf/f107 [932706,121531] 0 2024-03-24T10:46:21.057 INFO:tasks.workunit.client.1.smithi096.stdout:4/686: rename d1/d32/l8c to d1/d32/d42/ldd 0 2024-03-24T10:46:21.059 INFO:tasks.workunit.client.1.smithi096.stdout:2/943: dwrite d5/d7c/d8f/fc7 [0,4194304] 0 2024-03-24T10:46:21.064 INFO:tasks.workunit.client.1.smithi096.stdout:3/769: dwrite d0/d13/d45/d52/f66 [0,4194304] 0 2024-03-24T10:46:21.067 INFO:tasks.workunit.client.1.smithi096.stdout:0/867: fdatasync d0/f72 0 2024-03-24T10:46:21.069 INFO:tasks.workunit.client.1.smithi096.stdout:4/687: link d1/d2b/d6b/l95 d1/d32/d42/dba/lde 0 2024-03-24T10:46:21.076 INFO:tasks.workunit.client.1.smithi096.stdout:2/944: creat d5/dc/d12/d23/d105/f15b x:0 0 0 2024-03-24T10:46:21.076 INFO:tasks.workunit.client.1.smithi096.stdout:2/945: chown d5/de/d2a/d5e/f7f 7269 1 2024-03-24T10:46:21.085 INFO:tasks.workunit.client.1.smithi096.stdout:7/966: mknod d0/d13/df0/c137 0 2024-03-24T10:46:21.085 INFO:tasks.workunit.client.1.smithi096.stdout:7/967: fsync d0/f14 0 2024-03-24T10:46:21.090 INFO:tasks.workunit.client.1.smithi096.stdout:3/770: symlink d0/d9/lf9 0 2024-03-24T10:46:21.092 INFO:tasks.workunit.client.1.smithi096.stdout:0/868: mkdir d0/d11/d24/d127 0 2024-03-24T10:46:21.092 INFO:tasks.workunit.client.1.smithi096.stdout:0/869: chown d0/d11/d20/fd5 21191 1 2024-03-24T10:46:21.093 INFO:tasks.workunit.client.1.smithi096.stdout:0/870: stat d0/d11/d20/d65/f6e 0 2024-03-24T10:46:21.093 INFO:tasks.workunit.client.1.smithi096.stdout:0/871: stat d0/d11/d24/f45 0 2024-03-24T10:46:21.093 INFO:tasks.workunit.client.1.smithi096.stdout:0/872: chown d0/d11/d20/d91/f96 1624981275 1 2024-03-24T10:46:21.093 INFO:tasks.workunit.client.1.smithi096.stdout:2/946: dwrite d5/de/d2a/f10f [0,4194304] 0 2024-03-24T10:46:21.094 INFO:tasks.workunit.client.1.smithi096.stdout:0/873: write d0/d11/d20/d91/fe5 [214858,38338] 0 2024-03-24T10:46:21.094 INFO:tasks.workunit.client.1.smithi096.stdout:0/874: creat d0/d11/dc3/f128 x:0 0 0 2024-03-24T10:46:21.094 INFO:tasks.workunit.client.1.smithi096.stdout:0/875: fdatasync d0/d11/d16/d21/d25/f106 0 2024-03-24T10:46:21.094 INFO:tasks.workunit.client.1.smithi096.stdout:4/688: symlink d1/d32/d4c/d7d/ldf 0 2024-03-24T10:46:21.096 INFO:tasks.workunit.client.1.smithi096.stdout:7/968: creat d0/db/d4f/d125/f138 x:0 0 0 2024-03-24T10:46:21.100 INFO:tasks.workunit.client.1.smithi096.stdout:3/771: rename d0/d13/d45/d7d/d39/fd6 to d0/d13/d45/d7d/d39/d48/ffa 0 2024-03-24T10:46:21.100 INFO:tasks.workunit.client.1.smithi096.stdout:3/772: truncate d0/d13/d41/d7f/fb7 667651 0 2024-03-24T10:46:21.108 INFO:tasks.workunit.client.1.smithi096.stdout:0/876: creat d0/d11/d20/d91/f129 x:0 0 0 2024-03-24T10:46:21.111 INFO:tasks.workunit.client.1.smithi096.stdout:2/947: dwrite d5/dc/d12/d23/d105/f109 [0,4194304] 0 2024-03-24T10:46:21.112 INFO:tasks.workunit.client.1.smithi096.stdout:3/773: link d0/d13/d45/d7d/d71/ld7 d0/d92/lfb 0 2024-03-24T10:46:21.113 INFO:tasks.workunit.client.1.smithi096.stdout:0/877: creat d0/d11/d24/d127/f12a x:0 0 0 2024-03-24T10:46:21.113 INFO:tasks.workunit.client.1.smithi096.stdout:0/878: write d0/d11/d16/d21/fef [807877,112489] 0 2024-03-24T10:46:21.114 INFO:tasks.workunit.client.1.smithi096.stdout:7/969: dwrite d0/d13/df0/d3c/d68/d11c/f12c [0,4194304] 0 2024-03-24T10:46:21.115 INFO:tasks.workunit.client.1.smithi096.stdout:4/689: getdents d1/d2b/d67 0 2024-03-24T10:46:21.115 INFO:tasks.workunit.client.1.smithi096.stdout:4/690: dread - d1/d32/d4c/da6/faa zero size 2024-03-24T10:46:21.115 INFO:tasks.workunit.client.1.smithi096.stdout:4/691: dread - d1/d32/d42/f79 zero size 2024-03-24T10:46:21.115 INFO:tasks.workunit.client.1.smithi096.stdout:4/692: chown d1/d2b/c58 940631469 1 2024-03-24T10:46:21.116 INFO:tasks.workunit.client.1.smithi096.stdout:1/721: sync 2024-03-24T10:46:21.116 INFO:tasks.workunit.client.1.smithi096.stdout:5/796: sync 2024-03-24T10:46:21.122 INFO:tasks.workunit.client.1.smithi096.stdout:2/948: mknod d5/de/d120/c15c 0 2024-03-24T10:46:21.126 INFO:tasks.workunit.client.1.smithi096.stdout:0/879: getdents d0/d11/d16/d21/d81/d8f/d9a/d108 0 2024-03-24T10:46:21.127 INFO:tasks.workunit.client.1.smithi096.stdout:0/880: chown d0/d11/d16/d21/d25/c124 82483910 1 2024-03-24T10:46:21.127 INFO:tasks.workunit.client.1.smithi096.stdout:0/881: creat d0/d11/d16/d21/d25/f12b x:0 0 0 2024-03-24T10:46:21.127 INFO:tasks.workunit.client.1.smithi096.stdout:0/882: chown d0/d11/d16/d21/d81/d8f/d9a/ddd 48 1 2024-03-24T10:46:21.127 INFO:tasks.workunit.client.1.smithi096.stdout:0/883: readlink d0/d11/l35 0 2024-03-24T10:46:21.131 INFO:tasks.workunit.client.1.smithi096.stdout:5/797: dwrite d2/d7/d8/d18/db6/fd1 [0,4194304] 0 2024-03-24T10:46:21.131 INFO:tasks.workunit.client.1.smithi096.stdout:4/693: mkdir d1/d2b/d59/de0 0 2024-03-24T10:46:21.131 INFO:tasks.workunit.client.1.smithi096.stdout:4/694: read d1/d11/fae [147248,46428] 0 2024-03-24T10:46:21.133 INFO:tasks.workunit.client.1.smithi096.stdout:1/722: truncate d3/d23/d2f/f7e 4014082 0 2024-03-24T10:46:21.134 INFO:tasks.workunit.client.1.smithi096.stdout:1/723: fsync d3/d23/d2f/f3d 0 2024-03-24T10:46:21.134 INFO:tasks.workunit.client.1.smithi096.stdout:1/724: fsync d3/d23/f68 0 2024-03-24T10:46:21.135 INFO:tasks.workunit.client.1.smithi096.stdout:2/949: creat d5/f15d x:0 0 0 2024-03-24T10:46:21.135 INFO:tasks.workunit.client.1.smithi096.stdout:2/950: stat d5/de/l68 0 2024-03-24T10:46:21.144 INFO:tasks.workunit.client.1.smithi096.stdout:0/884: mknod d0/d11/d24/d9d/c12c 0 2024-03-24T10:46:21.149 INFO:tasks.workunit.client.1.smithi096.stdout:5/798: mkdir d2/d7/d8/df/d8a/d10a/d10b 0 2024-03-24T10:46:21.153 INFO:tasks.workunit.client.1.smithi096.stdout:4/695: dwrite d1/d32/d4c/d7e/fd4 [0,4194304] 0 2024-03-24T10:46:21.153 INFO:tasks.workunit.client.1.smithi096.stdout:2/951: unlink d5/d38/fc4 0 2024-03-24T10:46:21.154 INFO:tasks.workunit.client.1.smithi096.stdout:1/725: dwrite d3/d84/fd7 [0,4194304] 0 2024-03-24T10:46:21.155 INFO:tasks.workunit.client.1.smithi096.stdout:3/774: rename d0/d13/d45/d7d/d39/d48/dbd/d9c/da9 to d0/d92/ddc/dfc 0 2024-03-24T10:46:21.155 INFO:tasks.workunit.client.1.smithi096.stdout:3/775: chown d0/d13/d45/d47/cf8 16621 1 2024-03-24T10:46:21.158 INFO:tasks.workunit.client.1.smithi096.stdout:0/885: symlink d0/d11/d24/d11f/l12d 0 2024-03-24T10:46:21.160 INFO:tasks.workunit.client.1.smithi096.stdout:5/799: mknod d2/d29/d32/dbb/c10c 0 2024-03-24T10:46:21.163 INFO:tasks.workunit.client.1.smithi096.stdout:4/696: mknod d1/d32/d4c/d7d/ce1 0 2024-03-24T10:46:21.164 INFO:tasks.workunit.client.1.smithi096.stdout:2/952: mknod d5/d79/de8/c15e 0 2024-03-24T10:46:21.166 INFO:tasks.workunit.client.1.smithi096.stdout:6/781: sync 2024-03-24T10:46:21.166 INFO:tasks.workunit.client.1.smithi096.stdout:8/841: sync 2024-03-24T10:46:21.166 INFO:tasks.workunit.client.1.smithi096.stdout:9/812: sync 2024-03-24T10:46:21.166 INFO:tasks.workunit.client.1.smithi096.stdout:8/842: readlink d8/d72/d96/le3 0 2024-03-24T10:46:21.170 INFO:tasks.workunit.client.1.smithi096.stdout:1/726: link d3/d23/d71/dbe/fc9 d3/d23/d2f/d65/d7c/ddf/ff5 0 2024-03-24T10:46:21.170 INFO:tasks.workunit.client.1.smithi096.stdout:1/727: stat d3/d23/d8b/f57 0 2024-03-24T10:46:21.173 INFO:tasks.workunit.client.1.smithi096.stdout:8/843: write d8/d11/d22/d4c/d65/d1d/d3d/f56 [809230,100634] 0 2024-03-24T10:46:21.174 INFO:tasks.workunit.client.1.smithi096.stdout:3/776: truncate d0/d9/f32 885190 0 2024-03-24T10:46:21.175 INFO:tasks.workunit.client.1.smithi096.stdout:0/886: symlink d0/d11/d43/d75/l12e 0 2024-03-24T10:46:21.176 INFO:tasks.workunit.client.1.smithi096.stdout:5/800: symlink d2/d29/d32/d3f/l10d 0 2024-03-24T10:46:21.178 INFO:tasks.workunit.client.1.smithi096.stdout:2/953: unlink d5/d38/dea/cff 0 2024-03-24T10:46:21.181 INFO:tasks.workunit.client.1.smithi096.stdout:6/782: creat d8/d12/d32/d38/d59/db8/ff5 x:0 0 0 2024-03-24T10:46:21.183 INFO:tasks.workunit.client.1.smithi096.stdout:4/697: dwrite d1/d2b/d6b/d80/fb6 [0,4194304] 0 2024-03-24T10:46:21.183 INFO:tasks.workunit.client.1.smithi096.stdout:5/801: dread d2/f5 [0,4194304] 0 2024-03-24T10:46:21.187 INFO:tasks.workunit.client.1.smithi096.stdout:8/844: dread d8/d11/d22/d4c/d65/d1d/f2b [0,4194304] 0 2024-03-24T10:46:21.192 INFO:tasks.workunit.client.1.smithi096.stdout:3/777: dwrite d0/d9/f36 [0,4194304] 0 2024-03-24T10:46:21.195 INFO:tasks.workunit.client.1.smithi096.stdout:0/887: unlink d0/d11/d24/d2c/d84/de8/l11c 0 2024-03-24T10:46:21.195 INFO:tasks.workunit.client.1.smithi096.stdout:0/888: fdatasync d0/d11/d20/f26 0 2024-03-24T10:46:21.199 INFO:tasks.workunit.client.1.smithi096.stdout:6/783: creat d8/d10/d13/d63/d51/ff6 x:0 0 0 2024-03-24T10:46:21.201 INFO:tasks.workunit.client.1.smithi096.stdout:9/813: symlink d9/d15/d26/d29/d73/d49/db7/l107 0 2024-03-24T10:46:21.206 INFO:tasks.workunit.client.1.smithi096.stdout:5/802: creat d2/d7/d69/dce/df0/f10e x:0 0 0 2024-03-24T10:46:21.206 INFO:tasks.workunit.client.1.smithi096.stdout:5/803: read - d2/d29/d32/d33/d37/dab/fed zero size 2024-03-24T10:46:21.208 INFO:tasks.workunit.client.1.smithi096.stdout:6/784: write d8/d10/d13/d63/f2c [121758,81440] 0 2024-03-24T10:46:21.208 INFO:tasks.workunit.client.1.smithi096.stdout:6/785: fsync d8/d12/d28/d36/d65/d6b/dec/fed 0 2024-03-24T10:46:21.209 INFO:tasks.workunit.client.1.smithi096.stdout:8/845: dwrite d8/d11/d22/d4c/d65/d1a/d46/f10b [0,4194304] 0 2024-03-24T10:46:21.209 INFO:tasks.workunit.client.1.smithi096.stdout:3/778: dwrite d0/d13/d45/d52/fc1 [0,4194304] 0 2024-03-24T10:46:21.209 INFO:tasks.workunit.client.1.smithi096.stdout:8/846: write d8/f58 [1442863,54028] 0 2024-03-24T10:46:21.211 INFO:tasks.workunit.client.1.smithi096.stdout:4/698: creat d1/d32/fe2 x:0 0 0 2024-03-24T10:46:21.214 INFO:tasks.workunit.client.1.smithi096.stdout:7/970: rename d0/d13/df0/d3c/d68/d11c to d0/d139 0 2024-03-24T10:46:21.216 INFO:tasks.workunit.client.1.smithi096.stdout:2/954: rmdir d5/dc/d6b/d72/d91 39 2024-03-24T10:46:21.217 INFO:tasks.workunit.client.1.smithi096.stdout:9/814: truncate d9/db/d6d/fe2 819507 0 2024-03-24T10:46:21.222 INFO:tasks.workunit.client.1.smithi096.stdout:6/786: link d8/d10/d13/d63/f2c d8/d12/d28/d36/d65/d71/dd0/da8/dbe/ff7 0 2024-03-24T10:46:21.223 INFO:tasks.workunit.client.1.smithi096.stdout:4/699: write d1/d2b/d67/f52 [1100551,65294] 0 2024-03-24T10:46:21.230 INFO:tasks.workunit.client.1.smithi096.stdout:8/847: creat d8/d11/d22/d4c/d65/d52/dc0/dd9/d10a/f10d x:0 0 0 2024-03-24T10:46:21.232 INFO:tasks.workunit.client.1.smithi096.stdout:4/700: write d1/d11/f66 [1342457,110959] 0 2024-03-24T10:46:21.232 INFO:tasks.workunit.client.1.smithi096.stdout:4/701: creat d1/d2b/d6b/d80/d9c/fe3 x:0 0 0 2024-03-24T10:46:21.237 INFO:tasks.workunit.client.1.smithi096.stdout:1/728: rename d3/l20 to d3/d23/d2f/d65/d7c/lf6 0 2024-03-24T10:46:21.237 INFO:tasks.workunit.client.1.smithi096.stdout:1/729: chown d3/d36/f4d 150 1 2024-03-24T10:46:21.238 INFO:tasks.workunit.client.1.smithi096.stdout:7/971: creat d0/d111/f13a x:0 0 0 2024-03-24T10:46:21.240 INFO:tasks.workunit.client.1.smithi096.stdout:4/702: dread d1/d11/f12 [0,4194304] 0 2024-03-24T10:46:21.240 INFO:tasks.workunit.client.1.smithi096.stdout:4/703: dread - d1/d2b/d6b/d80/d9c/fc8 zero size 2024-03-24T10:46:21.240 INFO:tasks.workunit.client.1.smithi096.stdout:0/889: link d0/f60 d0/d11/d24/d9d/df5/f12f 0 2024-03-24T10:46:21.241 INFO:tasks.workunit.client.1.smithi096.stdout:9/815: mknod d9/d102/c108 0 2024-03-24T10:46:21.241 INFO:tasks.workunit.client.1.smithi096.stdout:9/816: chown d9/d15/d26/d29/d73/d43/d48 498317997 1 2024-03-24T10:46:21.242 INFO:tasks.workunit.client.1.smithi096.stdout:9/817: chown d9/db/d6d/d4f/de8 890 1 2024-03-24T10:46:21.243 INFO:tasks.workunit.client.1.smithi096.stdout:5/804: mkdir d2/d29/d32/d33/d3c/d3d/d10f 0 2024-03-24T10:46:21.245 INFO:tasks.workunit.client.1.smithi096.stdout:9/818: dread d9/db/d6d/d7f/f3f [0,4194304] 0 2024-03-24T10:46:21.247 INFO:tasks.workunit.client.1.smithi096.stdout:6/787: mkdir d8/d12/d28/d36/d65/d6b/da6/db2/df8 0 2024-03-24T10:46:21.251 INFO:tasks.workunit.client.1.smithi096.stdout:8/848: link d8/f16 d8/d11/d22/d4c/d65/d1a/d46/f10e 0 2024-03-24T10:46:21.258 INFO:tasks.workunit.client.1.smithi096.stdout:2/955: rename d5/dc/d12/f31 to d5/d79/d9f/f15f 0 2024-03-24T10:46:21.258 INFO:tasks.workunit.client.1.smithi096.stdout:2/956: stat d5/dc/d12/f26 0 2024-03-24T10:46:21.258 INFO:tasks.workunit.client.1.smithi096.stdout:2/957: chown d5/dc/ldd 30 1 2024-03-24T10:46:21.260 INFO:tasks.workunit.client.1.smithi096.stdout:1/730: creat d3/d23/d2f/d65/d7c/ddf/ff7 x:0 0 0 2024-03-24T10:46:21.263 INFO:tasks.workunit.client.1.smithi096.stdout:7/972: creat d0/db/d26/d11b/f13b x:0 0 0 2024-03-24T10:46:21.263 INFO:tasks.workunit.client.1.smithi096.stdout:7/973: truncate d0/db/d26/dc3/f9d 791482 0 2024-03-24T10:46:21.265 INFO:tasks.workunit.client.1.smithi096.stdout:4/704: symlink d1/d32/d4c/d7d/le4 0 2024-03-24T10:46:21.268 INFO:tasks.workunit.client.1.smithi096.stdout:5/805: mknod d2/d29/d32/d3f/d5f/dc0/c110 0 2024-03-24T10:46:21.268 INFO:tasks.workunit.client.1.smithi096.stdout:1/731: write d3/d23/d71/f7d [2851271,27248] 0 2024-03-24T10:46:21.269 INFO:tasks.workunit.client.1.smithi096.stdout:9/819: rmdir d9/d15/d26/d29/dcd 39 2024-03-24T10:46:21.273 INFO:tasks.workunit.client.1.smithi096.stdout:7/974: dread d0/db/d1c/d39/f4a [4194304,4194304] 0 2024-03-24T10:46:21.273 INFO:tasks.workunit.client.1.smithi096.stdout:7/975: dread - d0/d13/df0/d3c/d68/d85/fe4 zero size 2024-03-24T10:46:21.277 INFO:tasks.workunit.client.1.smithi096.stdout:8/849: creat d8/d11/d22/d4c/d65/df0/f10f x:0 0 0 2024-03-24T10:46:21.279 INFO:tasks.workunit.client.1.smithi096.stdout:2/958: truncate d5/dc/f83 301183 0 2024-03-24T10:46:21.279 INFO:tasks.workunit.client.1.smithi096.stdout:2/959: chown d5/d79/d9f/f15f 3535 1 2024-03-24T10:46:21.280 INFO:tasks.workunit.client.1.smithi096.stdout:0/890: getdents d0/d11/dc3 0 2024-03-24T10:46:21.282 INFO:tasks.workunit.client.1.smithi096.stdout:5/806: symlink d2/d49/l111 0 2024-03-24T10:46:21.285 INFO:tasks.workunit.client.1.smithi096.stdout:9/820: rename d9/d79/fd1 to d9/df1/f109 0 2024-03-24T10:46:21.285 INFO:tasks.workunit.client.1.smithi096.stdout:9/821: read d9/db/d6d/f35 [3422678,95959] 0 2024-03-24T10:46:21.285 INFO:tasks.workunit.client.1.smithi096.stdout:9/822: write d9/db/f8c [1380061,32825] 0 2024-03-24T10:46:21.285 INFO:tasks.workunit.client.1.smithi096.stdout:9/823: creat d9/d7c/f10a x:0 0 0 2024-03-24T10:46:21.286 INFO:tasks.workunit.client.1.smithi096.stdout:6/788: getdents d8/d12/d32/d38/d59 0 2024-03-24T10:46:21.286 INFO:tasks.workunit.client.1.smithi096.stdout:4/705: dwrite d1/d2b/d40/fdc [0,4194304] 0 2024-03-24T10:46:21.287 INFO:tasks.workunit.client.1.smithi096.stdout:8/850: mkdir d8/d110 0 2024-03-24T10:46:21.287 INFO:tasks.workunit.client.1.smithi096.stdout:8/851: fdatasync d8/d11/d44/fa7 0 2024-03-24T10:46:21.288 INFO:tasks.workunit.client.1.smithi096.stdout:2/960: mkdir d5/dc/d11f/d160 0 2024-03-24T10:46:21.289 INFO:tasks.workunit.client.1.smithi096.stdout:1/732: dwrite d3/d23/d8b/d6a/fd6 [0,4194304] 0 2024-03-24T10:46:21.290 INFO:tasks.workunit.client.1.smithi096.stdout:0/891: creat d0/d11/d16/d21/d25/d47/f130 x:0 0 0 2024-03-24T10:46:21.290 INFO:tasks.workunit.client.1.smithi096.stdout:0/892: truncate d0/d11/d43/fa6 961404 0 2024-03-24T10:46:21.291 INFO:tasks.workunit.client.1.smithi096.stdout:5/807: mkdir d2/d112 0 2024-03-24T10:46:21.293 INFO:tasks.workunit.client.1.smithi096.stdout:7/976: rename d0/db/d1c/d39/da2/c113 to d0/d139/c13c 0 2024-03-24T10:46:21.293 INFO:tasks.workunit.client.1.smithi096.stdout:7/977: creat d0/f13d x:0 0 0 2024-03-24T10:46:21.293 INFO:tasks.workunit.client.1.smithi096.stdout:7/978: chown d0/d13/df0/c137 8043 1 2024-03-24T10:46:21.293 INFO:tasks.workunit.client.1.smithi096.stdout:7/979: stat d0/db/d26/d48/da8/db1/l121 0 2024-03-24T10:46:21.293 INFO:tasks.workunit.client.1.smithi096.stdout:7/980: truncate d0/d13/df0/d3c/d80/fcf 448136 0 2024-03-24T10:46:21.298 INFO:tasks.workunit.client.1.smithi096.stdout:4/706: write d1/d2b/d6b/d80/f9f [4120211,5188] 0 2024-03-24T10:46:21.300 INFO:tasks.workunit.client.1.smithi096.stdout:0/893: unlink d0/d11/d24/f68 0 2024-03-24T10:46:21.301 INFO:tasks.workunit.client.1.smithi096.stdout:0/894: chown d0/d11/dc3 174173755 1 2024-03-24T10:46:21.302 INFO:tasks.workunit.client.1.smithi096.stdout:5/808: mkdir d2/d29/d32/d33/d8d/d94/d113 0 2024-03-24T10:46:21.302 INFO:tasks.workunit.client.1.smithi096.stdout:2/961: read d5/dc/f2f [1322917,127680] 0 2024-03-24T10:46:21.304 INFO:tasks.workunit.client.1.smithi096.stdout:1/733: write d3/d23/d8b/d6a/fd6 [2566500,71688] 0 2024-03-24T10:46:21.306 INFO:tasks.workunit.client.1.smithi096.stdout:9/824: rename d9/d7c/lad to d9/d15/d26/d29/d73/d49/db7/dca/l10b 0 2024-03-24T10:46:21.310 INFO:tasks.workunit.client.1.smithi096.stdout:7/981: symlink d0/db/d4f/d125/l13e 0 2024-03-24T10:46:21.314 INFO:tasks.workunit.client.1.smithi096.stdout:6/789: rmdir d8/d12/d61 39 2024-03-24T10:46:21.316 INFO:tasks.workunit.client.1.smithi096.stdout:9/825: dread d9/db/fe [0,4194304] 0 2024-03-24T10:46:21.316 INFO:tasks.workunit.client.1.smithi096.stdout:9/826: stat d9/db/d6d/d4f/d75/dcc 0 2024-03-24T10:46:21.316 INFO:tasks.workunit.client.1.smithi096.stdout:9/827: stat d9/d15/d26/d29/d73/f101 0 2024-03-24T10:46:21.316 INFO:tasks.workunit.client.1.smithi096.stdout:1/734: dread d3/d23/d8b/d79/fd1 [0,4194304] 0 2024-03-24T10:46:21.316 INFO:tasks.workunit.client.1.smithi096.stdout:8/852: rmdir d8/d11/d22/d4c/d65/d52/dc0/dd9/d10a 39 2024-03-24T10:46:21.316 INFO:tasks.workunit.client.1.smithi096.stdout:1/735: creat d3/d23/d71/ff8 x:0 0 0 2024-03-24T10:46:21.316 INFO:tasks.workunit.client.1.smithi096.stdout:1/736: chown d3/d23/d2f/d65/la0 0 1 2024-03-24T10:46:21.316 INFO:tasks.workunit.client.1.smithi096.stdout:1/737: fdatasync d3/d23/d71/dbe/fc9 0 2024-03-24T10:46:21.316 INFO:tasks.workunit.client.1.smithi096.stdout:1/738: chown d3/d23/d2f/d65/d7c/ddf/df2 0 1 2024-03-24T10:46:21.318 INFO:tasks.workunit.client.1.smithi096.stdout:8/853: dread d8/d11/d22/d4c/d67/f7a [0,4194304] 0 2024-03-24T10:46:21.323 INFO:tasks.workunit.client.1.smithi096.stdout:0/895: dwrite d0/d11/d24/f2d [0,4194304] 0 2024-03-24T10:46:21.323 INFO:tasks.workunit.client.1.smithi096.stdout:0/896: write d0/d11/dc3/f103 [4022645,120137] 0 2024-03-24T10:46:21.323 INFO:tasks.workunit.client.1.smithi096.stdout:4/707: creat d1/d2b/d59/de0/fe5 x:0 0 0 2024-03-24T10:46:21.333 INFO:tasks.workunit.client.1.smithi096.stdout:5/809: unlink d2/lc7 0 2024-03-24T10:46:21.334 INFO:tasks.workunit.client.1.smithi096.stdout:9/828: dwrite d9/db/f8c [0,4194304] 0 2024-03-24T10:46:21.334 INFO:tasks.workunit.client.1.smithi096.stdout:9/829: write d9/db/d6d/f35 [175729,11512] 0 2024-03-24T10:46:21.334 INFO:tasks.workunit.client.1.smithi096.stdout:9/830: creat d9/d7c/f10c x:0 0 0 2024-03-24T10:46:21.334 INFO:tasks.workunit.client.1.smithi096.stdout:9/831: chown d9/d15/d26/d29/d73/d49/l5f 1202 1 2024-03-24T10:46:21.334 INFO:tasks.workunit.client.1.smithi096.stdout:9/832: chown d9/d15/ca9 65771999 1 2024-03-24T10:46:21.334 INFO:tasks.workunit.client.1.smithi096.stdout:2/962: mknod d5/dc/d6b/dcf/c161 0 2024-03-24T10:46:21.334 INFO:tasks.workunit.client.1.smithi096.stdout:2/963: stat d5/d7c/f99 0 2024-03-24T10:46:21.337 INFO:tasks.workunit.client.1.smithi096.stdout:7/982: unlink d0/d13/df0/d3c/c46 0 2024-03-24T10:46:21.337 INFO:tasks.workunit.client.1.smithi096.stdout:7/983: creat d0/f13f x:0 0 0 2024-03-24T10:46:21.340 INFO:tasks.workunit.client.1.smithi096.stdout:6/790: creat d8/d12/d28/d36/d65/d71/dd0/da8/dbe/ff9 x:0 0 0 2024-03-24T10:46:21.340 INFO:tasks.workunit.client.1.smithi096.stdout:6/791: truncate d8/d12/d32/d38/d59/db8/ff5 565409 0 2024-03-24T10:46:21.340 INFO:tasks.workunit.client.1.smithi096.stdout:6/792: read f2 [1582894,96803] 0 2024-03-24T10:46:21.340 INFO:tasks.workunit.client.1.smithi096.stdout:1/739: dwrite d3/d84/fe3 [0,4194304] 0 2024-03-24T10:46:21.344 INFO:tasks.workunit.client.1.smithi096.stdout:8/854: unlink d8/d11/d22/d4c/d65/d1a/d46/f10b 0 2024-03-24T10:46:21.344 INFO:tasks.workunit.client.1.smithi096.stdout:8/855: fsync d8/d72/f105 0 2024-03-24T10:46:21.345 INFO:tasks.workunit.client.1.smithi096.stdout:6/793: dread d8/d12/d32/d68/fcb [0,4194304] 0 2024-03-24T10:46:21.347 INFO:tasks.workunit.client.1.smithi096.stdout:0/897: creat d0/d11/d24/d9d/df5/f131 x:0 0 0 2024-03-24T10:46:21.347 INFO:tasks.workunit.client.1.smithi096.stdout:0/898: write d0/d11/d24/f4c [2964590,19685] 0 2024-03-24T10:46:21.347 INFO:tasks.workunit.client.1.smithi096.stdout:0/899: chown d0/d11/d24/lb7 66 1 2024-03-24T10:46:21.347 INFO:tasks.workunit.client.1.smithi096.stdout:7/984: dread d0/d13/df0/d59/d87/f11e [0,4194304] 0 2024-03-24T10:46:21.347 INFO:tasks.workunit.client.1.smithi096.stdout:0/900: write d0/d11/d16/d21/d81/d9c/dcd/d10a/ff7 [32598,51424] 0 2024-03-24T10:46:21.347 INFO:tasks.workunit.client.1.smithi096.stdout:7/985: chown d0/c6b 12078 1 2024-03-24T10:46:21.348 INFO:tasks.workunit.client.1.smithi096.stdout:4/708: rmdir d1/d2b/d67/d5f 39 2024-03-24T10:46:21.348 INFO:tasks.workunit.client.1.smithi096.stdout:5/810: dwrite d2/d7/d8/d18/db6/fd1 [0,4194304] 0 2024-03-24T10:46:21.348 INFO:tasks.workunit.client.1.smithi096.stdout:5/811: write d2/d29/d32/d33/d3c/dd2/fd3 [970475,83567] 0 2024-03-24T10:46:21.352 INFO:tasks.workunit.client.1.smithi096.stdout:9/833: symlink d9/d15/l10d 0 2024-03-24T10:46:21.353 INFO:tasks.workunit.client.1.smithi096.stdout:2/964: rename d5/dc/f2f to d5/d7c/d124/f162 0 2024-03-24T10:46:21.362 INFO:tasks.workunit.client.1.smithi096.stdout:1/740: creat d3/d23/d71/d85/ff9 x:0 0 0 2024-03-24T10:46:21.362 INFO:tasks.workunit.client.1.smithi096.stdout:1/741: write d3/d23/f42 [1040464,51062] 0 2024-03-24T10:46:21.362 INFO:tasks.workunit.client.1.smithi096.stdout:1/742: creat d3/d23/d8b/d8c/ddc/ffa x:0 0 0 2024-03-24T10:46:21.362 INFO:tasks.workunit.client.1.smithi096.stdout:1/743: dread - d3/d23/d71/fb8 zero size 2024-03-24T10:46:21.364 INFO:tasks.workunit.client.1.smithi096.stdout:8/856: symlink d8/d11/d22/d4c/d65/d52/l111 0 2024-03-24T10:46:21.366 INFO:tasks.workunit.client.1.smithi096.stdout:6/794: rmdir d8/d10/d13/df3 0 2024-03-24T10:46:21.367 INFO:tasks.workunit.client.1.smithi096.stdout:2/965: dwrite d5/d79/d9f/f123 [0,4194304] 0 2024-03-24T10:46:21.370 INFO:tasks.workunit.client.1.smithi096.stdout:7/986: creat d0/d13/df0/d3c/d68/f140 x:0 0 0 2024-03-24T10:46:21.370 INFO:tasks.workunit.client.1.smithi096.stdout:1/744: dread d3/d23/f42 [0,4194304] 0 2024-03-24T10:46:21.373 INFO:tasks.workunit.client.1.smithi096.stdout:0/901: creat d0/d11/d16/d21/d81/d8f/d9a/ddd/f132 x:0 0 0 2024-03-24T10:46:21.374 INFO:tasks.workunit.client.1.smithi096.stdout:4/709: unlink d1/d2b/d6b/f83 0 2024-03-24T10:46:21.374 INFO:tasks.workunit.client.1.smithi096.stdout:4/710: dread - d1/d2b/d6b/d80/fd1 zero size 2024-03-24T10:46:21.377 INFO:tasks.workunit.client.1.smithi096.stdout:5/812: creat d2/d7/dba/f114 x:0 0 0 2024-03-24T10:46:21.377 INFO:tasks.workunit.client.1.smithi096.stdout:5/813: fsync d2/d7/d8/d18/ffa 0 2024-03-24T10:46:21.379 INFO:tasks.workunit.client.1.smithi096.stdout:9/834: creat d9/f10e x:0 0 0 2024-03-24T10:46:21.384 INFO:tasks.workunit.client.1.smithi096.stdout:6/795: rename d8/d10/f3e to d8/d12/d28/d36/d65/d6b/dec/ffa 0 2024-03-24T10:46:21.391 INFO:tasks.workunit.client.1.smithi096.stdout:8/857: dwrite d8/d11/d22/d4c/d65/d1a/d49/f6a [0,4194304] 0 2024-03-24T10:46:21.391 INFO:tasks.workunit.client.1.smithi096.stdout:1/745: mknod d3/d23/d8b/d8c/cfb 0 2024-03-24T10:46:21.393 INFO:tasks.workunit.client.1.smithi096.stdout:5/814: link d2/d7/d8/df/d14/f23 d2/d29/d32/d33/d8d/df3/f115 0 2024-03-24T10:46:21.393 INFO:tasks.workunit.client.1.smithi096.stdout:5/815: stat d2/d7/d69/de4/deb 0 2024-03-24T10:46:21.395 INFO:tasks.workunit.client.1.smithi096.stdout:6/796: symlink d8/d10/d13/dcc/lfb 0 2024-03-24T10:46:21.397 INFO:tasks.workunit.client.1.smithi096.stdout:1/746: mkdir d3/d23/d71/d7b/dce/dfc 0 2024-03-24T10:46:21.399 INFO:tasks.workunit.client.1.smithi096.stdout:5/816: mkdir d2/d29/d32/d33/d116 0 2024-03-24T10:46:21.399 INFO:tasks.workunit.client.1.smithi096.stdout:5/817: write d2/d7/d69/dce/df0/f10e [826025,102859] 0 2024-03-24T10:46:21.399 INFO:tasks.workunit.client.1.smithi096.stdout:5/818: chown d2/d29/d32/d3f 15 1 2024-03-24T10:46:21.401 INFO:tasks.workunit.client.1.smithi096.stdout:1/747: creat d3/d23/d8b/d5c/d6d/ffd x:0 0 0 2024-03-24T10:46:21.401 INFO:tasks.workunit.client.1.smithi096.stdout:1/748: chown d3/f66 2829 1 2024-03-24T10:46:21.402 INFO:tasks.workunit.client.1.smithi096.stdout:5/819: symlink d2/d29/d32/db9/l117 0 2024-03-24T10:46:21.403 INFO:tasks.workunit.client.1.smithi096.stdout:6/797: dread d8/d12/d28/d36/d65/d71/dd0/da8/dbe/ff7 [0,4194304] 0 2024-03-24T10:46:21.404 INFO:tasks.workunit.client.1.smithi096.stdout:8/858: write d8/f14 [3595373,121662] 0 2024-03-24T10:46:21.405 INFO:tasks.workunit.client.1.smithi096.stdout:1/749: symlink d3/d23/d2f/d65/d7c/lfe 0 2024-03-24T10:46:21.405 INFO:tasks.workunit.client.1.smithi096.stdout:1/750: creat d3/d23/d2f/d65/d7c/ddf/fff x:0 0 0 2024-03-24T10:46:21.405 INFO:tasks.workunit.client.1.smithi096.stdout:1/751: truncate d3/d23/d2f/fc4 587671 0 2024-03-24T10:46:21.405 INFO:tasks.workunit.client.1.smithi096.stdout:1/752: dread - d3/d36/fa3 zero size 2024-03-24T10:46:21.405 INFO:tasks.workunit.client.1.smithi096.stdout:1/753: fdatasync d3/d23/d2f/d65/d7c/ddf/ff7 0 2024-03-24T10:46:21.406 INFO:tasks.workunit.client.1.smithi096.stdout:4/711: dwrite d1/d32/d4c/d7e/fd4 [0,4194304] 0 2024-03-24T10:46:21.406 INFO:tasks.workunit.client.1.smithi096.stdout:7/987: dwrite d0/db/d4f/f58 [0,4194304] 0 2024-03-24T10:46:21.406 INFO:tasks.workunit.client.1.smithi096.stdout:2/966: dwrite d5/d79/d9f/dad/fdf [0,4194304] 0 2024-03-24T10:46:21.406 INFO:tasks.workunit.client.1.smithi096.stdout:5/820: creat d2/d7/d8/df/d8a/d10a/f118 x:0 0 0 2024-03-24T10:46:21.406 INFO:tasks.workunit.client.1.smithi096.stdout:5/821: write d2/d29/d32/d33/d37/fe8 [1389258,94477] 0 2024-03-24T10:46:21.408 INFO:tasks.workunit.client.1.smithi096.stdout:9/835: dwrite d9/f5c [4194304,4194304] 0 2024-03-24T10:46:21.411 INFO:tasks.workunit.client.1.smithi096.stdout:6/798: unlink d8/d12/d32/d38/d59/db8/le9 0 2024-03-24T10:46:21.411 INFO:tasks.workunit.client.1.smithi096.stdout:6/799: write d8/d12/d32/d38/d59/fcf [882108,126229] 0 2024-03-24T10:46:21.414 INFO:tasks.workunit.client.1.smithi096.stdout:8/859: rename d8/d11/f7f to d8/d11/d22/d4c/d65/d1a/da0/da6/f112 0 2024-03-24T10:46:21.414 INFO:tasks.workunit.client.1.smithi096.stdout:0/902: dwrite d0/d11/d24/f50 [0,4194304] 0 2024-03-24T10:46:21.418 INFO:tasks.workunit.client.1.smithi096.stdout:1/754: chown d3/d23/d8b/d5c/f67 7255125 1 2024-03-24T10:46:21.420 INFO:tasks.workunit.client.1.smithi096.stdout:8/860: dread d8/d11/d22/d4c/d65/d1a/d46/f10e [0,4194304] 0 2024-03-24T10:46:21.420 INFO:tasks.workunit.client.1.smithi096.stdout:9/836: dread d9/d15/d26/d29/d73/f91 [0,4194304] 0 2024-03-24T10:46:21.421 INFO:tasks.workunit.client.1.smithi096.stdout:9/837: write d9/db/d6d/f35 [4132714,23604] 0 2024-03-24T10:46:21.422 INFO:tasks.workunit.client.1.smithi096.stdout:2/967: rmdir d5/d93/d115 39 2024-03-24T10:46:21.424 INFO:tasks.workunit.client.1.smithi096.stdout:5/822: dwrite d2/d7/fdc [4194304,4194304] 0 2024-03-24T10:46:21.426 INFO:tasks.workunit.client.1.smithi096.stdout:6/800: unlink d8/c8b 0 2024-03-24T10:46:21.429 INFO:tasks.workunit.client.1.smithi096.stdout:4/712: rename d1/d32/d4c/dc1 to d1/d2b/d40/de6 0 2024-03-24T10:46:21.434 INFO:tasks.workunit.client.1.smithi096.stdout:5/823: dread d2/d29/d32/d33/d37/f6d [0,4194304] 0 2024-03-24T10:46:21.459 INFO:tasks.workunit.client.1.smithi096.stdout:0/903: mknod d0/d11/d16/d21/d78/c133 0 2024-03-24T10:46:21.460 INFO:tasks.workunit.client.1.smithi096.stdout:1/755: truncate d3/d23/d8b/d5c/fa5 5444978 0 2024-03-24T10:46:21.460 INFO:tasks.workunit.client.1.smithi096.stdout:1/756: chown d3/d36/f54 396826882 1 2024-03-24T10:46:21.460 INFO:tasks.workunit.client.1.smithi096.stdout:0/904: dread d0/f72 [0,4194304] 0 2024-03-24T10:46:21.462 INFO:tasks.workunit.client.1.smithi096.stdout:1/757: dread d3/d23/d8b/fbd [0,4194304] 0 2024-03-24T10:46:21.462 INFO:tasks.workunit.client.1.smithi096.stdout:1/758: stat d3/d23/f62 0 2024-03-24T10:46:21.463 INFO:tasks.workunit.client.1.smithi096.stdout:9/838: link d9/d15/d26/d29/f4e d9/d15/d26/d29/d73/df4/f10f 0 2024-03-24T10:46:21.463 INFO:tasks.workunit.client.1.smithi096.stdout:9/839: dread - d9/d15/d26/d29/d73/d43/d48/dc0/dd8/fe9 zero size 2024-03-24T10:46:21.463 INFO:tasks.workunit.client.1.smithi096.stdout:9/840: fdatasync d9/db/d6d/f56 0 2024-03-24T10:46:21.464 INFO:tasks.workunit.client.1.smithi096.stdout:2/968: mknod d5/de/d2a/df3/c163 0 2024-03-24T10:46:21.466 INFO:tasks.workunit.client.1.smithi096.stdout:6/801: mkdir d8/d10/d13/d63/d51/dfc 0 2024-03-24T10:46:21.467 INFO:tasks.workunit.client.1.smithi096.stdout:6/802: chown d8/d12/d32/d38/d59/db8/fbb 1437025 1 2024-03-24T10:46:21.467 INFO:tasks.workunit.client.1.smithi096.stdout:6/803: stat d8/d10/d13/d63/c20 0 2024-03-24T10:46:21.470 INFO:tasks.workunit.client.1.smithi096.stdout:5/824: rmdir d2/d29/d32/d3f/d5f/dc0 39 2024-03-24T10:46:21.471 INFO:tasks.workunit.client.1.smithi096.stdout:0/905: dread d0/d11/d16/d21/d81/d7f/f5e [0,4194304] 0 2024-03-24T10:46:21.471 INFO:tasks.workunit.client.1.smithi096.stdout:1/759: dread d3/d84/fd7 [0,4194304] 0 2024-03-24T10:46:21.473 INFO:tasks.workunit.client.1.smithi096.stdout:0/906: dread d0/d11/d24/d9d/fa7 [0,4194304] 0 2024-03-24T10:46:21.473 INFO:tasks.workunit.client.1.smithi096.stdout:9/841: unlink d9/d15/d26/d29/d73/d49/db7/dc1/cda 0 2024-03-24T10:46:21.474 INFO:tasks.workunit.client.1.smithi096.stdout:0/907: dread d0/d11/d20/d9e/d80/fed [0,4194304] 0 2024-03-24T10:46:21.476 INFO:tasks.workunit.client.1.smithi096.stdout:3/779: sync 2024-03-24T10:46:21.477 INFO:tasks.workunit.client.1.smithi096.stdout:2/969: mknod d5/dc/d12/d23/d105/c164 0 2024-03-24T10:46:21.477 INFO:tasks.workunit.client.1.smithi096.stdout:2/970: write d5/dc/d12/f41 [113345,91727] 0 2024-03-24T10:46:21.478 INFO:tasks.workunit.client.1.smithi096.stdout:2/971: chown d5/dc/d6b/dcf/f11d 3234061 1 2024-03-24T10:46:21.480 INFO:tasks.workunit.client.1.smithi096.stdout:1/760: dread d3/ff [0,4194304] 0 2024-03-24T10:46:21.483 INFO:tasks.workunit.client.1.smithi096.stdout:6/804: dread d8/d10/d76/f8d [0,4194304] 0 2024-03-24T10:46:21.483 INFO:tasks.workunit.client.1.smithi096.stdout:6/805: truncate d8/d10/d19/f60 4565891 0 2024-03-24T10:46:21.484 INFO:tasks.workunit.client.1.smithi096.stdout:5/825: unlink d2/d29/d32/d33/d3c/dd2/cf6 0 2024-03-24T10:46:21.489 INFO:tasks.workunit.client.1.smithi096.stdout:9/842: symlink d9/db/d6d/dc6/l110 0 2024-03-24T10:46:21.490 INFO:tasks.workunit.client.1.smithi096.stdout:3/780: link d0/d13/d1e/da4/cae d0/d13/d45/d7d/d39/d48/cfd 0 2024-03-24T10:46:21.490 INFO:tasks.workunit.client.1.smithi096.stdout:3/781: fsync d0/d13/d41/ddf/fd4 0 2024-03-24T10:46:21.495 INFO:tasks.workunit.client.1.smithi096.stdout:7/988: rename d0/d13/df0/d59/d87 to d0/d13/df0/d3c/d68/d141 0 2024-03-24T10:46:21.497 INFO:tasks.workunit.client.1.smithi096.stdout:5/826: mkdir d2/d29/d32/d89/d119 0 2024-03-24T10:46:21.500 INFO:tasks.workunit.client.1.smithi096.stdout:9/843: mknod d9/db/d6d/c111 0 2024-03-24T10:46:21.501 INFO:tasks.workunit.client.1.smithi096.stdout:8/861: dwrite d8/d11/f19 [0,4194304] 0 2024-03-24T10:46:21.502 INFO:tasks.workunit.client.1.smithi096.stdout:8/862: write d8/f14 [4321692,64001] 0 2024-03-24T10:46:21.502 INFO:tasks.workunit.client.1.smithi096.stdout:8/863: truncate d8/d72/f105 235283 0 2024-03-24T10:46:21.502 INFO:tasks.workunit.client.1.smithi096.stdout:6/806: dwrite d8/d10/d58/f85 [0,4194304] 0 2024-03-24T10:46:21.502 INFO:tasks.workunit.client.1.smithi096.stdout:3/782: symlink d0/d92/ddc/dfc/de5/lfe 0 2024-03-24T10:46:21.502 INFO:tasks.workunit.client.1.smithi096.stdout:3/783: creat d0/d13/d45/d7d/fff x:0 0 0 2024-03-24T10:46:21.503 INFO:tasks.workunit.client.1.smithi096.stdout:3/784: dread - d0/d13/d45/d7d/fff zero size 2024-03-24T10:46:21.503 INFO:tasks.workunit.client.1.smithi096.stdout:1/761: mknod d3/d84/c100 0 2024-03-24T10:46:21.503 INFO:tasks.workunit.client.1.smithi096.stdout:1/762: chown d3/c8f 521274387 1 2024-03-24T10:46:21.505 INFO:tasks.workunit.client.1.smithi096.stdout:4/713: rename d1/d32/d4c/d7d/le4 to d1/d32/le7 0 2024-03-24T10:46:21.508 INFO:tasks.workunit.client.1.smithi096.stdout:7/989: truncate d0/d139/f12c 3869984 0 2024-03-24T10:46:21.509 INFO:tasks.workunit.client.1.smithi096.stdout:9/844: mknod d9/db/d6d/d4f/c112 0 2024-03-24T10:46:21.509 INFO:tasks.workunit.client.1.smithi096.stdout:9/845: dread - d9/db/d6d/fc2 zero size 2024-03-24T10:46:21.509 INFO:tasks.workunit.client.1.smithi096.stdout:9/846: dread - d9/d15/d26/d29/d73/d49/d86/fcf zero size 2024-03-24T10:46:21.513 INFO:tasks.workunit.client.1.smithi096.stdout:8/864: rmdir d8/d11/d22/d4c/db0 39 2024-03-24T10:46:21.518 INFO:tasks.workunit.client.1.smithi096.stdout:3/785: dwrite d0/d9/fbe [0,4194304] 0 2024-03-24T10:46:21.518 INFO:tasks.workunit.client.1.smithi096.stdout:3/786: fsync d0/d13/d45/d47/f76 0 2024-03-24T10:46:21.519 INFO:tasks.workunit.client.1.smithi096.stdout:3/787: stat d0/d9/lf9 0 2024-03-24T10:46:21.520 INFO:tasks.workunit.client.1.smithi096.stdout:6/807: truncate d8/d10/f3d 6973006 0 2024-03-24T10:46:21.520 INFO:tasks.workunit.client.1.smithi096.stdout:6/808: fsync d8/d10/d13/fc5 0 2024-03-24T10:46:21.526 INFO:tasks.workunit.client.1.smithi096.stdout:1/763: symlink d3/d23/d2f/l101 0 2024-03-24T10:46:21.530 INFO:tasks.workunit.client.1.smithi096.stdout:0/908: rename d0/d11/d43/la1 to d0/d11/d20/d9e/d80/d110/l134 0 2024-03-24T10:46:21.531 INFO:tasks.workunit.client.1.smithi096.stdout:4/714: symlink d1/d32/d4c/d9d/le8 0 2024-03-24T10:46:21.531 INFO:tasks.workunit.client.1.smithi096.stdout:4/715: dread - d1/d32/d42/f79 zero size 2024-03-24T10:46:21.531 INFO:tasks.workunit.client.1.smithi096.stdout:4/716: chown d1/d11/c98 21251 1 2024-03-24T10:46:21.531 INFO:tasks.workunit.client.1.smithi096.stdout:4/717: chown d1/d32/l8f 3638836 1 2024-03-24T10:46:21.533 INFO:tasks.workunit.client.1.smithi096.stdout:9/847: creat d9/d65/f113 x:0 0 0 2024-03-24T10:46:21.535 INFO:tasks.workunit.client.1.smithi096.stdout:8/865: link d8/d11/d22/d4c/d65/d1d/f3b d8/d11/d22/d4c/d65/d1a/d49/f113 0 2024-03-24T10:46:21.538 INFO:tasks.workunit.client.1.smithi096.stdout:1/764: symlink d3/d23/d71/d7b/dce/de1/l102 0 2024-03-24T10:46:21.538 INFO:tasks.workunit.client.1.smithi096.stdout:1/765: stat d3/d23/d8b/lf3 0 2024-03-24T10:46:21.538 INFO:tasks.workunit.client.1.smithi096.stdout:1/766: write d3/d23/d71/dbe/fc9 [749356,21889] 0 2024-03-24T10:46:21.539 INFO:tasks.workunit.client.1.smithi096.stdout:8/866: dread d8/d11/d22/d4c/d65/d1a/f26 [0,4194304] 0 2024-03-24T10:46:21.539 INFO:tasks.workunit.client.1.smithi096.stdout:6/809: dwrite d8/d10/d13/dcc/fda [0,4194304] 0 2024-03-24T10:46:21.540 INFO:tasks.workunit.client.1.smithi096.stdout:3/788: dwrite d0/d9/f5e [0,4194304] 0 2024-03-24T10:46:21.540 INFO:tasks.workunit.client.1.smithi096.stdout:3/789: chown d0/c35 2544 1 2024-03-24T10:46:21.541 INFO:tasks.workunit.client.1.smithi096.stdout:2/972: rename d5/d38/f66 to d5/d7c/d88/d114/f165 0 2024-03-24T10:46:21.544 INFO:tasks.workunit.client.1.smithi096.stdout:4/718: mkdir d1/d2b/d89/dbf/de9 0 2024-03-24T10:46:21.544 INFO:tasks.workunit.client.1.smithi096.stdout:4/719: creat d1/d2b/d6b/d80/fea x:0 0 0 2024-03-24T10:46:21.549 INFO:tasks.workunit.client.1.smithi096.stdout:1/767: mkdir d3/d23/d71/d7b/dce/d103 0 2024-03-24T10:46:21.550 INFO:tasks.workunit.client.1.smithi096.stdout:9/848: read d9/f18 [292977,15796] 0 2024-03-24T10:46:21.554 INFO:tasks.workunit.client.1.smithi096.stdout:8/867: unlink d8/d11/d44/ld5 0 2024-03-24T10:46:21.555 INFO:tasks.workunit.client.1.smithi096.stdout:6/810: mkdir d8/d10/d19/dfd 0 2024-03-24T10:46:21.560 INFO:tasks.workunit.client.1.smithi096.stdout:5/827: rename d2/d29/d32/dbb/fe7 to d2/d29/d32/d33/d8d/f11a 0 2024-03-24T10:46:21.562 INFO:tasks.workunit.client.1.smithi096.stdout:8/868: write d8/d11/d22/d4c/d82/f5e [1087394,80596] 0 2024-03-24T10:46:21.563 INFO:tasks.workunit.client.1.smithi096.stdout:8/869: stat d8/d11/d22/d4c/d65/d52/dc0/dd9/d10a/f10d 0 2024-03-24T10:46:21.563 INFO:tasks.workunit.client.1.smithi096.stdout:8/870: readlink d8/d11/d44/d45/la8 0 2024-03-24T10:46:21.564 INFO:tasks.workunit.client.1.smithi096.stdout:6/811: dread d8/d12/f5c [0,4194304] 0 2024-03-24T10:46:21.565 INFO:tasks.workunit.client.1.smithi096.stdout:4/720: unlink d1/d2b/d6b/d80/d9c/fa1 0 2024-03-24T10:46:21.565 INFO:tasks.workunit.client.1.smithi096.stdout:4/721: creat d1/d2b/d40/feb x:0 0 0 2024-03-24T10:46:21.565 INFO:tasks.workunit.client.1.smithi096.stdout:2/973: dwrite d5/d93/ff8 [0,4194304] 0 2024-03-24T10:46:21.568 INFO:tasks.workunit.client.1.smithi096.stdout:1/768: unlink d3/d23/d2f/d65/d7c/ff4 0 2024-03-24T10:46:21.571 INFO:tasks.workunit.client.1.smithi096.stdout:1/769: write d3/d36/fd5 [267429,31332] 0 2024-03-24T10:46:21.571 INFO:tasks.workunit.client.1.smithi096.stdout:9/849: dwrite d9/d7c/f9c [0,4194304] 0 2024-03-24T10:46:21.571 INFO:tasks.workunit.client.1.smithi096.stdout:9/850: fdatasync d9/d15/d26/d29/d73/d49/f50 0 2024-03-24T10:46:21.571 INFO:tasks.workunit.client.1.smithi096.stdout:1/770: write d3/d23/d8b/d8c/ddc/fde [780087,113799] 0 2024-03-24T10:46:21.571 INFO:tasks.workunit.client.1.smithi096.stdout:7/990: rename d0/d111/f13a to d0/db/d26/d48/f142 0 2024-03-24T10:46:21.571 INFO:tasks.workunit.client.1.smithi096.stdout:7/991: chown d0/f38 53423 1 2024-03-24T10:46:21.571 INFO:tasks.workunit.client.1.smithi096.stdout:7/992: write d0/d139/f129 [768941,36306] 0 2024-03-24T10:46:21.571 INFO:tasks.workunit.client.1.smithi096.stdout:7/993: write d0/db/d4f/fb7 [4744782,123209] 0 2024-03-24T10:46:21.572 INFO:tasks.workunit.client.1.smithi096.stdout:5/828: creat d2/d7/d8/df/d14/f11b x:0 0 0 2024-03-24T10:46:21.573 INFO:tasks.workunit.client.1.smithi096.stdout:6/812: mkdir d8/d12/d61/dfe 0 2024-03-24T10:46:21.579 INFO:tasks.workunit.client.1.smithi096.stdout:2/974: mkdir d5/dc/d6b/d72/d14c/d166 0 2024-03-24T10:46:21.584 INFO:tasks.workunit.client.1.smithi096.stdout:9/851: mkdir d9/df1/dd9/d114 0 2024-03-24T10:46:21.588 INFO:tasks.workunit.client.1.smithi096.stdout:1/771: symlink d3/d23/d2f/d65/d7c/ddf/df2/l104 0 2024-03-24T10:46:21.589 INFO:tasks.workunit.client.1.smithi096.stdout:6/813: dwrite d8/d10/f8a [0,4194304] 0 2024-03-24T10:46:21.590 INFO:tasks.workunit.client.1.smithi096.stdout:0/909: rename d0/d11/c3c to d0/d11/d16/d21/d81/d9c/dcd/c135 0 2024-03-24T10:46:21.591 INFO:tasks.workunit.client.1.smithi096.stdout:0/910: readlink d0/d11/d16/d21/d81/d7f/le4 0 2024-03-24T10:46:21.591 INFO:tasks.workunit.client.1.smithi096.stdout:0/911: readlink d0/d11/d24/l86 0 2024-03-24T10:46:21.597 INFO:tasks.workunit.client.1.smithi096.stdout:5/829: link d2/d7/f52 d2/d29/df7/f11c 0 2024-03-24T10:46:21.601 INFO:tasks.workunit.client.1.smithi096.stdout:2/975: mkdir d5/d79/de8/d167 0 2024-03-24T10:46:21.601 INFO:tasks.workunit.client.1.smithi096.stdout:6/814: dread d8/d10/d19/f96 [0,4194304] 0 2024-03-24T10:46:21.603 INFO:tasks.workunit.client.1.smithi096.stdout:1/772: link d3/d23/d71/f8a d3/d73/f105 0 2024-03-24T10:46:21.606 INFO:tasks.workunit.client.1.smithi096.stdout:3/790: rename d0/c3f to d0/d13/d45/d7d/d39/d48/dbd/c100 0 2024-03-24T10:46:21.609 INFO:tasks.workunit.client.1.smithi096.stdout:5/830: mknod d2/d29/d32/dbb/c11d 0 2024-03-24T10:46:21.609 INFO:tasks.workunit.client.1.smithi096.stdout:5/831: fsync d2/d29/d32/d33/d3c/dd2/fda 0 2024-03-24T10:46:21.610 INFO:tasks.workunit.client.1.smithi096.stdout:9/852: mknod d9/d15/d26/d29/d73/d49/db7/c115 0 2024-03-24T10:46:21.615 INFO:tasks.workunit.client.1.smithi096.stdout:8/871: rename d8/d11/d22/d4c/d65/d1d/f9a to d8/d11/d22/d4c/d65/d1a/da0/df6/f114 0 2024-03-24T10:46:21.616 INFO:tasks.workunit.client.1.smithi096.stdout:8/872: stat d8/d72/d96/fba 0 2024-03-24T10:46:21.617 INFO:tasks.workunit.client.1.smithi096.stdout:3/791: dread d0/d9/f5e [0,4194304] 0 2024-03-24T10:46:21.617 INFO:tasks.workunit.client.1.smithi096.stdout:0/912: rmdir d0/d11/d16/d21/d78/d92 39 2024-03-24T10:46:21.619 INFO:tasks.workunit.client.1.smithi096.stdout:2/976: dwrite d5/de/f51 [0,4194304] 0 2024-03-24T10:46:21.634 INFO:tasks.workunit.client.1.smithi096.stdout:6/815: dwrite d8/d10/d13/d63/f2c [0,4194304] 0 2024-03-24T10:46:21.635 INFO:tasks.workunit.client.1.smithi096.stdout:5/832: dwrite d2/d29/d32/d33/d37/dab/fed [0,4194304] 0 2024-03-24T10:46:21.649 INFO:tasks.workunit.client.1.smithi096.stdout:0/913: dwrite d0/d11/f6d [0,4194304] 0 2024-03-24T10:46:21.649 INFO:tasks.workunit.client.1.smithi096.stdout:2/977: dwrite d5/dc/d6b/d72/d91/fd0 [0,4194304] 0 2024-03-24T10:46:21.649 INFO:tasks.workunit.client.1.smithi096.stdout:2/978: write d5/d38/f7b [378617,96589] 0 2024-03-24T10:46:21.672 INFO:tasks.workunit.client.1.smithi096.stdout:9/853: creat d9/d15/d26/d29/d73/d43/d48/f116 x:0 0 0 2024-03-24T10:46:21.673 INFO:tasks.workunit.client.1.smithi096.stdout:9/854: creat d9/d7c/f117 x:0 0 0 2024-03-24T10:46:21.673 INFO:tasks.workunit.client.1.smithi096.stdout:7/994: write d0/d13/df0/d3c/d80/fcf [1136483,124848] 0 2024-03-24T10:46:21.674 INFO:tasks.workunit.client.1.smithi096.stdout:1/773: getdents d3/d23/d8b/d5c/d9c 0 2024-03-24T10:46:21.687 INFO:tasks.workunit.client.1.smithi096.stdout:4/722: rename d1/d2b/d40/fc5 to d1/d2b/d67/d5f/fec 0 2024-03-24T10:46:21.687 INFO:tasks.workunit.client.1.smithi096.stdout:2/979: rename d5/d7c to d5/d7c/d8f/d128/d168 22 2024-03-24T10:46:21.687 INFO:tasks.workunit.client.1.smithi096.stdout:2/980: chown d5/dc/d12/d23/l4e 7440 1 2024-03-24T10:46:21.688 INFO:tasks.workunit.client.1.smithi096.stdout:2/981: read d5/dc/d12/f4a [4461654,67734] 0 2024-03-24T10:46:21.688 INFO:tasks.workunit.client.1.smithi096.stdout:2/982: write d5/de/d2a/d5e/f65 [573449,86587] 0 2024-03-24T10:46:21.692 INFO:tasks.workunit.client.1.smithi096.stdout:3/792: creat d0/d13/df1/f101 x:0 0 0 2024-03-24T10:46:21.696 INFO:tasks.workunit.client.1.smithi096.stdout:1/774: dwrite d3/d23/d2f/d65/fb4 [0,4194304] 0 2024-03-24T10:46:21.696 INFO:tasks.workunit.client.1.smithi096.stdout:1/775: dread - d3/d23/d71/fb8 zero size 2024-03-24T10:46:21.698 INFO:tasks.workunit.client.1.smithi096.stdout:7/995: dwrite d0/db/d26/d11b/f13b [0,4194304] 0 2024-03-24T10:46:21.708 INFO:tasks.workunit.client.1.smithi096.stdout:3/793: dwrite d0/d13/d45/d7d/d39/d48/f6a [4194304,4194304] 0 2024-03-24T10:46:21.709 INFO:tasks.workunit.client.1.smithi096.stdout:4/723: dwrite d1/d32/d4c/d7e/fd4 [0,4194304] 0 2024-03-24T10:46:21.709 INFO:tasks.workunit.client.1.smithi096.stdout:4/724: write d1/f8 [2984105,13893] 0 2024-03-24T10:46:21.709 INFO:tasks.workunit.client.1.smithi096.stdout:7/996: dread d0/d139/f12c [0,4194304] 0 2024-03-24T10:46:21.710 INFO:tasks.workunit.client.1.smithi096.stdout:5/833: creat d2/d7/d8/d18/db6/ddd/d103/f11e x:0 0 0 2024-03-24T10:46:21.710 INFO:tasks.workunit.client.1.smithi096.stdout:5/834: truncate d2/d7/d8/df/d14/f11b 686246 0 2024-03-24T10:46:21.731 INFO:tasks.workunit.client.1.smithi096.stdout:7/997: dwrite d0/f10f [0,4194304] 0 2024-03-24T10:46:21.731 INFO:tasks.workunit.client.1.smithi096.stdout:4/725: dwrite d1/d2b/d67/d5f/fec [0,4194304] 0 2024-03-24T10:46:21.738 INFO:tasks.workunit.client.1.smithi096.stdout:0/914: symlink d0/d11/d16/d21/d78/d92/l136 0 2024-03-24T10:46:21.814 INFO:tasks.workunit.client.1.smithi096.stdout:9/855: mknod d9/d15/d26/c118 0 2024-03-24T10:46:21.815 INFO:tasks.workunit.client.1.smithi096.stdout:2/983: mkdir d5/d7c/d124/d169 0 2024-03-24T10:46:21.815 INFO:tasks.workunit.client.1.smithi096.stdout:2/984: chown d5/d7c/dfd/d140 13 1 2024-03-24T10:46:21.815 INFO:tasks.workunit.client.1.smithi096.stdout:2/985: fdatasync d5/d7c/d88/d114/f165 0 2024-03-24T10:46:21.818 INFO:tasks.workunit.client.1.smithi096.stdout:1/776: symlink d3/d23/d2f/d65/d7c/ddf/df2/l106 0 2024-03-24T10:46:21.828 INFO:tasks.workunit.client.1.smithi096.stdout:9/856: dwrite d9/db/f28 [0,4194304] 0 2024-03-24T10:46:21.830 INFO:tasks.workunit.client.1.smithi096.stdout:0/915: symlink d0/d11/d24/d11f/l137 0 2024-03-24T10:46:21.830 INFO:tasks.workunit.client.1.smithi096.stdout:0/916: write d0/d11/d20/f26 [1490889,99695] 0 2024-03-24T10:46:21.830 INFO:tasks.workunit.client.1.smithi096.stdout:0/917: readlink d0/d11/d24/d2c/d84/lc2 0 2024-03-24T10:46:21.840 INFO:tasks.workunit.client.1.smithi096.stdout:2/986: truncate d5/dc/f13d 1648919 0 2024-03-24T10:46:21.840 INFO:tasks.workunit.client.1.smithi096.stdout:2/987: chown d5/dc/d6b/d72/d91/l9d 7 1 2024-03-24T10:46:21.845 INFO:tasks.workunit.client.1.smithi096.stdout:9/857: symlink d9/d102/l119 0 2024-03-24T10:46:21.845 INFO:tasks.workunit.client.1.smithi096.stdout:9/858: write d9/d15/d26/d29/d73/df4/f10f [4472665,25295] 0 2024-03-24T10:46:21.846 INFO:tasks.workunit.client.1.smithi096.stdout:0/918: dwrite d0/d11/d24/f10f [0,4194304] 0 2024-03-24T10:46:21.846 INFO:tasks.workunit.client.1.smithi096.stdout:0/919: readlink d0/d11/d16/d21/d81/d9c/l114 0 2024-03-24T10:46:21.848 INFO:tasks.workunit.client.1.smithi096.stdout:9/859: symlink d9/db/d6d/d4f/d75/dcc/l11a 0 2024-03-24T10:46:21.848 INFO:tasks.workunit.client.1.smithi096.stdout:9/860: fsync d9/d79/fd7 0 2024-03-24T10:46:21.850 INFO:tasks.workunit.client.1.smithi096.stdout:2/988: dread d5/de/fe2 [0,4194304] 0 2024-03-24T10:46:21.851 INFO:tasks.workunit.client.1.smithi096.stdout:2/989: chown d5/d79/d9f 2 1 2024-03-24T10:46:21.864 INFO:tasks.workunit.client.1.smithi096.stdout:2/990: dwrite d5/de/d2a/d5e/f10c [0,4194304] 0 2024-03-24T10:46:21.868 INFO:tasks.workunit.client.1.smithi096.stdout:2/991: truncate d5/d93/ff8 1160791 0 2024-03-24T10:46:21.883 INFO:tasks.workunit.client.1.smithi096.stdout:6/816: unlink d8/d12/d28/d36/d65/d6b/fc2 0 2024-03-24T10:46:21.883 INFO:tasks.workunit.client.1.smithi096.stdout:6/817: creat d8/d12/d32/fff x:0 0 0 2024-03-24T10:46:21.887 INFO:tasks.workunit.client.1.smithi096.stdout:6/818: creat d8/d12/d61/f100 x:0 0 0 2024-03-24T10:46:21.888 INFO:tasks.workunit.client.1.smithi096.stdout:6/819: link d8/d12/d28/d36/d65/f6f d8/d12/d32/f101 0 2024-03-24T10:46:21.890 INFO:tasks.workunit.client.1.smithi096.stdout:6/820: symlink d8/d7b/l102 0 2024-03-24T10:46:21.890 INFO:tasks.workunit.client.1.smithi096.stdout:6/821: fdatasync d8/d10/ff1 0 2024-03-24T10:46:21.893 INFO:tasks.workunit.client.1.smithi096.stdout:2/992: dwrite d5/d38/f7b [0,4194304] 0 2024-03-24T10:46:21.893 INFO:tasks.workunit.client.1.smithi096.stdout:2/993: chown d5/dc/d6b/c135 0 1 2024-03-24T10:46:21.893 INFO:tasks.workunit.client.1.smithi096.stdout:2/994: write d5/d7c/f10b [4970453,53198] 0 2024-03-24T10:46:21.908 INFO:tasks.workunit.client.1.smithi096.stdout:6/822: dwrite d8/fdc [0,4194304] 0 2024-03-24T10:46:21.916 INFO:tasks.workunit.client.1.smithi096.stdout:6/823: read d8/d12/d32/f47 [3213850,106887] 0 2024-03-24T10:46:21.916 INFO:tasks.workunit.client.1.smithi096.stdout:6/824: write d8/d12/d32/f6e [1014039,15537] 0 2024-03-24T10:46:21.921 INFO:tasks.workunit.client.1.smithi096.stdout:9/861: getdents d9/d102 0 2024-03-24T10:46:21.924 INFO:tasks.workunit.client.1.smithi096.stdout:0/920: write d0/d11/d16/d21/d78/d92/f99 [1068678,27611] 0 2024-03-24T10:46:21.924 INFO:tasks.workunit.client.1.smithi096.stdout:0/921: fdatasync d0/d11/d20/fd5 0 2024-03-24T10:46:21.924 INFO:tasks.workunit.client.1.smithi096.stdout:0/922: dread - d0/d11/d16/d21/d78/deb/ffa zero size 2024-03-24T10:46:21.924 INFO:tasks.workunit.client.1.smithi096.stdout:0/923: write d0/d11/d16/d21/d81/d8f/fc6 [1576850,66445] 0 2024-03-24T10:46:21.925 INFO:tasks.workunit.client.1.smithi096.stdout:9/862: write d9/db/d6d/d7f/f3f [1021337,75145] 0 2024-03-24T10:46:21.925 INFO:tasks.workunit.client.1.smithi096.stdout:9/863: write d9/d15/d26/d29/fc5 [606784,85914] 0 2024-03-24T10:46:21.926 INFO:tasks.workunit.client.1.smithi096.stdout:9/864: mknod d9/d15/d26/d29/d73/d49/d86/c11b 0 2024-03-24T10:46:21.926 INFO:tasks.workunit.client.1.smithi096.stdout:9/865: chown d9/d15/d26/d29/ca5 9860 1 2024-03-24T10:46:21.928 INFO:tasks.workunit.client.1.smithi096.stdout:9/866: mkdir d9/d102/d11c 0 2024-03-24T10:46:21.928 INFO:tasks.workunit.client.1.smithi096.stdout:9/867: readlink d9/db/d6d/d4f/d75/d8b/d8e/lee 0 2024-03-24T10:46:21.929 INFO:tasks.workunit.client.1.smithi096.stdout:9/868: creat d9/d15/f11d x:0 0 0 2024-03-24T10:46:21.934 INFO:tasks.workunit.client.1.smithi096.stdout:3/794: rename d0/d13/d45/d7d/d71 to d0/d9/d22/d102 0 2024-03-24T10:46:21.936 INFO:tasks.workunit.client.1.smithi096.stdout:9/869: rename d9/d15/d26/d29/d73/d43/d48/d51/l68 to d9/db/d6d/d4f/de8/l11e 0 2024-03-24T10:46:21.937 INFO:tasks.workunit.client.1.smithi096.stdout:3/795: creat d0/d13/d45/d7d/d39/d48/d85/f103 x:0 0 0 2024-03-24T10:46:21.938 INFO:tasks.workunit.client.1.smithi096.stdout:9/870: link d9/d102/c108 d9/db/d6d/dc6/df0/de4/c11f 0 2024-03-24T10:46:21.939 INFO:tasks.workunit.client.1.smithi096.stdout:9/871: write d9/d15/d26/d29/d73/d43/d48/dc0/dd8/fe9 [871661,53684] 0 2024-03-24T10:46:21.939 INFO:tasks.workunit.client.1.smithi096.stdout:9/872: write d9/d15/d26/f33 [2924062,51750] 0 2024-03-24T10:46:21.939 INFO:tasks.workunit.client.1.smithi096.stdout:3/796: symlink d0/d9/d22/d102/l104 0 2024-03-24T10:46:21.942 INFO:tasks.workunit.client.1.smithi096.stdout:9/873: rename d9/db/d6d/d4f/f7b to d9/db/d6d/dc6/df0/de4/f120 0 2024-03-24T10:46:21.942 INFO:tasks.workunit.client.1.smithi096.stdout:9/874: write d9/d15/d26/d29/d73/ffe [174172,65536] 0 2024-03-24T10:46:21.943 INFO:tasks.workunit.client.1.smithi096.stdout:9/875: chown d9/d15/d26/d29 3 1 2024-03-24T10:46:21.943 INFO:tasks.workunit.client.1.smithi096.stdout:0/924: dwrite d0/d11/d24/f2d [0,4194304] 0 2024-03-24T10:46:21.943 INFO:tasks.workunit.client.1.smithi096.stdout:0/925: chown d0/d11/dc3/f103 64 1 2024-03-24T10:46:21.943 INFO:tasks.workunit.client.1.smithi096.stdout:3/797: getdents d0/d92/ddc 0 2024-03-24T10:46:21.944 INFO:tasks.workunit.client.1.smithi096.stdout:9/876: creat d9/df1/df8/f121 x:0 0 0 2024-03-24T10:46:21.946 INFO:tasks.workunit.client.1.smithi096.stdout:0/926: mknod d0/d11/d16/c138 0 2024-03-24T10:46:21.949 INFO:tasks.workunit.client.1.smithi096.stdout:3/798: read d0/d13/d45/d47/d8f/fdb [276580,64219] 0 2024-03-24T10:46:21.950 INFO:tasks.workunit.client.1.smithi096.stdout:1/777: symlink d3/d23/d2f/d65/l107 0 2024-03-24T10:46:21.950 INFO:tasks.workunit.client.1.smithi096.stdout:1/778: fsync d3/d23/d71/fb8 0 2024-03-24T10:46:21.950 INFO:tasks.workunit.client.1.smithi096.stdout:1/779: chown d3/d73/fcf 59 1 2024-03-24T10:46:21.951 INFO:tasks.workunit.client.1.smithi096.stdout:3/799: mknod d0/d13/d45/d7d/d39/d48/dbd/daa/c105 0 2024-03-24T10:46:21.951 INFO:tasks.workunit.client.1.smithi096.stdout:3/800: readlink d0/d13/d41/lcb 0 2024-03-24T10:46:21.953 INFO:tasks.workunit.client.1.smithi096.stdout:1/780: creat d3/d23/d8b/d79/f108 x:0 0 0 2024-03-24T10:46:21.953 INFO:tasks.workunit.client.1.smithi096.stdout:1/781: creat d3/d84/d9a/dd2/f109 x:0 0 0 2024-03-24T10:46:21.953 INFO:tasks.workunit.client.1.smithi096.stdout:1/782: creat d3/d23/d71/d7b/f10a x:0 0 0 2024-03-24T10:46:21.956 INFO:tasks.workunit.client.1.smithi096.stdout:1/783: link d3/d23/d8b/d99/lb6 d3/d23/d71/d7b/d93/l10b 0 2024-03-24T10:46:21.956 INFO:tasks.workunit.client.1.smithi096.stdout:1/784: dread - d3/d23/d71/fb8 zero size 2024-03-24T10:46:21.956 INFO:tasks.workunit.client.1.smithi096.stdout:1/785: readlink d3/d84/d9a/lae 0 2024-03-24T10:46:21.957 INFO:tasks.workunit.client.1.smithi096.stdout:1/786: rename d3/l1a to d3/l10c 0 2024-03-24T10:46:21.962 INFO:tasks.workunit.client.1.smithi096.stdout:1/787: mkdir d3/d36/d10d 0 2024-03-24T10:46:21.962 INFO:tasks.workunit.client.1.smithi096.stdout:1/788: readlink d3/d23/d2f/l49 0 2024-03-24T10:46:21.964 INFO:tasks.workunit.client.1.smithi096.stdout:1/789: creat d3/d23/d8b/d99/f10e x:0 0 0 2024-03-24T10:46:21.964 INFO:tasks.workunit.client.1.smithi096.stdout:3/801: dread d0/d9/d22/fa7 [0,4194304] 0 2024-03-24T10:46:21.966 INFO:tasks.workunit.client.1.smithi096.stdout:1/790: mkdir d3/d23/d71/d7b/dce/d103/d10f 0 2024-03-24T10:46:21.967 INFO:tasks.workunit.client.1.smithi096.stdout:3/802: rmdir d0/d13/d41/ddf 39 2024-03-24T10:46:21.970 INFO:tasks.workunit.client.1.smithi096.stdout:1/791: symlink d3/d23/d8b/d8c/ddc/l110 0 2024-03-24T10:46:21.972 INFO:tasks.workunit.client.1.smithi096.stdout:3/803: mkdir d0/d13/d45/d7d/d39/d48/dbd/d106 0 2024-03-24T10:46:21.974 INFO:tasks.workunit.client.1.smithi096.stdout:1/792: symlink d3/d9b/l111 0 2024-03-24T10:46:21.975 INFO:tasks.workunit.client.1.smithi096.stdout:9/877: dwrite d9/d15/d26/d29/d73/d43/d48/fae [0,4194304] 0 2024-03-24T10:46:21.978 INFO:tasks.workunit.client.1.smithi096.stdout:3/804: link d0/fe9 d0/f107 0 2024-03-24T10:46:21.978 INFO:tasks.workunit.client.1.smithi096.stdout:3/805: truncate d0/d92/ddc/dfc/de5/ff6 385463 0 2024-03-24T10:46:21.981 INFO:tasks.workunit.client.1.smithi096.stdout:1/793: symlink d3/d23/d2f/d65/d7c/ddf/df2/l112 0 2024-03-24T10:46:21.982 INFO:tasks.workunit.client.1.smithi096.stdout:0/927: dwrite d0/d11/d16/d21/d25/d47/f6b [0,4194304] 0 2024-03-24T10:46:21.982 INFO:tasks.workunit.client.1.smithi096.stdout:0/928: fsync d0/d11/d20/d91/fe5 0 2024-03-24T10:46:21.992 INFO:tasks.workunit.client.1.smithi096.stdout:1/794: read d3/d23/d2f/f59 [2779098,121617] 0 2024-03-24T10:46:21.992 INFO:tasks.workunit.client.1.smithi096.stdout:1/795: write d3/d23/d71/fb8 [423629,82377] 0 2024-03-24T10:46:21.993 INFO:tasks.workunit.client.1.smithi096.stdout:1/796: dread - d3/d23/d8b/d5c/d9c/fec zero size 2024-03-24T10:46:21.995 INFO:tasks.workunit.client.1.smithi096.stdout:9/878: symlink d9/d102/d11c/l122 0 2024-03-24T10:46:21.997 INFO:tasks.workunit.client.1.smithi096.stdout:0/929: dwrite d0/f60 [0,4194304] 0 2024-03-24T10:46:21.998 INFO:tasks.workunit.client.1.smithi096.stdout:7/998: sync 2024-03-24T10:46:21.999 INFO:tasks.workunit.client.1.smithi096.stdout:4/726: sync 2024-03-24T10:46:21.999 INFO:tasks.workunit.client.1.smithi096.stdout:5/835: sync 2024-03-24T10:46:21.999 INFO:tasks.workunit.client.1.smithi096.stdout:8/873: sync 2024-03-24T10:46:21.999 INFO:tasks.workunit.client.1.smithi096.stdout:5/836: chown d2/d29/d32 60212002 1 2024-03-24T10:46:21.999 INFO:tasks.workunit.client.1.smithi096.stdout:5/837: fsync d2/d29/d32/fcb 0 2024-03-24T10:46:21.999 INFO:tasks.workunit.client.1.smithi096.stdout:8/874: write d8/d11/d22/d4c/d65/d1a/d49/fae [3651,64888] 0 2024-03-24T10:46:21.999 INFO:tasks.workunit.client.1.smithi096.stdout:8/875: read d8/d11/d22/d4c/d65/d1a/f2d [331446,64031] 0 2024-03-24T10:46:22.003 INFO:tasks.workunit.client.1.smithi096.stdout:7/999: dread d0/d13/df0/d59/f79 [0,4194304] 0 2024-03-24T10:46:22.004 INFO:tasks.workunit.client.1.smithi096.stdout:3/806: mkdir d0/d13/d45/d7d/d39/d48/dbd/d108 0 2024-03-24T10:46:22.005 INFO:tasks.workunit.client.1.smithi096.stdout:0/930: write d0/d11/d20/f111 [2505390,67678] 0 2024-03-24T10:46:22.009 INFO:tasks.workunit.client.1.smithi096.stdout:1/797: mknod d3/db2/c113 0 2024-03-24T10:46:22.010 INFO:tasks.workunit.client.1.smithi096.stdout:1/798: read d3/d23/d8b/d8c/ddc/fde [22738,8529] 0 2024-03-24T10:46:22.010 INFO:tasks.workunit.client.1.smithi096.stdout:9/879: mkdir d9/d15/d26/d29/d123 0 2024-03-24T10:46:22.010 INFO:tasks.workunit.client.1.smithi096.stdout:9/880: stat d9/d60 0 2024-03-24T10:46:22.010 INFO:tasks.workunit.client.1.smithi096.stdout:9/881: fdatasync d9/d7c/f10c 0 2024-03-24T10:46:22.014 INFO:tasks.workunit.client.1.smithi096.stdout:4/727: link d1/d2b/d6b/d80/d9c/lb2 d1/d32/d4c/led 0 2024-03-24T10:46:22.015 INFO:tasks.workunit.client.1.smithi096.stdout:5/838: truncate d2/d7/fdc 5471338 0 2024-03-24T10:46:22.018 INFO:tasks.workunit.client.1.smithi096.stdout:8/876: truncate d8/d11/f27 425051 0 2024-03-24T10:46:22.021 INFO:tasks.workunit.client.1.smithi096.stdout:3/807: rename d0/d13/d45/d7d/d39/le1 to d0/d13/d45/l109 0 2024-03-24T10:46:22.024 INFO:tasks.workunit.client.1.smithi096.stdout:0/931: link d0/d11/d20/d56/l5b d0/d11/d24/l139 0 2024-03-24T10:46:22.025 INFO:tasks.workunit.client.1.smithi096.stdout:9/882: creat d9/d15/d26/f124 x:0 0 0 2024-03-24T10:46:22.025 INFO:tasks.workunit.client.1.smithi096.stdout:9/883: fsync d9/d15/d26/d29/d73/df4/f10f 0 2024-03-24T10:46:22.025 INFO:tasks.workunit.client.1.smithi096.stdout:9/884: readlink d9/db/l93 0 2024-03-24T10:46:22.025 INFO:tasks.workunit.client.1.smithi096.stdout:9/885: fdatasync d9/d7c/f10a 0 2024-03-24T10:46:22.025 INFO:tasks.workunit.client.1.smithi096.stdout:9/886: write d9/d7c/f9a [728985,8593] 0 2024-03-24T10:46:22.027 INFO:tasks.workunit.client.1.smithi096.stdout:5/839: mkdir d2/d29/d32/d33/d116/d11f 0 2024-03-24T10:46:22.031 INFO:tasks.workunit.client.1.smithi096.stdout:8/877: rmdir d8/d11/d22/d4c/dbf 0 2024-03-24T10:46:22.031 INFO:tasks.workunit.client.1.smithi096.stdout:8/878: write d8/fd6 [579113,61851] 0 2024-03-24T10:46:22.031 INFO:tasks.workunit.client.1.smithi096.stdout:8/879: write d8/d11/d22/d4c/d65/d1a/d49/fae [1081720,114813] 0 2024-03-24T10:46:22.031 INFO:tasks.workunit.client.1.smithi096.stdout:8/880: write d8/d72/f83 [756386,16971] 0 2024-03-24T10:46:22.033 INFO:tasks.workunit.client.1.smithi096.stdout:1/799: rename d3/d23/d71/d7b/dce/d103/d10f to d3/d23/d71/d7b/dce/de1/d114 0 2024-03-24T10:46:22.033 INFO:tasks.workunit.client.1.smithi096.stdout:1/800: fdatasync d3/d23/d8b/d5c/d9c/fda 0 2024-03-24T10:46:22.039 INFO:tasks.workunit.client.1.smithi096.stdout:0/932: mkdir d0/d11/d24/d127/d13a 0 2024-03-24T10:46:22.039 INFO:tasks.workunit.client.1.smithi096.stdout:0/933: readlink d0/d11/d16/l3b 0 2024-03-24T10:46:22.043 INFO:tasks.workunit.client.1.smithi096.stdout:8/881: rmdir d8/d11/d22/d4c/d82 39 2024-03-24T10:46:22.045 INFO:tasks.workunit.client.1.smithi096.stdout:9/887: dwrite d9/d15/f3c [0,4194304] 0 2024-03-24T10:46:22.046 INFO:tasks.workunit.client.1.smithi096.stdout:8/882: dread d8/d11/d22/d4c/d65/d1a/f80 [0,4194304] 0 2024-03-24T10:46:22.048 INFO:tasks.workunit.client.1.smithi096.stdout:4/728: rename d1/d32/d42/c4a to d1/d32/d4c/cee 0 2024-03-24T10:46:22.048 INFO:tasks.workunit.client.1.smithi096.stdout:4/729: chown d1/c17 29 1 2024-03-24T10:46:22.049 INFO:tasks.workunit.client.1.smithi096.stdout:4/730: write d1/d32/d4c/d7e/f99 [6043851,22849] 0 2024-03-24T10:46:22.049 INFO:tasks.workunit.client.1.smithi096.stdout:4/731: truncate d1/d2b/d6b/d80/d9c/fe3 864600 0 2024-03-24T10:46:22.049 INFO:tasks.workunit.client.1.smithi096.stdout:1/801: link d3/c4 d3/d23/d8b/d5c/d9c/c115 0 2024-03-24T10:46:22.053 INFO:tasks.workunit.client.1.smithi096.stdout:0/934: mkdir d0/d11/d20/d9e/d80/d110/d13b 0 2024-03-24T10:46:22.053 INFO:tasks.workunit.client.1.smithi096.stdout:0/935: stat d0/d11/d16/d21/d25/f106 0 2024-03-24T10:46:22.054 INFO:tasks.workunit.client.1.smithi096.stdout:5/840: dwrite d2/d29/d32/f40 [4194304,4194304] 0 2024-03-24T10:46:22.054 INFO:tasks.workunit.client.1.smithi096.stdout:5/841: chown d2/d29/d32/d33/d116 17 1 2024-03-24T10:46:22.054 INFO:tasks.workunit.client.1.smithi096.stdout:5/842: chown d2/d29/d32/cac 131231993 1 2024-03-24T10:46:22.058 INFO:tasks.workunit.client.1.smithi096.stdout:9/888: unlink d9/d15/l12 0 2024-03-24T10:46:22.059 INFO:tasks.workunit.client.1.smithi096.stdout:9/889: write d9/d15/d26/d29/d73/d43/fa3 [1288704,75195] 0 2024-03-24T10:46:22.059 INFO:tasks.workunit.client.1.smithi096.stdout:1/802: write d3/d23/d8b/d5c/fa5 [60476,118737] 0 2024-03-24T10:46:22.059 INFO:tasks.workunit.client.1.smithi096.stdout:1/803: write d3/d23/d71/d7b/faa [1097268,82041] 0 2024-03-24T10:46:22.060 INFO:tasks.workunit.client.1.smithi096.stdout:8/883: rename d8/d11/d22/d4c/d65/d1d/fcf to d8/d72/d96/d91/dfc/f115 0 2024-03-24T10:46:22.065 INFO:tasks.workunit.client.1.smithi096.stdout:4/732: unlink d1/d2b/l90 0 2024-03-24T10:46:22.074 INFO:tasks.workunit.client.1.smithi096.stdout:9/890: symlink d9/d15/d26/l125 0 2024-03-24T10:46:22.076 INFO:tasks.workunit.client.1.smithi096.stdout:1/804: dwrite d3/d23/d71/d85/ff9 [0,4194304] 0 2024-03-24T10:46:22.078 INFO:tasks.workunit.client.1.smithi096.stdout:8/884: mkdir d8/d110/d116 0 2024-03-24T10:46:22.078 INFO:tasks.workunit.client.1.smithi096.stdout:8/885: stat d8/d11/d22/d4c/d67/f7a 0 2024-03-24T10:46:22.078 INFO:tasks.workunit.client.1.smithi096.stdout:3/808: write d0/d9/f32 [46359,4805] 0 2024-03-24T10:46:22.079 INFO:tasks.workunit.client.1.smithi096.stdout:4/733: rmdir d1/d2b/d89/dbf 39 2024-03-24T10:46:22.079 INFO:tasks.workunit.client.1.smithi096.stdout:4/734: stat d1/c77 0 2024-03-24T10:46:22.079 INFO:tasks.workunit.client.1.smithi096.stdout:4/735: stat d1/f35 0 2024-03-24T10:46:22.080 INFO:tasks.workunit.client.1.smithi096.stdout:0/936: rmdir d0/d11/d16/d21/d78/deb 39 2024-03-24T10:46:22.086 INFO:tasks.workunit.client.1.smithi096.stdout:1/805: readlink d3/d23/d8b/d5c/l64 0 2024-03-24T10:46:22.087 INFO:tasks.workunit.client.1.smithi096.stdout:5/843: dread d2/d7/d69/f87 [0,4194304] 0 2024-03-24T10:46:22.087 INFO:tasks.workunit.client.1.smithi096.stdout:3/809: mknod d0/d13/d77/c10a 0 2024-03-24T10:46:22.089 INFO:tasks.workunit.client.1.smithi096.stdout:4/736: mkdir d1/d2b/d89/def 0 2024-03-24T10:46:22.091 INFO:tasks.workunit.client.1.smithi096.stdout:0/937: unlink d0/d11/d16/d21/d78/d82/cf3 0 2024-03-24T10:46:22.092 INFO:tasks.workunit.client.1.smithi096.stdout:5/844: dread d2/d7/f3e [0,4194304] 0 2024-03-24T10:46:22.092 INFO:tasks.workunit.client.1.smithi096.stdout:5/845: read - d2/d7/dba/f114 zero size 2024-03-24T10:46:22.093 INFO:tasks.workunit.client.1.smithi096.stdout:5/846: chown d2/d29/d32/d33/d3c/le9 2034 1 2024-03-24T10:46:22.094 INFO:tasks.workunit.client.1.smithi096.stdout:4/737: dread d1/d32/d4c/d7e/f99 [4194304,4194304] 0 2024-03-24T10:46:22.098 INFO:tasks.workunit.client.1.smithi096.stdout:0/938: read d0/d11/d16/d21/d25/f36 [4172514,128809] 0 2024-03-24T10:46:22.103 INFO:tasks.workunit.client.1.smithi096.stdout:1/806: mkdir d3/d84/d9a/d116 0 2024-03-24T10:46:22.104 INFO:tasks.workunit.client.1.smithi096.stdout:3/810: symlink d0/d13/d45/d7d/d39/d48/dbd/d106/l10b 0 2024-03-24T10:46:22.105 INFO:tasks.workunit.client.1.smithi096.stdout:3/811: chown d0/d13/d41/ddf 1010217953 1 2024-03-24T10:46:22.105 INFO:tasks.workunit.client.1.smithi096.stdout:3/812: fsync d0/d13/d45/d47/d8f/fef 0 2024-03-24T10:46:22.105 INFO:tasks.workunit.client.1.smithi096.stdout:5/847: creat d2/d29/d32/d33/d116/d11f/f120 x:0 0 0 2024-03-24T10:46:22.107 INFO:tasks.workunit.client.1.smithi096.stdout:4/738: mkdir d1/d32/d42/dba/df0 0 2024-03-24T10:46:22.108 INFO:tasks.workunit.client.1.smithi096.stdout:0/939: creat d0/d11/d24/dcc/f13c x:0 0 0 2024-03-24T10:46:22.108 INFO:tasks.workunit.client.1.smithi096.stdout:0/940: dread - d0/d11/d43/d75/f88 zero size 2024-03-24T10:46:22.109 INFO:tasks.workunit.client.1.smithi096.stdout:9/891: rename d9/d15/d26 to d9/d65/d126 0 2024-03-24T10:46:22.112 INFO:tasks.workunit.client.1.smithi096.stdout:5/848: creat d2/d29/d32/d89/f121 x:0 0 0 2024-03-24T10:46:22.129 INFO:tasks.workunit.client.1.smithi096.stdout:4/739: rename d1/d2b/l2d to d1/d2b/lf1 0 2024-03-24T10:46:22.133 INFO:tasks.workunit.client.1.smithi096.stdout:8/886: rename d8/d11/d22/d4c/d65/d37/df2 to d8/d11/d22/d4c/d65/d1a/d46/d117 0 2024-03-24T10:46:22.135 INFO:tasks.workunit.client.1.smithi096.stdout:9/892: symlink d9/d65/d126/d29/d73/d49/db7/dca/de3/l127 0 2024-03-24T10:46:22.136 INFO:tasks.workunit.client.1.smithi096.stdout:4/740: dread d1/d32/d42/f5e [0,4194304] 0 2024-03-24T10:46:22.137 INFO:tasks.workunit.client.1.smithi096.stdout:3/813: getdents d0/d13/d45/d47/d8f 0 2024-03-24T10:46:22.137 INFO:tasks.workunit.client.1.smithi096.stdout:3/814: chown d0/d13/d45/d7d/d39/l93 225899897 1 2024-03-24T10:46:22.138 INFO:tasks.workunit.client.1.smithi096.stdout:5/849: creat d2/d29/d32/d33/d3c/d3d/d10f/f122 x:0 0 0 2024-03-24T10:46:22.138 INFO:tasks.workunit.client.1.smithi096.stdout:5/850: write d2/d7/fa7 [2517883,80903] 0 2024-03-24T10:46:22.138 INFO:tasks.workunit.client.1.smithi096.stdout:9/893: dread d9/db/d6d/f35 [0,4194304] 0 2024-03-24T10:46:22.144 INFO:tasks.workunit.client.1.smithi096.stdout:9/894: dread d9/d15/f66 [0,4194304] 0 2024-03-24T10:46:22.144 INFO:tasks.workunit.client.1.smithi096.stdout:9/895: write d9/f5c [1364177,98760] 0 2024-03-24T10:46:22.144 INFO:tasks.workunit.client.1.smithi096.stdout:2/995: sync 2024-03-24T10:46:22.145 INFO:tasks.workunit.client.1.smithi096.stdout:6/825: sync 2024-03-24T10:46:22.150 INFO:tasks.workunit.client.1.smithi096.stdout:0/941: rmdir d0/d11/d16/d21 39 2024-03-24T10:46:22.152 INFO:tasks.workunit.client.1.smithi096.stdout:1/807: rename d3/d23/d8b/f46 to d3/d36/db3/f117 0 2024-03-24T10:46:22.152 INFO:tasks.workunit.client.1.smithi096.stdout:1/808: chown d3/d23/d8b/d5c/f92 360196076 1 2024-03-24T10:46:22.152 INFO:tasks.workunit.client.1.smithi096.stdout:1/809: getdents d3/d36/df1 0 2024-03-24T10:46:22.152 INFO:tasks.workunit.client.1.smithi096.stdout:1/810: chown d3/d23/d2f/d65/d7c/ddf/df2 936035 1 2024-03-24T10:46:22.153 INFO:tasks.workunit.client.1.smithi096.stdout:8/887: getdents d8/d11/d22/d4c/d65/d37 0 2024-03-24T10:46:22.154 INFO:tasks.workunit.client.1.smithi096.stdout:4/741: symlink d1/d32/d42/dba/lf2 0 2024-03-24T10:46:22.156 INFO:tasks.workunit.client.1.smithi096.stdout:3/815: link d0/d13/d45/d7d/d39/d48/d8d/fe0 d0/d13/d45/d47/d8f/f10c 0 2024-03-24T10:46:22.159 INFO:tasks.workunit.client.1.smithi096.stdout:9/896: mkdir d9/db/d6d/d7f/d128 0 2024-03-24T10:46:22.161 INFO:tasks.workunit.client.1.smithi096.stdout:6/826: getdents d8/d10/d76/dd6 0 2024-03-24T10:46:22.161 INFO:tasks.workunit.client.1.smithi096.stdout:8/888: dread d8/d11/d22/d4c/d65/d1a/f2d [0,4194304] 0 2024-03-24T10:46:22.164 INFO:tasks.workunit.client.1.smithi096.stdout:0/942: creat d0/d11/d16/d21/d81/d8f/d9a/ddd/f13d x:0 0 0 2024-03-24T10:46:22.164 INFO:tasks.workunit.client.1.smithi096.stdout:0/943: write d0/d11/d24/d2c/d84/f11d [318079,127796] 0 2024-03-24T10:46:22.164 INFO:tasks.workunit.client.1.smithi096.stdout:0/944: fdatasync d0/d11/d24/d2c/d84/ff6 0 2024-03-24T10:46:22.169 INFO:tasks.workunit.client.1.smithi096.stdout:8/889: write d8/d11/d22/d4c/d65/d1a/d46/d2e/fbe [3980256,97875] 0 2024-03-24T10:46:22.172 INFO:tasks.workunit.client.1.smithi096.stdout:2/996: rename d5/d79/d9f/dad to d5/dc/d6b/d72/d91/d16a 0 2024-03-24T10:46:22.176 INFO:tasks.workunit.client.1.smithi096.stdout:3/816: creat d0/d92/ddc/dfc/de5/f10d x:0 0 0 2024-03-24T10:46:22.177 INFO:tasks.workunit.client.1.smithi096.stdout:9/897: dwrite d9/f13 [4194304,4194304] 0 2024-03-24T10:46:22.177 INFO:tasks.workunit.client.1.smithi096.stdout:9/898: chown d9/d65/d126/c118 3204 1 2024-03-24T10:46:22.177 INFO:tasks.workunit.client.1.smithi096.stdout:5/851: rmdir d2/d7/dba 39 2024-03-24T10:46:22.180 INFO:tasks.workunit.client.1.smithi096.stdout:6/827: mkdir d8/d12/d28/d36/d65/d6b/da6/d103 0 2024-03-24T10:46:22.186 INFO:tasks.workunit.client.1.smithi096.stdout:0/945: creat d0/d11/d16/d21/d25/f13e x:0 0 0 2024-03-24T10:46:22.187 INFO:tasks.workunit.client.1.smithi096.stdout:8/890: rmdir d8/d11/d22/d4c/d65/d1d/d3d/d5f 39 2024-03-24T10:46:22.188 INFO:tasks.workunit.client.1.smithi096.stdout:6/828: dread d8/d12/d32/f6e [0,4194304] 0 2024-03-24T10:46:22.192 INFO:tasks.workunit.client.1.smithi096.stdout:2/997: getdents d5 0 2024-03-24T10:46:22.192 INFO:tasks.workunit.client.1.smithi096.stdout:2/998: fdatasync d5/d7c/d124/f162 0 2024-03-24T10:46:22.193 INFO:tasks.workunit.client.1.smithi096.stdout:1/811: rmdir d3/d23/d71/d7b 39 2024-03-24T10:46:22.193 INFO:tasks.workunit.client.1.smithi096.stdout:1/812: write d3/d23/d8b/d79/fa8 [783390,87256] 0 2024-03-24T10:46:22.195 INFO:tasks.workunit.client.1.smithi096.stdout:5/852: dwrite d2/d7/f3e [0,4194304] 0 2024-03-24T10:46:22.197 INFO:tasks.workunit.client.1.smithi096.stdout:5/853: dread d2/d29/d32/d89/f9c [0,4194304] 0 2024-03-24T10:46:22.198 INFO:tasks.workunit.client.1.smithi096.stdout:3/817: dwrite d0/d13/d45/d7d/d39/d48/dbd/d9c/ff0 [0,4194304] 0 2024-03-24T10:46:22.198 INFO:tasks.workunit.client.1.smithi096.stdout:3/818: write d0/d13/d45/d7d/d39/d48/ffa [774799,69427] 0 2024-03-24T10:46:22.199 INFO:tasks.workunit.client.1.smithi096.stdout:9/899: mknod d9/db/d6d/d4f/c129 0 2024-03-24T10:46:22.199 INFO:tasks.workunit.client.1.smithi096.stdout:9/900: write d9/db/f80 [2562463,60191] 0 2024-03-24T10:46:22.202 INFO:tasks.workunit.client.1.smithi096.stdout:0/946: mknod d0/d11/d20/c13f 0 2024-03-24T10:46:22.202 INFO:tasks.workunit.client.1.smithi096.stdout:0/947: creat d0/d11/d20/d91/f140 x:0 0 0 2024-03-24T10:46:22.203 INFO:tasks.workunit.client.1.smithi096.stdout:8/891: mkdir d8/d11/d22/d4c/d65/d118 0 2024-03-24T10:46:22.203 INFO:tasks.workunit.client.1.smithi096.stdout:8/892: chown d8/d11/d22/d4c/db4/lb7 238661 1 2024-03-24T10:46:22.208 INFO:tasks.workunit.client.1.smithi096.stdout:2/999: getdents d5/de/d2a/d5e 0 2024-03-24T10:46:22.209 INFO:tasks.workunit.client.1.smithi096.stdout:6/829: dwrite d8/d10/d19/f30 [0,4194304] 0 2024-03-24T10:46:22.210 INFO:tasks.workunit.client.1.smithi096.stdout:1/813: mknod d3/d23/d8b/d5c/d6d/c118 0 2024-03-24T10:46:22.213 INFO:tasks.workunit.client.1.smithi096.stdout:5/854: symlink d2/d29/d32/d3f/d84/l123 0 2024-03-24T10:46:22.216 INFO:tasks.workunit.client.1.smithi096.stdout:3/819: stat d0/d13/d45/d7d/de7/cf5 0 2024-03-24T10:46:22.218 INFO:tasks.workunit.client.1.smithi096.stdout:9/901: creat d9/db/d6d/d4f/d75/f12a x:0 0 0 2024-03-24T10:46:22.218 INFO:tasks.workunit.client.1.smithi096.stdout:9/902: chown d9/d102/l119 13 1 2024-03-24T10:46:22.220 INFO:tasks.workunit.client.1.smithi096.stdout:1/814: dread d3/d23/fb7 [0,4194304] 0 2024-03-24T10:46:22.224 INFO:tasks.workunit.client.1.smithi096.stdout:4/742: rename d1/d2b/d40 to d1/d2b/d67/d5f/da9/db5/df3 0 2024-03-24T10:46:22.229 INFO:tasks.workunit.client.1.smithi096.stdout:5/855: link d2/d7/d8/df/d14/dea/lfb d2/d29/d32/l124 0 2024-03-24T10:46:22.229 INFO:tasks.workunit.client.1.smithi096.stdout:9/903: dread d9/d65/d126/d29/d73/d43/d48/fae [0,4194304] 0 2024-03-24T10:46:22.229 INFO:tasks.workunit.client.1.smithi096.stdout:5/856: read d2/d29/d32/d3f/d5f/f8f [74762,23199] 0 2024-03-24T10:46:22.232 INFO:tasks.workunit.client.1.smithi096.stdout:3/820: mknod d0/d13/d1e/c10e 0 2024-03-24T10:46:22.232 INFO:tasks.workunit.client.1.smithi096.stdout:3/821: write d0/d13/d45/d7d/d39/fbf [4847376,42905] 0 2024-03-24T10:46:22.232 INFO:tasks.workunit.client.1.smithi096.stdout:3/822: dread - d0/d13/d45/d7d/fff zero size 2024-03-24T10:46:22.232 INFO:tasks.workunit.client.1.smithi096.stdout:8/893: dwrite d8/d11/d22/d4c/d65/d1a/d46/d2e/fbe [4194304,4194304] 0 2024-03-24T10:46:22.232 INFO:tasks.workunit.client.1.smithi096.stdout:8/894: fdatasync d8/d11/d22/d4c/d65/d52/f57 0 2024-03-24T10:46:22.233 INFO:tasks.workunit.client.1.smithi096.stdout:8/895: write d8/d11/fe9 [226039,90381] 0 2024-03-24T10:46:22.233 INFO:tasks.workunit.client.1.smithi096.stdout:6/830: dwrite d8/d12/f2d [4194304,4194304] 0 2024-03-24T10:46:22.234 INFO:tasks.workunit.client.1.smithi096.stdout:4/743: mknod d1/d32/d42/cf4 0 2024-03-24T10:46:22.239 INFO:tasks.workunit.client.1.smithi096.stdout:9/904: dread d9/d65/d126/d29/d73/d49/db7/fea [0,4194304] 0 2024-03-24T10:46:22.239 INFO:tasks.workunit.client.1.smithi096.stdout:9/905: fsync d9/d65/d126/d29/d73/d49/d86/f9e 0 2024-03-24T10:46:22.239 INFO:tasks.workunit.client.1.smithi096.stdout:9/906: chown d9/d102/c108 2783 1 2024-03-24T10:46:22.241 INFO:tasks.workunit.client.1.smithi096.stdout:0/948: rename d0/d11/d24/f87 to d0/d11/d20/d2b/ddf/f141 0 2024-03-24T10:46:22.241 INFO:tasks.workunit.client.1.smithi096.stdout:5/857: mknod d2/d29/d32/d33/d8d/d94/df9/c125 0 2024-03-24T10:46:22.246 INFO:tasks.workunit.client.1.smithi096.stdout:3/823: creat d0/d13/d45/d7d/d39/d48/dbd/d9c/f10f x:0 0 0 2024-03-24T10:46:22.247 INFO:tasks.workunit.client.1.smithi096.stdout:3/824: readlink d0/d9/d22/d102/l8a 0 2024-03-24T10:46:22.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:21 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:46:22.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:21 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:46:22.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:21 smithi094 ceph-mon[29446]: pgmap v254: 65 pgs: 65 active+clean; 2.8 GiB data, 9.8 GiB used, 527 GiB / 536 GiB avail; 169 MiB/s rd, 206 MiB/s wr, 354 op/s 2024-03-24T10:46:22.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:21 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:46:22.249 INFO:tasks.workunit.client.1.smithi096.stdout:8/896: dwrite d8/f16 [0,4194304] 0 2024-03-24T10:46:22.249 INFO:tasks.workunit.client.1.smithi096.stdout:6/831: mknod d8/d12/d28/d36/d65/d6b/da6/d103/c104 0 2024-03-24T10:46:22.250 INFO:tasks.workunit.client.1.smithi096.stdout:6/832: chown d8/f9 3508952 1 2024-03-24T10:46:22.251 INFO:tasks.workunit.client.1.smithi096.stdout:4/744: mknod d1/d32/d4c/d7d/dc2/cf5 0 2024-03-24T10:46:22.254 INFO:tasks.workunit.client.1.smithi096.stdout:0/949: mkdir d0/d11/d20/d9e/d80/d142 0 2024-03-24T10:46:22.256 INFO:tasks.workunit.client.1.smithi096.stdout:3/825: symlink d0/d92/ddc/dfc/l110 0 2024-03-24T10:46:22.258 INFO:tasks.workunit.client.1.smithi096.stdout:8/897: mknod d8/d11/d22/d4c/d65/d1d/c119 0 2024-03-24T10:46:22.260 INFO:tasks.workunit.client.1.smithi096.stdout:5/858: dwrite d2/d7/d8/df/f2d [0,4194304] 0 2024-03-24T10:46:22.260 INFO:tasks.workunit.client.1.smithi096.stdout:4/745: dread d1/d11/f12 [0,4194304] 0 2024-03-24T10:46:22.260 INFO:tasks.workunit.client.1.smithi096.stdout:4/746: chown d1/d32/d42/cc7 199 1 2024-03-24T10:46:22.260 INFO:tasks.workunit.client.1.smithi096.stdout:6/833: creat d8/d10/d13/d63/d51/dfc/f105 x:0 0 0 2024-03-24T10:46:22.262 INFO:tasks.workunit.client.1.smithi096.stdout:9/907: truncate d9/db/d6d/fe2 1022687 0 2024-03-24T10:46:22.262 INFO:tasks.workunit.client.1.smithi096.stdout:9/908: write d9/d15/fec [523702,122988] 0 2024-03-24T10:46:22.264 INFO:tasks.workunit.client.1.smithi096.stdout:0/950: unlink d0/f48 0 2024-03-24T10:46:22.264 INFO:tasks.workunit.client.1.smithi096.stdout:0/951: write d0/d11/d16/f53 [1720,89983] 0 2024-03-24T10:46:22.264 INFO:tasks.workunit.client.1.smithi096.stdout:0/952: fdatasync d0/d11/d24/dcc/fd3 0 2024-03-24T10:46:22.264 INFO:tasks.workunit.client.1.smithi096.stdout:4/747: dread d1/d32/d42/f5e [0,4194304] 0 2024-03-24T10:46:22.271 INFO:tasks.workunit.client.1.smithi096.stdout:5/859: creat d2/d7/f126 x:0 0 0 2024-03-24T10:46:22.271 INFO:tasks.workunit.client.1.smithi096.stdout:5/860: chown d2/d7/f126 1019 1 2024-03-24T10:46:22.272 INFO:tasks.workunit.client.1.smithi096.stdout:9/909: dread d9/f5c [0,4194304] 0 2024-03-24T10:46:22.272 INFO:tasks.workunit.client.1.smithi096.stdout:9/910: dread - d9/db/d6d/d4f/fb2 zero size 2024-03-24T10:46:22.272 INFO:tasks.workunit.client.1.smithi096.stdout:9/911: stat d9/d65/d126/d29/d73/d49/db7/dc1 0 2024-03-24T10:46:22.273 INFO:tasks.workunit.client.1.smithi096.stdout:4/748: dread d1/d11/f41 [0,4194304] 0 2024-03-24T10:46:22.273 INFO:tasks.workunit.client.1.smithi096.stdout:4/749: creat d1/d2b/d59/d74/ff6 x:0 0 0 2024-03-24T10:46:22.274 INFO:tasks.workunit.client.1.smithi096.stdout:0/953: creat d0/d11/d20/d56/d109/f143 x:0 0 0 2024-03-24T10:46:22.275 INFO:tasks.workunit.client.1.smithi096.stdout:3/826: getdents d0/d13/df1 0 2024-03-24T10:46:22.276 INFO:tasks.workunit.client.1.smithi096.stdout:8/898: dwrite d8/d11/d22/d4c/d65/d1a/da0/fbd [0,4194304] 0 2024-03-24T10:46:22.276 INFO:tasks.workunit.client.1.smithi096.stdout:8/899: write d8/d11/d22/d4c/d82/f70 [1180642,63499] 0 2024-03-24T10:46:22.276 INFO:tasks.workunit.client.1.smithi096.stdout:8/900: dread - d8/d72/d98/f9c zero size 2024-03-24T10:46:22.276 INFO:tasks.workunit.client.1.smithi096.stdout:8/901: chown d8/d11/d22/d4c/d65/d1d/d3d/d5f/de2/ce6 244890 1 2024-03-24T10:46:22.277 INFO:tasks.workunit.client.1.smithi096.stdout:5/861: rename d2/d29/d32/d3f/f9b to d2/d7/d8/df/d14/d96/f127 0 2024-03-24T10:46:22.278 INFO:tasks.workunit.client.1.smithi096.stdout:5/862: fsync d2/d29/d32/d33/d3c/d3d/fb5 0 2024-03-24T10:46:22.282 INFO:tasks.workunit.client.1.smithi096.stdout:4/750: truncate d1/fd 498451 0 2024-03-24T10:46:22.284 INFO:tasks.workunit.client.1.smithi096.stdout:6/834: link d8/d12/d28/d36/d65/f92 d8/d10/d76/f106 0 2024-03-24T10:46:22.284 INFO:tasks.workunit.client.1.smithi096.stdout:6/835: truncate d8/d12/d32/fee 192164 0 2024-03-24T10:46:22.287 INFO:tasks.workunit.client.1.smithi096.stdout:0/954: creat d0/d11/d24/d102/f144 x:0 0 0 2024-03-24T10:46:22.288 INFO:tasks.workunit.client.1.smithi096.stdout:3/827: symlink d0/d13/d41/ddf/l111 0 2024-03-24T10:46:22.292 INFO:tasks.workunit.client.1.smithi096.stdout:8/902: rename d8/d11/d22/d4c/f68 to d8/d11/d85/f11a 0 2024-03-24T10:46:22.292 INFO:tasks.workunit.client.1.smithi096.stdout:8/903: readlink d8/d11/d22/d4c/d65/d1d/l9e 0 2024-03-24T10:46:22.292 INFO:tasks.workunit.client.1.smithi096.stdout:8/904: read d8/d11/d44/ff4 [22084,102495] 0 2024-03-24T10:46:22.294 INFO:tasks.workunit.client.1.smithi096.stdout:5/863: mkdir d2/d49/d128 0 2024-03-24T10:46:22.294 INFO:tasks.workunit.client.1.smithi096.stdout:5/864: fsync d2/d29/d32/d33/d8d/f11a 0 2024-03-24T10:46:22.294 INFO:tasks.workunit.client.1.smithi096.stdout:9/912: rmdir d9/db/d6d/dc6/df0 39 2024-03-24T10:46:22.295 INFO:tasks.workunit.client.1.smithi096.stdout:9/913: stat d9/d65/f9f 0 2024-03-24T10:46:22.295 INFO:tasks.workunit.client.1.smithi096.stdout:9/914: fdatasync d9/d65/d126/d29/d73/d49/f50 0 2024-03-24T10:46:22.297 INFO:tasks.workunit.client.1.smithi096.stdout:4/751: mknod d1/d32/d4c/d7d/cf7 0 2024-03-24T10:46:22.301 INFO:tasks.workunit.client.1.smithi096.stdout:6/836: creat d8/d12/d28/d36/d65/d6b/f107 x:0 0 0 2024-03-24T10:46:22.302 INFO:tasks.workunit.client.1.smithi096.stdout:3/828: truncate d0/d9/f3b 953113 0 2024-03-24T10:46:22.302 INFO:tasks.workunit.client.1.smithi096.stdout:3/829: dread - d0/d13/d77/fb4 zero size 2024-03-24T10:46:22.304 INFO:tasks.workunit.client.1.smithi096.stdout:8/905: rename d8/d11/d22/d4c/d67/c9d to d8/d11/d22/d4c/d65/d1a/d49/c11b 0 2024-03-24T10:46:22.305 INFO:tasks.workunit.client.1.smithi096.stdout:0/955: dwrite d0/d11/dc3/f128 [0,4194304] 0 2024-03-24T10:46:22.307 INFO:tasks.workunit.client.1.smithi096.stdout:9/915: mknod d9/db/d6d/d7f/c12b 0 2024-03-24T10:46:22.311 INFO:tasks.workunit.client.1.smithi096.stdout:0/956: dread d0/d11/d20/d65/fe3 [0,4194304] 0 2024-03-24T10:46:22.314 INFO:tasks.workunit.client.1.smithi096.stdout:8/906: dread d8/f16 [0,4194304] 0 2024-03-24T10:46:22.314 INFO:tasks.workunit.client.1.smithi096.stdout:8/907: chown d8/d11/d22/d4c/d65/d52/lf3 15 1 2024-03-24T10:46:22.317 INFO:tasks.workunit.client.1.smithi096.stdout:3/830: truncate d0/d9/fbe 2127874 0 2024-03-24T10:46:22.317 INFO:tasks.workunit.client.1.smithi096.stdout:3/831: fsync d0/d13/d1e/fe8 0 2024-03-24T10:46:22.319 INFO:tasks.workunit.client.1.smithi096.stdout:5/865: dwrite d2/d29/d32/d33/d8d/f9e [0,4194304] 0 2024-03-24T10:46:22.324 INFO:tasks.workunit.client.1.smithi096.stdout:9/916: mkdir d9/df1/d12c 0 2024-03-24T10:46:22.325 INFO:tasks.workunit.client.1.smithi096.stdout:9/917: dread - d9/d7c/f10c zero size 2024-03-24T10:46:22.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:21 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:46:22.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:21 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:46:22.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:21 smithi096 ceph-mon[38097]: pgmap v254: 65 pgs: 65 active+clean; 2.8 GiB data, 9.8 GiB used, 527 GiB / 536 GiB avail; 169 MiB/s rd, 206 MiB/s wr, 354 op/s 2024-03-24T10:46:22.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:21 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:46:22.326 INFO:tasks.workunit.client.1.smithi096.stdout:0/957: symlink d0/d11/d20/d9e/l145 0 2024-03-24T10:46:22.327 INFO:tasks.workunit.client.1.smithi096.stdout:6/837: unlink d8/d12/d28/f33 0 2024-03-24T10:46:22.329 INFO:tasks.workunit.client.1.smithi096.stdout:8/908: rmdir d8/d11/d22/d4c/d65/d52/dc0/dd9 39 2024-03-24T10:46:22.329 INFO:tasks.workunit.client.1.smithi096.stdout:8/909: dread - d8/d11/d22/d4c/d65/df0/f10f zero size 2024-03-24T10:46:22.330 INFO:tasks.workunit.client.1.smithi096.stdout:6/838: dread d8/d10/d76/f106 [0,4194304] 0 2024-03-24T10:46:22.332 INFO:tasks.workunit.client.1.smithi096.stdout:3/832: dwrite d0/d13/f83 [0,4194304] 0 2024-03-24T10:46:22.332 INFO:tasks.workunit.client.1.smithi096.stdout:3/833: fsync d0/d13/d45/d7d/d39/d48/f6a 0 2024-03-24T10:46:22.332 INFO:tasks.workunit.client.1.smithi096.stdout:3/834: chown d0/d13/d45/d7d/fea 11409198 1 2024-03-24T10:46:22.334 INFO:tasks.workunit.client.1.smithi096.stdout:5/866: creat d2/d7/d8/df/d8a/d10a/d10b/f129 x:0 0 0 2024-03-24T10:46:22.335 INFO:tasks.workunit.client.1.smithi096.stdout:9/918: rename d9/d65/d126/d29/d73/d43/f85 to d9/d65/d126/d29/d73/d49/d86/f12d 0 2024-03-24T10:46:22.339 INFO:tasks.workunit.client.1.smithi096.stdout:0/958: creat d0/d11/d16/d21/d81/f146 x:0 0 0 2024-03-24T10:46:22.350 INFO:tasks.workunit.client.1.smithi096.stdout:8/910: mknod d8/d11/d85/d93/c11c 0 2024-03-24T10:46:22.350 INFO:tasks.workunit.client.1.smithi096.stdout:8/911: dread - d8/d11/d22/d4c/d82/da1/dd4/de7/ffd zero size 2024-03-24T10:46:22.350 INFO:tasks.workunit.client.1.smithi096.stdout:8/912: write d8/d11/fe9 [1243072,55935] 0 2024-03-24T10:46:22.351 INFO:tasks.workunit.client.1.smithi096.stdout:3/835: creat d0/d9/f112 x:0 0 0 2024-03-24T10:46:22.354 INFO:tasks.workunit.client.1.smithi096.stdout:5/867: creat d2/d29/d32/d33/d3c/dd2/f12a x:0 0 0 2024-03-24T10:46:22.354 INFO:tasks.workunit.client.1.smithi096.stdout:5/868: truncate d2/d7/f126 699762 0 2024-03-24T10:46:22.355 INFO:tasks.workunit.client.1.smithi096.stdout:5/869: write d2/d7/f4a [423893,108668] 0 2024-03-24T10:46:22.355 INFO:tasks.workunit.client.1.smithi096.stdout:9/919: dwrite d9/d65/d126/d29/d73/d43/d48/fed [0,4194304] 0 2024-03-24T10:46:22.356 INFO:tasks.workunit.client.1.smithi096.stdout:6/839: rename d8/d12/l78 to d8/d12/d32/d38/d59/db8/l108 0 2024-03-24T10:46:22.358 INFO:tasks.workunit.client.1.smithi096.stdout:8/913: creat d8/d11/d22/d4c/d65/d52/dc0/dd9/d10a/f11d x:0 0 0 2024-03-24T10:46:22.358 INFO:tasks.workunit.client.1.smithi096.stdout:8/914: write d8/d11/d22/d4c/d65/d52/dc0/dd9/d10a/f10d [539906,96016] 0 2024-03-24T10:46:22.363 INFO:tasks.workunit.client.1.smithi096.stdout:3/836: rename d0/d9/c6f to d0/d9/c113 0 2024-03-24T10:46:22.365 INFO:tasks.workunit.client.1.smithi096.stdout:9/920: dread d9/d65/d126/f33 [0,4194304] 0 2024-03-24T10:46:22.365 INFO:tasks.workunit.client.1.smithi096.stdout:9/921: truncate d9/db/d6d/dc6/fe0 635525 0 2024-03-24T10:46:22.365 INFO:tasks.workunit.client.1.smithi096.stdout:5/870: chown d2/d7/l22 1883255 1 2024-03-24T10:46:22.365 INFO:tasks.workunit.client.1.smithi096.stdout:0/959: dwrite d0/d11/d24/f45 [4194304,4194304] 0 2024-03-24T10:46:22.365 INFO:tasks.workunit.client.1.smithi096.stdout:9/922: write d9/d65/d126/d29/d73/d43/d48/f98 [4344614,22668] 0 2024-03-24T10:46:22.366 INFO:tasks.workunit.client.1.smithi096.stdout:9/923: chown d9/db/fa4 20134 1 2024-03-24T10:46:22.372 INFO:tasks.workunit.client.1.smithi096.stdout:6/840: symlink d8/d10/d13/d63/l109 0 2024-03-24T10:46:22.372 INFO:tasks.workunit.client.1.smithi096.stdout:6/841: write d8/d12/d32/fee [664285,23297] 0 2024-03-24T10:46:22.373 INFO:tasks.workunit.client.1.smithi096.stdout:8/915: dwrite d8/d11/d22/d4c/d65/d1d/f2b [4194304,4194304] 0 2024-03-24T10:46:22.373 INFO:tasks.workunit.client.1.smithi096.stdout:8/916: chown d8/d11/d22/d4c/d65/d1a/c75 8 1 2024-03-24T10:46:22.375 INFO:tasks.workunit.client.1.smithi096.stdout:0/960: rmdir d0/d11/d20/d2b/d11a 0 2024-03-24T10:46:22.376 INFO:tasks.workunit.client.1.smithi096.stdout:0/961: truncate d0/d11/d16/d21/d78/d92/f112 969088 0 2024-03-24T10:46:22.376 INFO:tasks.workunit.client.1.smithi096.stdout:0/962: truncate d0/d11/d16/d21/d25/f101 21025 0 2024-03-24T10:46:22.376 INFO:tasks.workunit.client.1.smithi096.stdout:9/924: dread d9/f3a [0,4194304] 0 2024-03-24T10:46:22.376 INFO:tasks.workunit.client.1.smithi096.stdout:9/925: dread - d9/d79/fd7 zero size 2024-03-24T10:46:22.377 INFO:tasks.workunit.client.1.smithi096.stdout:6/842: mknod d8/d12/d28/d36/d65/d6b/da6/c10a 0 2024-03-24T10:46:22.379 INFO:tasks.workunit.client.1.smithi096.stdout:3/837: getdents d0/d13/d77 0 2024-03-24T10:46:22.379 INFO:tasks.workunit.client.1.smithi096.stdout:3/838: write d0/d13/d41/ddf/fd4 [554354,26074] 0 2024-03-24T10:46:22.380 INFO:tasks.workunit.client.1.smithi096.stdout:8/917: mknod d8/d11/d22/d4c/d65/df0/c11e 0 2024-03-24T10:46:22.380 INFO:tasks.workunit.client.1.smithi096.stdout:8/918: write d8/d11/d85/fda [4146444,16065] 0 2024-03-24T10:46:22.383 INFO:tasks.workunit.client.1.smithi096.stdout:0/963: mkdir d0/d11/d20/d56/d109/d147 0 2024-03-24T10:46:22.385 INFO:tasks.workunit.client.1.smithi096.stdout:6/843: unlink d8/d12/d32/d38/l56 0 2024-03-24T10:46:22.387 INFO:tasks.workunit.client.1.smithi096.stdout:3/839: symlink d0/d13/d77/l114 0 2024-03-24T10:46:22.389 INFO:tasks.workunit.client.1.smithi096.stdout:8/919: mkdir d8/d11/d22/d4c/d65/d1a/d46/d117/d11f 0 2024-03-24T10:46:22.392 INFO:tasks.workunit.client.1.smithi096.stdout:6/844: symlink d8/d12/d28/d36/d65/d71/l10b 0 2024-03-24T10:46:22.392 INFO:tasks.workunit.client.1.smithi096.stdout:6/845: readlink d8/d12/d28/d36/d65/d71/l87 0 2024-03-24T10:46:22.393 INFO:tasks.workunit.client.1.smithi096.stdout:0/964: getdents d0/d11/d20/d91 0 2024-03-24T10:46:22.396 INFO:tasks.workunit.client.1.smithi096.stdout:0/965: creat d0/d11/d16/f148 x:0 0 0 2024-03-24T10:46:22.398 INFO:tasks.workunit.client.1.smithi096.stdout:0/966: symlink d0/d11/d24/d2c/l149 0 2024-03-24T10:46:22.398 INFO:tasks.workunit.client.1.smithi096.stdout:0/967: fdatasync d0/d11/d20/d56/d109/f143 0 2024-03-24T10:46:22.398 INFO:tasks.workunit.client.1.smithi096.stdout:6/846: write d8/d12/d32/d38/d59/db8/fbb [447409,15798] 0 2024-03-24T10:46:22.398 INFO:tasks.workunit.client.1.smithi096.stdout:0/968: chown d0/d11/d16/d21/d25/c124 39 1 2024-03-24T10:46:22.406 INFO:tasks.workunit.client.1.smithi096.stdout:6/847: rename d8/d10/d13/d63/d51 to d8/d7b/d10c 0 2024-03-24T10:46:22.409 INFO:tasks.workunit.client.1.smithi096.stdout:6/848: rename d8/d12/d28/d36/d65/d71/dd0 to d8/d12/d28/d36/d65/d71/d10d 0 2024-03-24T10:46:22.417 INFO:tasks.workunit.client.1.smithi096.stdout:3/840: dwrite d0/d13/d1e/f2f [4194304,4194304] 0 2024-03-24T10:46:22.419 INFO:tasks.workunit.client.1.smithi096.stdout:3/841: symlink d0/d13/d45/d7d/d39/d48/dbd/dbc/l115 0 2024-03-24T10:46:22.421 INFO:tasks.workunit.client.1.smithi096.stdout:8/920: dwrite d8/d11/d22/d4c/d65/d1a/d49/f6a [4194304,4194304] 0 2024-03-24T10:46:22.423 INFO:tasks.workunit.client.1.smithi096.stdout:8/921: creat d8/d11/d44/f120 x:0 0 0 2024-03-24T10:46:22.423 INFO:tasks.workunit.client.1.smithi096.stdout:8/922: chown d8/d11/d22/d4c/d65/d37 453437 1 2024-03-24T10:46:22.423 INFO:tasks.workunit.client.1.smithi096.stdout:8/923: read d8/d11/d22/d4c/d65/d1d/d3d/f56 [2866422,57748] 0 2024-03-24T10:46:22.429 INFO:tasks.workunit.client.1.smithi096.stdout:6/849: dwrite d8/d12/f2d [0,4194304] 0 2024-03-24T10:46:22.429 INFO:tasks.workunit.client.1.smithi096.stdout:6/850: dread - d8/d12/d28/d36/d65/d6b/f107 zero size 2024-03-24T10:46:22.434 INFO:tasks.workunit.client.1.smithi096.stdout:8/924: write d8/d11/d22/d4c/d65/d1a/d49/fc8 [2661503,65942] 0 2024-03-24T10:46:22.434 INFO:tasks.workunit.client.1.smithi096.stdout:0/969: dwrite d0/d11/d16/d21/d81/d9c/dcd/d10a/fba [4194304,4194304] 0 2024-03-24T10:46:22.434 INFO:tasks.workunit.client.1.smithi096.stdout:0/970: truncate d0/d11/d16/fbe 566769 0 2024-03-24T10:46:22.434 INFO:tasks.workunit.client.1.smithi096.stdout:0/971: chown d0/d11/d20/d2b/ddf/f141 3477425 1 2024-03-24T10:46:22.437 INFO:tasks.workunit.client.1.smithi096.stdout:3/842: dwrite d0/d13/d45/d47/d8f/f10c [0,4194304] 0 2024-03-24T10:46:22.437 INFO:tasks.workunit.client.1.smithi096.stdout:3/843: fdatasync d0/d13/d1e/f2f 0 2024-03-24T10:46:22.439 INFO:tasks.workunit.client.1.smithi096.stdout:6/851: dread d8/d12/d32/d68/fc9 [0,4194304] 0 2024-03-24T10:46:22.439 INFO:tasks.workunit.client.1.smithi096.stdout:6/852: stat d8/d10/d19/f30 0 2024-03-24T10:46:22.440 INFO:tasks.workunit.client.1.smithi096.stdout:8/925: mknod d8/d72/d96/d91/dfc/c121 0 2024-03-24T10:46:22.449 INFO:tasks.workunit.client.1.smithi096.stdout:6/853: dread d8/d10/d13/dcc/fda [0,4194304] 0 2024-03-24T10:46:22.449 INFO:tasks.workunit.client.1.smithi096.stdout:6/854: readlink d8/d10/d76/d91/de0/lf4 0 2024-03-24T10:46:22.451 INFO:tasks.workunit.client.1.smithi096.stdout:3/844: creat d0/d13/d41/f116 x:0 0 0 2024-03-24T10:46:22.453 INFO:tasks.workunit.client.1.smithi096.stdout:6/855: mkdir d8/d12/d32/d10e 0 2024-03-24T10:46:22.456 INFO:tasks.workunit.client.1.smithi096.stdout:6/856: mknod d8/d12/d32/d38/c10f 0 2024-03-24T10:46:22.459 INFO:tasks.workunit.client.1.smithi096.stdout:3/845: mkdir d0/d13/d77/d117 0 2024-03-24T10:46:22.461 INFO:tasks.workunit.client.1.smithi096.stdout:3/846: symlink d0/d9/d22/d102/db6/l118 0 2024-03-24T10:46:22.463 INFO:tasks.workunit.client.1.smithi096.stdout:0/972: dwrite d0/fb0 [0,4194304] 0 2024-03-24T10:46:22.463 INFO:tasks.workunit.client.1.smithi096.stdout:0/973: rename d0/d11/d20/d9e/d80/d110 to d0/d11/d20/d9e/d80/d110/d13b/d14a 22 2024-03-24T10:46:22.463 INFO:tasks.workunit.client.1.smithi096.stdout:8/926: dwrite d8/f14 [0,4194304] 0 2024-03-24T10:46:22.464 INFO:tasks.workunit.client.1.smithi096.stdout:8/927: dread - d8/d11/d22/d4c/d82/da1/f108 zero size 2024-03-24T10:46:22.464 INFO:tasks.workunit.client.1.smithi096.stdout:3/847: link d0/d13/d45/d7d/f63 d0/d92/ddc/dfc/de5/f119 0 2024-03-24T10:46:22.465 INFO:tasks.workunit.client.1.smithi096.stdout:3/848: chown d0/d13/d45/d7d/d39/d48/dbd/d9c/fa6 15 1 2024-03-24T10:46:22.466 INFO:tasks.workunit.client.1.smithi096.stdout:6/857: dread f2 [0,4194304] 0 2024-03-24T10:46:22.470 INFO:tasks.workunit.client.1.smithi096.stdout:0/974: truncate d0/d11/d16/d21/d25/f46 2876420 0 2024-03-24T10:46:22.470 INFO:tasks.workunit.client.1.smithi096.stdout:0/975: chown d0/d11/d24/l86 13007901 1 2024-03-24T10:46:22.470 INFO:tasks.workunit.client.1.smithi096.stdout:0/976: fdatasync d0/d11/d24/f45 0 2024-03-24T10:46:22.470 INFO:tasks.workunit.client.1.smithi096.stdout:0/977: creat d0/d11/d24/d2c/d84/f14b x:0 0 0 2024-03-24T10:46:22.470 INFO:tasks.workunit.client.1.smithi096.stdout:8/928: getdents d8/d72/d96/d91/dfc 0 2024-03-24T10:46:22.473 INFO:tasks.workunit.client.1.smithi096.stdout:6/858: unlink d8/fdc 0 2024-03-24T10:46:22.475 INFO:tasks.workunit.client.1.smithi096.stdout:8/929: mkdir d8/d72/dd0/d122 0 2024-03-24T10:46:22.479 INFO:tasks.workunit.client.1.smithi096.stdout:6/859: rename d8/d10/d13/dd8/dde to d8/d12/d28/d36/d65/d6b/da6/db2/df8/d110 0 2024-03-24T10:46:22.479 INFO:tasks.workunit.client.1.smithi096.stdout:6/860: chown d8/d10/d19/c84 646535 1 2024-03-24T10:46:22.479 INFO:tasks.workunit.client.1.smithi096.stdout:6/861: write d8/d12/d28/d36/d65/f92 [893363,8409] 0 2024-03-24T10:46:22.480 INFO:tasks.workunit.client.1.smithi096.stdout:0/978: dread d0/d11/d20/d56/fb2 [0,4194304] 0 2024-03-24T10:46:22.481 INFO:tasks.workunit.client.1.smithi096.stdout:3/849: dwrite d0/d9/d22/fc2 [0,4194304] 0 2024-03-24T10:46:22.484 INFO:tasks.workunit.client.1.smithi096.stdout:8/930: dread d8/d11/d22/d4c/f9f [0,4194304] 0 2024-03-24T10:46:22.485 INFO:tasks.workunit.client.1.smithi096.stdout:6/862: unlink d8/f31 0 2024-03-24T10:46:22.485 INFO:tasks.workunit.client.1.smithi096.stdout:6/863: write d8/d12/d32/d38/fd1 [6002,95296] 0 2024-03-24T10:46:22.489 INFO:tasks.workunit.client.1.smithi096.stdout:0/979: unlink d0/d11/d24/d11f/l12d 0 2024-03-24T10:46:22.489 INFO:tasks.workunit.client.1.smithi096.stdout:0/980: chown d0/d11/d20/f26 1 1 2024-03-24T10:46:22.503 INFO:tasks.workunit.client.1.smithi096.stdout:3/850: dwrite d0/d13/d1e/f2a [0,4194304] 0 2024-03-24T10:46:22.513 INFO:tasks.workunit.client.1.smithi096.stdout:6/864: link d8/d12/d28/d36/d65/d71/l87 d8/d12/d32/d38/d59/db8/l111 0 2024-03-24T10:46:22.513 INFO:tasks.workunit.client.1.smithi096.stdout:6/865: chown d8 2677725 1 2024-03-24T10:46:22.514 INFO:tasks.workunit.client.1.smithi096.stdout:0/981: dwrite d0/d11/d16/d21/d81/d9c/dcd/d10a/f11e [0,4194304] 0 2024-03-24T10:46:22.514 INFO:tasks.workunit.client.1.smithi096.stdout:8/931: dwrite d8/d72/f83 [0,4194304] 0 2024-03-24T10:46:22.518 INFO:tasks.workunit.client.1.smithi096.stdout:0/982: dread d0/d11/d24/f30 [0,4194304] 0 2024-03-24T10:46:22.518 INFO:tasks.workunit.client.1.smithi096.stdout:3/851: chown d0/d13/d45/d47/d8f/lf2 1 1 2024-03-24T10:46:22.518 INFO:tasks.workunit.client.1.smithi096.stdout:3/852: stat d0/d92/ddc/dfc/l110 0 2024-03-24T10:46:22.522 INFO:tasks.workunit.client.1.smithi096.stdout:6/866: mkdir d8/d12/d28/d36/d65/d71/d10d/da8/dbe/d112 0 2024-03-24T10:46:22.532 INFO:tasks.workunit.client.1.smithi096.stdout:8/932: creat d8/d11/d22/ded/f123 x:0 0 0 2024-03-24T10:46:22.535 INFO:tasks.workunit.client.1.smithi096.stdout:3/853: getdents d0/d13/d45/d7d/d39/d48/dbd/dbc 0 2024-03-24T10:46:22.538 INFO:tasks.workunit.client.1.smithi096.stdout:6/867: getdents d8/d12/d28/d36/d65/d71/d10d/da8 0 2024-03-24T10:46:22.540 INFO:tasks.workunit.client.1.smithi096.stdout:8/933: creat d8/d11/d22/d4c/d65/d1a/da0/f124 x:0 0 0 2024-03-24T10:46:22.544 INFO:tasks.workunit.client.1.smithi096.stdout:6/868: mkdir d8/d12/d61/d113 0 2024-03-24T10:46:22.545 INFO:tasks.workunit.client.1.smithi096.stdout:8/934: symlink d8/d11/d22/d4c/d65/d37/l125 0 2024-03-24T10:46:22.546 INFO:tasks.workunit.client.1.smithi096.stdout:3/854: mknod d0/d13/d45/d47/d8f/c11a 0 2024-03-24T10:46:22.548 INFO:tasks.workunit.client.1.smithi096.stdout:6/869: rename d8/d10/d13/l2f to d8/d12/d32/d38/l114 0 2024-03-24T10:46:22.548 INFO:tasks.workunit.client.1.smithi096.stdout:1/815: sync 2024-03-24T10:46:22.548 INFO:tasks.workunit.client.1.smithi096.stdout:4/752: sync 2024-03-24T10:46:22.548 INFO:tasks.workunit.client.1.smithi096.stdout:1/816: fsync d3/d23/d8b/d99/f10e 0 2024-03-24T10:46:22.549 INFO:tasks.workunit.client.1.smithi096.stdout:4/753: creat d1/d32/d4c/d7e/ff8 x:0 0 0 2024-03-24T10:46:22.549 INFO:tasks.workunit.client.1.smithi096.stdout:4/754: chown d1/d2b/d67/d5f/l85 10549556 1 2024-03-24T10:46:22.549 INFO:tasks.workunit.client.1.smithi096.stdout:4/755: fdatasync d1/d2b/d6b/d80/fea 0 2024-03-24T10:46:22.554 INFO:tasks.workunit.client.1.smithi096.stdout:1/817: rmdir d3/d23/d8b/d6a 39 2024-03-24T10:46:22.555 INFO:tasks.workunit.client.1.smithi096.stdout:8/935: dread d8/fb3 [0,4194304] 0 2024-03-24T10:46:22.556 INFO:tasks.workunit.client.1.smithi096.stdout:3/855: dread d0/d13/d45/d7d/d39/d48/dbd/d9c/ff0 [0,4194304] 0 2024-03-24T10:46:22.557 INFO:tasks.workunit.client.1.smithi096.stdout:1/818: stat d3/d23/d71/d7b/d93/lf0 0 2024-03-24T10:46:22.558 INFO:tasks.workunit.client.1.smithi096.stdout:6/870: write d8/d12/d32/d38/d59/db8/fb1 [1816986,103707] 0 2024-03-24T10:46:22.558 INFO:tasks.workunit.client.1.smithi096.stdout:8/936: rename d8/f14 to d8/d11/d22/d4c/d65/d1a/d46/d117/f126 0 2024-03-24T10:46:22.561 INFO:tasks.workunit.client.1.smithi096.stdout:1/819: creat d3/d73/ded/f119 x:0 0 0 2024-03-24T10:46:22.561 INFO:tasks.workunit.client.1.smithi096.stdout:1/820: chown d3/d23/fb7 0 1 2024-03-24T10:46:22.561 INFO:tasks.workunit.client.1.smithi096.stdout:1/821: write d3/d73/f7a [1281134,27284] 0 2024-03-24T10:46:22.564 INFO:tasks.workunit.client.1.smithi096.stdout:4/756: dwrite d1/d32/d4c/da6/faa [0,4194304] 0 2024-03-24T10:46:22.566 INFO:tasks.workunit.client.1.smithi096.stdout:3/856: rename d0/d13/d45/d52 to d0/d13/d41/ddf/d11b 0 2024-03-24T10:46:22.566 INFO:tasks.workunit.client.1.smithi096.stdout:3/857: readlink d0/d9/la2 0 2024-03-24T10:46:22.566 INFO:tasks.workunit.client.1.smithi096.stdout:3/858: chown d0/d92/ddc/dfc/de5/lfe 778 1 2024-03-24T10:46:22.567 INFO:tasks.workunit.client.1.smithi096.stdout:8/937: truncate d8/d11/d22/d4c/d65/d52/f57 2235240 0 2024-03-24T10:46:22.570 INFO:tasks.workunit.client.1.smithi096.stdout:1/822: link d3/le d3/d23/d8b/l11a 0 2024-03-24T10:46:22.572 INFO:tasks.workunit.client.1.smithi096.stdout:6/871: creat d8/d10/d19/dfd/f115 x:0 0 0 2024-03-24T10:46:22.578 INFO:tasks.workunit.client.1.smithi096.stdout:4/757: dwrite d1/d2b/d6b/d80/d9c/fe3 [0,4194304] 0 2024-03-24T10:46:22.580 INFO:tasks.workunit.client.1.smithi096.stdout:8/938: mkdir d8/d11/d85/d93/d127 0 2024-03-24T10:46:22.580 INFO:tasks.workunit.client.1.smithi096.stdout:8/939: stat d8/d11/f19 0 2024-03-24T10:46:22.582 INFO:tasks.workunit.client.1.smithi096.stdout:6/872: symlink d8/d12/d32/d10e/l116 0 2024-03-24T10:46:22.583 INFO:tasks.workunit.client.1.smithi096.stdout:6/873: rename d8/d12/d28/d36/d65 to d8/d12/d28/d36/d65/d71/d117 22 2024-03-24T10:46:22.587 INFO:tasks.workunit.client.1.smithi096.stdout:1/823: dwrite d3/d23/d8b/d5c/f92 [0,4194304] 0 2024-03-24T10:46:22.590 INFO:tasks.workunit.client.1.smithi096.stdout:3/859: symlink d0/d13/d77/l11c 0 2024-03-24T10:46:22.590 INFO:tasks.workunit.client.1.smithi096.stdout:3/860: chown d0/d13/d45/d7d/fff 87 1 2024-03-24T10:46:22.591 INFO:tasks.workunit.client.1.smithi096.stdout:8/940: mkdir d8/d11/d22/d4c/d65/d1a/d46/d2e/d128 0 2024-03-24T10:46:22.591 INFO:tasks.workunit.client.1.smithi096.stdout:8/941: readlink d8/d11/l7c 0 2024-03-24T10:46:22.602 INFO:tasks.workunit.client.1.smithi096.stdout:1/824: dwrite d3/d23/d2f/f7e [0,4194304] 0 2024-03-24T10:46:22.603 INFO:tasks.workunit.client.1.smithi096.stdout:1/825: stat d3/d9b/fa4 0 2024-03-24T10:46:22.604 INFO:tasks.workunit.client.1.smithi096.stdout:8/942: mknod d8/d72/d96/d91/dfc/c129 0 2024-03-24T10:46:22.605 INFO:tasks.workunit.client.1.smithi096.stdout:8/943: fdatasync d8/d11/d22/d4c/d65/d1a/da0/da6/f112 0 2024-03-24T10:46:22.607 INFO:tasks.workunit.client.1.smithi096.stdout:1/826: rename d3/d23/d2f/f7e to d3/d36/f11b 0 2024-03-24T10:46:22.607 INFO:tasks.workunit.client.1.smithi096.stdout:1/827: creat d3/d23/d2f/d65/d7c/ddf/f11c x:0 0 0 2024-03-24T10:46:22.610 INFO:tasks.workunit.client.1.smithi096.stdout:3/861: getdents d0/d9/d22 0 2024-03-24T10:46:22.610 INFO:tasks.workunit.client.1.smithi096.stdout:3/862: creat d0/d13/d45/d7d/d39/d48/d8d/f11d x:0 0 0 2024-03-24T10:46:22.610 INFO:tasks.workunit.client.1.smithi096.stdout:8/944: creat d8/d11/d22/d4c/d65/d1a/d46/d117/f12a x:0 0 0 2024-03-24T10:46:22.610 INFO:tasks.workunit.client.1.smithi096.stdout:8/945: chown d8/d11/d22/d4c/d65/d1a/d46/d2e/fbe 4630 1 2024-03-24T10:46:22.611 INFO:tasks.workunit.client.1.smithi096.stdout:8/946: readlink d8/d11/d22/d4c/d82/da1/dd4/l100 0 2024-03-24T10:46:22.627 INFO:tasks.workunit.client.1.smithi096.stdout:1/828: dwrite d3/d23/d2f/d65/d7c/ddf/ff5 [0,4194304] 0 2024-03-24T10:46:22.628 INFO:tasks.workunit.client.1.smithi096.stdout:1/829: chown d3/d23/d8b/d5c/cbb 39641352 1 2024-03-24T10:46:22.631 INFO:tasks.workunit.client.1.smithi096.stdout:1/830: rename d3/d23/d8b/d5c/f75 to d3/d36/f11d 0 2024-03-24T10:46:22.632 INFO:tasks.workunit.client.1.smithi096.stdout:1/831: fdatasync d3/d23/d8b/d8c/ddc/fde 0 2024-03-24T10:46:22.638 INFO:tasks.workunit.client.1.smithi096.stdout:8/947: dwrite d8/f92 [0,4194304] 0 2024-03-24T10:46:22.638 INFO:tasks.workunit.client.1.smithi096.stdout:3/863: dwrite d0/d13/d77/fb4 [0,4194304] 0 2024-03-24T10:46:22.638 INFO:tasks.workunit.client.1.smithi096.stdout:8/948: chown d8/d72/dd0 89739 1 2024-03-24T10:46:22.658 INFO:tasks.workunit.client.1.smithi096.stdout:8/949: dwrite d8/d72/f105 [0,4194304] 0 2024-03-24T10:46:22.658 INFO:tasks.workunit.client.1.smithi096.stdout:3/864: dwrite d0/d13/d45/d47/f8e [0,4194304] 0 2024-03-24T10:46:22.665 INFO:tasks.workunit.client.1.smithi096.stdout:3/865: getdents d0/d13/d45/d7d/d39/d48/dbd/d106 0 2024-03-24T10:46:22.672 INFO:tasks.workunit.client.1.smithi096.stdout:8/950: dwrite d8/d11/d44/d45/f4d [4194304,4194304] 0 2024-03-24T10:46:22.672 INFO:tasks.workunit.client.1.smithi096.stdout:8/951: creat d8/d11/d44/f12b x:0 0 0 2024-03-24T10:46:22.672 INFO:tasks.workunit.client.1.smithi096.stdout:8/952: truncate d8/d11/d22/d4c/d65/fa2 378405 0 2024-03-24T10:46:22.680 INFO:tasks.workunit.client.1.smithi096.stdout:3/866: dwrite d0/d13/d45/d47/f76 [0,4194304] 0 2024-03-24T10:46:22.680 INFO:tasks.workunit.client.1.smithi096.stdout:3/867: fdatasync d0/d9b/fca 0 2024-03-24T10:46:22.694 INFO:tasks.workunit.client.1.smithi096.stdout:3/868: dwrite d0/d13/d45/d7d/d39/d48/f6a [0,4194304] 0 2024-03-24T10:46:22.694 INFO:tasks.workunit.client.1.smithi096.stdout:3/869: write d0/d13/d41/ddf/fd2 [341316,104279] 0 2024-03-24T10:46:22.696 INFO:tasks.workunit.client.1.smithi096.stdout:3/870: creat d0/d13/d45/d7d/d39/d48/dbd/d106/f11e x:0 0 0 2024-03-24T10:46:22.717 INFO:tasks.workunit.client.1.smithi096.stdout:3/871: dwrite d0/d9/f112 [0,4194304] 0 2024-03-24T10:46:22.735 INFO:tasks.workunit.client.1.smithi096.stdout:3/872: dwrite d0/d9/f49 [0,4194304] 0 2024-03-24T10:46:22.863 INFO:tasks.workunit.client.1.smithi096.stdout:9/926: sync 2024-03-24T10:46:22.864 INFO:tasks.workunit.client.1.smithi096.stdout:0/983: sync 2024-03-24T10:46:22.864 INFO:tasks.workunit.client.1.smithi096.stdout:5/871: sync 2024-03-24T10:46:22.864 INFO:tasks.workunit.client.1.smithi096.stdout:0/984: chown d0/d11/d16/d21/d81/d8f/l120 59074 1 2024-03-24T10:46:22.864 INFO:tasks.workunit.client.1.smithi096.stdout:5/872: write d2/d29/d32/d33/d8d/d94/fe3 [825707,24861] 0 2024-03-24T10:46:22.865 INFO:tasks.workunit.client.1.smithi096.stdout:9/927: mkdir d9/db/d6d/d4f/d12e 0 2024-03-24T10:46:22.866 INFO:tasks.workunit.client.1.smithi096.stdout:0/985: getdents d0/d11/d20/d9e/d80/d110 0 2024-03-24T10:46:22.866 INFO:tasks.workunit.client.1.smithi096.stdout:0/986: chown d0/d11/d20/d9e/d80/d142 870777 1 2024-03-24T10:46:22.870 INFO:tasks.workunit.client.1.smithi096.stdout:5/873: mknod d2/d29/d32/d3f/d84/c12b 0 2024-03-24T10:46:22.881 INFO:tasks.workunit.client.1.smithi096.stdout:9/928: rmdir d9/db/d6d/d4f/d75 39 2024-03-24T10:46:22.882 INFO:tasks.workunit.client.1.smithi096.stdout:9/929: write d9/db/d6d/d7f/f3f [2001211,29220] 0 2024-03-24T10:46:22.883 INFO:tasks.workunit.client.1.smithi096.stdout:0/987: write d0/d11/d16/d21/d25/d47/f7c [5681099,128942] 0 2024-03-24T10:46:22.885 INFO:tasks.workunit.client.1.smithi096.stdout:9/930: rename d9/d15/f66 to d9/d102/f12f 0 2024-03-24T10:46:22.885 INFO:tasks.workunit.client.1.smithi096.stdout:9/931: chown d9/db/l55 130610 1 2024-03-24T10:46:22.886 INFO:tasks.workunit.client.1.smithi096.stdout:0/988: rmdir d0/d11/d16/d21/d78/d82 39 2024-03-24T10:46:22.888 INFO:tasks.workunit.client.1.smithi096.stdout:0/989: mknod d0/d11/d24/d9d/df5/c14c 0 2024-03-24T10:46:22.889 INFO:tasks.workunit.client.1.smithi096.stdout:0/990: truncate d0/d11/d16/d21/d81/d7f/f64 1414907 0 2024-03-24T10:46:22.891 INFO:tasks.workunit.client.1.smithi096.stdout:0/991: mkdir d0/d11/d16/d21/d78/d82/d14d 0 2024-03-24T10:46:22.895 INFO:tasks.workunit.client.1.smithi096.stdout:3/873: dwrite d0/d92/ddc/dfc/de5/ff6 [0,4194304] 0 2024-03-24T10:46:22.897 INFO:tasks.workunit.client.1.smithi096.stdout:5/874: dwrite d2/d29/d32/d33/d8d/fa2 [0,4194304] 0 2024-03-24T10:46:22.898 INFO:tasks.workunit.client.1.smithi096.stdout:9/932: read d9/d65/d126/d29/f4e [1220396,9572] 0 2024-03-24T10:46:22.898 INFO:tasks.workunit.client.1.smithi096.stdout:9/933: chown d9/d65/d126/d29/d73/d43/d48/d51/lbb 507 1 2024-03-24T10:46:22.901 INFO:tasks.workunit.client.1.smithi096.stdout:5/875: mknod d2/d7/d8/df/d8a/d10a/c12c 0 2024-03-24T10:46:22.902 INFO:tasks.workunit.client.1.smithi096.stdout:5/876: dread - d2/d29/d32/d33/d3c/dc3/f105 zero size 2024-03-24T10:46:22.934 INFO:tasks.workunit.client.1.smithi096.stdout:0/992: dwrite d0/d11/d20/d91/f96 [0,4194304] 0 2024-03-24T10:46:22.934 INFO:tasks.workunit.client.1.smithi096.stdout:0/993: mkdir d0/d11/d16/d21/d78/d82/d14d/d14e 0 2024-03-24T10:46:22.936 INFO:tasks.workunit.client.1.smithi096.stdout:3/874: dwrite d0/d13/d41/ddf/fd2 [0,4194304] 0 2024-03-24T10:46:22.936 INFO:tasks.workunit.client.1.smithi096.stdout:3/875: dread - d0/d92/ddc/dfc/fd5 zero size 2024-03-24T10:46:22.937 INFO:tasks.workunit.client.1.smithi096.stdout:5/877: dwrite d2/d29/d32/fe5 [0,4194304] 0 2024-03-24T10:46:22.937 INFO:tasks.workunit.client.1.smithi096.stdout:9/934: dwrite d9/d65/d126/d29/d73/d43/f45 [0,4194304] 0 2024-03-24T10:46:22.937 INFO:tasks.workunit.client.1.smithi096.stdout:9/935: truncate d9/d65/d126/d29/f6a 4656373 0 2024-03-24T10:46:22.937 INFO:tasks.workunit.client.1.smithi096.stdout:9/936: fdatasync d9/db/d6d/fc2 0 2024-03-24T10:46:22.946 INFO:tasks.workunit.client.1.smithi096.stdout:0/994: write d0/d11/d16/d21/d81/d7f/f4a [3633685,101595] 0 2024-03-24T10:46:22.950 INFO:tasks.workunit.client.1.smithi096.stdout:0/995: creat d0/d11/d24/d11f/f14f x:0 0 0 2024-03-24T10:46:22.954 INFO:tasks.workunit.client.1.smithi096.stdout:9/937: dread d9/d65/d126/d29/d73/d43/d48/fed [0,4194304] 0 2024-03-24T10:46:22.956 INFO:tasks.workunit.client.1.smithi096.stdout:0/996: getdents d0/d11/d24/d102 0 2024-03-24T10:46:22.957 INFO:tasks.workunit.client.1.smithi096.stdout:0/997: truncate d0/d11/d24/d2c/d84/ff6 779607 0 2024-03-24T10:46:22.958 INFO:tasks.workunit.client.1.smithi096.stdout:9/938: rmdir d9/db/d6d/d4f/d75/d8b/d8e 39 2024-03-24T10:46:22.968 INFO:tasks.workunit.client.1.smithi096.stdout:5/878: dwrite d2/d29/d32/d33/d8d/f9e [0,4194304] 0 2024-03-24T10:46:22.968 INFO:tasks.workunit.client.1.smithi096.stdout:5/879: write d2/d29/fcf [550602,76929] 0 2024-03-24T10:46:22.970 INFO:tasks.workunit.client.1.smithi096.stdout:5/880: creat d2/d29/d32/d33/d8d/d94/df9/f12d x:0 0 0 2024-03-24T10:46:22.972 INFO:tasks.workunit.client.1.smithi096.stdout:5/881: symlink d2/d29/d32/d89/d119/l12e 0 2024-03-24T10:46:22.975 INFO:tasks.workunit.client.1.smithi096.stdout:9/939: dwrite d9/d7c/f117 [0,4194304] 0 2024-03-24T10:46:22.980 INFO:tasks.workunit.client.1.smithi096.stdout:9/940: truncate d9/d65/d126/d29/d73/d49/f50 1434290 0 2024-03-24T10:46:22.982 INFO:tasks.workunit.client.1.smithi096.stdout:9/941: rename d9/db/d6d/dc6/df0/fef to d9/d65/d126/d29/d123/f130 0 2024-03-24T10:46:22.989 INFO:tasks.workunit.client.1.smithi096.stdout:5/882: dwrite d2/d7/d8/df/d14/f11b [0,4194304] 0 2024-03-24T10:46:22.989 INFO:tasks.workunit.client.1.smithi096.stdout:5/883: readlink d2/d7/d8/df/d14/dea/lfb 0 2024-03-24T10:46:22.989 INFO:tasks.workunit.client.1.smithi096.stdout:5/884: creat d2/d7/d8/df/d8a/f12f x:0 0 0 2024-03-24T10:46:22.990 INFO:tasks.workunit.client.1.smithi096.stdout:9/942: write d9/db/d6d/d4f/d75/dcc/fd2 [6339229,28313] 0 2024-03-24T10:46:22.991 INFO:tasks.workunit.client.1.smithi096.stdout:9/943: readlink d9/d79/lc3 0 2024-03-24T10:46:22.991 INFO:tasks.workunit.client.1.smithi096.stdout:9/944: read - d9/db/d6d/dc6/df0/ffc zero size 2024-03-24T10:46:22.994 INFO:tasks.workunit.client.1.smithi096.stdout:9/945: write d9/d65/d126/d29/d73/f91 [179544,15056] 0 2024-03-24T10:46:22.994 INFO:tasks.workunit.client.1.smithi096.stdout:9/946: readlink d9/d79/lf3 0 2024-03-24T10:46:22.994 INFO:tasks.workunit.client.1.smithi096.stdout:9/947: write d9/d65/d126/d29/d73/d49/d86/fb9 [959985,91402] 0 2024-03-24T10:46:22.994 INFO:tasks.workunit.client.1.smithi096.stdout:9/948: chown d9/d15/f11d 1663 1 2024-03-24T10:46:22.996 INFO:tasks.workunit.client.1.smithi096.stdout:4/758: sync 2024-03-24T10:46:22.997 INFO:tasks.workunit.client.1.smithi096.stdout:4/759: write d1/d2b/d67/d5f/fc0 [699280,11564] 0 2024-03-24T10:46:22.997 INFO:tasks.workunit.client.1.smithi096.stdout:4/760: write d1/d11/f29 [935449,62080] 0 2024-03-24T10:46:22.997 INFO:tasks.workunit.client.1.smithi096.stdout:4/761: dread - d1/d32/fe2 zero size 2024-03-24T10:46:22.999 INFO:tasks.workunit.client.1.smithi096.stdout:5/885: symlink d2/d29/d32/d33/d8d/d94/d113/l130 0 2024-03-24T10:46:23.001 INFO:tasks.workunit.client.1.smithi096.stdout:4/762: symlink d1/d2b/d67/d5f/lf9 0 2024-03-24T10:46:23.003 INFO:tasks.workunit.client.1.smithi096.stdout:4/763: mknod d1/d11/cfa 0 2024-03-24T10:46:23.009 INFO:tasks.workunit.client.1.smithi096.stdout:5/886: dread d2/f5 [0,4194304] 0 2024-03-24T10:46:23.010 INFO:tasks.workunit.client.1.smithi096.stdout:9/949: dwrite d9/db/d6d/d4f/f63 [0,4194304] 0 2024-03-24T10:46:23.012 INFO:tasks.workunit.client.1.smithi096.stdout:5/887: unlink d2/d29/d32/fcb 0 2024-03-24T10:46:23.014 INFO:tasks.workunit.client.1.smithi096.stdout:9/950: mknod d9/d65/d126/d29/d73/d43/d48/c131 0 2024-03-24T10:46:23.015 INFO:tasks.workunit.client.1.smithi096.stdout:5/888: symlink d2/d7/l131 0 2024-03-24T10:46:23.016 INFO:tasks.workunit.client.1.smithi096.stdout:9/951: mknod d9/d65/d126/d29/d73/d49/db7/dca/de3/c132 0 2024-03-24T10:46:23.016 INFO:tasks.workunit.client.1.smithi096.stdout:9/952: write d9/d15/f3c [4630766,59920] 0 2024-03-24T10:46:23.018 INFO:tasks.workunit.client.1.smithi096.stdout:5/889: unlink d2/d29/d32/d33/d3c/c9f 0 2024-03-24T10:46:23.018 INFO:tasks.workunit.client.1.smithi096.stdout:5/890: truncate d2/d7/d8/df/d8a/f12f 260348 0 2024-03-24T10:46:23.018 INFO:tasks.workunit.client.1.smithi096.stdout:5/891: write d2/d7/d8/df/d14/f31 [506837,99236] 0 2024-03-24T10:46:23.020 INFO:tasks.workunit.client.1.smithi096.stdout:4/764: dwrite d1/d32/d42/dbe/fd9 [0,4194304] 0 2024-03-24T10:46:23.020 INFO:tasks.workunit.client.1.smithi096.stdout:9/953: mknod d9/db/d6d/d4f/c133 0 2024-03-24T10:46:23.020 INFO:tasks.workunit.client.1.smithi096.stdout:4/765: chown d1/d32/c63 9166 1 2024-03-24T10:46:23.020 INFO:tasks.workunit.client.1.smithi096.stdout:4/766: creat d1/d32/d42/dbe/ffb x:0 0 0 2024-03-24T10:46:23.022 INFO:tasks.workunit.client.1.smithi096.stdout:5/892: getdents d2/d29/d32/d33/d37 0 2024-03-24T10:46:23.022 INFO:tasks.workunit.client.1.smithi096.stdout:5/893: write d2/d7/d8/df/d14/d96/f127 [553921,62704] 0 2024-03-24T10:46:23.023 INFO:tasks.workunit.client.1.smithi096.stdout:9/954: creat d9/d65/d126/d29/d73/d43/db1/f134 x:0 0 0 2024-03-24T10:46:23.023 INFO:tasks.workunit.client.1.smithi096.stdout:9/955: truncate d9/fb3 1460118 0 2024-03-24T10:46:23.028 INFO:tasks.workunit.client.1.smithi096.stdout:5/894: mkdir d2/d29/d32/d3f/d84/db8/d132 0 2024-03-24T10:46:23.033 INFO:tasks.workunit.client.1.smithi096.stdout:4/767: dwrite d1/d2b/d6b/d80/fea [0,4194304] 0 2024-03-24T10:46:23.041 INFO:tasks.workunit.client.1.smithi096.stdout:4/768: dread d1/d2b/d67/f52 [0,4194304] 0 2024-03-24T10:46:23.042 INFO:tasks.workunit.client.1.smithi096.stdout:9/956: dwrite d9/df1/fde [0,4194304] 0 2024-03-24T10:46:23.045 INFO:tasks.workunit.client.1.smithi096.stdout:5/895: dwrite d2/d7/fa7 [4194304,4194304] 0 2024-03-24T10:46:23.050 INFO:tasks.workunit.client.1.smithi096.stdout:9/957: symlink d9/d15/l135 0 2024-03-24T10:46:23.053 INFO:tasks.workunit.client.1.smithi096.stdout:4/769: write d1/f24 [4125103,17321] 0 2024-03-24T10:46:23.053 INFO:tasks.workunit.client.1.smithi096.stdout:4/770: getdents d1/d32/d42/dba/df0 0 2024-03-24T10:46:23.053 INFO:tasks.workunit.client.1.smithi096.stdout:4/771: write d1/d2b/d67/f82 [218718,121412] 0 2024-03-24T10:46:23.054 INFO:tasks.workunit.client.1.smithi096.stdout:5/896: unlink d2/d29/d32/d33/d8d/df3/f115 0 2024-03-24T10:46:23.066 INFO:tasks.workunit.client.1.smithi096.stdout:4/772: dread d1/f24 [0,4194304] 0 2024-03-24T10:46:23.067 INFO:tasks.workunit.client.1.smithi096.stdout:4/773: chown d1/d2b/d6b/l95 1 1 2024-03-24T10:46:23.070 INFO:tasks.workunit.client.1.smithi096.stdout:9/958: dwrite d9/d65/d126/d29/d73/d49/f50 [0,4194304] 0 2024-03-24T10:46:23.070 INFO:tasks.workunit.client.1.smithi096.stdout:4/774: link d1/f1f d1/d2b/d6b/ffc 0 2024-03-24T10:46:23.072 INFO:tasks.workunit.client.1.smithi096.stdout:5/897: dwrite d2/d7/d69/dce/df0/f10e [0,4194304] 0 2024-03-24T10:46:23.072 INFO:tasks.workunit.client.1.smithi096.stdout:5/898: chown d2/d29/f54 1909 1 2024-03-24T10:46:23.073 INFO:tasks.workunit.client.1.smithi096.stdout:4/775: mkdir d1/d2b/d67/d5f/dfd 0 2024-03-24T10:46:23.076 INFO:tasks.workunit.client.1.smithi096.stdout:4/776: creat d1/d2b/ffe x:0 0 0 2024-03-24T10:46:23.077 INFO:tasks.workunit.client.1.smithi096.stdout:4/777: mknod d1/d2b/d6b/d80/d9c/cff 0 2024-03-24T10:46:23.081 INFO:tasks.workunit.client.1.smithi096.stdout:5/899: write d2/d7/d8/d18/f72 [6171317,82620] 0 2024-03-24T10:46:23.083 INFO:tasks.workunit.client.1.smithi096.stdout:5/900: rename d2/d7/f7f to d2/d29/d32/d33/d3c/dc3/f133 0 2024-03-24T10:46:23.087 INFO:tasks.workunit.client.1.smithi096.stdout:9/959: dwrite d9/d65/d126/d29/d73/d43/db1/f134 [0,4194304] 0 2024-03-24T10:46:23.090 INFO:tasks.workunit.client.1.smithi096.stdout:9/960: dread d9/db/d6d/d4f/d75/d8b/d8e/f8f [0,4194304] 0 2024-03-24T10:46:23.090 INFO:tasks.workunit.client.1.smithi096.stdout:9/961: chown d9/db 22215041 1 2024-03-24T10:46:23.091 INFO:tasks.workunit.client.1.smithi096.stdout:4/778: dread d1/d2b/d6b/ffc [0,4194304] 0 2024-03-24T10:46:23.092 INFO:tasks.workunit.client.1.smithi096.stdout:4/779: truncate d1/d2b/d89/fa0 948585 0 2024-03-24T10:46:23.092 INFO:tasks.workunit.client.1.smithi096.stdout:4/780: truncate d1/d32/d42/dbe/fca 1073329 0 2024-03-24T10:46:23.092 INFO:tasks.workunit.client.1.smithi096.stdout:4/781: write d1/d32/d4c/d7e/ff8 [825423,89590] 0 2024-03-24T10:46:23.092 INFO:tasks.workunit.client.1.smithi096.stdout:4/782: dread - d1/d2b/d67/d5f/da9/db5/df3/feb zero size 2024-03-24T10:46:23.093 INFO:tasks.workunit.client.1.smithi096.stdout:9/962: creat d9/d65/d126/d29/d73/d43/d48/d104/f136 x:0 0 0 2024-03-24T10:46:23.093 INFO:tasks.workunit.client.1.smithi096.stdout:9/963: fdatasync d9/d15/f20 0 2024-03-24T10:46:23.094 INFO:tasks.workunit.client.1.smithi096.stdout:4/783: creat d1/d32/d4c/d9d/db7/f100 x:0 0 0 2024-03-24T10:46:23.098 INFO:tasks.workunit.client.1.smithi096.stdout:9/964: creat d9/d102/f137 x:0 0 0 2024-03-24T10:46:23.100 INFO:tasks.workunit.client.1.smithi096.stdout:5/901: dwrite d2/d7/d8/d18/ffa [0,4194304] 0 2024-03-24T10:46:23.102 INFO:tasks.workunit.client.1.smithi096.stdout:4/784: rmdir d1/d32/d42/dbe 39 2024-03-24T10:46:23.103 INFO:tasks.workunit.client.1.smithi096.stdout:4/785: fdatasync d1/d2b/d67/d5f/da9/db5/df3/f70 0 2024-03-24T10:46:23.107 INFO:tasks.workunit.client.1.smithi096.stdout:9/965: dread d9/d65/d126/d29/d73/d43/d48/d51/f72 [0,4194304] 0 2024-03-24T10:46:23.111 INFO:tasks.workunit.client.1.smithi096.stdout:4/786: rename d1/d2b/d89/dbf/de9 to d1/d32/d4c/d9d/db7/d101 0 2024-03-24T10:46:23.112 INFO:tasks.workunit.client.1.smithi096.stdout:4/787: write d1/d2b/d67/d5f/da9/db5/df3/fa7 [557440,96381] 0 2024-03-24T10:46:23.120 INFO:tasks.workunit.client.1.smithi096.stdout:9/966: mknod d9/d102/d11c/c138 0 2024-03-24T10:46:23.121 INFO:tasks.workunit.client.1.smithi096.stdout:5/902: rmdir d2/d29/d32/d33/d3c/dd2 39 2024-03-24T10:46:23.136 INFO:tasks.workunit.client.1.smithi096.stdout:4/788: rename d1/d32/d42/c65 to d1/d32/d4c/d9d/db7/c102 0 2024-03-24T10:46:23.139 INFO:tasks.workunit.client.1.smithi096.stdout:4/789: link d1/d32/d4c/d7d/cf7 d1/d2b/d67/d5f/da9/c103 0 2024-03-24T10:46:23.140 INFO:tasks.workunit.client.1.smithi096.stdout:4/790: chown d1/d2b/d67/d5f/fd7 1 1 2024-03-24T10:46:23.140 INFO:tasks.workunit.client.1.smithi096.stdout:4/791: dread - d1/d32/d4c/d7e/fc6 zero size 2024-03-24T10:46:23.140 INFO:tasks.workunit.client.1.smithi096.stdout:9/967: dwrite d9/d7c/f9c [0,4194304] 0 2024-03-24T10:46:23.140 INFO:tasks.workunit.client.1.smithi096.stdout:5/903: dread d2/d29/d32/fe5 [0,4194304] 0 2024-03-24T10:46:23.141 INFO:tasks.workunit.client.1.smithi096.stdout:9/968: write d9/d65/d126/d29/d73/d43/fcb [1329742,108721] 0 2024-03-24T10:46:23.141 INFO:tasks.workunit.client.1.smithi096.stdout:9/969: dread - d9/d65/d126/d29/d73/f101 zero size 2024-03-24T10:46:23.141 INFO:tasks.workunit.client.1.smithi096.stdout:9/970: read - d9/d15/f106 zero size 2024-03-24T10:46:23.141 INFO:tasks.workunit.client.1.smithi096.stdout:9/971: truncate d9/db/d6d/d4f/d75/dcc/fe6 683707 0 2024-03-24T10:46:23.142 INFO:tasks.workunit.client.1.smithi096.stdout:5/904: creat d2/d29/d32/d33/d3c/d3d/f134 x:0 0 0 2024-03-24T10:46:23.149 INFO:tasks.workunit.client.1.smithi096.stdout:5/905: truncate d2/d29/d32/d33/f44 5154234 0 2024-03-24T10:46:23.149 INFO:tasks.workunit.client.1.smithi096.stdout:5/906: stat d2/d7/d8/df/d14/c24 0 2024-03-24T10:46:23.149 INFO:tasks.workunit.client.1.smithi096.stdout:5/907: chown d2/d7/d8/d18/l28 217078640 1 2024-03-24T10:46:23.149 INFO:tasks.workunit.client.1.smithi096.stdout:5/908: chown d2/d7/d8/d18/f93 10481 1 2024-03-24T10:46:23.149 INFO:tasks.workunit.client.1.smithi096.stdout:5/909: write d2/d29/d32/d33/d3c/d3d/ffe [698498,18411] 0 2024-03-24T10:46:23.151 INFO:tasks.workunit.client.1.smithi096.stdout:9/972: rmdir d9/db/d6d 39 2024-03-24T10:46:23.164 INFO:tasks.workunit.client.1.smithi096.stdout:5/910: mkdir d2/d29/d32/d33/d135 0 2024-03-24T10:46:23.166 INFO:tasks.workunit.client.1.smithi096.stdout:9/973: dwrite d9/db/d6d/d4f/f63 [0,4194304] 0 2024-03-24T10:46:23.166 INFO:tasks.workunit.client.1.smithi096.stdout:5/911: rmdir d2/d7/d8/d18/db6/ddd 39 2024-03-24T10:46:23.168 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:22 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:46:23.168 INFO:tasks.workunit.client.1.smithi096.stdout:9/974: mkdir d9/d65/d126/d29/d73/d43/d48/d51/d139 0 2024-03-24T10:46:23.170 INFO:tasks.workunit.client.1.smithi096.stdout:5/912: mknod d2/d49/d128/c136 0 2024-03-24T10:46:23.173 INFO:tasks.workunit.client.1.smithi096.stdout:4/792: truncate d1/d2b/d67/d5f/da9/db5/df3/fa7 640126 0 2024-03-24T10:46:23.173 INFO:tasks.workunit.client.1.smithi096.stdout:4/793: chown d1/d2b/d6b/d80/d9c/dcb/fcc 125429 1 2024-03-24T10:46:23.182 INFO:tasks.workunit.client.1.smithi096.stdout:9/975: creat d9/db/d6d/d4f/d75/d8b/d8e/dfb/f13a x:0 0 0 2024-03-24T10:46:23.182 INFO:tasks.workunit.client.1.smithi096.stdout:9/976: dread - d9/d102/f137 zero size 2024-03-24T10:46:23.184 INFO:tasks.workunit.client.1.smithi096.stdout:5/913: truncate d2/d7/d8/d18/db6/fd1 973273 0 2024-03-24T10:46:23.184 INFO:tasks.workunit.client.1.smithi096.stdout:5/914: fdatasync d2/d29/d32/d33/d3c/dd2/fd3 0 2024-03-24T10:46:23.188 INFO:tasks.workunit.client.1.smithi096.stdout:9/977: creat d9/d65/d126/d29/d73/d43/d48/dc0/dd8/f13b x:0 0 0 2024-03-24T10:46:23.188 INFO:tasks.workunit.client.1.smithi096.stdout:9/978: chown d9/d79/lf3 0 1 2024-03-24T10:46:23.191 INFO:tasks.workunit.client.1.smithi096.stdout:4/794: dwrite d1/d2b/d59/d74/f57 [4194304,4194304] 0 2024-03-24T10:46:23.191 INFO:tasks.workunit.client.1.smithi096.stdout:9/979: creat d9/db/d6d/f13c x:0 0 0 2024-03-24T10:46:23.193 INFO:tasks.workunit.client.1.smithi096.stdout:5/915: getdents d2/d49 0 2024-03-24T10:46:23.195 INFO:tasks.workunit.client.1.smithi096.stdout:4/795: dread d1/f1f [0,4194304] 0 2024-03-24T10:46:23.195 INFO:tasks.workunit.client.1.smithi096.stdout:1/832: sync 2024-03-24T10:46:23.196 INFO:tasks.workunit.client.1.smithi096.stdout:6/874: sync 2024-03-24T10:46:23.196 INFO:tasks.workunit.client.1.smithi096.stdout:8/953: sync 2024-03-24T10:46:23.196 INFO:tasks.workunit.client.1.smithi096.stdout:1/833: truncate d3/db2/fc0 5051286 0 2024-03-24T10:46:23.196 INFO:tasks.workunit.client.1.smithi096.stdout:1/834: fsync d3/d23/d8b/d99/fa6 0 2024-03-24T10:46:23.196 INFO:tasks.workunit.client.1.smithi096.stdout:1/835: dread - d3/d23/d2f/d65/d7c/ddf/fff zero size 2024-03-24T10:46:23.198 INFO:tasks.workunit.client.1.smithi096.stdout:4/796: dread d1/d11/f29 [0,4194304] 0 2024-03-24T10:46:23.199 INFO:tasks.workunit.client.1.smithi096.stdout:4/797: dread - d1/d32/d4c/d7e/fcf zero size 2024-03-24T10:46:23.202 INFO:tasks.workunit.client.1.smithi096.stdout:9/980: rename d9/db/d6d/fc7 to d9/db/d6d/d4f/d75/d8b/f13d 0 2024-03-24T10:46:23.204 INFO:tasks.workunit.client.1.smithi096.stdout:8/954: mknod d8/d11/d22/d4c/d65/df0/c12c 0 2024-03-24T10:46:23.205 INFO:tasks.workunit.client.1.smithi096.stdout:8/955: chown d8/d11/ldc 64940 1 2024-03-24T10:46:23.205 INFO:tasks.workunit.client.1.smithi096.stdout:8/956: truncate d8/d11/d22/d4c/d65/d1a/da0/df6/f114 794729 0 2024-03-24T10:46:23.239 INFO:tasks.workunit.client.1.smithi096.stdout:3/876: sync 2024-03-24T10:46:23.239 INFO:tasks.workunit.client.1.smithi096.stdout:0/998: sync 2024-03-24T10:46:23.241 INFO:tasks.workunit.client.1.smithi096.stdout:8/957: creat d8/d11/d22/d4c/d67/f12d x:0 0 0 2024-03-24T10:46:23.241 INFO:tasks.workunit.client.1.smithi096.stdout:8/958: readlink d8/d11/d22/d4c/d65/d1a/d46/d117/l107 0 2024-03-24T10:46:23.244 INFO:tasks.workunit.client.1.smithi096.stdout:9/981: link d9/d65/d126/d29/d73/d49/db7/dca/de3/l127 d9/d65/d126/d29/d73/d49/l13e 0 2024-03-24T10:46:23.246 INFO:tasks.workunit.client.1.smithi096.stdout:8/959: getdents d8/d11/d22/d4c/d65/d1a 0 2024-03-24T10:46:23.246 INFO:tasks.workunit.client.1.smithi096.stdout:5/916: dwrite d2/d29/d32/d3f/f55 [4194304,4194304] 0 2024-03-24T10:46:23.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:22 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:46:23.249 INFO:tasks.workunit.client.1.smithi096.stdout:0/999: dread d0/d11/d20/d2b/ddf/f107 [0,4194304] 0 2024-03-24T10:46:23.250 INFO:tasks.workunit.client.1.smithi096.stdout:8/960: unlink d8/d11/d22/d4c/d65/d1a/d46/d2e/l3f 0 2024-03-24T10:46:23.250 INFO:tasks.workunit.client.1.smithi096.stdout:8/961: write d8/d72/d96/fba [83174,72386] 0 2024-03-24T10:46:23.253 INFO:tasks.workunit.client.1.smithi096.stdout:5/917: mkdir d2/d137 0 2024-03-24T10:46:23.254 INFO:tasks.workunit.client.1.smithi096.stdout:1/836: dwrite d3/d23/d71/fb8 [0,4194304] 0 2024-03-24T10:46:23.254 INFO:tasks.workunit.client.1.smithi096.stdout:6/875: dwrite d8/d12/d32/d38/d59/f5a [0,4194304] 0 2024-03-24T10:46:23.254 INFO:tasks.workunit.client.1.smithi096.stdout:6/876: fsync d8/d12/d28/d36/d65/d71/d10d/fba 0 2024-03-24T10:46:23.255 INFO:tasks.workunit.client.1.smithi096.stdout:5/918: creat d2/dd7/f138 x:0 0 0 2024-03-24T10:46:23.261 INFO:tasks.workunit.client.1.smithi096.stdout:5/919: dread d2/d7/d8/d18/db6/fff [0,4194304] 0 2024-03-24T10:46:23.264 INFO:tasks.workunit.client.1.smithi096.stdout:9/982: dwrite d9/db/d6d/f35 [0,4194304] 0 2024-03-24T10:46:23.264 INFO:tasks.workunit.client.1.smithi096.stdout:1/837: link d3/d9b/fa4 d3/d23/d71/f11e 0 2024-03-24T10:46:23.267 INFO:tasks.workunit.client.1.smithi096.stdout:6/877: truncate d8/d12/d32/d38/d59/f98 233293 0 2024-03-24T10:46:23.268 INFO:tasks.workunit.client.1.smithi096.stdout:9/983: mknod d9/d65/d126/d29/d73/d43/d48/dc0/dd8/c13f 0 2024-03-24T10:46:23.269 INFO:tasks.workunit.client.1.smithi096.stdout:1/838: truncate d3/d23/d71/f8a 93197 0 2024-03-24T10:46:23.269 INFO:tasks.workunit.client.1.smithi096.stdout:1/839: creat d3/d84/d9a/dd2/f11f x:0 0 0 2024-03-24T10:46:23.270 INFO:tasks.workunit.client.1.smithi096.stdout:1/840: dread - d3/d23/d71/dbe/fdd zero size 2024-03-24T10:46:23.277 INFO:tasks.workunit.client.1.smithi096.stdout:9/984: write d9/d65/d126/d29/d73/d43/d48/fed [2230701,94890] 0 2024-03-24T10:46:23.277 INFO:tasks.workunit.client.1.smithi096.stdout:9/985: chown d9/db/d6d/d4f/d75/d8b/d8e/f95 68 1 2024-03-24T10:46:23.277 INFO:tasks.workunit.client.1.smithi096.stdout:9/986: write d9/d65/d126/d29/d73/d49/db7/fdf [919462,75285] 0 2024-03-24T10:46:23.277 INFO:tasks.workunit.client.1.smithi096.stdout:9/987: chown d9/d65/d126/d29/d73/d43/f6c 1 1 2024-03-24T10:46:23.278 INFO:tasks.workunit.client.1.smithi096.stdout:1/841: creat d3/d23/d71/d7b/dce/d103/f120 x:0 0 0 2024-03-24T10:46:23.279 INFO:tasks.workunit.client.1.smithi096.stdout:1/842: chown d3/d23/d8b/d5c/d9c/fa9 10 1 2024-03-24T10:46:23.282 INFO:tasks.workunit.client.1.smithi096.stdout:6/878: dread d8/d12/d32/d38/d59/f98 [0,4194304] 0 2024-03-24T10:46:23.282 INFO:tasks.workunit.client.1.smithi096.stdout:1/843: mknod d3/d23/d2f/d65/d7c/c121 0 2024-03-24T10:46:23.287 INFO:tasks.workunit.client.1.smithi096.stdout:5/920: dwrite d2/d7/f126 [0,4194304] 0 2024-03-24T10:46:23.288 INFO:tasks.workunit.client.1.smithi096.stdout:1/844: mkdir d3/d23/d8b/d5c/d6d/d122 0 2024-03-24T10:46:23.293 INFO:tasks.workunit.client.1.smithi096.stdout:6/879: mknod d8/d12/d28/c118 0 2024-03-24T10:46:23.294 INFO:tasks.workunit.client.1.smithi096.stdout:9/988: dwrite d9/d65/d126/d29/d73/f4b [0,4194304] 0 2024-03-24T10:46:23.300 INFO:tasks.workunit.client.1.smithi096.stdout:5/921: getdents d2/d7/d8/df/d8a 0 2024-03-24T10:46:23.301 INFO:tasks.workunit.client.1.smithi096.stdout:5/922: chown d2/d7/f3e 21958687 1 2024-03-24T10:46:23.303 INFO:tasks.workunit.client.1.smithi096.stdout:6/880: dread d8/d10/d58/f85 [0,4194304] 0 2024-03-24T10:46:23.303 INFO:tasks.workunit.client.1.smithi096.stdout:6/881: readlink d8/d12/d32/d38/ld9 0 2024-03-24T10:46:23.305 INFO:tasks.workunit.client.1.smithi096.stdout:9/989: dread d9/db/d6d/f35 [0,4194304] 0 2024-03-24T10:46:23.306 INFO:tasks.workunit.client.1.smithi096.stdout:1/845: dwrite d3/d23/d2f/fc4 [0,4194304] 0 2024-03-24T10:46:23.306 INFO:tasks.workunit.client.1.smithi096.stdout:6/882: symlink d8/d10/d13/d63/dbf/l119 0 2024-03-24T10:46:23.311 INFO:tasks.workunit.client.1.smithi096.stdout:9/990: rename d9/d65/d126/d29/d73/df4/f10f to d9/d65/d126/d29/f140 0 2024-03-24T10:46:23.317 INFO:tasks.workunit.client.1.smithi096.stdout:9/991: mknod d9/d79/c141 0 2024-03-24T10:46:23.317 INFO:tasks.workunit.client.1.smithi096.stdout:9/992: fsync d9/d65/d126/d29/f6a 0 2024-03-24T10:46:23.325 INFO:tasks.workunit.client.1.smithi096.stdout:6/883: dwrite d8/d12/d28/d36/d65/d6b/fce [0,4194304] 0 2024-03-24T10:46:23.333 INFO:tasks.workunit.client.1.smithi096.stdout:6/884: creat d8/d12/d28/d36/f11a x:0 0 0 2024-03-24T10:46:23.335 INFO:tasks.workunit.client.1.smithi096.stdout:1/846: dwrite d3/d23/d8b/d5c/f61 [4194304,4194304] 0 2024-03-24T10:46:23.339 INFO:tasks.workunit.client.1.smithi096.stdout:1/847: creat d3/d23/d2f/d65/f123 x:0 0 0 2024-03-24T10:46:23.341 INFO:tasks.workunit.client.1.smithi096.stdout:9/993: dwrite d9/d65/d126/d29/d73/f101 [0,4194304] 0 2024-03-24T10:46:23.341 INFO:tasks.workunit.client.1.smithi096.stdout:9/994: readlink d9/db/l55 0 2024-03-24T10:46:23.343 INFO:tasks.workunit.client.1.smithi096.stdout:1/848: mknod d3/d23/d8b/d79/c124 0 2024-03-24T10:46:23.343 INFO:tasks.workunit.client.1.smithi096.stdout:1/849: read d3/d23/d8b/d5c/fa5 [1601645,90152] 0 2024-03-24T10:46:23.343 INFO:tasks.workunit.client.1.smithi096.stdout:1/850: chown d3/d23/d8b/d5c/f61 222 1 2024-03-24T10:46:23.343 INFO:tasks.workunit.client.1.smithi096.stdout:1/851: write d3/d23/d71/f8a [607690,38632] 0 2024-03-24T10:46:23.343 INFO:tasks.workunit.client.1.smithi096.stdout:1/852: readlink d3/d23/d2f/d65/d7c/ddf/df2/l112 0 2024-03-24T10:46:23.343 INFO:tasks.workunit.client.1.smithi096.stdout:1/853: write d3/f66 [1700276,84393] 0 2024-03-24T10:46:23.343 INFO:tasks.workunit.client.1.smithi096.stdout:1/854: readlink d3/d23/d2f/d65/l107 0 2024-03-24T10:46:23.346 INFO:tasks.workunit.client.1.smithi096.stdout:6/885: read d8/f62 [3038149,25999] 0 2024-03-24T10:46:23.348 INFO:tasks.workunit.client.1.smithi096.stdout:9/995: write d9/d65/d126/d29/d73/d49/d86/f9e [545788,47545] 0 2024-03-24T10:46:23.349 INFO:tasks.workunit.client.1.smithi096.stdout:9/996: chown d9/d102 103933 1 2024-03-24T10:46:23.350 INFO:tasks.workunit.client.1.smithi096.stdout:6/886: chown d8/d10/d19/f60 0 1 2024-03-24T10:46:23.350 INFO:tasks.workunit.client.1.smithi096.stdout:1/855: dread d3/d36/f11b [0,4194304] 0 2024-03-24T10:46:23.354 INFO:tasks.workunit.client.1.smithi096.stdout:6/887: truncate d8/d12/d28/d36/d65/f92 9707 0 2024-03-24T10:46:23.354 INFO:tasks.workunit.client.1.smithi096.stdout:6/888: fsync d8/d12/d28/d36/d65/d6b/dec/ffa 0 2024-03-24T10:46:23.354 INFO:tasks.workunit.client.1.smithi096.stdout:6/889: write d8/d12/d32/f101 [4232563,128185] 0 2024-03-24T10:46:23.355 INFO:tasks.workunit.client.1.smithi096.stdout:1/856: mknod d3/d23/d2f/d65/c125 0 2024-03-24T10:46:23.355 INFO:tasks.workunit.client.1.smithi096.stdout:6/890: dread d8/d12/d32/d38/d59/db8/ff5 [0,4194304] 0 2024-03-24T10:46:23.359 INFO:tasks.workunit.client.1.smithi096.stdout:1/857: mkdir d3/d23/d71/d7b/d93/d126 0 2024-03-24T10:46:23.359 INFO:tasks.workunit.client.1.smithi096.stdout:1/858: write d3/d23/d8b/d6a/f77 [1301966,94188] 0 2024-03-24T10:46:23.364 INFO:tasks.workunit.client.1.smithi096.stdout:6/891: unlink d8/d12/d32/d68/fcb 0 2024-03-24T10:46:23.364 INFO:tasks.workunit.client.1.smithi096.stdout:6/892: chown d8/d10/d76 315 1 2024-03-24T10:46:23.366 INFO:tasks.workunit.client.1.smithi096.stdout:9/997: truncate d9/d15/fa6 583903 0 2024-03-24T10:46:23.366 INFO:tasks.workunit.client.1.smithi096.stdout:9/998: chown d9/d15/f3c 175447 1 2024-03-24T10:46:23.366 INFO:tasks.workunit.client.1.smithi096.stdout:9/999: readlink d9/db/d6d/l90 0 2024-03-24T10:46:23.368 INFO:tasks.workunit.client.1.smithi096.stdout:6/893: link d8/d7b/fca d8/d12/d61/d113/f11b 0 2024-03-24T10:46:23.371 INFO:tasks.workunit.client.1.smithi096.stdout:6/894: rename d8/d12/f5c to d8/d10/d13/dcc/dad/f11c 0 2024-03-24T10:46:23.376 INFO:tasks.workunit.client.1.smithi096.stdout:1/859: dwrite d3/d23/d8b/d5c/d9c/fcb [0,4194304] 0 2024-03-24T10:46:23.385 INFO:tasks.workunit.client.1.smithi096.stdout:1/860: write d3/d23/d2f/f3d [2853974,14501] 0 2024-03-24T10:46:23.388 INFO:tasks.workunit.client.1.smithi096.stdout:1/861: creat d3/d84/d9a/d116/f127 x:0 0 0 2024-03-24T10:46:23.388 INFO:tasks.workunit.client.1.smithi096.stdout:1/862: fsync d3/d23/d8b/f4c 0 2024-03-24T10:46:23.390 INFO:tasks.workunit.client.1.smithi096.stdout:1/863: rename d3/d23/d8b/d99/f10e to d3/d23/d2f/d65/d7c/ddf/df2/f128 0 2024-03-24T10:46:23.390 INFO:tasks.workunit.client.1.smithi096.stdout:1/864: write d3/d23/d71/ff8 [473428,44218] 0 2024-03-24T10:46:23.390 INFO:tasks.workunit.client.1.smithi096.stdout:1/865: fdatasync d3/d23/d8b/f3e 0 2024-03-24T10:46:23.391 INFO:tasks.workunit.client.1.smithi096.stdout:1/866: fsync d3/d23/d71/f8a 0 2024-03-24T10:46:23.395 INFO:tasks.workunit.client.1.smithi096.stdout:1/867: write d3/d23/fb7 [2694458,79453] 0 2024-03-24T10:46:23.395 INFO:tasks.workunit.client.1.smithi096.stdout:1/868: stat d3/d23/d8b/d5c/l64 0 2024-03-24T10:46:23.395 INFO:tasks.workunit.client.1.smithi096.stdout:1/869: chown d3/d23/d2f/f59 458893 1 2024-03-24T10:46:23.395 INFO:tasks.workunit.client.1.smithi096.stdout:1/870: mkdir d3/d23/d8b/d99/d129 0 2024-03-24T10:46:23.402 INFO:tasks.workunit.client.1.smithi096.stdout:1/871: dread d3/d73/fcf [0,4194304] 0 2024-03-24T10:46:23.404 INFO:tasks.workunit.client.1.smithi096.stdout:1/872: symlink d3/d23/d71/d85/l12a 0 2024-03-24T10:46:23.405 INFO:tasks.workunit.client.1.smithi096.stdout:1/873: mknod d3/d23/d8b/d8c/ddc/c12b 0 2024-03-24T10:46:23.405 INFO:tasks.workunit.client.1.smithi096.stdout:1/874: write d3/d23/d2f/f37 [2252008,78603] 0 2024-03-24T10:46:23.406 INFO:tasks.workunit.client.1.smithi096.stdout:1/875: symlink d3/d23/d71/d7b/d93/d126/l12c 0 2024-03-24T10:46:23.420 INFO:tasks.workunit.client.1.smithi096.stdout:1/876: dwrite d3/d23/d71/d7b/d93/fc5 [0,4194304] 0 2024-03-24T10:46:23.422 INFO:tasks.workunit.client.1.smithi096.stdout:1/877: creat d3/d23/d8b/d5c/d6d/d122/f12d x:0 0 0 2024-03-24T10:46:23.437 INFO:tasks.workunit.client.1.smithi096.stdout:1/878: dwrite d3/d23/f69 [0,4194304] 0 2024-03-24T10:46:23.440 INFO:tasks.workunit.client.1.smithi096.stdout:1/879: mkdir d3/d84/d9a/dd2/d12e 0 2024-03-24T10:46:23.452 INFO:tasks.workunit.client.1.smithi096.stdout:1/880: dwrite d3/d23/d8b/d6a/f70 [4194304,4194304] 0 2024-03-24T10:46:23.468 INFO:tasks.workunit.client.1.smithi096.stdout:1/881: dwrite d3/d23/d8b/d5c/d6d/ffd [0,4194304] 0 2024-03-24T10:46:23.468 INFO:tasks.workunit.client.1.smithi096.stdout:1/882: creat d3/d23/d71/d85/f12f x:0 0 0 2024-03-24T10:46:23.476 INFO:tasks.workunit.client.1.smithi096.stdout:1/883: write d3/d84/fe3 [1732836,97642] 0 2024-03-24T10:46:23.478 INFO:tasks.workunit.client.1.smithi096.stdout:1/884: rename d3/d23/d8b/d5c/d6d/fac to d3/d23/d8b/d99/d129/f130 0 2024-03-24T10:46:23.485 INFO:tasks.workunit.client.1.smithi096.stdout:1/885: dread d3/d9b/fa4 [0,4194304] 0 2024-03-24T10:46:23.500 INFO:tasks.workunit.client.1.smithi096.stdout:1/886: dwrite d3/d73/fcf [0,4194304] 0 2024-03-24T10:46:23.500 INFO:tasks.workunit.client.1.smithi096.stdout:1/887: chown d3/d73/f7a 21278270 1 2024-03-24T10:46:23.500 INFO:tasks.workunit.client.1.smithi096.stdout:1/888: write d3/d23/d8b/d8c/ddc/ffa [328081,67652] 0 2024-03-24T10:46:23.501 INFO:tasks.workunit.client.1.smithi096.stdout:1/889: unlink d3/d23/d8b/d8c/l9e 0 2024-03-24T10:46:23.502 INFO:tasks.workunit.client.1.smithi096.stdout:1/890: rmdir d3/d23/d71 39 2024-03-24T10:46:23.503 INFO:tasks.workunit.client.1.smithi096.stdout:1/891: readlink d3/dbc/le7 0 2024-03-24T10:46:23.503 INFO:tasks.workunit.client.1.smithi096.stdout:1/892: write d3/d23/d71/dbe/fc9 [4998566,747] 0 2024-03-24T10:46:23.506 INFO:tasks.workunit.client.1.smithi096.stdout:1/893: link d3/d73/f9f d3/f131 0 2024-03-24T10:46:23.509 INFO:tasks.workunit.client.1.smithi096.stdout:1/894: unlink d3/d23/d2f/f5a 0 2024-03-24T10:46:23.509 INFO:tasks.workunit.client.1.smithi096.stdout:1/895: chown d3/d23/d8b/d99 15987378 1 2024-03-24T10:46:23.510 INFO:tasks.workunit.client.1.smithi096.stdout:1/896: mknod d3/d23/d2f/c132 0 2024-03-24T10:46:23.512 INFO:tasks.workunit.client.1.smithi096.stdout:1/897: creat d3/d23/d71/d7b/dce/dfc/f133 x:0 0 0 2024-03-24T10:46:23.512 INFO:tasks.workunit.client.1.smithi096.stdout:1/898: write d3/f34 [1554039,57558] 0 2024-03-24T10:46:23.529 INFO:tasks.workunit.client.1.smithi096.stdout:1/899: dwrite d3/d23/d8b/d5c/d6d/ffd [0,4194304] 0 2024-03-24T10:46:23.530 INFO:tasks.workunit.client.1.smithi096.stdout:1/900: unlink d3/d23/d8b/d8c/fcc 0 2024-03-24T10:46:23.530 INFO:tasks.workunit.client.1.smithi096.stdout:1/901: truncate d3/d23/d8b/f4c 1660809 0 2024-03-24T10:46:23.534 INFO:tasks.workunit.client.1.smithi096.stdout:1/902: rename d3/d73/ded/f119 to d3/d36/d10d/f134 0 2024-03-24T10:46:23.534 INFO:tasks.workunit.client.1.smithi096.stdout:1/903: write d3/d23/d71/d85/ff9 [4978747,78838] 0 2024-03-24T10:46:23.534 INFO:tasks.workunit.client.1.smithi096.stdout:1/904: fsync d3/d23/d8b/d5c/f61 0 2024-03-24T10:46:23.539 INFO:tasks.workunit.client.1.smithi096.stdout:1/905: rename d3/d23/f56 to d3/d23/d2f/f135 0 2024-03-24T10:46:23.566 INFO:tasks.workunit.client.1.smithi096.stdout:1/906: mkdir d3/d23/d8b/d79/d136 0 2024-03-24T10:46:23.575 INFO:tasks.workunit.client.1.smithi096.stdout:1/907: dread d3/d36/fc2 [0,4194304] 0 2024-03-24T10:46:23.575 INFO:tasks.workunit.client.1.smithi096.stdout:1/908: chown d3/l51 34 1 2024-03-24T10:46:23.592 INFO:tasks.workunit.client.1.smithi096.stdout:1/909: dwrite d3/d23/d8b/d6a/f96 [0,4194304] 0 2024-03-24T10:46:23.605 INFO:tasks.workunit.client.1.smithi096.stdout:1/910: dwrite d3/d23/d8b/d5c/d9c/fec [0,4194304] 0 2024-03-24T10:46:23.611 INFO:tasks.workunit.client.1.smithi096.stdout:1/911: dread d3/d23/d8b/d99/fa6 [0,4194304] 0 2024-03-24T10:46:23.613 INFO:tasks.workunit.client.1.smithi096.stdout:5/923: sync 2024-03-24T10:46:23.613 INFO:tasks.workunit.client.1.smithi096.stdout:6/895: sync 2024-03-24T10:46:23.613 INFO:tasks.workunit.client.1.smithi096.stdout:4/798: sync 2024-03-24T10:46:23.613 INFO:tasks.workunit.client.1.smithi096.stdout:3/877: sync 2024-03-24T10:46:23.613 INFO:tasks.workunit.client.1.smithi096.stdout:8/962: sync 2024-03-24T10:46:23.613 INFO:tasks.workunit.client.1.smithi096.stdout:6/896: creat d8/d12/d32/f11d x:0 0 0 2024-03-24T10:46:23.614 INFO:tasks.workunit.client.1.smithi096.stdout:1/912: rmdir d3/d23/d71/d7b/dce/d103 39 2024-03-24T10:46:23.614 INFO:tasks.workunit.client.1.smithi096.stdout:1/913: chown d3/d23/d8b/d99/fa6 50083 1 2024-03-24T10:46:23.617 INFO:tasks.workunit.client.1.smithi096.stdout:5/924: rename d2/d29/d32/d3f/d5f/l67 to d2/d29/d32/d33/d116/d11f/l139 0 2024-03-24T10:46:23.622 INFO:tasks.workunit.client.1.smithi096.stdout:4/799: mkdir d1/d32/d4c/d7d/dc2/d104 0 2024-03-24T10:46:23.622 INFO:tasks.workunit.client.1.smithi096.stdout:4/800: fdatasync d1/d32/d42/dbe/ffb 0 2024-03-24T10:46:23.623 INFO:tasks.workunit.client.1.smithi096.stdout:4/801: chown d1/d32/d4c/d7d/dc2/cf5 16147027 1 2024-03-24T10:46:23.626 INFO:tasks.workunit.client.1.smithi096.stdout:3/878: truncate d0/d13/d45/d47/f8e 417074 0 2024-03-24T10:46:23.627 INFO:tasks.workunit.client.1.smithi096.stdout:6/897: getdents d8/d12/d28/d36/d65/d6b/da6 0 2024-03-24T10:46:23.628 INFO:tasks.workunit.client.1.smithi096.stdout:1/914: dwrite d3/d23/d8b/d5c/d9c/fda [0,4194304] 0 2024-03-24T10:46:23.628 INFO:tasks.workunit.client.1.smithi096.stdout:1/915: fsync d3/d23/d8b/d79/fd1 0 2024-03-24T10:46:23.628 INFO:tasks.workunit.client.1.smithi096.stdout:1/916: fsync d3/d23/d8b/d6a/fad 0 2024-03-24T10:46:23.629 INFO:tasks.workunit.client.1.smithi096.stdout:1/917: readlink d3/d23/d2f/d65/d7c/l90 0 2024-03-24T10:46:23.629 INFO:tasks.workunit.client.1.smithi096.stdout:5/925: mknod d2/d29/d32/d33/d8d/d94/dad/c13a 0 2024-03-24T10:46:23.636 INFO:tasks.workunit.client.1.smithi096.stdout:4/802: dread d1/d2b/d6b/d80/fb6 [4194304,4194304] 0 2024-03-24T10:46:23.637 INFO:tasks.workunit.client.1.smithi096.stdout:3/879: dread d0/d9/d22/fc2 [4194304,4194304] 0 2024-03-24T10:46:23.642 INFO:tasks.workunit.client.1.smithi096.stdout:1/918: truncate d3/d36/fc2 1308297 0 2024-03-24T10:46:23.643 INFO:tasks.workunit.client.1.smithi096.stdout:8/963: dwrite d8/d11/d44/f120 [0,4194304] 0 2024-03-24T10:46:23.645 INFO:tasks.workunit.client.1.smithi096.stdout:5/926: mkdir d2/d29/d32/d3f/d13b 0 2024-03-24T10:46:23.647 INFO:tasks.workunit.client.1.smithi096.stdout:6/898: mkdir d8/d10/d13/dcc/dad/d11e 0 2024-03-24T10:46:23.649 INFO:tasks.workunit.client.1.smithi096.stdout:4/803: creat d1/d2b/d67/d5f/da9/db5/df3/f105 x:0 0 0 2024-03-24T10:46:23.651 INFO:tasks.workunit.client.1.smithi096.stdout:3/880: mkdir d0/d13/d41/ddf/d11b/d11f 0 2024-03-24T10:46:23.651 INFO:tasks.workunit.client.1.smithi096.stdout:3/881: chown d0/d13/d45/d7d/d39/d48/d8d/f11d 29822285 1 2024-03-24T10:46:23.657 INFO:tasks.workunit.client.1.smithi096.stdout:4/804: read d1/d32/d42/fb0 [1213532,64618] 0 2024-03-24T10:46:23.657 INFO:tasks.workunit.client.1.smithi096.stdout:4/805: readlink d1/l4b 0 2024-03-24T10:46:23.658 INFO:tasks.workunit.client.1.smithi096.stdout:8/964: mknod d8/d11/d22/d4c/d65/d1a/d46/c12e 0 2024-03-24T10:46:23.658 INFO:tasks.workunit.client.1.smithi096.stdout:8/965: fdatasync d8/d11/d22/d4c/d65/d1a/f80 0 2024-03-24T10:46:23.660 INFO:tasks.workunit.client.1.smithi096.stdout:5/927: creat d2/d29/d32/d33/d8d/df3/f13c x:0 0 0 2024-03-24T10:46:23.661 INFO:tasks.workunit.client.1.smithi096.stdout:6/899: creat d8/d12/d28/f11f x:0 0 0 2024-03-24T10:46:23.664 INFO:tasks.workunit.client.1.smithi096.stdout:3/882: mkdir d0/d9/d120 0 2024-03-24T10:46:23.664 INFO:tasks.workunit.client.1.smithi096.stdout:3/883: chown d0/d92/ddc/dfc/de5/lfe 0 1 2024-03-24T10:46:23.664 INFO:tasks.workunit.client.1.smithi096.stdout:3/884: chown d0/d92/ddc/dfc 8539838 1 2024-03-24T10:46:23.664 INFO:tasks.workunit.client.1.smithi096.stdout:3/885: readlink d0/d13/d45/d7d/d39/d48/dbd/lda 0 2024-03-24T10:46:23.665 INFO:tasks.workunit.client.1.smithi096.stdout:4/806: dread d1/d32/d4c/d7e/f99 [0,4194304] 0 2024-03-24T10:46:23.666 INFO:tasks.workunit.client.1.smithi096.stdout:1/919: rename d3/d84/c97 to d3/d23/d71/d7b/dce/de1/d114/c137 0 2024-03-24T10:46:23.669 INFO:tasks.workunit.client.1.smithi096.stdout:8/966: rmdir d8/d11/d22/d4c/d65/d37 39 2024-03-24T10:46:23.669 INFO:tasks.workunit.client.1.smithi096.stdout:6/900: read d8/d12/d32/d38/f82 [2281418,14277] 0 2024-03-24T10:46:23.670 INFO:tasks.workunit.client.1.smithi096.stdout:5/928: creat d2/d7/d69/de4/deb/f13d x:0 0 0 2024-03-24T10:46:23.673 INFO:tasks.workunit.client.1.smithi096.stdout:4/807: mkdir d1/d32/d4c/d7d/dc2/d106 0 2024-03-24T10:46:23.674 INFO:tasks.workunit.client.1.smithi096.stdout:8/967: mkdir d8/d11/d22/d4c/d65/d52/dc0/dd9/d12f 0 2024-03-24T10:46:23.675 INFO:tasks.workunit.client.1.smithi096.stdout:1/920: write d3/d23/d2f/d65/d7c/ddf/ff5 [2415858,74143] 0 2024-03-24T10:46:23.675 INFO:tasks.workunit.client.1.smithi096.stdout:5/929: truncate d2/d29/d32/d33/f83 440585 0 2024-03-24T10:46:23.676 INFO:tasks.workunit.client.1.smithi096.stdout:5/930: fdatasync d2/d7/d8/df/f2d 0 2024-03-24T10:46:23.678 INFO:tasks.workunit.client.1.smithi096.stdout:1/921: getdents d3/d23/d2f/d65/dd9 0 2024-03-24T10:46:23.678 INFO:tasks.workunit.client.1.smithi096.stdout:1/922: write d3/d84/d9a/d116/f127 [813707,78270] 0 2024-03-24T10:46:23.685 INFO:tasks.workunit.client.1.smithi096.stdout:1/923: mkdir d3/d23/d71/d7b/d138 0 2024-03-24T10:46:23.687 INFO:tasks.workunit.client.1.smithi096.stdout:3/886: dwrite d0/d9/f36 [0,4194304] 0 2024-03-24T10:46:23.687 INFO:tasks.workunit.client.1.smithi096.stdout:4/808: dread d1/f1c [0,4194304] 0 2024-03-24T10:46:23.687 INFO:tasks.workunit.client.1.smithi096.stdout:1/924: mkdir d3/d23/d71/d7b/dce/d103/d139 0 2024-03-24T10:46:23.688 INFO:tasks.workunit.client.1.smithi096.stdout:1/925: creat d3/d23/d8b/d5c/d6d/d122/f13a x:0 0 0 2024-03-24T10:46:23.688 INFO:tasks.workunit.client.1.smithi096.stdout:1/926: truncate d3/d73/fcf 4997848 0 2024-03-24T10:46:23.688 INFO:tasks.workunit.client.1.smithi096.stdout:1/927: write d3/d23/d8b/d6a/fd6 [3350110,34757] 0 2024-03-24T10:46:23.688 INFO:tasks.workunit.client.1.smithi096.stdout:1/928: readlink d3/d9b/l111 0 2024-03-24T10:46:23.690 INFO:tasks.workunit.client.1.smithi096.stdout:3/887: mknod d0/d13/d45/d7d/d39/c121 0 2024-03-24T10:46:23.692 INFO:tasks.workunit.client.1.smithi096.stdout:4/809: symlink d1/d2b/l107 0 2024-03-24T10:46:23.696 INFO:tasks.workunit.client.1.smithi096.stdout:4/810: mkdir d1/d2b/d67/d5f/dfd/d108 0 2024-03-24T10:46:23.705 INFO:tasks.workunit.client.1.smithi096.stdout:4/811: dread d1/d2b/d59/d74/f57 [4194304,4194304] 0 2024-03-24T10:46:23.705 INFO:tasks.workunit.client.1.smithi096.stdout:4/812: write d1/d2b/d59/d74/ff6 [720963,23550] 0 2024-03-24T10:46:23.711 INFO:tasks.workunit.client.1.smithi096.stdout:4/813: link d1/d2b/d67/d5f/da9/db5/df3/f70 d1/d32/d4c/d7e/f109 0 2024-03-24T10:46:23.711 INFO:tasks.workunit.client.1.smithi096.stdout:4/814: write d1/d2b/d67/d5f/da9/db5/df3/dbd/fd6 [609129,44569] 0 2024-03-24T10:46:23.716 INFO:tasks.workunit.client.1.smithi096.stdout:8/968: dwrite d8/d72/d96/ff5 [0,4194304] 0 2024-03-24T10:46:23.717 INFO:tasks.workunit.client.1.smithi096.stdout:5/931: dwrite d2/d7/d8/df/d14/f23 [0,4194304] 0 2024-03-24T10:46:23.717 INFO:tasks.workunit.client.1.smithi096.stdout:5/932: fsync d2/d7/d8/df/f2d 0 2024-03-24T10:46:23.717 INFO:tasks.workunit.client.1.smithi096.stdout:6/901: dwrite d8/d10/d58/fe8 [0,4194304] 0 2024-03-24T10:46:23.717 INFO:tasks.workunit.client.1.smithi096.stdout:3/888: dwrite d0/d13/d45/d47/f8e [0,4194304] 0 2024-03-24T10:46:23.720 INFO:tasks.workunit.client.1.smithi096.stdout:8/969: symlink d8/d11/d22/d4c/d65/d1a/d46/d117/d11f/l130 0 2024-03-24T10:46:23.722 INFO:tasks.workunit.client.1.smithi096.stdout:5/933: rmdir d2/d7/d8/df/d14/dea 39 2024-03-24T10:46:23.722 INFO:tasks.workunit.client.1.smithi096.stdout:4/815: dread d1/f8 [0,4194304] 0 2024-03-24T10:46:23.726 INFO:tasks.workunit.client.1.smithi096.stdout:6/902: rmdir d8/d12/d28/d36/d65/d71/d10d/da8/dbe 39 2024-03-24T10:46:23.734 INFO:tasks.workunit.client.1.smithi096.stdout:6/903: mkdir d8/d12/d32/d38/d120 0 2024-03-24T10:46:23.737 INFO:tasks.workunit.client.1.smithi096.stdout:8/970: rename d8/d72 to d8/d11/d22/d4c/d65/df0/d131 0 2024-03-24T10:46:23.740 INFO:tasks.workunit.client.1.smithi096.stdout:6/904: mkdir d8/d12/d28/d36/d65/d6b/da6/d121 0 2024-03-24T10:46:23.740 INFO:tasks.workunit.client.1.smithi096.stdout:6/905: truncate d8/d10/d58/f85 4575307 0 2024-03-24T10:46:23.743 INFO:tasks.workunit.client.1.smithi096.stdout:3/889: dwrite d0/d13/d45/d7d/d39/d48/dbd/d106/f11e [0,4194304] 0 2024-03-24T10:46:23.746 INFO:tasks.workunit.client.1.smithi096.stdout:4/816: dwrite d1/d32/d42/f86 [0,4194304] 0 2024-03-24T10:46:23.750 INFO:tasks.workunit.client.1.smithi096.stdout:5/934: dwrite d2/d29/d32/d33/d3c/f106 [0,4194304] 0 2024-03-24T10:46:23.754 INFO:tasks.workunit.client.1.smithi096.stdout:3/890: mkdir d0/d13/d45/d7d/d122 0 2024-03-24T10:46:23.756 INFO:tasks.workunit.client.1.smithi096.stdout:8/971: dwrite d8/d11/d22/d4c/d65/d1d/f3b [0,4194304] 0 2024-03-24T10:46:23.766 INFO:tasks.workunit.client.1.smithi096.stdout:3/891: mkdir d0/d13/d1e/da4/d123 0 2024-03-24T10:46:23.766 INFO:tasks.workunit.client.1.smithi096.stdout:8/972: dread d8/d11/d22/d4c/d65/d1d/d3d/fce [0,4194304] 0 2024-03-24T10:46:23.767 INFO:tasks.workunit.client.1.smithi096.stdout:8/973: write d8/d11/d22/d4c/d65/df0/d131/d96/fee [687353,18813] 0 2024-03-24T10:46:23.767 INFO:tasks.workunit.client.1.smithi096.stdout:8/974: stat d8/d11/d44/d45/cc3 0 2024-03-24T10:46:23.770 INFO:tasks.workunit.client.1.smithi096.stdout:6/906: unlink d8/d10/d76/f95 0 2024-03-24T10:46:23.770 INFO:tasks.workunit.client.1.smithi096.stdout:6/907: chown d8/d12/d28/d36/d65/f92 3917 1 2024-03-24T10:46:23.771 INFO:tasks.workunit.client.1.smithi096.stdout:3/892: link d0/d13/d45/d7d/d39/d48/d8d/fc8 d0/d13/d45/d7d/d39/d48/dbd/daa/f124 0 2024-03-24T10:46:23.771 INFO:tasks.workunit.client.1.smithi096.stdout:3/893: readlink d0/d13/d45/d7d/l73 0 2024-03-24T10:46:23.774 INFO:tasks.workunit.client.1.smithi096.stdout:8/975: creat d8/d11/d22/d4c/db0/dcc/f132 x:0 0 0 2024-03-24T10:46:23.774 INFO:tasks.workunit.client.1.smithi096.stdout:5/935: dwrite d2/d29/d32/d33/d3c/dd2/fca [0,4194304] 0 2024-03-24T10:46:23.775 INFO:tasks.workunit.client.1.smithi096.stdout:5/936: dread - d2/d29/d32/d33/d8d/f11a zero size 2024-03-24T10:46:23.775 INFO:tasks.workunit.client.1.smithi096.stdout:6/908: mknod d8/d12/d28/d36/d65/d71/d10d/da8/c122 0 2024-03-24T10:46:23.775 INFO:tasks.workunit.client.1.smithi096.stdout:4/817: dwrite d1/d11/f29 [0,4194304] 0 2024-03-24T10:46:23.781 INFO:tasks.workunit.client.1.smithi096.stdout:3/894: symlink d0/d13/d45/d7d/d39/l125 0 2024-03-24T10:46:23.794 INFO:tasks.workunit.client.1.smithi096.stdout:5/937: rename d2/d29/d32/d33/d8d/d94/fd8 to d2/d29/d32/d33/d37/dab/f13e 0 2024-03-24T10:46:23.800 INFO:tasks.workunit.client.1.smithi096.stdout:6/909: symlink d8/d12/d32/d38/d59/db8/l123 0 2024-03-24T10:46:23.800 INFO:tasks.workunit.client.1.smithi096.stdout:4/818: mkdir d1/d2b/d10a 0 2024-03-24T10:46:23.803 INFO:tasks.workunit.client.1.smithi096.stdout:3/895: dwrite d0/d9/d22/f3d [4194304,4194304] 0 2024-03-24T10:46:23.803 INFO:tasks.workunit.client.1.smithi096.stdout:3/896: chown d0/d13/d45/d7d/d39/d48/d8d 245935741 1 2024-03-24T10:46:23.804 INFO:tasks.workunit.client.1.smithi096.stdout:8/976: rmdir d8/d11/d22/d4c/d65/d1a/d46/d2e 39 2024-03-24T10:46:23.804 INFO:tasks.workunit.client.1.smithi096.stdout:8/977: write d8/d11/f43 [628740,50267] 0 2024-03-24T10:46:23.806 INFO:tasks.workunit.client.1.smithi096.stdout:5/938: getdents d2/d7/d8/df/d14 0 2024-03-24T10:46:23.806 INFO:tasks.workunit.client.1.smithi096.stdout:5/939: chown d2/d29/d32/d33/d37/l70 224 1 2024-03-24T10:46:23.807 INFO:tasks.workunit.client.1.smithi096.stdout:5/940: dread - d2/d7/d69/de4/deb/f13d zero size 2024-03-24T10:46:23.812 INFO:tasks.workunit.client.1.smithi096.stdout:6/910: creat d8/d12/d28/d36/d65/d71/d10d/da8/dbe/d112/f124 x:0 0 0 2024-03-24T10:46:23.815 INFO:tasks.workunit.client.1.smithi096.stdout:4/819: write d1/d11/f41 [2791076,17547] 0 2024-03-24T10:46:23.817 INFO:tasks.workunit.client.1.smithi096.stdout:4/820: dread d1/d32/d4c/d7d/fab [0,4194304] 0 2024-03-24T10:46:23.817 INFO:tasks.workunit.client.1.smithi096.stdout:4/821: fsync d1/d2b/d59/d74/fb4 0 2024-03-24T10:46:23.818 INFO:tasks.workunit.client.1.smithi096.stdout:5/941: unlink d2/d7/f126 0 2024-03-24T10:46:23.822 INFO:tasks.workunit.client.1.smithi096.stdout:5/942: creat d2/d29/d32/d3f/d5f/dc0/f13f x:0 0 0 2024-03-24T10:46:23.825 INFO:tasks.workunit.client.1.smithi096.stdout:6/911: truncate d8/d10/d19/f48 6644827 0 2024-03-24T10:46:23.828 INFO:tasks.workunit.client.1.smithi096.stdout:6/912: mknod d8/d10/d19/dfd/c125 0 2024-03-24T10:46:23.828 INFO:tasks.workunit.client.1.smithi096.stdout:8/978: dwrite d8/d11/d22/d4c/d65/d52/dc0/dd9/d10a/f11d [0,4194304] 0 2024-03-24T10:46:23.832 INFO:tasks.workunit.client.1.smithi096.stdout:3/897: dwrite d0/d13/d45/d7d/d39/d48/d8d/f11d [0,4194304] 0 2024-03-24T10:46:23.833 INFO:tasks.workunit.client.1.smithi096.stdout:6/913: mknod d8/d10/d13/d63/dbf/c126 0 2024-03-24T10:46:23.833 INFO:tasks.workunit.client.1.smithi096.stdout:6/914: fsync d8/d7b/d10c/ff6 0 2024-03-24T10:46:23.833 INFO:tasks.workunit.client.1.smithi096.stdout:6/915: fsync d8/d12/d28/d36/d65/fc1 0 2024-03-24T10:46:23.846 INFO:tasks.workunit.client.1.smithi096.stdout:3/898: write d0/f1 [2258112,27380] 0 2024-03-24T10:46:23.846 INFO:tasks.workunit.client.1.smithi096.stdout:3/899: write d0/d13/d41/ddf/d11b/f5b [4040761,18651] 0 2024-03-24T10:46:23.847 INFO:tasks.workunit.client.1.smithi096.stdout:5/943: dwrite d2/f8b [0,4194304] 0 2024-03-24T10:46:23.850 INFO:tasks.workunit.client.1.smithi096.stdout:8/979: dwrite d8/d11/d22/d4c/d65/d1d/f5c [0,4194304] 0 2024-03-24T10:46:23.850 INFO:tasks.workunit.client.1.smithi096.stdout:8/980: stat d8/d11/d22/d4c/d65/d1a/da0/da6 0 2024-03-24T10:46:23.852 INFO:tasks.workunit.client.1.smithi096.stdout:6/916: mknod d8/d12/d32/d68/c127 0 2024-03-24T10:46:23.852 INFO:tasks.workunit.client.1.smithi096.stdout:6/917: truncate d8/d10/d19/dfd/f115 80055 0 2024-03-24T10:46:23.855 INFO:tasks.workunit.client.1.smithi096.stdout:3/900: rename d0/d92/ddc/dfc/fd5 to d0/d13/d45/d7d/d39/d48/dbd/d108/f126 0 2024-03-24T10:46:23.860 INFO:tasks.workunit.client.1.smithi096.stdout:5/944: mknod d2/d29/d32/d3f/d5f/c140 0 2024-03-24T10:46:23.862 INFO:tasks.workunit.client.1.smithi096.stdout:5/945: dread d2/d7/fd4 [0,4194304] 0 2024-03-24T10:46:23.862 INFO:tasks.workunit.client.1.smithi096.stdout:5/946: write d2/d7/f86 [4682848,47051] 0 2024-03-24T10:46:23.864 INFO:tasks.workunit.client.1.smithi096.stdout:8/981: symlink d8/d11/d22/d4c/d65/d52/dc0/dd9/d12f/l133 0 2024-03-24T10:46:23.868 INFO:tasks.workunit.client.1.smithi096.stdout:3/901: creat d0/d13/d45/d7d/f127 x:0 0 0 2024-03-24T10:46:23.873 INFO:tasks.workunit.client.1.smithi096.stdout:5/947: symlink d2/d7/d8/l141 0 2024-03-24T10:46:23.875 INFO:tasks.workunit.client.1.smithi096.stdout:3/902: symlink d0/d13/d45/d7d/d39/d48/l128 0 2024-03-24T10:46:23.880 INFO:tasks.workunit.client.1.smithi096.stdout:8/982: dwrite d8/d11/d22/d4c/faf [4194304,4194304] 0 2024-03-24T10:46:23.883 INFO:tasks.workunit.client.1.smithi096.stdout:8/983: creat d8/d11/d22/d4c/db0/f134 x:0 0 0 2024-03-24T10:46:23.883 INFO:tasks.workunit.client.1.smithi096.stdout:8/984: fdatasync d8/d11/d22/d4c/d67/f12d 0 2024-03-24T10:46:23.883 INFO:tasks.workunit.client.1.smithi096.stdout:8/985: write d8/d11/d22/d4c/d65/d1a/da0/f124 [517818,94389] 0 2024-03-24T10:46:23.883 INFO:tasks.workunit.client.1.smithi096.stdout:8/986: chown d8/d11/d22/d4c/d82 3 1 2024-03-24T10:46:23.887 INFO:tasks.workunit.client.1.smithi096.stdout:3/903: dread d0/d9/f36 [0,4194304] 0 2024-03-24T10:46:23.887 INFO:tasks.workunit.client.1.smithi096.stdout:3/904: write d0/d13/d41/ddf/d11b/f66 [4885465,96550] 0 2024-03-24T10:46:23.891 INFO:tasks.workunit.client.1.smithi096.stdout:8/987: creat d8/d11/d22/d4c/f135 x:0 0 0 2024-03-24T10:46:23.891 INFO:tasks.workunit.client.1.smithi096.stdout:8/988: readlink d8/l9 0 2024-03-24T10:46:23.892 INFO:tasks.workunit.client.1.smithi096.stdout:5/948: dwrite d2/d29/d32/d33/f63 [0,4194304] 0 2024-03-24T10:46:23.894 INFO:tasks.workunit.client.1.smithi096.stdout:3/905: unlink d0/d13/d41/c6c 0 2024-03-24T10:46:23.895 INFO:tasks.workunit.client.1.smithi096.stdout:8/989: rmdir d8/d11/d22/d4c/d65/d1a/d46/d2e/d128 0 2024-03-24T10:46:23.900 INFO:tasks.workunit.client.1.smithi096.stdout:3/906: truncate d0/d13/d45/d47/f64 3207211 0 2024-03-24T10:46:23.901 INFO:tasks.workunit.client.1.smithi096.stdout:8/990: symlink d8/d11/d22/d4c/d65/d1d/d3d/l136 0 2024-03-24T10:46:23.902 INFO:tasks.workunit.client.1.smithi096.stdout:3/907: rmdir d0/d9/d22/d102 39 2024-03-24T10:46:23.903 INFO:tasks.workunit.client.1.smithi096.stdout:8/991: creat d8/d11/d85/f137 x:0 0 0 2024-03-24T10:46:23.904 INFO:tasks.workunit.client.1.smithi096.stdout:3/908: dread d0/d13/d41/d7f/fab [0,4194304] 0 2024-03-24T10:46:23.905 INFO:tasks.workunit.client.1.smithi096.stdout:3/909: truncate d0/d13/d1e/da4/fd9 687262 0 2024-03-24T10:46:23.905 INFO:tasks.workunit.client.1.smithi096.stdout:8/992: symlink d8/d11/d22/ded/l138 0 2024-03-24T10:46:23.908 INFO:tasks.workunit.client.1.smithi096.stdout:3/910: creat d0/d92/ddc/f129 x:0 0 0 2024-03-24T10:46:23.916 INFO:tasks.workunit.client.1.smithi096.stdout:5/949: dwrite d2/d29/d32/d3f/d84/faf [0,4194304] 0 2024-03-24T10:46:23.921 INFO:tasks.workunit.client.1.smithi096.stdout:5/950: mkdir d2/d29/d142 0 2024-03-24T10:46:23.921 INFO:tasks.workunit.client.1.smithi096.stdout:5/951: readlink d2/d7/l2f 0 2024-03-24T10:46:23.922 INFO:tasks.workunit.client.1.smithi096.stdout:5/952: truncate d2/d7/d8/df/d14/f11b 2982661 0 2024-03-24T10:46:23.922 INFO:tasks.workunit.client.1.smithi096.stdout:5/953: creat d2/d7/d8/d18/db6/f143 x:0 0 0 2024-03-24T10:46:23.922 INFO:tasks.workunit.client.1.smithi096.stdout:5/954: chown d2/d29/d32/d33/d37/dab/d100 651036 1 2024-03-24T10:46:23.923 INFO:tasks.workunit.client.1.smithi096.stdout:3/911: dwrite d0/d92/ddc/dfc/de5/f10d [0,4194304] 0 2024-03-24T10:46:23.924 INFO:tasks.workunit.client.1.smithi096.stdout:5/955: mknod d2/d7/d8/d18/db6/c144 0 2024-03-24T10:46:23.925 INFO:tasks.workunit.client.1.smithi096.stdout:5/956: write d2/d29/d32/d33/f83 [320476,75200] 0 2024-03-24T10:46:23.925 INFO:tasks.workunit.client.1.smithi096.stdout:5/957: chown d2/d29/d32/d33/d3c/cd6 130607 1 2024-03-24T10:46:23.927 INFO:tasks.workunit.client.1.smithi096.stdout:3/912: mknod d0/d13/d41/ddf/c12a 0 2024-03-24T10:46:23.928 INFO:tasks.workunit.client.1.smithi096.stdout:5/958: mknod d2/d29/d32/d3f/d5f/c145 0 2024-03-24T10:46:23.928 INFO:tasks.workunit.client.1.smithi096.stdout:5/959: dread - d2/d29/d32/dbb/ff2 zero size 2024-03-24T10:46:23.929 INFO:tasks.workunit.client.1.smithi096.stdout:3/913: symlink d0/d13/d77/l12b 0 2024-03-24T10:46:23.930 INFO:tasks.workunit.client.1.smithi096.stdout:5/960: fdatasync d2/d29/d32/d33/f83 0 2024-03-24T10:46:23.930 INFO:tasks.workunit.client.1.smithi096.stdout:3/914: mkdir d0/d13/d12c 0 2024-03-24T10:46:23.930 INFO:tasks.workunit.client.1.smithi096.stdout:3/915: readlink d0/d9/lf9 0 2024-03-24T10:46:23.931 INFO:tasks.workunit.client.1.smithi096.stdout:5/961: link d2/d29/d32/dbb/ff2 d2/d29/d142/f146 0 2024-03-24T10:46:23.940 INFO:tasks.workunit.client.1.smithi096.stdout:5/962: dread d2/d29/d32/d33/d3c/f106 [0,4194304] 0 2024-03-24T10:46:23.945 INFO:tasks.workunit.client.1.smithi096.stdout:5/963: rename d2/d29/d32/d33/d3c/dc3 to d2/d29/d32/dbb/d147 0 2024-03-24T10:46:23.947 INFO:tasks.workunit.client.1.smithi096.stdout:5/964: mkdir d2/d29/d32/d33/d135/d148 0 2024-03-24T10:46:23.949 INFO:tasks.workunit.client.1.smithi096.stdout:5/965: mknod d2/d7/d69/c149 0 2024-03-24T10:46:23.949 INFO:tasks.workunit.client.1.smithi096.stdout:5/966: stat d2/d29/d32/d3f/d84/db8/ce0 0 2024-03-24T10:46:23.950 INFO:tasks.workunit.client.1.smithi096.stdout:5/967: creat d2/d29/d32/d33/f14a x:0 0 0 2024-03-24T10:46:23.952 INFO:tasks.workunit.client.1.smithi096.stdout:5/968: rename d2/d7/d8/d18/db6/ddd/c104 to d2/d7/d69/de4/deb/c14b 0 2024-03-24T10:46:23.954 INFO:tasks.workunit.client.1.smithi096.stdout:5/969: symlink d2/d29/d142/l14c 0 2024-03-24T10:46:23.954 INFO:tasks.workunit.client.1.smithi096.stdout:5/970: chown d2/d7/d8/f56 0 1 2024-03-24T10:46:23.956 INFO:tasks.workunit.client.1.smithi096.stdout:5/971: dread d2/d29/d32/d89/fa1 [0,4194304] 0 2024-03-24T10:46:23.957 INFO:tasks.workunit.client.1.smithi096.stdout:5/972: getdents d2/dd7 0 2024-03-24T10:46:23.958 INFO:tasks.workunit.client.1.smithi096.stdout:5/973: mknod d2/d29/d32/d3f/d13b/c14d 0 2024-03-24T10:46:23.969 INFO:tasks.workunit.client.1.smithi096.stdout:3/916: write d0/d9/f3b [389939,55962] 0 2024-03-24T10:46:23.970 INFO:tasks.workunit.client.1.smithi096.stdout:3/917: creat d0/d13/d45/d7d/d39/f12d x:0 0 0 2024-03-24T10:46:23.971 INFO:tasks.workunit.client.1.smithi096.stdout:5/974: dwrite d2/d29/d32/d33/d3c/d3d/ffe [0,4194304] 0 2024-03-24T10:46:23.971 INFO:tasks.workunit.client.1.smithi096.stdout:5/975: write d2/d29/d32/d33/d37/dab/f13e [1116694,8342] 0 2024-03-24T10:46:23.971 INFO:tasks.workunit.client.1.smithi096.stdout:5/976: fdatasync d2/d7/d8/f56 0 2024-03-24T10:46:23.972 INFO:tasks.workunit.client.1.smithi096.stdout:5/977: write d2/d29/d32/d33/d3c/dd2/f12a [984780,9673] 0 2024-03-24T10:46:23.975 INFO:tasks.workunit.client.1.smithi096.stdout:5/978: mknod d2/d7/d8/df/d14/dea/c14e 0 2024-03-24T10:46:23.978 INFO:tasks.workunit.client.1.smithi096.stdout:5/979: link d2/d29/d32/d33/d8d/df3/f13c d2/d29/d32/dbb/d147/f14f 0 2024-03-24T10:46:23.978 INFO:tasks.workunit.client.1.smithi096.stdout:5/980: fdatasync d2/d29/d32/d33/d37/dab/fb1 0 2024-03-24T10:46:23.979 INFO:tasks.workunit.client.1.smithi096.stdout:5/981: creat d2/d7/d8/df/d8a/d10a/f150 x:0 0 0 2024-03-24T10:46:23.981 INFO:tasks.workunit.client.1.smithi096.stdout:5/982: creat d2/d29/d32/d33/d135/d148/f151 x:0 0 0 2024-03-24T10:46:23.982 INFO:tasks.workunit.client.1.smithi096.stdout:5/983: creat d2/d137/f152 x:0 0 0 2024-03-24T10:46:23.984 INFO:tasks.workunit.client.1.smithi096.stdout:3/918: dwrite d0/d13/d41/ddf/d11b/f6d [0,4194304] 0 2024-03-24T10:46:23.987 INFO:tasks.workunit.client.1.smithi096.stdout:3/919: truncate d0/d13/d45/d7d/d39/d48/f6a 3911461 0 2024-03-24T10:46:24.007 INFO:tasks.workunit.client.1.smithi096.stdout:3/920: dwrite d0/d9/d22/d102/db6/fe6 [0,4194304] 0 2024-03-24T10:46:24.007 INFO:tasks.workunit.client.1.smithi096.stdout:5/984: dwrite d2/d7/d8/d18/db6/fff [0,4194304] 0 2024-03-24T10:46:24.013 INFO:tasks.workunit.client.1.smithi096.stdout:3/921: rename d0/f107 to d0/d9b/f12e 0 2024-03-24T10:46:24.014 INFO:tasks.workunit.client.1.smithi096.stdout:5/985: creat d2/d29/d32/d3f/f153 x:0 0 0 2024-03-24T10:46:24.015 INFO:tasks.workunit.client.1.smithi096.stdout:3/922: creat d0/d13/d77/d117/f12f x:0 0 0 2024-03-24T10:46:24.016 INFO:tasks.workunit.client.1.smithi096.stdout:5/986: creat d2/d29/d32/d3f/d84/db8/d132/f154 x:0 0 0 2024-03-24T10:46:24.021 INFO:tasks.workunit.client.1.smithi096.stdout:5/987: getdents d2/d29/d32/d3f/d5f/dc0 0 2024-03-24T10:46:24.023 INFO:tasks.workunit.client.1.smithi096.stdout:5/988: getdents d2/d29/d32/d33/d116 0 2024-03-24T10:46:24.023 INFO:tasks.workunit.client.1.smithi096.stdout:5/989: fdatasync d2/d29/d32/d89/fa1 0 2024-03-24T10:46:24.024 INFO:tasks.workunit.client.1.smithi096.stdout:5/990: rmdir d2 39 2024-03-24T10:46:24.024 INFO:tasks.workunit.client.1.smithi096.stdout:5/991: dread - d2/d29/d32/d33/d3c/d3d/d10f/f122 zero size 2024-03-24T10:46:24.024 INFO:tasks.workunit.client.1.smithi096.stdout:5/992: chown d2/d29/d32/d3f/d84/faf 1903418664 1 2024-03-24T10:46:24.024 INFO:tasks.workunit.client.1.smithi096.stdout:5/993: truncate d2/d29/d32/d33/d8d/d94/df9/f12d 1046805 0 2024-03-24T10:46:24.048 INFO:tasks.workunit.client.1.smithi096.stdout:5/994: dwrite d2/d7/d8/df/d8a/d10a/d10b/f129 [0,4194304] 0 2024-03-24T10:46:24.048 INFO:tasks.workunit.client.1.smithi096.stdout:5/995: creat d2/d29/d32/d33/f155 x:0 0 0 2024-03-24T10:46:24.062 INFO:tasks.workunit.client.1.smithi096.stdout:5/996: rmdir d2/d29/d32/d3f 39 2024-03-24T10:46:24.062 INFO:tasks.workunit.client.1.smithi096.stdout:5/997: dread - d2/d29/d32/d3f/d5f/dc0/f13f zero size 2024-03-24T10:46:24.064 INFO:tasks.workunit.client.1.smithi096.stdout:5/998: symlink d2/d7/d8/l156 0 2024-03-24T10:46:24.065 INFO:tasks.workunit.client.1.smithi096.stdout:5/999: dread d2/d7/d8/df/d8a/f12f [0,4194304] 0 2024-03-24T10:46:24.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:23 smithi094 ceph-mon[29446]: pgmap v255: 65 pgs: 65 active+clean; 2.8 GiB data, 9.8 GiB used, 527 GiB / 536 GiB avail; 150 MiB/s rd, 185 MiB/s wr, 382 op/s 2024-03-24T10:46:24.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:23 smithi096 ceph-mon[38097]: pgmap v255: 65 pgs: 65 active+clean; 2.8 GiB data, 9.8 GiB used, 527 GiB / 536 GiB avail; 150 MiB/s rd, 185 MiB/s wr, 382 op/s 2024-03-24T10:46:24.431 INFO:tasks.workunit.client.1.smithi096.stdout:1/929: sync 2024-03-24T10:46:24.431 INFO:tasks.workunit.client.1.smithi096.stdout:1/930: write d3/d84/d9a/dd2/f11f [266834,124093] 0 2024-03-24T10:46:24.449 INFO:tasks.workunit.client.1.smithi096.stdout:1/931: dwrite d3/fd8 [0,4194304] 0 2024-03-24T10:46:24.449 INFO:tasks.workunit.client.1.smithi096.stdout:1/932: fsync d3/d23/d2f/d65/d7c/ddf/f11c 0 2024-03-24T10:46:24.449 INFO:tasks.workunit.client.1.smithi096.stdout:1/933: fsync d3/d36/f4d 0 2024-03-24T10:46:24.450 INFO:tasks.workunit.client.1.smithi096.stdout:1/934: link d3/c1d d3/d23/d71/d7b/c13b 0 2024-03-24T10:46:24.474 INFO:tasks.workunit.client.1.smithi096.stdout:1/935: dwrite d3/d23/d71/d85/ff9 [4194304,4194304] 0 2024-03-24T10:46:24.491 INFO:tasks.workunit.client.1.smithi096.stdout:1/936: dwrite d3/d23/d2f/d65/f123 [0,4194304] 0 2024-03-24T10:46:24.495 INFO:tasks.workunit.client.1.smithi096.stdout:6/918: sync 2024-03-24T10:46:24.495 INFO:tasks.workunit.client.1.smithi096.stdout:4/822: sync 2024-03-24T10:46:24.495 INFO:tasks.workunit.client.1.smithi096.stdout:1/937: write d3/d36/f4d [256645,79148] 0 2024-03-24T10:46:24.495 INFO:tasks.workunit.client.1.smithi096.stdout:8/993: sync 2024-03-24T10:46:24.495 INFO:tasks.workunit.client.1.smithi096.stdout:3/923: sync 2024-03-24T10:46:24.498 INFO:tasks.workunit.client.1.smithi096.stdout:6/919: rmdir d8/d10/d13/d63/dbf 39 2024-03-24T10:46:24.499 INFO:tasks.workunit.client.1.smithi096.stdout:4/823: rmdir d1/d2b/d67 39 2024-03-24T10:46:24.500 INFO:tasks.workunit.client.1.smithi096.stdout:6/920: dread d8/d12/d28/d36/fae [0,4194304] 0 2024-03-24T10:46:24.501 INFO:tasks.workunit.client.1.smithi096.stdout:1/938: creat d3/d23/d71/d7b/d93/d126/f13c x:0 0 0 2024-03-24T10:46:24.501 INFO:tasks.workunit.client.1.smithi096.stdout:1/939: stat d3/d73/f9f 0 2024-03-24T10:46:24.501 INFO:tasks.workunit.client.1.smithi096.stdout:1/940: chown d3/c4 789286 1 2024-03-24T10:46:24.503 INFO:tasks.workunit.client.1.smithi096.stdout:8/994: link d8/la d8/d11/d22/d4c/d65/d118/l139 0 2024-03-24T10:46:24.503 INFO:tasks.workunit.client.1.smithi096.stdout:8/995: fsync d8/f15 0 2024-03-24T10:46:24.503 INFO:tasks.workunit.client.1.smithi096.stdout:8/996: creat d8/d11/d22/d4c/d65/df0/f13a x:0 0 0 2024-03-24T10:46:24.506 INFO:tasks.workunit.client.1.smithi096.stdout:4/824: rename d1/d2b/d59/d74/fcd to d1/d2b/d6b/d80/d9c/f10b 0 2024-03-24T10:46:24.506 INFO:tasks.workunit.client.1.smithi096.stdout:6/921: mkdir d8/d10/d13/d63/d128 0 2024-03-24T10:46:24.507 INFO:tasks.workunit.client.1.smithi096.stdout:1/941: mknod d3/d23/d8b/d99/c13d 0 2024-03-24T10:46:24.507 INFO:tasks.workunit.client.1.smithi096.stdout:1/942: readlink d3/d23/d71/laf 0 2024-03-24T10:46:24.508 INFO:tasks.workunit.client.1.smithi096.stdout:3/924: symlink d0/d13/d45/d47/d8f/l130 0 2024-03-24T10:46:24.513 INFO:tasks.workunit.client.1.smithi096.stdout:4/825: rename d1/d32/d4c/l7a to d1/d32/d4c/da6/l10c 0 2024-03-24T10:46:24.515 INFO:tasks.workunit.client.1.smithi096.stdout:6/922: mkdir d8/d12/d28/d36/d65/d6b/d129 0 2024-03-24T10:46:24.518 INFO:tasks.workunit.client.1.smithi096.stdout:3/925: link d0/d9/f112 d0/d13/d41/f131 0 2024-03-24T10:46:24.519 INFO:tasks.workunit.client.1.smithi096.stdout:8/997: getdents d8/d11/d22/d4c/db0/dcc 0 2024-03-24T10:46:24.521 INFO:tasks.workunit.client.1.smithi096.stdout:4/826: symlink d1/d2b/d67/d5f/da9/l10d 0 2024-03-24T10:46:24.523 INFO:tasks.workunit.client.1.smithi096.stdout:3/926: link d0/d13/d45/d47/f64 d0/d13/d45/d7d/d39/d48/dbd/d108/f132 0 2024-03-24T10:46:24.525 INFO:tasks.workunit.client.1.smithi096.stdout:8/998: mknod d8/d11/d22/d4c/d65/d1d/d3d/d5f/c13b 0 2024-03-24T10:46:24.526 INFO:tasks.workunit.client.1.smithi096.stdout:8/999: read - d8/d11/d22/d4c/d82/da1/dd4/de7/ffd zero size 2024-03-24T10:46:24.527 INFO:tasks.workunit.client.1.smithi096.stdout:4/827: symlink d1/d2b/d67/d5f/l10e 0 2024-03-24T10:46:24.530 INFO:tasks.workunit.client.1.smithi096.stdout:1/943: dwrite d3/d23/d8b/d99/fa6 [0,4194304] 0 2024-03-24T10:46:24.530 INFO:tasks.workunit.client.1.smithi096.stdout:1/944: dread - d3/d23/d2f/d65/d7c/ddf/f11c zero size 2024-03-24T10:46:24.530 INFO:tasks.workunit.client.1.smithi096.stdout:1/945: chown d3/le8 0 1 2024-03-24T10:46:24.530 INFO:tasks.workunit.client.1.smithi096.stdout:1/946: creat d3/d23/d8b/d79/f13e x:0 0 0 2024-03-24T10:46:24.530 INFO:tasks.workunit.client.1.smithi096.stdout:4/828: truncate d1/d2b/d67/d5f/da9/db5/df3/f60 1021917 0 2024-03-24T10:46:24.530 INFO:tasks.workunit.client.1.smithi096.stdout:4/829: read d1/d11/f66 [3534422,56251] 0 2024-03-24T10:46:24.530 INFO:tasks.workunit.client.1.smithi096.stdout:4/830: truncate d1/fd 771180 0 2024-03-24T10:46:24.531 INFO:tasks.workunit.client.1.smithi096.stdout:3/927: rmdir d0/d9b 39 2024-03-24T10:46:24.534 INFO:tasks.workunit.client.1.smithi096.stdout:6/923: dwrite d8/d10/d19/dfd/f115 [0,4194304] 0 2024-03-24T10:46:24.536 INFO:tasks.workunit.client.1.smithi096.stdout:4/831: rename d1/c77 to d1/d2b/d89/dbf/c10f 0 2024-03-24T10:46:24.538 INFO:tasks.workunit.client.1.smithi096.stdout:3/928: mknod d0/c133 0 2024-03-24T10:46:24.545 INFO:tasks.workunit.client.1.smithi096.stdout:6/924: creat d8/d12/d28/d36/d65/d6b/da6/d103/f12a x:0 0 0 2024-03-24T10:46:24.550 INFO:tasks.workunit.client.1.smithi096.stdout:4/832: rmdir d1/d2b/d10a 0 2024-03-24T10:46:24.552 INFO:tasks.workunit.client.1.smithi096.stdout:3/929: unlink d0/d13/d41/ddf/c12a 0 2024-03-24T10:46:24.553 INFO:tasks.workunit.client.1.smithi096.stdout:6/925: symlink d8/d10/d76/d91/dd5/de5/l12b 0 2024-03-24T10:46:24.555 INFO:tasks.workunit.client.1.smithi096.stdout:4/833: rename d1/d2b/d6b/f96 to d1/d2b/d67/d5f/da9/db5/df3/dbd/f110 0 2024-03-24T10:46:24.557 INFO:tasks.workunit.client.1.smithi096.stdout:3/930: rename d0/d92/lfb to d0/d13/d41/ddf/l134 0 2024-03-24T10:46:24.557 INFO:tasks.workunit.client.1.smithi096.stdout:6/926: dread d8/d12/d32/d38/d59/db8/fbb [0,4194304] 0 2024-03-24T10:46:24.559 INFO:tasks.workunit.client.1.smithi096.stdout:6/927: mknod d8/d12/d32/d38/c12c 0 2024-03-24T10:46:24.560 INFO:tasks.workunit.client.1.smithi096.stdout:6/928: truncate d8/d12/d28/d36/f7d 3893635 0 2024-03-24T10:46:24.560 INFO:tasks.workunit.client.1.smithi096.stdout:6/929: read d8/d12/d32/d38/d59/db8/ff5 [251919,130336] 0 2024-03-24T10:46:24.560 INFO:tasks.workunit.client.1.smithi096.stdout:6/930: mknod d8/d10/d13/d63/c12d 0 2024-03-24T10:46:24.561 INFO:tasks.workunit.client.1.smithi096.stdout:6/931: creat d8/d12/d28/d36/d65/d6b/da6/d121/f12e x:0 0 0 2024-03-24T10:46:24.562 INFO:tasks.workunit.client.1.smithi096.stdout:6/932: creat d8/d7b/d10c/f12f x:0 0 0 2024-03-24T10:46:24.562 INFO:tasks.workunit.client.1.smithi096.stdout:6/933: chown d8/d12/d32/d38/d59/db8/fb1 38531 1 2024-03-24T10:46:24.563 INFO:tasks.workunit.client.1.smithi096.stdout:6/934: getdents d8/d12/d28/d36 0 2024-03-24T10:46:24.563 INFO:tasks.workunit.client.1.smithi096.stdout:6/935: fsync d8/d12/d32/f11d 0 2024-03-24T10:46:24.565 INFO:tasks.workunit.client.1.smithi096.stdout:6/936: creat d8/d12/d28/d36/d65/d6b/d129/f130 x:0 0 0 2024-03-24T10:46:24.566 INFO:tasks.workunit.client.1.smithi096.stdout:6/937: rename d8/d12/d32/d38/c10f to d8/d12/d28/d36/d65/d6b/da6/db2/c131 0 2024-03-24T10:46:24.567 INFO:tasks.workunit.client.1.smithi096.stdout:6/938: unlink d8/d12/f1f 0 2024-03-24T10:46:24.568 INFO:tasks.workunit.client.1.smithi096.stdout:6/939: truncate d8/d10/d58/fe8 4069275 0 2024-03-24T10:46:24.574 INFO:tasks.workunit.client.1.smithi096.stdout:3/931: dwrite d0/d9/f49 [8388608,4194304] 0 2024-03-24T10:46:24.575 INFO:tasks.workunit.client.1.smithi096.stdout:3/932: creat d0/d9/d22/d102/f135 x:0 0 0 2024-03-24T10:46:24.576 INFO:tasks.workunit.client.1.smithi096.stdout:3/933: unlink d0/d9/d22/c40 0 2024-03-24T10:46:24.577 INFO:tasks.workunit.client.1.smithi096.stdout:3/934: mkdir d0/d9/d22/d136 0 2024-03-24T10:46:24.577 INFO:tasks.workunit.client.1.smithi096.stdout:3/935: rmdir d0/d13/d45/d7d/d39/d48/dbd 39 2024-03-24T10:46:24.677 INFO:tasks.workunit.client.1.smithi096.stdout:3/936: dwrite d0/d13/d77/f90 [0,4194304] 0 2024-03-24T10:46:24.677 INFO:tasks.workunit.client.1.smithi096.stdout:3/937: readlink d0/d13/d41/l62 0 2024-03-24T10:46:24.692 INFO:tasks.workunit.client.1.smithi096.stdout:6/940: rmdir d8/d12/d32 39 2024-03-24T10:46:24.694 INFO:tasks.workunit.client.1.smithi096.stdout:6/941: link d8/d10/d76/fa9 d8/d10/d76/dd6/f132 0 2024-03-24T10:46:24.698 INFO:tasks.workunit.client.1.smithi096.stdout:6/942: truncate d8/d12/d32/fee 416007 0 2024-03-24T10:46:24.719 INFO:tasks.workunit.client.1.smithi096.stdout:6/943: unlink d8/d12/d28/d36/c7c 0 2024-03-24T10:46:24.721 INFO:tasks.workunit.client.1.smithi096.stdout:6/944: creat d8/f133 x:0 0 0 2024-03-24T10:46:25.070 INFO:tasks.workunit.client.1.smithi096.stdout:1/947: sync 2024-03-24T10:46:25.070 INFO:tasks.workunit.client.1.smithi096.stdout:4/834: sync 2024-03-24T10:46:25.073 INFO:tasks.workunit.client.1.smithi096.stdout:4/835: mkdir d1/d32/d42/dba/d111 0 2024-03-24T10:46:25.074 INFO:tasks.workunit.client.1.smithi096.stdout:4/836: mkdir d1/d32/d4c/d7d/dc2/d106/d112 0 2024-03-24T10:46:25.075 INFO:tasks.workunit.client.1.smithi096.stdout:4/837: truncate d1/d2b/d67/f82 288901 0 2024-03-24T10:46:25.075 INFO:tasks.workunit.client.1.smithi096.stdout:4/838: truncate d1/d32/d4c/d9d/db7/f100 271701 0 2024-03-24T10:46:25.076 INFO:tasks.workunit.client.1.smithi096.stdout:4/839: mknod d1/d2b/d6b/c113 0 2024-03-24T10:46:25.087 INFO:tasks.workunit.client.1.smithi096.stdout:1/948: dwrite d3/d23/d2f/d65/d7c/ddf/ff7 [0,4194304] 0 2024-03-24T10:46:25.092 INFO:tasks.workunit.client.1.smithi096.stdout:1/949: mkdir d3/d23/d71/d7b/dce/d103/d139/d13f 0 2024-03-24T10:46:25.093 INFO:tasks.workunit.client.1.smithi096.stdout:1/950: write d3/d23/d71/ff8 [721903,119676] 0 2024-03-24T10:46:25.093 INFO:tasks.workunit.client.1.smithi096.stdout:1/951: stat d3/d23/d71/d7b/d93/d126/l12c 0 2024-03-24T10:46:25.093 INFO:tasks.workunit.client.1.smithi096.stdout:4/840: dwrite d1/d32/d42/dbe/fd9 [8388608,4194304] 0 2024-03-24T10:46:25.095 INFO:tasks.workunit.client.1.smithi096.stdout:4/841: mkdir d1/d2b/d114 0 2024-03-24T10:46:25.097 INFO:tasks.workunit.client.1.smithi096.stdout:1/952: mknod d3/d84/c140 0 2024-03-24T10:46:25.098 INFO:tasks.workunit.client.1.smithi096.stdout:4/842: truncate d1/d2b/d6b/d80/f9f 99541 0 2024-03-24T10:46:25.100 INFO:tasks.workunit.client.1.smithi096.stdout:4/843: symlink d1/d2b/d67/d5f/dfd/l115 0 2024-03-24T10:46:25.100 INFO:tasks.workunit.client.1.smithi096.stdout:4/844: chown d1/d32/d4c/d9d/db7/c102 7 1 2024-03-24T10:46:25.101 INFO:tasks.workunit.client.1.smithi096.stdout:1/953: getdents d3/d84/d9a/dd2 0 2024-03-24T10:46:25.104 INFO:tasks.workunit.client.1.smithi096.stdout:4/845: creat d1/d2b/d89/f116 x:0 0 0 2024-03-24T10:46:25.108 INFO:tasks.workunit.client.1.smithi096.stdout:1/954: write d3/d73/fcf [1597060,116819] 0 2024-03-24T10:46:25.109 INFO:tasks.workunit.client.1.smithi096.stdout:1/955: mkdir d3/d23/d71/dbe/d141 0 2024-03-24T10:46:25.113 INFO:tasks.workunit.client.1.smithi096.stdout:4/846: dread d1/d2b/d6b/d80/fea [0,4194304] 0 2024-03-24T10:46:25.113 INFO:tasks.workunit.client.1.smithi096.stdout:4/847: creat d1/d11/f117 x:0 0 0 2024-03-24T10:46:25.114 INFO:tasks.workunit.client.1.smithi096.stdout:1/956: rename d3/d23/d8b/d5c/f67 to d3/d9b/f142 0 2024-03-24T10:46:25.114 INFO:tasks.workunit.client.1.smithi096.stdout:1/957: stat d3/d23/d8b/d79/fa8 0 2024-03-24T10:46:25.118 INFO:tasks.workunit.client.1.smithi096.stdout:4/848: truncate d1/d2b/d67/f82 227312 0 2024-03-24T10:46:25.119 INFO:tasks.workunit.client.1.smithi096.stdout:1/958: truncate d3/d23/f38 2794113 0 2024-03-24T10:46:25.121 INFO:tasks.workunit.client.1.smithi096.stdout:4/849: symlink d1/d2b/d67/d5f/da9/db5/l118 0 2024-03-24T10:46:25.123 INFO:tasks.workunit.client.1.smithi096.stdout:1/959: creat d3/dbc/f143 x:0 0 0 2024-03-24T10:46:25.125 INFO:tasks.workunit.client.1.smithi096.stdout:1/960: mkdir d3/d84/d9a/d144 0 2024-03-24T10:46:25.128 INFO:tasks.workunit.client.1.smithi096.stdout:1/961: symlink d3/d23/d2f/d65/l145 0 2024-03-24T10:46:25.130 INFO:tasks.workunit.client.1.smithi096.stdout:1/962: symlink d3/d9b/l146 0 2024-03-24T10:46:25.130 INFO:tasks.workunit.client.1.smithi096.stdout:1/963: write d3/d36/fc2 [1351182,83535] 0 2024-03-24T10:46:25.130 INFO:tasks.workunit.client.1.smithi096.stdout:1/964: creat d3/d73/f147 x:0 0 0 2024-03-24T10:46:25.134 INFO:tasks.workunit.client.1.smithi096.stdout:1/965: rename d3/d23/d8b/d5c/d9c/c115 to d3/d23/d2f/d65/d7c/ddf/c148 0 2024-03-24T10:46:25.136 INFO:tasks.workunit.client.1.smithi096.stdout:1/966: unlink d3/d23/d8b/fcd 0 2024-03-24T10:46:25.136 INFO:tasks.workunit.client.1.smithi096.stdout:1/967: fsync d3/d36/dd4/fe0 0 2024-03-24T10:46:25.137 INFO:tasks.workunit.client.1.smithi096.stdout:1/968: mknod d3/d23/d71/c149 0 2024-03-24T10:46:25.139 INFO:tasks.workunit.client.1.smithi096.stdout:1/969: rmdir d3/d23/d71/d7b/d138 0 2024-03-24T10:46:25.139 INFO:tasks.workunit.client.1.smithi096.stdout:1/970: write d3/d23/d2f/d65/fc6 [254753,94028] 0 2024-03-24T10:46:25.139 INFO:tasks.workunit.client.1.smithi096.stdout:1/971: write d3/d23/d71/dbe/fc9 [2934762,63065] 0 2024-03-24T10:46:25.139 INFO:tasks.workunit.client.1.smithi096.stdout:4/850: dwrite d1/d32/f75 [0,4194304] 0 2024-03-24T10:46:25.141 INFO:tasks.workunit.client.1.smithi096.stdout:1/972: write d3/d23/f42 [781735,85695] 0 2024-03-24T10:46:25.141 INFO:tasks.workunit.client.1.smithi096.stdout:1/973: truncate d3/d84/d9a/dd2/f11f 989780 0 2024-03-24T10:46:25.149 INFO:tasks.workunit.client.1.smithi096.stdout:4/851: mknod d1/d2b/d67/d5f/dfd/c119 0 2024-03-24T10:46:25.154 INFO:tasks.workunit.client.1.smithi096.stdout:4/852: getdents d1/d2b/d89/dbf 0 2024-03-24T10:46:25.156 INFO:tasks.workunit.client.1.smithi096.stdout:1/974: rename d3/d23/d8b/f57 to d3/d23/d8b/d5c/f14a 0 2024-03-24T10:46:25.156 INFO:tasks.workunit.client.1.smithi096.stdout:1/975: fsync d3/d23/d2f/f60 0 2024-03-24T10:46:25.156 INFO:tasks.workunit.client.1.smithi096.stdout:4/853: mknod d1/d32/d42/dba/d111/c11a 0 2024-03-24T10:46:25.157 INFO:tasks.workunit.client.1.smithi096.stdout:1/976: dread d3/d84/d9a/d116/f127 [0,4194304] 0 2024-03-24T10:46:25.158 INFO:tasks.workunit.client.1.smithi096.stdout:4/854: truncate d1/d2b/d59/d74/f57 6241753 0 2024-03-24T10:46:25.160 INFO:tasks.workunit.client.1.smithi096.stdout:1/977: link d3/d23/lb5 d3/d23/d71/d7b/l14b 0 2024-03-24T10:46:25.167 INFO:tasks.workunit.client.1.smithi096.stdout:3/938: sync 2024-03-24T10:46:25.169 INFO:tasks.workunit.client.1.smithi096.stdout:6/945: sync 2024-03-24T10:46:25.169 INFO:tasks.workunit.client.1.smithi096.stdout:6/946: read d8/d10/d19/f48 [2214886,125302] 0 2024-03-24T10:46:25.173 INFO:tasks.workunit.client.1.smithi096.stdout:4/855: dwrite d1/d32/d4c/d7e/ff8 [0,4194304] 0 2024-03-24T10:46:25.174 INFO:tasks.workunit.client.1.smithi096.stdout:6/947: unlink d8/d12/d28/d36/d65/fc1 0 2024-03-24T10:46:25.177 INFO:tasks.workunit.client.1.smithi096.stdout:6/948: rename d8/d12/d32/d38/c89 to d8/d12/d28/d36/d65/d6b/da6/db2/c134 0 2024-03-24T10:46:25.183 INFO:tasks.workunit.client.1.smithi096.stdout:1/978: dwrite d3/d84/d9a/dd2/f109 [0,4194304] 0 2024-03-24T10:46:25.183 INFO:tasks.workunit.client.1.smithi096.stdout:1/979: chown d3/d23/d71/d7b/dce/de1 2015489 1 2024-03-24T10:46:25.184 INFO:tasks.workunit.client.1.smithi096.stdout:6/949: dread d8/d12/d32/d38/d59/fcf [0,4194304] 0 2024-03-24T10:46:25.184 INFO:tasks.workunit.client.1.smithi096.stdout:6/950: write d8/d12/d32/d38/d59/db8/ff5 [296647,46986] 0 2024-03-24T10:46:25.184 INFO:tasks.workunit.client.1.smithi096.stdout:6/951: write d8/d12/d32/d38/d59/f98 [683015,33134] 0 2024-03-24T10:46:25.184 INFO:tasks.workunit.client.1.smithi096.stdout:6/952: fdatasync d8/d12/d32/f6e 0 2024-03-24T10:46:25.188 INFO:tasks.workunit.client.1.smithi096.stdout:6/953: mkdir d8/d10/d13/d63/dbf/d135 0 2024-03-24T10:46:25.189 INFO:tasks.workunit.client.1.smithi096.stdout:1/980: getdents d3/d23/d71/d85 0 2024-03-24T10:46:25.191 INFO:tasks.workunit.client.1.smithi096.stdout:3/939: dwrite d0/d13/d1e/f2c [0,4194304] 0 2024-03-24T10:46:25.192 INFO:tasks.workunit.client.1.smithi096.stdout:6/954: truncate d8/d10/d19/f34 5111675 0 2024-03-24T10:46:25.192 INFO:tasks.workunit.client.1.smithi096.stdout:6/955: fsync d8/d12/d61/fdb 0 2024-03-24T10:46:25.193 INFO:tasks.workunit.client.1.smithi096.stdout:1/981: mkdir d3/d84/d9a/d14c 0 2024-03-24T10:46:25.195 INFO:tasks.workunit.client.1.smithi096.stdout:4/856: dwrite d1/d2b/d6b/d80/fb6 [4194304,4194304] 0 2024-03-24T10:46:25.197 INFO:tasks.workunit.client.1.smithi096.stdout:3/940: mknod d0/d13/d77/d117/c137 0 2024-03-24T10:46:25.197 INFO:tasks.workunit.client.1.smithi096.stdout:3/941: write d0/d9/f5e [4395154,67609] 0 2024-03-24T10:46:25.197 INFO:tasks.workunit.client.1.smithi096.stdout:3/942: read - d0/d13/d41/ddf/d11b/f74 zero size 2024-03-24T10:46:25.197 INFO:tasks.workunit.client.1.smithi096.stdout:3/943: write d0/d13/d41/ddf/d11b/f74 [415426,23203] 0 2024-03-24T10:46:25.198 INFO:tasks.workunit.client.1.smithi096.stdout:1/982: rename d3/d36/fa3 to d3/d23/d8b/d5c/d6d/f14d 0 2024-03-24T10:46:25.201 INFO:tasks.workunit.client.1.smithi096.stdout:4/857: link d1/d2b/d67/d5f/l85 d1/d2b/d67/d5f/da9/db5/df3/dbd/l11b 0 2024-03-24T10:46:25.204 INFO:tasks.workunit.client.1.smithi096.stdout:3/944: mknod d0/d13/d45/c138 0 2024-03-24T10:46:25.204 INFO:tasks.workunit.client.1.smithi096.stdout:3/945: chown d0/d9/d22/d102/db6/fe6 4883 1 2024-03-24T10:46:25.204 INFO:tasks.workunit.client.1.smithi096.stdout:3/946: chown d0/c31 14231909 1 2024-03-24T10:46:25.204 INFO:tasks.workunit.client.1.smithi096.stdout:3/947: readlink d0/d13/d45/d7d/d39/d48/d8d/le3 0 2024-03-24T10:46:25.205 INFO:tasks.workunit.client.1.smithi096.stdout:3/948: creat d0/d13/d41/ddf/d11b/d11f/f139 x:0 0 0 2024-03-24T10:46:25.208 INFO:tasks.workunit.client.1.smithi096.stdout:3/949: mknod d0/d13/d45/d7d/d39/d48/dbd/daa/c13a 0 2024-03-24T10:46:25.208 INFO:tasks.workunit.client.1.smithi096.stdout:4/858: write d1/d2b/d67/d5f/da9/db5/df3/f46 [1374389,86903] 0 2024-03-24T10:46:25.210 INFO:tasks.workunit.client.1.smithi096.stdout:3/950: rename d0/d13/d45/d7d/d39/d48/f6a to d0/d13/d41/ddf/d11b/f13b 0 2024-03-24T10:46:25.210 INFO:tasks.workunit.client.1.smithi096.stdout:3/951: readlink d0/d13/d45/d7d/d39/d48/l55 0 2024-03-24T10:46:25.212 INFO:tasks.workunit.client.1.smithi096.stdout:4/859: link d1/f8 d1/d2b/d89/def/f11c 0 2024-03-24T10:46:25.214 INFO:tasks.workunit.client.1.smithi096.stdout:4/860: symlink d1/d2b/d59/de0/l11d 0 2024-03-24T10:46:25.215 INFO:tasks.workunit.client.1.smithi096.stdout:4/861: mkdir d1/d32/d4c/d7d/d11e 0 2024-03-24T10:46:25.216 INFO:tasks.workunit.client.1.smithi096.stdout:3/952: write d0/d9/d22/f3d [7162097,66006] 0 2024-03-24T10:46:25.218 INFO:tasks.workunit.client.1.smithi096.stdout:4/862: rename d1/d2b/d67/d5f/dfd to d1/d2b/d67/d5f/da9/db5/df3/dbd/d11f 0 2024-03-24T10:46:25.219 INFO:tasks.workunit.client.1.smithi096.stdout:3/953: mknod d0/d13/d45/d7d/d39/d48/dbd/dbc/c13c 0 2024-03-24T10:46:25.219 INFO:tasks.workunit.client.1.smithi096.stdout:3/954: write d0/d13/d45/d7d/d39/d48/ffa [1575072,100286] 0 2024-03-24T10:46:25.220 INFO:tasks.workunit.client.1.smithi096.stdout:3/955: fdatasync d0/d13/d45/d7d/d39/d48/fb0 0 2024-03-24T10:46:25.221 INFO:tasks.workunit.client.1.smithi096.stdout:4/863: mknod d1/d32/d4c/d9d/db7/d101/c120 0 2024-03-24T10:46:25.231 INFO:tasks.workunit.client.1.smithi096.stdout:4/864: truncate d1/d2b/d67/d5f/da9/db5/df3/f46 2068321 0 2024-03-24T10:46:25.231 INFO:tasks.workunit.client.1.smithi096.stdout:4/865: readlink d1/d2b/d67/d5f/da9/db5/df3/dbd/d11f/l115 0 2024-03-24T10:46:25.231 INFO:tasks.workunit.client.1.smithi096.stdout:4/866: write d1/d11/f41 [2491490,98471] 0 2024-03-24T10:46:25.232 INFO:tasks.workunit.client.1.smithi096.stdout:3/956: dwrite d0/d13/d45/d7d/d39/d48/dbd/d106/f11e [4194304,4194304] 0 2024-03-24T10:46:25.233 INFO:tasks.workunit.client.1.smithi096.stdout:4/867: rmdir d1/d2b/d6b 39 2024-03-24T10:46:25.237 INFO:tasks.workunit.client.1.smithi096.stdout:4/868: symlink d1/d2b/l121 0 2024-03-24T10:46:25.241 INFO:tasks.workunit.client.1.smithi096.stdout:4/869: creat d1/d32/d42/dbe/f122 x:0 0 0 2024-03-24T10:46:25.245 INFO:tasks.workunit.client.1.smithi096.stdout:4/870: rmdir d1/d32/d4c/d7d/dc2/d106/d112 0 2024-03-24T10:46:25.245 INFO:tasks.workunit.client.1.smithi096.stdout:4/871: write d1/d32/d42/dbe/f122 [434191,113520] 0 2024-03-24T10:46:25.247 INFO:tasks.workunit.client.1.smithi096.stdout:3/957: dwrite d0/d9/fbe [0,4194304] 0 2024-03-24T10:46:25.247 INFO:tasks.workunit.client.1.smithi096.stdout:4/872: symlink d1/d32/d4c/d7d/d11e/l123 0 2024-03-24T10:46:25.271 INFO:tasks.workunit.client.1.smithi096.stdout:3/958: dwrite d0/d13/d41/d7f/fb7 [0,4194304] 0 2024-03-24T10:46:25.271 INFO:tasks.workunit.client.1.smithi096.stdout:3/959: creat d0/d13/d41/d7f/f13d x:0 0 0 2024-03-24T10:46:25.271 INFO:tasks.workunit.client.1.smithi096.stdout:3/960: stat d0/d9/d22/l33 0 2024-03-24T10:46:25.271 INFO:tasks.workunit.client.1.smithi096.stdout:4/873: dwrite d1/d32/fe2 [0,4194304] 0 2024-03-24T10:46:25.281 INFO:tasks.workunit.client.1.smithi096.stdout:3/961: creat d0/d13/d45/d7d/d39/d48/f13e x:0 0 0 2024-03-24T10:46:25.284 INFO:tasks.workunit.client.1.smithi096.stdout:3/962: mkdir d0/d9/d120/d13f 0 2024-03-24T10:46:25.284 INFO:tasks.workunit.client.1.smithi096.stdout:3/963: dread - d0/d13/d45/d7d/d39/d48/fb0 zero size 2024-03-24T10:46:25.292 INFO:tasks.workunit.client.1.smithi096.stdout:4/874: dwrite d1/d2b/d67/d5f/fec [0,4194304] 0 2024-03-24T10:46:25.295 INFO:tasks.workunit.client.1.smithi096.stdout:4/875: dread d1/d2b/d67/d5f/da9/db5/df3/dbd/fd6 [0,4194304] 0 2024-03-24T10:46:25.296 INFO:tasks.workunit.client.1.smithi096.stdout:4/876: mkdir d1/d32/d4c/d7d/dc2/d104/d124 0 2024-03-24T10:46:25.296 INFO:tasks.workunit.client.1.smithi096.stdout:4/877: chown d1/d2b/d6b/d80/f9f 83119891 1 2024-03-24T10:46:25.297 INFO:tasks.workunit.client.1.smithi096.stdout:4/878: rmdir d1/d32/d4c/d9d 39 2024-03-24T10:46:25.298 INFO:tasks.workunit.client.1.smithi096.stdout:3/964: write d0/d13/d45/d47/f8e [2102359,46212] 0 2024-03-24T10:46:25.305 INFO:tasks.workunit.client.1.smithi096.stdout:4/879: write d1/d32/d42/f86 [523427,97748] 0 2024-03-24T10:46:25.305 INFO:tasks.workunit.client.1.smithi096.stdout:4/880: write d1/d11/f41 [5154015,30459] 0 2024-03-24T10:46:25.305 INFO:tasks.workunit.client.1.smithi096.stdout:4/881: dread - d1/d32/d4c/d7e/fd0 zero size 2024-03-24T10:46:25.321 INFO:tasks.workunit.client.1.smithi096.stdout:3/965: dwrite d0/fe9 [0,4194304] 0 2024-03-24T10:46:25.322 INFO:tasks.workunit.client.1.smithi096.stdout:4/882: dwrite d1/d32/d4c/d7e/ff8 [0,4194304] 0 2024-03-24T10:46:25.324 INFO:tasks.workunit.client.1.smithi096.stdout:3/966: creat d0/d13/d45/f140 x:0 0 0 2024-03-24T10:46:25.324 INFO:tasks.workunit.client.1.smithi096.stdout:3/967: write d0/d13/d1e/fdd [707133,11713] 0 2024-03-24T10:46:25.324 INFO:tasks.workunit.client.1.smithi096.stdout:3/968: truncate d0/d13/d45/d7d/d39/d48/dbd/d9c/f10f 502457 0 2024-03-24T10:46:25.325 INFO:tasks.workunit.client.1.smithi096.stdout:4/883: creat d1/f125 x:0 0 0 2024-03-24T10:46:25.328 INFO:tasks.workunit.client.1.smithi096.stdout:3/969: symlink d0/d13/d12c/l141 0 2024-03-24T10:46:25.331 INFO:tasks.workunit.client.1.smithi096.stdout:4/884: dread d1/d2b/d67/d5f/fd7 [0,4194304] 0 2024-03-24T10:46:25.334 INFO:tasks.workunit.client.1.smithi096.stdout:4/885: rename d1/ld3 to d1/d2b/d59/de0/l126 0 2024-03-24T10:46:25.336 INFO:tasks.workunit.client.1.smithi096.stdout:4/886: mknod d1/d2b/d6b/d80/c127 0 2024-03-24T10:46:25.339 INFO:tasks.workunit.client.1.smithi096.stdout:4/887: creat d1/d32/d42/dba/df0/f128 x:0 0 0 2024-03-24T10:46:25.344 INFO:tasks.workunit.client.1.smithi096.stdout:3/970: dwrite d0/d13/d45/d7d/d39/d48/ffa [0,4194304] 0 2024-03-24T10:46:25.349 INFO:tasks.workunit.client.1.smithi096.stdout:3/971: symlink d0/d9/l142 0 2024-03-24T10:46:25.350 INFO:tasks.workunit.client.1.smithi096.stdout:3/972: creat d0/d13/d45/d7d/d39/d48/dbd/dbc/f143 x:0 0 0 2024-03-24T10:46:25.357 INFO:tasks.workunit.client.1.smithi096.stdout:4/888: dwrite d1/d2b/d67/d5f/fec [0,4194304] 0 2024-03-24T10:46:25.357 INFO:tasks.workunit.client.1.smithi096.stdout:4/889: fdatasync d1/d2b/d89/fa0 0 2024-03-24T10:46:25.360 INFO:tasks.workunit.client.1.smithi096.stdout:3/973: write d0/d13/d45/d7d/d39/fbf [2223015,18058] 0 2024-03-24T10:46:25.361 INFO:tasks.workunit.client.1.smithi096.stdout:4/890: rename d1/d2b/d89/def to d1/d129 0 2024-03-24T10:46:25.361 INFO:tasks.workunit.client.1.smithi096.stdout:4/891: readlink d1/d32/d4c/d7e/l8d 0 2024-03-24T10:46:25.362 INFO:tasks.workunit.client.1.smithi096.stdout:3/974: dread d0/d13/d45/d7d/f63 [4194304,4194304] 0 2024-03-24T10:46:25.363 INFO:tasks.workunit.client.1.smithi096.stdout:4/892: mkdir d1/d2b/d67/d5f/d12a 0 2024-03-24T10:46:25.364 INFO:tasks.workunit.client.1.smithi096.stdout:3/975: link d0/d9/d22/fa7 d0/d13/d1e/da4/d123/f144 0 2024-03-24T10:46:25.367 INFO:tasks.workunit.client.1.smithi096.stdout:4/893: rename d1/d32/d4c/d7d/dc2 to d1/d2b/d67/d5f/d12b 0 2024-03-24T10:46:25.369 INFO:tasks.workunit.client.1.smithi096.stdout:4/894: symlink d1/d2b/d67/d5f/da9/db5/l12c 0 2024-03-24T10:46:25.369 INFO:tasks.workunit.client.1.smithi096.stdout:4/895: write d1/d32/d4c/d7e/fd4 [4164169,53749] 0 2024-03-24T10:46:25.369 INFO:tasks.workunit.client.1.smithi096.stdout:4/896: write d1/d11/f66 [2487669,89949] 0 2024-03-24T10:46:25.369 INFO:tasks.workunit.client.1.smithi096.stdout:4/897: readlink d1/d32/le7 0 2024-03-24T10:46:25.372 INFO:tasks.workunit.client.1.smithi096.stdout:4/898: dread d1/d2b/d67/f52 [0,4194304] 0 2024-03-24T10:46:25.373 INFO:tasks.workunit.client.1.smithi096.stdout:4/899: symlink d1/d2b/d67/d5f/d12b/d104/d124/l12d 0 2024-03-24T10:46:25.383 INFO:tasks.workunit.client.1.smithi096.stdout:3/976: dwrite d0/d13/d41/ddf/d11b/f74 [0,4194304] 0 2024-03-24T10:46:25.383 INFO:tasks.workunit.client.1.smithi096.stdout:4/900: write d1/d2b/d67/d5f/fd7 [4712915,121618] 0 2024-03-24T10:46:25.384 INFO:tasks.workunit.client.1.smithi096.stdout:3/977: symlink d0/d13/d45/d47/l145 0 2024-03-24T10:46:25.385 INFO:tasks.workunit.client.1.smithi096.stdout:3/978: creat d0/d13/d12c/f146 x:0 0 0 2024-03-24T10:46:25.386 INFO:tasks.workunit.client.1.smithi096.stdout:3/979: rename d0/lc5 to d0/d92/ddc/l147 0 2024-03-24T10:46:25.389 INFO:tasks.workunit.client.1.smithi096.stdout:3/980: getdents d0/d13/d45/d7d/d39/d48/dbd/d9c 0 2024-03-24T10:46:25.391 INFO:tasks.workunit.client.1.smithi096.stdout:4/901: dread d1/d32/f75 [0,4194304] 0 2024-03-24T10:46:25.391 INFO:tasks.workunit.client.1.smithi096.stdout:3/981: truncate d0/d13/d45/d47/fb9 175311 0 2024-03-24T10:46:25.391 INFO:tasks.workunit.client.1.smithi096.stdout:3/982: write d0/d13/d41/ddf/d11b/f5b [731587,12619] 0 2024-03-24T10:46:25.404 INFO:tasks.workunit.client.1.smithi096.stdout:3/983: dread d0/d9b/f12e [0,4194304] 0 2024-03-24T10:46:25.406 INFO:tasks.workunit.client.1.smithi096.stdout:3/984: mkdir d0/d13/d45/d7d/d39/d48/dbd/d9c/db1/d148 0 2024-03-24T10:46:25.406 INFO:tasks.workunit.client.1.smithi096.stdout:4/902: dwrite d1/d2b/d89/f116 [0,4194304] 0 2024-03-24T10:46:25.411 INFO:tasks.workunit.client.1.smithi096.stdout:4/903: dread d1/d32/d42/dbe/f122 [0,4194304] 0 2024-03-24T10:46:25.414 INFO:tasks.workunit.client.1.smithi096.stdout:4/904: rmdir d1/d32/d4c/d9d 39 2024-03-24T10:46:25.415 INFO:tasks.workunit.client.1.smithi096.stdout:3/985: dread d0/d9/f112 [0,4194304] 0 2024-03-24T10:46:25.417 INFO:tasks.workunit.client.1.smithi096.stdout:3/986: mkdir d0/d92/ddc/dfc/de5/d149 0 2024-03-24T10:46:25.421 INFO:tasks.workunit.client.1.smithi096.stdout:3/987: rename d0/d13/d41/ddf/l134 to d0/d13/d41/l14a 0 2024-03-24T10:46:25.441 INFO:tasks.workunit.client.1.smithi096.stdout:4/905: dwrite d1/fd [0,4194304] 0 2024-03-24T10:46:25.441 INFO:tasks.workunit.client.1.smithi096.stdout:4/906: fsync d1/d2b/d67/d5f/da9/db5/df3/fdc 0 2024-03-24T10:46:25.441 INFO:tasks.workunit.client.1.smithi096.stdout:3/988: dwrite d0/d13/d1e/fe8 [0,4194304] 0 2024-03-24T10:46:25.441 INFO:tasks.workunit.client.1.smithi096.stdout:3/989: readlink d0/d13/d41/d7f/l81 0 2024-03-24T10:46:25.444 INFO:tasks.workunit.client.1.smithi096.stdout:4/907: creat d1/d2b/d67/f12e x:0 0 0 2024-03-24T10:46:25.456 INFO:tasks.workunit.client.1.smithi096.stdout:3/990: dwrite d0/d13/d41/fcd [0,4194304] 0 2024-03-24T10:46:25.471 INFO:tasks.workunit.client.1.smithi096.stdout:4/908: creat d1/d2b/d6b/f12f x:0 0 0 2024-03-24T10:46:25.473 INFO:tasks.workunit.client.1.smithi096.stdout:3/991: dwrite d0/d13/d45/d7d/d39/d48/d85/f103 [0,4194304] 0 2024-03-24T10:46:25.490 INFO:tasks.workunit.client.1.smithi096.stdout:4/909: dwrite d1/ff [0,4194304] 0 2024-03-24T10:46:25.491 INFO:tasks.workunit.client.1.smithi096.stdout:3/992: mknod d0/d13/d41/c14b 0 2024-03-24T10:46:25.494 INFO:tasks.workunit.client.1.smithi096.stdout:4/910: rmdir d1/d2b/d67/d5f/d12b/d104 39 2024-03-24T10:46:25.498 INFO:tasks.workunit.client.1.smithi096.stdout:4/911: creat d1/d2b/d67/d5f/d12b/d104/d124/f130 x:0 0 0 2024-03-24T10:46:25.502 INFO:tasks.workunit.client.1.smithi096.stdout:3/993: rename d0/d92/ddc to d0/d13/d1e/d14c 0 2024-03-24T10:46:25.504 INFO:tasks.workunit.client.1.smithi096.stdout:4/912: rename d1/d2b/l121 to d1/d2b/d67/d5f/da9/db5/df3/dbd/d11f/l131 0 2024-03-24T10:46:25.506 INFO:tasks.workunit.client.1.smithi096.stdout:4/913: symlink d1/d2b/d6b/d80/d9c/l132 0 2024-03-24T10:46:25.510 INFO:tasks.workunit.client.1.smithi096.stdout:4/914: dread d1/d2b/d67/f6a [0,4194304] 0 2024-03-24T10:46:25.512 INFO:tasks.workunit.client.1.smithi096.stdout:4/915: creat d1/d32/d4c/da6/f133 x:0 0 0 2024-03-24T10:46:25.514 INFO:tasks.workunit.client.1.smithi096.stdout:4/916: link d1/d2b/d59/d74/l84 d1/d32/d42/dba/df0/l134 0 2024-03-24T10:46:25.516 INFO:tasks.workunit.client.1.smithi096.stdout:4/917: unlink d1/d2b/d6b/ffc 0 2024-03-24T10:46:25.519 INFO:tasks.workunit.client.1.smithi096.stdout:3/994: dwrite d0/d13/d45/d47/d8f/fdb [0,4194304] 0 2024-03-24T10:46:25.521 INFO:tasks.workunit.client.1.smithi096.stdout:3/995: getdents d0/d13/d77 0 2024-03-24T10:46:25.521 INFO:tasks.workunit.client.1.smithi096.stdout:3/996: chown d0/d13/d45/d47/f8e 0 1 2024-03-24T10:46:25.532 INFO:tasks.workunit.client.1.smithi096.stdout:3/997: write d0/d9/d22/fa7 [1551652,39363] 0 2024-03-24T10:46:25.542 INFO:tasks.workunit.client.1.smithi096.stdout:4/918: dwrite d1/d11/f29 [0,4194304] 0 2024-03-24T10:46:25.546 INFO:tasks.workunit.client.1.smithi096.stdout:4/919: truncate d1/d32/d42/fb0 1977471 0 2024-03-24T10:46:25.546 INFO:tasks.workunit.client.1.smithi096.stdout:4/920: truncate d1/d32/d4c/d7e/ff8 5085739 0 2024-03-24T10:46:25.551 INFO:tasks.workunit.client.1.smithi096.stdout:3/998: dwrite d0/d13/d1e/d14c/dfc/de5/ff6 [0,4194304] 0 2024-03-24T10:46:25.551 INFO:tasks.workunit.client.1.smithi096.stdout:3/999: fsync d0/d13/d41/ddf/fd4 0 2024-03-24T10:46:25.561 INFO:tasks.workunit.client.1.smithi096.stdout:4/921: dwrite d1/d32/d4c/d7d/fab [0,4194304] 0 2024-03-24T10:46:25.565 INFO:tasks.workunit.client.1.smithi096.stdout:4/922: creat d1/d2b/d89/dbf/f135 x:0 0 0 2024-03-24T10:46:25.566 INFO:tasks.workunit.client.1.smithi096.stdout:4/923: truncate d1/d2b/d6b/d80/d9c/fc8 502584 0 2024-03-24T10:46:25.566 INFO:tasks.workunit.client.1.smithi096.stdout:4/924: stat d1/d2b/d67/f6a 0 2024-03-24T10:46:25.567 INFO:tasks.workunit.client.1.smithi096.stdout:4/925: unlink d1/d11/c98 0 2024-03-24T10:46:25.568 INFO:tasks.workunit.client.1.smithi096.stdout:4/926: symlink d1/d32/d42/dbe/l136 0 2024-03-24T10:46:25.568 INFO:tasks.workunit.client.1.smithi096.stdout:4/927: creat d1/d2b/d6b/f137 x:0 0 0 2024-03-24T10:46:25.568 INFO:tasks.workunit.client.1.smithi096.stdout:1/983: sync 2024-03-24T10:46:25.568 INFO:tasks.workunit.client.1.smithi096.stdout:6/956: sync 2024-03-24T10:46:25.577 INFO:tasks.workunit.client.1.smithi096.stdout:1/984: dread d3/d36/f54 [0,4194304] 0 2024-03-24T10:46:25.584 INFO:tasks.workunit.client.1.smithi096.stdout:6/957: dwrite d8/d10/d13/f2b [0,4194304] 0 2024-03-24T10:46:25.587 INFO:tasks.workunit.client.1.smithi096.stdout:6/958: creat d8/d12/d32/d10e/f136 x:0 0 0 2024-03-24T10:46:25.587 INFO:tasks.workunit.client.1.smithi096.stdout:6/959: readlink d8/d12/d28/d36/d65/lc0 0 2024-03-24T10:46:25.587 INFO:tasks.workunit.client.1.smithi096.stdout:1/985: dread d3/d23/d71/d85/ff9 [0,4194304] 0 2024-03-24T10:46:25.588 INFO:tasks.workunit.client.1.smithi096.stdout:6/960: mknod d8/d12/d32/d38/d120/c137 0 2024-03-24T10:46:25.590 INFO:tasks.workunit.client.1.smithi096.stdout:1/986: dread d3/d23/d2f/f60 [0,4194304] 0 2024-03-24T10:46:25.590 INFO:tasks.workunit.client.1.smithi096.stdout:1/987: write d3/d36/dd4/fe0 [371474,45272] 0 2024-03-24T10:46:25.593 INFO:tasks.workunit.client.1.smithi096.stdout:1/988: link d3/d23/d2f/d65/d7c/ddf/ff7 d3/d84/f14e 0 2024-03-24T10:46:25.595 INFO:tasks.workunit.client.1.smithi096.stdout:1/989: mkdir d3/d23/d8b/d5c/d9c/d14f 0 2024-03-24T10:46:25.615 INFO:tasks.workunit.client.1.smithi096.stdout:1/990: dwrite d3/d23/d8b/fbd [0,4194304] 0 2024-03-24T10:46:25.615 INFO:tasks.workunit.client.1.smithi096.stdout:1/991: chown d3/d23/d2f/d65/d7c/l90 142 1 2024-03-24T10:46:25.616 INFO:tasks.workunit.client.1.smithi096.stdout:6/961: dwrite d8/d10/d13/d63/f2c [0,4194304] 0 2024-03-24T10:46:25.620 INFO:tasks.workunit.client.1.smithi096.stdout:6/962: rename d8/d12/d32/d38/d59/f5a to d8/d12/d28/d36/d65/d6b/da6/d103/f138 0 2024-03-24T10:46:25.623 INFO:tasks.workunit.client.1.smithi096.stdout:6/963: symlink d8/d12/d28/d36/l139 0 2024-03-24T10:46:25.624 INFO:tasks.workunit.client.1.smithi096.stdout:6/964: creat d8/d10/f13a x:0 0 0 2024-03-24T10:46:25.624 INFO:tasks.workunit.client.1.smithi096.stdout:6/965: read - d8/d12/d28/d36/d65/d71/d10d/da8/dbe/d112/f124 zero size 2024-03-24T10:46:25.632 INFO:tasks.workunit.client.1.smithi096.stdout:1/992: dwrite d3/d23/d2f/f135 [0,4194304] 0 2024-03-24T10:46:25.633 INFO:tasks.workunit.client.1.smithi096.stdout:6/966: write d8/d7b/fab [2733654,106385] 0 2024-03-24T10:46:25.633 INFO:tasks.workunit.client.1.smithi096.stdout:6/967: stat d8/d12/d28/d36/d65/d6b/da6/db2/df8 0 2024-03-24T10:46:25.646 INFO:tasks.workunit.client.1.smithi096.stdout:1/993: mkdir d3/d23/d8b/d5c/d150 0 2024-03-24T10:46:25.646 INFO:tasks.workunit.client.1.smithi096.stdout:1/994: chown d3/d23/d71/d7b/dce 15286838 1 2024-03-24T10:46:25.647 INFO:tasks.workunit.client.1.smithi096.stdout:1/995: write d3/d23/d71/d7b/faa [450074,15773] 0 2024-03-24T10:46:25.647 INFO:tasks.workunit.client.1.smithi096.stdout:1/996: chown d3/d23/d8b/d8c/fe2 0 1 2024-03-24T10:46:25.649 INFO:tasks.workunit.client.1.smithi096.stdout:1/997: rename d3/d23/d2f/l40 to d3/d23/d71/d7b/dce/l151 0 2024-03-24T10:46:25.650 INFO:tasks.workunit.client.1.smithi096.stdout:1/998: mknod d3/d73/ded/c152 0 2024-03-24T10:46:25.954 INFO:tasks.workunit.client.1.smithi096.stdout:6/968: sync 2024-03-24T10:46:25.954 INFO:tasks.workunit.client.1.smithi096.stdout:4/928: sync 2024-03-24T10:46:25.958 INFO:tasks.workunit.client.1.smithi096.stdout:4/929: creat d1/d32/d4c/f138 x:0 0 0 2024-03-24T10:46:25.958 INFO:tasks.workunit.client.1.smithi096.stdout:4/930: chown d1/d32/le7 372 1 2024-03-24T10:46:25.963 INFO:tasks.workunit.client.1.smithi096.stdout:6/969: write d8/d10/d13/d63/f2c [975055,46654] 0 2024-03-24T10:46:25.972 INFO:tasks.workunit.client.1.smithi096.stdout:6/970: write d8/d12/d28/d36/d65/d6b/da6/d103/f138 [1102493,60024] 0 2024-03-24T10:46:25.972 INFO:tasks.workunit.client.1.smithi096.stdout:6/971: write d8/d10/d19/dfd/f115 [5076919,27993] 0 2024-03-24T10:46:25.972 INFO:tasks.workunit.client.1.smithi096.stdout:6/972: write d8/d12/d28/d36/d65/d6b/f107 [1041137,12465] 0 2024-03-24T10:46:25.973 INFO:tasks.workunit.client.1.smithi096.stdout:6/973: dread - d8/d12/d28/d36/d65/d71/d10d/da8/dbe/ff9 zero size 2024-03-24T10:46:25.974 INFO:tasks.workunit.client.1.smithi096.stdout:6/974: write d8/d10/f3d [2120265,68391] 0 2024-03-24T10:46:25.978 INFO:tasks.workunit.client.1.smithi096.stdout:4/931: dwrite d1/d32/d42/dbe/f122 [0,4194304] 0 2024-03-24T10:46:25.978 INFO:tasks.workunit.client.1.smithi096.stdout:4/932: chown d1/d2b/d67/d5f/cb3 196764 1 2024-03-24T10:46:25.985 INFO:tasks.workunit.client.1.smithi096.stdout:4/933: getdents d1/d32/d42/dba/df0 0 2024-03-24T10:46:25.987 INFO:tasks.workunit.client.1.smithi096.stdout:4/934: link d1/d2b/d6b/d80/d9c/l132 d1/d2b/d67/d5f/da9/db5/df3/dbd/d11f/d108/l139 0 2024-03-24T10:46:25.987 INFO:tasks.workunit.client.1.smithi096.stdout:4/935: write d1/d2b/d6b/f12f [720747,77214] 0 2024-03-24T10:46:25.990 INFO:tasks.workunit.client.1.smithi096.stdout:4/936: truncate d1/d32/d4c/d7e/ff8 2034449 0 2024-03-24T10:46:25.990 INFO:tasks.workunit.client.1.smithi096.stdout:4/937: write d1/d32/d4c/d7e/fcf [449429,115244] 0 2024-03-24T10:46:25.992 INFO:tasks.workunit.client.1.smithi096.stdout:4/938: dread d1/d2b/d59/d74/ff6 [0,4194304] 0 2024-03-24T10:46:25.993 INFO:tasks.workunit.client.1.smithi096.stdout:4/939: mknod d1/d2b/d6b/d80/c13a 0 2024-03-24T10:46:25.993 INFO:tasks.workunit.client.1.smithi096.stdout:4/940: write d1/d32/d42/f5e [8986291,54020] 0 2024-03-24T10:46:25.995 INFO:tasks.workunit.client.1.smithi096.stdout:4/941: unlink d1/d2b/d67/d5f/da9/db5/df3/dbd/d11f/l115 0 2024-03-24T10:46:25.998 INFO:tasks.workunit.client.1.smithi096.stdout:6/975: dwrite d8/d7b/d10c/dfc/f105 [0,4194304] 0 2024-03-24T10:46:26.026 INFO:tasks.workunit.client.1.smithi096.stdout:6/976: dwrite d8/d12/d28/d36/d65/d6b/da6/d103/f12a [0,4194304] 0 2024-03-24T10:46:26.026 INFO:tasks.workunit.client.1.smithi096.stdout:6/977: write d8/d10/d76/fa9 [655905,74846] 0 2024-03-24T10:46:26.026 INFO:tasks.workunit.client.1.smithi096.stdout:6/978: write d8/d12/d61/faa [838378,20817] 0 2024-03-24T10:46:26.027 INFO:tasks.workunit.client.1.smithi096.stdout:4/942: dwrite d1/d2b/d67/d5f/d12b/d104/d124/f130 [0,4194304] 0 2024-03-24T10:46:26.027 INFO:tasks.workunit.client.1.smithi096.stdout:4/943: chown d1/d2b/d6b/d80/c127 60 1 2024-03-24T10:46:26.027 INFO:tasks.workunit.client.1.smithi096.stdout:4/944: fdatasync d1/d2b/d89/f116 0 2024-03-24T10:46:26.029 INFO:tasks.workunit.client.1.smithi096.stdout:4/945: dread d1/d2b/d67/d5f/da9/db5/df3/f46 [0,4194304] 0 2024-03-24T10:46:26.030 INFO:tasks.workunit.client.1.smithi096.stdout:4/946: read d1/f3e [371567,4672] 0 2024-03-24T10:46:26.030 INFO:tasks.workunit.client.1.smithi096.stdout:4/947: creat d1/d2b/d67/d5f/d12b/d104/d124/f13b x:0 0 0 2024-03-24T10:46:26.031 INFO:tasks.workunit.client.1.smithi096.stdout:6/979: symlink d8/d12/d32/d68/l13b 0 2024-03-24T10:46:26.035 INFO:tasks.workunit.client.1.smithi096.stdout:6/980: rmdir d8/d12/d61/d113 39 2024-03-24T10:46:26.047 INFO:tasks.workunit.client.1.smithi096.stdout:4/948: dwrite d1/f125 [0,4194304] 0 2024-03-24T10:46:26.051 INFO:tasks.workunit.client.1.smithi096.stdout:4/949: mkdir d1/d32/d42/dba/df0/d13c 0 2024-03-24T10:46:26.058 INFO:tasks.workunit.client.1.smithi096.stdout:1/999: sync 2024-03-24T10:46:26.059 INFO:tasks.workunit.client.1.smithi096.stdout:4/950: getdents d1/d32/d42/dba/df0 0 2024-03-24T10:46:26.060 INFO:tasks.workunit.client.1.smithi096.stdout:4/951: creat d1/d2b/d67/d5f/da9/db5/df3/f13d x:0 0 0 2024-03-24T10:46:26.062 INFO:tasks.workunit.client.1.smithi096.stdout:4/952: unlink d1/d32/fe2 0 2024-03-24T10:46:26.062 INFO:tasks.workunit.client.1.smithi096.stdout:4/953: dread - d1/d32/d42/f79 zero size 2024-03-24T10:46:26.062 INFO:tasks.workunit.client.1.smithi096.stdout:4/954: mknod d1/d32/c13e 0 2024-03-24T10:46:26.063 INFO:tasks.workunit.client.1.smithi096.stdout:4/955: mknod d1/d32/d42/dbe/c13f 0 2024-03-24T10:46:26.065 INFO:tasks.workunit.client.1.smithi096.stdout:4/956: symlink d1/d2b/d67/d5f/da9/db5/df3/l140 0 2024-03-24T10:46:26.066 INFO:tasks.workunit.client.1.smithi096.stdout:4/957: link d1/d2b/d67/f12e d1/d2b/f141 0 2024-03-24T10:46:26.067 INFO:tasks.workunit.client.1.smithi096.stdout:4/958: creat d1/d2b/d67/d5f/d12b/d104/d124/f142 x:0 0 0 2024-03-24T10:46:26.090 INFO:tasks.workunit.client.1.smithi096.stdout:4/959: dwrite d1/f3e [0,4194304] 0 2024-03-24T10:46:26.090 INFO:tasks.workunit.client.1.smithi096.stdout:4/960: creat d1/d2b/d67/d5f/da9/db5/df3/f143 x:0 0 0 2024-03-24T10:46:26.096 INFO:tasks.workunit.client.1.smithi096.stdout:4/961: creat d1/d2b/d67/d5f/da9/db5/df3/dbd/d11f/d108/f144 x:0 0 0 2024-03-24T10:46:26.098 INFO:tasks.workunit.client.1.smithi096.stdout:4/962: link d1/d32/d4c/cee d1/d2b/d6b/d80/c145 0 2024-03-24T10:46:26.098 INFO:tasks.workunit.client.1.smithi096.stdout:4/963: chown d1/c3b 4 1 2024-03-24T10:46:26.100 INFO:tasks.workunit.client.1.smithi096.stdout:4/964: symlink d1/d11/l146 0 2024-03-24T10:46:26.100 INFO:tasks.workunit.client.1.smithi096.stdout:4/965: chown d1/f24 4017070 1 2024-03-24T10:46:26.102 INFO:tasks.workunit.client.1.smithi096.stdout:4/966: creat d1/d32/d4c/d7d/d11e/f147 x:0 0 0 2024-03-24T10:46:26.102 INFO:tasks.workunit.client.1.smithi096.stdout:4/967: write d1/d2b/d6b/f12f [805358,104094] 0 2024-03-24T10:46:26.102 INFO:tasks.workunit.client.1.smithi096.stdout:4/968: write d1/d32/d4c/d7d/d11e/f147 [609821,63230] 0 2024-03-24T10:46:26.102 INFO:tasks.workunit.client.1.smithi096.stdout:4/969: chown d1/d2b/d6b/f12f 370070 1 2024-03-24T10:46:26.108 INFO:tasks.workunit.client.1.smithi096.stdout:4/970: dread d1/f35 [0,4194304] 0 2024-03-24T10:46:26.121 INFO:tasks.workunit.client.1.smithi096.stdout:4/971: dwrite d1/d2b/d6b/f137 [0,4194304] 0 2024-03-24T10:46:26.125 INFO:tasks.workunit.client.1.smithi096.stdout:4/972: mkdir d1/d2b/d114/d148 0 2024-03-24T10:46:26.126 INFO:tasks.workunit.client.1.smithi096.stdout:4/973: rename d1/d32/d4c/d7d/d11e/f147 to d1/d2b/d67/d5f/da9/db5/df3/de6/f149 0 2024-03-24T10:46:26.126 INFO:tasks.workunit.client.1.smithi096.stdout:4/974: truncate d1/d32/d42/dba/df0/f128 197791 0 2024-03-24T10:46:26.140 INFO:tasks.workunit.client.1.smithi096.stdout:4/975: dwrite d1/d2b/d67/d5f/da9/db5/fd2 [0,4194304] 0 2024-03-24T10:46:26.145 INFO:tasks.workunit.client.1.smithi096.stdout:4/976: mknod d1/d2b/d59/d74/c14a 0 2024-03-24T10:46:26.145 INFO:tasks.workunit.client.1.smithi096.stdout:4/977: stat d1/d32/d42/f86 0 2024-03-24T10:46:26.146 INFO:tasks.workunit.client.1.smithi096.stdout:4/978: mknod d1/d2b/d67/d5f/c14b 0 2024-03-24T10:46:26.149 INFO:tasks.workunit.client.1.smithi096.stdout:4/979: getdents d1/d2b/d67/d5f/d12b/d106 0 2024-03-24T10:46:26.149 INFO:tasks.workunit.client.1.smithi096.stdout:4/980: fsync d1/d2b/d67/d5f/da9/db5/df3/f64 0 2024-03-24T10:46:26.151 INFO:tasks.workunit.client.1.smithi096.stdout:4/981: creat d1/d129/f14c x:0 0 0 2024-03-24T10:46:26.154 INFO:tasks.workunit.client.1.smithi096.stdout:4/982: mknod d1/d2b/d59/c14d 0 2024-03-24T10:46:26.161 INFO:tasks.workunit.client.1.smithi096.stdout:4/983: link d1/l4b d1/d32/d42/dba/df0/d13c/l14e 0 2024-03-24T10:46:26.165 INFO:tasks.workunit.client.1.smithi096.stdout:4/984: rename d1/d32/d42/dba/df0/f128 to d1/d32/d42/dba/df0/f14f 0 2024-03-24T10:46:26.165 INFO:tasks.workunit.client.1.smithi096.stdout:4/985: write d1/d2b/d67/d5f/da9/db5/df3/f70 [533729,98200] 0 2024-03-24T10:46:26.165 INFO:tasks.workunit.client.1.smithi096.stdout:4/986: read - d1/d2b/d89/dbf/f135 zero size 2024-03-24T10:46:26.165 INFO:tasks.workunit.client.1.smithi096.stdout:4/987: fsync d1/d2b/f141 0 2024-03-24T10:46:26.167 INFO:tasks.workunit.client.1.smithi096.stdout:4/988: symlink d1/d32/d4c/d9d/l150 0 2024-03-24T10:46:26.167 INFO:tasks.workunit.client.1.smithi096.stdout:4/989: fsync d1/d32/d4c/da6/f133 0 2024-03-24T10:46:26.169 INFO:tasks.workunit.client.1.smithi096.stdout:4/990: mkdir d1/d129/d151 0 2024-03-24T10:46:26.178 INFO:tasks.workunit.client.1.smithi096.stdout:4/991: dread d1/d32/d42/dbe/fd9 [0,4194304] 0 2024-03-24T10:46:26.178 INFO:tasks.workunit.client.1.smithi096.stdout:4/992: chown d1/d2b/d6b/d80/d9c 331 1 2024-03-24T10:46:26.179 INFO:tasks.workunit.client.1.smithi096.stdout:4/993: read d1/f51 [7463929,20403] 0 2024-03-24T10:46:26.179 INFO:tasks.workunit.client.1.smithi096.stdout:4/994: dread d1/d32/d42/dba/df0/f14f [0,4194304] 0 2024-03-24T10:46:26.180 INFO:tasks.workunit.client.1.smithi096.stdout:4/995: read d1/d2b/d67/d5f/fc0 [348880,61383] 0 2024-03-24T10:46:26.181 INFO:tasks.workunit.client.1.smithi096.stdout:4/996: creat d1/d2b/d67/d5f/f152 x:0 0 0 2024-03-24T10:46:26.181 INFO:tasks.workunit.client.1.smithi096.stdout:4/997: dread - d1/d2b/d89/dbf/f135 zero size 2024-03-24T10:46:26.182 INFO:tasks.workunit.client.1.smithi096.stdout:4/998: mknod d1/d2b/c153 0 2024-03-24T10:46:26.184 INFO:tasks.workunit.client.1.smithi096.stdout:4/999: rename d1/d2b/d6b/f12f to d1/d32/d4c/d7d/f154 0 2024-03-24T10:46:26.245 INFO:tasks.workunit.client.1.smithi096.stdout:6/981: sync 2024-03-24T10:46:26.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:25 smithi094 ceph-mon[29446]: pgmap v256: 65 pgs: 65 active+clean; 2.9 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 146 MiB/s rd, 179 MiB/s wr, 348 op/s 2024-03-24T10:46:26.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:25 smithi096 ceph-mon[38097]: pgmap v256: 65 pgs: 65 active+clean; 2.9 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 146 MiB/s rd, 179 MiB/s wr, 348 op/s 2024-03-24T10:46:26.431 INFO:tasks.workunit.client.1.smithi096.stdout:6/982: sync 2024-03-24T10:46:26.433 INFO:tasks.workunit.client.1.smithi096.stdout:6/983: symlink d8/l13c 0 2024-03-24T10:46:26.433 INFO:tasks.workunit.client.1.smithi096.stdout:6/984: read d8/d12/d32/d38/d59/db8/fb1 [1672761,110345] 0 2024-03-24T10:46:26.433 INFO:tasks.workunit.client.1.smithi096.stdout:6/985: chown d8/d10/d76/fc3 338 1 2024-03-24T10:46:26.433 INFO:tasks.workunit.client.1.smithi096.stdout:6/986: dread - d8/d12/d61/fdb zero size 2024-03-24T10:46:26.434 INFO:tasks.workunit.client.1.smithi096.stdout:6/987: mkdir d8/d12/d32/d38/d59/db8/d13d 0 2024-03-24T10:46:26.435 INFO:tasks.workunit.client.1.smithi096.stdout:6/988: creat d8/d12/d61/f13e x:0 0 0 2024-03-24T10:46:26.435 INFO:tasks.workunit.client.1.smithi096.stdout:6/989: readlink d8/d12/d32/l90 0 2024-03-24T10:46:26.448 INFO:tasks.workunit.client.1.smithi096.stdout:6/990: dwrite d8/d10/d76/f97 [0,4194304] 0 2024-03-24T10:46:26.448 INFO:tasks.workunit.client.1.smithi096.stdout:6/991: stat d8/d12/d28/d36/d65/d6b/d129/f130 0 2024-03-24T10:46:26.451 INFO:tasks.workunit.client.1.smithi096.stdout:6/992: mknod d8/d12/d28/d36/d65/d71/d10d/da8/c13f 0 2024-03-24T10:46:26.453 INFO:tasks.workunit.client.1.smithi096.stdout:6/993: getdents d8/d10/d13/d63/dbf 0 2024-03-24T10:46:26.455 INFO:tasks.workunit.client.1.smithi096.stdout:6/994: mknod d8/d12/d61/d113/c140 0 2024-03-24T10:46:26.653 INFO:tasks.workunit.client.1.smithi096.stdout:6/995: sync 2024-03-24T10:46:26.865 INFO:tasks.workunit.client.1.smithi096.stdout:6/996: mknod d8/d12/d28/c141 0 2024-03-24T10:46:26.879 INFO:tasks.workunit.client.1.smithi096.stdout:6/997: write d8/d10/d13/dcc/fda [2904957,52868] 0 2024-03-24T10:46:26.885 INFO:tasks.workunit.client.1.smithi096.stdout:6/998: mknod d8/d12/d28/d36/d65/d6b/dec/c142 0 2024-03-24T10:46:26.891 INFO:tasks.workunit.client.1.smithi096.stdout:6/999: rename d8/d10/d76/d91/dd5/de5/l12b to d8/d12/d28/d36/d65/d6b/da6/db2/df2/l143 0 2024-03-24T10:46:26.896 INFO:tasks.workunit.client.1.smithi096.stderr:+ rm -rf -- ./tmp.NH14HyWfuo 2024-03-24T10:46:27.872 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-24T10:46:27.872 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-24T10:46:28.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:27 smithi096 ceph-mon[38097]: pgmap v257: 65 pgs: 65 active+clean; 2.8 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 164 MiB/s rd, 216 MiB/s wr, 386 op/s 2024-03-24T10:46:28.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:27 smithi094 ceph-mon[29446]: pgmap v257: 65 pgs: 65 active+clean; 2.8 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 164 MiB/s rd, 216 MiB/s wr, 386 op/s 2024-03-24T10:46:30.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:30 smithi094 ceph-mon[29446]: pgmap v258: 65 pgs: 65 active+clean; 2.7 GiB data, 9.9 GiB used, 527 GiB / 536 GiB avail; 128 MiB/s rd, 172 MiB/s wr, 314 op/s 2024-03-24T10:46:30.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:30 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:46:30.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:30 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:46:30.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:30 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:46:30.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:30 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:46:30.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:30 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:46:30.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:30 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:46:30.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:30 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mgr fail", "who": "smithi094.yaethu"}]: dispatch 2024-03-24T10:46:30.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:30 smithi094 ceph-mon[29446]: osdmap e47: 6 total, 6 up, 6 in 2024-03-24T10:46:30.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:30 smithi094 ceph-mon[29446]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "mgr fail", "who": "smithi094.yaethu"}]': finished 2024-03-24T10:46:30.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:30 smithi094 ceph-mon[29446]: mgrmap e21: smithi096.gfonnu(active, starting, since 0.00637532s) 2024-03-24T10:46:30.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:30 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "mon metadata", "id": "smithi094"}]: dispatch 2024-03-24T10:46:30.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:30 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "mon metadata", "id": "smithi096"}]: dispatch 2024-03-24T10:46:30.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:30 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.llefgu"}]: dispatch 2024-03-24T10:46:30.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:30 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.efiqte"}]: dispatch 2024-03-24T10:46:30.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:30 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi096.egbmoq"}]: dispatch 2024-03-24T10:46:30.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:30 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi096.jtebhh"}]: dispatch 2024-03-24T10:46:30.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:30 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "mgr metadata", "who": "smithi096.gfonnu", "id": "smithi096.gfonnu"}]: dispatch 2024-03-24T10:46:30.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:30 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T10:46:30.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:30 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T10:46:30.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:30 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T10:46:30.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:30 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T10:46:30.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:30 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T10:46:30.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:30 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T10:46:30.500 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:30 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-24T10:46:30.500 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:30 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-24T10:46:30.500 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:30 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-24T10:46:30.500 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:30 smithi094 ceph-mon[29446]: Manager daemon smithi096.gfonnu is now available 2024-03-24T10:46:30.500 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:30 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:46:30.500 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:30 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:46:30.500 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:30 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:46:30.500 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:30 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi096.gfonnu/mirror_snapshot_schedule"}]: dispatch 2024-03-24T10:46:30.500 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:30 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi096.gfonnu/trash_purge_schedule"}]: dispatch 2024-03-24T10:46:30.500 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:30 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:30.500 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:30 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:30.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:30 smithi096 ceph-mon[38097]: pgmap v258: 65 pgs: 65 active+clean; 2.7 GiB data, 9.9 GiB used, 527 GiB / 536 GiB avail; 128 MiB/s rd, 172 MiB/s wr, 314 op/s 2024-03-24T10:46:30.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:30 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:46:30.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:30 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:46:30.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:30 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:46:30.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:30 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:46:30.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:30 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' 2024-03-24T10:46:30.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:30 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:46:30.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:30 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mgr fail", "who": "smithi094.yaethu"}]: dispatch 2024-03-24T10:46:30.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:30 smithi096 ceph-mon[38097]: osdmap e47: 6 total, 6 up, 6 in 2024-03-24T10:46:30.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:30 smithi096 ceph-mon[38097]: from='mgr.14232 172.21.15.94:0/3888632041' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "mgr fail", "who": "smithi094.yaethu"}]': finished 2024-03-24T10:46:30.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:30 smithi096 ceph-mon[38097]: mgrmap e21: smithi096.gfonnu(active, starting, since 0.00637532s) 2024-03-24T10:46:30.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:30 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "mon metadata", "id": "smithi094"}]: dispatch 2024-03-24T10:46:30.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:30 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "mon metadata", "id": "smithi096"}]: dispatch 2024-03-24T10:46:30.577 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:30 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.llefgu"}]: dispatch 2024-03-24T10:46:30.577 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:30 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.efiqte"}]: dispatch 2024-03-24T10:46:30.577 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:30 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi096.egbmoq"}]: dispatch 2024-03-24T10:46:30.577 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:30 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi096.jtebhh"}]: dispatch 2024-03-24T10:46:30.577 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:30 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "mgr metadata", "who": "smithi096.gfonnu", "id": "smithi096.gfonnu"}]: dispatch 2024-03-24T10:46:30.577 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:30 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T10:46:30.577 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:30 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T10:46:30.577 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:30 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T10:46:30.577 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:30 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T10:46:30.577 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:30 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T10:46:30.577 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:30 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T10:46:30.577 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:30 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-24T10:46:30.577 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:30 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-24T10:46:30.577 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:30 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-24T10:46:30.577 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:30 smithi096 ceph-mon[38097]: Manager daemon smithi096.gfonnu is now available 2024-03-24T10:46:30.578 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:30 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:46:30.578 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:30 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:46:30.578 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:30 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:46:30.578 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:30 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi096.gfonnu/mirror_snapshot_schedule"}]: dispatch 2024-03-24T10:46:30.578 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:30 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi096.gfonnu/trash_purge_schedule"}]: dispatch 2024-03-24T10:46:30.578 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:30 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:30.578 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:30 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:31.612 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:31 smithi094 ceph-mon[29446]: Deploying cephadm binary to smithi094 2024-03-24T10:46:31.612 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:31 smithi094 ceph-mon[29446]: Deploying cephadm binary to smithi096 2024-03-24T10:46:31.613 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:31 smithi094 ceph-mon[29446]: mgrmap e22: smithi096.gfonnu(active, since 1.00853s) 2024-03-24T10:46:31.613 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:31 smithi094 ceph-mon[29446]: pgmap v3: 65 pgs: 65 active+clean; 2.1 GiB data, 7.9 GiB used, 529 GiB / 536 GiB avail 2024-03-24T10:46:31.613 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:31 smithi094 ceph-mon[29446]: [24/Mar/2024:10:46:30] ENGINE Bus STARTING 2024-03-24T10:46:31.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:31 smithi096 ceph-mon[38097]: Deploying cephadm binary to smithi094 2024-03-24T10:46:31.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:31 smithi096 ceph-mon[38097]: Deploying cephadm binary to smithi096 2024-03-24T10:46:31.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:31 smithi096 ceph-mon[38097]: mgrmap e22: smithi096.gfonnu(active, since 1.00853s) 2024-03-24T10:46:31.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:31 smithi096 ceph-mon[38097]: pgmap v3: 65 pgs: 65 active+clean; 2.1 GiB data, 7.9 GiB used, 529 GiB / 536 GiB avail 2024-03-24T10:46:31.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:31 smithi096 ceph-mon[38097]: [24/Mar/2024:10:46:30] ENGINE Bus STARTING 2024-03-24T10:46:32.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:32 smithi094 ceph-mon[29446]: [24/Mar/2024:10:46:31] ENGINE Serving on https://172.21.15.96:7150 2024-03-24T10:46:32.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:32 smithi094 ceph-mon[29446]: [24/Mar/2024:10:46:31] ENGINE Serving on http://172.21.15.96:8765 2024-03-24T10:46:32.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:32 smithi094 ceph-mon[29446]: [24/Mar/2024:10:46:31] ENGINE Bus STARTED 2024-03-24T10:46:32.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:32 smithi094 ceph-mon[29446]: pgmap v4: 65 pgs: 65 active+clean; 2.1 GiB data, 7.9 GiB used, 529 GiB / 536 GiB avail 2024-03-24T10:46:33.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:32 smithi096 ceph-mon[38097]: [24/Mar/2024:10:46:31] ENGINE Serving on https://172.21.15.96:7150 2024-03-24T10:46:33.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:32 smithi096 ceph-mon[38097]: [24/Mar/2024:10:46:31] ENGINE Serving on http://172.21.15.96:8765 2024-03-24T10:46:33.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:32 smithi096 ceph-mon[38097]: [24/Mar/2024:10:46:31] ENGINE Bus STARTED 2024-03-24T10:46:33.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:32 smithi096 ceph-mon[38097]: pgmap v4: 65 pgs: 65 active+clean; 2.1 GiB data, 7.9 GiB used, 529 GiB / 536 GiB avail 2024-03-24T10:46:33.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:33 smithi094 ceph-mon[29446]: mgrmap e23: smithi096.gfonnu(active, since 3s) 2024-03-24T10:46:34.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:33 smithi096 ceph-mon[38097]: mgrmap e23: smithi096.gfonnu(active, since 3s) 2024-03-24T10:46:34.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:34 smithi094 ceph-mon[29446]: pgmap v5: 65 pgs: 65 active+clean; 2.1 GiB data, 7.9 GiB used, 529 GiB / 536 GiB avail 2024-03-24T10:46:35.077 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:34 smithi096 ceph-mon[38097]: pgmap v5: 65 pgs: 65 active+clean; 2.1 GiB data, 7.9 GiB used, 529 GiB / 536 GiB avail 2024-03-24T10:46:36.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:36 smithi094 ceph-mon[29446]: pgmap v6: 65 pgs: 65 active+clean; 1.8 GiB data, 7.2 GiB used, 529 GiB / 536 GiB avail; 492 KiB/s rd, 845 KiB/s wr, 106 op/s 2024-03-24T10:46:37.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:36 smithi096 ceph-mon[38097]: pgmap v6: 65 pgs: 65 active+clean; 1.8 GiB data, 7.2 GiB used, 529 GiB / 536 GiB avail; 492 KiB/s rd, 845 KiB/s wr, 106 op/s 2024-03-24T10:46:37.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:37 smithi096 ceph-mon[38097]: Standby manager daemon smithi094.yaethu started 2024-03-24T10:46:37.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:37 smithi096 ceph-mon[38097]: from='mgr.? 172.21.15.94:0/3629028016' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi094.yaethu/crt"}]: dispatch 2024-03-24T10:46:37.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:37 smithi096 ceph-mon[38097]: from='mgr.? 172.21.15.94:0/3629028016' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T10:46:37.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:37 smithi096 ceph-mon[38097]: from='mgr.? 172.21.15.94:0/3629028016' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi094.yaethu/key"}]: dispatch 2024-03-24T10:46:37.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:37 smithi096 ceph-mon[38097]: from='mgr.? 172.21.15.94:0/3629028016' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T10:46:37.899 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:37 smithi094 ceph-mon[29446]: Standby manager daemon smithi094.yaethu started 2024-03-24T10:46:37.899 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:37 smithi094 ceph-mon[29446]: from='mgr.? 172.21.15.94:0/3629028016' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi094.yaethu/crt"}]: dispatch 2024-03-24T10:46:37.899 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:37 smithi094 ceph-mon[29446]: from='mgr.? 172.21.15.94:0/3629028016' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T10:46:37.899 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:37 smithi094 ceph-mon[29446]: from='mgr.? 172.21.15.94:0/3629028016' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi094.yaethu/key"}]: dispatch 2024-03-24T10:46:37.899 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:37 smithi094 ceph-mon[29446]: from='mgr.? 172.21.15.94:0/3629028016' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T10:46:38.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:38 smithi094 ceph-mon[29446]: pgmap v7: 65 pgs: 65 active+clean; 1.4 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 971 KiB/s rd, 1.1 MiB/s wr, 241 op/s 2024-03-24T10:46:38.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:38 smithi094 ceph-mon[29446]: mgrmap e24: smithi096.gfonnu(active, since 8s), standbys: smithi094.yaethu 2024-03-24T10:46:38.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:38 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "mgr metadata", "who": "smithi094.yaethu", "id": "smithi094.yaethu"}]: dispatch 2024-03-24T10:46:39.044 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:38 smithi096 ceph-mon[38097]: pgmap v7: 65 pgs: 65 active+clean; 1.4 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 971 KiB/s rd, 1.1 MiB/s wr, 241 op/s 2024-03-24T10:46:39.044 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:38 smithi096 ceph-mon[38097]: mgrmap e24: smithi096.gfonnu(active, since 8s), standbys: smithi094.yaethu 2024-03-24T10:46:39.045 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:38 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "mgr metadata", "who": "smithi094.yaethu", "id": "smithi094.yaethu"}]: dispatch 2024-03-24T10:46:41.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:40 smithi094 ceph-mon[29446]: pgmap v8: 65 pgs: 65 active+clean; 1.4 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 755 KiB/s rd, 886 KiB/s wr, 187 op/s 2024-03-24T10:46:41.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:40 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:41.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:40 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:41.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:40 smithi096 ceph-mon[38097]: pgmap v8: 65 pgs: 65 active+clean; 1.4 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 755 KiB/s rd, 886 KiB/s wr, 187 op/s 2024-03-24T10:46:41.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:40 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:41.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:40 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:42.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:42 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:42.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:42 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:42.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:42 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:42.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:42 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:42.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:42 smithi096 ceph-mon[38097]: pgmap v9: 65 pgs: 65 active+clean; 823 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 293 op/s 2024-03-24T10:46:42.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:42 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:42.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:42 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:42.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:42 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:42.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:42 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:42.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:42 smithi094 ceph-mon[29446]: pgmap v9: 65 pgs: 65 active+clean; 823 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 293 op/s 2024-03-24T10:46:43.880 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:43 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:43.881 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:43 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:43.881 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:43 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T10:46:43.881 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:43 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T10:46:43.881 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:43 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T10:46:43.881 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:43 smithi096 ceph-mon[38097]: Adjusting osd_memory_target on smithi094 to 1557M 2024-03-24T10:46:43.881 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:43 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:43.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:43 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:43.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:43 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:43.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:43 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T10:46:43.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:43 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T10:46:43.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:43 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T10:46:43.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:43 smithi094 ceph-mon[29446]: Adjusting osd_memory_target on smithi094 to 1557M 2024-03-24T10:46:43.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:43 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:44.901 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:44 smithi096 ceph-mon[38097]: pgmap v10: 65 pgs: 65 active+clean; 823 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 268 op/s 2024-03-24T10:46:44.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:44 smithi094 ceph-mon[29446]: pgmap v10: 65 pgs: 65 active+clean; 823 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 268 op/s 2024-03-24T10:46:45.626 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-24T10:46:45.627 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-03-24T10:46:45.697 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:45 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:46:45.909 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:45 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:46:45.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:45.909+0000 7f1d23ef5700 1 -- 172.21.15.94:0/3302933763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d1c0662e0 msgr2=0x7f1d1c0ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:45.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:45.909+0000 7f1d23ef5700 1 --2- 172.21.15.94:0/3302933763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d1c0662e0 0x7f1d1c0ff9c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1d18004940 tx=0x7f1d18009e30 comp rx=0 tx=0).stop 2024-03-24T10:46:45.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:45.909+0000 7f1d23ef5700 1 -- 172.21.15.94:0/3302933763 shutdown_connections 2024-03-24T10:46:45.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:45.909+0000 7f1d23ef5700 1 --2- 172.21.15.94:0/3302933763 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d1c0fff00 0x7f1d1c102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:45.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:45.909+0000 7f1d23ef5700 1 --2- 172.21.15.94:0/3302933763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d1c0662e0 0x7f1d1c0ff9c0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:45.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:45.909+0000 7f1d23ef5700 1 -- 172.21.15.94:0/3302933763 >> 172.21.15.94:0/3302933763 conn(0x7f1d1c0f7430 msgr2=0x7f1d1c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:46:45.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:45.909+0000 7f1d23ef5700 1 -- 172.21.15.94:0/3302933763 shutdown_connections 2024-03-24T10:46:45.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:45.909+0000 7f1d23ef5700 1 -- 172.21.15.94:0/3302933763 wait complete. 2024-03-24T10:46:45.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:45.910+0000 7f1d23ef5700 1 Processor -- start 2024-03-24T10:46:45.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:45.910+0000 7f1d23ef5700 1 -- start start 2024-03-24T10:46:45.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:45.910+0000 7f1d23ef5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d1c0662e0 0x7f1d1c10da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:46:45.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:45.910+0000 7f1d23ef5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d1c0fff00 0x7f1d1c10df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:46:45.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:45.910+0000 7f1d23ef5700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1c10e570 con 0x7f1d1c0662e0 2024-03-24T10:46:45.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:45.910+0000 7f1d23ef5700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1c10edc0 con 0x7f1d1c0fff00 2024-03-24T10:46:45.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:45.910+0000 7f1d21c91700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d1c0662e0 0x7f1d1c10da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:46:45.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:45.910+0000 7f1d21c91700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d1c0662e0 0x7f1d1c10da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37168/0 (socket says 172.21.15.94:37168) 2024-03-24T10:46:45.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:45.910+0000 7f1d21c91700 1 -- 172.21.15.94:0/243105137 learned_addr learned my addr 172.21.15.94:0/243105137 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:46:45.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:45.910+0000 7f1d21490700 1 --2- 172.21.15.94:0/243105137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d1c0fff00 0x7f1d1c10df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:46:45.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:45.910+0000 7f1d21c91700 1 -- 172.21.15.94:0/243105137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d1c0fff00 msgr2=0x7f1d1c10df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:45.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:45.910+0000 7f1d21c91700 1 --2- 172.21.15.94:0/243105137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d1c0fff00 0x7f1d1c10df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:45.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:45.910+0000 7f1d21c91700 1 -- 172.21.15.94:0/243105137 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d18005040 con 0x7f1d1c0662e0 2024-03-24T10:46:45.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:45.910+0000 7f1d21c91700 1 --2- 172.21.15.94:0/243105137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d1c0662e0 0x7f1d1c10da30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1d1800b3b0 tx=0x7f1d18009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:46:45.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:45.911+0000 7f1d127fc700 1 -- 172.21.15.94:0/243105137 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d1800e040 con 0x7f1d1c0662e0 2024-03-24T10:46:45.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:45.911+0000 7f1d23ef5700 1 -- 172.21.15.94:0/243105137 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d1c10f040 con 0x7f1d1c0662e0 2024-03-24T10:46:45.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:45.911+0000 7f1d127fc700 1 -- 172.21.15.94:0/243105137 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1d1800bbf0 con 0x7f1d1c0662e0 2024-03-24T10:46:45.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:45.911+0000 7f1d127fc700 1 -- 172.21.15.94:0/243105137 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d1801c630 con 0x7f1d1c0662e0 2024-03-24T10:46:45.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:45.911+0000 7f1d23ef5700 1 -- 172.21.15.94:0/243105137 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d1c10f580 con 0x7f1d1c0662e0 2024-03-24T10:46:45.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:45.912+0000 7f1d127fc700 1 -- 172.21.15.94:0/243105137 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) v1 ==== 89012+0+0 (secure 0 0 0) 0x7f1d180123c0 con 0x7f1d1c0662e0 2024-03-24T10:46:45.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:45.912+0000 7f1d23ef5700 1 -- 172.21.15.94:0/243105137 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d1c05f5f0 con 0x7f1d1c0662e0 2024-03-24T10:46:45.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:45.913+0000 7f1d127fc700 1 --2- 172.21.15.94:0/243105137 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f1d0806b2e0 0x7f1d0806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:46:45.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:45.913+0000 7f1d127fc700 1 -- 172.21.15.94:0/243105137 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f1d1808a7d0 con 0x7f1d1c0662e0 2024-03-24T10:46:45.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:45.913+0000 7f1d21490700 1 --2- 172.21.15.94:0/243105137 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f1d0806b2e0 0x7f1d0806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:46:45.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:45.913+0000 7f1d21490700 1 --2- 172.21.15.94:0/243105137 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f1d0806b2e0 0x7f1d0806d790 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c004f40 tx=0x7f1d0c00b040 comp rx=0 tx=0).ready entity=mgr.24397 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:46:45.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:45.918+0000 7f1d127fc700 1 -- 172.21.15.94:0/243105137 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7f1d180583a0 con 0x7f1d1c0662e0 2024-03-24T10:46:46.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.140+0000 7f1d23ef5700 1 -- 172.21.15.94:0/243105137 --> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1d1c10f910 con 0x7f1d0806b2e0 2024-03-24T10:46:46.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.142+0000 7f1d127fc700 1 -- 172.21.15.94:0/243105137 <== mgr.24397 v2:172.21.15.96:6828/1632816545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f1d1c10f910 con 0x7f1d0806b2e0 2024-03-24T10:46:46.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.144+0000 7f1d23ef5700 1 -- 172.21.15.94:0/243105137 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f1d0806b2e0 msgr2=0x7f1d0806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:46.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.144+0000 7f1d23ef5700 1 --2- 172.21.15.94:0/243105137 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f1d0806b2e0 0x7f1d0806d790 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c004f40 tx=0x7f1d0c00b040 comp rx=0 tx=0).stop 2024-03-24T10:46:46.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.144+0000 7f1d23ef5700 1 -- 172.21.15.94:0/243105137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d1c0662e0 msgr2=0x7f1d1c10da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:46.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.144+0000 7f1d23ef5700 1 --2- 172.21.15.94:0/243105137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d1c0662e0 0x7f1d1c10da30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1d1800b3b0 tx=0x7f1d18009e30 comp rx=0 tx=0).stop 2024-03-24T10:46:46.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.145+0000 7f1d23ef5700 1 -- 172.21.15.94:0/243105137 shutdown_connections 2024-03-24T10:46:46.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.145+0000 7f1d23ef5700 1 --2- 172.21.15.94:0/243105137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d1c0662e0 0x7f1d1c10da30 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:46.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.145+0000 7f1d23ef5700 1 --2- 172.21.15.94:0/243105137 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f1d0806b2e0 0x7f1d0806d790 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:46.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.145+0000 7f1d23ef5700 1 --2- 172.21.15.94:0/243105137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d1c0fff00 0x7f1d1c10df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:46.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.145+0000 7f1d23ef5700 1 -- 172.21.15.94:0/243105137 >> 172.21.15.94:0/243105137 conn(0x7f1d1c0f7430 msgr2=0x7f1d1c0f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:46:46.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.145+0000 7f1d23ef5700 1 -- 172.21.15.94:0/243105137 shutdown_connections 2024-03-24T10:46:46.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.145+0000 7f1d23ef5700 1 -- 172.21.15.94:0/243105137 wait complete. 2024-03-24T10:46:46.155 INFO:teuthology.orchestra.run.smithi094.stdout:true 2024-03-24T10:46:46.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.281+0000 7f6a475dd700 1 -- 172.21.15.94:0/2146438146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a400fbcd0 msgr2=0x7f6a400fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:46.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.281+0000 7f6a475dd700 1 --2- 172.21.15.94:0/2146438146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a400fbcd0 0x7f6a400fc0e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c004e90 tx=0x7f6a3c004990 comp rx=0 tx=0).stop 2024-03-24T10:46:46.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.282+0000 7f6a475dd700 1 -- 172.21.15.94:0/2146438146 shutdown_connections 2024-03-24T10:46:46.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.282+0000 7f6a475dd700 1 --2- 172.21.15.94:0/2146438146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a400fcf80 0x7f6a400fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:46.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.282+0000 7f6a475dd700 1 --2- 172.21.15.94:0/2146438146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a400fbcd0 0x7f6a400fc0e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:46.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.282+0000 7f6a475dd700 1 -- 172.21.15.94:0/2146438146 >> 172.21.15.94:0/2146438146 conn(0x7f6a400f7430 msgr2=0x7f6a400f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:46:46.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.282+0000 7f6a475dd700 1 -- 172.21.15.94:0/2146438146 shutdown_connections 2024-03-24T10:46:46.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.282+0000 7f6a475dd700 1 -- 172.21.15.94:0/2146438146 wait complete. 2024-03-24T10:46:46.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.282+0000 7f6a475dd700 1 Processor -- start 2024-03-24T10:46:46.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.283+0000 7f6a475dd700 1 -- start start 2024-03-24T10:46:46.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.283+0000 7f6a475dd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a400fbcd0 0x7f6a4006e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:46:46.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.283+0000 7f6a475dd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a400fcf80 0x7f6a4006e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:46:46.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.283+0000 7f6a45379700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a400fbcd0 0x7f6a4006e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:46:46.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.283+0000 7f6a45379700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a400fbcd0 0x7f6a4006e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.94:59014/0 (socket says 172.21.15.94:59014) 2024-03-24T10:46:46.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.283+0000 7f6a45379700 1 -- 172.21.15.94:0/792054530 learned_addr learned my addr 172.21.15.94:0/792054530 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:46:46.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.283+0000 7f6a475dd700 1 -- 172.21.15.94:0/792054530 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a40070050 con 0x7f6a400fcf80 2024-03-24T10:46:46.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.283+0000 7f6a475dd700 1 -- 172.21.15.94:0/792054530 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4006ee00 con 0x7f6a400fbcd0 2024-03-24T10:46:46.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.284+0000 7f6a44b78700 1 --2- 172.21.15.94:0/792054530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a400fcf80 0x7f6a4006e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:46:46.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.284+0000 7f6a45379700 1 -- 172.21.15.94:0/792054530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a400fcf80 msgr2=0x7f6a4006e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:46.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.284+0000 7f6a45379700 1 --2- 172.21.15.94:0/792054530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a400fcf80 0x7f6a4006e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:46.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.284+0000 7f6a45379700 1 -- 172.21.15.94:0/792054530 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a3c005040 con 0x7f6a400fbcd0 2024-03-24T10:46:46.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.284+0000 7f6a44b78700 1 --2- 172.21.15.94:0/792054530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a400fcf80 0x7f6a4006e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T10:46:46.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.284+0000 7f6a45379700 1 --2- 172.21.15.94:0/792054530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a400fbcd0 0x7f6a4006e380 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c016500 tx=0x7f6a3c004990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:46:46.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.285+0000 7f6a35ffb700 1 -- 172.21.15.94:0/792054530 <== mon.1 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a3c012050 con 0x7f6a400fbcd0 2024-03-24T10:46:46.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.285+0000 7f6a35ffb700 1 -- 172.21.15.94:0/792054530 <== mon.1 v2:172.21.15.96:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6a3c004290 con 0x7f6a400fbcd0 2024-03-24T10:46:46.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.285+0000 7f6a475dd700 1 -- 172.21.15.94:0/792054530 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a4006f020 con 0x7f6a400fbcd0 2024-03-24T10:46:46.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.285+0000 7f6a35ffb700 1 -- 172.21.15.94:0/792054530 <== mon.1 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a3c0219f0 con 0x7f6a400fbcd0 2024-03-24T10:46:46.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.285+0000 7f6a475dd700 1 -- 172.21.15.94:0/792054530 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a40122570 con 0x7f6a400fbcd0 2024-03-24T10:46:46.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.287+0000 7f6a475dd700 1 -- 172.21.15.94:0/792054530 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a4005f5f0 con 0x7f6a400fbcd0 2024-03-24T10:46:46.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.289+0000 7f6a35ffb700 1 -- 172.21.15.94:0/792054530 <== mon.1 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 24) v1 ==== 89012+0+0 (secure 0 0 0) 0x7f6a3c021c20 con 0x7f6a400fbcd0 2024-03-24T10:46:46.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.289+0000 7f6a35ffb700 1 --2- 172.21.15.94:0/792054530 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f6a2c06b330 0x7f6a2c06d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:46:46.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.290+0000 7f6a35ffb700 1 -- 172.21.15.94:0/792054530 <== mon.1 v2:172.21.15.96:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f6a3c08b150 con 0x7f6a400fbcd0 2024-03-24T10:46:46.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.291+0000 7f6a44b78700 1 --2- 172.21.15.94:0/792054530 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f6a2c06b330 0x7f6a2c06d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:46:46.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.292+0000 7f6a44b78700 1 --2- 172.21.15.94:0/792054530 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f6a2c06b330 0x7f6a2c06d7e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6a30009b10 tx=0x7f6a30008040 comp rx=0 tx=0).ready entity=mgr.24397 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:46:46.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.299+0000 7f6a35ffb700 1 -- 172.21.15.94:0/792054530 <== mon.1 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7f6a3c055180 con 0x7f6a400fbcd0 2024-03-24T10:46:46.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.582+0000 7f6a475dd700 1 -- 172.21.15.94:0/792054530 --> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a401012a0 con 0x7f6a2c06b330 2024-03-24T10:46:46.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.584+0000 7f6a35ffb700 1 -- 172.21.15.94:0/792054530 <== mgr.24397 v2:172.21.15.96:6828/1632816545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f6a401012a0 con 0x7f6a2c06b330 2024-03-24T10:46:46.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.586+0000 7f6a475dd700 1 -- 172.21.15.94:0/792054530 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f6a2c06b330 msgr2=0x7f6a2c06d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:46.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.586+0000 7f6a475dd700 1 --2- 172.21.15.94:0/792054530 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f6a2c06b330 0x7f6a2c06d7e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6a30009b10 tx=0x7f6a30008040 comp rx=0 tx=0).stop 2024-03-24T10:46:46.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.586+0000 7f6a475dd700 1 -- 172.21.15.94:0/792054530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a400fbcd0 msgr2=0x7f6a4006e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:46.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.586+0000 7f6a475dd700 1 --2- 172.21.15.94:0/792054530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a400fbcd0 0x7f6a4006e380 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c016500 tx=0x7f6a3c004990 comp rx=0 tx=0).stop 2024-03-24T10:46:46.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.586+0000 7f6a475dd700 1 -- 172.21.15.94:0/792054530 shutdown_connections 2024-03-24T10:46:46.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.586+0000 7f6a475dd700 1 --2- 172.21.15.94:0/792054530 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f6a2c06b330 0x7f6a2c06d7e0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:46.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.586+0000 7f6a475dd700 1 --2- 172.21.15.94:0/792054530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a400fbcd0 0x7f6a4006e380 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:46.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.586+0000 7f6a475dd700 1 --2- 172.21.15.94:0/792054530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a400fcf80 0x7f6a4006e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:46.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.586+0000 7f6a475dd700 1 -- 172.21.15.94:0/792054530 >> 172.21.15.94:0/792054530 conn(0x7f6a400f7430 msgr2=0x7f6a400f8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:46:46.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.587+0000 7f6a475dd700 1 -- 172.21.15.94:0/792054530 shutdown_connections 2024-03-24T10:46:46.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.587+0000 7f6a475dd700 1 -- 172.21.15.94:0/792054530 wait complete. 2024-03-24T10:46:46.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.717+0000 7f67a9f1a700 1 -- 172.21.15.94:0/581390277 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67a40fd9c0 msgr2=0x7f67a40ffda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:46.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.717+0000 7f67a9f1a700 1 --2- 172.21.15.94:0/581390277 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67a40fd9c0 0x7f67a40ffda0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6798004940 tx=0x7f6798009e30 comp rx=0 tx=0).stop 2024-03-24T10:46:46.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.717+0000 7f67a9f1a700 1 -- 172.21.15.94:0/581390277 shutdown_connections 2024-03-24T10:46:46.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.717+0000 7f67a9f1a700 1 --2- 172.21.15.94:0/581390277 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67a41002e0 0x7f67a41026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:46.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.717+0000 7f67a9f1a700 1 --2- 172.21.15.94:0/581390277 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67a40fd9c0 0x7f67a40ffda0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:46.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.717+0000 7f67a9f1a700 1 -- 172.21.15.94:0/581390277 >> 172.21.15.94:0/581390277 conn(0x7f67a40f7480 msgr2=0x7f67a40f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:46:46.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.717+0000 7f67a9f1a700 1 -- 172.21.15.94:0/581390277 shutdown_connections 2024-03-24T10:46:46.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.717+0000 7f67a9f1a700 1 -- 172.21.15.94:0/581390277 wait complete. 2024-03-24T10:46:46.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.717+0000 7f67a9f1a700 1 Processor -- start 2024-03-24T10:46:46.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.718+0000 7f67a9f1a700 1 -- start start 2024-03-24T10:46:46.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.718+0000 7f67a9f1a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67a40fd9c0 0x7f67a40fd400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:46:46.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.718+0000 7f67a9f1a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67a41002e0 0x7f67a40fb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:46:46.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.718+0000 7f67a9f1a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a40fbe80 con 0x7f67a40fd9c0 2024-03-24T10:46:46.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.718+0000 7f67a9f1a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a40fbff0 con 0x7f67a41002e0 2024-03-24T10:46:46.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.718+0000 7f67a37fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67a40fd9c0 0x7f67a40fd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:46:46.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.718+0000 7f67a37fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67a40fd9c0 0x7f67a40fd400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37210/0 (socket says 172.21.15.94:37210) 2024-03-24T10:46:46.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.718+0000 7f67a37fe700 1 -- 172.21.15.94:0/594519043 learned_addr learned my addr 172.21.15.94:0/594519043 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:46:46.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.718+0000 7f67a2ffd700 1 --2- 172.21.15.94:0/594519043 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67a41002e0 0x7f67a40fb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:46:46.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.718+0000 7f67a37fe700 1 -- 172.21.15.94:0/594519043 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67a41002e0 msgr2=0x7f67a40fb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:46.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.718+0000 7f67a37fe700 1 --2- 172.21.15.94:0/594519043 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67a41002e0 0x7f67a40fb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:46.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.718+0000 7f67a37fe700 1 -- 172.21.15.94:0/594519043 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6798005040 con 0x7f67a40fd9c0 2024-03-24T10:46:46.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.719+0000 7f67a37fe700 1 --2- 172.21.15.94:0/594519043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67a40fd9c0 0x7f67a40fd400 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6798000c00 tx=0x7f6798009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:46:46.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.719+0000 7f678bfff700 1 -- 172.21.15.94:0/594519043 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f679800e040 con 0x7f67a40fd9c0 2024-03-24T10:46:46.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.719+0000 7f67a9f1a700 1 -- 172.21.15.94:0/594519043 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67a40fc270 con 0x7f67a40fd9c0 2024-03-24T10:46:46.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.719+0000 7f678bfff700 1 -- 172.21.15.94:0/594519043 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f679800bbf0 con 0x7f67a40fd9c0 2024-03-24T10:46:46.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.719+0000 7f678bfff700 1 -- 172.21.15.94:0/594519043 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f679801c630 con 0x7f67a40fd9c0 2024-03-24T10:46:46.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.719+0000 7f67a9f1a700 1 -- 172.21.15.94:0/594519043 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67a4115a10 con 0x7f67a40fd9c0 2024-03-24T10:46:46.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.720+0000 7f67a9f1a700 1 -- 172.21.15.94:0/594519043 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67a405f5f0 con 0x7f67a40fd9c0 2024-03-24T10:46:46.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.721+0000 7f678bfff700 1 -- 172.21.15.94:0/594519043 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) v1 ==== 89012+0+0 (secure 0 0 0) 0x7f679800bd60 con 0x7f67a40fd9c0 2024-03-24T10:46:46.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.721+0000 7f678bfff700 1 --2- 172.21.15.94:0/594519043 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f678c06b2e0 0x7f678c06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:46:46.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.721+0000 7f678bfff700 1 -- 172.21.15.94:0/594519043 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f679808b0c0 con 0x7f67a40fd9c0 2024-03-24T10:46:46.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.721+0000 7f67a2ffd700 1 --2- 172.21.15.94:0/594519043 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f678c06b2e0 0x7f678c06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:46:46.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.722+0000 7f67a2ffd700 1 --2- 172.21.15.94:0/594519043 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f678c06b2e0 0x7f678c06d790 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6794009fd0 tx=0x7f679400b040 comp rx=0 tx=0).ready entity=mgr.24397 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:46:46.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.726+0000 7f678bfff700 1 -- 172.21.15.94:0/594519043 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7f67980550d0 con 0x7f67a40fd9c0 2024-03-24T10:46:46.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.947+0000 7f67a9f1a700 1 -- 172.21.15.94:0/594519043 --> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f67a40f8f70 con 0x7f678c06b2e0 2024-03-24T10:46:46.947 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:46 smithi094 ceph-mon[29446]: pgmap v11: 65 pgs: 65 active+clean; 704 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 309 op/s 2024-03-24T10:46:46.948 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:46 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:46.948 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:46 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:46.948 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:46 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T10:46:46.948 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:46 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T10:46:46.948 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:46 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T10:46:46.948 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:46 smithi094 ceph-mon[29446]: Adjusting osd_memory_target on smithi096 to 2581M 2024-03-24T10:46:46.948 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:46 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:46.948 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:46 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:46:46.948 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:46 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:46:46.948 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:46 smithi094 ceph-mon[29446]: Updating smithi094:/etc/ceph/ceph.conf 2024-03-24T10:46:46.948 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:46 smithi094 ceph-mon[29446]: Updating smithi096:/etc/ceph/ceph.conf 2024-03-24T10:46:46.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.967+0000 7f678bfff700 1 -- 172.21.15.94:0/594519043 <== mgr.24397 v2:172.21.15.96:6828/1632816545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f67a40f8f70 con 0x7f678c06b2e0 2024-03-24T10:46:46.967 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T10:46:46.967 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.smithi094 smithi094 *:9093,9094 running (7m) 5s ago 8m 25.2M - 0.25.0 c8568f914cd2 8072d4b65986 2024-03-24T10:46:46.967 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi094 smithi094 running (9m) 5s ago 9m 6660k - 17.2.7-726-g9c4e0c34 837124da804f fb5772af46c5 2024-03-24T10:46:46.967 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi096 smithi096 running (7m) 7s ago 7m 6669k - 17.2.7-726-g9c4e0c34 837124da804f 6087ad4e0734 2024-03-24T10:46:46.967 INFO:teuthology.orchestra.run.smithi094.stdout:grafana.smithi094 smithi094 *:3000 running (6m) 5s ago 8m 83.1M - 9.4.7 954c08fa6188 7b534968fb36 2024-03-24T10:46:46.967 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.efiqte smithi094 running (3m) 5s ago 3m 11.7M - 17.2.7-726-g9c4e0c34 837124da804f f0df0a056c21 2024-03-24T10:46:46.968 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.llefgu smithi094 running (2m) 5s ago 2m 149M - 17.2.7-726-g9c4e0c34 837124da804f 52a9771dc8e0 2024-03-24T10:46:46.968 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi096.egbmoq smithi096 running (3m) 7s ago 3m 226M - 17.2.7-726-g9c4e0c34 837124da804f 1cfc930dbab0 2024-03-24T10:46:46.968 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi096.jtebhh smithi096 running (2m) 7s ago 2m 14.8M - 17.2.7-726-g9c4e0c34 837124da804f 4ff3451a6f6e 2024-03-24T10:46:46.968 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi094.yaethu smithi094 *:9283 running (10m) 5s ago 10m 428M - 17.2.7-726-g9c4e0c34 837124da804f 327d527697e0 2024-03-24T10:46:46.968 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi096.gfonnu smithi096 *:8443,9283 running (43s) 7s ago 7m 488M - 19.0.0-1855-g21b3bd67 553463a90fd5 b6483bc88c49 2024-03-24T10:46:46.968 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi094 smithi094 running (10m) 5s ago 10m 45.9M 2048M 17.2.7-726-g9c4e0c34 837124da804f 76584fceb32b 2024-03-24T10:46:46.968 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi096 smithi096 running (7m) 7s ago 7m 39.0M 2048M 17.2.7-726-g9c4e0c34 837124da804f 710030b222cb 2024-03-24T10:46:46.968 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi094 smithi094 *:9100 running (9m) 5s ago 9m 15.9M - 1.5.0 0da6a335fe13 59895363f7e6 2024-03-24T10:46:46.968 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi096 smithi096 *:9100 running (7m) 7s ago 7m 15.6M - 1.5.0 0da6a335fe13 fc629282835f 2024-03-24T10:46:46.968 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (6m) 5s ago 6m 224M 1557M 17.2.7-726-g9c4e0c34 837124da804f b664fd3557bd 2024-03-24T10:46:46.968 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi094 running (6m) 5s ago 6m 226M 1557M 17.2.7-726-g9c4e0c34 837124da804f 4e737a80d733 2024-03-24T10:46:46.968 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi094 running (5m) 5s ago 5m 188M 1557M 17.2.7-726-g9c4e0c34 837124da804f cf0aefd20c76 2024-03-24T10:46:46.968 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi096 running (5m) 7s ago 5m 248M 2581M 17.2.7-726-g9c4e0c34 837124da804f f4650d0f6241 2024-03-24T10:46:46.968 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi096 running (4m) 7s ago 4m 216M 2581M 17.2.7-726-g9c4e0c34 837124da804f a63d7c9e705f 2024-03-24T10:46:46.969 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi096 running (4m) 7s ago 4m 240M 2581M 17.2.7-726-g9c4e0c34 837124da804f f56fad2959c3 2024-03-24T10:46:46.969 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.smithi094 smithi094 *:9095 running (6m) 5s ago 7m 59.2M - 2.43.0 a07b618ecd1d 023eb0a49108 2024-03-24T10:46:46.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.969+0000 7f67a9f1a700 1 -- 172.21.15.94:0/594519043 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f678c06b2e0 msgr2=0x7f678c06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:46.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.969+0000 7f67a9f1a700 1 --2- 172.21.15.94:0/594519043 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f678c06b2e0 0x7f678c06d790 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6794009fd0 tx=0x7f679400b040 comp rx=0 tx=0).stop 2024-03-24T10:46:46.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.969+0000 7f67a9f1a700 1 -- 172.21.15.94:0/594519043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67a40fd9c0 msgr2=0x7f67a40fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:46.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.969+0000 7f67a9f1a700 1 --2- 172.21.15.94:0/594519043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67a40fd9c0 0x7f67a40fd400 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6798000c00 tx=0x7f6798009e30 comp rx=0 tx=0).stop 2024-03-24T10:46:46.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.969+0000 7f67a9f1a700 1 -- 172.21.15.94:0/594519043 shutdown_connections 2024-03-24T10:46:46.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.969+0000 7f67a9f1a700 1 --2- 172.21.15.94:0/594519043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67a40fd9c0 0x7f67a40fd400 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:46.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.969+0000 7f67a9f1a700 1 --2- 172.21.15.94:0/594519043 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f678c06b2e0 0x7f678c06d790 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:46.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.969+0000 7f67a9f1a700 1 --2- 172.21.15.94:0/594519043 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67a41002e0 0x7f67a40fb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:46.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.969+0000 7f67a9f1a700 1 -- 172.21.15.94:0/594519043 >> 172.21.15.94:0/594519043 conn(0x7f67a40f7480 msgr2=0x7f67a40f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:46:46.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.970+0000 7f67a9f1a700 1 -- 172.21.15.94:0/594519043 shutdown_connections 2024-03-24T10:46:46.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:46.970+0000 7f67a9f1a700 1 -- 172.21.15.94:0/594519043 wait complete. 2024-03-24T10:46:47.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.074+0000 7fd094d5c700 1 -- 172.21.15.94:0/2489510975 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0900fd9c0 msgr2=0x7fd0900ffda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:47.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.074+0000 7fd094d5c700 1 --2- 172.21.15.94:0/2489510975 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0900fd9c0 0x7fd0900ffda0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd080004940 tx=0x7fd080009e30 comp rx=0 tx=0).stop 2024-03-24T10:46:47.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.074+0000 7fd094d5c700 1 -- 172.21.15.94:0/2489510975 shutdown_connections 2024-03-24T10:46:47.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.074+0000 7fd094d5c700 1 --2- 172.21.15.94:0/2489510975 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0901002e0 0x7fd0901026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:47.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.074+0000 7fd094d5c700 1 --2- 172.21.15.94:0/2489510975 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0900fd9c0 0x7fd0900ffda0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:47.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.074+0000 7fd094d5c700 1 -- 172.21.15.94:0/2489510975 >> 172.21.15.94:0/2489510975 conn(0x7fd0900f7480 msgr2=0x7fd0900f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:46:47.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.075+0000 7fd094d5c700 1 -- 172.21.15.94:0/2489510975 shutdown_connections 2024-03-24T10:46:47.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.075+0000 7fd094d5c700 1 -- 172.21.15.94:0/2489510975 wait complete. 2024-03-24T10:46:47.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.075+0000 7fd094d5c700 1 Processor -- start 2024-03-24T10:46:47.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.075+0000 7fd094d5c700 1 -- start start 2024-03-24T10:46:47.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.075+0000 7fd094d5c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0900fd9c0 0x7fd0900fd400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:46:47.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.075+0000 7fd094d5c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0901002e0 0x7fd0900fb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:46:47.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.075+0000 7fd094d5c700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0900fbe80 con 0x7fd0900fd9c0 2024-03-24T10:46:47.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.075+0000 7fd094d5c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0900fbff0 con 0x7fd0901002e0 2024-03-24T10:46:47.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.075+0000 7fd08e59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0900fd9c0 0x7fd0900fd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:46:47.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.075+0000 7fd08e59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0900fd9c0 0x7fd0900fd400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37236/0 (socket says 172.21.15.94:37236) 2024-03-24T10:46:47.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.075+0000 7fd08e59c700 1 -- 172.21.15.94:0/3901748197 learned_addr learned my addr 172.21.15.94:0/3901748197 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:46:47.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.075+0000 7fd08dd9b700 1 --2- 172.21.15.94:0/3901748197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0901002e0 0x7fd0900fb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:46:47.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.076+0000 7fd08e59c700 1 -- 172.21.15.94:0/3901748197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0901002e0 msgr2=0x7fd0900fb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:47.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.076+0000 7fd08e59c700 1 --2- 172.21.15.94:0/3901748197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0901002e0 0x7fd0900fb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:47.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.076+0000 7fd08e59c700 1 -- 172.21.15.94:0/3901748197 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd080005040 con 0x7fd0900fd9c0 2024-03-24T10:46:47.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.076+0000 7fd08e59c700 1 --2- 172.21.15.94:0/3901748197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0900fd9c0 0x7fd0900fd400 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd08000b970 tx=0x7fd080009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:46:47.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.076+0000 7fd07effd700 1 -- 172.21.15.94:0/3901748197 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd08000e040 con 0x7fd0900fd9c0 2024-03-24T10:46:47.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.076+0000 7fd07effd700 1 -- 172.21.15.94:0/3901748197 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd080004290 con 0x7fd0900fd9c0 2024-03-24T10:46:47.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.076+0000 7fd094d5c700 1 -- 172.21.15.94:0/3901748197 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0900fc270 con 0x7fd0900fd9c0 2024-03-24T10:46:47.078 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:46 smithi096 ceph-mon[38097]: pgmap v11: 65 pgs: 65 active+clean; 704 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 309 op/s 2024-03-24T10:46:47.078 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:46 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:47.078 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:46 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:47.078 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:46 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T10:46:47.078 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:46 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T10:46:47.078 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:46 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T10:46:47.079 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:46 smithi096 ceph-mon[38097]: Adjusting osd_memory_target on smithi096 to 2581M 2024-03-24T10:46:47.079 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:46 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:47.079 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:46 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:46:47.079 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:46 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:46:47.079 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:46 smithi096 ceph-mon[38097]: Updating smithi094:/etc/ceph/ceph.conf 2024-03-24T10:46:47.079 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:46 smithi096 ceph-mon[38097]: Updating smithi096:/etc/ceph/ceph.conf 2024-03-24T10:46:47.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.076+0000 7fd07effd700 1 -- 172.21.15.94:0/3901748197 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd08001c3f0 con 0x7fd0900fd9c0 2024-03-24T10:46:47.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.076+0000 7fd094d5c700 1 -- 172.21.15.94:0/3901748197 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd090115a10 con 0x7fd0900fd9c0 2024-03-24T10:46:47.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.077+0000 7fd094d5c700 1 -- 172.21.15.94:0/3901748197 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd09005f5f0 con 0x7fd0900fd9c0 2024-03-24T10:46:47.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.077+0000 7fd07effd700 1 -- 172.21.15.94:0/3901748197 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) v1 ==== 89012+0+0 (secure 0 0 0) 0x7fd08000bbf0 con 0x7fd0900fd9c0 2024-03-24T10:46:47.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.078+0000 7fd07effd700 1 --2- 172.21.15.94:0/3901748197 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7fd07806b0d0 0x7fd07806d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:46:47.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.078+0000 7fd07effd700 1 -- 172.21.15.94:0/3901748197 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fd08008ad10 con 0x7fd0900fd9c0 2024-03-24T10:46:47.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.078+0000 7fd08dd9b700 1 --2- 172.21.15.94:0/3901748197 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7fd07806b0d0 0x7fd07806d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:46:47.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.078+0000 7fd08dd9b700 1 --2- 172.21.15.94:0/3901748197 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7fd07806b0d0 0x7fd07806d580 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd084009fd0 tx=0x7fd08400b040 comp rx=0 tx=0).ready entity=mgr.24397 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:46:47.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.082+0000 7fd07effd700 1 -- 172.21.15.94:0/3901748197 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7fd080054de0 con 0x7fd0900fd9c0 2024-03-24T10:46:47.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.382+0000 7fd094d5c700 1 -- 172.21.15.94:0/3901748197 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd090105970 con 0x7fd0900fd9c0 2024-03-24T10:46:47.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.383+0000 7fd07effd700 1 -- 172.21.15.94:0/3901748197 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7fd08000bea0 con 0x7fd0900fd9c0 2024-03-24T10:46:47.383 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-03-24T10:46:47.383 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": { 2024-03-24T10:46:47.383 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-24T10:46:47.383 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:46:47.383 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": { 2024-03-24T10:46:47.383 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 1, 2024-03-24T10:46:47.383 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 1 2024-03-24T10:46:47.383 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:46:47.383 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": { 2024-03-24T10:46:47.383 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-24T10:46:47.383 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:46:47.383 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-03-24T10:46:47.383 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-24T10:46:47.383 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:46:47.383 INFO:teuthology.orchestra.run.smithi094.stdout: "overall": { 2024-03-24T10:46:47.383 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 13, 2024-03-24T10:46:47.384 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 1 2024-03-24T10:46:47.384 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-03-24T10:46:47.384 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-03-24T10:46:47.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.385+0000 7fd094d5c700 1 -- 172.21.15.94:0/3901748197 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7fd07806b0d0 msgr2=0x7fd07806d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:47.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.385+0000 7fd094d5c700 1 --2- 172.21.15.94:0/3901748197 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7fd07806b0d0 0x7fd07806d580 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd084009fd0 tx=0x7fd08400b040 comp rx=0 tx=0).stop 2024-03-24T10:46:47.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.385+0000 7fd094d5c700 1 -- 172.21.15.94:0/3901748197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0900fd9c0 msgr2=0x7fd0900fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:47.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.385+0000 7fd094d5c700 1 --2- 172.21.15.94:0/3901748197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0900fd9c0 0x7fd0900fd400 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd08000b970 tx=0x7fd080009e30 comp rx=0 tx=0).stop 2024-03-24T10:46:47.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.385+0000 7fd094d5c700 1 -- 172.21.15.94:0/3901748197 shutdown_connections 2024-03-24T10:46:47.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.385+0000 7fd094d5c700 1 --2- 172.21.15.94:0/3901748197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0900fd9c0 0x7fd0900fd400 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:47.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.385+0000 7fd094d5c700 1 --2- 172.21.15.94:0/3901748197 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7fd07806b0d0 0x7fd07806d580 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:47.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.385+0000 7fd094d5c700 1 --2- 172.21.15.94:0/3901748197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0901002e0 0x7fd0900fb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:47.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.385+0000 7fd094d5c700 1 -- 172.21.15.94:0/3901748197 >> 172.21.15.94:0/3901748197 conn(0x7fd0900f7480 msgr2=0x7fd0900f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:46:47.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.385+0000 7fd094d5c700 1 -- 172.21.15.94:0/3901748197 shutdown_connections 2024-03-24T10:46:47.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.385+0000 7fd094d5c700 1 -- 172.21.15.94:0/3901748197 wait complete. 2024-03-24T10:46:47.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.491+0000 7fb9493a0700 1 -- 172.21.15.94:0/235391456 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9440fcdc0 msgr2=0x7fb9440fd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:47.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.491+0000 7fb9493a0700 1 --2- 172.21.15.94:0/235391456 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9440fcdc0 0x7fb9440fd230 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb938004940 tx=0x7fb938009e30 comp rx=0 tx=0).stop 2024-03-24T10:46:47.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.491+0000 7fb9493a0700 1 -- 172.21.15.94:0/235391456 shutdown_connections 2024-03-24T10:46:47.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.491+0000 7fb9493a0700 1 --2- 172.21.15.94:0/235391456 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9440fcdc0 0x7fb9440fd230 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:47.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.491+0000 7fb9493a0700 1 --2- 172.21.15.94:0/235391456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9440fbb10 0x7fb9440fbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:47.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.491+0000 7fb9493a0700 1 -- 172.21.15.94:0/235391456 >> 172.21.15.94:0/235391456 conn(0x7fb9440f7270 msgr2=0x7fb9440f96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:46:47.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.491+0000 7fb9493a0700 1 -- 172.21.15.94:0/235391456 shutdown_connections 2024-03-24T10:46:47.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.491+0000 7fb9493a0700 1 -- 172.21.15.94:0/235391456 wait complete. 2024-03-24T10:46:47.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.492+0000 7fb9493a0700 1 Processor -- start 2024-03-24T10:46:47.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.492+0000 7fb9493a0700 1 -- start start 2024-03-24T10:46:47.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.492+0000 7fb9493a0700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9440fbb10 0x7fb944110530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:46:47.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.492+0000 7fb9493a0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9440fcdc0 0x7fb944112a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:46:47.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.492+0000 7fb9493a0700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb944066350 con 0x7fb9440fcdc0 2024-03-24T10:46:47.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.492+0000 7fb9493a0700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb944066490 con 0x7fb9440fbb10 2024-03-24T10:46:47.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.492+0000 7fb9437fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9440fcdc0 0x7fb944112a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:46:47.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.492+0000 7fb9437fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9440fcdc0 0x7fb944112a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37254/0 (socket says 172.21.15.94:37254) 2024-03-24T10:46:47.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.492+0000 7fb943fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9440fbb10 0x7fb944110530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:46:47.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.492+0000 7fb9437fe700 1 -- 172.21.15.94:0/279105872 learned_addr learned my addr 172.21.15.94:0/279105872 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:46:47.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.493+0000 7fb9437fe700 1 -- 172.21.15.94:0/279105872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9440fbb10 msgr2=0x7fb944110530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:47.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.493+0000 7fb9437fe700 1 --2- 172.21.15.94:0/279105872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9440fbb10 0x7fb944110530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:47.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.493+0000 7fb9437fe700 1 -- 172.21.15.94:0/279105872 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb938005040 con 0x7fb9440fcdc0 2024-03-24T10:46:47.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.493+0000 7fb943fff700 1 --2- 172.21.15.94:0/279105872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9440fbb10 0x7fb944110530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T10:46:47.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.493+0000 7fb9437fe700 1 --2- 172.21.15.94:0/279105872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9440fcdc0 0x7fb944112a80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb938013870 tx=0x7fb938009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:46:47.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.493+0000 7fb940ff9700 1 -- 172.21.15.94:0/279105872 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb938021070 con 0x7fb9440fcdc0 2024-03-24T10:46:47.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.493+0000 7fb940ff9700 1 -- 172.21.15.94:0/279105872 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb938009430 con 0x7fb9440fcdc0 2024-03-24T10:46:47.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.493+0000 7fb9493a0700 1 -- 172.21.15.94:0/279105872 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb944112fc0 con 0x7fb9440fcdc0 2024-03-24T10:46:47.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.493+0000 7fb940ff9700 1 -- 172.21.15.94:0/279105872 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb93800e070 con 0x7fb9440fcdc0 2024-03-24T10:46:47.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.493+0000 7fb9493a0700 1 -- 172.21.15.94:0/279105872 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9441134d0 con 0x7fb9440fcdc0 2024-03-24T10:46:47.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.494+0000 7fb9493a0700 1 -- 172.21.15.94:0/279105872 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb94405f5f0 con 0x7fb9440fcdc0 2024-03-24T10:46:47.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.495+0000 7fb940ff9700 1 -- 172.21.15.94:0/279105872 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) v1 ==== 89012+0+0 (secure 0 0 0) 0x7fb93801c750 con 0x7fb9440fcdc0 2024-03-24T10:46:47.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.495+0000 7fb940ff9700 1 --2- 172.21.15.94:0/279105872 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7fb92c06b280 0x7fb92c06d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:46:47.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.495+0000 7fb940ff9700 1 -- 172.21.15.94:0/279105872 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fb93808afe0 con 0x7fb9440fcdc0 2024-03-24T10:46:47.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.495+0000 7fb943fff700 1 --2- 172.21.15.94:0/279105872 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7fb92c06b280 0x7fb92c06d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:46:47.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.496+0000 7fb943fff700 1 --2- 172.21.15.94:0/279105872 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7fb92c06b280 0x7fb92c06d730 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb944065fd0 tx=0x7fb93400b040 comp rx=0 tx=0).ready entity=mgr.24397 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:46:47.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.500+0000 7fb940ff9700 1 -- 172.21.15.94:0/279105872 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7fb938055110 con 0x7fb9440fcdc0 2024-03-24T10:46:47.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.845+0000 7fb9493a0700 1 -- 172.21.15.94:0/279105872 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb94404bed0 con 0x7fb9440fcdc0 2024-03-24T10:46:47.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.845+0000 7fb940ff9700 1 -- 172.21.15.94:0/279105872 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1833 (secure 0 0 0) 0x7fb9380587d0 con 0x7fb9440fcdc0 2024-03-24T10:46:47.845 INFO:teuthology.orchestra.run.smithi094.stdout:e18 2024-03-24T10:46:47.845 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T10:46:47.845 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:46:47.846 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-03-24T10:46:47.846 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:46:47.846 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-03-24T10:46:47.846 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-03-24T10:46:47.846 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 18 2024-03-24T10:46:47.846 INFO:teuthology.orchestra.run.smithi094.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-24T10:46:47.846 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-03-24T10:43:39.435738+0000 2024-03-24T10:46:47.846 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-03-24T10:43:58.133901+0000 2024-03-24T10:46:47.846 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-03-24T10:46:47.846 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-03-24T10:46:47.846 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 60 2024-03-24T10:46:47.846 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-03-24T10:46:47.846 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-03-24T10:46:47.846 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-03-24T10:46:47.847 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-03-24T10:46:47.847 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 46 2024-03-24T10:46:47.847 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:46:47.847 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-03-24T10:46:47.847 INFO:teuthology.orchestra.run.smithi094.stdout:in 0 2024-03-24T10:46:47.847 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=24251} 2024-03-24T10:46:47.847 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-03-24T10:46:47.847 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-03-24T10:46:47.847 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-03-24T10:46:47.847 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-03-24T10:46:47.847 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-03-24T10:46:47.847 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-03-24T10:46:47.847 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-03-24T10:46:47.847 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 1 2024-03-24T10:46:47.848 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi096.egbmoq{0:24251} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.96:6824/1539752208,v1:172.21.15.96:6825/1539752208] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:46:47.848 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.llefgu{0:14500} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.94:6828/126762980,v1:172.21.15.94:6829/126762980] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:46:47.848 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:46:47.848 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:46:47.848 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-03-24T10:46:47.848 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:46:47.848 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.efiqte{-1:14512} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.94:6826/1636528860,v1:172.21.15.94:6827/1636528860] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:46:47.848 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi096.jtebhh{-1:24261} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.96:6826/1321087930,v1:172.21.15.96:6827/1321087930] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:46:47.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.847+0000 7fb9493a0700 1 -- 172.21.15.94:0/279105872 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7fb92c06b280 msgr2=0x7fb92c06d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:47.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.847+0000 7fb9493a0700 1 --2- 172.21.15.94:0/279105872 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7fb92c06b280 0x7fb92c06d730 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb944065fd0 tx=0x7fb93400b040 comp rx=0 tx=0).stop 2024-03-24T10:46:47.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.847+0000 7fb9493a0700 1 -- 172.21.15.94:0/279105872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9440fcdc0 msgr2=0x7fb944112a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:47.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.847+0000 7fb9493a0700 1 --2- 172.21.15.94:0/279105872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9440fcdc0 0x7fb944112a80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb938013870 tx=0x7fb938009e30 comp rx=0 tx=0).stop 2024-03-24T10:46:47.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.847+0000 7fb9493a0700 1 -- 172.21.15.94:0/279105872 shutdown_connections 2024-03-24T10:46:47.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.847+0000 7fb9493a0700 1 --2- 172.21.15.94:0/279105872 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7fb92c06b280 0x7fb92c06d730 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:47.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.847+0000 7fb9493a0700 1 --2- 172.21.15.94:0/279105872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9440fbb10 0x7fb944110530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:47.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.847+0000 7fb9493a0700 1 --2- 172.21.15.94:0/279105872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9440fcdc0 0x7fb944112a80 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:47.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.847+0000 7fb9493a0700 1 -- 172.21.15.94:0/279105872 >> 172.21.15.94:0/279105872 conn(0x7fb9440f7270 msgr2=0x7fb9440f8940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:46:47.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.848+0000 7fb9493a0700 1 -- 172.21.15.94:0/279105872 shutdown_connections 2024-03-24T10:46:47.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.848+0000 7fb9493a0700 1 -- 172.21.15.94:0/279105872 wait complete. 2024-03-24T10:46:47.850 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 18 2024-03-24T10:46:47.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.954+0000 7fc00b1b2700 1 -- 172.21.15.94:0/2128482496 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc004066ab0 msgr2=0x7fc0040ffb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:47.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.954+0000 7fc00b1b2700 1 --2- 172.21.15.94:0/2128482496 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc004066ab0 0x7fc0040ffb10 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fbff0004940 tx=0x7fbff0009e30 comp rx=0 tx=0).stop 2024-03-24T10:46:47.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.954+0000 7fc00b1b2700 1 -- 172.21.15.94:0/2128482496 shutdown_connections 2024-03-24T10:46:47.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.954+0000 7fc00b1b2700 1 --2- 172.21.15.94:0/2128482496 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc004100050 0x7fc004102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:47.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.954+0000 7fc00b1b2700 1 --2- 172.21.15.94:0/2128482496 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc004066ab0 0x7fc0040ffb10 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:47.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.954+0000 7fc00b1b2700 1 -- 172.21.15.94:0/2128482496 >> 172.21.15.94:0/2128482496 conn(0x7fc0040f74e0 msgr2=0x7fc0040f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:46:47.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.954+0000 7fc00b1b2700 1 -- 172.21.15.94:0/2128482496 shutdown_connections 2024-03-24T10:46:47.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.954+0000 7fc00b1b2700 1 -- 172.21.15.94:0/2128482496 wait complete. 2024-03-24T10:46:47.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.955+0000 7fc00b1b2700 1 Processor -- start 2024-03-24T10:46:47.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.955+0000 7fc00b1b2700 1 -- start start 2024-03-24T10:46:47.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.955+0000 7fc00b1b2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc004066ab0 0x7fc0040fd540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:46:47.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.955+0000 7fc00b1b2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc004100050 0x7fc0040fba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:46:47.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.955+0000 7fc00b1b2700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0040fbfc0 con 0x7fc004100050 2024-03-24T10:46:47.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.955+0000 7fc00b1b2700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0040fc130 con 0x7fc004066ab0 2024-03-24T10:46:47.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.955+0000 7fc003fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc004100050 0x7fc0040fba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:46:47.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.955+0000 7fc003fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc004100050 0x7fc0040fba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37274/0 (socket says 172.21.15.94:37274) 2024-03-24T10:46:47.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.955+0000 7fc003fff700 1 -- 172.21.15.94:0/3956714334 learned_addr learned my addr 172.21.15.94:0/3956714334 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:46:47.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.955+0000 7fc008f4e700 1 --2- 172.21.15.94:0/3956714334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc004066ab0 0x7fc0040fd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:46:47.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.956+0000 7fc008f4e700 1 -- 172.21.15.94:0/3956714334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc004100050 msgr2=0x7fc0040fba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:47.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.956+0000 7fc008f4e700 1 --2- 172.21.15.94:0/3956714334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc004100050 0x7fc0040fba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:47.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.956+0000 7fc008f4e700 1 -- 172.21.15.94:0/3956714334 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff0005040 con 0x7fc004066ab0 2024-03-24T10:46:47.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.956+0000 7fc008f4e700 1 --2- 172.21.15.94:0/3956714334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc004066ab0 0x7fc0040fd540 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbff0000c00 tx=0x7fbff0009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:46:47.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.956+0000 7fc0017fa700 1 -- 172.21.15.94:0/3956714334 <== mon.1 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbff000e030 con 0x7fc004066ab0 2024-03-24T10:46:47.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.956+0000 7fc00b1b2700 1 -- 172.21.15.94:0/3956714334 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0040fc3b0 con 0x7fc004066ab0 2024-03-24T10:46:47.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.956+0000 7fc0017fa700 1 -- 172.21.15.94:0/3956714334 <== mon.1 v2:172.21.15.96:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbff0013de0 con 0x7fc004066ab0 2024-03-24T10:46:47.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.956+0000 7fc0017fa700 1 -- 172.21.15.94:0/3956714334 <== mon.1 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbff001cdc0 con 0x7fc004066ab0 2024-03-24T10:46:47.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.956+0000 7fc00b1b2700 1 -- 172.21.15.94:0/3956714334 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc004115a80 con 0x7fc004066ab0 2024-03-24T10:46:47.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.957+0000 7fc00b1b2700 1 -- 172.21.15.94:0/3956714334 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc00405f5f0 con 0x7fc004066ab0 2024-03-24T10:46:47.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.958+0000 7fc0017fa700 1 -- 172.21.15.94:0/3956714334 <== mon.1 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 24) v1 ==== 89012+0+0 (secure 0 0 0) 0x7fbff0025400 con 0x7fc004066ab0 2024-03-24T10:46:47.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.958+0000 7fc0017fa700 1 --2- 172.21.15.94:0/3956714334 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7fbff406b3b0 0x7fbff406d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:46:47.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.959+0000 7fc0017fa700 1 -- 172.21.15.94:0/3956714334 <== mon.1 v2:172.21.15.96:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fbff008b620 con 0x7fc004066ab0 2024-03-24T10:46:47.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.959+0000 7fc003fff700 1 --2- 172.21.15.94:0/3956714334 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7fbff406b3b0 0x7fbff406d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:46:47.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.959+0000 7fc003fff700 1 --2- 172.21.15.94:0/3956714334 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7fbff406b3b0 0x7fbff406d860 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fbff8009b80 tx=0x7fbff800b040 comp rx=0 tx=0).ready entity=mgr.24397 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:46:47.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:47.963+0000 7fc0017fa700 1 -- 172.21.15.94:0/3956714334 <== mon.1 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7fbff0055740 con 0x7fc004066ab0 2024-03-24T10:46:48.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:47 smithi096 ceph-mon[38097]: Updating smithi096:/var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T10:46:48.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:47 smithi096 ceph-mon[38097]: Updating smithi094:/var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T10:46:48.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:47 smithi096 ceph-mon[38097]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:46:48.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:47 smithi096 ceph-mon[38097]: Updating smithi096:/etc/ceph/ceph.client.admin.keyring 2024-03-24T10:46:48.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:47 smithi096 ceph-mon[38097]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-03-24T10:46:48.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:47 smithi096 ceph-mon[38097]: from='client.24449 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:46:48.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:47 smithi096 ceph-mon[38097]: Updating smithi094:/var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T10:46:48.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:47 smithi096 ceph-mon[38097]: Updating smithi096:/var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T10:46:48.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:47 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:48.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:47 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:48.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:47 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:48.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:47 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:48.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:47 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:48.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:47 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:48.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:47 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:48.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:47 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:48.077 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:47 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:48.077 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:47 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/3901748197' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:46:48.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.196+0000 7fc00b1b2700 1 -- 172.21.15.94:0/3956714334 --> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc0040f8fd0 con 0x7fbff406b3b0 2024-03-24T10:46:48.197 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:47 smithi094 ceph-mon[29446]: Updating smithi096:/var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T10:46:48.197 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:47 smithi094 ceph-mon[29446]: Updating smithi094:/var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T10:46:48.197 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:47 smithi094 ceph-mon[29446]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:46:48.197 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:47 smithi094 ceph-mon[29446]: Updating smithi096:/etc/ceph/ceph.client.admin.keyring 2024-03-24T10:46:48.197 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:47 smithi094 ceph-mon[29446]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-03-24T10:46:48.197 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:47 smithi094 ceph-mon[29446]: from='client.24449 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:46:48.197 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:47 smithi094 ceph-mon[29446]: Updating smithi094:/var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T10:46:48.197 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:47 smithi094 ceph-mon[29446]: Updating smithi096:/var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T10:46:48.197 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:47 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:48.198 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:47 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:48.198 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:47 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:48.198 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:47 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:48.198 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:47 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:48.198 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:47 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:48.198 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:47 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:48.198 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:47 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:48.198 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:47 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:48.198 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:47 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/3901748197' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:46:48.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.198+0000 7fc0017fa700 1 -- 172.21.15.94:0/3956714334 <== mgr.24397 v2:172.21.15.96:6828/1632816545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7fc0040f8fd0 con 0x7fbff406b3b0 2024-03-24T10:46:48.199 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-03-24T10:46:48.199 INFO:teuthology.orchestra.run.smithi094.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919", 2024-03-24T10:46:48.199 INFO:teuthology.orchestra.run.smithi094.stdout: "in_progress": true, 2024-03-24T10:46:48.199 INFO:teuthology.orchestra.run.smithi094.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T10:46:48.199 INFO:teuthology.orchestra.run.smithi094.stdout: "services_complete": [], 2024-03-24T10:46:48.199 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": "1/21 daemons upgraded", 2024-03-24T10:46:48.199 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "", 2024-03-24T10:46:48.199 INFO:teuthology.orchestra.run.smithi094.stdout: "is_paused": false 2024-03-24T10:46:48.199 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-03-24T10:46:48.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.203+0000 7fc00b1b2700 1 -- 172.21.15.94:0/3956714334 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7fbff406b3b0 msgr2=0x7fbff406d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:48.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.203+0000 7fc00b1b2700 1 --2- 172.21.15.94:0/3956714334 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7fbff406b3b0 0x7fbff406d860 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fbff8009b80 tx=0x7fbff800b040 comp rx=0 tx=0).stop 2024-03-24T10:46:48.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.203+0000 7fc00b1b2700 1 -- 172.21.15.94:0/3956714334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc004066ab0 msgr2=0x7fc0040fd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:48.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.203+0000 7fc00b1b2700 1 --2- 172.21.15.94:0/3956714334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc004066ab0 0x7fc0040fd540 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbff0000c00 tx=0x7fbff0009e30 comp rx=0 tx=0).stop 2024-03-24T10:46:48.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.203+0000 7fc00b1b2700 1 -- 172.21.15.94:0/3956714334 shutdown_connections 2024-03-24T10:46:48.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.203+0000 7fc00b1b2700 1 --2- 172.21.15.94:0/3956714334 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7fbff406b3b0 0x7fbff406d860 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:48.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.203+0000 7fc00b1b2700 1 --2- 172.21.15.94:0/3956714334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc004066ab0 0x7fc0040fd540 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:48.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.203+0000 7fc00b1b2700 1 --2- 172.21.15.94:0/3956714334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc004100050 0x7fc0040fba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:48.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.203+0000 7fc00b1b2700 1 -- 172.21.15.94:0/3956714334 >> 172.21.15.94:0/3956714334 conn(0x7fc0040f74e0 msgr2=0x7fc0040f78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:46:48.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.203+0000 7fc00b1b2700 1 -- 172.21.15.94:0/3956714334 shutdown_connections 2024-03-24T10:46:48.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.203+0000 7fc00b1b2700 1 -- 172.21.15.94:0/3956714334 wait complete. 2024-03-24T10:46:48.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.337+0000 7f1c16c20700 1 -- 172.21.15.94:0/2790301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c100fbd00 msgr2=0x7f1c100fc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:48.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.337+0000 7f1c16c20700 1 --2- 172.21.15.94:0/2790301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c100fbd00 0x7f1c100fc110 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1c04004940 tx=0x7f1c04009e30 comp rx=0 tx=0).stop 2024-03-24T10:46:48.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.338+0000 7f1c16c20700 1 -- 172.21.15.94:0/2790301 shutdown_connections 2024-03-24T10:46:48.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.338+0000 7f1c16c20700 1 --2- 172.21.15.94:0/2790301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c100fcfb0 0x7f1c100fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:48.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.338+0000 7f1c16c20700 1 --2- 172.21.15.94:0/2790301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c100fbd00 0x7f1c100fc110 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:48.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.338+0000 7f1c16c20700 1 -- 172.21.15.94:0/2790301 >> 172.21.15.94:0/2790301 conn(0x7f1c100f7440 msgr2=0x7f1c100f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:46:48.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.338+0000 7f1c16c20700 1 -- 172.21.15.94:0/2790301 shutdown_connections 2024-03-24T10:46:48.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.338+0000 7f1c16c20700 1 -- 172.21.15.94:0/2790301 wait complete. 2024-03-24T10:46:48.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.338+0000 7f1c16c20700 1 Processor -- start 2024-03-24T10:46:48.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.338+0000 7f1c16c20700 1 -- start start 2024-03-24T10:46:48.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.338+0000 7f1c16c20700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c100fbd00 0x7f1c10066c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:46:48.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.338+0000 7f1c16c20700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c100fcfb0 0x7f1c101126c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:46:48.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.338+0000 7f1c16c20700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c10112c00 con 0x7f1c100fbd00 2024-03-24T10:46:48.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.338+0000 7f1c16c20700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c10112d40 con 0x7f1c100fcfb0 2024-03-24T10:46:48.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.339+0000 7f1c149bc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c100fbd00 0x7f1c10066c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:46:48.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.339+0000 7f1c149bc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c100fbd00 0x7f1c10066c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37286/0 (socket says 172.21.15.94:37286) 2024-03-24T10:46:48.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.339+0000 7f1c149bc700 1 -- 172.21.15.94:0/3667153916 learned_addr learned my addr 172.21.15.94:0/3667153916 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:46:48.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.339+0000 7f1c0ffff700 1 --2- 172.21.15.94:0/3667153916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c100fcfb0 0x7f1c101126c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:46:48.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.339+0000 7f1c149bc700 1 -- 172.21.15.94:0/3667153916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c100fcfb0 msgr2=0x7f1c101126c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:48.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.339+0000 7f1c149bc700 1 --2- 172.21.15.94:0/3667153916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c100fcfb0 0x7f1c101126c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:48.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.339+0000 7f1c149bc700 1 -- 172.21.15.94:0/3667153916 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c04005040 con 0x7f1c100fbd00 2024-03-24T10:46:48.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.339+0000 7f1c149bc700 1 --2- 172.21.15.94:0/3667153916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c100fbd00 0x7f1c10066c70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1c04009150 tx=0x7f1c04009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:46:48.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.339+0000 7f1c0d7fa700 1 -- 172.21.15.94:0/3667153916 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c0400e040 con 0x7f1c100fbd00 2024-03-24T10:46:48.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.339+0000 7f1c0d7fa700 1 -- 172.21.15.94:0/3667153916 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1c04004290 con 0x7f1c100fbd00 2024-03-24T10:46:48.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.339+0000 7f1c16c20700 1 -- 172.21.15.94:0/3667153916 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c10112f40 con 0x7f1c100fbd00 2024-03-24T10:46:48.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.339+0000 7f1c0d7fa700 1 -- 172.21.15.94:0/3667153916 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c0401c630 con 0x7f1c100fbd00 2024-03-24T10:46:48.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.339+0000 7f1c16c20700 1 -- 172.21.15.94:0/3667153916 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c10113470 con 0x7f1c100fbd00 2024-03-24T10:46:48.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.340+0000 7f1c16c20700 1 -- 172.21.15.94:0/3667153916 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c1004c710 con 0x7f1c100fbd00 2024-03-24T10:46:48.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.341+0000 7f1c0d7fa700 1 -- 172.21.15.94:0/3667153916 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) v1 ==== 89012+0+0 (secure 0 0 0) 0x7f1c04026440 con 0x7f1c100fbd00 2024-03-24T10:46:48.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.341+0000 7f1c0d7fa700 1 --2- 172.21.15.94:0/3667153916 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f1c0006b2f0 0x7f1c0006d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:46:48.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.341+0000 7f1c0d7fa700 1 -- 172.21.15.94:0/3667153916 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f1c0408a8d0 con 0x7f1c100fbd00 2024-03-24T10:46:48.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.341+0000 7f1c0ffff700 1 --2- 172.21.15.94:0/3667153916 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f1c0006b2f0 0x7f1c0006d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:46:48.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.342+0000 7f1c0ffff700 1 --2- 172.21.15.94:0/3667153916 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f1c0006b2f0 0x7f1c0006d7a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc009fd0 tx=0x7f1bfc00b040 comp rx=0 tx=0).ready entity=mgr.24397 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:46:48.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.346+0000 7f1c0d7fa700 1 -- 172.21.15.94:0/3667153916 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7f1c040585f0 con 0x7f1c100fbd00 2024-03-24T10:46:48.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.627+0000 7f1c16c20700 1 -- 172.21.15.94:0/3667153916 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1c101047f0 con 0x7f1c100fbd00 2024-03-24T10:46:48.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.627+0000 7f1c0d7fa700 1 -- 172.21.15.94:0/3667153916 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f1c04058180 con 0x7f1c100fbd00 2024-03-24T10:46:48.627 INFO:teuthology.orchestra.run.smithi094.stdout:HEALTH_OK 2024-03-24T10:46:48.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.629+0000 7f1c16c20700 1 -- 172.21.15.94:0/3667153916 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f1c0006b2f0 msgr2=0x7f1c0006d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:48.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.629+0000 7f1c16c20700 1 --2- 172.21.15.94:0/3667153916 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f1c0006b2f0 0x7f1c0006d7a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc009fd0 tx=0x7f1bfc00b040 comp rx=0 tx=0).stop 2024-03-24T10:46:48.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.629+0000 7f1c16c20700 1 -- 172.21.15.94:0/3667153916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c100fbd00 msgr2=0x7f1c10066c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:46:48.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.629+0000 7f1c16c20700 1 --2- 172.21.15.94:0/3667153916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c100fbd00 0x7f1c10066c70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1c04009150 tx=0x7f1c04009e30 comp rx=0 tx=0).stop 2024-03-24T10:46:48.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.629+0000 7f1c16c20700 1 -- 172.21.15.94:0/3667153916 shutdown_connections 2024-03-24T10:46:48.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.629+0000 7f1c16c20700 1 --2- 172.21.15.94:0/3667153916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c100fbd00 0x7f1c10066c70 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:48.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.629+0000 7f1c16c20700 1 --2- 172.21.15.94:0/3667153916 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f1c0006b2f0 0x7f1c0006d7a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:48.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.629+0000 7f1c16c20700 1 --2- 172.21.15.94:0/3667153916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c100fcfb0 0x7f1c101126c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:46:48.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.630+0000 7f1c16c20700 1 -- 172.21.15.94:0/3667153916 >> 172.21.15.94:0/3667153916 conn(0x7f1c100f7440 msgr2=0x7f1c100f8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:46:48.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.630+0000 7f1c16c20700 1 -- 172.21.15.94:0/3667153916 shutdown_connections 2024-03-24T10:46:48.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:46:48.630+0000 7f1c16c20700 1 -- 172.21.15.94:0/3667153916 wait complete. 2024-03-24T10:46:48.994 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:48 smithi094 ceph-mon[29446]: from='client.14726 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:46:48.994 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:48 smithi094 ceph-mon[29446]: Reconfiguring node-exporter.smithi094 (dependencies changed)... 2024-03-24T10:46:48.994 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:48 smithi094 ceph-mon[29446]: Reconfiguring daemon node-exporter.smithi094 on smithi094 2024-03-24T10:46:48.994 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:48 smithi094 ceph-mon[29446]: pgmap v12: 65 pgs: 65 active+clean; 407 MiB data, 3.2 GiB used, 533 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 356 op/s 2024-03-24T10:46:48.994 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:48 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/279105872' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T10:46:48.994 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:48 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/3667153916' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T10:46:49.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:48 smithi096 ceph-mon[38097]: from='client.14726 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:46:49.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:48 smithi096 ceph-mon[38097]: Reconfiguring node-exporter.smithi094 (dependencies changed)... 2024-03-24T10:46:49.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:48 smithi096 ceph-mon[38097]: Reconfiguring daemon node-exporter.smithi094 on smithi094 2024-03-24T10:46:49.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:48 smithi096 ceph-mon[38097]: pgmap v12: 65 pgs: 65 active+clean; 407 MiB data, 3.2 GiB used, 533 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 356 op/s 2024-03-24T10:46:49.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:48 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/279105872' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T10:46:49.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:48 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/3667153916' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T10:46:50.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:49 smithi096 ceph-mon[38097]: from='client.24465 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:46:50.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:49 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:50.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:49 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:50.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:49 smithi094 ceph-mon[29446]: from='client.24465 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:46:50.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:49 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:50.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:49 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:51.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:50 smithi096 ceph-mon[38097]: pgmap v13: 65 pgs: 65 active+clean; 407 MiB data, 3.2 GiB used, 533 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 259 op/s 2024-03-24T10:46:51.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:50 smithi096 ceph-mon[38097]: Reconfiguring alertmanager.smithi094 (dependencies changed)... 2024-03-24T10:46:51.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:50 smithi096 ceph-mon[38097]: Reconfiguring daemon alertmanager.smithi094 on smithi094 2024-03-24T10:46:51.082 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:50 smithi094 ceph-mon[29446]: pgmap v13: 65 pgs: 65 active+clean; 407 MiB data, 3.2 GiB used, 533 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 259 op/s 2024-03-24T10:46:51.082 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:50 smithi094 ceph-mon[29446]: Reconfiguring alertmanager.smithi094 (dependencies changed)... 2024-03-24T10:46:51.082 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:50 smithi094 ceph-mon[29446]: Reconfiguring daemon alertmanager.smithi094 on smithi094 2024-03-24T10:46:52.960 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:52 smithi094 ceph-mon[29446]: pgmap v14: 65 pgs: 65 active+clean; 299 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 332 op/s 2024-03-24T10:46:53.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:52 smithi096 ceph-mon[38097]: pgmap v14: 65 pgs: 65 active+clean; 299 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 332 op/s 2024-03-24T10:46:55.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:54 smithi096 ceph-mon[38097]: pgmap v15: 65 pgs: 65 active+clean; 299 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 204 op/s 2024-03-24T10:46:55.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:54 smithi094 ceph-mon[29446]: pgmap v15: 65 pgs: 65 active+clean; 299 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 204 op/s 2024-03-24T10:46:57.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:56 smithi096 ceph-mon[38097]: pgmap v16: 65 pgs: 65 active+clean; 303 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 212 op/s 2024-03-24T10:46:57.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:56 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:57.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:56 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:57.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:56 smithi094 ceph-mon[29446]: pgmap v16: 65 pgs: 65 active+clean; 303 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 212 op/s 2024-03-24T10:46:57.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:56 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:57.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:56 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:46:58.037 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:57 smithi094 ceph-mon[29446]: Reconfiguring grafana.smithi094 (dependencies changed)... 2024-03-24T10:46:58.037 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:57 smithi094 ceph-mon[29446]: Reconfiguring daemon grafana.smithi094 on smithi094 2024-03-24T10:46:58.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:57 smithi096 ceph-mon[38097]: Reconfiguring grafana.smithi094 (dependencies changed)... 2024-03-24T10:46:58.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:57 smithi096 ceph-mon[38097]: Reconfiguring daemon grafana.smithi094 on smithi094 2024-03-24T10:46:59.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:58 smithi096 ceph-mon[38097]: pgmap v17: 65 pgs: 65 active+clean; 305 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 189 op/s 2024-03-24T10:46:59.170 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:58 smithi094 ceph-mon[29446]: pgmap v17: 65 pgs: 65 active+clean; 305 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 189 op/s 2024-03-24T10:47:00.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:46:59 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:47:00.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:46:59 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:47:01.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:00 smithi094 ceph-mon[29446]: pgmap v18: 65 pgs: 65 active+clean; 305 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 98 op/s 2024-03-24T10:47:01.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:00 smithi096 ceph-mon[38097]: pgmap v18: 65 pgs: 65 active+clean; 305 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 98 op/s 2024-03-24T10:47:03.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:02 smithi094 ceph-mon[29446]: pgmap v19: 65 pgs: 65 active+clean; 299 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 288 op/s 2024-03-24T10:47:03.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:02 smithi096 ceph-mon[38097]: pgmap v19: 65 pgs: 65 active+clean; 299 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 288 op/s 2024-03-24T10:47:05.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:04 smithi094 ceph-mon[29446]: pgmap v20: 65 pgs: 65 active+clean; 299 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 215 op/s 2024-03-24T10:47:05.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:04 smithi096 ceph-mon[38097]: pgmap v20: 65 pgs: 65 active+clean; 299 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 215 op/s 2024-03-24T10:47:07.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:06 smithi094 ceph-mon[29446]: pgmap v21: 65 pgs: 65 active+clean; 304 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 277 op/s 2024-03-24T10:47:07.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:06 smithi096 ceph-mon[38097]: pgmap v21: 65 pgs: 65 active+clean; 304 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 277 op/s 2024-03-24T10:47:09.232 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:08 smithi094 ceph-mon[29446]: pgmap v22: 65 pgs: 65 active+clean; 302 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 395 op/s 2024-03-24T10:47:09.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:08 smithi096 ceph-mon[38097]: pgmap v22: 65 pgs: 65 active+clean; 302 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 395 op/s 2024-03-24T10:47:10.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:10 smithi094 ceph-mon[29446]: pgmap v23: 65 pgs: 65 active+clean; 302 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 377 op/s 2024-03-24T10:47:11.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:10 smithi096 ceph-mon[38097]: pgmap v23: 65 pgs: 65 active+clean; 302 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 377 op/s 2024-03-24T10:47:12.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:12 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:47:12.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:12 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:47:12.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:12 smithi094 ceph-mon[29446]: Reconfiguring prometheus.smithi094 (dependencies changed)... 2024-03-24T10:47:12.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:12 smithi094 ceph-mon[29446]: pgmap v24: 65 pgs: 65 active+clean; 308 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 533 op/s 2024-03-24T10:47:12.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:12 smithi094 ceph-mon[29446]: Reconfiguring daemon prometheus.smithi094 on smithi094 2024-03-24T10:47:12.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:12 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:47:12.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:12 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:47:12.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:12 smithi096 ceph-mon[38097]: Reconfiguring prometheus.smithi094 (dependencies changed)... 2024-03-24T10:47:12.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:12 smithi096 ceph-mon[38097]: pgmap v24: 65 pgs: 65 active+clean; 308 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 533 op/s 2024-03-24T10:47:12.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:12 smithi096 ceph-mon[38097]: Reconfiguring daemon prometheus.smithi094 on smithi094 2024-03-24T10:47:14.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:14 smithi094 ceph-mon[29446]: pgmap v25: 65 pgs: 65 active+clean; 308 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 344 op/s 2024-03-24T10:47:15.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:14 smithi096 ceph-mon[38097]: pgmap v25: 65 pgs: 65 active+clean; 308 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 344 op/s 2024-03-24T10:47:15.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:15 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:47:16.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:15 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:47:16.816 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:16 smithi094 ceph-mon[29446]: pgmap v26: 65 pgs: 65 active+clean; 312 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 669 op/s 2024-03-24T10:47:17.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:16 smithi096 ceph-mon[38097]: pgmap v26: 65 pgs: 65 active+clean; 312 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 669 op/s 2024-03-24T10:47:18.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:18.782+0000 7f85a05bc700 1 -- 172.21.15.94:0/1603203785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85980fcee0 msgr2=0x7f85980fd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:18.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:18.782+0000 7f85a05bc700 1 --2- 172.21.15.94:0/1603203785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85980fcee0 0x7f85980fd310 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8594004940 tx=0x7f8594009e30 comp rx=0 tx=0).stop 2024-03-24T10:47:18.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:18.782+0000 7f85a05bc700 1 -- 172.21.15.94:0/1603203785 shutdown_connections 2024-03-24T10:47:18.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:18.782+0000 7f85a05bc700 1 --2- 172.21.15.94:0/1603203785 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85980fe0d0 0x7f85980fe540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:18.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:18.782+0000 7f85a05bc700 1 --2- 172.21.15.94:0/1603203785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85980fcee0 0x7f85980fd310 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:18.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:18.782+0000 7f85a05bc700 1 -- 172.21.15.94:0/1603203785 >> 172.21.15.94:0/1603203785 conn(0x7f85980f8560 msgr2=0x7f85980fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:47:18.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:18.782+0000 7f85a05bc700 1 -- 172.21.15.94:0/1603203785 shutdown_connections 2024-03-24T10:47:18.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:18.782+0000 7f85a05bc700 1 -- 172.21.15.94:0/1603203785 wait complete. 2024-03-24T10:47:18.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:18.783+0000 7f85a05bc700 1 Processor -- start 2024-03-24T10:47:18.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:18.783+0000 7f85a05bc700 1 -- start start 2024-03-24T10:47:18.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:18.783+0000 7f85a05bc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85980fcee0 0x7f859810d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:47:18.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:18.783+0000 7f85a05bc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85980fe0d0 0x7f859810df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:47:18.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:18.783+0000 7f85a05bc700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f859810e510 con 0x7f85980fe0d0 2024-03-24T10:47:18.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:18.783+0000 7f85a05bc700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f859810ed60 con 0x7f85980fcee0 2024-03-24T10:47:18.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:18.783+0000 7f859db57700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85980fe0d0 0x7f859810df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:47:18.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:18.783+0000 7f859db57700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85980fe0d0 0x7f859810df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33234/0 (socket says 172.21.15.94:33234) 2024-03-24T10:47:18.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:18.783+0000 7f859db57700 1 -- 172.21.15.94:0/3669563964 learned_addr learned my addr 172.21.15.94:0/3669563964 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:47:18.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:18.783+0000 7f859e358700 1 --2- 172.21.15.94:0/3669563964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85980fcee0 0x7f859810d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:47:18.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:18.783+0000 7f859db57700 1 -- 172.21.15.94:0/3669563964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85980fcee0 msgr2=0x7f859810d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:18.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:18.783+0000 7f859db57700 1 --2- 172.21.15.94:0/3669563964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85980fcee0 0x7f859810d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:18.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:18.783+0000 7f859db57700 1 -- 172.21.15.94:0/3669563964 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8594005040 con 0x7f85980fe0d0 2024-03-24T10:47:18.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:18.784+0000 7f859db57700 1 --2- 172.21.15.94:0/3669563964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85980fe0d0 0x7f859810df10 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f858800eb40 tx=0x7f858800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:47:18.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:18.784+0000 7f858effd700 1 -- 172.21.15.94:0/3669563964 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8588007cc0 con 0x7f85980fe0d0 2024-03-24T10:47:18.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:18.784+0000 7f85a05bc700 1 -- 172.21.15.94:0/3669563964 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f859810efe0 con 0x7f85980fe0d0 2024-03-24T10:47:18.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:18.784+0000 7f858effd700 1 -- 172.21.15.94:0/3669563964 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f858800f040 con 0x7f85980fe0d0 2024-03-24T10:47:18.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:18.784+0000 7f85a05bc700 1 -- 172.21.15.94:0/3669563964 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f859810f520 con 0x7f85980fe0d0 2024-03-24T10:47:18.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:18.784+0000 7f858effd700 1 -- 172.21.15.94:0/3669563964 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f858800ca10 con 0x7f85980fe0d0 2024-03-24T10:47:18.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:18.785+0000 7f85a05bc700 1 -- 172.21.15.94:0/3669563964 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f859805f5f0 con 0x7f85980fe0d0 2024-03-24T10:47:18.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:18.785+0000 7f858effd700 1 -- 172.21.15.94:0/3669563964 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) v1 ==== 89012+0+0 (secure 0 0 0) 0x7f858800cc80 con 0x7f85980fe0d0 2024-03-24T10:47:18.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:18.786+0000 7f858effd700 1 --2- 172.21.15.94:0/3669563964 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f858406b2e0 0x7f858406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:47:18.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:18.786+0000 7f859e358700 1 --2- 172.21.15.94:0/3669563964 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f858406b2e0 0x7f858406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:47:18.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:18.786+0000 7f858effd700 1 -- 172.21.15.94:0/3669563964 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f8588089a20 con 0x7f85980fe0d0 2024-03-24T10:47:18.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:18.786+0000 7f859e358700 1 --2- 172.21.15.94:0/3669563964 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f858406b2e0 0x7f858406d790 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8594004890 tx=0x7f8594009e30 comp rx=0 tx=0).ready entity=mgr.24397 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:47:18.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:18.790+0000 7f858effd700 1 -- 172.21.15.94:0/3669563964 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7f858804fb80 con 0x7f85980fe0d0 2024-03-24T10:47:18.983 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:18 smithi096 ceph-mon[38097]: pgmap v27: 65 pgs: 65 active+clean; 311 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 1.31k op/s 2024-03-24T10:47:18.983 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:18 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:47:18.983 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:18 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:47:18.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:18 smithi094 ceph-mon[29446]: pgmap v27: 65 pgs: 65 active+clean; 311 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 1.31k op/s 2024-03-24T10:47:18.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:18 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:47:18.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:18 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:47:19.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.004+0000 7f85a05bc700 1 -- 172.21.15.94:0/3669563964 --> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f859810f850 con 0x7f858406b2e0 2024-03-24T10:47:19.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.006+0000 7f858effd700 1 -- 172.21.15.94:0/3669563964 <== mgr.24397 v2:172.21.15.96:6828/1632816545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f859810f850 con 0x7f858406b2e0 2024-03-24T10:47:19.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.008+0000 7f85a05bc700 1 -- 172.21.15.94:0/3669563964 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f858406b2e0 msgr2=0x7f858406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:19.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.008+0000 7f85a05bc700 1 --2- 172.21.15.94:0/3669563964 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f858406b2e0 0x7f858406d790 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8594004890 tx=0x7f8594009e30 comp rx=0 tx=0).stop 2024-03-24T10:47:19.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.008+0000 7f85a05bc700 1 -- 172.21.15.94:0/3669563964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85980fe0d0 msgr2=0x7f859810df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:19.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.008+0000 7f85a05bc700 1 --2- 172.21.15.94:0/3669563964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85980fe0d0 0x7f859810df10 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f858800eb40 tx=0x7f858800b040 comp rx=0 tx=0).stop 2024-03-24T10:47:19.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.008+0000 7f85a05bc700 1 -- 172.21.15.94:0/3669563964 shutdown_connections 2024-03-24T10:47:19.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.008+0000 7f85a05bc700 1 --2- 172.21.15.94:0/3669563964 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f858406b2e0 0x7f858406d790 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:19.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.008+0000 7f85a05bc700 1 --2- 172.21.15.94:0/3669563964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85980fcee0 0x7f859810d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:19.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.008+0000 7f85a05bc700 1 --2- 172.21.15.94:0/3669563964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85980fe0d0 0x7f859810df10 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:19.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.008+0000 7f85a05bc700 1 -- 172.21.15.94:0/3669563964 >> 172.21.15.94:0/3669563964 conn(0x7f85980f8560 msgr2=0x7f85980f9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:47:19.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.008+0000 7f85a05bc700 1 -- 172.21.15.94:0/3669563964 shutdown_connections 2024-03-24T10:47:19.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.008+0000 7f85a05bc700 1 -- 172.21.15.94:0/3669563964 wait complete. 2024-03-24T10:47:19.019 INFO:teuthology.orchestra.run.smithi094.stdout:true 2024-03-24T10:47:19.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.110+0000 7f69cdbec700 1 -- 172.21.15.94:0/651802541 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69c80fcf80 msgr2=0x7f69c80fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:19.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.110+0000 7f69cdbec700 1 --2- 172.21.15.94:0/651802541 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69c80fcf80 0x7f69c80fd3f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f69bc004940 tx=0x7f69bc009e30 comp rx=0 tx=0).stop 2024-03-24T10:47:19.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.110+0000 7f69cdbec700 1 -- 172.21.15.94:0/651802541 shutdown_connections 2024-03-24T10:47:19.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.110+0000 7f69cdbec700 1 --2- 172.21.15.94:0/651802541 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69c80fcf80 0x7f69c80fd3f0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:19.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.110+0000 7f69cdbec700 1 --2- 172.21.15.94:0/651802541 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f69c80fbcd0 0x7f69c80fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:19.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.110+0000 7f69cdbec700 1 -- 172.21.15.94:0/651802541 >> 172.21.15.94:0/651802541 conn(0x7f69c80f7430 msgr2=0x7f69c80f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:47:19.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.110+0000 7f69cdbec700 1 -- 172.21.15.94:0/651802541 shutdown_connections 2024-03-24T10:47:19.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.110+0000 7f69cdbec700 1 -- 172.21.15.94:0/651802541 wait complete. 2024-03-24T10:47:19.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.111+0000 7f69cdbec700 1 Processor -- start 2024-03-24T10:47:19.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.111+0000 7f69cdbec700 1 -- start start 2024-03-24T10:47:19.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.111+0000 7f69cdbec700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69c80fbcd0 0x7f69c810c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:47:19.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.111+0000 7f69cdbec700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f69c80fcf80 0x7f69c810cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:47:19.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.111+0000 7f69cdbec700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c810d3c0 con 0x7f69c80fbcd0 2024-03-24T10:47:19.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.111+0000 7f69cdbec700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c810dc10 con 0x7f69c80fcf80 2024-03-24T10:47:19.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.111+0000 7f69c77fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69c80fbcd0 0x7f69c810c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:47:19.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.111+0000 7f69c77fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69c80fbcd0 0x7f69c810c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33248/0 (socket says 172.21.15.94:33248) 2024-03-24T10:47:19.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.111+0000 7f69c77fe700 1 -- 172.21.15.94:0/3148671508 learned_addr learned my addr 172.21.15.94:0/3148671508 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:47:19.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.111+0000 7f69c6ffd700 1 --2- 172.21.15.94:0/3148671508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f69c80fcf80 0x7f69c810cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:47:19.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.112+0000 7f69c77fe700 1 -- 172.21.15.94:0/3148671508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f69c80fcf80 msgr2=0x7f69c810cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:19.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.112+0000 7f69c77fe700 1 --2- 172.21.15.94:0/3148671508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f69c80fcf80 0x7f69c810cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:19.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.112+0000 7f69c77fe700 1 -- 172.21.15.94:0/3148671508 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69bc005040 con 0x7f69c80fbcd0 2024-03-24T10:47:19.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.112+0000 7f69c6ffd700 1 --2- 172.21.15.94:0/3148671508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f69c80fcf80 0x7f69c810cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T10:47:19.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.112+0000 7f69c77fe700 1 --2- 172.21.15.94:0/3148671508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69c80fbcd0 0x7f69c810c880 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f69b800eb40 tx=0x7f69b800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:47:19.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.112+0000 7f69affff700 1 -- 172.21.15.94:0/3148671508 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69b8007cc0 con 0x7f69c80fbcd0 2024-03-24T10:47:19.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.112+0000 7f69affff700 1 -- 172.21.15.94:0/3148671508 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f69b800f040 con 0x7f69c80fbcd0 2024-03-24T10:47:19.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.112+0000 7f69cdbec700 1 -- 172.21.15.94:0/3148671508 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69c810de90 con 0x7f69c80fbcd0 2024-03-24T10:47:19.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.112+0000 7f69affff700 1 -- 172.21.15.94:0/3148671508 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69b800ca10 con 0x7f69c80fbcd0 2024-03-24T10:47:19.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.112+0000 7f69cdbec700 1 -- 172.21.15.94:0/3148671508 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69c810e3d0 con 0x7f69c80fbcd0 2024-03-24T10:47:19.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.113+0000 7f69cdbec700 1 -- 172.21.15.94:0/3148671508 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69c805f5f0 con 0x7f69c80fbcd0 2024-03-24T10:47:19.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.113+0000 7f69affff700 1 -- 172.21.15.94:0/3148671508 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) v1 ==== 89012+0+0 (secure 0 0 0) 0x7f69b800cb70 con 0x7f69c80fbcd0 2024-03-24T10:47:19.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.114+0000 7f69affff700 1 --2- 172.21.15.94:0/3148671508 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f69b006b2e0 0x7f69b006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:47:19.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.114+0000 7f69affff700 1 -- 172.21.15.94:0/3148671508 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f69b8089920 con 0x7f69c80fbcd0 2024-03-24T10:47:19.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.114+0000 7f69c6ffd700 1 --2- 172.21.15.94:0/3148671508 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f69b006b2e0 0x7f69b006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:47:19.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.114+0000 7f69c6ffd700 1 --2- 172.21.15.94:0/3148671508 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f69b006b2e0 0x7f69b006d790 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f69bc00b0d0 tx=0x7f69bc009e30 comp rx=0 tx=0).ready entity=mgr.24397 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:47:19.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.118+0000 7f69affff700 1 -- 172.21.15.94:0/3148671508 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7f69b804fb80 con 0x7f69c80fbcd0 2024-03-24T10:47:19.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.341+0000 7f69cdbec700 1 -- 172.21.15.94:0/3148671508 --> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f69c810e6b0 con 0x7f69b006b2e0 2024-03-24T10:47:19.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.343+0000 7f69affff700 1 -- 172.21.15.94:0/3148671508 <== mgr.24397 v2:172.21.15.96:6828/1632816545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f69c810e6b0 con 0x7f69b006b2e0 2024-03-24T10:47:19.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.345+0000 7f69cdbec700 1 -- 172.21.15.94:0/3148671508 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f69b006b2e0 msgr2=0x7f69b006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:19.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.345+0000 7f69cdbec700 1 --2- 172.21.15.94:0/3148671508 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f69b006b2e0 0x7f69b006d790 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f69bc00b0d0 tx=0x7f69bc009e30 comp rx=0 tx=0).stop 2024-03-24T10:47:19.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.345+0000 7f69cdbec700 1 -- 172.21.15.94:0/3148671508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69c80fbcd0 msgr2=0x7f69c810c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:19.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.345+0000 7f69cdbec700 1 --2- 172.21.15.94:0/3148671508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69c80fbcd0 0x7f69c810c880 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f69b800eb40 tx=0x7f69b800b040 comp rx=0 tx=0).stop 2024-03-24T10:47:19.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.345+0000 7f69cdbec700 1 -- 172.21.15.94:0/3148671508 shutdown_connections 2024-03-24T10:47:19.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.345+0000 7f69cdbec700 1 --2- 172.21.15.94:0/3148671508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69c80fbcd0 0x7f69c810c880 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:19.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.345+0000 7f69cdbec700 1 --2- 172.21.15.94:0/3148671508 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f69b006b2e0 0x7f69b006d790 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:19.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.345+0000 7f69cdbec700 1 --2- 172.21.15.94:0/3148671508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f69c80fcf80 0x7f69c810cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:19.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.345+0000 7f69cdbec700 1 -- 172.21.15.94:0/3148671508 >> 172.21.15.94:0/3148671508 conn(0x7f69c80f7430 msgr2=0x7f69c80f8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:47:19.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.345+0000 7f69cdbec700 1 -- 172.21.15.94:0/3148671508 shutdown_connections 2024-03-24T10:47:19.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.345+0000 7f69cdbec700 1 -- 172.21.15.94:0/3148671508 wait complete. 2024-03-24T10:47:19.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.448+0000 7f60e4704700 1 -- 172.21.15.94:0/3929971634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60dc0fff60 msgr2=0x7f60dc102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:19.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.448+0000 7f60e4704700 1 --2- 172.21.15.94:0/3929971634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60dc0fff60 0x7f60dc102340 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f60cc004940 tx=0x7f60cc009e30 comp rx=0 tx=0).stop 2024-03-24T10:47:19.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.448+0000 7f60e4704700 1 -- 172.21.15.94:0/3929971634 shutdown_connections 2024-03-24T10:47:19.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.448+0000 7f60e4704700 1 --2- 172.21.15.94:0/3929971634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60dc0fff60 0x7f60dc102340 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:19.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.448+0000 7f60e4704700 1 --2- 172.21.15.94:0/3929971634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60dc066280 0x7f60dc0ffa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:19.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.448+0000 7f60e4704700 1 -- 172.21.15.94:0/3929971634 >> 172.21.15.94:0/3929971634 conn(0x7f60dc0f7430 msgr2=0x7f60dc0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:47:19.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.448+0000 7f60e4704700 1 -- 172.21.15.94:0/3929971634 shutdown_connections 2024-03-24T10:47:19.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.448+0000 7f60e4704700 1 -- 172.21.15.94:0/3929971634 wait complete. 2024-03-24T10:47:19.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.449+0000 7f60e4704700 1 Processor -- start 2024-03-24T10:47:19.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.449+0000 7f60e4704700 1 -- start start 2024-03-24T10:47:19.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.449+0000 7f60e4704700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60dc066280 0x7f60dc10d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:47:19.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.449+0000 7f60e4704700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60dc0fff60 0x7f60dc10df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:47:19.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.449+0000 7f60e4704700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60dc10e510 con 0x7f60dc0fff60 2024-03-24T10:47:19.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.449+0000 7f60e4704700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60dc10ed60 con 0x7f60dc066280 2024-03-24T10:47:19.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.449+0000 7f60e1c9f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60dc0fff60 0x7f60dc10df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:47:19.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.449+0000 7f60e1c9f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60dc0fff60 0x7f60dc10df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33266/0 (socket says 172.21.15.94:33266) 2024-03-24T10:47:19.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.449+0000 7f60e1c9f700 1 -- 172.21.15.94:0/937750161 learned_addr learned my addr 172.21.15.94:0/937750161 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:47:19.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.449+0000 7f60e24a0700 1 --2- 172.21.15.94:0/937750161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60dc066280 0x7f60dc10d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:47:19.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.449+0000 7f60e1c9f700 1 -- 172.21.15.94:0/937750161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60dc066280 msgr2=0x7f60dc10d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:19.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.449+0000 7f60e1c9f700 1 --2- 172.21.15.94:0/937750161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60dc066280 0x7f60dc10d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:19.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.449+0000 7f60e1c9f700 1 -- 172.21.15.94:0/937750161 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60cc005040 con 0x7f60dc0fff60 2024-03-24T10:47:19.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.449+0000 7f60e1c9f700 1 --2- 172.21.15.94:0/937750161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60dc0fff60 0x7f60dc10df10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f60cc013870 tx=0x7f60cc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:47:19.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.449+0000 7f60d2ffd700 1 -- 172.21.15.94:0/937750161 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60cc021070 con 0x7f60dc0fff60 2024-03-24T10:47:19.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.449+0000 7f60d2ffd700 1 -- 172.21.15.94:0/937750161 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f60cc009430 con 0x7f60dc0fff60 2024-03-24T10:47:19.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.449+0000 7f60e4704700 1 -- 172.21.15.94:0/937750161 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60dc10efe0 con 0x7f60dc0fff60 2024-03-24T10:47:19.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.450+0000 7f60d2ffd700 1 -- 172.21.15.94:0/937750161 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60cc00e070 con 0x7f60dc0fff60 2024-03-24T10:47:19.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.450+0000 7f60e4704700 1 -- 172.21.15.94:0/937750161 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60dc10f520 con 0x7f60dc0fff60 2024-03-24T10:47:19.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.450+0000 7f60e4704700 1 -- 172.21.15.94:0/937750161 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60dc05f5f0 con 0x7f60dc0fff60 2024-03-24T10:47:19.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.451+0000 7f60d2ffd700 1 -- 172.21.15.94:0/937750161 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) v1 ==== 89012+0+0 (secure 0 0 0) 0x7f60cc01c750 con 0x7f60dc0fff60 2024-03-24T10:47:19.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.451+0000 7f60d2ffd700 1 --2- 172.21.15.94:0/937750161 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f60c806b280 0x7f60c806d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:47:19.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.451+0000 7f60d2ffd700 1 -- 172.21.15.94:0/937750161 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f60cc08afb0 con 0x7f60dc0fff60 2024-03-24T10:47:19.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.452+0000 7f60e24a0700 1 --2- 172.21.15.94:0/937750161 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f60c806b280 0x7f60c806d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:47:19.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.452+0000 7f60e24a0700 1 --2- 172.21.15.94:0/937750161 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f60c806b280 0x7f60c806d730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f60d8009b80 tx=0x7f60d800b040 comp rx=0 tx=0).ready entity=mgr.24397 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:47:19.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.456+0000 7f60d2ffd700 1 -- 172.21.15.94:0/937750161 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7f60cc0beb80 con 0x7f60dc0fff60 2024-03-24T10:47:19.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.739+0000 7f60e4704700 1 -- 172.21.15.94:0/937750161 --> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f60dc10f800 con 0x7f60c806b280 2024-03-24T10:47:19.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.759+0000 7f60d2ffd700 1 -- 172.21.15.94:0/937750161 <== mgr.24397 v2:172.21.15.96:6828/1632816545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f60dc10f800 con 0x7f60c806b280 2024-03-24T10:47:19.760 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T10:47:19.760 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.smithi094 smithi094 *:9093,9094 starting - - - - 2024-03-24T10:47:19.760 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi094 smithi094 running (9m) 38s ago 9m 6660k - 17.2.7-726-g9c4e0c34 837124da804f fb5772af46c5 2024-03-24T10:47:19.760 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi096 smithi096 running (8m) 39s ago 8m 6669k - 17.2.7-726-g9c4e0c34 837124da804f 6087ad4e0734 2024-03-24T10:47:19.761 INFO:teuthology.orchestra.run.smithi094.stdout:grafana.smithi094 smithi094 *:3000 starting - - - - 2024-03-24T10:47:19.761 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.efiqte smithi094 running (3m) 38s ago 3m 11.7M - 17.2.7-726-g9c4e0c34 837124da804f f0df0a056c21 2024-03-24T10:47:19.761 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.llefgu smithi094 running (3m) 38s ago 3m 149M - 17.2.7-726-g9c4e0c34 837124da804f 52a9771dc8e0 2024-03-24T10:47:19.761 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi096.egbmoq smithi096 running (3m) 39s ago 3m 226M - 17.2.7-726-g9c4e0c34 837124da804f 1cfc930dbab0 2024-03-24T10:47:19.761 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi096.jtebhh smithi096 running (3m) 39s ago 3m 14.8M - 17.2.7-726-g9c4e0c34 837124da804f 4ff3451a6f6e 2024-03-24T10:47:19.761 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi094.yaethu smithi094 *:9283 running (10m) 38s ago 10m 428M - 17.2.7-726-g9c4e0c34 837124da804f 327d527697e0 2024-03-24T10:47:19.761 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi096.gfonnu smithi096 *:8443,9283 running (76s) 39s ago 8m 488M - 19.0.0-1855-g21b3bd67 553463a90fd5 b6483bc88c49 2024-03-24T10:47:19.761 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi094 smithi094 running (10m) 38s ago 11m 45.9M 2048M 17.2.7-726-g9c4e0c34 837124da804f 76584fceb32b 2024-03-24T10:47:19.761 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi096 smithi096 running (7m) 39s ago 7m 39.0M 2048M 17.2.7-726-g9c4e0c34 837124da804f 710030b222cb 2024-03-24T10:47:19.761 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi094 smithi094 *:9100 starting - - - - 2024-03-24T10:47:19.761 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi096 smithi096 *:9100 running (8m) 39s ago 8m 15.6M - 1.5.0 0da6a335fe13 fc629282835f 2024-03-24T10:47:19.761 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (7m) 38s ago 7m 224M 1557M 17.2.7-726-g9c4e0c34 837124da804f b664fd3557bd 2024-03-24T10:47:19.761 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi094 running (6m) 38s ago 6m 226M 1557M 17.2.7-726-g9c4e0c34 837124da804f 4e737a80d733 2024-03-24T10:47:19.761 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi094 running (6m) 38s ago 6m 188M 1557M 17.2.7-726-g9c4e0c34 837124da804f cf0aefd20c76 2024-03-24T10:47:19.762 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi096 running (5m) 39s ago 5m 248M 2581M 17.2.7-726-g9c4e0c34 837124da804f f4650d0f6241 2024-03-24T10:47:19.762 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi096 running (5m) 39s ago 5m 216M 2581M 17.2.7-726-g9c4e0c34 837124da804f a63d7c9e705f 2024-03-24T10:47:19.762 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi096 running (5m) 39s ago 5m 240M 2581M 17.2.7-726-g9c4e0c34 837124da804f f56fad2959c3 2024-03-24T10:47:19.762 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.smithi094 smithi094 *:9095 starting - - - - 2024-03-24T10:47:19.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.762+0000 7f60e4704700 1 -- 172.21.15.94:0/937750161 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f60c806b280 msgr2=0x7f60c806d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:19.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.762+0000 7f60e4704700 1 --2- 172.21.15.94:0/937750161 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f60c806b280 0x7f60c806d730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f60d8009b80 tx=0x7f60d800b040 comp rx=0 tx=0).stop 2024-03-24T10:47:19.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.762+0000 7f60e4704700 1 -- 172.21.15.94:0/937750161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60dc0fff60 msgr2=0x7f60dc10df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:19.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.762+0000 7f60e4704700 1 --2- 172.21.15.94:0/937750161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60dc0fff60 0x7f60dc10df10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f60cc013870 tx=0x7f60cc009e30 comp rx=0 tx=0).stop 2024-03-24T10:47:19.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.762+0000 7f60e4704700 1 -- 172.21.15.94:0/937750161 shutdown_connections 2024-03-24T10:47:19.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.762+0000 7f60e4704700 1 --2- 172.21.15.94:0/937750161 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f60c806b280 0x7f60c806d730 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:19.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.762+0000 7f60e4704700 1 --2- 172.21.15.94:0/937750161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60dc066280 0x7f60dc10d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:19.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.762+0000 7f60e4704700 1 --2- 172.21.15.94:0/937750161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60dc0fff60 0x7f60dc10df10 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:19.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.762+0000 7f60e4704700 1 -- 172.21.15.94:0/937750161 >> 172.21.15.94:0/937750161 conn(0x7f60dc0f7430 msgr2=0x7f60dc0f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:47:19.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.762+0000 7f60e4704700 1 -- 172.21.15.94:0/937750161 shutdown_connections 2024-03-24T10:47:19.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.762+0000 7f60e4704700 1 -- 172.21.15.94:0/937750161 wait complete. 2024-03-24T10:47:19.884 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:19 smithi096 ceph-mon[38097]: Reconfiguring node-exporter.smithi096 (dependencies changed)... 2024-03-24T10:47:19.884 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:19 smithi096 ceph-mon[38097]: Reconfiguring daemon node-exporter.smithi096 on smithi096 2024-03-24T10:47:19.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.884+0000 7f9d53a84700 1 -- 172.21.15.94:0/1148426532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d4c066240 msgr2=0x7f9d4c0ffa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:19.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.884+0000 7f9d53a84700 1 --2- 172.21.15.94:0/1148426532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d4c066240 0x7f9d4c0ffa60 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9d48004940 tx=0x7f9d48009e30 comp rx=0 tx=0).stop 2024-03-24T10:47:19.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.884+0000 7f9d53a84700 1 -- 172.21.15.94:0/1148426532 shutdown_connections 2024-03-24T10:47:19.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.884+0000 7f9d53a84700 1 --2- 172.21.15.94:0/1148426532 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d4c0fffa0 0x7f9d4c102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:19.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.884+0000 7f9d53a84700 1 --2- 172.21.15.94:0/1148426532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d4c066240 0x7f9d4c0ffa60 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:19.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.884+0000 7f9d53a84700 1 -- 172.21.15.94:0/1148426532 >> 172.21.15.94:0/1148426532 conn(0x7f9d4c0f7430 msgr2=0x7f9d4c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:47:19.885 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:19 smithi094 ceph-mon[29446]: Reconfiguring node-exporter.smithi096 (dependencies changed)... 2024-03-24T10:47:19.885 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:19 smithi094 ceph-mon[29446]: Reconfiguring daemon node-exporter.smithi096 on smithi096 2024-03-24T10:47:19.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.884+0000 7f9d53a84700 1 -- 172.21.15.94:0/1148426532 shutdown_connections 2024-03-24T10:47:19.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.884+0000 7f9d53a84700 1 -- 172.21.15.94:0/1148426532 wait complete. 2024-03-24T10:47:19.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.885+0000 7f9d53a84700 1 Processor -- start 2024-03-24T10:47:19.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.885+0000 7f9d53a84700 1 -- start start 2024-03-24T10:47:19.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.885+0000 7f9d53a84700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d4c066240 0x7f9d4c10da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:47:19.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.885+0000 7f9d53a84700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d4c0fffa0 0x7f9d4c10dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:47:19.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.885+0000 7f9d53a84700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4c10e5c0 con 0x7f9d4c066240 2024-03-24T10:47:19.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.885+0000 7f9d53a84700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4c10ed70 con 0x7f9d4c0fffa0 2024-03-24T10:47:19.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.885+0000 7f9d51820700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d4c066240 0x7f9d4c10da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:47:19.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.885+0000 7f9d51820700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d4c066240 0x7f9d4c10da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43648/0 (socket says 172.21.15.94:43648) 2024-03-24T10:47:19.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.885+0000 7f9d51820700 1 -- 172.21.15.94:0/468095594 learned_addr learned my addr 172.21.15.94:0/468095594 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:47:19.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.885+0000 7f9d5101f700 1 --2- 172.21.15.94:0/468095594 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d4c0fffa0 0x7f9d4c10dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:47:19.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.886+0000 7f9d51820700 1 -- 172.21.15.94:0/468095594 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d4c0fffa0 msgr2=0x7f9d4c10dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:19.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.886+0000 7f9d51820700 1 --2- 172.21.15.94:0/468095594 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d4c0fffa0 0x7f9d4c10dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:19.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.886+0000 7f9d51820700 1 -- 172.21.15.94:0/468095594 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d48005040 con 0x7f9d4c066240 2024-03-24T10:47:19.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.886+0000 7f9d51820700 1 --2- 172.21.15.94:0/468095594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d4c066240 0x7f9d4c10da80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9d48000c00 tx=0x7f9d48009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:47:19.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.886+0000 7f9d427fc700 1 -- 172.21.15.94:0/468095594 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d4800e030 con 0x7f9d4c066240 2024-03-24T10:47:19.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.886+0000 7f9d427fc700 1 -- 172.21.15.94:0/468095594 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9d48013de0 con 0x7f9d4c066240 2024-03-24T10:47:19.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.886+0000 7f9d427fc700 1 -- 172.21.15.94:0/468095594 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d4800b410 con 0x7f9d4c066240 2024-03-24T10:47:19.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.886+0000 7f9d53a84700 1 -- 172.21.15.94:0/468095594 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d4c10eff0 con 0x7f9d4c066240 2024-03-24T10:47:19.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.886+0000 7f9d53a84700 1 -- 172.21.15.94:0/468095594 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d4c10f530 con 0x7f9d4c066240 2024-03-24T10:47:19.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.888+0000 7f9d53a84700 1 -- 172.21.15.94:0/468095594 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d4c05f5f0 con 0x7f9d4c066240 2024-03-24T10:47:19.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.889+0000 7f9d427fc700 1 -- 172.21.15.94:0/468095594 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) v1 ==== 89012+0+0 (secure 0 0 0) 0x7f9d4801c770 con 0x7f9d4c066240 2024-03-24T10:47:19.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.889+0000 7f9d427fc700 1 --2- 172.21.15.94:0/468095594 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f9d3806b2e0 0x7f9d3806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:47:19.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.889+0000 7f9d427fc700 1 -- 172.21.15.94:0/468095594 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f9d4808b730 con 0x7f9d4c066240 2024-03-24T10:47:19.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.889+0000 7f9d5101f700 1 --2- 172.21.15.94:0/468095594 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f9d3806b2e0 0x7f9d3806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:47:19.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.890+0000 7f9d5101f700 1 --2- 172.21.15.94:0/468095594 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f9d3806b2e0 0x7f9d3806d790 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c009b80 tx=0x7f9d3c00b040 comp rx=0 tx=0).ready entity=mgr.24397 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:47:19.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:19.897+0000 7f9d427fc700 1 -- 172.21.15.94:0/468095594 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7f9d48059390 con 0x7f9d4c066240 2024-03-24T10:47:20.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.177+0000 7f9d53a84700 1 -- 172.21.15.94:0/468095594 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9d4c10f7a0 con 0x7f9d4c066240 2024-03-24T10:47:20.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.178+0000 7f9d427fc700 1 -- 172.21.15.94:0/468095594 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7f9d4801c400 con 0x7f9d4c066240 2024-03-24T10:47:20.179 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-03-24T10:47:20.179 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": { 2024-03-24T10:47:20.179 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-24T10:47:20.179 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:47:20.179 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": { 2024-03-24T10:47:20.179 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 1, 2024-03-24T10:47:20.179 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 1 2024-03-24T10:47:20.179 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:47:20.179 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": { 2024-03-24T10:47:20.179 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-24T10:47:20.179 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:47:20.179 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-03-24T10:47:20.180 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-24T10:47:20.180 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:47:20.180 INFO:teuthology.orchestra.run.smithi094.stdout: "overall": { 2024-03-24T10:47:20.180 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 13, 2024-03-24T10:47:20.180 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 1 2024-03-24T10:47:20.180 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-03-24T10:47:20.180 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-03-24T10:47:20.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.180+0000 7f9d53a84700 1 -- 172.21.15.94:0/468095594 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f9d3806b2e0 msgr2=0x7f9d3806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:20.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.180+0000 7f9d53a84700 1 --2- 172.21.15.94:0/468095594 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f9d3806b2e0 0x7f9d3806d790 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c009b80 tx=0x7f9d3c00b040 comp rx=0 tx=0).stop 2024-03-24T10:47:20.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.180+0000 7f9d53a84700 1 -- 172.21.15.94:0/468095594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d4c066240 msgr2=0x7f9d4c10da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:20.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.180+0000 7f9d53a84700 1 --2- 172.21.15.94:0/468095594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d4c066240 0x7f9d4c10da80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9d48000c00 tx=0x7f9d48009e30 comp rx=0 tx=0).stop 2024-03-24T10:47:20.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.180+0000 7f9d53a84700 1 -- 172.21.15.94:0/468095594 shutdown_connections 2024-03-24T10:47:20.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.180+0000 7f9d53a84700 1 --2- 172.21.15.94:0/468095594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d4c066240 0x7f9d4c10da80 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:20.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.180+0000 7f9d53a84700 1 --2- 172.21.15.94:0/468095594 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f9d3806b2e0 0x7f9d3806d790 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:20.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.180+0000 7f9d53a84700 1 --2- 172.21.15.94:0/468095594 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d4c0fffa0 0x7f9d4c10dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:20.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.180+0000 7f9d53a84700 1 -- 172.21.15.94:0/468095594 >> 172.21.15.94:0/468095594 conn(0x7f9d4c0f7430 msgr2=0x7f9d4c0f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:47:20.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.180+0000 7f9d53a84700 1 -- 172.21.15.94:0/468095594 shutdown_connections 2024-03-24T10:47:20.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.180+0000 7f9d53a84700 1 -- 172.21.15.94:0/468095594 wait complete. 2024-03-24T10:47:20.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.281+0000 7f6084b12700 1 -- 172.21.15.94:0/1790230632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60800fcf80 msgr2=0x7f60800fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:20.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.281+0000 7f6084b12700 1 --2- 172.21.15.94:0/1790230632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60800fcf80 0x7f60800fd3f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6070004940 tx=0x7f6070009e30 comp rx=0 tx=0).stop 2024-03-24T10:47:20.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.281+0000 7f6084b12700 1 -- 172.21.15.94:0/1790230632 shutdown_connections 2024-03-24T10:47:20.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.281+0000 7f6084b12700 1 --2- 172.21.15.94:0/1790230632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60800fcf80 0x7f60800fd3f0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:20.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.281+0000 7f6084b12700 1 --2- 172.21.15.94:0/1790230632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60800fbcd0 0x7f60800fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:20.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.281+0000 7f6084b12700 1 -- 172.21.15.94:0/1790230632 >> 172.21.15.94:0/1790230632 conn(0x7f60800f7430 msgr2=0x7f60800f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:47:20.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.281+0000 7f6084b12700 1 -- 172.21.15.94:0/1790230632 shutdown_connections 2024-03-24T10:47:20.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.281+0000 7f6084b12700 1 -- 172.21.15.94:0/1790230632 wait complete. 2024-03-24T10:47:20.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.282+0000 7f6084b12700 1 Processor -- start 2024-03-24T10:47:20.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.282+0000 7f6084b12700 1 -- start start 2024-03-24T10:47:20.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.282+0000 7f6084b12700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60800fbcd0 0x7f608010c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:47:20.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.282+0000 7f6084b12700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60800fcf80 0x7f608010cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:47:20.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.282+0000 7f6084b12700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608010d300 con 0x7f60800fcf80 2024-03-24T10:47:20.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.282+0000 7f6084b12700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608010d470 con 0x7f60800fbcd0 2024-03-24T10:47:20.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.282+0000 7f607dd9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60800fcf80 0x7f608010cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:47:20.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.282+0000 7f607dd9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60800fcf80 0x7f608010cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43652/0 (socket says 172.21.15.94:43652) 2024-03-24T10:47:20.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.282+0000 7f607dd9b700 1 -- 172.21.15.94:0/2494727745 learned_addr learned my addr 172.21.15.94:0/2494727745 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:47:20.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.282+0000 7f607e59c700 1 --2- 172.21.15.94:0/2494727745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60800fbcd0 0x7f608010c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:47:20.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.282+0000 7f607dd9b700 1 -- 172.21.15.94:0/2494727745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60800fbcd0 msgr2=0x7f608010c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:20.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.282+0000 7f607dd9b700 1 --2- 172.21.15.94:0/2494727745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60800fbcd0 0x7f608010c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:20.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.282+0000 7f607dd9b700 1 -- 172.21.15.94:0/2494727745 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6070005040 con 0x7f60800fcf80 2024-03-24T10:47:20.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.283+0000 7f607dd9b700 1 --2- 172.21.15.94:0/2494727745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60800fcf80 0x7f608010cd00 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6070013870 tx=0x7f6070009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:47:20.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.283+0000 7f606effd700 1 -- 172.21.15.94:0/2494727745 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6070021070 con 0x7f60800fcf80 2024-03-24T10:47:20.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.283+0000 7f606effd700 1 -- 172.21.15.94:0/2494727745 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6070009430 con 0x7f60800fcf80 2024-03-24T10:47:20.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.283+0000 7f6084b12700 1 -- 172.21.15.94:0/2494727745 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6080111e70 con 0x7f60800fcf80 2024-03-24T10:47:20.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.283+0000 7f606effd700 1 -- 172.21.15.94:0/2494727745 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f607000e070 con 0x7f60800fcf80 2024-03-24T10:47:20.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.283+0000 7f6084b12700 1 -- 172.21.15.94:0/2494727745 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6080112380 con 0x7f60800fcf80 2024-03-24T10:47:20.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.284+0000 7f6084b12700 1 -- 172.21.15.94:0/2494727745 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f608004c710 con 0x7f60800fcf80 2024-03-24T10:47:20.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.284+0000 7f606effd700 1 -- 172.21.15.94:0/2494727745 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) v1 ==== 89012+0+0 (secure 0 0 0) 0x7f607001c750 con 0x7f60800fcf80 2024-03-24T10:47:20.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.285+0000 7f606effd700 1 --2- 172.21.15.94:0/2494727745 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f606806b280 0x7f606806d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:47:20.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.285+0000 7f606effd700 1 -- 172.21.15.94:0/2494727745 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f607008afe0 con 0x7f60800fcf80 2024-03-24T10:47:20.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.285+0000 7f607e59c700 1 --2- 172.21.15.94:0/2494727745 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f606806b280 0x7f606806d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:47:20.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.285+0000 7f607e59c700 1 --2- 172.21.15.94:0/2494727745 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f606806b280 0x7f606806d730 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6074009b80 tx=0x7f607400b040 comp rx=0 tx=0).ready entity=mgr.24397 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:47:20.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.289+0000 7f606effd700 1 -- 172.21.15.94:0/2494727745 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7f6070058c40 con 0x7f60800fcf80 2024-03-24T10:47:20.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.531+0000 7f6084b12700 1 -- 172.21.15.94:0/2494727745 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f60801047c0 con 0x7f60800fcf80 2024-03-24T10:47:20.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.532+0000 7f606effd700 1 -- 172.21.15.94:0/2494727745 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1833 (secure 0 0 0) 0x7f60700587d0 con 0x7f60800fcf80 2024-03-24T10:47:20.532 INFO:teuthology.orchestra.run.smithi094.stdout:e18 2024-03-24T10:47:20.533 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T10:47:20.533 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:47:20.533 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-03-24T10:47:20.533 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:47:20.533 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-03-24T10:47:20.533 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-03-24T10:47:20.533 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 18 2024-03-24T10:47:20.533 INFO:teuthology.orchestra.run.smithi094.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-24T10:47:20.533 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-03-24T10:43:39.435738+0000 2024-03-24T10:47:20.533 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-03-24T10:43:58.133901+0000 2024-03-24T10:47:20.533 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-03-24T10:47:20.533 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-03-24T10:47:20.533 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 60 2024-03-24T10:47:20.533 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-03-24T10:47:20.533 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-03-24T10:47:20.533 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-03-24T10:47:20.534 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-03-24T10:47:20.534 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 46 2024-03-24T10:47:20.534 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:47:20.534 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-03-24T10:47:20.534 INFO:teuthology.orchestra.run.smithi094.stdout:in 0 2024-03-24T10:47:20.534 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=24251} 2024-03-24T10:47:20.534 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-03-24T10:47:20.534 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-03-24T10:47:20.534 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-03-24T10:47:20.534 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-03-24T10:47:20.534 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-03-24T10:47:20.534 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-03-24T10:47:20.534 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-03-24T10:47:20.534 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 1 2024-03-24T10:47:20.535 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi096.egbmoq{0:24251} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.96:6824/1539752208,v1:172.21.15.96:6825/1539752208] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:47:20.535 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.llefgu{0:14500} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.94:6828/126762980,v1:172.21.15.94:6829/126762980] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:47:20.535 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:47:20.535 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:47:20.535 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-03-24T10:47:20.535 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:47:20.535 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.efiqte{-1:14512} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.94:6826/1636528860,v1:172.21.15.94:6827/1636528860] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:47:20.535 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi096.jtebhh{-1:24261} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.96:6826/1321087930,v1:172.21.15.96:6827/1321087930] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:47:20.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.534+0000 7f6084b12700 1 -- 172.21.15.94:0/2494727745 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f606806b280 msgr2=0x7f606806d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:20.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.534+0000 7f6084b12700 1 --2- 172.21.15.94:0/2494727745 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f606806b280 0x7f606806d730 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6074009b80 tx=0x7f607400b040 comp rx=0 tx=0).stop 2024-03-24T10:47:20.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.534+0000 7f6084b12700 1 -- 172.21.15.94:0/2494727745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60800fcf80 msgr2=0x7f608010cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:20.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.534+0000 7f6084b12700 1 --2- 172.21.15.94:0/2494727745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60800fcf80 0x7f608010cd00 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6070013870 tx=0x7f6070009e30 comp rx=0 tx=0).stop 2024-03-24T10:47:20.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.534+0000 7f6084b12700 1 -- 172.21.15.94:0/2494727745 shutdown_connections 2024-03-24T10:47:20.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.534+0000 7f6084b12700 1 --2- 172.21.15.94:0/2494727745 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f606806b280 0x7f606806d730 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:20.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.534+0000 7f6084b12700 1 --2- 172.21.15.94:0/2494727745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60800fbcd0 0x7f608010c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:20.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.534+0000 7f6084b12700 1 --2- 172.21.15.94:0/2494727745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60800fcf80 0x7f608010cd00 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:20.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.534+0000 7f6084b12700 1 -- 172.21.15.94:0/2494727745 >> 172.21.15.94:0/2494727745 conn(0x7f60800f7430 msgr2=0x7f60800f8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:47:20.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.534+0000 7f6084b12700 1 -- 172.21.15.94:0/2494727745 shutdown_connections 2024-03-24T10:47:20.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.534+0000 7f6084b12700 1 -- 172.21.15.94:0/2494727745 wait complete. 2024-03-24T10:47:20.537 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 18 2024-03-24T10:47:20.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.663+0000 7fdba48d4700 1 -- 172.21.15.94:0/4106153398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdba00fcf90 msgr2=0x7fdba00fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:20.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.663+0000 7fdba48d4700 1 --2- 172.21.15.94:0/4106153398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdba00fcf90 0x7fdba00fd400 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fdb94004940 tx=0x7fdb94009e30 comp rx=0 tx=0).stop 2024-03-24T10:47:20.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.663+0000 7fdba48d4700 1 -- 172.21.15.94:0/4106153398 shutdown_connections 2024-03-24T10:47:20.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.663+0000 7fdba48d4700 1 --2- 172.21.15.94:0/4106153398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdba00fcf90 0x7fdba00fd400 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:20.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.663+0000 7fdba48d4700 1 --2- 172.21.15.94:0/4106153398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdba00fbce0 0x7fdba00fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:20.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.663+0000 7fdba48d4700 1 -- 172.21.15.94:0/4106153398 >> 172.21.15.94:0/4106153398 conn(0x7fdba00f7440 msgr2=0x7fdba00f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:47:20.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.663+0000 7fdba48d4700 1 -- 172.21.15.94:0/4106153398 shutdown_connections 2024-03-24T10:47:20.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.663+0000 7fdba48d4700 1 -- 172.21.15.94:0/4106153398 wait complete. 2024-03-24T10:47:20.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.663+0000 7fdba48d4700 1 Processor -- start 2024-03-24T10:47:20.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.663+0000 7fdba48d4700 1 -- start start 2024-03-24T10:47:20.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.663+0000 7fdba48d4700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdba00fbce0 0x7fdba00ffc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:47:20.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.663+0000 7fdba48d4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdba00fcf90 0x7fdba00fe2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:47:20.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.663+0000 7fdba48d4700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba0100220 con 0x7fdba00fcf90 2024-03-24T10:47:20.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.663+0000 7fdba48d4700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba01115d0 con 0x7fdba00fbce0 2024-03-24T10:47:20.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.664+0000 7fdb9dd9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdba00fcf90 0x7fdba00fe2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:47:20.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.664+0000 7fdb9dd9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdba00fcf90 0x7fdba00fe2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43670/0 (socket says 172.21.15.94:43670) 2024-03-24T10:47:20.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.664+0000 7fdb9e59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdba00fbce0 0x7fdba00ffc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:47:20.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.664+0000 7fdb9dd9b700 1 -- 172.21.15.94:0/540184073 learned_addr learned my addr 172.21.15.94:0/540184073 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:47:20.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.664+0000 7fdb9dd9b700 1 -- 172.21.15.94:0/540184073 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdba00fbce0 msgr2=0x7fdba00ffc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:20.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.664+0000 7fdb9dd9b700 1 --2- 172.21.15.94:0/540184073 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdba00fbce0 0x7fdba00ffc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:20.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.664+0000 7fdb9dd9b700 1 -- 172.21.15.94:0/540184073 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb94005040 con 0x7fdba00fcf90 2024-03-24T10:47:20.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.664+0000 7fdb9e59c700 1 --2- 172.21.15.94:0/540184073 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdba00fbce0 0x7fdba00ffc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T10:47:20.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.664+0000 7fdb9dd9b700 1 --2- 172.21.15.94:0/540184073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdba00fcf90 0x7fdba00fe2d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdb94013870 tx=0x7fdb94009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:47:20.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.664+0000 7fdb8effd700 1 -- 172.21.15.94:0/540184073 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb94021070 con 0x7fdba00fcf90 2024-03-24T10:47:20.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.664+0000 7fdb8effd700 1 -- 172.21.15.94:0/540184073 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdb94009430 con 0x7fdba00fcf90 2024-03-24T10:47:20.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.664+0000 7fdba48d4700 1 -- 172.21.15.94:0/540184073 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdba0111820 con 0x7fdba00fcf90 2024-03-24T10:47:20.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.664+0000 7fdb8effd700 1 -- 172.21.15.94:0/540184073 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb9400e070 con 0x7fdba00fcf90 2024-03-24T10:47:20.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.664+0000 7fdba48d4700 1 -- 172.21.15.94:0/540184073 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdba00febc0 con 0x7fdba00fcf90 2024-03-24T10:47:20.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.665+0000 7fdba48d4700 1 -- 172.21.15.94:0/540184073 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdba0101b20 con 0x7fdba00fcf90 2024-03-24T10:47:20.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.666+0000 7fdb8effd700 1 -- 172.21.15.94:0/540184073 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) v1 ==== 89012+0+0 (secure 0 0 0) 0x7fdb9401c750 con 0x7fdba00fcf90 2024-03-24T10:47:20.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.666+0000 7fdb8effd700 1 --2- 172.21.15.94:0/540184073 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7fdb8806b2f0 0x7fdb8806d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:47:20.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.667+0000 7fdb8effd700 1 -- 172.21.15.94:0/540184073 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fdb9408acd0 con 0x7fdba00fcf90 2024-03-24T10:47:20.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.667+0000 7fdb9e59c700 1 --2- 172.21.15.94:0/540184073 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7fdb8806b2f0 0x7fdb8806d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:47:20.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.667+0000 7fdb9e59c700 1 --2- 172.21.15.94:0/540184073 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7fdb8806b2f0 0x7fdb8806d7a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdb90004fd0 tx=0x7fdb9000b040 comp rx=0 tx=0).ready entity=mgr.24397 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:47:20.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.671+0000 7fdb8effd700 1 -- 172.21.15.94:0/540184073 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7fdb940589f0 con 0x7fdba00fcf90 2024-03-24T10:47:20.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:20 smithi096 ceph-mon[38097]: from='client.14746 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:47:20.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:20 smithi096 ceph-mon[38097]: from='client.14750 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:47:20.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:20 smithi096 ceph-mon[38097]: pgmap v28: 65 pgs: 65 active+clean; 311 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 1.18k op/s 2024-03-24T10:47:20.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:20 smithi096 ceph-mon[38097]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:47:20.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:20 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/468095594' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:47:20.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:20 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:47:20.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:20 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:47:20.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:20 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/2494727745' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T10:47:20.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:20 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-24T10:47:20.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:20 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-24T10:47:20.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:20 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T10:47:20.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:20 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:47:20.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.982+0000 7fdba48d4700 1 -- 172.21.15.94:0/540184073 --> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdba0111820 con 0x7fdb8806b2f0 2024-03-24T10:47:20.983 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:20 smithi094 ceph-mon[29446]: from='client.14746 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:47:20.983 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:20 smithi094 ceph-mon[29446]: from='client.14750 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:47:20.983 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:20 smithi094 ceph-mon[29446]: pgmap v28: 65 pgs: 65 active+clean; 311 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 1.18k op/s 2024-03-24T10:47:20.983 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:20 smithi094 ceph-mon[29446]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:47:20.983 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:20 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/468095594' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:47:20.983 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:20 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:47:20.983 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:20 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:47:20.983 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:20 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/2494727745' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T10:47:20.983 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:20 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-24T10:47:20.983 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:20 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-24T10:47:20.983 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:20 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T10:47:20.983 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:20 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:47:20.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.983+0000 7fdb8effd700 1 -- 172.21.15.94:0/540184073 <== mgr.24397 v2:172.21.15.96:6828/1632816545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7fdba0111820 con 0x7fdb8806b2f0 2024-03-24T10:47:20.984 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-03-24T10:47:20.984 INFO:teuthology.orchestra.run.smithi094.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919", 2024-03-24T10:47:20.984 INFO:teuthology.orchestra.run.smithi094.stdout: "in_progress": true, 2024-03-24T10:47:20.984 INFO:teuthology.orchestra.run.smithi094.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T10:47:20.984 INFO:teuthology.orchestra.run.smithi094.stdout: "services_complete": [], 2024-03-24T10:47:20.984 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": "1/21 daemons upgraded", 2024-03-24T10:47:20.984 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "Currently upgrading mgr daemons", 2024-03-24T10:47:20.984 INFO:teuthology.orchestra.run.smithi094.stdout: "is_paused": false 2024-03-24T10:47:20.984 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-03-24T10:47:20.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.986+0000 7fdba48d4700 1 -- 172.21.15.94:0/540184073 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7fdb8806b2f0 msgr2=0x7fdb8806d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:20.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.986+0000 7fdba48d4700 1 --2- 172.21.15.94:0/540184073 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7fdb8806b2f0 0x7fdb8806d7a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdb90004fd0 tx=0x7fdb9000b040 comp rx=0 tx=0).stop 2024-03-24T10:47:20.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.986+0000 7fdba48d4700 1 -- 172.21.15.94:0/540184073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdba00fcf90 msgr2=0x7fdba00fe2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:20.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.986+0000 7fdba48d4700 1 --2- 172.21.15.94:0/540184073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdba00fcf90 0x7fdba00fe2d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdb94013870 tx=0x7fdb94009e30 comp rx=0 tx=0).stop 2024-03-24T10:47:20.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.986+0000 7fdba48d4700 1 -- 172.21.15.94:0/540184073 shutdown_connections 2024-03-24T10:47:20.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.986+0000 7fdba48d4700 1 --2- 172.21.15.94:0/540184073 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7fdb8806b2f0 0x7fdb8806d7a0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:20.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.986+0000 7fdba48d4700 1 --2- 172.21.15.94:0/540184073 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdba00fbce0 0x7fdba00ffc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:20.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.986+0000 7fdba48d4700 1 --2- 172.21.15.94:0/540184073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdba00fcf90 0x7fdba00fe2d0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:20.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.986+0000 7fdba48d4700 1 -- 172.21.15.94:0/540184073 >> 172.21.15.94:0/540184073 conn(0x7fdba00f7440 msgr2=0x7fdba00667c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:47:20.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.987+0000 7fdba48d4700 1 -- 172.21.15.94:0/540184073 shutdown_connections 2024-03-24T10:47:20.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:20.987+0000 7fdba48d4700 1 -- 172.21.15.94:0/540184073 wait complete. 2024-03-24T10:47:21.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:21.099+0000 7f6673769700 1 -- 172.21.15.94:0/3948526757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f666c0fe040 msgr2=0x7f666c0fe4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:21.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:21.099+0000 7f6673769700 1 --2- 172.21.15.94:0/3948526757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f666c0fe040 0x7f666c0fe4b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f665c004940 tx=0x7f665c009e30 comp rx=0 tx=0).stop 2024-03-24T10:47:21.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:21.099+0000 7f6673769700 1 -- 172.21.15.94:0/3948526757 shutdown_connections 2024-03-24T10:47:21.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:21.099+0000 7f6673769700 1 --2- 172.21.15.94:0/3948526757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f666c0fe040 0x7f666c0fe4b0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:21.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:21.099+0000 7f6673769700 1 --2- 172.21.15.94:0/3948526757 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f666c0fce40 0x7f666c0fd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:21.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:21.099+0000 7f6673769700 1 -- 172.21.15.94:0/3948526757 >> 172.21.15.94:0/3948526757 conn(0x7f666c0f8570 msgr2=0x7f666c0fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:47:21.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:21.099+0000 7f6673769700 1 -- 172.21.15.94:0/3948526757 shutdown_connections 2024-03-24T10:47:21.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:21.099+0000 7f6673769700 1 -- 172.21.15.94:0/3948526757 wait complete. 2024-03-24T10:47:21.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:21.099+0000 7f6673769700 1 Processor -- start 2024-03-24T10:47:21.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:21.100+0000 7f6673769700 1 -- start start 2024-03-24T10:47:21.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:21.100+0000 7f6673769700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f666c0fce40 0x7f666c071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:47:21.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:21.100+0000 7f6673769700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f666c0fe040 0x7f666c070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:47:21.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:21.100+0000 7f6673769700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f666c0708c0 con 0x7f666c0fe040 2024-03-24T10:47:21.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:21.100+0000 7f6673769700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f666c070a30 con 0x7f666c0fce40 2024-03-24T10:47:21.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:21.100+0000 7f6670d04700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f666c0fe040 0x7f666c070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:47:21.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:21.100+0000 7f6670d04700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f666c0fe040 0x7f666c070310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43680/0 (socket says 172.21.15.94:43680) 2024-03-24T10:47:21.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:21.100+0000 7f6670d04700 1 -- 172.21.15.94:0/2425491342 learned_addr learned my addr 172.21.15.94:0/2425491342 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:47:21.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:21.100+0000 7f6671505700 1 --2- 172.21.15.94:0/2425491342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f666c0fce40 0x7f666c071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:47:21.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:21.100+0000 7f6670d04700 1 -- 172.21.15.94:0/2425491342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f666c0fce40 msgr2=0x7f666c071e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:21.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:21.100+0000 7f6670d04700 1 --2- 172.21.15.94:0/2425491342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f666c0fce40 0x7f666c071e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:21.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:21.100+0000 7f6670d04700 1 -- 172.21.15.94:0/2425491342 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f665c005040 con 0x7f666c0fe040 2024-03-24T10:47:21.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:21.100+0000 7f6670d04700 1 --2- 172.21.15.94:0/2425491342 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f666c0fe040 0x7f666c070310 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f665c013870 tx=0x7f665c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:47:21.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:21.100+0000 7f6661ffb700 1 -- 172.21.15.94:0/2425491342 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f665c021070 con 0x7f666c0fe040 2024-03-24T10:47:21.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:21.100+0000 7f6661ffb700 1 -- 172.21.15.94:0/2425491342 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f665c009430 con 0x7f666c0fe040 2024-03-24T10:47:21.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:21.100+0000 7f6673769700 1 -- 172.21.15.94:0/2425491342 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f666c070cb0 con 0x7f666c0fe040 2024-03-24T10:47:21.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:21.100+0000 7f6661ffb700 1 -- 172.21.15.94:0/2425491342 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f665c00e070 con 0x7f666c0fe040 2024-03-24T10:47:21.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:21.101+0000 7f6673769700 1 -- 172.21.15.94:0/2425491342 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f666c1159e0 con 0x7f666c0fe040 2024-03-24T10:47:21.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:21.102+0000 7f6673769700 1 -- 172.21.15.94:0/2425491342 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f666c05f5f0 con 0x7f666c0fe040 2024-03-24T10:47:21.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:21.102+0000 7f6661ffb700 1 -- 172.21.15.94:0/2425491342 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) v1 ==== 89012+0+0 (secure 0 0 0) 0x7f665c026410 con 0x7f666c0fe040 2024-03-24T10:47:21.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:21.103+0000 7f6661ffb700 1 --2- 172.21.15.94:0/2425491342 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f665806b280 0x7f665806d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:47:21.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:21.103+0000 7f6661ffb700 1 -- 172.21.15.94:0/2425491342 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f665c08af70 con 0x7f666c0fe040 2024-03-24T10:47:21.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:21.103+0000 7f6671505700 1 --2- 172.21.15.94:0/2425491342 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f665806b280 0x7f665806d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:47:21.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:21.103+0000 7f6671505700 1 --2- 172.21.15.94:0/2425491342 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f665806b280 0x7f665806d730 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f666c071bd0 tx=0x7f666800b040 comp rx=0 tx=0).ready entity=mgr.24397 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:47:21.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:21.107+0000 7f6661ffb700 1 -- 172.21.15.94:0/2425491342 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7f665c0beb80 con 0x7f666c0fe040 2024-03-24T10:47:21.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:21.401+0000 7f6673769700 1 -- 172.21.15.94:0/2425491342 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f666c04bed0 con 0x7f666c0fe040 2024-03-24T10:47:21.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:21.402+0000 7f6661ffb700 1 -- 172.21.15.94:0/2425491342 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f665c0550a0 con 0x7f666c0fe040 2024-03-24T10:47:21.403 INFO:teuthology.orchestra.run.smithi094.stdout:HEALTH_OK 2024-03-24T10:47:21.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:21.404+0000 7f6673769700 1 -- 172.21.15.94:0/2425491342 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f665806b280 msgr2=0x7f665806d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:21.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:21.404+0000 7f6673769700 1 --2- 172.21.15.94:0/2425491342 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f665806b280 0x7f665806d730 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f666c071bd0 tx=0x7f666800b040 comp rx=0 tx=0).stop 2024-03-24T10:47:21.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:21.404+0000 7f6673769700 1 -- 172.21.15.94:0/2425491342 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f666c0fe040 msgr2=0x7f666c070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:21.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:21.404+0000 7f6673769700 1 --2- 172.21.15.94:0/2425491342 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f666c0fe040 0x7f666c070310 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f665c013870 tx=0x7f665c009e30 comp rx=0 tx=0).stop 2024-03-24T10:47:21.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:21.404+0000 7f6673769700 1 -- 172.21.15.94:0/2425491342 shutdown_connections 2024-03-24T10:47:21.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:21.404+0000 7f6673769700 1 --2- 172.21.15.94:0/2425491342 >> [v2:172.21.15.96:6828/1632816545,v1:172.21.15.96:6829/1632816545] conn(0x7f665806b280 0x7f665806d730 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:21.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:21.404+0000 7f6673769700 1 --2- 172.21.15.94:0/2425491342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f666c0fce40 0x7f666c071e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:21.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:21.404+0000 7f6673769700 1 --2- 172.21.15.94:0/2425491342 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f666c0fe040 0x7f666c070310 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:21.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:21.404+0000 7f6673769700 1 -- 172.21.15.94:0/2425491342 >> 172.21.15.94:0/2425491342 conn(0x7f666c0f8570 msgr2=0x7f666c0f9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:47:21.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:21.404+0000 7f6673769700 1 -- 172.21.15.94:0/2425491342 shutdown_connections 2024-03-24T10:47:21.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:21.404+0000 7f6673769700 1 -- 172.21.15.94:0/2425491342 wait complete. 2024-03-24T10:47:21.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:21 smithi094 ceph-mon[29446]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-24T10:47:21.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:21 smithi094 ceph-mon[29446]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-24T10:47:21.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:21 smithi094 ceph-mon[29446]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T10:47:21.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:21 smithi094 ceph-mon[29446]: Upgrade: Need to upgrade myself (mgr.smithi096.gfonnu) 2024-03-24T10:47:21.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:21 smithi094 ceph-mon[29446]: Upgrade: Need to upgrade myself (mgr.smithi096.gfonnu) 2024-03-24T10:47:21.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:21 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/2425491342' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T10:47:22.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:21 smithi096 ceph-mon[38097]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-24T10:47:22.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:21 smithi096 ceph-mon[38097]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-24T10:47:22.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:21 smithi096 ceph-mon[38097]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T10:47:22.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:21 smithi096 ceph-mon[38097]: Upgrade: Need to upgrade myself (mgr.smithi096.gfonnu) 2024-03-24T10:47:22.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:21 smithi096 ceph-mon[38097]: Upgrade: Need to upgrade myself (mgr.smithi096.gfonnu) 2024-03-24T10:47:22.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:21 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/2425491342' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T10:47:22.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:22 smithi094 ceph-mon[29446]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:47:22.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:22 smithi094 ceph-mon[29446]: pgmap v29: 65 pgs: 65 active+clean; 293 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 1.34k op/s 2024-03-24T10:47:23.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:22 smithi096 ceph-mon[38097]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:47:23.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:22 smithi096 ceph-mon[38097]: pgmap v29: 65 pgs: 65 active+clean; 293 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 1.34k op/s 2024-03-24T10:47:24.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:24 smithi094 ceph-mon[29446]: pgmap v30: 65 pgs: 65 active+clean; 293 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 1.19k op/s 2024-03-24T10:47:25.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:24 smithi096 ceph-mon[38097]: pgmap v30: 65 pgs: 65 active+clean; 293 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 1.19k op/s 2024-03-24T10:47:26.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:26 smithi094 ceph-mon[29446]: pgmap v31: 65 pgs: 65 active+clean; 293 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 1.37k op/s 2024-03-24T10:47:27.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:26 smithi096 ceph-mon[38097]: pgmap v31: 65 pgs: 65 active+clean; 293 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 1.37k op/s 2024-03-24T10:47:28.877 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:28 smithi096 ceph-mon[38097]: pgmap v32: 65 pgs: 65 active+clean; 271 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 1.25k op/s 2024-03-24T10:47:28.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:28 smithi094 ceph-mon[29446]: pgmap v32: 65 pgs: 65 active+clean; 271 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 1.25k op/s 2024-03-24T10:47:30.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:30 smithi094 ceph-mon[29446]: Upgrade: Updating mgr.smithi094.yaethu 2024-03-24T10:47:30.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:30 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:47:30.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:30 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi094.yaethu", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T10:47:30.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:30 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T10:47:30.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:30 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:47:30.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:30 smithi094 ceph-mon[29446]: Deploying daemon mgr.smithi094.yaethu on smithi094 2024-03-24T10:47:30.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:30 smithi094 ceph-mon[29446]: pgmap v33: 65 pgs: 65 active+clean; 271 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 547 op/s 2024-03-24T10:47:30.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:30 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:47:30.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:30 smithi096 ceph-mon[38097]: Upgrade: Updating mgr.smithi094.yaethu 2024-03-24T10:47:30.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:30 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:47:30.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:30 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi094.yaethu", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T10:47:30.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:30 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T10:47:30.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:30 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:47:30.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:30 smithi096 ceph-mon[38097]: Deploying daemon mgr.smithi094.yaethu on smithi094 2024-03-24T10:47:30.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:30 smithi096 ceph-mon[38097]: pgmap v33: 65 pgs: 65 active+clean; 271 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 547 op/s 2024-03-24T10:47:30.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:30 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:47:32.840 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:32 smithi094 ceph-mon[29446]: pgmap v34: 65 pgs: 65 active+clean; 267 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 1.01k op/s 2024-03-24T10:47:33.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:32 smithi096 ceph-mon[38097]: pgmap v34: 65 pgs: 65 active+clean; 267 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 1.01k op/s 2024-03-24T10:47:34.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:34 smithi094 ceph-mon[29446]: pgmap v35: 65 pgs: 65 active+clean; 267 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 846 op/s 2024-03-24T10:47:34.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:34 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:47:34.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:34 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:47:34.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:34 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:47:34.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:34 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:47:34.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:34 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:47:35.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:34 smithi096 ceph-mon[38097]: pgmap v35: 65 pgs: 65 active+clean; 267 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 846 op/s 2024-03-24T10:47:35.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:34 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:47:35.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:34 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:47:35.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:34 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:47:35.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:34 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:47:35.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:34 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:47:36.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:36 smithi096 ceph-mon[38097]: pgmap v36: 65 pgs: 65 active+clean; 263 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 942 op/s 2024-03-24T10:47:37.075 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:36 smithi094 ceph-mon[29446]: pgmap v36: 65 pgs: 65 active+clean; 263 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 942 op/s 2024-03-24T10:47:39.003 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:38 smithi094 ceph-mon[29446]: pgmap v37: 65 pgs: 65 active+clean; 248 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 850 op/s 2024-03-24T10:47:39.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:38 smithi096 ceph-mon[38097]: pgmap v37: 65 pgs: 65 active+clean; 248 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 850 op/s 2024-03-24T10:47:40.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:40 smithi094 ceph-mon[29446]: pgmap v38: 65 pgs: 65 active+clean; 248 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 652 op/s 2024-03-24T10:47:40.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:40 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:47:40.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:40 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:47:41.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:40 smithi096 ceph-mon[38097]: pgmap v38: 65 pgs: 65 active+clean; 248 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 652 op/s 2024-03-24T10:47:41.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:40 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:47:41.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:40 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:47:41.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:41 smithi094 ceph-mon[29446]: Standby manager daemon smithi094.yaethu restarted 2024-03-24T10:47:41.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:41 smithi094 ceph-mon[29446]: Standby manager daemon smithi094.yaethu started 2024-03-24T10:47:41.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:41 smithi094 ceph-mon[29446]: from='mgr.? 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi094.yaethu/crt"}]: dispatch 2024-03-24T10:47:41.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:41 smithi094 ceph-mon[29446]: from='mgr.? 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T10:47:41.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:41 smithi094 ceph-mon[29446]: from='mgr.? 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi094.yaethu/key"}]: dispatch 2024-03-24T10:47:41.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:41 smithi094 ceph-mon[29446]: from='mgr.? 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T10:47:41.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:41 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:47:41.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:41 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:47:42.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:41 smithi096 ceph-mon[38097]: Standby manager daemon smithi094.yaethu restarted 2024-03-24T10:47:42.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:41 smithi096 ceph-mon[38097]: Standby manager daemon smithi094.yaethu started 2024-03-24T10:47:42.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:41 smithi096 ceph-mon[38097]: from='mgr.? 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi094.yaethu/crt"}]: dispatch 2024-03-24T10:47:42.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:41 smithi096 ceph-mon[38097]: from='mgr.? 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T10:47:42.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:41 smithi096 ceph-mon[38097]: from='mgr.? 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi094.yaethu/key"}]: dispatch 2024-03-24T10:47:42.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:41 smithi096 ceph-mon[38097]: from='mgr.? 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T10:47:42.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:41 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:47:42.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:41 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:47:42.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:42 smithi094 ceph-mon[29446]: pgmap v39: 65 pgs: 65 active+clean; 244 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 943 op/s 2024-03-24T10:47:42.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:42 smithi094 ceph-mon[29446]: mgrmap e25: smithi096.gfonnu(active, since 72s), standbys: smithi094.yaethu 2024-03-24T10:47:43.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:42 smithi096 ceph-mon[38097]: pgmap v39: 65 pgs: 65 active+clean; 244 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 943 op/s 2024-03-24T10:47:43.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:42 smithi096 ceph-mon[38097]: mgrmap e25: smithi096.gfonnu(active, since 72s), standbys: smithi094.yaethu 2024-03-24T10:47:44.654 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-03-24T10:47:44.655 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-24T10:47:44.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:44 smithi094 ceph-mon[29446]: pgmap v40: 65 pgs: 65 active+clean; 244 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 481 op/s 2024-03-24T10:47:44.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:44 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:47:45.068 DEBUG:teuthology.parallel:result is None 2024-03-24T10:47:45.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:44 smithi096 ceph-mon[38097]: pgmap v40: 65 pgs: 65 active+clean; 244 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 481 op/s 2024-03-24T10:47:45.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:44 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:47:46.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:46 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:47:46.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:46 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:47:46.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:46 smithi096 ceph-mon[38097]: pgmap v41: 65 pgs: 65 active+clean; 240 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 759 op/s 2024-03-24T10:47:46.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:46 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:47:46.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:46 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:47:46.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:46 smithi094 ceph-mon[29446]: pgmap v41: 65 pgs: 65 active+clean; 240 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 759 op/s 2024-03-24T10:47:48.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:48 smithi094 ceph-mon[29446]: pgmap v42: 65 pgs: 65 active+clean; 239 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 691 op/s 2024-03-24T10:47:49.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:48 smithi096 ceph-mon[38097]: pgmap v42: 65 pgs: 65 active+clean; 239 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 691 op/s 2024-03-24T10:47:50.925 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:50 smithi094 ceph-mon[29446]: pgmap v43: 65 pgs: 65 active+clean; 239 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 598 op/s 2024-03-24T10:47:50.926 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:50 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:47:50.926 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:50 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:47:50.926 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:50 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:47:50.926 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:50 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:47:50.926 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:50 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:47:50.926 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:50 smithi096 ceph-mon[38097]: pgmap v43: 65 pgs: 65 active+clean; 239 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 598 op/s 2024-03-24T10:47:50.926 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:50 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:47:50.926 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:50 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:47:50.926 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:50 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:47:50.927 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:50 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:47:50.927 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:50 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' 2024-03-24T10:47:51.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.521+0000 7fe5bba86700 1 -- 172.21.15.94:0/619135921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5b40fe0d0 msgr2=0x7fe5b40fe540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:51.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.521+0000 7fe5bba86700 1 --2- 172.21.15.94:0/619135921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5b40fe0d0 0x7fe5b40fe540 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4004940 tx=0x7fe5a4009e30 comp rx=0 tx=0).stop 2024-03-24T10:47:51.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.521+0000 7fe5bba86700 1 -- 172.21.15.94:0/619135921 shutdown_connections 2024-03-24T10:47:51.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.521+0000 7fe5bba86700 1 --2- 172.21.15.94:0/619135921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5b40fe0d0 0x7fe5b40fe540 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:51.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.521+0000 7fe5bba86700 1 --2- 172.21.15.94:0/619135921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5b40fcee0 0x7fe5b40fd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:51.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.521+0000 7fe5bba86700 1 -- 172.21.15.94:0/619135921 >> 172.21.15.94:0/619135921 conn(0x7fe5b40f8560 msgr2=0x7fe5b40fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:47:51.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.521+0000 7fe5bba86700 1 -- 172.21.15.94:0/619135921 shutdown_connections 2024-03-24T10:47:51.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.521+0000 7fe5bba86700 1 -- 172.21.15.94:0/619135921 wait complete. 2024-03-24T10:47:51.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.522+0000 7fe5bba86700 1 Processor -- start 2024-03-24T10:47:51.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.522+0000 7fe5bba86700 1 -- start start 2024-03-24T10:47:51.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.522+0000 7fe5bba86700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5b40fcee0 0x7fe5b410da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:47:51.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.522+0000 7fe5bba86700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5b40fe0d0 0x7fe5b410df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:47:51.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.522+0000 7fe5bba86700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b410e570 con 0x7fe5b40fcee0 2024-03-24T10:47:51.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.522+0000 7fe5bba86700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b410ed70 con 0x7fe5b40fe0d0 2024-03-24T10:47:51.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.522+0000 7fe5b9822700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5b40fcee0 0x7fe5b410da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:47:51.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.522+0000 7fe5b9822700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5b40fcee0 0x7fe5b410da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44910/0 (socket says 172.21.15.94:44910) 2024-03-24T10:47:51.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.522+0000 7fe5b9822700 1 -- 172.21.15.94:0/578265248 learned_addr learned my addr 172.21.15.94:0/578265248 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:47:51.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.522+0000 7fe5b9021700 1 --2- 172.21.15.94:0/578265248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5b40fe0d0 0x7fe5b410df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:47:51.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.522+0000 7fe5b9822700 1 -- 172.21.15.94:0/578265248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5b40fe0d0 msgr2=0x7fe5b410df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:51.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.522+0000 7fe5b9822700 1 --2- 172.21.15.94:0/578265248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5b40fe0d0 0x7fe5b410df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:51.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.522+0000 7fe5b9822700 1 -- 172.21.15.94:0/578265248 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a4005040 con 0x7fe5b40fcee0 2024-03-24T10:47:51.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.523+0000 7fe5b9822700 1 --2- 172.21.15.94:0/578265248 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5b40fcee0 0x7fe5b410da30 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe5b000eb40 tx=0x7fe5b000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:47:51.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.523+0000 7fe5aa7fc700 1 -- 172.21.15.94:0/578265248 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5b0007cc0 con 0x7fe5b40fcee0 2024-03-24T10:47:51.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.523+0000 7fe5bba86700 1 -- 172.21.15.94:0/578265248 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5b410eff0 con 0x7fe5b40fcee0 2024-03-24T10:47:51.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.523+0000 7fe5aa7fc700 1 -- 172.21.15.94:0/578265248 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe5b000f040 con 0x7fe5b40fcee0 2024-03-24T10:47:51.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.523+0000 7fe5aa7fc700 1 -- 172.21.15.94:0/578265248 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5b000caa0 con 0x7fe5b40fcee0 2024-03-24T10:47:51.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.523+0000 7fe5bba86700 1 -- 172.21.15.94:0/578265248 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5b410f530 con 0x7fe5b40fcee0 2024-03-24T10:47:51.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.524+0000 7fe5aa7fc700 1 -- 172.21.15.94:0/578265248 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) v1 ==== 45451+0+0 (secure 0 0 0) 0x7fe5b000cc00 con 0x7fe5b40fcee0 2024-03-24T10:47:51.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.524+0000 7fe5bba86700 1 -- 172.21.15.94:0/578265248 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5b405f5f0 con 0x7fe5b40fcee0 2024-03-24T10:47:51.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.524+0000 7fe5aa7fc700 1 -- 172.21.15.94:0/578265248 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7fe5b004b9e0 con 0x7fe5b40fcee0 2024-03-24T10:47:51.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.529+0000 7fe5aa7fc700 1 -- 172.21.15.94:0/578265248 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7fe5b00133e0 con 0x7fe5b40fcee0 2024-03-24T10:47:51.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.753+0000 7fe5aa7fc700 1 -- 172.21.15.94:0/578265248 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 27) v1 ==== 45872+0+0 (secure 0 0 0) 0x7fe5b001e440 con 0x7fe5b40fcee0 2024-03-24T10:47:51.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.753+0000 7fe5aa7fc700 1 --2- 172.21.15.94:0/578265248 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7fe5a003dc40 0x7fe5a0040020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:47:51.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.764+0000 7fe5b9021700 1 --2- 172.21.15.94:0/578265248 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7fe5a003dc40 0x7fe5a0040020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:47:51.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.765+0000 7fe5b9021700 1 --2- 172.21.15.94:0/578265248 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7fe5a003dc40 0x7fe5a0040020 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4009b30 tx=0x7fe5a4009e30 comp rx=0 tx=0).ready entity=mgr.14774 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:47:51.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.785+0000 7fe5bba86700 1 -- 172.21.15.94:0/578265248 --> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe5b410f860 con 0x7fe5a003dc40 2024-03-24T10:47:51.786 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:51 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:47:51.786 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:51 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "mgr fail", "who": "smithi096.gfonnu"}]: dispatch 2024-03-24T10:47:51.786 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:51 smithi094 ceph-mon[29446]: osdmap e48: 6 total, 6 up, 6 in 2024-03-24T10:47:51.786 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:51 smithi094 ceph-mon[29446]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd='[{"prefix": "mgr fail", "who": "smithi096.gfonnu"}]': finished 2024-03-24T10:47:51.786 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:51 smithi094 ceph-mon[29446]: mgrmap e26: smithi094.yaethu(active, starting, since 0.00720482s) 2024-03-24T10:47:51.786 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:51 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata", "id": "smithi094"}]: dispatch 2024-03-24T10:47:51.786 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:51 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata", "id": "smithi096"}]: dispatch 2024-03-24T10:47:51.786 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:51 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.llefgu"}]: dispatch 2024-03-24T10:47:51.786 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:51 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.efiqte"}]: dispatch 2024-03-24T10:47:51.786 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:51 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi096.egbmoq"}]: dispatch 2024-03-24T10:47:51.786 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:51 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi096.jtebhh"}]: dispatch 2024-03-24T10:47:51.786 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:51 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mgr metadata", "who": "smithi094.yaethu", "id": "smithi094.yaethu"}]: dispatch 2024-03-24T10:47:51.786 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:51 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T10:47:51.787 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:51 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T10:47:51.787 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:51 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T10:47:51.787 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:51 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T10:47:51.787 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:51 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T10:47:51.787 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:51 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T10:47:51.787 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:51 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-24T10:47:51.787 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:51 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-24T10:47:51.787 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:51 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-24T10:47:51.787 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:51 smithi094 ceph-mon[29446]: Manager daemon smithi094.yaethu is now available 2024-03-24T10:47:51.787 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:51 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:47:51.787 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:51 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:47:51.787 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:51 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:47:51.787 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:51 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi094.yaethu/mirror_snapshot_schedule"}]: dispatch 2024-03-24T10:47:51.787 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:51 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi094.yaethu/trash_purge_schedule"}]: dispatch 2024-03-24T10:47:51.787 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:51 smithi094 ceph-mon[29446]: [24/Mar/2024:10:47:51] ENGINE Bus STARTING 2024-03-24T10:47:51.787 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:51 smithi094 ceph-mon[29446]: [24/Mar/2024:10:47:51] ENGINE Serving on http://172.21.15.94:8765 2024-03-24T10:47:51.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.787+0000 7fe5aa7fc700 1 -- 172.21.15.94:0/578265248 <== mgr.14774 v2:172.21.15.94:6800/4143126939 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7fe5b410f860 con 0x7fe5a003dc40 2024-03-24T10:47:51.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.790+0000 7fe5bba86700 1 -- 172.21.15.94:0/578265248 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7fe5a003dc40 msgr2=0x7fe5a0040020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:51.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.790+0000 7fe5bba86700 1 --2- 172.21.15.94:0/578265248 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7fe5a003dc40 0x7fe5a0040020 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4009b30 tx=0x7fe5a4009e30 comp rx=0 tx=0).stop 2024-03-24T10:47:51.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.790+0000 7fe5bba86700 1 -- 172.21.15.94:0/578265248 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5b40fcee0 msgr2=0x7fe5b410da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:51.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.790+0000 7fe5bba86700 1 --2- 172.21.15.94:0/578265248 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5b40fcee0 0x7fe5b410da30 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe5b000eb40 tx=0x7fe5b000b040 comp rx=0 tx=0).stop 2024-03-24T10:47:51.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.790+0000 7fe5bba86700 1 -- 172.21.15.94:0/578265248 shutdown_connections 2024-03-24T10:47:51.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.790+0000 7fe5bba86700 1 --2- 172.21.15.94:0/578265248 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7fe5a003dc40 0x7fe5a0040020 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:51.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.790+0000 7fe5bba86700 1 --2- 172.21.15.94:0/578265248 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5b40fcee0 0x7fe5b410da30 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:51.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.790+0000 7fe5bba86700 1 --2- 172.21.15.94:0/578265248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5b40fe0d0 0x7fe5b410df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:51.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.790+0000 7fe5bba86700 1 -- 172.21.15.94:0/578265248 >> 172.21.15.94:0/578265248 conn(0x7fe5b40f8560 msgr2=0x7fe5b40f9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:47:51.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.790+0000 7fe5bba86700 1 -- 172.21.15.94:0/578265248 shutdown_connections 2024-03-24T10:47:51.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.790+0000 7fe5bba86700 1 -- 172.21.15.94:0/578265248 wait complete. 2024-03-24T10:47:51.805 INFO:teuthology.orchestra.run.smithi094.stdout:true 2024-03-24T10:47:51.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:51 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:47:51.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:51 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "mgr fail", "who": "smithi096.gfonnu"}]: dispatch 2024-03-24T10:47:51.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:51 smithi096 ceph-mon[38097]: osdmap e48: 6 total, 6 up, 6 in 2024-03-24T10:47:51.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:51 smithi096 ceph-mon[38097]: from='mgr.24397 172.21.15.96:0/933527187' entity='mgr.smithi096.gfonnu' cmd='[{"prefix": "mgr fail", "who": "smithi096.gfonnu"}]': finished 2024-03-24T10:47:51.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:51 smithi096 ceph-mon[38097]: mgrmap e26: smithi094.yaethu(active, starting, since 0.00720482s) 2024-03-24T10:47:51.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:51 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata", "id": "smithi094"}]: dispatch 2024-03-24T10:47:51.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:51 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata", "id": "smithi096"}]: dispatch 2024-03-24T10:47:51.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:51 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.llefgu"}]: dispatch 2024-03-24T10:47:51.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:51 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.efiqte"}]: dispatch 2024-03-24T10:47:51.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:51 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi096.egbmoq"}]: dispatch 2024-03-24T10:47:51.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:51 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi096.jtebhh"}]: dispatch 2024-03-24T10:47:51.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:51 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mgr metadata", "who": "smithi094.yaethu", "id": "smithi094.yaethu"}]: dispatch 2024-03-24T10:47:51.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:51 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T10:47:51.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:51 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T10:47:51.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:51 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T10:47:51.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:51 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T10:47:51.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:51 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T10:47:51.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:51 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T10:47:51.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:51 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-24T10:47:51.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:51 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-24T10:47:51.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:51 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-24T10:47:51.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:51 smithi096 ceph-mon[38097]: Manager daemon smithi094.yaethu is now available 2024-03-24T10:47:51.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:51 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:47:51.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:51 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:47:51.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:51 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:47:51.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:51 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi094.yaethu/mirror_snapshot_schedule"}]: dispatch 2024-03-24T10:47:51.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:51 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi094.yaethu/trash_purge_schedule"}]: dispatch 2024-03-24T10:47:51.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:51 smithi096 ceph-mon[38097]: [24/Mar/2024:10:47:51] ENGINE Bus STARTING 2024-03-24T10:47:51.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:51 smithi096 ceph-mon[38097]: [24/Mar/2024:10:47:51] ENGINE Serving on http://172.21.15.94:8765 2024-03-24T10:47:51.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.939+0000 7f09d628f700 1 -- 172.21.15.94:0/2683236256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09d00fbcd0 msgr2=0x7f09d00fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:51.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.939+0000 7f09d628f700 1 --2- 172.21.15.94:0/2683236256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09d00fbcd0 0x7f09d00fc0e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f09c0004940 tx=0x7f09c0009e30 comp rx=0 tx=0).stop 2024-03-24T10:47:51.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.939+0000 7f09d628f700 1 -- 172.21.15.94:0/2683236256 shutdown_connections 2024-03-24T10:47:51.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.939+0000 7f09d628f700 1 --2- 172.21.15.94:0/2683236256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09d00fcf80 0x7f09d00fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:51.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.939+0000 7f09d628f700 1 --2- 172.21.15.94:0/2683236256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09d00fbcd0 0x7f09d00fc0e0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:51.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.939+0000 7f09d628f700 1 -- 172.21.15.94:0/2683236256 >> 172.21.15.94:0/2683236256 conn(0x7f09d00f7410 msgr2=0x7f09d00f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:47:51.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.939+0000 7f09d628f700 1 -- 172.21.15.94:0/2683236256 shutdown_connections 2024-03-24T10:47:51.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.939+0000 7f09d628f700 1 -- 172.21.15.94:0/2683236256 wait complete. 2024-03-24T10:47:51.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.940+0000 7f09d628f700 1 Processor -- start 2024-03-24T10:47:51.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.940+0000 7f09d628f700 1 -- start start 2024-03-24T10:47:51.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.940+0000 7f09d628f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09d00fbcd0 0x7f09d010d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:47:51.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.940+0000 7f09d628f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09d00fcf80 0x7f09d010def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:47:51.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.940+0000 7f09d628f700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d010e4f0 con 0x7f09d00fbcd0 2024-03-24T10:47:51.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.940+0000 7f09d628f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d010ed40 con 0x7f09d00fcf80 2024-03-24T10:47:51.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.940+0000 7f09cffff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09d00fbcd0 0x7f09d010d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:47:51.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.940+0000 7f09cffff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09d00fbcd0 0x7f09d010d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44916/0 (socket says 172.21.15.94:44916) 2024-03-24T10:47:51.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.940+0000 7f09cffff700 1 -- 172.21.15.94:0/3019780402 learned_addr learned my addr 172.21.15.94:0/3019780402 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:47:51.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.940+0000 7f09cf7fe700 1 --2- 172.21.15.94:0/3019780402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09d00fcf80 0x7f09d010def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:47:51.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.941+0000 7f09cffff700 1 -- 172.21.15.94:0/3019780402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09d00fcf80 msgr2=0x7f09d010def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:51.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.941+0000 7f09cffff700 1 --2- 172.21.15.94:0/3019780402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09d00fcf80 0x7f09d010def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:51.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.941+0000 7f09cffff700 1 -- 172.21.15.94:0/3019780402 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c0005040 con 0x7f09d00fbcd0 2024-03-24T10:47:51.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.941+0000 7f09cffff700 1 --2- 172.21.15.94:0/3019780402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09d00fbcd0 0x7f09d010d9b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f09c0013870 tx=0x7f09c0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:47:51.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.941+0000 7f09ccff9700 1 -- 172.21.15.94:0/3019780402 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09c0021070 con 0x7f09d00fbcd0 2024-03-24T10:47:51.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.941+0000 7f09ccff9700 1 -- 172.21.15.94:0/3019780402 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f09c0009430 con 0x7f09d00fbcd0 2024-03-24T10:47:51.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.941+0000 7f09d628f700 1 -- 172.21.15.94:0/3019780402 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09d010efc0 con 0x7f09d00fbcd0 2024-03-24T10:47:51.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.942+0000 7f09d628f700 1 -- 172.21.15.94:0/3019780402 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09d010f500 con 0x7f09d00fbcd0 2024-03-24T10:47:51.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.941+0000 7f09ccff9700 1 -- 172.21.15.94:0/3019780402 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09c000e070 con 0x7f09d00fbcd0 2024-03-24T10:47:51.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.942+0000 7f09ccff9700 1 -- 172.21.15.94:0/3019780402 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 45872+0+0 (secure 0 0 0) 0x7f09c0026410 con 0x7f09d00fbcd0 2024-03-24T10:47:51.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.942+0000 7f09ccff9700 1 --2- 172.21.15.94:0/3019780402 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f09b8038980 0x7f09b803ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:47:51.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.942+0000 7f09ccff9700 1 -- 172.21.15.94:0/3019780402 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f09c004d160 con 0x7f09d00fbcd0 2024-03-24T10:47:51.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.943+0000 7f09cf7fe700 1 --2- 172.21.15.94:0/3019780402 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f09b8038980 0x7f09b803ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:47:51.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.943+0000 7f09d628f700 1 -- 172.21.15.94:0/3019780402 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09d0101b10 con 0x7f09d00fbcd0 2024-03-24T10:47:51.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.943+0000 7f09cf7fe700 1 --2- 172.21.15.94:0/3019780402 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f09b8038980 0x7f09b803ae30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f09c4009b80 tx=0x7f09c400b040 comp rx=0 tx=0).ready entity=mgr.14774 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:47:51.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:51.948+0000 7f09ccff9700 1 -- 172.21.15.94:0/3019780402 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7f09c0052060 con 0x7f09d00fbcd0 2024-03-24T10:47:52.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.178+0000 7f09d628f700 1 -- 172.21.15.94:0/3019780402 --> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f09d0104930 con 0x7f09b8038980 2024-03-24T10:47:52.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.179+0000 7f09ccff9700 1 -- 172.21.15.94:0/3019780402 <== mgr.14774 v2:172.21.15.94:6800/4143126939 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f09d0104930 con 0x7f09b8038980 2024-03-24T10:47:52.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.181+0000 7f09d628f700 1 -- 172.21.15.94:0/3019780402 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f09b8038980 msgr2=0x7f09b803ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:52.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.181+0000 7f09d628f700 1 --2- 172.21.15.94:0/3019780402 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f09b8038980 0x7f09b803ae30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f09c4009b80 tx=0x7f09c400b040 comp rx=0 tx=0).stop 2024-03-24T10:47:52.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.181+0000 7f09d628f700 1 -- 172.21.15.94:0/3019780402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09d00fbcd0 msgr2=0x7f09d010d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:52.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.181+0000 7f09d628f700 1 --2- 172.21.15.94:0/3019780402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09d00fbcd0 0x7f09d010d9b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f09c0013870 tx=0x7f09c0009e30 comp rx=0 tx=0).stop 2024-03-24T10:47:52.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.182+0000 7f09d628f700 1 -- 172.21.15.94:0/3019780402 shutdown_connections 2024-03-24T10:47:52.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.182+0000 7f09d628f700 1 --2- 172.21.15.94:0/3019780402 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f09b8038980 0x7f09b803ae30 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:52.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.182+0000 7f09d628f700 1 --2- 172.21.15.94:0/3019780402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09d00fbcd0 0x7f09d010d9b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:52.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.182+0000 7f09d628f700 1 --2- 172.21.15.94:0/3019780402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09d00fcf80 0x7f09d010def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:52.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.182+0000 7f09d628f700 1 -- 172.21.15.94:0/3019780402 >> 172.21.15.94:0/3019780402 conn(0x7f09d00f7410 msgr2=0x7f09d00f8a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:47:52.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.182+0000 7f09d628f700 1 -- 172.21.15.94:0/3019780402 shutdown_connections 2024-03-24T10:47:52.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.182+0000 7f09d628f700 1 -- 172.21.15.94:0/3019780402 wait complete. 2024-03-24T10:47:52.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.290+0000 7f0f19d84700 1 -- 172.21.15.94:0/1889985767 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f140fcf80 msgr2=0x7f0f140fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:52.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.290+0000 7f0f19d84700 1 --2- 172.21.15.94:0/1889985767 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f140fcf80 0x7f0f140fd3f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0f08004940 tx=0x7f0f08009e30 comp rx=0 tx=0).stop 2024-03-24T10:47:52.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.290+0000 7f0f19d84700 1 -- 172.21.15.94:0/1889985767 shutdown_connections 2024-03-24T10:47:52.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.290+0000 7f0f19d84700 1 --2- 172.21.15.94:0/1889985767 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f140fcf80 0x7f0f140fd3f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:52.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.290+0000 7f0f19d84700 1 --2- 172.21.15.94:0/1889985767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f140fbcd0 0x7f0f140fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:52.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.290+0000 7f0f19d84700 1 -- 172.21.15.94:0/1889985767 >> 172.21.15.94:0/1889985767 conn(0x7f0f140f7430 msgr2=0x7f0f140f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:47:52.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.290+0000 7f0f19d84700 1 -- 172.21.15.94:0/1889985767 shutdown_connections 2024-03-24T10:47:52.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.290+0000 7f0f19d84700 1 -- 172.21.15.94:0/1889985767 wait complete. 2024-03-24T10:47:52.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.291+0000 7f0f19d84700 1 Processor -- start 2024-03-24T10:47:52.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.294+0000 7f0f19d84700 1 -- start start 2024-03-24T10:47:52.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.294+0000 7f0f19d84700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f140fbcd0 0x7f0f140ffdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:47:52.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.294+0000 7f0f19d84700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f140fcf80 0x7f0f140fe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:47:52.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.294+0000 7f0f19d84700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f14111660 con 0x7f0f140fbcd0 2024-03-24T10:47:52.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.294+0000 7f0f19d84700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f141117d0 con 0x7f0f140fcf80 2024-03-24T10:47:52.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.295+0000 7f0f12ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f140fcf80 0x7f0f140fe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:47:52.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.295+0000 7f0f137fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f140fbcd0 0x7f0f140ffdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:47:52.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.295+0000 7f0f137fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f140fbcd0 0x7f0f140ffdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44934/0 (socket says 172.21.15.94:44934) 2024-03-24T10:47:52.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.295+0000 7f0f137fe700 1 -- 172.21.15.94:0/2651300551 learned_addr learned my addr 172.21.15.94:0/2651300551 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:47:52.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.295+0000 7f0f137fe700 1 -- 172.21.15.94:0/2651300551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f140fcf80 msgr2=0x7f0f140fe2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:52.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.295+0000 7f0f137fe700 1 --2- 172.21.15.94:0/2651300551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f140fcf80 0x7f0f140fe2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:52.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.295+0000 7f0f137fe700 1 -- 172.21.15.94:0/2651300551 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f08005040 con 0x7f0f140fbcd0 2024-03-24T10:47:52.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.295+0000 7f0f137fe700 1 --2- 172.21.15.94:0/2651300551 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f140fbcd0 0x7f0f140ffdf0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0f0400eb70 tx=0x7f0f0400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:47:52.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.296+0000 7f0efbfff700 1 -- 172.21.15.94:0/2651300551 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f04007cc0 con 0x7f0f140fbcd0 2024-03-24T10:47:52.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.296+0000 7f0efbfff700 1 -- 172.21.15.94:0/2651300551 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0f0400f040 con 0x7f0f140fbcd0 2024-03-24T10:47:52.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.296+0000 7f0f19d84700 1 -- 172.21.15.94:0/2651300551 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f14111a50 con 0x7f0f140fbcd0 2024-03-24T10:47:52.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.296+0000 7f0efbfff700 1 -- 172.21.15.94:0/2651300551 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f0400ca90 con 0x7f0f140fbcd0 2024-03-24T10:47:52.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.296+0000 7f0f19d84700 1 -- 172.21.15.94:0/2651300551 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f14111f90 con 0x7f0f140fbcd0 2024-03-24T10:47:52.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.297+0000 7f0efbfff700 1 -- 172.21.15.94:0/2651300551 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 45872+0+0 (secure 0 0 0) 0x7f0f0400cbf0 con 0x7f0f140fbcd0 2024-03-24T10:47:52.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.297+0000 7f0efbfff700 1 --2- 172.21.15.94:0/2651300551 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f0efc038a10 0x7f0efc03aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:47:52.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.297+0000 7f0efbfff700 1 -- 172.21.15.94:0/2651300551 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f0f0404c100 con 0x7f0f140fbcd0 2024-03-24T10:47:52.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.297+0000 7f0f19d84700 1 -- 172.21.15.94:0/2651300551 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f1405f5f0 con 0x7f0f140fbcd0 2024-03-24T10:47:52.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.297+0000 7f0f12ffd700 1 --2- 172.21.15.94:0/2651300551 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f0efc038a10 0x7f0efc03aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:47:52.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.298+0000 7f0f12ffd700 1 --2- 172.21.15.94:0/2651300551 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f0efc038a10 0x7f0efc03aec0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0f08009b30 tx=0x7f0f08009e30 comp rx=0 tx=0).ready entity=mgr.14774 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:47:52.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.303+0000 7f0efbfff700 1 -- 172.21.15.94:0/2651300551 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7f0f04019020 con 0x7f0f140fbcd0 2024-03-24T10:47:52.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.527+0000 7f0f19d84700 1 -- 172.21.15.94:0/2651300551 --> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f140ff040 con 0x7f0efc038a10 2024-03-24T10:47:52.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.538+0000 7f0efbfff700 1 -- 172.21.15.94:0/2651300551 <== mgr.14774 v2:172.21.15.94:6800/4143126939 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f0f140ff040 con 0x7f0efc038a10 2024-03-24T10:47:52.537 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T10:47:52.537 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.smithi094 smithi094 *:9093,9094 running (56s) 12s ago 10m 16.5M - 0.25.0 c8568f914cd2 5c583af60554 2024-03-24T10:47:52.538 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi094 smithi094 running (10m) 12s ago 10m 6660k - 17.2.7-726-g9c4e0c34 837124da804f fb5772af46c5 2024-03-24T10:47:52.538 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi096 smithi096 running (8m) 2s ago 8m 6669k - 17.2.7-726-g9c4e0c34 837124da804f 6087ad4e0734 2024-03-24T10:47:52.538 INFO:teuthology.orchestra.run.smithi094.stdout:grafana.smithi094 smithi094 *:3000 running (41s) 12s ago 9m 80.2M - 9.4.7 954c08fa6188 d3d425bf1b7c 2024-03-24T10:47:52.538 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.efiqte smithi094 running (4m) 12s ago 4m 12.0M - 17.2.7-726-g9c4e0c34 837124da804f f0df0a056c21 2024-03-24T10:47:52.538 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.llefgu smithi094 running (4m) 12s ago 4m 128M - 17.2.7-726-g9c4e0c34 837124da804f 52a9771dc8e0 2024-03-24T10:47:52.538 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi096.egbmoq smithi096 running (4m) 2s ago 4m 209M - 17.2.7-726-g9c4e0c34 837124da804f 1cfc930dbab0 2024-03-24T10:47:52.538 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi096.jtebhh smithi096 running (3m) 2s ago 3m 15.0M - 17.2.7-726-g9c4e0c34 837124da804f 4ff3451a6f6e 2024-03-24T10:47:52.538 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi094.yaethu smithi094 *:8443,9283,8765 running (19s) 12s ago 11m 94.9M - 19.0.0-1855-g21b3bd67 553463a90fd5 d3adab76b18f 2024-03-24T10:47:52.538 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi096.gfonnu smithi096 *:8443,9283 running (109s) 2s ago 8m 500M - 19.0.0-1855-g21b3bd67 553463a90fd5 b6483bc88c49 2024-03-24T10:47:52.538 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi094 smithi094 running (11m) 12s ago 11m 47.8M 2048M 17.2.7-726-g9c4e0c34 837124da804f 76584fceb32b 2024-03-24T10:47:52.538 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi096 smithi096 running (8m) 2s ago 8m 40.9M 2048M 17.2.7-726-g9c4e0c34 837124da804f 710030b222cb 2024-03-24T10:47:52.538 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi094 smithi094 *:9100 running (63s) 12s ago 10m 8024k - 1.5.0 0da6a335fe13 ca67b53c64e2 2024-03-24T10:47:52.538 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi096 smithi096 *:9100 running (32s) 2s ago 8m 8158k - 1.5.0 0da6a335fe13 65137a7a7549 2024-03-24T10:47:52.539 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (7m) 12s ago 7m 234M 1557M 17.2.7-726-g9c4e0c34 837124da804f b664fd3557bd 2024-03-24T10:47:52.539 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi094 running (7m) 12s ago 7m 246M 1557M 17.2.7-726-g9c4e0c34 837124da804f 4e737a80d733 2024-03-24T10:47:52.539 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi094 running (6m) 12s ago 6m 211M 1557M 17.2.7-726-g9c4e0c34 837124da804f cf0aefd20c76 2024-03-24T10:47:52.539 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi096 running (6m) 2s ago 6m 274M 2581M 17.2.7-726-g9c4e0c34 837124da804f f4650d0f6241 2024-03-24T10:47:52.539 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi096 running (6m) 2s ago 6m 248M 2581M 17.2.7-726-g9c4e0c34 837124da804f a63d7c9e705f 2024-03-24T10:47:52.539 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi096 running (5m) 2s ago 5m 230M 2581M 17.2.7-726-g9c4e0c34 837124da804f f56fad2959c3 2024-03-24T10:47:52.539 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.smithi094 smithi094 *:9095 running (34s) 12s ago 8m 42.2M - 2.43.0 a07b618ecd1d 0751fffb4d9c 2024-03-24T10:47:52.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.540+0000 7f0f19d84700 1 -- 172.21.15.94:0/2651300551 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f0efc038a10 msgr2=0x7f0efc03aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:52.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.540+0000 7f0f19d84700 1 --2- 172.21.15.94:0/2651300551 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f0efc038a10 0x7f0efc03aec0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0f08009b30 tx=0x7f0f08009e30 comp rx=0 tx=0).stop 2024-03-24T10:47:52.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.540+0000 7f0f19d84700 1 -- 172.21.15.94:0/2651300551 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f140fbcd0 msgr2=0x7f0f140ffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:52.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.540+0000 7f0f19d84700 1 --2- 172.21.15.94:0/2651300551 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f140fbcd0 0x7f0f140ffdf0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0f0400eb70 tx=0x7f0f0400b040 comp rx=0 tx=0).stop 2024-03-24T10:47:52.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.540+0000 7f0f19d84700 1 -- 172.21.15.94:0/2651300551 shutdown_connections 2024-03-24T10:47:52.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.540+0000 7f0f19d84700 1 --2- 172.21.15.94:0/2651300551 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f0efc038a10 0x7f0efc03aec0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:52.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.540+0000 7f0f19d84700 1 --2- 172.21.15.94:0/2651300551 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f140fbcd0 0x7f0f140ffdf0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:52.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.540+0000 7f0f19d84700 1 --2- 172.21.15.94:0/2651300551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f140fcf80 0x7f0f140fe2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:52.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.540+0000 7f0f19d84700 1 -- 172.21.15.94:0/2651300551 >> 172.21.15.94:0/2651300551 conn(0x7f0f140f7430 msgr2=0x7f0f140f8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:47:52.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.540+0000 7f0f19d84700 1 -- 172.21.15.94:0/2651300551 shutdown_connections 2024-03-24T10:47:52.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.540+0000 7f0f19d84700 1 -- 172.21.15.94:0/2651300551 wait complete. 2024-03-24T10:47:52.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.641+0000 7f118ca7f700 1 -- 172.21.15.94:0/247333231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11880fcf90 msgr2=0x7f11880fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:52.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.641+0000 7f118ca7f700 1 --2- 172.21.15.94:0/247333231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11880fcf90 0x7f11880fd400 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1170004940 tx=0x7f1170009e30 comp rx=0 tx=0).stop 2024-03-24T10:47:52.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.642+0000 7f118ca7f700 1 -- 172.21.15.94:0/247333231 shutdown_connections 2024-03-24T10:47:52.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.642+0000 7f118ca7f700 1 --2- 172.21.15.94:0/247333231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11880fcf90 0x7f11880fd400 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:52.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.642+0000 7f118ca7f700 1 --2- 172.21.15.94:0/247333231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11880fbce0 0x7f11880fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:52.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.642+0000 7f118ca7f700 1 -- 172.21.15.94:0/247333231 >> 172.21.15.94:0/247333231 conn(0x7f11880f7440 msgr2=0x7f11880f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:47:52.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.642+0000 7f118ca7f700 1 -- 172.21.15.94:0/247333231 shutdown_connections 2024-03-24T10:47:52.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.642+0000 7f118ca7f700 1 -- 172.21.15.94:0/247333231 wait complete. 2024-03-24T10:47:52.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.642+0000 7f118ca7f700 1 Processor -- start 2024-03-24T10:47:52.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.642+0000 7f118ca7f700 1 -- start start 2024-03-24T10:47:52.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.642+0000 7f118ca7f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11880fbce0 0x7f118810fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:47:52.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.642+0000 7f118ca7f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11880fcf90 0x7f11881100a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:47:52.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.643+0000 7f118ca7f700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11881106a0 con 0x7f11880fbce0 2024-03-24T10:47:52.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.643+0000 7f118ca7f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1188110810 con 0x7f11880fcf90 2024-03-24T10:47:52.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.643+0000 7f118659c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11880fbce0 0x7f118810fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:47:52.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.643+0000 7f118659c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11880fbce0 0x7f118810fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44952/0 (socket says 172.21.15.94:44952) 2024-03-24T10:47:52.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.643+0000 7f118659c700 1 -- 172.21.15.94:0/307606340 learned_addr learned my addr 172.21.15.94:0/307606340 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:47:52.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.643+0000 7f1185d9b700 1 --2- 172.21.15.94:0/307606340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11880fcf90 0x7f11881100a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:47:52.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.643+0000 7f118659c700 1 -- 172.21.15.94:0/307606340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11880fcf90 msgr2=0x7f11881100a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:52.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.643+0000 7f118659c700 1 --2- 172.21.15.94:0/307606340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11880fcf90 0x7f11881100a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:52.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.643+0000 7f118659c700 1 -- 172.21.15.94:0/307606340 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1170005040 con 0x7f11880fbce0 2024-03-24T10:47:52.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.643+0000 7f118659c700 1 --2- 172.21.15.94:0/307606340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11880fbce0 0x7f118810fb60 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f118000ec90 tx=0x7f118000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:47:52.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.643+0000 7f117effd700 1 -- 172.21.15.94:0/307606340 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1180007b30 con 0x7f11880fbce0 2024-03-24T10:47:52.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.643+0000 7f117effd700 1 -- 172.21.15.94:0/307606340 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f118000f040 con 0x7f11880fbce0 2024-03-24T10:47:52.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.643+0000 7f118ca7f700 1 -- 172.21.15.94:0/307606340 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1188115220 con 0x7f11880fbce0 2024-03-24T10:47:52.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.643+0000 7f117effd700 1 -- 172.21.15.94:0/307606340 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f118000cc80 con 0x7f11880fbce0 2024-03-24T10:47:52.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.643+0000 7f118ca7f700 1 -- 172.21.15.94:0/307606340 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1188115760 con 0x7f11880fbce0 2024-03-24T10:47:52.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.644+0000 7f117effd700 1 -- 172.21.15.94:0/307606340 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 45872+0+0 (secure 0 0 0) 0x7f1180007c90 con 0x7f11880fbce0 2024-03-24T10:47:52.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.644+0000 7f117effd700 1 --2- 172.21.15.94:0/307606340 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f11740389d0 0x7f117403ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:47:52.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.644+0000 7f118ca7f700 1 -- 172.21.15.94:0/307606340 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1188101920 con 0x7f11880fbce0 2024-03-24T10:47:52.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.645+0000 7f117effd700 1 -- 172.21.15.94:0/307606340 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f118004ca70 con 0x7f11880fbce0 2024-03-24T10:47:52.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.645+0000 7f1185d9b700 1 --2- 172.21.15.94:0/307606340 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f11740389d0 0x7f117403ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:47:52.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.645+0000 7f1185d9b700 1 --2- 172.21.15.94:0/307606340 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f11740389d0 0x7f117403ae80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1170009b30 tx=0x7f1170009e30 comp rx=0 tx=0).ready entity=mgr.14774 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:47:52.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:52.649+0000 7f117effd700 1 -- 172.21.15.94:0/307606340 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7f11800133e0 con 0x7f11880fbce0 2024-03-24T10:47:52.755 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:52 smithi096 ceph-mon[38097]: [24/Mar/2024:10:47:51] ENGINE Serving on https://172.21.15.94:7150 2024-03-24T10:47:52.756 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:52 smithi096 ceph-mon[38097]: [24/Mar/2024:10:47:51] ENGINE Bus STARTED 2024-03-24T10:47:52.756 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:52 smithi096 ceph-mon[38097]: mgrmap e27: smithi094.yaethu(active, since 1.01021s) 2024-03-24T10:47:52.756 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:52 smithi096 ceph-mon[38097]: pgmap v3: 65 pgs: 65 active+clean; 216 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-03-24T10:47:52.756 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:52 smithi096 ceph-mon[38097]: from='client.14798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:47:52.756 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:52 smithi096 ceph-mon[38097]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:47:52.756 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:52 smithi096 ceph-mon[38097]: from='client.14806 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:47:52.973 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:52 smithi094 ceph-mon[29446]: [24/Mar/2024:10:47:51] ENGINE Serving on https://172.21.15.94:7150 2024-03-24T10:47:52.973 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:52 smithi094 ceph-mon[29446]: [24/Mar/2024:10:47:51] ENGINE Bus STARTED 2024-03-24T10:47:52.974 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:52 smithi094 ceph-mon[29446]: mgrmap e27: smithi094.yaethu(active, since 1.01021s) 2024-03-24T10:47:52.974 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:52 smithi094 ceph-mon[29446]: pgmap v3: 65 pgs: 65 active+clean; 216 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-03-24T10:47:52.974 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:52 smithi094 ceph-mon[29446]: from='client.14798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:47:52.974 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:52 smithi094 ceph-mon[29446]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:47:52.974 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:52 smithi094 ceph-mon[29446]: from='client.14806 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:47:53.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.042+0000 7f118ca7f700 1 -- 172.21.15.94:0/307606340 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f11881047d0 con 0x7f11880fbce0 2024-03-24T10:47:53.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.042+0000 7f117effd700 1 -- 172.21.15.94:0/307606340 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f11800112c0 con 0x7f11880fbce0 2024-03-24T10:47:53.042 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-03-24T10:47:53.042 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": { 2024-03-24T10:47:53.042 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-24T10:47:53.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:47:53.042 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": { 2024-03-24T10:47:53.042 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 1 2024-03-24T10:47:53.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:47:53.042 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": { 2024-03-24T10:47:53.043 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-24T10:47:53.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:47:53.043 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-03-24T10:47:53.043 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-24T10:47:53.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:47:53.043 INFO:teuthology.orchestra.run.smithi094.stdout: "overall": { 2024-03-24T10:47:53.043 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 12, 2024-03-24T10:47:53.043 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 1 2024-03-24T10:47:53.043 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-03-24T10:47:53.043 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-03-24T10:47:53.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.044+0000 7f118ca7f700 1 -- 172.21.15.94:0/307606340 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f11740389d0 msgr2=0x7f117403ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:53.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.044+0000 7f118ca7f700 1 --2- 172.21.15.94:0/307606340 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f11740389d0 0x7f117403ae80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1170009b30 tx=0x7f1170009e30 comp rx=0 tx=0).stop 2024-03-24T10:47:53.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.044+0000 7f118ca7f700 1 -- 172.21.15.94:0/307606340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11880fbce0 msgr2=0x7f118810fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:53.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.044+0000 7f118ca7f700 1 --2- 172.21.15.94:0/307606340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11880fbce0 0x7f118810fb60 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f118000ec90 tx=0x7f118000b040 comp rx=0 tx=0).stop 2024-03-24T10:47:53.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.044+0000 7f118ca7f700 1 -- 172.21.15.94:0/307606340 shutdown_connections 2024-03-24T10:47:53.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.044+0000 7f118ca7f700 1 --2- 172.21.15.94:0/307606340 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f11740389d0 0x7f117403ae80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:53.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.044+0000 7f118ca7f700 1 --2- 172.21.15.94:0/307606340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11880fbce0 0x7f118810fb60 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:53.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.044+0000 7f118ca7f700 1 --2- 172.21.15.94:0/307606340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11880fcf90 0x7f11881100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:53.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.044+0000 7f118ca7f700 1 -- 172.21.15.94:0/307606340 >> 172.21.15.94:0/307606340 conn(0x7f11880f7440 msgr2=0x7f1188066880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:47:53.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.044+0000 7f118ca7f700 1 -- 172.21.15.94:0/307606340 shutdown_connections 2024-03-24T10:47:53.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.044+0000 7f118ca7f700 1 -- 172.21.15.94:0/307606340 wait complete. 2024-03-24T10:47:53.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.151+0000 7f7bdab1b700 1 -- 172.21.15.94:0/3683948645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7bd40fff10 msgr2=0x7f7bd4102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:53.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.151+0000 7f7bdab1b700 1 --2- 172.21.15.94:0/3683948645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7bd40fff10 0x7f7bd4102330 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0004940 tx=0x7f7bc0009e30 comp rx=0 tx=0).stop 2024-03-24T10:47:53.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.151+0000 7f7bdab1b700 1 -- 172.21.15.94:0/3683948645 shutdown_connections 2024-03-24T10:47:53.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.151+0000 7f7bdab1b700 1 --2- 172.21.15.94:0/3683948645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7bd40fff10 0x7f7bd4102330 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:53.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.151+0000 7f7bdab1b700 1 --2- 172.21.15.94:0/3683948645 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7bd4066fd0 0x7f7bd40ff9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:53.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.151+0000 7f7bdab1b700 1 -- 172.21.15.94:0/3683948645 >> 172.21.15.94:0/3683948645 conn(0x7f7bd40f7440 msgr2=0x7f7bd40f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:47:53.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.151+0000 7f7bdab1b700 1 -- 172.21.15.94:0/3683948645 shutdown_connections 2024-03-24T10:47:53.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.151+0000 7f7bdab1b700 1 -- 172.21.15.94:0/3683948645 wait complete. 2024-03-24T10:47:53.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.152+0000 7f7bdab1b700 1 Processor -- start 2024-03-24T10:47:53.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.152+0000 7f7bdab1b700 1 -- start start 2024-03-24T10:47:53.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.152+0000 7f7bdab1b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7bd4066fd0 0x7f7bd410d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:47:53.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.152+0000 7f7bdab1b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7bd40fff10 0x7f7bd410de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:47:53.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.152+0000 7f7bdab1b700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd410e460 con 0x7f7bd40fff10 2024-03-24T10:47:53.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.152+0000 7f7bdab1b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd410e5d0 con 0x7f7bd4066fd0 2024-03-24T10:47:53.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.152+0000 7f7bd3fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7bd40fff10 0x7f7bd410de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:47:53.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.152+0000 7f7bd3fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7bd40fff10 0x7f7bd410de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45000/0 (socket says 172.21.15.94:45000) 2024-03-24T10:47:53.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.152+0000 7f7bd3fff700 1 -- 172.21.15.94:0/1740188704 learned_addr learned my addr 172.21.15.94:0/1740188704 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:47:53.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.152+0000 7f7bd88b7700 1 --2- 172.21.15.94:0/1740188704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7bd4066fd0 0x7f7bd410d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:47:53.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.153+0000 7f7bd88b7700 1 -- 172.21.15.94:0/1740188704 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7bd40fff10 msgr2=0x7f7bd410de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:53.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.153+0000 7f7bd88b7700 1 --2- 172.21.15.94:0/1740188704 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7bd40fff10 0x7f7bd410de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:53.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.153+0000 7f7bd88b7700 1 -- 172.21.15.94:0/1740188704 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc0005040 con 0x7f7bd4066fd0 2024-03-24T10:47:53.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.153+0000 7f7bd3fff700 1 --2- 172.21.15.94:0/1740188704 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7bd40fff10 0x7f7bd410de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T10:47:53.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.153+0000 7f7bd88b7700 1 --2- 172.21.15.94:0/1740188704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7bd4066fd0 0x7f7bd410d920 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4066d10 tx=0x7f7bc800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:47:53.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.153+0000 7f7bd17fa700 1 -- 172.21.15.94:0/1740188704 <== mon.1 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bc8004e90 con 0x7f7bd4066fd0 2024-03-24T10:47:53.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.153+0000 7f7bdab1b700 1 -- 172.21.15.94:0/1740188704 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bd4112f80 con 0x7f7bd4066fd0 2024-03-24T10:47:53.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.153+0000 7f7bd17fa700 1 -- 172.21.15.94:0/1740188704 <== mon.1 v2:172.21.15.96:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7bc800f040 con 0x7f7bd4066fd0 2024-03-24T10:47:53.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.153+0000 7f7bd17fa700 1 -- 172.21.15.94:0/1740188704 <== mon.1 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bc800ca20 con 0x7f7bd4066fd0 2024-03-24T10:47:53.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.153+0000 7f7bdab1b700 1 -- 172.21.15.94:0/1740188704 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bd4113430 con 0x7f7bd4066fd0 2024-03-24T10:47:53.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.154+0000 7f7bdab1b700 1 -- 172.21.15.94:0/1740188704 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bd404c710 con 0x7f7bd4066fd0 2024-03-24T10:47:53.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.154+0000 7f7bd17fa700 1 -- 172.21.15.94:0/1740188704 <== mon.1 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 27) v1 ==== 45872+0+0 (secure 0 0 0) 0x7f7bc800cbd0 con 0x7f7bd4066fd0 2024-03-24T10:47:53.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.154+0000 7f7bd17fa700 1 --2- 172.21.15.94:0/1740188704 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f7bc40389d0 0x7f7bc403ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:47:53.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.154+0000 7f7bd17fa700 1 -- 172.21.15.94:0/1740188704 <== mon.1 v2:172.21.15.96:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f7bc804c080 con 0x7f7bd4066fd0 2024-03-24T10:47:53.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.154+0000 7f7bd3fff700 1 --2- 172.21.15.94:0/1740188704 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f7bc40389d0 0x7f7bc403ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:47:53.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.155+0000 7f7bd3fff700 1 --2- 172.21.15.94:0/1740188704 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f7bc40389d0 0x7f7bc403ae80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7bc000b0d0 tx=0x7f7bc0009e30 comp rx=0 tx=0).ready entity=mgr.14774 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:47:53.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.159+0000 7f7bd17fa700 1 -- 172.21.15.94:0/1740188704 <== mon.1 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7f7bc8009730 con 0x7f7bd4066fd0 2024-03-24T10:47:53.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.412+0000 7f7bdab1b700 1 -- 172.21.15.94:0/1740188704 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7bd4113650 con 0x7f7bd4066fd0 2024-03-24T10:47:53.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.413+0000 7f7bd17fa700 1 -- 172.21.15.94:0/1740188704 <== mon.1 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1833 (secure 0 0 0) 0x7f7bc80129f0 con 0x7f7bd4066fd0 2024-03-24T10:47:53.412 INFO:teuthology.orchestra.run.smithi094.stdout:e18 2024-03-24T10:47:53.412 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T10:47:53.413 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:47:53.413 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-03-24T10:47:53.413 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:47:53.413 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-03-24T10:47:53.413 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-03-24T10:47:53.413 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 18 2024-03-24T10:47:53.413 INFO:teuthology.orchestra.run.smithi094.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-24T10:47:53.413 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-03-24T10:43:39.435738+0000 2024-03-24T10:47:53.413 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-03-24T10:43:58.133901+0000 2024-03-24T10:47:53.413 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-03-24T10:47:53.413 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-03-24T10:47:53.413 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 60 2024-03-24T10:47:53.413 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-03-24T10:47:53.413 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-03-24T10:47:53.413 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-03-24T10:47:53.413 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-03-24T10:47:53.414 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 46 2024-03-24T10:47:53.414 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:47:53.414 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-03-24T10:47:53.414 INFO:teuthology.orchestra.run.smithi094.stdout:in 0 2024-03-24T10:47:53.414 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=24251} 2024-03-24T10:47:53.414 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-03-24T10:47:53.414 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-03-24T10:47:53.414 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-03-24T10:47:53.414 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-03-24T10:47:53.414 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-03-24T10:47:53.414 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-03-24T10:47:53.414 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-03-24T10:47:53.414 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 1 2024-03-24T10:47:53.414 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi096.egbmoq{0:24251} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.96:6824/1539752208,v1:172.21.15.96:6825/1539752208] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:47:53.414 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.llefgu{0:14500} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.94:6828/126762980,v1:172.21.15.94:6829/126762980] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:47:53.415 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:47:53.415 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:47:53.415 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-03-24T10:47:53.415 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:47:53.415 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.efiqte{-1:14512} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.94:6826/1636528860,v1:172.21.15.94:6827/1636528860] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:47:53.415 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi096.jtebhh{-1:24261} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.96:6826/1321087930,v1:172.21.15.96:6827/1321087930] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:47:53.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.415+0000 7f7bdab1b700 1 -- 172.21.15.94:0/1740188704 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f7bc40389d0 msgr2=0x7f7bc403ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:53.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.415+0000 7f7bdab1b700 1 --2- 172.21.15.94:0/1740188704 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f7bc40389d0 0x7f7bc403ae80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7bc000b0d0 tx=0x7f7bc0009e30 comp rx=0 tx=0).stop 2024-03-24T10:47:53.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.415+0000 7f7bdab1b700 1 -- 172.21.15.94:0/1740188704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7bd4066fd0 msgr2=0x7f7bd410d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:53.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.415+0000 7f7bdab1b700 1 --2- 172.21.15.94:0/1740188704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7bd4066fd0 0x7f7bd410d920 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4066d10 tx=0x7f7bc800b040 comp rx=0 tx=0).stop 2024-03-24T10:47:53.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.415+0000 7f7bdab1b700 1 -- 172.21.15.94:0/1740188704 shutdown_connections 2024-03-24T10:47:53.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.415+0000 7f7bdab1b700 1 --2- 172.21.15.94:0/1740188704 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f7bc40389d0 0x7f7bc403ae80 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:53.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.415+0000 7f7bdab1b700 1 --2- 172.21.15.94:0/1740188704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7bd4066fd0 0x7f7bd410d920 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:53.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.415+0000 7f7bdab1b700 1 --2- 172.21.15.94:0/1740188704 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7bd40fff10 0x7f7bd410de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:53.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.415+0000 7f7bdab1b700 1 -- 172.21.15.94:0/1740188704 >> 172.21.15.94:0/1740188704 conn(0x7f7bd40f7440 msgr2=0x7f7bd40f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:47:53.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.415+0000 7f7bdab1b700 1 -- 172.21.15.94:0/1740188704 shutdown_connections 2024-03-24T10:47:53.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.415+0000 7f7bdab1b700 1 -- 172.21.15.94:0/1740188704 wait complete. 2024-03-24T10:47:53.416 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 18 2024-03-24T10:47:53.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.534+0000 7ff4d64f2700 1 -- 172.21.15.94:0/200184826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4d00662e0 msgr2=0x7ff4d00ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:53.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.534+0000 7ff4d64f2700 1 --2- 172.21.15.94:0/200184826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4d00662e0 0x7ff4d00ff9c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0004940 tx=0x7ff4c0009e30 comp rx=0 tx=0).stop 2024-03-24T10:47:53.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.534+0000 7ff4d64f2700 1 -- 172.21.15.94:0/200184826 shutdown_connections 2024-03-24T10:47:53.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.534+0000 7ff4d64f2700 1 --2- 172.21.15.94:0/200184826 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4d00fff00 0x7ff4d0102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:53.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.534+0000 7ff4d64f2700 1 --2- 172.21.15.94:0/200184826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4d00662e0 0x7ff4d00ff9c0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:53.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.534+0000 7ff4d64f2700 1 -- 172.21.15.94:0/200184826 >> 172.21.15.94:0/200184826 conn(0x7ff4d00f7430 msgr2=0x7ff4d00f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:47:53.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.534+0000 7ff4d64f2700 1 -- 172.21.15.94:0/200184826 shutdown_connections 2024-03-24T10:47:53.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.534+0000 7ff4d64f2700 1 -- 172.21.15.94:0/200184826 wait complete. 2024-03-24T10:47:53.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.534+0000 7ff4d64f2700 1 Processor -- start 2024-03-24T10:47:53.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.535+0000 7ff4d64f2700 1 -- start start 2024-03-24T10:47:53.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.535+0000 7ff4d64f2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4d00662e0 0x7ff4d010da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:47:53.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.535+0000 7ff4d64f2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4d00fff00 0x7ff4d010df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:47:53.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.535+0000 7ff4d64f2700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d010e540 con 0x7ff4d00fff00 2024-03-24T10:47:53.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.535+0000 7ff4d64f2700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d010ed40 con 0x7ff4d00662e0 2024-03-24T10:47:53.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.535+0000 7ff4cf7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4d00fff00 0x7ff4d010df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:47:53.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.535+0000 7ff4cf7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4d00fff00 0x7ff4d010df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45012/0 (socket says 172.21.15.94:45012) 2024-03-24T10:47:53.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.535+0000 7ff4cf7fe700 1 -- 172.21.15.94:0/2989798563 learned_addr learned my addr 172.21.15.94:0/2989798563 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:47:53.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.535+0000 7ff4cffff700 1 --2- 172.21.15.94:0/2989798563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4d00662e0 0x7ff4d010da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:47:53.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.535+0000 7ff4cf7fe700 1 -- 172.21.15.94:0/2989798563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4d00662e0 msgr2=0x7ff4d010da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:53.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.535+0000 7ff4cf7fe700 1 --2- 172.21.15.94:0/2989798563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4d00662e0 0x7ff4d010da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:53.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.535+0000 7ff4cf7fe700 1 -- 172.21.15.94:0/2989798563 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c0005040 con 0x7ff4d00fff00 2024-03-24T10:47:53.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.535+0000 7ff4cf7fe700 1 --2- 172.21.15.94:0/2989798563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4d00fff00 0x7ff4d010df40 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff4c400eb40 tx=0x7ff4c400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:47:53.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.535+0000 7ff4ccff9700 1 -- 172.21.15.94:0/2989798563 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4c4007cc0 con 0x7ff4d00fff00 2024-03-24T10:47:53.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.536+0000 7ff4ccff9700 1 -- 172.21.15.94:0/2989798563 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff4c400f040 con 0x7ff4d00fff00 2024-03-24T10:47:53.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.536+0000 7ff4d64f2700 1 -- 172.21.15.94:0/2989798563 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4d010efc0 con 0x7ff4d00fff00 2024-03-24T10:47:53.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.536+0000 7ff4ccff9700 1 -- 172.21.15.94:0/2989798563 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4c400ca10 con 0x7ff4d00fff00 2024-03-24T10:47:53.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.536+0000 7ff4d64f2700 1 -- 172.21.15.94:0/2989798563 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4d010f500 con 0x7ff4d00fff00 2024-03-24T10:47:53.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.536+0000 7ff4ccff9700 1 -- 172.21.15.94:0/2989798563 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 45872+0+0 (secure 0 0 0) 0x7ff4c400cb70 con 0x7ff4d00fff00 2024-03-24T10:47:53.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.536+0000 7ff4d64f2700 1 -- 172.21.15.94:0/2989798563 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4d005f5f0 con 0x7ff4d00fff00 2024-03-24T10:47:53.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.537+0000 7ff4ccff9700 1 --2- 172.21.15.94:0/2989798563 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7ff4b80389e0 0x7ff4b803ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:47:53.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.537+0000 7ff4ccff9700 1 -- 172.21.15.94:0/2989798563 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7ff4c404c400 con 0x7ff4d00fff00 2024-03-24T10:47:53.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.537+0000 7ff4cffff700 1 --2- 172.21.15.94:0/2989798563 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7ff4b80389e0 0x7ff4b803ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:47:53.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.537+0000 7ff4cffff700 1 --2- 172.21.15.94:0/2989798563 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7ff4b80389e0 0x7ff4b803ae90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0004890 tx=0x7ff4c0009e30 comp rx=0 tx=0).ready entity=mgr.14774 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:47:53.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.541+0000 7ff4ccff9700 1 -- 172.21.15.94:0/2989798563 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7ff4c4012db0 con 0x7ff4d00fff00 2024-03-24T10:47:53.745 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-03-24T10:47:53.746 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-03-24T10:47:53.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.762+0000 7ff4ccff9700 1 -- 172.21.15.94:0/2989798563 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 28) v1 ==== 45928+0+0 (secure 0 0 0) 0x7ff4c4012340 con 0x7ff4d00fff00 2024-03-24T10:47:53.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.779+0000 7ff4d64f2700 1 -- 172.21.15.94:0/2989798563 --> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff4d010f8a0 con 0x7ff4b80389e0 2024-03-24T10:47:53.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.780+0000 7ff4ccff9700 1 -- 172.21.15.94:0/2989798563 <== mgr.14774 v2:172.21.15.94:6800/4143126939 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7ff4d010f8a0 con 0x7ff4b80389e0 2024-03-24T10:47:53.780 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-03-24T10:47:53.780 INFO:teuthology.orchestra.run.smithi094.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919", 2024-03-24T10:47:53.780 INFO:teuthology.orchestra.run.smithi094.stdout: "in_progress": true, 2024-03-24T10:47:53.780 INFO:teuthology.orchestra.run.smithi094.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T10:47:53.780 INFO:teuthology.orchestra.run.smithi094.stdout: "services_complete": [ 2024-03-24T10:47:53.780 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr" 2024-03-24T10:47:53.780 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-03-24T10:47:53.780 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": "2/21 daemons upgraded", 2024-03-24T10:47:53.780 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "", 2024-03-24T10:47:53.780 INFO:teuthology.orchestra.run.smithi094.stdout: "is_paused": false 2024-03-24T10:47:53.781 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-03-24T10:47:53.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.783+0000 7ff4d64f2700 1 -- 172.21.15.94:0/2989798563 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7ff4b80389e0 msgr2=0x7ff4b803ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:53.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.783+0000 7ff4d64f2700 1 --2- 172.21.15.94:0/2989798563 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7ff4b80389e0 0x7ff4b803ae90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0004890 tx=0x7ff4c0009e30 comp rx=0 tx=0).stop 2024-03-24T10:47:53.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.783+0000 7ff4d64f2700 1 -- 172.21.15.94:0/2989798563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4d00fff00 msgr2=0x7ff4d010df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:53.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.783+0000 7ff4d64f2700 1 --2- 172.21.15.94:0/2989798563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4d00fff00 0x7ff4d010df40 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff4c400eb40 tx=0x7ff4c400b040 comp rx=0 tx=0).stop 2024-03-24T10:47:53.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.783+0000 7ff4d64f2700 1 -- 172.21.15.94:0/2989798563 shutdown_connections 2024-03-24T10:47:53.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.783+0000 7ff4d64f2700 1 --2- 172.21.15.94:0/2989798563 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7ff4b80389e0 0x7ff4b803ae90 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:53.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.783+0000 7ff4d64f2700 1 --2- 172.21.15.94:0/2989798563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4d00662e0 0x7ff4d010da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:53.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.783+0000 7ff4d64f2700 1 --2- 172.21.15.94:0/2989798563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4d00fff00 0x7ff4d010df40 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:53.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.783+0000 7ff4d64f2700 1 -- 172.21.15.94:0/2989798563 >> 172.21.15.94:0/2989798563 conn(0x7ff4d00f7430 msgr2=0x7ff4d00f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:47:53.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.783+0000 7ff4d64f2700 1 -- 172.21.15.94:0/2989798563 shutdown_connections 2024-03-24T10:47:53.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.783+0000 7ff4d64f2700 1 -- 172.21.15.94:0/2989798563 wait complete. 2024-03-24T10:47:53.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.937+0000 7f7132ba5700 1 -- 172.21.15.94:0/830714105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f712c0fbcd0 msgr2=0x7f712c0fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:53.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.937+0000 7f7132ba5700 1 --2- 172.21.15.94:0/830714105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f712c0fbcd0 0x7f712c0fc0e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7120004940 tx=0x7f7120009e30 comp rx=0 tx=0).stop 2024-03-24T10:47:53.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.938+0000 7f7132ba5700 1 -- 172.21.15.94:0/830714105 shutdown_connections 2024-03-24T10:47:53.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.938+0000 7f7132ba5700 1 --2- 172.21.15.94:0/830714105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f712c0fcf80 0x7f712c0fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:53.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.938+0000 7f7132ba5700 1 --2- 172.21.15.94:0/830714105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f712c0fbcd0 0x7f712c0fc0e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:53.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.938+0000 7f7132ba5700 1 -- 172.21.15.94:0/830714105 >> 172.21.15.94:0/830714105 conn(0x7f712c0f7430 msgr2=0x7f712c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:47:53.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.938+0000 7f7132ba5700 1 -- 172.21.15.94:0/830714105 shutdown_connections 2024-03-24T10:47:53.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.938+0000 7f7132ba5700 1 -- 172.21.15.94:0/830714105 wait complete. 2024-03-24T10:47:53.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.938+0000 7f7132ba5700 1 Processor -- start 2024-03-24T10:47:53.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.938+0000 7f7132ba5700 1 -- start start 2024-03-24T10:47:53.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.939+0000 7f7132ba5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f712c0fbcd0 0x7f712c10da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:47:53.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.939+0000 7f7132ba5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f712c0fcf80 0x7f712c10df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:47:53.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.939+0000 7f7132ba5700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712c10e610 con 0x7f712c0fcf80 2024-03-24T10:47:53.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.939+0000 7f7132ba5700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712c10ee00 con 0x7f712c0fbcd0 2024-03-24T10:47:53.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.939+0000 7f712bfff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f712c0fcf80 0x7f712c10df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:47:53.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.939+0000 7f7130941700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f712c0fbcd0 0x7f712c10da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:47:53.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.939+0000 7f712bfff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f712c0fcf80 0x7f712c10df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45030/0 (socket says 172.21.15.94:45030) 2024-03-24T10:47:53.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.939+0000 7f712bfff700 1 -- 172.21.15.94:0/3015991421 learned_addr learned my addr 172.21.15.94:0/3015991421 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:47:53.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.940+0000 7f712bfff700 1 -- 172.21.15.94:0/3015991421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f712c0fbcd0 msgr2=0x7f712c10da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:53.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.940+0000 7f712bfff700 1 --2- 172.21.15.94:0/3015991421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f712c0fbcd0 0x7f712c10da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:53.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.940+0000 7f712bfff700 1 -- 172.21.15.94:0/3015991421 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7120005040 con 0x7f712c0fcf80 2024-03-24T10:47:53.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.940+0000 7f712bfff700 1 --2- 172.21.15.94:0/3015991421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f712c0fcf80 0x7f712c10df80 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7118009f50 tx=0x7f7118009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:47:53.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.940+0000 7f71297fa700 1 -- 172.21.15.94:0/3015991421 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7118010a20 con 0x7f712c0fcf80 2024-03-24T10:47:53.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.940+0000 7f71297fa700 1 -- 172.21.15.94:0/3015991421 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7118010b80 con 0x7f712c0fcf80 2024-03-24T10:47:53.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.941+0000 7f71297fa700 1 -- 172.21.15.94:0/3015991421 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71180175b0 con 0x7f712c0fcf80 2024-03-24T10:47:53.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.941+0000 7f7132ba5700 1 -- 172.21.15.94:0/3015991421 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f712c10f0e0 con 0x7f712c0fcf80 2024-03-24T10:47:53.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.941+0000 7f7132ba5700 1 -- 172.21.15.94:0/3015991421 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f712c10f650 con 0x7f712c0fcf80 2024-03-24T10:47:53.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.942+0000 7f71297fa700 1 -- 172.21.15.94:0/3015991421 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 28) v1 ==== 45928+0+0 (secure 0 0 0) 0x7f7118016070 con 0x7f712c0fcf80 2024-03-24T10:47:53.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.942+0000 7f7132ba5700 1 -- 172.21.15.94:0/3015991421 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f712c05f5f0 con 0x7f712c0fcf80 2024-03-24T10:47:53.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.942+0000 7f71297fa700 1 --2- 172.21.15.94:0/3015991421 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f711c038b00 0x7f711c03afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:47:53.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.942+0000 7f71297fa700 1 -- 172.21.15.94:0/3015991421 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f7118050190 con 0x7f712c0fcf80 2024-03-24T10:47:53.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.943+0000 7f7130941700 1 --2- 172.21.15.94:0/3015991421 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f711c038b00 0x7f711c03afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:47:53.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.944+0000 7f7130941700 1 --2- 172.21.15.94:0/3015991421 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f711c038b00 0x7f711c03afb0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f712000b3b0 tx=0x7f7120009e30 comp rx=0 tx=0).ready entity=mgr.14774 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:47:53.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:53.950+0000 7f71297fa700 1 -- 172.21.15.94:0/3015991421 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7f71180166e0 con 0x7f712c0fcf80 2024-03-24T10:47:53.950 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:53 smithi094 ceph-mon[29446]: pgmap v4: 65 pgs: 65 active+clean; 216 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-03-24T10:47:53.950 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:53 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/307606340' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:47:53.950 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:53 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/1740188704' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T10:47:54.041 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:53 smithi096 ceph-mon[38097]: pgmap v4: 65 pgs: 65 active+clean; 216 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-03-24T10:47:54.042 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:53 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/307606340' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:47:54.042 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:53 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/1740188704' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T10:47:54.118 DEBUG:teuthology.parallel:result is None 2024-03-24T10:47:54.118 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-24T10:47:54.151 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-24T10:47:54.152 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-03-24T10:47:54.187 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-03-24T10:47:54.188 DEBUG:teuthology.parallel:result is None 2024-03-24T10:47:54.282 INFO:teuthology.orchestra.run.smithi094.stdout:HEALTH_OK 2024-03-24T10:47:54.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:54.282+0000 7f7132ba5700 1 -- 172.21.15.94:0/3015991421 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f712c04bed0 con 0x7f712c0fcf80 2024-03-24T10:47:54.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:54.282+0000 7f71297fa700 1 -- 172.21.15.94:0/3015991421 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f7118020550 con 0x7f712c0fcf80 2024-03-24T10:47:54.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:54.285+0000 7f7132ba5700 1 -- 172.21.15.94:0/3015991421 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f711c038b00 msgr2=0x7f711c03afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:54.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:54.285+0000 7f7132ba5700 1 --2- 172.21.15.94:0/3015991421 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f711c038b00 0x7f711c03afb0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f712000b3b0 tx=0x7f7120009e30 comp rx=0 tx=0).stop 2024-03-24T10:47:54.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:54.285+0000 7f7132ba5700 1 -- 172.21.15.94:0/3015991421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f712c0fcf80 msgr2=0x7f712c10df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:47:54.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:54.285+0000 7f7132ba5700 1 --2- 172.21.15.94:0/3015991421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f712c0fcf80 0x7f712c10df80 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7118009f50 tx=0x7f7118009250 comp rx=0 tx=0).stop 2024-03-24T10:47:54.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:54.285+0000 7f7132ba5700 1 -- 172.21.15.94:0/3015991421 shutdown_connections 2024-03-24T10:47:54.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:54.285+0000 7f7132ba5700 1 --2- 172.21.15.94:0/3015991421 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f711c038b00 0x7f711c03afb0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:54.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:54.285+0000 7f7132ba5700 1 --2- 172.21.15.94:0/3015991421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f712c0fbcd0 0x7f712c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:54.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:54.285+0000 7f7132ba5700 1 --2- 172.21.15.94:0/3015991421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f712c0fcf80 0x7f712c10df80 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:47:54.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:54.285+0000 7f7132ba5700 1 -- 172.21.15.94:0/3015991421 >> 172.21.15.94:0/3015991421 conn(0x7f712c0f7430 msgr2=0x7f712c0f8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:47:54.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:54.285+0000 7f7132ba5700 1 -- 172.21.15.94:0/3015991421 shutdown_connections 2024-03-24T10:47:54.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:47:54.285+0000 7f7132ba5700 1 -- 172.21.15.94:0/3015991421 wait complete. 2024-03-24T10:47:54.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:54 smithi094 ceph-mon[29446]: mgrmap e28: smithi094.yaethu(active, since 3s) 2024-03-24T10:47:54.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:54 smithi094 ceph-mon[29446]: from='client.14822 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:47:54.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:54 smithi094 ceph-mon[29446]: from='client.? 172.21.15.94:0/3015991421' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T10:47:55.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:54 smithi096 ceph-mon[38097]: mgrmap e28: smithi094.yaethu(active, since 3s) 2024-03-24T10:47:55.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:54 smithi096 ceph-mon[38097]: from='client.14822 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:47:55.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:54 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/3015991421' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T10:47:56.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:55 smithi096 ceph-mon[38097]: pgmap v5: 65 pgs: 65 active+clean; 216 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-03-24T10:47:56.096 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:55 smithi094 ceph-mon[29446]: pgmap v5: 65 pgs: 65 active+clean; 216 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-03-24T10:47:57.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:57 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:47:57.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:57 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:47:57.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:57 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:47:57.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:57 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:47:57.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:57 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:47:57.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:57 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:47:57.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:57 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:47:57.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:57 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:47:58.623 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:58 smithi094 ceph-mon[29446]: pgmap v6: 65 pgs: 65 active+clean; 216 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-03-24T10:47:58.623 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:58 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:47:58.623 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:58 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:47:58.623 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:58 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:47:58.623 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:58 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:47:58.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:58 smithi096 ceph-mon[38097]: pgmap v6: 65 pgs: 65 active+clean; 216 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-03-24T10:47:58.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:58 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:47:58.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:58 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:47:58.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:58 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:47:58.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:58 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:47:59.638 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:59 smithi094 ceph-mon[29446]: Standby manager daemon smithi096.gfonnu started 2024-03-24T10:47:59.639 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:59 smithi094 ceph-mon[29446]: from='mgr.? 172.21.15.96:0/1242234477' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi096.gfonnu/crt"}]: dispatch 2024-03-24T10:47:59.639 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:59 smithi094 ceph-mon[29446]: from='mgr.? 172.21.15.96:0/1242234477' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T10:47:59.639 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:59 smithi094 ceph-mon[29446]: from='mgr.? 172.21.15.96:0/1242234477' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi096.gfonnu/key"}]: dispatch 2024-03-24T10:47:59.639 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:47:59 smithi094 ceph-mon[29446]: from='mgr.? 172.21.15.96:0/1242234477' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T10:47:59.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:59 smithi096 ceph-mon[38097]: Standby manager daemon smithi096.gfonnu started 2024-03-24T10:47:59.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:59 smithi096 ceph-mon[38097]: from='mgr.? 172.21.15.96:0/1242234477' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi096.gfonnu/crt"}]: dispatch 2024-03-24T10:47:59.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:59 smithi096 ceph-mon[38097]: from='mgr.? 172.21.15.96:0/1242234477' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T10:47:59.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:59 smithi096 ceph-mon[38097]: from='mgr.? 172.21.15.96:0/1242234477' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi096.gfonnu/key"}]: dispatch 2024-03-24T10:47:59.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:47:59 smithi096 ceph-mon[38097]: from='mgr.? 172.21.15.96:0/1242234477' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T10:48:00.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:00 smithi094 ceph-mon[29446]: pgmap v7: 65 pgs: 65 active+clean; 201 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 538 KiB/s rd, 521 KiB/s wr, 172 op/s 2024-03-24T10:48:00.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:00 smithi094 ceph-mon[29446]: mgrmap e29: smithi094.yaethu(active, since 8s), standbys: smithi096.gfonnu 2024-03-24T10:48:00.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:00 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mgr metadata", "who": "smithi096.gfonnu", "id": "smithi096.gfonnu"}]: dispatch 2024-03-24T10:48:00.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:00 smithi096 ceph-mon[38097]: pgmap v7: 65 pgs: 65 active+clean; 201 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 538 KiB/s rd, 521 KiB/s wr, 172 op/s 2024-03-24T10:48:00.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:00 smithi096 ceph-mon[38097]: mgrmap e29: smithi094.yaethu(active, since 8s), standbys: smithi096.gfonnu 2024-03-24T10:48:00.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:00 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mgr metadata", "who": "smithi096.gfonnu", "id": "smithi096.gfonnu"}]: dispatch 2024-03-24T10:48:02.750 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:02 smithi094 ceph-mon[29446]: pgmap v8: 65 pgs: 65 active+clean; 201 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 418 KiB/s rd, 405 KiB/s wr, 133 op/s 2024-03-24T10:48:02.750 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:02 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:02.750 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:02 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:02.750 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:02 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:48:02.750 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:02 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:48:02.751 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:02 smithi094 ceph-mon[29446]: Updating smithi094:/etc/ceph/ceph.conf 2024-03-24T10:48:02.751 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:02 smithi094 ceph-mon[29446]: Updating smithi096:/etc/ceph/ceph.conf 2024-03-24T10:48:02.751 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:02 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:02.751 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:02 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:02.751 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:02 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:02.751 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:02 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:02.751 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:02 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:02.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:02 smithi096 ceph-mon[38097]: pgmap v8: 65 pgs: 65 active+clean; 201 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 418 KiB/s rd, 405 KiB/s wr, 133 op/s 2024-03-24T10:48:02.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:02 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:02.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:02 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:02.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:02 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:48:02.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:02 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:48:02.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:02 smithi096 ceph-mon[38097]: Updating smithi094:/etc/ceph/ceph.conf 2024-03-24T10:48:02.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:02 smithi096 ceph-mon[38097]: Updating smithi096:/etc/ceph/ceph.conf 2024-03-24T10:48:02.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:02 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:02.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:02 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:02.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:02 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:02.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:02 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:02.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:02 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:03.552 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:03 smithi094 ceph-mon[29446]: Updating smithi094:/var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T10:48:03.552 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:03 smithi094 ceph-mon[29446]: Updating smithi096:/var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T10:48:03.552 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:03 smithi094 ceph-mon[29446]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-03-24T10:48:03.552 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:03 smithi094 ceph-mon[29446]: Updating smithi096:/etc/ceph/ceph.client.admin.keyring 2024-03-24T10:48:03.552 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:03 smithi094 ceph-mon[29446]: Updating smithi094:/var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T10:48:03.552 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:03 smithi094 ceph-mon[29446]: Updating smithi096:/var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T10:48:03.552 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:03 smithi094 ceph-mon[29446]: Reconfiguring prometheus.smithi094 (dependencies changed)... 2024-03-24T10:48:03.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:03 smithi096 ceph-mon[38097]: Updating smithi094:/var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T10:48:03.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:03 smithi096 ceph-mon[38097]: Updating smithi096:/var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T10:48:03.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:03 smithi096 ceph-mon[38097]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-03-24T10:48:03.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:03 smithi096 ceph-mon[38097]: Updating smithi096:/etc/ceph/ceph.client.admin.keyring 2024-03-24T10:48:03.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:03 smithi096 ceph-mon[38097]: Updating smithi094:/var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T10:48:03.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:03 smithi096 ceph-mon[38097]: Updating smithi096:/var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T10:48:03.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:03 smithi096 ceph-mon[38097]: Reconfiguring prometheus.smithi094 (dependencies changed)... 2024-03-24T10:48:04.613 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:04 smithi094 ceph-mon[29446]: Reconfiguring daemon prometheus.smithi094 on smithi094 2024-03-24T10:48:04.613 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:04 smithi094 ceph-mon[29446]: pgmap v9: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 344 KiB/s rd, 339 KiB/s wr, 121 op/s 2024-03-24T10:48:04.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:04 smithi096 ceph-mon[38097]: Reconfiguring daemon prometheus.smithi094 on smithi094 2024-03-24T10:48:04.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:04 smithi096 ceph-mon[38097]: pgmap v9: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 344 KiB/s rd, 339 KiB/s wr, 121 op/s 2024-03-24T10:48:06.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:06 smithi094 ceph-mon[29446]: pgmap v10: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 315 KiB/s rd, 310 KiB/s wr, 111 op/s 2024-03-24T10:48:06.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:06 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:48:06.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:06 smithi096 ceph-mon[38097]: pgmap v10: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 315 KiB/s rd, 310 KiB/s wr, 111 op/s 2024-03-24T10:48:06.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:06 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:48:08.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:08 smithi096 ceph-mon[38097]: pgmap v11: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 315 KiB/s rd, 310 KiB/s wr, 111 op/s 2024-03-24T10:48:08.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:08 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:08.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:08 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:08.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:08 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T10:48:08.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:08 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:48:08.839 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:08 smithi094 ceph-mon[29446]: pgmap v11: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 315 KiB/s rd, 310 KiB/s wr, 111 op/s 2024-03-24T10:48:08.839 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:08 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:08.839 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:08 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:08.839 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:08 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T10:48:08.839 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:08 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:48:09.835 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:09 smithi096 ceph-mon[38097]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T10:48:09.835 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:09 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi096.gfonnu", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T10:48:09.835 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:09 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T10:48:09.835 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:09 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:48:09.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:09 smithi094 ceph-mon[29446]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T10:48:09.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:09 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi096.gfonnu", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T10:48:09.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:09 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T10:48:09.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:09 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:48:10.790 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:10 smithi096 ceph-mon[38097]: pgmap v12: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 321 KiB/s rd, 310 KiB/s wr, 112 op/s 2024-03-24T10:48:10.790 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:10 smithi096 ceph-mon[38097]: Upgrade: Updating mgr.smithi096.gfonnu 2024-03-24T10:48:10.790 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:10 smithi096 ceph-mon[38097]: Deploying daemon mgr.smithi096.gfonnu on smithi096 2024-03-24T10:48:10.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:10 smithi094 ceph-mon[29446]: pgmap v12: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 321 KiB/s rd, 310 KiB/s wr, 112 op/s 2024-03-24T10:48:10.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:10 smithi094 ceph-mon[29446]: Upgrade: Updating mgr.smithi096.gfonnu 2024-03-24T10:48:10.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:10 smithi094 ceph-mon[29446]: Deploying daemon mgr.smithi096.gfonnu on smithi096 2024-03-24T10:48:11.646 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:11 smithi096 ceph-mon[38097]: pgmap v13: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 7.6 KiB/s rd, 6.6 KiB/s wr, 11 op/s 2024-03-24T10:48:11.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:11 smithi094 ceph-mon[29446]: pgmap v13: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 7.6 KiB/s rd, 6.6 KiB/s wr, 11 op/s 2024-03-24T10:48:14.086 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:14 smithi096 ceph-mon[38097]: pgmap v14: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 8.0 KiB/s rd, 6.6 KiB/s wr, 12 op/s 2024-03-24T10:48:14.086 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:14 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:14.086 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:14 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:14.086 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:14 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:48:14.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:14 smithi094 ceph-mon[29446]: pgmap v14: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 8.0 KiB/s rd, 6.6 KiB/s wr, 12 op/s 2024-03-24T10:48:14.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:14 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:14.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:14 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:14.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:14 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:48:16.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:16 smithi096 ceph-mon[38097]: pgmap v15: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 6.0 KiB/s rd, 1 op/s 2024-03-24T10:48:16.341 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:16 smithi094 ceph-mon[29446]: pgmap v15: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 6.0 KiB/s rd, 1 op/s 2024-03-24T10:48:17.284 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:17 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:17.284 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:17 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:17.480 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:17 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:17.480 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:17 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:18.658 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:18 smithi094 ceph-mon[29446]: pgmap v16: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 6.0 KiB/s rd, 1 op/s 2024-03-24T10:48:18.658 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:18 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:18.658 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:18 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:18.658 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:18 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:18.658 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:18 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:18.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:18 smithi096 ceph-mon[38097]: pgmap v16: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 6.0 KiB/s rd, 1 op/s 2024-03-24T10:48:18.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:18 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:18.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:18 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:18.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:18 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:18.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:18 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:20.701 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:20 smithi096 ceph-mon[38097]: pgmap v17: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 6.4 KiB/s rd, 2 op/s 2024-03-24T10:48:20.701 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:20 smithi096 ceph-mon[38097]: Standby manager daemon smithi096.gfonnu restarted 2024-03-24T10:48:20.701 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:20 smithi096 ceph-mon[38097]: Standby manager daemon smithi096.gfonnu started 2024-03-24T10:48:20.701 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:20 smithi096 ceph-mon[38097]: from='mgr.? 172.21.15.96:0/1227175982' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi096.gfonnu/crt"}]: dispatch 2024-03-24T10:48:20.701 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:20 smithi096 ceph-mon[38097]: from='mgr.? 172.21.15.96:0/1227175982' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T10:48:20.701 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:20 smithi096 ceph-mon[38097]: from='mgr.? 172.21.15.96:0/1227175982' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi096.gfonnu/key"}]: dispatch 2024-03-24T10:48:20.701 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:20 smithi096 ceph-mon[38097]: from='mgr.? 172.21.15.96:0/1227175982' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T10:48:20.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:20 smithi094 ceph-mon[29446]: pgmap v17: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 6.4 KiB/s rd, 2 op/s 2024-03-24T10:48:20.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:20 smithi094 ceph-mon[29446]: Standby manager daemon smithi096.gfonnu restarted 2024-03-24T10:48:20.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:20 smithi094 ceph-mon[29446]: Standby manager daemon smithi096.gfonnu started 2024-03-24T10:48:20.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:20 smithi094 ceph-mon[29446]: from='mgr.? 172.21.15.96:0/1227175982' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi096.gfonnu/crt"}]: dispatch 2024-03-24T10:48:20.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:20 smithi094 ceph-mon[29446]: from='mgr.? 172.21.15.96:0/1227175982' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T10:48:20.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:20 smithi094 ceph-mon[29446]: from='mgr.? 172.21.15.96:0/1227175982' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi096.gfonnu/key"}]: dispatch 2024-03-24T10:48:20.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:20 smithi094 ceph-mon[29446]: from='mgr.? 172.21.15.96:0/1227175982' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T10:48:21.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:21 smithi094 ceph-mon[29446]: mgrmap e30: smithi094.yaethu(active, since 29s), standbys: smithi096.gfonnu 2024-03-24T10:48:21.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:21 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:48:21.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:21 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:21.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:21 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:21.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:21 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:48:21.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:21 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:48:21.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:21 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:21.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:21 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:48:21.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:21 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:48:21.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:21 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:21.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:21 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi094.yaethu"}]: dispatch 2024-03-24T10:48:21.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:21 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi094.yaethu"}]': finished 2024-03-24T10:48:21.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:21 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi096.gfonnu"}]: dispatch 2024-03-24T10:48:21.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:21 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi096.gfonnu"}]': finished 2024-03-24T10:48:21.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:21 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-24T10:48:21.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:21 smithi096 ceph-mon[38097]: mgrmap e30: smithi094.yaethu(active, since 29s), standbys: smithi096.gfonnu 2024-03-24T10:48:21.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:21 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:48:21.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:21 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:21.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:21 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:21.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:21 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:48:21.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:21 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:48:21.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:21 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:21.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:21 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:48:21.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:21 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:48:21.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:21 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:21.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:21 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi094.yaethu"}]: dispatch 2024-03-24T10:48:21.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:21 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi094.yaethu"}]': finished 2024-03-24T10:48:21.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:21 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi096.gfonnu"}]: dispatch 2024-03-24T10:48:21.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:21 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi096.gfonnu"}]': finished 2024-03-24T10:48:21.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:21 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-24T10:48:22.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:22 smithi094 ceph-mon[29446]: pgmap v18: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:48:22.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:22 smithi094 ceph-mon[29446]: Upgrade: Setting container_image for all mgr 2024-03-24T10:48:22.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:22 smithi096 ceph-mon[38097]: pgmap v18: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:48:22.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:22 smithi096 ceph-mon[38097]: Upgrade: Setting container_image for all mgr 2024-03-24T10:48:24.157 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:23 smithi094 ceph-mon[29446]: pgmap v19: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:48:24.157 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:23 smithi094 ceph-mon[29446]: Upgrade: Updating mon.smithi094 2024-03-24T10:48:24.157 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:23 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:24.157 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:23 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T10:48:24.157 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:23 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T10:48:24.157 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:23 smithi094 ceph-mon[29446]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:48:24.157 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:23 smithi094 ceph-mon[29446]: Deploying daemon mon.smithi094 on smithi094 2024-03-24T10:48:24.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:23 smithi096 ceph-mon[38097]: pgmap v19: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:48:24.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:23 smithi096 ceph-mon[38097]: Upgrade: Updating mon.smithi094 2024-03-24T10:48:24.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:23 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:24.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:23 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T10:48:24.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:23 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T10:48:24.327 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:23 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:48:24.327 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:23 smithi096 ceph-mon[38097]: Deploying daemon mon.smithi094 on smithi094 2024-03-24T10:48:24.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.390+0000 7f1e19e25700 1 -- 172.21.15.94:0/3983358233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e140fce70 msgr2=0x7f1e140fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:24.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.390+0000 7f1e19e25700 1 --2- 172.21.15.94:0/3983358233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e140fce70 0x7f1e140fd280 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1e08004940 tx=0x7f1e08009e30 comp rx=0 tx=0).stop 2024-03-24T10:48:24.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.390+0000 7f1e19e25700 1 -- 172.21.15.94:0/3983358233 shutdown_connections 2024-03-24T10:48:24.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.390+0000 7f1e19e25700 1 --2- 172.21.15.94:0/3983358233 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e140fdfe0 0x7f1e140fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:24.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.390+0000 7f1e19e25700 1 --2- 172.21.15.94:0/3983358233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e140fce70 0x7f1e140fd280 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:24.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.390+0000 7f1e19e25700 1 -- 172.21.15.94:0/3983358233 >> 172.21.15.94:0/3983358233 conn(0x7f1e140f8550 msgr2=0x7f1e140fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:48:24.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.390+0000 7f1e19e25700 1 -- 172.21.15.94:0/3983358233 shutdown_connections 2024-03-24T10:48:24.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.390+0000 7f1e19e25700 1 -- 172.21.15.94:0/3983358233 wait complete. 2024-03-24T10:48:24.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.391+0000 7f1e19e25700 1 Processor -- start 2024-03-24T10:48:24.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.391+0000 7f1e19e25700 1 -- start start 2024-03-24T10:48:24.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.391+0000 7f1e19e25700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e140fce70 0x7f1e1410d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:48:24.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.391+0000 7f1e19e25700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e140fdfe0 0x7f1e1410de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:48:24.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.391+0000 7f1e19e25700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1410e510 con 0x7f1e140fdfe0 2024-03-24T10:48:24.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.391+0000 7f1e19e25700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e14112eb0 con 0x7f1e140fce70 2024-03-24T10:48:24.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.391+0000 7f1e12ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e140fdfe0 0x7f1e1410de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:48:24.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.391+0000 7f1e12ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e140fdfe0 0x7f1e1410de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54212/0 (socket says 172.21.15.94:54212) 2024-03-24T10:48:24.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.391+0000 7f1e12ffd700 1 -- 172.21.15.94:0/1712376135 learned_addr learned my addr 172.21.15.94:0/1712376135 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:48:24.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.392+0000 7f1e137fe700 1 --2- 172.21.15.94:0/1712376135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e140fce70 0x7f1e1410d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:48:24.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.392+0000 7f1e12ffd700 1 -- 172.21.15.94:0/1712376135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e140fce70 msgr2=0x7f1e1410d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:24.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.392+0000 7f1e12ffd700 1 --2- 172.21.15.94:0/1712376135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e140fce70 0x7f1e1410d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:24.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.392+0000 7f1e12ffd700 1 -- 172.21.15.94:0/1712376135 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e08005040 con 0x7f1e140fdfe0 2024-03-24T10:48:24.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.392+0000 7f1e12ffd700 1 --2- 172.21.15.94:0/1712376135 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e140fdfe0 0x7f1e1410de80 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1e0400b790 tx=0x7f1e04009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:48:24.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.392+0000 7f1dfbfff700 1 -- 172.21.15.94:0/1712376135 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e0400f770 con 0x7f1e140fdfe0 2024-03-24T10:48:24.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.392+0000 7f1dfbfff700 1 -- 172.21.15.94:0/1712376135 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1e0400fda0 con 0x7f1e140fdfe0 2024-03-24T10:48:24.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.392+0000 7f1e19e25700 1 -- 172.21.15.94:0/1712376135 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e14113110 con 0x7f1e140fdfe0 2024-03-24T10:48:24.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.392+0000 7f1dfbfff700 1 -- 172.21.15.94:0/1712376135 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e0400d310 con 0x7f1e140fdfe0 2024-03-24T10:48:24.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.392+0000 7f1e19e25700 1 -- 172.21.15.94:0/1712376135 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e14113680 con 0x7f1e140fdfe0 2024-03-24T10:48:24.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.393+0000 7f1e19e25700 1 -- 172.21.15.94:0/1712376135 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e1404c710 con 0x7f1e140fdfe0 2024-03-24T10:48:24.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.394+0000 7f1dfbfff700 1 -- 172.21.15.94:0/1712376135 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 30) v1 ==== 91301+0+0 (secure 0 0 0) 0x7f1e04010420 con 0x7f1e140fdfe0 2024-03-24T10:48:24.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.394+0000 7f1dfbfff700 1 --2- 172.21.15.94:0/1712376135 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f1dfc06d210 0x7f1dfc06f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:48:24.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.394+0000 7f1e137fe700 1 --2- 172.21.15.94:0/1712376135 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f1dfc06d210 0x7f1dfc06f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:48:24.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.394+0000 7f1dfbfff700 1 -- 172.21.15.94:0/1712376135 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f1e0408c770 con 0x7f1e140fdfe0 2024-03-24T10:48:24.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.395+0000 7f1e137fe700 1 --2- 172.21.15.94:0/1712376135 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f1dfc06d210 0x7f1dfc06f6c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1e0800b3b0 tx=0x7f1e08009e30 comp rx=0 tx=0).ready entity=mgr.14774 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:48:24.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.399+0000 7f1dfbfff700 1 -- 172.21.15.94:0/1712376135 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7f1e0405a260 con 0x7f1e140fdfe0 2024-03-24T10:48:24.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.616+0000 7f1e19e25700 1 -- 172.21.15.94:0/1712376135 --> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e14102280 con 0x7f1dfc06d210 2024-03-24T10:48:24.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.617+0000 7f1dfbfff700 1 -- 172.21.15.94:0/1712376135 <== mgr.14774 v2:172.21.15.94:6800/4143126939 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f1e14102280 con 0x7f1dfc06d210 2024-03-24T10:48:24.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.621+0000 7f1e19e25700 1 -- 172.21.15.94:0/1712376135 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f1dfc06d210 msgr2=0x7f1dfc06f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:24.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.621+0000 7f1e19e25700 1 --2- 172.21.15.94:0/1712376135 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f1dfc06d210 0x7f1dfc06f6c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1e0800b3b0 tx=0x7f1e08009e30 comp rx=0 tx=0).stop 2024-03-24T10:48:24.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.621+0000 7f1e19e25700 1 -- 172.21.15.94:0/1712376135 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e140fdfe0 msgr2=0x7f1e1410de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:24.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.621+0000 7f1e19e25700 1 --2- 172.21.15.94:0/1712376135 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e140fdfe0 0x7f1e1410de80 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1e0400b790 tx=0x7f1e04009250 comp rx=0 tx=0).stop 2024-03-24T10:48:24.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.621+0000 7f1e19e25700 1 -- 172.21.15.94:0/1712376135 shutdown_connections 2024-03-24T10:48:24.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.621+0000 7f1e19e25700 1 --2- 172.21.15.94:0/1712376135 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f1dfc06d210 0x7f1dfc06f6c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:24.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.621+0000 7f1e19e25700 1 --2- 172.21.15.94:0/1712376135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e140fce70 0x7f1e1410d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:24.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.621+0000 7f1e19e25700 1 --2- 172.21.15.94:0/1712376135 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e140fdfe0 0x7f1e1410de80 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:24.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.621+0000 7f1e19e25700 1 -- 172.21.15.94:0/1712376135 >> 172.21.15.94:0/1712376135 conn(0x7f1e140f8550 msgr2=0x7f1e140f9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:48:24.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.621+0000 7f1e19e25700 1 -- 172.21.15.94:0/1712376135 shutdown_connections 2024-03-24T10:48:24.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.621+0000 7f1e19e25700 1 -- 172.21.15.94:0/1712376135 wait complete. 2024-03-24T10:48:24.632 INFO:teuthology.orchestra.run.smithi094.stdout:true 2024-03-24T10:48:24.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.731+0000 7fdbee0f1700 1 -- 172.21.15.94:0/2543654845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbe80fcf80 msgr2=0x7fdbe80fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:24.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.731+0000 7fdbee0f1700 1 --2- 172.21.15.94:0/2543654845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbe80fcf80 0x7fdbe80fd3f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc004940 tx=0x7fdbdc009e30 comp rx=0 tx=0).stop 2024-03-24T10:48:24.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.731+0000 7fdbee0f1700 1 -- 172.21.15.94:0/2543654845 shutdown_connections 2024-03-24T10:48:24.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.731+0000 7fdbee0f1700 1 --2- 172.21.15.94:0/2543654845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbe80fcf80 0x7fdbe80fd3f0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:24.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.731+0000 7fdbee0f1700 1 --2- 172.21.15.94:0/2543654845 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbe80fbcd0 0x7fdbe80fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:24.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.731+0000 7fdbee0f1700 1 -- 172.21.15.94:0/2543654845 >> 172.21.15.94:0/2543654845 conn(0x7fdbe80f7430 msgr2=0x7fdbe80f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:48:24.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.731+0000 7fdbee0f1700 1 -- 172.21.15.94:0/2543654845 shutdown_connections 2024-03-24T10:48:24.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.731+0000 7fdbee0f1700 1 -- 172.21.15.94:0/2543654845 wait complete. 2024-03-24T10:48:24.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.731+0000 7fdbee0f1700 1 Processor -- start 2024-03-24T10:48:24.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.732+0000 7fdbee0f1700 1 -- start start 2024-03-24T10:48:24.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.732+0000 7fdbee0f1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbe80fbcd0 0x7fdbe8110720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:48:24.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.732+0000 7fdbee0f1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbe80fcf80 0x7fdbe8112c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:48:24.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.732+0000 7fdbee0f1700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe8066350 con 0x7fdbe80fcf80 2024-03-24T10:48:24.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.732+0000 7fdbee0f1700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe8066490 con 0x7fdbe80fbcd0 2024-03-24T10:48:24.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.732+0000 7fdbe6ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbe80fcf80 0x7fdbe8112c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:48:24.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.732+0000 7fdbe6ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbe80fcf80 0x7fdbe8112c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54226/0 (socket says 172.21.15.94:54226) 2024-03-24T10:48:24.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.732+0000 7fdbe6ffd700 1 -- 172.21.15.94:0/901825124 learned_addr learned my addr 172.21.15.94:0/901825124 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:48:24.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.732+0000 7fdbe77fe700 1 --2- 172.21.15.94:0/901825124 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbe80fbcd0 0x7fdbe8110720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:48:24.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.732+0000 7fdbe6ffd700 1 -- 172.21.15.94:0/901825124 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbe80fbcd0 msgr2=0x7fdbe8110720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:24.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.732+0000 7fdbe6ffd700 1 --2- 172.21.15.94:0/901825124 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbe80fbcd0 0x7fdbe8110720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:24.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.732+0000 7fdbe6ffd700 1 -- 172.21.15.94:0/901825124 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbdc005040 con 0x7fdbe80fcf80 2024-03-24T10:48:24.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.732+0000 7fdbe6ffd700 1 --2- 172.21.15.94:0/901825124 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbe80fcf80 0x7fdbe8112c70 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc009150 tx=0x7fdbdc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:48:24.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.732+0000 7fdbcffff700 1 -- 172.21.15.94:0/901825124 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdbdc00e040 con 0x7fdbe80fcf80 2024-03-24T10:48:24.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.733+0000 7fdbee0f1700 1 -- 172.21.15.94:0/901825124 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbe81131b0 con 0x7fdbe80fcf80 2024-03-24T10:48:24.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.733+0000 7fdbcffff700 1 -- 172.21.15.94:0/901825124 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdbdc004290 con 0x7fdbe80fcf80 2024-03-24T10:48:24.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.733+0000 7fdbcffff700 1 -- 172.21.15.94:0/901825124 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdbdc01c630 con 0x7fdbe80fcf80 2024-03-24T10:48:24.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.733+0000 7fdbee0f1700 1 -- 172.21.15.94:0/901825124 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbe8113770 con 0x7fdbe80fcf80 2024-03-24T10:48:24.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.734+0000 7fdbee0f1700 1 -- 172.21.15.94:0/901825124 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbe805f5f0 con 0x7fdbe80fcf80 2024-03-24T10:48:24.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.734+0000 7fdbcffff700 1 -- 172.21.15.94:0/901825124 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 30) v1 ==== 91301+0+0 (secure 0 0 0) 0x7fdbdc00bbf0 con 0x7fdbe80fcf80 2024-03-24T10:48:24.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.734+0000 7fdbcffff700 1 --2- 172.21.15.94:0/901825124 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7fdbd006d200 0x7fdbd006f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:48:24.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.735+0000 7fdbcffff700 1 -- 172.21.15.94:0/901825124 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7fdbdc08d3e0 con 0x7fdbe80fcf80 2024-03-24T10:48:24.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.735+0000 7fdbe77fe700 1 --2- 172.21.15.94:0/901825124 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7fdbd006d200 0x7fdbd006f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:48:24.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.735+0000 7fdbe77fe700 1 --2- 172.21.15.94:0/901825124 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7fdbd006d200 0x7fdbd006f6b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8065fd0 tx=0x7fdbd800b040 comp rx=0 tx=0).ready entity=mgr.14774 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:48:24.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.739+0000 7fdbcffff700 1 -- 172.21.15.94:0/901825124 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7fdbdc057290 con 0x7fdbe80fcf80 2024-03-24T10:48:24.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.994+0000 7fdbee0f1700 1 -- 172.21.15.94:0/901825124 --> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdbe8101360 con 0x7fdbd006d200 2024-03-24T10:48:24.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.995+0000 7fdbcffff700 1 -- 172.21.15.94:0/901825124 <== mgr.14774 v2:172.21.15.94:6800/4143126939 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fdbe8101360 con 0x7fdbd006d200 2024-03-24T10:48:24.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.997+0000 7fdbee0f1700 1 -- 172.21.15.94:0/901825124 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7fdbd006d200 msgr2=0x7fdbd006f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:24.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.997+0000 7fdbee0f1700 1 --2- 172.21.15.94:0/901825124 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7fdbd006d200 0x7fdbd006f6b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8065fd0 tx=0x7fdbd800b040 comp rx=0 tx=0).stop 2024-03-24T10:48:24.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.997+0000 7fdbee0f1700 1 -- 172.21.15.94:0/901825124 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbe80fcf80 msgr2=0x7fdbe8112c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:24.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.997+0000 7fdbee0f1700 1 --2- 172.21.15.94:0/901825124 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbe80fcf80 0x7fdbe8112c70 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc009150 tx=0x7fdbdc009e30 comp rx=0 tx=0).stop 2024-03-24T10:48:24.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.997+0000 7fdbee0f1700 1 -- 172.21.15.94:0/901825124 shutdown_connections 2024-03-24T10:48:24.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.997+0000 7fdbee0f1700 1 --2- 172.21.15.94:0/901825124 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7fdbd006d200 0x7fdbd006f6b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:24.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.997+0000 7fdbee0f1700 1 --2- 172.21.15.94:0/901825124 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbe80fbcd0 0x7fdbe8110720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:24.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.998+0000 7fdbee0f1700 1 --2- 172.21.15.94:0/901825124 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbe80fcf80 0x7fdbe8112c70 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:24.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.998+0000 7fdbee0f1700 1 -- 172.21.15.94:0/901825124 >> 172.21.15.94:0/901825124 conn(0x7fdbe80f7430 msgr2=0x7fdbe80f8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:48:24.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.998+0000 7fdbee0f1700 1 -- 172.21.15.94:0/901825124 shutdown_connections 2024-03-24T10:48:24.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:24.998+0000 7fdbee0f1700 1 -- 172.21.15.94:0/901825124 wait complete. 2024-03-24T10:48:25.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.101+0000 7f582913b700 1 -- 172.21.15.94:0/3063240042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58240fbd00 msgr2=0x7f58240fc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:25.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.101+0000 7f582913b700 1 --2- 172.21.15.94:0/3063240042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58240fbd00 0x7f58240fc110 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f581c009ce0 tx=0x7f581c004d10 comp rx=0 tx=0).stop 2024-03-24T10:48:25.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.101+0000 7f582913b700 1 -- 172.21.15.94:0/3063240042 shutdown_connections 2024-03-24T10:48:25.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.101+0000 7f582913b700 1 --2- 172.21.15.94:0/3063240042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58240fcfb0 0x7f58240fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:25.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.101+0000 7f582913b700 1 --2- 172.21.15.94:0/3063240042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58240fbd00 0x7f58240fc110 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:25.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.101+0000 7f582913b700 1 -- 172.21.15.94:0/3063240042 >> 172.21.15.94:0/3063240042 conn(0x7f58240f7480 msgr2=0x7f58240f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:48:25.102 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:24 smithi094 systemd[1]: Stopping Ceph mon.smithi094 for 386110d4-e9ca-11ee-95cd-87774f69a715... 2024-03-24T10:48:25.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.101+0000 7f582913b700 1 -- 172.21.15.94:0/3063240042 shutdown_connections 2024-03-24T10:48:25.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.101+0000 7f582913b700 1 -- 172.21.15.94:0/3063240042 wait complete. 2024-03-24T10:48:25.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.102+0000 7f582913b700 1 Processor -- start 2024-03-24T10:48:25.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.102+0000 7f582913b700 1 -- start start 2024-03-24T10:48:25.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.102+0000 7f582913b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58240fbd00 0x7f582410da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:48:25.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.102+0000 7f582913b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58240fcfb0 0x7f582410dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:48:25.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.102+0000 7f582913b700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582410e5d0 con 0x7f58240fcfb0 2024-03-24T10:48:25.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.102+0000 7f582913b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582410e740 con 0x7f58240fbd00 2024-03-24T10:48:25.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.102+0000 7f582259c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58240fcfb0 0x7f582410dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:48:25.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.102+0000 7f582259c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58240fcfb0 0x7f582410dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54242/0 (socket says 172.21.15.94:54242) 2024-03-24T10:48:25.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.102+0000 7f582259c700 1 -- 172.21.15.94:0/1303307530 learned_addr learned my addr 172.21.15.94:0/1303307530 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:48:25.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.102+0000 7f5822d9d700 1 --2- 172.21.15.94:0/1303307530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58240fbd00 0x7f582410da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:48:25.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.102+0000 7f582259c700 1 -- 172.21.15.94:0/1303307530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58240fbd00 msgr2=0x7f582410da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:25.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.102+0000 7f582259c700 1 --2- 172.21.15.94:0/1303307530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58240fbd00 0x7f582410da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:25.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.102+0000 7f582259c700 1 -- 172.21.15.94:0/1303307530 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f581c009940 con 0x7f58240fcfb0 2024-03-24T10:48:25.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.103+0000 7f582259c700 1 --2- 172.21.15.94:0/1303307530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58240fcfb0 0x7f582410dfd0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f580c002a50 tx=0x7f580c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:48:25.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.103+0000 7f581b7fe700 1 -- 172.21.15.94:0/1303307530 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f580c007cc0 con 0x7f58240fcfb0 2024-03-24T10:48:25.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.103+0000 7f581b7fe700 1 -- 172.21.15.94:0/1303307530 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f580c00f040 con 0x7f58240fcfb0 2024-03-24T10:48:25.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.103+0000 7f582913b700 1 -- 172.21.15.94:0/1303307530 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58241131a0 con 0x7f58240fcfb0 2024-03-24T10:48:25.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.103+0000 7f581b7fe700 1 -- 172.21.15.94:0/1303307530 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f580c00da30 con 0x7f58240fcfb0 2024-03-24T10:48:25.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.103+0000 7f582913b700 1 -- 172.21.15.94:0/1303307530 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5824113760 con 0x7f58240fcfb0 2024-03-24T10:48:25.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.104+0000 7f582913b700 1 -- 172.21.15.94:0/1303307530 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5824101b40 con 0x7f58240fcfb0 2024-03-24T10:48:25.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.104+0000 7f581b7fe700 1 -- 172.21.15.94:0/1303307530 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 30) v1 ==== 91301+0+0 (secure 0 0 0) 0x7f580c00db90 con 0x7f58240fcfb0 2024-03-24T10:48:25.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.104+0000 7f581b7fe700 1 --2- 172.21.15.94:0/1303307530 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f581006d210 0x7f581006f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:48:25.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.105+0000 7f581b7fe700 1 -- 172.21.15.94:0/1303307530 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f580c08c5d0 con 0x7f58240fcfb0 2024-03-24T10:48:25.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.105+0000 7f5822d9d700 1 --2- 172.21.15.94:0/1303307530 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f581006d210 0x7f581006f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:48:25.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.105+0000 7f5822d9d700 1 --2- 172.21.15.94:0/1303307530 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f581006d210 0x7f581006f6c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f581c015040 tx=0x7f581c004d10 comp rx=0 tx=0).ready entity=mgr.14774 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:48:25.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.109+0000 7f581b7fe700 1 -- 172.21.15.94:0/1303307530 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7f580c059f30 con 0x7f58240fcfb0 2024-03-24T10:48:25.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.114+0000 7f582259c700 1 -- 172.21.15.94:0/1303307530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58240fcfb0 msgr2=0x7f582410dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-24T10:48:25.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.114+0000 7f582259c700 1 -- 172.21.15.94:0/1303307530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58240fcfb0 msgr2=0x7f582410dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-24T10:48:25.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.114+0000 7f582259c700 1 --2- 172.21.15.94:0/1303307530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58240fcfb0 0x7f582410dfd0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f580c002a50 tx=0x7f580c0046b0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-24T10:48:25.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.114+0000 7f582259c700 1 --2- 172.21.15.94:0/1303307530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58240fcfb0 0x7f582410dfd0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f580c002a50 tx=0x7f580c0046b0 comp rx=0 tx=0).stop 2024-03-24T10:48:25.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.114+0000 7f581b7fe700 1 -- 172.21.15.94:0/1303307530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58240fcfb0 msgr2=0x7f582410dfd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-24T10:48:25.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.114+0000 7f581b7fe700 1 --2- 172.21.15.94:0/1303307530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58240fcfb0 0x7f582410dfd0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:25.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.114+0000 7f581b7fe700 1 --2- 172.21.15.94:0/1303307530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5810074f00 0x7f58100772e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:48:25.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.114+0000 7f581b7fe700 1 --2- 172.21.15.94:0/1303307530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58240fbd00 0x7f5810079b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:48:25.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.114+0000 7f581b7fe700 1 -- 172.21.15.94:0/1303307530 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581007a4b0 con 0x7f5810074f00 2024-03-24T10:48:25.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.114+0000 7f581b7fe700 1 -- 172.21.15.94:0/1303307530 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581007a5f0 con 0x7f58240fbd00 2024-03-24T10:48:25.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.114+0000 7f582259c700 1 -- 172.21.15.94:0/1303307530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5810074f00 msgr2=0x7f58100772e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:3300/0 2024-03-24T10:48:25.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.114+0000 7f582259c700 1 --2- 172.21.15.94:0/1303307530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5810074f00 0x7f58100772e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-24T10:48:25.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.128+0000 7f582359e700 1 --2- 172.21.15.94:0/1303307530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58240fbd00 0x7f5810079b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:48:25.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.129+0000 7f582359e700 1 -- 172.21.15.94:0/1303307530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5810074f00 msgr2=0x7f58100772e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-24T10:48:25.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.129+0000 7f582359e700 1 --2- 172.21.15.94:0/1303307530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5810074f00 0x7f58100772e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:25.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.129+0000 7f582359e700 1 -- 172.21.15.94:0/1303307530 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,mgrmap=0+,monmap=3+}) v3 -- 0x7f581007ebf0 con 0x7f58240fbd00 2024-03-24T10:48:25.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.129+0000 7f582359e700 1 --2- 172.21.15.94:0/1303307530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58240fbd00 0x7f5810079b30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5814009920 tx=0x7f5814006ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:48:25.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.130+0000 7f581b7fe700 1 -- 172.21.15.94:0/1303307530 <== mon.1 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58140115e0 con 0x7f58240fbd00 2024-03-24T10:48:25.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.130+0000 7f581b7fe700 1 -- 172.21.15.94:0/1303307530 <== mon.1 v2:172.21.15.96:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5814011c10 con 0x7f58240fbd00 2024-03-24T10:48:25.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.132+0000 7f581b7fe700 1 -- 172.21.15.94:0/1303307530 <== mon.1 v2:172.21.15.96:3300/0 3 ==== mgrmap(e 30) v1 ==== 91301+0+0 (secure 0 0 0) 0x7f5814010990 con 0x7f58240fbd00 2024-03-24T10:48:25.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.327+0000 7f582913b700 1 -- 172.21.15.94:0/1303307530 --> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5808000d30 con 0x7f581006d210 2024-03-24T10:48:25.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.339+0000 7f581b7fe700 1 -- 172.21.15.94:0/1303307530 <== mgr.14774 v2:172.21.15.94:6800/4143126939 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f5808000d30 con 0x7f581006d210 2024-03-24T10:48:25.340 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T10:48:25.340 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.smithi094 smithi094 *:9093,9094 running (89s) 7s ago 10m 16.6M - 0.25.0 c8568f914cd2 5c583af60554 2024-03-24T10:48:25.340 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi094 smithi094 running (10m) 7s ago 10m 6660k - 17.2.7-726-g9c4e0c34 837124da804f fb5772af46c5 2024-03-24T10:48:25.340 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi096 smithi096 running (9m) 9s ago 9m 6669k - 17.2.7-726-g9c4e0c34 837124da804f 6087ad4e0734 2024-03-24T10:48:25.340 INFO:teuthology.orchestra.run.smithi094.stdout:grafana.smithi094 smithi094 *:3000 running (74s) 7s ago 9m 77.6M - 9.4.7 954c08fa6188 d3d425bf1b7c 2024-03-24T10:48:25.340 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.efiqte smithi094 running (4m) 7s ago 4m 12.4M - 17.2.7-726-g9c4e0c34 837124da804f f0df0a056c21 2024-03-24T10:48:25.340 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.llefgu smithi094 running (4m) 7s ago 4m 109M - 17.2.7-726-g9c4e0c34 837124da804f 52a9771dc8e0 2024-03-24T10:48:25.340 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi096.egbmoq smithi096 running (4m) 9s ago 4m 156M - 17.2.7-726-g9c4e0c34 837124da804f 1cfc930dbab0 2024-03-24T10:48:25.340 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi096.jtebhh smithi096 running (4m) 9s ago 4m 15.4M - 17.2.7-726-g9c4e0c34 837124da804f 4ff3451a6f6e 2024-03-24T10:48:25.340 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi094.yaethu smithi094 *:8443,9283,8765 running (52s) 7s ago 11m 493M - 19.0.0-1855-g21b3bd67 553463a90fd5 d3adab76b18f 2024-03-24T10:48:25.340 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi096.gfonnu smithi096 *:8443,9283,8765 running (12s) 9s ago 9m 64.4M - 19.0.0-1855-g21b3bd67 553463a90fd5 22937a059eb8 2024-03-24T10:48:25.341 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi094 smithi094 running (11m) 7s ago 12m 50.5M 2048M 17.2.7-726-g9c4e0c34 837124da804f 76584fceb32b 2024-03-24T10:48:25.341 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi096 smithi096 running (9m) 9s ago 9m 41.3M 2048M 17.2.7-726-g9c4e0c34 837124da804f 710030b222cb 2024-03-24T10:48:25.341 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi094 smithi094 *:9100 running (96s) 7s ago 10m 8497k - 1.5.0 0da6a335fe13 ca67b53c64e2 2024-03-24T10:48:25.341 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi096 smithi096 *:9100 running (65s) 9s ago 9m 7872k - 1.5.0 0da6a335fe13 65137a7a7549 2024-03-24T10:48:25.341 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (8m) 7s ago 8m 247M 1557M 17.2.7-726-g9c4e0c34 837124da804f b664fd3557bd 2024-03-24T10:48:25.341 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi094 running (7m) 7s ago 7m 259M 1557M 17.2.7-726-g9c4e0c34 837124da804f 4e737a80d733 2024-03-24T10:48:25.341 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi094 running (7m) 7s ago 7m 220M 1557M 17.2.7-726-g9c4e0c34 837124da804f cf0aefd20c76 2024-03-24T10:48:25.341 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi096 running (7m) 9s ago 6m 284M 2581M 17.2.7-726-g9c4e0c34 837124da804f f4650d0f6241 2024-03-24T10:48:25.341 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi096 running (6m) 9s ago 6m 256M 2581M 17.2.7-726-g9c4e0c34 837124da804f a63d7c9e705f 2024-03-24T10:48:25.341 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi096 running (6m) 9s ago 6m 241M 2581M 17.2.7-726-g9c4e0c34 837124da804f f56fad2959c3 2024-03-24T10:48:25.341 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.smithi094 smithi094 *:9095 running (18s) 7s ago 9m 40.0M - 2.43.0 a07b618ecd1d 6ce091a0c326 2024-03-24T10:48:25.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.341+0000 7f582913b700 1 -- 172.21.15.94:0/1303307530 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f581006d210 msgr2=0x7f581006f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:25.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.341+0000 7f582913b700 1 --2- 172.21.15.94:0/1303307530 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f581006d210 0x7f581006f6c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f581c015040 tx=0x7f581c004d10 comp rx=0 tx=0).stop 2024-03-24T10:48:25.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.341+0000 7f582913b700 1 -- 172.21.15.94:0/1303307530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58240fbd00 msgr2=0x7f5810079b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:25.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.341+0000 7f582913b700 1 --2- 172.21.15.94:0/1303307530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58240fbd00 0x7f5810079b30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5814009920 tx=0x7f5814006ed0 comp rx=0 tx=0).stop 2024-03-24T10:48:25.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.341+0000 7f582913b700 1 -- 172.21.15.94:0/1303307530 shutdown_connections 2024-03-24T10:48:25.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.341+0000 7f582913b700 1 --2- 172.21.15.94:0/1303307530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58240fbd00 0x7f5810079b30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:25.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.341+0000 7f582913b700 1 --2- 172.21.15.94:0/1303307530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5810074f00 0x7f58100772e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:25.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.341+0000 7f582913b700 1 --2- 172.21.15.94:0/1303307530 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f581006d210 0x7f581006f6c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:25.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.341+0000 7f582913b700 1 -- 172.21.15.94:0/1303307530 >> 172.21.15.94:0/1303307530 conn(0x7f58240f7480 msgr2=0x7f5824066840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:48:25.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.341+0000 7f582913b700 1 -- 172.21.15.94:0/1303307530 shutdown_connections 2024-03-24T10:48:25.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.341+0000 7f582913b700 1 -- 172.21.15.94:0/1303307530 wait complete. 2024-03-24T10:48:25.422 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:25 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-mon-smithi094[29442]: 2024-03-24T10:48:25.112+0000 7febb1067700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi094 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T10:48:25.422 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:25 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-mon-smithi094[29442]: 2024-03-24T10:48:25.112+0000 7febb1067700 -1 mon.smithi094@0(leader) e2 *** Got Signal Terminated *** 2024-03-24T10:48:25.422 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:25 smithi094 podman[93672]: 2024-03-24 10:48:25.134586584 +0000 UTC m=+0.123158369 container died 76584fceb32b32ae4332ea8019fd710a55ac7733c1b6c174d655481b8e7115ae (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-mon-smithi094, GIT_BRANCH=HEAD, RELEASE=quincy-9c4e0c3, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-24T10:48:25.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.442+0000 7f7579fde700 1 -- 172.21.15.94:0/3311436091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75740fff60 msgr2=0x7f7574102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:25.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.442+0000 7f7579fde700 1 --2- 172.21.15.94:0/3311436091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75740fff60 0x7f7574102340 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f755c004940 tx=0x7f755c009e30 comp rx=0 tx=0).stop 2024-03-24T10:48:25.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.442+0000 7f7579fde700 1 -- 172.21.15.94:0/3311436091 shutdown_connections 2024-03-24T10:48:25.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.442+0000 7f7579fde700 1 --2- 172.21.15.94:0/3311436091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75740fff60 0x7f7574102340 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:25.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.442+0000 7f7579fde700 1 --2- 172.21.15.94:0/3311436091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7574066280 0x7f75740ffa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:25.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.442+0000 7f7579fde700 1 -- 172.21.15.94:0/3311436091 >> 172.21.15.94:0/3311436091 conn(0x7f75740f7430 msgr2=0x7f75740f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:48:25.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.442+0000 7f7579fde700 1 -- 172.21.15.94:0/3311436091 shutdown_connections 2024-03-24T10:48:25.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.442+0000 7f7579fde700 1 -- 172.21.15.94:0/3311436091 wait complete. 2024-03-24T10:48:25.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.443+0000 7f7579fde700 1 Processor -- start 2024-03-24T10:48:25.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.443+0000 7f7579fde700 1 -- start start 2024-03-24T10:48:25.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.443+0000 7f7579fde700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7574066280 0x7f757410daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:48:25.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.443+0000 7f7579fde700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75740fff60 0x7f757410e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:48:25.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.443+0000 7f7579fde700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757410e6c0 con 0x7f75740fff60 2024-03-24T10:48:25.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.443+0000 7f7579fde700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757410eeb0 con 0x7f7574066280 2024-03-24T10:48:25.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.443+0000 7f7572ffd700 1 -- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75740fff60 msgr2=0x7f757410e030 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:3300/0 2024-03-24T10:48:25.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.443+0000 7f7572ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75740fff60 0x7f757410e030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-24T10:48:25.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.443+0000 7f75737fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7574066280 0x7f757410daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:48:25.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.443+0000 7f75737fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7574066280 0x7f757410daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.94:55766/0 (socket says 172.21.15.94:55766) 2024-03-24T10:48:25.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.443+0000 7f75737fe700 1 -- 172.21.15.94:0/384569478 learned_addr learned my addr 172.21.15.94:0/384569478 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:48:25.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.444+0000 7f75737fe700 1 -- 172.21.15.94:0/384569478 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75740fff60 msgr2=0x7f757410e030 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-24T10:48:25.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.444+0000 7f75737fe700 1 --2- 172.21.15.94:0/384569478 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75740fff60 0x7f757410e030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:25.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.444+0000 7f75737fe700 1 -- 172.21.15.94:0/384569478 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f755c005040 con 0x7f7574066280 2024-03-24T10:48:25.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.444+0000 7f75737fe700 1 --2- 172.21.15.94:0/384569478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7574066280 0x7f757410daf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f756c00bb00 tx=0x7f756c009810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:48:25.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.444+0000 7f756bfff700 1 -- 172.21.15.94:0/384569478 <== mon.1 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f756c00c8d0 con 0x7f7574066280 2024-03-24T10:48:25.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.444+0000 7f756bfff700 1 -- 172.21.15.94:0/384569478 <== mon.1 v2:172.21.15.96:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f756c015470 con 0x7f7574066280 2024-03-24T10:48:25.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.444+0000 7f7579fde700 1 -- 172.21.15.94:0/384569478 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f757410f190 con 0x7f7574066280 2024-03-24T10:48:25.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.444+0000 7f756bfff700 1 -- 172.21.15.94:0/384569478 <== mon.1 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f756c001640 con 0x7f7574066280 2024-03-24T10:48:25.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.444+0000 7f7579fde700 1 -- 172.21.15.94:0/384569478 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f757410f750 con 0x7f7574066280 2024-03-24T10:48:25.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.445+0000 7f7579fde700 1 -- 172.21.15.94:0/384569478 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f757405f5f0 con 0x7f7574066280 2024-03-24T10:48:25.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.446+0000 7f756bfff700 1 -- 172.21.15.94:0/384569478 <== mon.1 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 30) v1 ==== 91301+0+0 (secure 0 0 0) 0x7f756c012070 con 0x7f7574066280 2024-03-24T10:48:25.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.446+0000 7f756bfff700 1 --2- 172.21.15.94:0/384569478 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f756006d150 0x7f756006f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:48:25.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.446+0000 7f7572ffd700 1 --2- 172.21.15.94:0/384569478 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f756006d150 0x7f756006f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:48:25.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.446+0000 7f756bfff700 1 -- 172.21.15.94:0/384569478 <== mon.1 v2:172.21.15.96:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f756c08c6b0 con 0x7f7574066280 2024-03-24T10:48:25.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.447+0000 7f7572ffd700 1 --2- 172.21.15.94:0/384569478 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f756006d150 0x7f756006f600 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f755c004890 tx=0x7f755c009e30 comp rx=0 tx=0).ready entity=mgr.14774 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:48:25.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.451+0000 7f756bfff700 1 -- 172.21.15.94:0/384569478 <== mon.1 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7f756c05a140 con 0x7f7574066280 2024-03-24T10:48:25.682 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:25 smithi094 podman[93672]: 2024-03-24 10:48:25.664164625 +0000 UTC m=+0.652736412 container cleanup 76584fceb32b32ae4332ea8019fd710a55ac7733c1b6c174d655481b8e7115ae (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-mon-smithi094, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=quincy-9c4e0c3, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T10:48:25.683 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:25 smithi094 bash[93672]: ceph-386110d4-e9ca-11ee-95cd-87774f69a715-mon-smithi094 2024-03-24T10:48:25.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.733+0000 7f7579fde700 1 -- 172.21.15.94:0/384569478 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7574105950 con 0x7f7574066280 2024-03-24T10:48:25.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.734+0000 7f756bfff700 1 -- 172.21.15.94:0/384569478 <== mon.1 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f756c059cd0 con 0x7f7574066280 2024-03-24T10:48:25.735 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-03-24T10:48:25.735 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": { 2024-03-24T10:48:25.735 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-24T10:48:25.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:48:25.735 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": { 2024-03-24T10:48:25.735 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 2 2024-03-24T10:48:25.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:48:25.735 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": { 2024-03-24T10:48:25.735 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-24T10:48:25.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:48:25.735 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-03-24T10:48:25.735 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-24T10:48:25.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:48:25.735 INFO:teuthology.orchestra.run.smithi094.stdout: "overall": { 2024-03-24T10:48:25.736 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 12, 2024-03-24T10:48:25.736 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 2 2024-03-24T10:48:25.736 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-03-24T10:48:25.736 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-03-24T10:48:25.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.736+0000 7f7579fde700 1 -- 172.21.15.94:0/384569478 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f756006d150 msgr2=0x7f756006f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:25.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.736+0000 7f7579fde700 1 --2- 172.21.15.94:0/384569478 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f756006d150 0x7f756006f600 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f755c004890 tx=0x7f755c009e30 comp rx=0 tx=0).stop 2024-03-24T10:48:25.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.736+0000 7f7579fde700 1 -- 172.21.15.94:0/384569478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7574066280 msgr2=0x7f757410daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:25.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.736+0000 7f7579fde700 1 --2- 172.21.15.94:0/384569478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7574066280 0x7f757410daf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f756c00bb00 tx=0x7f756c009810 comp rx=0 tx=0).stop 2024-03-24T10:48:25.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.736+0000 7f7579fde700 1 -- 172.21.15.94:0/384569478 shutdown_connections 2024-03-24T10:48:25.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.736+0000 7f7579fde700 1 --2- 172.21.15.94:0/384569478 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f756006d150 0x7f756006f600 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:25.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.736+0000 7f7579fde700 1 --2- 172.21.15.94:0/384569478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7574066280 0x7f757410daf0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:25.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.736+0000 7f7579fde700 1 --2- 172.21.15.94:0/384569478 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75740fff60 0x7f757410e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:25.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.736+0000 7f7579fde700 1 -- 172.21.15.94:0/384569478 >> 172.21.15.94:0/384569478 conn(0x7f75740f7430 msgr2=0x7f75740f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:48:25.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.736+0000 7f7579fde700 1 -- 172.21.15.94:0/384569478 shutdown_connections 2024-03-24T10:48:25.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.736+0000 7f7579fde700 1 -- 172.21.15.94:0/384569478 wait complete. 2024-03-24T10:48:25.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.840+0000 7f4712989700 1 -- 172.21.15.94:0/3541346502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f470c066260 msgr2=0x7f470c0ff9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:25.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.840+0000 7f4712989700 1 --2- 172.21.15.94:0/3541346502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f470c066260 0x7f470c0ff9b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f46fc004940 tx=0x7f46fc009e30 comp rx=0 tx=0).stop 2024-03-24T10:48:25.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.840+0000 7f4712989700 1 -- 172.21.15.94:0/3541346502 shutdown_connections 2024-03-24T10:48:25.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.840+0000 7f4712989700 1 --2- 172.21.15.94:0/3541346502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f470c0ffef0 0x7f470c1022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:25.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.840+0000 7f4712989700 1 --2- 172.21.15.94:0/3541346502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f470c066260 0x7f470c0ff9b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:25.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.840+0000 7f4712989700 1 -- 172.21.15.94:0/3541346502 >> 172.21.15.94:0/3541346502 conn(0x7f470c0f73c0 msgr2=0x7f470c0f97d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:48:25.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.840+0000 7f4712989700 1 -- 172.21.15.94:0/3541346502 shutdown_connections 2024-03-24T10:48:25.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.840+0000 7f4712989700 1 -- 172.21.15.94:0/3541346502 wait complete. 2024-03-24T10:48:25.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.841+0000 7f4712989700 1 Processor -- start 2024-03-24T10:48:25.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.841+0000 7f4712989700 1 -- start start 2024-03-24T10:48:25.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.841+0000 7f4712989700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f470c066260 0x7f470c10da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:48:25.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.841+0000 7f4712989700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f470c0ffef0 0x7f470c10df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:48:25.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.841+0000 7f4712989700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470c10e620 con 0x7f470c0ffef0 2024-03-24T10:48:25.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.841+0000 7f4712989700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470c10ee10 con 0x7f470c066260 2024-03-24T10:48:25.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.841+0000 7f470b7fe700 1 -- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f470c0ffef0 msgr2=0x7f470c10df90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:3300/0 2024-03-24T10:48:25.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.841+0000 7f470b7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f470c0ffef0 0x7f470c10df90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-24T10:48:25.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.841+0000 7f470bfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f470c066260 0x7f470c10da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:48:25.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.841+0000 7f470bfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f470c066260 0x7f470c10da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.94:55772/0 (socket says 172.21.15.94:55772) 2024-03-24T10:48:25.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.841+0000 7f470bfff700 1 -- 172.21.15.94:0/3132577461 learned_addr learned my addr 172.21.15.94:0/3132577461 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:48:25.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.841+0000 7f470bfff700 1 -- 172.21.15.94:0/3132577461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f470c0ffef0 msgr2=0x7f470c10df90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-24T10:48:25.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.841+0000 7f470bfff700 1 --2- 172.21.15.94:0/3132577461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f470c0ffef0 0x7f470c10df90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:25.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.841+0000 7f470bfff700 1 -- 172.21.15.94:0/3132577461 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46fc005040 con 0x7f470c066260 2024-03-24T10:48:25.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.842+0000 7f470bfff700 1 --2- 172.21.15.94:0/3132577461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f470c066260 0x7f470c10da50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f46fc000c00 tx=0x7f46fc009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:48:25.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.842+0000 7f4708ff9700 1 -- 172.21.15.94:0/3132577461 <== mon.1 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46fc00e030 con 0x7f470c066260 2024-03-24T10:48:25.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.842+0000 7f4708ff9700 1 -- 172.21.15.94:0/3132577461 <== mon.1 v2:172.21.15.96:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f46fc013420 con 0x7f470c066260 2024-03-24T10:48:25.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.842+0000 7f4708ff9700 1 -- 172.21.15.94:0/3132577461 <== mon.1 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46fc00bee0 con 0x7f470c066260 2024-03-24T10:48:25.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.842+0000 7f4712989700 1 -- 172.21.15.94:0/3132577461 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f470c10f090 con 0x7f470c066260 2024-03-24T10:48:25.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.842+0000 7f4712989700 1 -- 172.21.15.94:0/3132577461 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f470c10f650 con 0x7f470c066260 2024-03-24T10:48:25.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.843+0000 7f4712989700 1 -- 172.21.15.94:0/3132577461 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f470c05f5f0 con 0x7f470c066260 2024-03-24T10:48:25.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.844+0000 7f4708ff9700 1 -- 172.21.15.94:0/3132577461 <== mon.1 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 30) v1 ==== 91301+0+0 (secure 0 0 0) 0x7f46fc013a20 con 0x7f470c066260 2024-03-24T10:48:25.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.844+0000 7f4708ff9700 1 --2- 172.21.15.94:0/3132577461 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f46f406d1a0 0x7f46f406f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:48:25.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.844+0000 7f4708ff9700 1 -- 172.21.15.94:0/3132577461 <== mon.1 v2:172.21.15.96:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f46fc08def0 con 0x7f470c066260 2024-03-24T10:48:25.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.844+0000 7f470b7fe700 1 --2- 172.21.15.94:0/3132577461 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f46f406d1a0 0x7f46f406f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:48:25.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.844+0000 7f470b7fe700 1 --2- 172.21.15.94:0/3132577461 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f46f406d1a0 0x7f46f406f650 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4700006fd0 tx=0x7f4700006df0 comp rx=0 tx=0).ready entity=mgr.14774 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:48:25.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:25.849+0000 7f4708ff9700 1 -- 172.21.15.94:0/3132577461 <== mon.1 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7f46fc05b900 con 0x7f470c066260 2024-03-24T10:48:25.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:25 smithi094 podman[93705]: 2024-03-24 10:48:25.874242895 +0000 UTC m=+0.737308796 container remove 76584fceb32b32ae4332ea8019fd710a55ac7733c1b6c174d655481b8e7115ae (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-mon-smithi094, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, RELEASE=quincy-9c4e0c3, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS) 2024-03-24T10:48:26.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.105+0000 7f4712989700 1 -- 172.21.15.94:0/3132577461 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f470c1058c0 con 0x7f470c066260 2024-03-24T10:48:26.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.106+0000 7f4708ff9700 1 -- 172.21.15.94:0/3132577461 <== mon.1 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1833 (secure 0 0 0) 0x7f46fc020740 con 0x7f470c066260 2024-03-24T10:48:26.107 INFO:teuthology.orchestra.run.smithi094.stdout:e18 2024-03-24T10:48:26.107 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T10:48:26.107 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:48:26.107 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-03-24T10:48:26.107 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:48:26.107 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-03-24T10:48:26.107 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-03-24T10:48:26.107 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 18 2024-03-24T10:48:26.107 INFO:teuthology.orchestra.run.smithi094.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-24T10:48:26.107 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-03-24T10:43:39.435738+0000 2024-03-24T10:48:26.107 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-03-24T10:43:58.133901+0000 2024-03-24T10:48:26.108 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-03-24T10:48:26.108 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-03-24T10:48:26.108 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 60 2024-03-24T10:48:26.108 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-03-24T10:48:26.108 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-03-24T10:48:26.108 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-03-24T10:48:26.108 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-03-24T10:48:26.108 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 46 2024-03-24T10:48:26.108 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:48:26.108 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-03-24T10:48:26.108 INFO:teuthology.orchestra.run.smithi094.stdout:in 0 2024-03-24T10:48:26.108 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=24251} 2024-03-24T10:48:26.108 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-03-24T10:48:26.108 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-03-24T10:48:26.108 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-03-24T10:48:26.108 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-03-24T10:48:26.109 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-03-24T10:48:26.109 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-03-24T10:48:26.109 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-03-24T10:48:26.109 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 1 2024-03-24T10:48:26.109 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi096.egbmoq{0:24251} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.96:6824/1539752208,v1:172.21.15.96:6825/1539752208] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:48:26.109 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.llefgu{0:14500} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.94:6828/126762980,v1:172.21.15.94:6829/126762980] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:48:26.109 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:48:26.109 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:48:26.109 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-03-24T10:48:26.109 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:48:26.109 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.efiqte{-1:14512} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.94:6826/1636528860,v1:172.21.15.94:6827/1636528860] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:48:26.109 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi096.jtebhh{-1:24261} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.96:6826/1321087930,v1:172.21.15.96:6827/1321087930] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:48:26.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.108+0000 7f4712989700 1 -- 172.21.15.94:0/3132577461 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f46f406d1a0 msgr2=0x7f46f406f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:26.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.108+0000 7f4712989700 1 --2- 172.21.15.94:0/3132577461 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f46f406d1a0 0x7f46f406f650 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4700006fd0 tx=0x7f4700006df0 comp rx=0 tx=0).stop 2024-03-24T10:48:26.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.108+0000 7f4712989700 1 -- 172.21.15.94:0/3132577461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f470c066260 msgr2=0x7f470c10da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:26.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.108+0000 7f4712989700 1 --2- 172.21.15.94:0/3132577461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f470c066260 0x7f470c10da50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f46fc000c00 tx=0x7f46fc009e30 comp rx=0 tx=0).stop 2024-03-24T10:48:26.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.108+0000 7f4712989700 1 -- 172.21.15.94:0/3132577461 shutdown_connections 2024-03-24T10:48:26.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.108+0000 7f4712989700 1 --2- 172.21.15.94:0/3132577461 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7f46f406d1a0 0x7f46f406f650 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:26.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.108+0000 7f4712989700 1 --2- 172.21.15.94:0/3132577461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f470c066260 0x7f470c10da50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:26.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.108+0000 7f4712989700 1 --2- 172.21.15.94:0/3132577461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f470c0ffef0 0x7f470c10df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:26.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.108+0000 7f4712989700 1 -- 172.21.15.94:0/3132577461 >> 172.21.15.94:0/3132577461 conn(0x7f470c0f73c0 msgr2=0x7f470c0f7790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:48:26.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.108+0000 7f4712989700 1 -- 172.21.15.94:0/3132577461 shutdown_connections 2024-03-24T10:48:26.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.108+0000 7f4712989700 1 -- 172.21.15.94:0/3132577461 wait complete. 2024-03-24T10:48:26.111 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 18 2024-03-24T10:48:26.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.228+0000 7fb7b1201700 1 -- 172.21.15.94:0/2937184235 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7ac0fbb10 msgr2=0x7fb7ac0fbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:26.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.228+0000 7fb7b1201700 1 --2- 172.21.15.94:0/2937184235 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7ac0fbb10 0x7fb7ac0fbf20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb79c004940 tx=0x7fb79c009e30 comp rx=0 tx=0).stop 2024-03-24T10:48:26.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.228+0000 7fb7b1201700 1 -- 172.21.15.94:0/2937184235 shutdown_connections 2024-03-24T10:48:26.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.228+0000 7fb7b1201700 1 --2- 172.21.15.94:0/2937184235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7ac0fc460 0x7fb7ac0fce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:26.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.228+0000 7fb7b1201700 1 --2- 172.21.15.94:0/2937184235 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7ac0fbb10 0x7fb7ac0fbf20 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:26.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.228+0000 7fb7b1201700 1 -- 172.21.15.94:0/2937184235 >> 172.21.15.94:0/2937184235 conn(0x7fb7ac0f72b0 msgr2=0x7fb7ac0f96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:48:26.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.228+0000 7fb7b1201700 1 -- 172.21.15.94:0/2937184235 shutdown_connections 2024-03-24T10:48:26.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.228+0000 7fb7b1201700 1 -- 172.21.15.94:0/2937184235 wait complete. 2024-03-24T10:48:26.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.229+0000 7fb7b1201700 1 Processor -- start 2024-03-24T10:48:26.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.229+0000 7fb7b1201700 1 -- start start 2024-03-24T10:48:26.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.229+0000 7fb7b1201700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7ac0fbb10 0x7fb7ac10c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:48:26.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.229+0000 7fb7b1201700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7ac0fc460 0x7fb7ac10cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:48:26.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.229+0000 7fb7b1201700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ac10d400 con 0x7fb7ac0fc460 2024-03-24T10:48:26.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.229+0000 7fb7b1201700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ac10dbf0 con 0x7fb7ac0fbb10 2024-03-24T10:48:26.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.229+0000 7fb7aa59c700 1 -- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7ac0fc460 msgr2=0x7fb7ac10cd70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:3300/0 2024-03-24T10:48:26.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.229+0000 7fb7aa59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7ac0fc460 0x7fb7ac10cd70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-24T10:48:26.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.230+0000 7fb7aad9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7ac0fbb10 0x7fb7ac10c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:48:26.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.230+0000 7fb7aad9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7ac0fbb10 0x7fb7ac10c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.94:55786/0 (socket says 172.21.15.94:55786) 2024-03-24T10:48:26.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.230+0000 7fb7aad9d700 1 -- 172.21.15.94:0/1454477735 learned_addr learned my addr 172.21.15.94:0/1454477735 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:48:26.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.230+0000 7fb7aad9d700 1 -- 172.21.15.94:0/1454477735 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7ac0fc460 msgr2=0x7fb7ac10cd70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-24T10:48:26.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.230+0000 7fb7aad9d700 1 --2- 172.21.15.94:0/1454477735 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7ac0fc460 0x7fb7ac10cd70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:26.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.230+0000 7fb7aad9d700 1 -- 172.21.15.94:0/1454477735 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb79c005040 con 0x7fb7ac0fbb10 2024-03-24T10:48:26.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.230+0000 7fb7aad9d700 1 --2- 172.21.15.94:0/1454477735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7ac0fbb10 0x7fb7ac10c830 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb79c000c00 tx=0x7fb79c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:48:26.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.230+0000 7fb7937fe700 1 -- 172.21.15.94:0/1454477735 <== mon.1 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb79c00e030 con 0x7fb7ac0fbb10 2024-03-24T10:48:26.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.230+0000 7fb7937fe700 1 -- 172.21.15.94:0/1454477735 <== mon.1 v2:172.21.15.96:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb79c013420 con 0x7fb7ac0fbb10 2024-03-24T10:48:26.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.230+0000 7fb7937fe700 1 -- 172.21.15.94:0/1454477735 <== mon.1 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb79c00bee0 con 0x7fb7ac0fbb10 2024-03-24T10:48:26.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.230+0000 7fb7b1201700 1 -- 172.21.15.94:0/1454477735 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7ac10de70 con 0x7fb7ac0fbb10 2024-03-24T10:48:26.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.231+0000 7fb7b1201700 1 -- 172.21.15.94:0/1454477735 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7ac10e430 con 0x7fb7ac0fbb10 2024-03-24T10:48:26.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.231+0000 7fb7b1201700 1 -- 172.21.15.94:0/1454477735 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7ac05f5f0 con 0x7fb7ac0fbb10 2024-03-24T10:48:26.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.232+0000 7fb7937fe700 1 -- 172.21.15.94:0/1454477735 <== mon.1 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 30) v1 ==== 91301+0+0 (secure 0 0 0) 0x7fb79c013a20 con 0x7fb7ac0fbb10 2024-03-24T10:48:26.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.232+0000 7fb7937fe700 1 --2- 172.21.15.94:0/1454477735 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7fb79406d1a0 0x7fb79406f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:48:26.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.232+0000 7fb7937fe700 1 -- 172.21.15.94:0/1454477735 <== mon.1 v2:172.21.15.96:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7fb79c08df90 con 0x7fb7ac0fbb10 2024-03-24T10:48:26.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.233+0000 7fb7aa59c700 1 --2- 172.21.15.94:0/1454477735 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7fb79406d1a0 0x7fb79406f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:48:26.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.233+0000 7fb7aa59c700 1 --2- 172.21.15.94:0/1454477735 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7fb79406d1a0 0x7fb79406f650 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb7a0006fd0 tx=0x7fb7a0006df0 comp rx=0 tx=0).ready entity=mgr.14774 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:48:26.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.237+0000 7fb7937fe700 1 -- 172.21.15.94:0/1454477735 <== mon.1 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7fb79c05b900 con 0x7fb7ac0fbb10 2024-03-24T10:48:26.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.463+0000 7fb7b1201700 1 -- 172.21.15.94:0/1454477735 --> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb7ac10e760 con 0x7fb79406d1a0 2024-03-24T10:48:26.464 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 systemd[1]: ceph-386110d4-e9ca-11ee-95cd-87774f69a715@mon.smithi094.service: Deactivated successfully. 2024-03-24T10:48:26.464 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 systemd[1]: Stopped Ceph mon.smithi094 for 386110d4-e9ca-11ee-95cd-87774f69a715. 2024-03-24T10:48:26.464 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 systemd[1]: ceph-386110d4-e9ca-11ee-95cd-87774f69a715@mon.smithi094.service: Consumed 10.872s CPU time. 2024-03-24T10:48:26.464 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 systemd[1]: Starting Ceph mon.smithi094 for 386110d4-e9ca-11ee-95cd-87774f69a715... 2024-03-24T10:48:26.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.464+0000 7fb7937fe700 1 -- 172.21.15.94:0/1454477735 <== mgr.14774 v2:172.21.15.94:6800/4143126939 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fb7ac10e760 con 0x7fb79406d1a0 2024-03-24T10:48:26.465 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-03-24T10:48:26.465 INFO:teuthology.orchestra.run.smithi094.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919", 2024-03-24T10:48:26.465 INFO:teuthology.orchestra.run.smithi094.stdout: "in_progress": true, 2024-03-24T10:48:26.465 INFO:teuthology.orchestra.run.smithi094.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T10:48:26.465 INFO:teuthology.orchestra.run.smithi094.stdout: "services_complete": [ 2024-03-24T10:48:26.466 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr" 2024-03-24T10:48:26.466 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-03-24T10:48:26.466 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": "2/21 daemons upgraded", 2024-03-24T10:48:26.466 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "Currently upgrading mon daemons", 2024-03-24T10:48:26.466 INFO:teuthology.orchestra.run.smithi094.stdout: "is_paused": false 2024-03-24T10:48:26.466 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-03-24T10:48:26.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.467+0000 7fb7b1201700 1 -- 172.21.15.94:0/1454477735 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7fb79406d1a0 msgr2=0x7fb79406f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:26.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.467+0000 7fb7b1201700 1 --2- 172.21.15.94:0/1454477735 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7fb79406d1a0 0x7fb79406f650 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb7a0006fd0 tx=0x7fb7a0006df0 comp rx=0 tx=0).stop 2024-03-24T10:48:26.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.467+0000 7fb7b1201700 1 -- 172.21.15.94:0/1454477735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7ac0fbb10 msgr2=0x7fb7ac10c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:26.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.467+0000 7fb7b1201700 1 --2- 172.21.15.94:0/1454477735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7ac0fbb10 0x7fb7ac10c830 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb79c000c00 tx=0x7fb79c009e30 comp rx=0 tx=0).stop 2024-03-24T10:48:26.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.467+0000 7fb7b1201700 1 -- 172.21.15.94:0/1454477735 shutdown_connections 2024-03-24T10:48:26.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.467+0000 7fb7b1201700 1 --2- 172.21.15.94:0/1454477735 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7fb79406d1a0 0x7fb79406f650 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:26.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.467+0000 7fb7b1201700 1 --2- 172.21.15.94:0/1454477735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7ac0fbb10 0x7fb7ac10c830 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:26.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.467+0000 7fb7b1201700 1 --2- 172.21.15.94:0/1454477735 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7ac0fc460 0x7fb7ac10cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:26.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.467+0000 7fb7b1201700 1 -- 172.21.15.94:0/1454477735 >> 172.21.15.94:0/1454477735 conn(0x7fb7ac0f72b0 msgr2=0x7fb7ac0f89f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:48:26.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.467+0000 7fb7b1201700 1 -- 172.21.15.94:0/1454477735 shutdown_connections 2024-03-24T10:48:26.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.467+0000 7fb7b1201700 1 -- 172.21.15.94:0/1454477735 wait complete. 2024-03-24T10:48:26.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.569+0000 7fdffbad5700 1 -- 172.21.15.94:0/1980532227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdff40fbcd0 msgr2=0x7fdff40fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:26.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.569+0000 7fdffbad5700 1 --2- 172.21.15.94:0/1980532227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdff40fbcd0 0x7fdff40fc0e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8004940 tx=0x7fdfe8009e30 comp rx=0 tx=0).stop 2024-03-24T10:48:26.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.569+0000 7fdffbad5700 1 -- 172.21.15.94:0/1980532227 shutdown_connections 2024-03-24T10:48:26.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.569+0000 7fdffbad5700 1 --2- 172.21.15.94:0/1980532227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdff40fcf80 0x7fdff40fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:26.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.569+0000 7fdffbad5700 1 --2- 172.21.15.94:0/1980532227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdff40fbcd0 0x7fdff40fc0e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:26.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.569+0000 7fdffbad5700 1 -- 172.21.15.94:0/1980532227 >> 172.21.15.94:0/1980532227 conn(0x7fdff40f7430 msgr2=0x7fdff40f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:48:26.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.569+0000 7fdffbad5700 1 -- 172.21.15.94:0/1980532227 shutdown_connections 2024-03-24T10:48:26.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.569+0000 7fdffbad5700 1 -- 172.21.15.94:0/1980532227 wait complete. 2024-03-24T10:48:26.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.570+0000 7fdffbad5700 1 Processor -- start 2024-03-24T10:48:26.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.570+0000 7fdffbad5700 1 -- start start 2024-03-24T10:48:26.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.570+0000 7fdffbad5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdff40fbcd0 0x7fdff4110750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:48:26.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.570+0000 7fdffbad5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdff40fcf80 0x7fdff4112ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:48:26.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.570+0000 7fdffbad5700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff4066440 con 0x7fdff40fbcd0 2024-03-24T10:48:26.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.570+0000 7fdffbad5700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff4066580 con 0x7fdff40fcf80 2024-03-24T10:48:26.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.570+0000 7fdff9871700 1 -- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdff40fbcd0 msgr2=0x7fdff4110750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:3300/0 2024-03-24T10:48:26.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.570+0000 7fdff9871700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdff40fbcd0 0x7fdff4110750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-24T10:48:26.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.571+0000 7fdff9070700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdff40fcf80 0x7fdff4112ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:48:26.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.571+0000 7fdff9070700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdff40fcf80 0x7fdff4112ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.94:55800/0 (socket says 172.21.15.94:55800) 2024-03-24T10:48:26.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.571+0000 7fdff9070700 1 -- 172.21.15.94:0/4222703942 learned_addr learned my addr 172.21.15.94:0/4222703942 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:48:26.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.571+0000 7fdff9070700 1 -- 172.21.15.94:0/4222703942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdff40fbcd0 msgr2=0x7fdff4110750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-24T10:48:26.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.571+0000 7fdff9070700 1 --2- 172.21.15.94:0/4222703942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdff40fbcd0 0x7fdff4110750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:26.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.571+0000 7fdff9070700 1 -- 172.21.15.94:0/4222703942 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe8005040 con 0x7fdff40fcf80 2024-03-24T10:48:26.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.571+0000 7fdff9070700 1 --2- 172.21.15.94:0/4222703942 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdff40fcf80 0x7fdff4112ca0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdff000b790 tx=0x7fdff00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:48:26.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.571+0000 7fdfe67fc700 1 -- 172.21.15.94:0/4222703942 <== mon.1 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdff0010790 con 0x7fdff40fcf80 2024-03-24T10:48:26.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.571+0000 7fdffbad5700 1 -- 172.21.15.94:0/4222703942 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdff4113330 con 0x7fdff40fcf80 2024-03-24T10:48:26.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.571+0000 7fdffbad5700 1 -- 172.21.15.94:0/4222703942 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdff41138f0 con 0x7fdff40fcf80 2024-03-24T10:48:26.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.571+0000 7fdfe67fc700 1 -- 172.21.15.94:0/4222703942 <== mon.1 v2:172.21.15.96:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdff0010dc0 con 0x7fdff40fcf80 2024-03-24T10:48:26.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.571+0000 7fdfe67fc700 1 -- 172.21.15.94:0/4222703942 <== mon.1 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdff000f540 con 0x7fdff40fcf80 2024-03-24T10:48:26.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.573+0000 7fdffbad5700 1 -- 172.21.15.94:0/4222703942 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdff405f5f0 con 0x7fdff40fcf80 2024-03-24T10:48:26.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.573+0000 7fdfe67fc700 1 -- 172.21.15.94:0/4222703942 <== mon.1 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 30) v1 ==== 91301+0+0 (secure 0 0 0) 0x7fdff00108f0 con 0x7fdff40fcf80 2024-03-24T10:48:26.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.573+0000 7fdfe67fc700 1 --2- 172.21.15.94:0/4222703942 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7fdfe006d1f0 0x7fdfe006f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:48:26.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.573+0000 7fdfe67fc700 1 -- 172.21.15.94:0/4222703942 <== mon.1 v2:172.21.15.96:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7fdff008c780 con 0x7fdff40fcf80 2024-03-24T10:48:26.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.574+0000 7fdff9871700 1 --2- 172.21.15.94:0/4222703942 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7fdfe006d1f0 0x7fdfe006f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:48:26.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.574+0000 7fdff9871700 1 --2- 172.21.15.94:0/4222703942 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7fdfe006d1f0 0x7fdfe006f6a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fdfe800b020 tx=0x7fdfe8009e30 comp rx=0 tx=0).ready entity=mgr.14774 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:48:26.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.578+0000 7fdfe67fc700 1 -- 172.21.15.94:0/4222703942 <== mon.1 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7fdff00566a0 con 0x7fdff40fcf80 2024-03-24T10:48:26.715 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 podman[93915]: 2024-03-24 10:48:26.431382335 +0000 UTC m=+0.018242079 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-24T10:48:26.716 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 podman[93915]: 2024-03-24 10:48:26.56804562 +0000 UTC m=+0.154905363 container create c779d611e29a976824345c01fe0209d2f25bbb268cb0ec946f706d92946b80f6 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-mon-smithi094, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-24T10:48:26.859 INFO:teuthology.orchestra.run.smithi094.stdout:HEALTH_OK 2024-03-24T10:48:26.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.857+0000 7fdffbad5700 1 -- 172.21.15.94:0/4222703942 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fdff404bed0 con 0x7fdff40fcf80 2024-03-24T10:48:26.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.858+0000 7fdfe67fc700 1 -- 172.21.15.94:0/4222703942 <== mon.1 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fdff000d290 con 0x7fdff40fcf80 2024-03-24T10:48:26.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.860+0000 7fdffbad5700 1 -- 172.21.15.94:0/4222703942 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7fdfe006d1f0 msgr2=0x7fdfe006f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:26.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.860+0000 7fdffbad5700 1 --2- 172.21.15.94:0/4222703942 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7fdfe006d1f0 0x7fdfe006f6a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fdfe800b020 tx=0x7fdfe8009e30 comp rx=0 tx=0).stop 2024-03-24T10:48:26.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.860+0000 7fdffbad5700 1 -- 172.21.15.94:0/4222703942 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdff40fcf80 msgr2=0x7fdff4112ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:26.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.860+0000 7fdffbad5700 1 --2- 172.21.15.94:0/4222703942 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdff40fcf80 0x7fdff4112ca0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdff000b790 tx=0x7fdff00091a0 comp rx=0 tx=0).stop 2024-03-24T10:48:26.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.860+0000 7fdffbad5700 1 -- 172.21.15.94:0/4222703942 shutdown_connections 2024-03-24T10:48:26.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.860+0000 7fdffbad5700 1 --2- 172.21.15.94:0/4222703942 >> [v2:172.21.15.94:6800/4143126939,v1:172.21.15.94:6801/4143126939] conn(0x7fdfe006d1f0 0x7fdfe006f6a0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:26.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.860+0000 7fdffbad5700 1 --2- 172.21.15.94:0/4222703942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdff40fbcd0 0x7fdff4110750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:26.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.860+0000 7fdffbad5700 1 --2- 172.21.15.94:0/4222703942 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdff40fcf80 0x7fdff4112ca0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:26.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.860+0000 7fdffbad5700 1 -- 172.21.15.94:0/4222703942 >> 172.21.15.94:0/4222703942 conn(0x7fdff40f7430 msgr2=0x7fdff40f8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:48:26.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.861+0000 7fdffbad5700 1 -- 172.21.15.94:0/4222703942 shutdown_connections 2024-03-24T10:48:26.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:26.861+0000 7fdffbad5700 1 -- 172.21.15.94:0/4222703942 wait complete. 2024-03-24T10:48:27.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 podman[93915]: 2024-03-24 10:48:26.838088351 +0000 UTC m=+0.424948095 container init c779d611e29a976824345c01fe0209d2f25bbb268cb0ec946f706d92946b80f6 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-mon-smithi094, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, ceph=True) 2024-03-24T10:48:27.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 podman[93915]: 2024-03-24 10:48:26.841349774 +0000 UTC m=+0.428209525 container start c779d611e29a976824345c01fe0209d2f25bbb268cb0ec946f706d92946b80f6 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-mon-smithi094, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6) 2024-03-24T10:48:27.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: set uid:gid to 167:167 (ceph:ceph) 2024-03-24T10:48:27.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev), process ceph-mon, pid 2 2024-03-24T10:48:27.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: pidfile_write: ignore empty --pid-file 2024-03-24T10:48:27.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: load: jerasure load: lrc 2024-03-24T10:48:27.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: RocksDB version: 7.9.2 2024-03-24T10:48:27.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Git sha 0 2024-03-24T10:48:27.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Compile date 2024-03-23 03:28:38 2024-03-24T10:48:27.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: DB SUMMARY 2024-03-24T10:48:27.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: DB Session ID: PZLUWCNDMLE2EUP3LHSP 2024-03-24T10:48:27.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: CURRENT file: CURRENT 2024-03-24T10:48:27.249 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: IDENTITY file: IDENTITY 2024-03-24T10:48:27.249 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: MANIFEST file: MANIFEST-000015 size: 967 Bytes 2024-03-24T10:48:27.249 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi094/store.db dir, Total Num: 1, files: 000030.sst 2024-03-24T10:48:27.249 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi094/store.db: 000028.log size: 1924984 ; 2024-03-24T10:48:27.249 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.error_if_exists: 0 2024-03-24T10:48:27.249 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.create_if_missing: 0 2024-03-24T10:48:27.249 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.paranoid_checks: 1 2024-03-24T10:48:27.249 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-24T10:48:27.249 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-24T10:48:27.249 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-24T10:48:27.249 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.env: 0x5572f481cc20 2024-03-24T10:48:27.249 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.fs: PosixFileSystem 2024-03-24T10:48:27.249 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.info_log: 0x5572f53ef980 2024-03-24T10:48:27.249 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.max_file_opening_threads: 16 2024-03-24T10:48:27.251 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.statistics: (nil) 2024-03-24T10:48:27.251 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.use_fsync: 0 2024-03-24T10:48:27.251 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.max_log_file_size: 0 2024-03-24T10:48:27.251 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-24T10:48:27.251 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-24T10:48:27.251 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.keep_log_file_num: 1000 2024-03-24T10:48:27.252 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.recycle_log_file_num: 0 2024-03-24T10:48:27.252 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.allow_fallocate: 1 2024-03-24T10:48:27.252 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.allow_mmap_reads: 0 2024-03-24T10:48:27.252 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.allow_mmap_writes: 0 2024-03-24T10:48:27.252 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.use_direct_reads: 0 2024-03-24T10:48:27.252 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-24T10:48:27.252 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.create_missing_column_families: 0 2024-03-24T10:48:27.252 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.db_log_dir: 2024-03-24T10:48:27.252 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.wal_dir: 2024-03-24T10:48:27.252 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-24T10:48:27.252 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-24T10:48:27.252 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-24T10:48:27.253 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-24T10:48:27.253 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-24T10:48:27.253 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-24T10:48:27.253 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.advise_random_on_open: 1 2024-03-24T10:48:27.253 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.db_write_buffer_size: 0 2024-03-24T10:48:27.253 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.write_buffer_manager: 0x5572f53c21e0 2024-03-24T10:48:27.253 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-24T10:48:27.253 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-24T10:48:27.253 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-24T10:48:27.253 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.rate_limiter: (nil) 2024-03-24T10:48:27.253 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-24T10:48:27.253 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.wal_recovery_mode: 2 2024-03-24T10:48:27.253 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.enable_thread_tracking: 0 2024-03-24T10:48:27.253 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.enable_pipelined_write: 0 2024-03-24T10:48:27.253 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.unordered_write: 0 2024-03-24T10:48:27.254 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-24T10:48:27.254 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-24T10:48:27.254 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-24T10:48:27.254 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-24T10:48:27.254 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.row_cache: None 2024-03-24T10:48:27.254 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.wal_filter: None 2024-03-24T10:48:27.254 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-24T10:48:27.254 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.allow_ingest_behind: 0 2024-03-24T10:48:27.254 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.two_write_queues: 0 2024-03-24T10:48:27.254 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.manual_wal_flush: 0 2024-03-24T10:48:27.254 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.wal_compression: 0 2024-03-24T10:48:27.254 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.atomic_flush: 0 2024-03-24T10:48:27.254 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-24T10:48:27.255 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-24T10:48:27.255 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-24T10:48:27.255 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.log_readahead_size: 0 2024-03-24T10:48:27.255 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-24T10:48:27.255 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.best_efforts_recovery: 0 2024-03-24T10:48:27.255 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-24T10:48:27.255 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-24T10:48:27.255 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.allow_data_in_errors: 0 2024-03-24T10:48:27.255 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.db_host_id: __hostname__ 2024-03-24T10:48:27.255 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-24T10:48:27.255 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.max_background_jobs: 2 2024-03-24T10:48:27.255 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.max_background_compactions: -1 2024-03-24T10:48:27.255 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.max_subcompactions: 1 2024-03-24T10:48:27.255 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-24T10:48:27.255 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-24T10:48:27.255 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-24T10:48:27.256 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.max_total_wal_size: 0 2024-03-24T10:48:27.256 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-24T10:48:27.256 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-24T10:48:27.256 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-24T10:48:27.256 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-24T10:48:27.256 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.max_open_files: -1 2024-03-24T10:48:27.256 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.bytes_per_sync: 0 2024-03-24T10:48:27.256 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-24T10:48:27.256 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-24T10:48:27.256 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.compaction_readahead_size: 0 2024-03-24T10:48:27.256 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.max_background_flushes: -1 2024-03-24T10:48:27.256 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Compression algorithms supported: 2024-03-24T10:48:27.256 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-24T10:48:27.256 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: kZSTD supported: 0 2024-03-24T10:48:27.256 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: kXpressCompression supported: 0 2024-03-24T10:48:27.256 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: kLZ4HCCompression supported: 1 2024-03-24T10:48:27.257 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: kZlibCompression supported: 1 2024-03-24T10:48:27.257 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: kSnappyCompression supported: 1 2024-03-24T10:48:27.257 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: kLZ4Compression supported: 1 2024-03-24T10:48:27.257 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: kBZip2Compression supported: 0 2024-03-24T10:48:27.257 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-24T10:48:27.257 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-24T10:48:27.257 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi094/store.db/MANIFEST-000015 2024-03-24T10:48:27.257 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-24T10:48:27.257 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-24T10:48:27.257 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.merge_operator: 2024-03-24T10:48:27.257 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.compaction_filter: None 2024-03-24T10:48:27.257 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.compaction_filter_factory: None 2024-03-24T10:48:27.257 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.sst_partitioner_factory: None 2024-03-24T10:48:27.257 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-24T10:48:27.257 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-24T10:48:27.258 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5572f53efa80) 2024-03-24T10:48:27.258 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: cache_index_and_filter_blocks: 1 2024-03-24T10:48:27.258 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-24T10:48:27.258 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-24T10:48:27.258 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: pin_top_level_index_and_filter: 1 2024-03-24T10:48:27.258 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: index_type: 0 2024-03-24T10:48:27.258 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: data_block_index_type: 0 2024-03-24T10:48:27.258 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: index_shortening: 1 2024-03-24T10:48:27.258 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-24T10:48:27.258 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: checksum: 4 2024-03-24T10:48:27.258 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: no_block_cache: 0 2024-03-24T10:48:27.258 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: block_cache: 0x5572f53f3090 2024-03-24T10:48:27.258 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: block_cache_name: BinnedLRUCache 2024-03-24T10:48:27.258 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: block_cache_options: 2024-03-24T10:48:27.259 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: capacity : 536870912 2024-03-24T10:48:27.259 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: num_shard_bits : 4 2024-03-24T10:48:27.259 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: strict_capacity_limit : 0 2024-03-24T10:48:27.259 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: high_pri_pool_ratio: 0.000 2024-03-24T10:48:27.259 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: block_cache_compressed: (nil) 2024-03-24T10:48:27.259 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: persistent_cache: (nil) 2024-03-24T10:48:27.259 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: block_size: 4096 2024-03-24T10:48:27.259 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: block_size_deviation: 10 2024-03-24T10:48:27.259 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: block_restart_interval: 16 2024-03-24T10:48:27.259 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: index_block_restart_interval: 1 2024-03-24T10:48:27.259 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: metadata_block_size: 4096 2024-03-24T10:48:27.259 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: partition_filters: 0 2024-03-24T10:48:27.259 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: use_delta_encoding: 1 2024-03-24T10:48:27.259 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: filter_policy: bloomfilter 2024-03-24T10:48:27.259 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: whole_key_filtering: 1 2024-03-24T10:48:27.260 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: verify_compression: 0 2024-03-24T10:48:27.260 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: read_amp_bytes_per_bit: 0 2024-03-24T10:48:27.260 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: format_version: 5 2024-03-24T10:48:27.260 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: enable_index_compression: 1 2024-03-24T10:48:27.260 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: block_align: 0 2024-03-24T10:48:27.260 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: max_auto_readahead_size: 262144 2024-03-24T10:48:27.260 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: prepopulate_block_cache: 0 2024-03-24T10:48:27.260 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: initial_auto_readahead_size: 8192 2024-03-24T10:48:27.260 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: num_file_reads_for_auto_readahead: 2 2024-03-24T10:48:27.260 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.write_buffer_size: 33554432 2024-03-24T10:48:27.260 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.max_write_buffer_number: 2 2024-03-24T10:48:27.260 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.compression: NoCompression 2024-03-24T10:48:27.260 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.bottommost_compression: Disabled 2024-03-24T10:48:27.260 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.prefix_extractor: nullptr 2024-03-24T10:48:27.260 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-24T10:48:27.261 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.num_levels: 7 2024-03-24T10:48:27.261 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-24T10:48:27.261 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-24T10:48:27.261 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-24T10:48:27.261 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-24T10:48:27.261 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-24T10:48:27.261 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-24T10:48:27.261 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-24T10:48:27.261 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-24T10:48:27.261 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-24T10:48:27.261 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-24T10:48:27.261 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-24T10:48:27.261 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-24T10:48:27.261 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-24T10:48:27.262 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.compression_opts.level: 32767 2024-03-24T10:48:27.262 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.compression_opts.strategy: 0 2024-03-24T10:48:27.262 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-24T10:48:27.262 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-24T10:48:27.262 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-24T10:48:27.262 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-24T10:48:27.262 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.compression_opts.enabled: false 2024-03-24T10:48:27.262 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-24T10:48:27.262 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-24T10:48:27.262 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-24T10:48:27.262 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-24T10:48:27.262 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.target_file_size_base: 67108864 2024-03-24T10:48:27.262 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-24T10:48:27.262 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-24T10:48:27.262 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-24T10:48:27.263 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-24T10:48:27.263 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-24T10:48:27.263 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-24T10:48:27.263 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-24T10:48:27.263 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-24T10:48:27.263 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-24T10:48:27.263 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-24T10:48:27.263 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-24T10:48:27.263 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-24T10:48:27.263 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-24T10:48:27.263 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-24T10:48:27.263 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.arena_block_size: 1048576 2024-03-24T10:48:27.263 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-24T10:48:27.263 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-24T10:48:27.263 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.disable_auto_compactions: 0 2024-03-24T10:48:27.264 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-24T10:48:27.264 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-24T10:48:27.264 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-24T10:48:27.264 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-24T10:48:27.264 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-24T10:48:27.264 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-24T10:48:27.264 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-24T10:48:27.264 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-24T10:48:27.264 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-24T10:48:27.264 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-24T10:48:27.264 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.table_properties_collectors: 2024-03-24T10:48:27.264 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.inplace_update_support: 0 2024-03-24T10:48:27.265 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-24T10:48:27.265 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-24T10:48:27.265 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-24T10:48:27.265 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-24T10:48:27.265 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.bloom_locality: 0 2024-03-24T10:48:27.265 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.max_successive_merges: 0 2024-03-24T10:48:27.265 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-24T10:48:27.265 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.paranoid_file_checks: 0 2024-03-24T10:48:27.265 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.force_consistency_checks: 1 2024-03-24T10:48:27.265 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.report_bg_io_stats: 0 2024-03-24T10:48:27.265 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.ttl: 2592000 2024-03-24T10:48:27.265 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-24T10:48:27.265 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-24T10:48:27.265 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-24T10:48:27.266 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.enable_blob_files: false 2024-03-24T10:48:27.266 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.min_blob_size: 0 2024-03-24T10:48:27.266 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.blob_file_size: 268435456 2024-03-24T10:48:27.266 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-24T10:48:27.266 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-24T10:48:27.266 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-24T10:48:27.266 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-24T10:48:27.266 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-24T10:48:27.266 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.blob_file_starting_level: 0 2024-03-24T10:48:27.266 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-24T10:48:27.266 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 30.sst 2024-03-24T10:48:27.266 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-24T10:48:27.266 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi094/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 32, last_sequence is 10719, log_number is 28,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-24T10:48:27.266 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 28 2024-03-24T10:48:27.266 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 1ae0d5c3-c4e6-450d-81ad-f4149dcd597c 2024-03-24T10:48:27.267 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711277306882571, "job": 1, "event": "recovery_started", "wal_files": [28]} 2024-03-24T10:48:27.267 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #28 mode 2 2024-03-24T10:48:27.267 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711277306888423, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 33, "file_size": 1764064, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 10720, "largest_seqno": 11478, "table_properties": {"data_size": 1759726, "index_size": 2364, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 965, "raw_key_size": 8533, "raw_average_key_size": 23, "raw_value_size": 1751885, "raw_average_value_size": 4866, "num_data_blocks": 108, "num_entries": 360, "num_filter_entries": 360, "num_deletions": 2, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711277306, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "1ae0d5c3-c4e6-450d-81ad-f4149dcd597c", "db_session_id": "PZLUWCNDMLE2EUP3LHSP", "orig_file_number": 33, "seqno_to_time_mapping": "N/A"}} 2024-03-24T10:48:27.267 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711277306888538, "job": 1, "event": "recovery_finished"} 2024-03-24T10:48:27.267 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: [db/version_set.cc:5047] Creating manifest 35 2024-03-24T10:48:27.267 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-24T10:48:27.267 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi094/store.db/000028.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-24T10:48:27.267 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5572f54e4000 2024-03-24T10:48:27.267 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: DB pointer 0x5572f54ce000 2024-03-24T10:48:27.267 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-24T10:48:27.267 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-24T10:48:27.268 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: ** DB Stats ** 2024-03-24T10:48:27.268 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-24T10:48:27.268 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-24T10:48:27.268 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-24T10:48:27.268 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-24T10:48:27.268 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-24T10:48:27.268 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-24T10:48:27.268 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-24T10:48:27.268 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: 2024-03-24T10:48:27.268 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: ** Compaction Stats [default] ** 2024-03-24T10:48:27.268 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-24T10:48:27.268 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-24T10:48:27.268 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: L0 1/0 1.68 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 524.7 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-03-24T10:48:27.268 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: L6 1/0 5.71 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-03-24T10:48:27.268 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Sum 2/0 7.39 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 524.7 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-03-24T10:48:27.269 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 524.7 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-03-24T10:48:27.269 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: 2024-03-24T10:48:27.269 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: ** Compaction Stats [default] ** 2024-03-24T10:48:27.269 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-24T10:48:27.269 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-24T10:48:27.269 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 524.7 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-03-24T10:48:27.269 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: 2024-03-24T10:48:27.269 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-24T10:48:27.269 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: 2024-03-24T10:48:27.269 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-24T10:48:27.269 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Flush(GB): cumulative 0.002, interval 0.002 2024-03-24T10:48:27.269 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-24T10:48:27.269 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-24T10:48:27.269 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-24T10:48:27.269 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-24T10:48:27.270 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Cumulative compaction: 0.00 GB write, 89.86 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-24T10:48:27.270 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Interval compaction: 0.00 GB write, 89.86 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-24T10:48:27.270 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-24T10:48:27.270 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Block cache BinnedLRUCache@0x5572f53f3090#2 capacity: 512.00 MB usage: 3.61 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-03-24T10:48:27.270 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,1.03 KB,0.000196695%) IndexBlock(1,2.58 KB,0.000491738%) Misc(1,0.00 KB,0%) 2024-03-24T10:48:27.270 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: 2024-03-24T10:48:27.270 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-24T10:48:27.270 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: starting mon.smithi094 rank 0 at public addrs [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] at bind addrs [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi094 fsid 386110d4-e9ca-11ee-95cd-87774f69a715 2024-03-24T10:48:27.270 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: mon.smithi094@-1(???) e2 preinit fsid 386110d4-e9ca-11ee-95cd-87774f69a715 2024-03-24T10:48:27.270 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: mon.smithi094@-1(???).mds e18 new map 2024-03-24T10:48:27.270 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: mon.smithi094@-1(???).mds e18 print_map 2024-03-24T10:48:27.270 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: e18 2024-03-24T10:48:27.270 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T10:48:27.270 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:48:27.270 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: legacy client fscid: 1 2024-03-24T10:48:27.271 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: 2024-03-24T10:48:27.271 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Filesystem 'cephfs' (1) 2024-03-24T10:48:27.271 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: fs_name cephfs 2024-03-24T10:48:27.271 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: epoch 18 2024-03-24T10:48:27.271 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-24T10:48:27.271 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: created 2024-03-24T10:43:39.435738+0000 2024-03-24T10:48:27.271 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: modified 2024-03-24T10:43:58.133901+0000 2024-03-24T10:48:27.271 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: tableserver 0 2024-03-24T10:48:27.271 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: root 0 2024-03-24T10:48:27.271 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: session_timeout 60 2024-03-24T10:48:27.271 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: session_autoclose 300 2024-03-24T10:48:27.271 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: max_file_size 1099511627776 2024-03-24T10:48:27.271 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: max_xattr_size 65536 2024-03-24T10:48:27.271 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: required_client_features {} 2024-03-24T10:48:27.271 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: last_failure 0 2024-03-24T10:48:27.272 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: last_failure_osd_epoch 46 2024-03-24T10:48:27.272 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:48:27.272 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: max_mds 1 2024-03-24T10:48:27.272 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: in 0 2024-03-24T10:48:27.272 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: up {0=24251} 2024-03-24T10:48:27.272 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: failed 2024-03-24T10:48:27.272 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: damaged 2024-03-24T10:48:27.272 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: stopped 2024-03-24T10:48:27.272 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: data_pools [3] 2024-03-24T10:48:27.272 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: metadata_pool 2 2024-03-24T10:48:27.272 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: inline_data disabled 2024-03-24T10:48:27.272 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: balancer 2024-03-24T10:48:27.272 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: bal_rank_mask -1 2024-03-24T10:48:27.272 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: standby_count_wanted 1 2024-03-24T10:48:27.272 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: qdb_cluster leader: 0 members: 2024-03-24T10:48:27.273 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: [mds.cephfs.smithi096.egbmoq{0:24251} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.96:6824/1539752208,v1:172.21.15.96:6825/1539752208] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:48:27.273 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: [mds.cephfs.smithi094.llefgu{0:14500} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.94:6828/126762980,v1:172.21.15.94:6829/126762980] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:48:27.273 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: 2024-03-24T10:48:27.273 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: 2024-03-24T10:48:27.273 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Standby daemons: 2024-03-24T10:48:27.273 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: 2024-03-24T10:48:27.273 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: [mds.cephfs.smithi094.efiqte{-1:14512} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.94:6826/1636528860,v1:172.21.15.94:6827/1636528860] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:48:27.273 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: [mds.cephfs.smithi096.jtebhh{-1:24261} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.96:6826/1321087930,v1:172.21.15.96:6827/1321087930] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:48:27.273 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: mon.smithi094@-1(???).osd e48 crush map has features 3314933000852226048, adjusting msgr requires 2024-03-24T10:48:27.273 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: mon.smithi094@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-03-24T10:48:27.273 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: mon.smithi094@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-03-24T10:48:27.273 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: mon.smithi094@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-03-24T10:48:27.273 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 ceph-mon[93957]: mon.smithi094@-1(???).paxosservice(auth 1..21) refresh upgraded, format 0 -> 3 2024-03-24T10:48:27.273 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 bash[93915]: c779d611e29a976824345c01fe0209d2f25bbb268cb0ec946f706d92946b80f6 2024-03-24T10:48:27.273 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:26 smithi094 systemd[1]: Started Ceph mon.smithi094 for 386110d4-e9ca-11ee-95cd-87774f69a715. 2024-03-24T10:48:28.659 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:28 smithi094 ceph-mon[93957]: pgmap v20: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:48:28.659 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:28 smithi094 ceph-mon[93957]: from='client.14834 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:48:28.659 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:28 smithi094 ceph-mon[93957]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata", "id": "smithi094"}]: dispatch 2024-03-24T10:48:28.659 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:28 smithi094 ceph-mon[93957]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata", "id": "smithi096"}]: dispatch 2024-03-24T10:48:28.659 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:28 smithi094 ceph-mon[93957]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:48:28.659 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:28 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/384569478' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:48:28.659 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:28 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/3132577461' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T10:48:28.659 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:28 smithi094 ceph-mon[93957]: from='client.24577 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:48:28.660 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:28 smithi094 ceph-mon[93957]: pgmap v21: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:48:28.660 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:28 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/4222703942' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T10:48:28.660 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:28 smithi094 ceph-mon[93957]: mon.smithi094 calling monitor election 2024-03-24T10:48:28.660 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:28 smithi094 ceph-mon[93957]: mon.smithi094 is new leader, mons smithi094,smithi096 in quorum (ranks 0,1) 2024-03-24T10:48:28.660 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:28 smithi094 ceph-mon[93957]: monmap epoch 2 2024-03-24T10:48:28.660 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:28 smithi094 ceph-mon[93957]: fsid 386110d4-e9ca-11ee-95cd-87774f69a715 2024-03-24T10:48:28.660 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:28 smithi094 ceph-mon[93957]: last_changed 2024-03-24T10:39:24.793140+0000 2024-03-24T10:48:28.660 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:28 smithi094 ceph-mon[93957]: created 2024-03-24T10:36:06.632144+0000 2024-03-24T10:48:28.660 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:28 smithi094 ceph-mon[93957]: min_mon_release 17 (quincy) 2024-03-24T10:48:28.660 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:28 smithi094 ceph-mon[93957]: election_strategy: 1 2024-03-24T10:48:28.660 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:28 smithi094 ceph-mon[93957]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.smithi094 2024-03-24T10:48:28.660 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:28 smithi094 ceph-mon[93957]: 1: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.smithi096 2024-03-24T10:48:28.660 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:28 smithi094 ceph-mon[93957]: fsmap cephfs:1 {0=cephfs.smithi096.egbmoq=up:active} 1 up:standby-replay 2 up:standby 2024-03-24T10:48:28.660 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:28 smithi094 ceph-mon[93957]: osdmap e48: 6 total, 6 up, 6 in 2024-03-24T10:48:28.660 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:28 smithi094 ceph-mon[93957]: mgrmap e30: smithi094.yaethu(active, since 37s), standbys: smithi096.gfonnu 2024-03-24T10:48:28.661 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:28 smithi094 ceph-mon[93957]: overall HEALTH_OK 2024-03-24T10:48:28.661 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:28 smithi094 ceph-mon[93957]: from='mgr.14774 ' entity='' 2024-03-24T10:48:28.661 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:28 smithi094 ceph-mon[93957]: mgrmap e31: smithi094.yaethu(active, since 37s), standbys: smithi096.gfonnu 2024-03-24T10:48:28.796 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:28 smithi096 ceph-mon[38097]: pgmap v20: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:48:28.796 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:28 smithi096 ceph-mon[38097]: from='client.14834 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:48:28.796 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:28 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata", "id": "smithi094"}]: dispatch 2024-03-24T10:48:28.796 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:28 smithi096 ceph-mon[38097]: from='mgr.14774 172.21.15.94:0/1573661964' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata", "id": "smithi096"}]: dispatch 2024-03-24T10:48:28.797 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:28 smithi096 ceph-mon[38097]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:48:28.797 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:28 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/384569478' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:48:28.797 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:28 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/3132577461' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T10:48:28.797 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:28 smithi096 ceph-mon[38097]: from='client.24577 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:48:28.797 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:28 smithi096 ceph-mon[38097]: pgmap v21: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:48:28.797 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:28 smithi096 ceph-mon[38097]: from='client.? 172.21.15.94:0/4222703942' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T10:48:28.797 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:28 smithi096 ceph-mon[38097]: mon.smithi094 calling monitor election 2024-03-24T10:48:28.797 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:28 smithi096 ceph-mon[38097]: mon.smithi094 is new leader, mons smithi094,smithi096 in quorum (ranks 0,1) 2024-03-24T10:48:28.797 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:28 smithi096 ceph-mon[38097]: monmap epoch 2 2024-03-24T10:48:28.797 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:28 smithi096 ceph-mon[38097]: fsid 386110d4-e9ca-11ee-95cd-87774f69a715 2024-03-24T10:48:28.797 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:28 smithi096 ceph-mon[38097]: last_changed 2024-03-24T10:39:24.793140+0000 2024-03-24T10:48:28.797 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:28 smithi096 ceph-mon[38097]: created 2024-03-24T10:36:06.632144+0000 2024-03-24T10:48:28.797 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:28 smithi096 ceph-mon[38097]: min_mon_release 17 (quincy) 2024-03-24T10:48:28.797 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:28 smithi096 ceph-mon[38097]: election_strategy: 1 2024-03-24T10:48:28.798 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:28 smithi096 ceph-mon[38097]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.smithi094 2024-03-24T10:48:28.798 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:28 smithi096 ceph-mon[38097]: 1: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.smithi096 2024-03-24T10:48:28.798 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:28 smithi096 ceph-mon[38097]: fsmap cephfs:1 {0=cephfs.smithi096.egbmoq=up:active} 1 up:standby-replay 2 up:standby 2024-03-24T10:48:28.798 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:28 smithi096 ceph-mon[38097]: osdmap e48: 6 total, 6 up, 6 in 2024-03-24T10:48:28.798 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:28 smithi096 ceph-mon[38097]: mgrmap e30: smithi094.yaethu(active, since 37s), standbys: smithi096.gfonnu 2024-03-24T10:48:28.798 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:28 smithi096 ceph-mon[38097]: overall HEALTH_OK 2024-03-24T10:48:28.798 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:28 smithi096 ceph-mon[38097]: from='mgr.14774 ' entity='' 2024-03-24T10:48:28.798 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:28 smithi096 ceph-mon[38097]: mgrmap e31: smithi094.yaethu(active, since 37s), standbys: smithi096.gfonnu 2024-03-24T10:48:35.252 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:35 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-mon-smithi096[38091]: 2024-03-24T10:48:35.237+0000 7f1f11059700 -1 mon.smithi096@1(peon).osd e49 update_from_paxos full map CRC mismatch, resetting to canonical 2024-03-24T10:48:35.252 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:35 smithi096 ceph-mon[38097]: Standby manager daemon smithi096.gfonnu restarted 2024-03-24T10:48:35.252 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:35 smithi096 ceph-mon[38097]: Standby manager daemon smithi096.gfonnu started 2024-03-24T10:48:35.252 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:35 smithi096 ceph-mon[38097]: from='mgr.? 172.21.15.96:0/3031448087' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi096.gfonnu/crt"}]: dispatch 2024-03-24T10:48:35.252 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:35 smithi096 ceph-mon[38097]: from='mgr.? 172.21.15.96:0/3031448087' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T10:48:35.252 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:35 smithi096 ceph-mon[38097]: from='mgr.? 172.21.15.96:0/3031448087' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi096.gfonnu/key"}]: dispatch 2024-03-24T10:48:35.252 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:35 smithi096 ceph-mon[38097]: from='mgr.? 172.21.15.96:0/3031448087' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T10:48:35.252 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:35 smithi096 ceph-mon[38097]: Active manager daemon smithi094.yaethu restarted 2024-03-24T10:48:35.252 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:35 smithi096 ceph-mon[38097]: Activating manager daemon smithi094.yaethu 2024-03-24T10:48:35.252 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:35 smithi096 ceph-mon[38097]: osdmap e49: 6 total, 6 up, 6 in 2024-03-24T10:48:35.252 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:35 smithi096 ceph-mon[38097]: mgrmap e32: smithi094.yaethu(active, starting, since 0.00511592s), standbys: smithi096.gfonnu 2024-03-24T10:48:35.252 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:35 smithi096 ceph-mon[38097]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata", "id": "smithi094"}]: dispatch 2024-03-24T10:48:35.252 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:35 smithi096 ceph-mon[38097]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata", "id": "smithi096"}]: dispatch 2024-03-24T10:48:35.253 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:35 smithi096 ceph-mon[38097]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.llefgu"}]: dispatch 2024-03-24T10:48:35.253 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:35 smithi096 ceph-mon[38097]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.efiqte"}]: dispatch 2024-03-24T10:48:35.253 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:35 smithi096 ceph-mon[38097]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi096.egbmoq"}]: dispatch 2024-03-24T10:48:35.253 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:35 smithi096 ceph-mon[38097]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi096.jtebhh"}]: dispatch 2024-03-24T10:48:35.253 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:35 smithi096 ceph-mon[38097]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mgr metadata", "who": "smithi094.yaethu", "id": "smithi094.yaethu"}]: dispatch 2024-03-24T10:48:35.253 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:35 smithi096 ceph-mon[38097]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mgr metadata", "who": "smithi096.gfonnu", "id": "smithi096.gfonnu"}]: dispatch 2024-03-24T10:48:35.253 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:35 smithi096 ceph-mon[38097]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T10:48:35.253 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:35 smithi096 ceph-mon[38097]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T10:48:35.253 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:35 smithi096 ceph-mon[38097]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T10:48:35.253 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:35 smithi096 ceph-mon[38097]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T10:48:35.253 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:35 smithi096 ceph-mon[38097]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T10:48:35.253 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:35 smithi096 ceph-mon[38097]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T10:48:35.253 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:35 smithi096 ceph-mon[38097]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-24T10:48:35.253 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:35 smithi096 ceph-mon[38097]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-24T10:48:35.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:35 smithi094 ceph-mon[93957]: Standby manager daemon smithi096.gfonnu restarted 2024-03-24T10:48:35.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:35 smithi094 ceph-mon[93957]: Standby manager daemon smithi096.gfonnu started 2024-03-24T10:48:35.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:35 smithi094 ceph-mon[93957]: from='mgr.? 172.21.15.96:0/3031448087' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi096.gfonnu/crt"}]: dispatch 2024-03-24T10:48:35.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:35 smithi094 ceph-mon[93957]: from='mgr.? 172.21.15.96:0/3031448087' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T10:48:35.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:35 smithi094 ceph-mon[93957]: from='mgr.? 172.21.15.96:0/3031448087' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi096.gfonnu/key"}]: dispatch 2024-03-24T10:48:35.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:35 smithi094 ceph-mon[93957]: from='mgr.? 172.21.15.96:0/3031448087' entity='mgr.smithi096.gfonnu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T10:48:35.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:35 smithi094 ceph-mon[93957]: Active manager daemon smithi094.yaethu restarted 2024-03-24T10:48:35.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:35 smithi094 ceph-mon[93957]: Activating manager daemon smithi094.yaethu 2024-03-24T10:48:35.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:35 smithi094 ceph-mon[93957]: osdmap e49: 6 total, 6 up, 6 in 2024-03-24T10:48:35.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:35 smithi094 ceph-mon[93957]: mgrmap e32: smithi094.yaethu(active, starting, since 0.00511592s), standbys: smithi096.gfonnu 2024-03-24T10:48:35.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:35 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata", "id": "smithi094"}]: dispatch 2024-03-24T10:48:35.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:35 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata", "id": "smithi096"}]: dispatch 2024-03-24T10:48:35.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:35 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.llefgu"}]: dispatch 2024-03-24T10:48:35.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:35 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.efiqte"}]: dispatch 2024-03-24T10:48:35.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:35 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi096.egbmoq"}]: dispatch 2024-03-24T10:48:35.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:35 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi096.jtebhh"}]: dispatch 2024-03-24T10:48:35.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:35 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mgr metadata", "who": "smithi094.yaethu", "id": "smithi094.yaethu"}]: dispatch 2024-03-24T10:48:35.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:35 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mgr metadata", "who": "smithi096.gfonnu", "id": "smithi096.gfonnu"}]: dispatch 2024-03-24T10:48:35.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:35 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T10:48:35.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:35 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T10:48:35.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:35 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T10:48:35.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:35 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T10:48:35.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:35 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T10:48:35.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:35 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T10:48:35.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:35 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-24T10:48:35.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:35 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-24T10:48:36.534 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:36 smithi094 ceph-mon[93957]: failed to encode map e49 with expected crc 2024-03-24T10:48:36.534 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:36 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-24T10:48:36.534 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:36 smithi094 ceph-mon[93957]: failed to encode map e49 with expected crc 2024-03-24T10:48:36.534 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:36 smithi094 ceph-mon[93957]: failed to encode map e49 with expected crc 2024-03-24T10:48:36.534 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:36 smithi094 ceph-mon[93957]: Manager daemon smithi094.yaethu is now available 2024-03-24T10:48:36.534 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:36 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:48:36.534 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:36 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:48:36.535 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:36 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:48:36.535 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:36 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi094.yaethu/mirror_snapshot_schedule"}]: dispatch 2024-03-24T10:48:36.535 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:36 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi094.yaethu/trash_purge_schedule"}]: dispatch 2024-03-24T10:48:36.536 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:36 smithi096 ceph-mon[38097]: failed to encode map e49 with expected crc 2024-03-24T10:48:36.537 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:36 smithi096 ceph-mon[38097]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-24T10:48:36.537 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:36 smithi096 ceph-mon[38097]: failed to encode map e49 with expected crc 2024-03-24T10:48:36.537 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:36 smithi096 ceph-mon[38097]: failed to encode map e49 with expected crc 2024-03-24T10:48:36.537 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:36 smithi096 ceph-mon[38097]: Manager daemon smithi094.yaethu is now available 2024-03-24T10:48:36.537 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:36 smithi096 ceph-mon[38097]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:48:36.537 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:36 smithi096 ceph-mon[38097]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:48:36.537 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:36 smithi096 ceph-mon[38097]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:48:36.537 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:36 smithi096 ceph-mon[38097]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi094.yaethu/mirror_snapshot_schedule"}]: dispatch 2024-03-24T10:48:36.537 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:36 smithi096 ceph-mon[38097]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi094.yaethu/trash_purge_schedule"}]: dispatch 2024-03-24T10:48:37.422 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:37 smithi094 ceph-mon[93957]: [24/Mar/2024:10:48:35] ENGINE Bus STARTING 2024-03-24T10:48:37.422 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:37 smithi094 ceph-mon[93957]: [24/Mar/2024:10:48:35] ENGINE Serving on http://172.21.15.94:8765 2024-03-24T10:48:37.422 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:37 smithi094 ceph-mon[93957]: [24/Mar/2024:10:48:35] ENGINE Serving on https://172.21.15.94:7150 2024-03-24T10:48:37.422 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:37 smithi094 ceph-mon[93957]: [24/Mar/2024:10:48:35] ENGINE Bus STARTED 2024-03-24T10:48:37.422 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:37 smithi094 ceph-mon[93957]: mgrmap e33: smithi094.yaethu(active, since 1.02254s), standbys: smithi096.gfonnu 2024-03-24T10:48:37.422 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:37 smithi094 ceph-mon[93957]: pgmap v3: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-24T10:48:37.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:37 smithi096 ceph-mon[38097]: [24/Mar/2024:10:48:35] ENGINE Bus STARTING 2024-03-24T10:48:37.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:37 smithi096 ceph-mon[38097]: [24/Mar/2024:10:48:35] ENGINE Serving on http://172.21.15.94:8765 2024-03-24T10:48:37.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:37 smithi096 ceph-mon[38097]: [24/Mar/2024:10:48:35] ENGINE Serving on https://172.21.15.94:7150 2024-03-24T10:48:37.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:37 smithi096 ceph-mon[38097]: [24/Mar/2024:10:48:35] ENGINE Bus STARTED 2024-03-24T10:48:37.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:37 smithi096 ceph-mon[38097]: mgrmap e33: smithi094.yaethu(active, since 1.02254s), standbys: smithi096.gfonnu 2024-03-24T10:48:37.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:37 smithi096 ceph-mon[38097]: pgmap v3: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-24T10:48:38.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:38 smithi096 ceph-mon[38097]: pgmap v4: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-24T10:48:38.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:38 smithi096 ceph-mon[38097]: mgrmap e34: smithi094.yaethu(active, since 2s), standbys: smithi096.gfonnu 2024-03-24T10:48:38.659 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:38 smithi094 ceph-mon[93957]: pgmap v4: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-24T10:48:38.659 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:38 smithi094 ceph-mon[93957]: mgrmap e34: smithi094.yaethu(active, since 2s), standbys: smithi096.gfonnu 2024-03-24T10:48:39.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:39 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:39.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:39 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:39.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:39 smithi094 ceph-mon[93957]: pgmap v5: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-24T10:48:39.811 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:39 smithi096 ceph-mon[38097]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:39.811 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:39 smithi096 ceph-mon[38097]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:39.811 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:39 smithi096 ceph-mon[38097]: pgmap v5: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-24T10:48:40.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:40 smithi096 ceph-mon[38097]: mgrmap e35: smithi094.yaethu(active, since 4s), standbys: smithi096.gfonnu 2024-03-24T10:48:40.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:40 smithi096 ceph-mon[38097]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:40.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:40 smithi096 ceph-mon[38097]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:40.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:40 smithi094 ceph-mon[93957]: mgrmap e35: smithi094.yaethu(active, since 4s), standbys: smithi096.gfonnu 2024-03-24T10:48:40.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:40 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:40.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:40 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:42.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:41 smithi096 ceph-mon[38097]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:42.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:41 smithi096 ceph-mon[38097]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:42.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:41 smithi096 ceph-mon[38097]: pgmap v6: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-24T10:48:42.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:41 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:42.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:41 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:42.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:41 smithi094 ceph-mon[93957]: pgmap v6: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-24T10:48:42.967 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:42 smithi096 ceph-mon[38097]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:42.967 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:42 smithi096 ceph-mon[38097]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:43.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:42 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:43.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:42 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:44.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:44 smithi096 ceph-mon[38097]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:44.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:44 smithi096 ceph-mon[38097]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:44.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:44 smithi096 ceph-mon[38097]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:48:44.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:44 smithi096 ceph-mon[38097]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:48:44.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:44 smithi096 ceph-mon[38097]: Updating smithi094:/etc/ceph/ceph.conf 2024-03-24T10:48:44.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:44 smithi096 ceph-mon[38097]: Updating smithi096:/etc/ceph/ceph.conf 2024-03-24T10:48:44.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:44 smithi096 ceph-mon[38097]: pgmap v7: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 0 B/s wr, 18 op/s 2024-03-24T10:48:44.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:44 smithi096 ceph-mon[38097]: Updating smithi094:/var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T10:48:44.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:44 smithi096 ceph-mon[38097]: Updating smithi096:/var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T10:48:44.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:44 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:44.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:44 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:44.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:44 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:48:44.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:44 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:48:44.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:44 smithi094 ceph-mon[93957]: Updating smithi094:/etc/ceph/ceph.conf 2024-03-24T10:48:44.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:44 smithi094 ceph-mon[93957]: Updating smithi096:/etc/ceph/ceph.conf 2024-03-24T10:48:44.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:44 smithi094 ceph-mon[93957]: pgmap v7: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 0 B/s wr, 18 op/s 2024-03-24T10:48:44.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:44 smithi094 ceph-mon[93957]: Updating smithi094:/var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T10:48:44.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:44 smithi094 ceph-mon[93957]: Updating smithi096:/var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T10:48:45.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:45 smithi096 ceph-mon[38097]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-03-24T10:48:45.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:45 smithi096 ceph-mon[38097]: Updating smithi096:/etc/ceph/ceph.client.admin.keyring 2024-03-24T10:48:45.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:45 smithi096 ceph-mon[38097]: Updating smithi094:/var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T10:48:45.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:45 smithi096 ceph-mon[38097]: Updating smithi096:/var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T10:48:45.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:45 smithi096 ceph-mon[38097]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:45.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:45 smithi096 ceph-mon[38097]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:45.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:45 smithi096 ceph-mon[38097]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:45.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:45 smithi096 ceph-mon[38097]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:45.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:45 smithi096 ceph-mon[38097]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:45.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:45 smithi096 ceph-mon[38097]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:48:45.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:45 smithi096 ceph-mon[38097]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:48:45.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:45 smithi096 ceph-mon[38097]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-24T10:48:45.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:45 smithi094 ceph-mon[93957]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-03-24T10:48:45.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:45 smithi094 ceph-mon[93957]: Updating smithi096:/etc/ceph/ceph.client.admin.keyring 2024-03-24T10:48:45.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:45 smithi094 ceph-mon[93957]: Updating smithi094:/var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T10:48:45.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:45 smithi094 ceph-mon[93957]: Updating smithi096:/var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T10:48:45.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:45 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:45.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:45 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:45.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:45 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:45.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:45 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:45.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:45 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:45.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:45 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:48:45.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:45 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:48:45.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:45 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-24T10:48:46.359 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:46 smithi096 ceph-mon[38097]: pgmap v8: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-24T10:48:46.359 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:46 smithi096 ceph-mon[38097]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:46.359 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:46 smithi096 ceph-mon[38097]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T10:48:46.359 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:46 smithi096 ceph-mon[38097]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T10:48:46.359 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:46 smithi096 ceph-mon[38097]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:48:46.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:46 smithi094 ceph-mon[93957]: pgmap v8: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-24T10:48:46.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:46 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:46.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:46 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T10:48:46.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:46 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T10:48:46.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:46 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:48:47.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:47 smithi094 ceph-mon[93957]: Upgrade: Updating mon.smithi096 2024-03-24T10:48:47.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:47 smithi094 ceph-mon[93957]: Deploying daemon mon.smithi096 on smithi096 2024-03-24T10:48:47.308 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:47 smithi096 ceph-mon[38097]: Upgrade: Updating mon.smithi096 2024-03-24T10:48:47.309 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:47 smithi096 ceph-mon[38097]: Deploying daemon mon.smithi096 on smithi096 2024-03-24T10:48:48.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:47 smithi096 systemd[1]: Stopping Ceph mon.smithi096 for 386110d4-e9ca-11ee-95cd-87774f69a715... 2024-03-24T10:48:48.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:47 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-mon-smithi096[38091]: 2024-03-24T10:48:47.987+0000 7f1f16864700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi096 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T10:48:48.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:47 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-mon-smithi096[38091]: 2024-03-24T10:48:47.987+0000 7f1f16864700 -1 mon.smithi096@1(peon) e2 *** Got Signal Terminated *** 2024-03-24T10:48:48.496 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:48 smithi096 podman[84085]: 2024-03-24 10:48:48.223432625 +0000 UTC m=+0.331840914 container died 710030b222cb7d230d798028067f9c7c15263afea0d120799e9f9958789d8e56 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-mon-smithi096, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=quincy-9c4e0c3, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T10:48:48.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:48 smithi096 podman[84085]: 2024-03-24 10:48:48.6046186 +0000 UTC m=+0.713026886 container cleanup 710030b222cb7d230d798028067f9c7c15263afea0d120799e9f9958789d8e56 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-mon-smithi096, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=quincy-9c4e0c3, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T10:48:48.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:48 smithi096 bash[84085]: ceph-386110d4-e9ca-11ee-95cd-87774f69a715-mon-smithi096 2024-03-24T10:48:49.185 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:48 smithi096 podman[84099]: 2024-03-24 10:48:48.881240463 +0000 UTC m=+0.652232972 container remove 710030b222cb7d230d798028067f9c7c15263afea0d120799e9f9958789d8e56 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-mon-smithi096, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=quincy-9c4e0c3, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-24T10:48:49.186 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:48 smithi096 systemd[1]: ceph-386110d4-e9ca-11ee-95cd-87774f69a715@mon.smithi096.service: Deactivated successfully. 2024-03-24T10:48:49.186 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:48 smithi096 systemd[1]: Stopped Ceph mon.smithi096 for 386110d4-e9ca-11ee-95cd-87774f69a715. 2024-03-24T10:48:49.186 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:48 smithi096 systemd[1]: ceph-386110d4-e9ca-11ee-95cd-87774f69a715@mon.smithi096.service: Consumed 5.523s CPU time. 2024-03-24T10:48:49.451 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 systemd[1]: Starting Ceph mon.smithi096 for 386110d4-e9ca-11ee-95cd-87774f69a715... 2024-03-24T10:48:49.451 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 podman[84231]: 2024-03-24 10:48:49.315656066 +0000 UTC m=+0.019557047 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-24T10:48:49.710 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 podman[84231]: 2024-03-24 10:48:49.474277201 +0000 UTC m=+0.178178171 container create ed9bd0bd57f6827b4d30ceb525471127356c7d2b1879c6048709be361c5a8f51 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-mon-smithi096, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS) 2024-03-24T10:48:49.710 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 podman[84231]: 2024-03-24 10:48:49.710444882 +0000 UTC m=+0.414345866 container init ed9bd0bd57f6827b4d30ceb525471127356c7d2b1879c6048709be361c5a8f51 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-mon-smithi096, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-24T10:48:50.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 podman[84231]: 2024-03-24 10:48:49.71412521 +0000 UTC m=+0.418026189 container start ed9bd0bd57f6827b4d30ceb525471127356c7d2b1879c6048709be361c5a8f51 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-mon-smithi096, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T10:48:50.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: set uid:gid to 167:167 (ceph:ceph) 2024-03-24T10:48:50.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev), process ceph-mon, pid 2 2024-03-24T10:48:50.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: pidfile_write: ignore empty --pid-file 2024-03-24T10:48:50.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: load: jerasure load: lrc 2024-03-24T10:48:50.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: RocksDB version: 7.9.2 2024-03-24T10:48:50.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Git sha 0 2024-03-24T10:48:50.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Compile date 2024-03-23 03:28:38 2024-03-24T10:48:50.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: DB SUMMARY 2024-03-24T10:48:50.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: DB Session ID: R1L5NPBXI7LCMZSCAT63 2024-03-24T10:48:50.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: CURRENT file: CURRENT 2024-03-24T10:48:50.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: IDENTITY file: IDENTITY 2024-03-24T10:48:50.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: MANIFEST file: MANIFEST-000009 size: 1079 Bytes 2024-03-24T10:48:50.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi096/store.db dir, Total Num: 1, files: 000027.sst 2024-03-24T10:48:50.077 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi096/store.db: 000025.log size: 299254 ; 2024-03-24T10:48:50.077 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.error_if_exists: 0 2024-03-24T10:48:50.077 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.create_if_missing: 0 2024-03-24T10:48:50.077 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.paranoid_checks: 1 2024-03-24T10:48:50.077 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-24T10:48:50.077 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-24T10:48:50.077 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-24T10:48:50.077 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.env: 0x564ad1565c20 2024-03-24T10:48:50.077 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.fs: PosixFileSystem 2024-03-24T10:48:50.077 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.info_log: 0x564ad3767980 2024-03-24T10:48:50.077 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.max_file_opening_threads: 16 2024-03-24T10:48:50.077 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.statistics: (nil) 2024-03-24T10:48:50.077 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.use_fsync: 0 2024-03-24T10:48:50.078 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.max_log_file_size: 0 2024-03-24T10:48:50.078 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-24T10:48:50.078 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-24T10:48:50.078 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.keep_log_file_num: 1000 2024-03-24T10:48:50.078 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.recycle_log_file_num: 0 2024-03-24T10:48:50.078 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.allow_fallocate: 1 2024-03-24T10:48:50.079 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.allow_mmap_reads: 0 2024-03-24T10:48:50.079 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.allow_mmap_writes: 0 2024-03-24T10:48:50.080 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.use_direct_reads: 0 2024-03-24T10:48:50.080 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-24T10:48:50.080 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.create_missing_column_families: 0 2024-03-24T10:48:50.080 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.db_log_dir: 2024-03-24T10:48:50.080 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.wal_dir: 2024-03-24T10:48:50.080 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-24T10:48:50.080 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-24T10:48:50.080 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-24T10:48:50.080 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-24T10:48:50.080 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-24T10:48:50.080 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-24T10:48:50.080 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.advise_random_on_open: 1 2024-03-24T10:48:50.080 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.db_write_buffer_size: 0 2024-03-24T10:48:50.080 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.write_buffer_manager: 0x564ad373a1e0 2024-03-24T10:48:50.080 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-24T10:48:50.081 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-24T10:48:50.081 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-24T10:48:50.081 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.rate_limiter: (nil) 2024-03-24T10:48:50.081 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-24T10:48:50.081 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.wal_recovery_mode: 2 2024-03-24T10:48:50.081 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.enable_thread_tracking: 0 2024-03-24T10:48:50.081 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.enable_pipelined_write: 0 2024-03-24T10:48:50.081 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.unordered_write: 0 2024-03-24T10:48:50.081 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-24T10:48:50.081 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-24T10:48:50.081 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-24T10:48:50.081 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-24T10:48:50.081 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.row_cache: None 2024-03-24T10:48:50.081 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.wal_filter: None 2024-03-24T10:48:50.081 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-24T10:48:50.082 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.allow_ingest_behind: 0 2024-03-24T10:48:50.082 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.two_write_queues: 0 2024-03-24T10:48:50.082 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.manual_wal_flush: 0 2024-03-24T10:48:50.082 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.wal_compression: 0 2024-03-24T10:48:50.082 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.atomic_flush: 0 2024-03-24T10:48:50.082 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-24T10:48:50.082 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-24T10:48:50.082 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-24T10:48:50.082 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.log_readahead_size: 0 2024-03-24T10:48:50.082 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-24T10:48:50.082 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.best_efforts_recovery: 0 2024-03-24T10:48:50.082 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-24T10:48:50.082 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-24T10:48:50.083 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.allow_data_in_errors: 0 2024-03-24T10:48:50.083 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.db_host_id: __hostname__ 2024-03-24T10:48:50.083 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-24T10:48:50.083 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.max_background_jobs: 2 2024-03-24T10:48:50.083 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.max_background_compactions: -1 2024-03-24T10:48:50.083 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.max_subcompactions: 1 2024-03-24T10:48:50.083 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-24T10:48:50.083 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-24T10:48:50.083 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-24T10:48:50.083 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.max_total_wal_size: 0 2024-03-24T10:48:50.083 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-24T10:48:50.083 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-24T10:48:50.083 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-24T10:48:50.083 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-24T10:48:50.083 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.max_open_files: -1 2024-03-24T10:48:50.084 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.bytes_per_sync: 0 2024-03-24T10:48:50.084 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-24T10:48:50.084 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-24T10:48:50.084 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.compaction_readahead_size: 0 2024-03-24T10:48:50.084 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.max_background_flushes: -1 2024-03-24T10:48:50.084 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Compression algorithms supported: 2024-03-24T10:48:50.084 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-24T10:48:50.084 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: kZSTD supported: 0 2024-03-24T10:48:50.084 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: kXpressCompression supported: 0 2024-03-24T10:48:50.084 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: kLZ4HCCompression supported: 1 2024-03-24T10:48:50.084 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: kZlibCompression supported: 1 2024-03-24T10:48:50.084 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: kSnappyCompression supported: 1 2024-03-24T10:48:50.084 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: kLZ4Compression supported: 1 2024-03-24T10:48:50.085 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: kBZip2Compression supported: 0 2024-03-24T10:48:50.085 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-24T10:48:50.085 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-24T10:48:50.085 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi096/store.db/MANIFEST-000009 2024-03-24T10:48:50.085 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-24T10:48:50.085 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-24T10:48:50.085 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.merge_operator: 2024-03-24T10:48:50.085 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.compaction_filter: None 2024-03-24T10:48:50.085 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.compaction_filter_factory: None 2024-03-24T10:48:50.085 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.sst_partitioner_factory: None 2024-03-24T10:48:50.085 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-24T10:48:50.085 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-24T10:48:50.085 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x564ad3767a80) 2024-03-24T10:48:50.085 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: cache_index_and_filter_blocks: 1 2024-03-24T10:48:50.085 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-24T10:48:50.086 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-24T10:48:50.086 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: pin_top_level_index_and_filter: 1 2024-03-24T10:48:50.086 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: index_type: 0 2024-03-24T10:48:50.086 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: data_block_index_type: 0 2024-03-24T10:48:50.086 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: index_shortening: 1 2024-03-24T10:48:50.086 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-24T10:48:50.086 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: checksum: 4 2024-03-24T10:48:50.086 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: no_block_cache: 0 2024-03-24T10:48:50.086 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: block_cache: 0x564ad376b090 2024-03-24T10:48:50.086 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: block_cache_name: BinnedLRUCache 2024-03-24T10:48:50.086 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: block_cache_options: 2024-03-24T10:48:50.086 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: capacity : 536870912 2024-03-24T10:48:50.087 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: num_shard_bits : 4 2024-03-24T10:48:50.087 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: strict_capacity_limit : 0 2024-03-24T10:48:50.087 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: high_pri_pool_ratio: 0.000 2024-03-24T10:48:50.087 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: block_cache_compressed: (nil) 2024-03-24T10:48:50.087 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: persistent_cache: (nil) 2024-03-24T10:48:50.087 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: block_size: 4096 2024-03-24T10:48:50.087 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: block_size_deviation: 10 2024-03-24T10:48:50.087 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: block_restart_interval: 16 2024-03-24T10:48:50.087 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: index_block_restart_interval: 1 2024-03-24T10:48:50.087 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: metadata_block_size: 4096 2024-03-24T10:48:50.087 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: partition_filters: 0 2024-03-24T10:48:50.087 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: use_delta_encoding: 1 2024-03-24T10:48:50.087 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: filter_policy: bloomfilter 2024-03-24T10:48:50.087 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: whole_key_filtering: 1 2024-03-24T10:48:50.087 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: verify_compression: 0 2024-03-24T10:48:50.088 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: read_amp_bytes_per_bit: 0 2024-03-24T10:48:50.088 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: format_version: 5 2024-03-24T10:48:50.088 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: enable_index_compression: 1 2024-03-24T10:48:50.088 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: block_align: 0 2024-03-24T10:48:50.088 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: max_auto_readahead_size: 262144 2024-03-24T10:48:50.088 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: prepopulate_block_cache: 0 2024-03-24T10:48:50.088 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: initial_auto_readahead_size: 8192 2024-03-24T10:48:50.088 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: num_file_reads_for_auto_readahead: 2 2024-03-24T10:48:50.088 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.write_buffer_size: 33554432 2024-03-24T10:48:50.088 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.max_write_buffer_number: 2 2024-03-24T10:48:50.088 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.compression: NoCompression 2024-03-24T10:48:50.088 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.bottommost_compression: Disabled 2024-03-24T10:48:50.088 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.prefix_extractor: nullptr 2024-03-24T10:48:50.088 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-24T10:48:50.089 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.num_levels: 7 2024-03-24T10:48:50.089 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-24T10:48:50.089 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-24T10:48:50.089 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-24T10:48:50.089 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-24T10:48:50.089 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-24T10:48:50.089 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-24T10:48:50.089 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-24T10:48:50.089 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-24T10:48:50.089 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-24T10:48:50.089 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-24T10:48:50.089 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-24T10:48:50.089 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-24T10:48:50.089 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-24T10:48:50.090 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.compression_opts.level: 32767 2024-03-24T10:48:50.090 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.compression_opts.strategy: 0 2024-03-24T10:48:50.090 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-24T10:48:50.090 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-24T10:48:50.090 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-24T10:48:50.090 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-24T10:48:50.090 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.compression_opts.enabled: false 2024-03-24T10:48:50.090 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-24T10:48:50.090 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-24T10:48:50.090 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-24T10:48:50.090 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-24T10:48:50.090 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.target_file_size_base: 67108864 2024-03-24T10:48:50.090 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-24T10:48:50.090 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-24T10:48:50.090 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-24T10:48:50.091 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-24T10:48:50.091 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-24T10:48:50.091 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-24T10:48:50.091 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-24T10:48:50.091 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-24T10:48:50.091 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-24T10:48:50.091 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-24T10:48:50.091 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-24T10:48:50.091 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-24T10:48:50.091 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-24T10:48:50.091 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-24T10:48:50.091 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.arena_block_size: 1048576 2024-03-24T10:48:50.091 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-24T10:48:50.092 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-24T10:48:50.092 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.disable_auto_compactions: 0 2024-03-24T10:48:50.092 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-24T10:48:50.092 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-24T10:48:50.092 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-24T10:48:50.092 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-24T10:48:50.092 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-24T10:48:50.092 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-24T10:48:50.092 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-24T10:48:50.092 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-24T10:48:50.092 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-24T10:48:50.092 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-24T10:48:50.092 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.table_properties_collectors: 2024-03-24T10:48:50.092 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.inplace_update_support: 0 2024-03-24T10:48:50.093 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-24T10:48:50.093 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-24T10:48:50.093 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-24T10:48:50.093 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-24T10:48:50.093 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.bloom_locality: 0 2024-03-24T10:48:50.093 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.max_successive_merges: 0 2024-03-24T10:48:50.093 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-24T10:48:50.093 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.paranoid_file_checks: 0 2024-03-24T10:48:50.093 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.force_consistency_checks: 1 2024-03-24T10:48:50.093 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.report_bg_io_stats: 0 2024-03-24T10:48:50.093 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.ttl: 2592000 2024-03-24T10:48:50.093 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-24T10:48:50.093 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-24T10:48:50.093 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-24T10:48:50.094 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.enable_blob_files: false 2024-03-24T10:48:50.094 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.min_blob_size: 0 2024-03-24T10:48:50.094 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.blob_file_size: 268435456 2024-03-24T10:48:50.094 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-24T10:48:50.094 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-24T10:48:50.094 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-24T10:48:50.094 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-24T10:48:50.094 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-24T10:48:50.094 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.blob_file_starting_level: 0 2024-03-24T10:48:50.094 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-24T10:48:50.094 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 27.sst 2024-03-24T10:48:50.094 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-24T10:48:50.094 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi096/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 29, last_sequence is 12112, log_number is 25,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-24T10:48:50.094 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 25 2024-03-24T10:48:50.095 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: a175449e-27d4-4789-b758-06edc3bb0eea 2024-03-24T10:48:50.095 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711277329743998, "job": 1, "event": "recovery_started", "wal_files": [25]} 2024-03-24T10:48:50.095 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #25 mode 2 2024-03-24T10:48:50.095 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711277329745824, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 30, "file_size": 187907, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12117, "largest_seqno": 12251, "table_properties": {"data_size": 186223, "index_size": 480, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 197, "raw_key_size": 1892, "raw_average_key_size": 26, "raw_value_size": 184487, "raw_average_value_size": 2562, "num_data_blocks": 20, "num_entries": 72, "num_filter_entries": 72, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711277329, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a175449e-27d4-4789-b758-06edc3bb0eea", "db_session_id": "R1L5NPBXI7LCMZSCAT63", "orig_file_number": 30, "seqno_to_time_mapping": "N/A"}} 2024-03-24T10:48:50.095 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711277329745929, "job": 1, "event": "recovery_finished"} 2024-03-24T10:48:50.095 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: [db/version_set.cc:5047] Creating manifest 32 2024-03-24T10:48:50.095 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-24T10:48:50.095 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi096/store.db/000025.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-24T10:48:50.095 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x564ad385c000 2024-03-24T10:48:50.095 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: DB pointer 0x564ad3846000 2024-03-24T10:48:50.095 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-24T10:48:50.095 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-24T10:48:50.095 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: ** DB Stats ** 2024-03-24T10:48:50.095 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-24T10:48:50.096 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-24T10:48:50.096 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-24T10:48:50.096 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-24T10:48:50.096 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-24T10:48:50.096 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-24T10:48:50.096 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-24T10:48:50.096 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: 2024-03-24T10:48:50.096 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: ** Compaction Stats [default] ** 2024-03-24T10:48:50.096 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-24T10:48:50.096 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-24T10:48:50.096 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: L0 1/0 183.50 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 154.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-24T10:48:50.096 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: L6 1/0 7.69 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-03-24T10:48:50.096 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: Sum 2/0 7.87 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 154.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-24T10:48:50.096 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 154.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-24T10:48:50.096 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: 2024-03-24T10:48:50.097 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: ** Compaction Stats [default] ** 2024-03-24T10:48:50.097 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-24T10:48:50.097 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-24T10:48:50.097 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 154.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-24T10:48:50.097 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: 2024-03-24T10:48:50.097 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-24T10:48:50.097 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: 2024-03-24T10:48:50.097 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-24T10:48:50.097 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-24T10:48:50.097 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-24T10:48:50.097 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-24T10:48:50.097 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-24T10:48:50.097 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-24T10:48:50.098 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: Cumulative compaction: 0.00 GB write, 41.42 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-24T10:48:50.098 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: Interval compaction: 0.00 GB write, 41.42 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-24T10:48:50.098 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-24T10:48:50.098 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: Block cache BinnedLRUCache@0x564ad376b090#2 capacity: 512.00 MB usage: 0.78 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-03-24T10:48:50.098 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.23 KB,4.47035e-05%) IndexBlock(1,0.55 KB,0.000104308%) Misc(1,0.00 KB,0%) 2024-03-24T10:48:50.098 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: 2024-03-24T10:48:50.098 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-24T10:48:50.098 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: starting mon.smithi096 rank 1 at public addrs [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] at bind addrs [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi096 fsid 386110d4-e9ca-11ee-95cd-87774f69a715 2024-03-24T10:48:50.098 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: mon.smithi096@-1(???) e2 preinit fsid 386110d4-e9ca-11ee-95cd-87774f69a715 2024-03-24T10:48:50.098 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: mon.smithi096@-1(???).mds e18 new map 2024-03-24T10:48:50.098 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: mon.smithi096@-1(???).mds e18 print_map 2024-03-24T10:48:50.098 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: e18 2024-03-24T10:48:50.098 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T10:48:50.098 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:48:50.099 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: legacy client fscid: 1 2024-03-24T10:48:50.099 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: 2024-03-24T10:48:50.099 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: Filesystem 'cephfs' (1) 2024-03-24T10:48:50.099 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: fs_name cephfs 2024-03-24T10:48:50.099 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: epoch 18 2024-03-24T10:48:50.099 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-24T10:48:50.099 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: created 2024-03-24T10:43:39.435738+0000 2024-03-24T10:48:50.099 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: modified 2024-03-24T10:43:58.133901+0000 2024-03-24T10:48:50.099 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: tableserver 0 2024-03-24T10:48:50.099 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: root 0 2024-03-24T10:48:50.099 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: session_timeout 60 2024-03-24T10:48:50.099 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: session_autoclose 300 2024-03-24T10:48:50.099 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: max_file_size 1099511627776 2024-03-24T10:48:50.099 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: max_xattr_size 65536 2024-03-24T10:48:50.099 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: required_client_features {} 2024-03-24T10:48:50.100 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: last_failure 0 2024-03-24T10:48:50.100 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: last_failure_osd_epoch 46 2024-03-24T10:48:50.100 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:48:50.100 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: max_mds 1 2024-03-24T10:48:50.100 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: in 0 2024-03-24T10:48:50.100 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: up {0=24251} 2024-03-24T10:48:50.100 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: failed 2024-03-24T10:48:50.100 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: damaged 2024-03-24T10:48:50.100 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: stopped 2024-03-24T10:48:50.100 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: data_pools [3] 2024-03-24T10:48:50.100 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: metadata_pool 2 2024-03-24T10:48:50.100 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: inline_data disabled 2024-03-24T10:48:50.100 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: balancer 2024-03-24T10:48:50.100 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: bal_rank_mask -1 2024-03-24T10:48:50.101 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: standby_count_wanted 1 2024-03-24T10:48:50.101 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: qdb_cluster leader: 0 members: 2024-03-24T10:48:50.101 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: [mds.cephfs.smithi096.egbmoq{0:24251} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.96:6824/1539752208,v1:172.21.15.96:6825/1539752208] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:48:50.101 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: [mds.cephfs.smithi094.llefgu{0:14500} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.94:6828/126762980,v1:172.21.15.94:6829/126762980] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:48:50.101 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: 2024-03-24T10:48:50.101 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: 2024-03-24T10:48:50.101 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: Standby daemons: 2024-03-24T10:48:50.101 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: 2024-03-24T10:48:50.101 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: [mds.cephfs.smithi094.efiqte{-1:14512} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.94:6826/1636528860,v1:172.21.15.94:6827/1636528860] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:48:50.101 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout: [mds.cephfs.smithi096.jtebhh{-1:24261} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.96:6826/1321087930,v1:172.21.15.96:6827/1321087930] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:48:50.101 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: mon.smithi096@-1(???).osd e49 crush map has features 3314933000852226048, adjusting msgr requires 2024-03-24T10:48:50.101 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: mon.smithi096@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-03-24T10:48:50.101 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: mon.smithi096@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-03-24T10:48:50.101 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: mon.smithi096@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-03-24T10:48:50.101 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 ceph-mon[84258]: mon.smithi096@-1(???).paxosservice(auth 1..22) refresh upgraded, format 0 -> 3 2024-03-24T10:48:50.102 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 bash[84231]: ed9bd0bd57f6827b4d30ceb525471127356c7d2b1879c6048709be361c5a8f51 2024-03-24T10:48:50.102 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:49 smithi096 systemd[1]: Started Ceph mon.smithi096 for 386110d4-e9ca-11ee-95cd-87774f69a715. 2024-03-24T10:48:52.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:52 smithi096 ceph-mon[84258]: election_strategy: 1 2024-03-24T10:48:52.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:52 smithi096 ceph-mon[84258]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.smithi094 2024-03-24T10:48:52.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:52 smithi096 ceph-mon[84258]: 1: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.smithi096 2024-03-24T10:48:52.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:52 smithi096 ceph-mon[84258]: fsmap cephfs:1 {0=cephfs.smithi096.egbmoq=up:active} 1 up:standby-replay 2 up:standby 2024-03-24T10:48:52.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:52 smithi096 ceph-mon[84258]: osdmap e49: 6 total, 6 up, 6 in 2024-03-24T10:48:52.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:52 smithi096 ceph-mon[84258]: mgrmap e35: smithi094.yaethu(active, since 15s), standbys: smithi096.gfonnu 2024-03-24T10:48:52.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:52 smithi096 ceph-mon[84258]: overall HEALTH_OK 2024-03-24T10:48:52.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:52 smithi096 ceph-mon[84258]: mon.smithi096 calling monitor election 2024-03-24T10:48:52.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:52 smithi096 ceph-mon[84258]: mon.smithi094 calling monitor election 2024-03-24T10:48:52.327 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:52 smithi096 ceph-mon[84258]: mon.smithi094 is new leader, mons smithi094,smithi096 in quorum (ranks 0,1) 2024-03-24T10:48:52.327 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:52 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata", "id": "smithi094"}]: dispatch 2024-03-24T10:48:52.327 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:52 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata", "id": "smithi096"}]: dispatch 2024-03-24T10:48:52.327 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:52 smithi096 ceph-mon[84258]: monmap epoch 3 2024-03-24T10:48:52.327 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:52 smithi096 ceph-mon[84258]: fsid 386110d4-e9ca-11ee-95cd-87774f69a715 2024-03-24T10:48:52.327 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:52 smithi096 ceph-mon[84258]: last_changed 2024-03-24T10:48:51.123053+0000 2024-03-24T10:48:52.327 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:52 smithi096 ceph-mon[84258]: created 2024-03-24T10:36:06.632144+0000 2024-03-24T10:48:52.327 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:52 smithi096 ceph-mon[84258]: min_mon_release 19 (squid) 2024-03-24T10:48:52.327 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:52 smithi096 ceph-mon[84258]: election_strategy: 1 2024-03-24T10:48:52.327 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:52 smithi096 ceph-mon[84258]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.smithi094 2024-03-24T10:48:52.327 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:52 smithi096 ceph-mon[84258]: 1: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.smithi096 2024-03-24T10:48:52.327 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:52 smithi096 ceph-mon[84258]: fsmap cephfs:1 {0=cephfs.smithi096.egbmoq=up:active} 1 up:standby-replay 2 up:standby 2024-03-24T10:48:52.327 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:52 smithi096 ceph-mon[84258]: osdmap e49: 6 total, 6 up, 6 in 2024-03-24T10:48:52.327 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:52 smithi096 ceph-mon[84258]: mgrmap e35: smithi094.yaethu(active, since 15s), standbys: smithi096.gfonnu 2024-03-24T10:48:52.327 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:52 smithi096 ceph-mon[84258]: overall HEALTH_OK 2024-03-24T10:48:52.327 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:52 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:52.328 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:52 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:52.328 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:52 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:48:52.328 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:52 smithi096 ceph-mon[84258]: pgmap v11: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 18 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-24T10:48:52.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:52 smithi094 ceph-mon[93957]: election_strategy: 1 2024-03-24T10:48:52.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:52 smithi094 ceph-mon[93957]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.smithi094 2024-03-24T10:48:52.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:52 smithi094 ceph-mon[93957]: 1: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.smithi096 2024-03-24T10:48:52.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:52 smithi094 ceph-mon[93957]: fsmap cephfs:1 {0=cephfs.smithi096.egbmoq=up:active} 1 up:standby-replay 2 up:standby 2024-03-24T10:48:52.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:52 smithi094 ceph-mon[93957]: osdmap e49: 6 total, 6 up, 6 in 2024-03-24T10:48:52.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:52 smithi094 ceph-mon[93957]: mgrmap e35: smithi094.yaethu(active, since 15s), standbys: smithi096.gfonnu 2024-03-24T10:48:52.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:52 smithi094 ceph-mon[93957]: overall HEALTH_OK 2024-03-24T10:48:52.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:52 smithi094 ceph-mon[93957]: mon.smithi096 calling monitor election 2024-03-24T10:48:52.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:52 smithi094 ceph-mon[93957]: mon.smithi094 calling monitor election 2024-03-24T10:48:52.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:52 smithi094 ceph-mon[93957]: mon.smithi094 is new leader, mons smithi094,smithi096 in quorum (ranks 0,1) 2024-03-24T10:48:52.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:52 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata", "id": "smithi094"}]: dispatch 2024-03-24T10:48:52.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:52 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mon metadata", "id": "smithi096"}]: dispatch 2024-03-24T10:48:52.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:52 smithi094 ceph-mon[93957]: monmap epoch 3 2024-03-24T10:48:52.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:52 smithi094 ceph-mon[93957]: fsid 386110d4-e9ca-11ee-95cd-87774f69a715 2024-03-24T10:48:52.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:52 smithi094 ceph-mon[93957]: last_changed 2024-03-24T10:48:51.123053+0000 2024-03-24T10:48:52.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:52 smithi094 ceph-mon[93957]: created 2024-03-24T10:36:06.632144+0000 2024-03-24T10:48:52.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:52 smithi094 ceph-mon[93957]: min_mon_release 19 (squid) 2024-03-24T10:48:52.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:52 smithi094 ceph-mon[93957]: election_strategy: 1 2024-03-24T10:48:52.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:52 smithi094 ceph-mon[93957]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.smithi094 2024-03-24T10:48:52.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:52 smithi094 ceph-mon[93957]: 1: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.smithi096 2024-03-24T10:48:52.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:52 smithi094 ceph-mon[93957]: fsmap cephfs:1 {0=cephfs.smithi096.egbmoq=up:active} 1 up:standby-replay 2 up:standby 2024-03-24T10:48:52.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:52 smithi094 ceph-mon[93957]: osdmap e49: 6 total, 6 up, 6 in 2024-03-24T10:48:52.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:52 smithi094 ceph-mon[93957]: mgrmap e35: smithi094.yaethu(active, since 15s), standbys: smithi096.gfonnu 2024-03-24T10:48:52.500 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:52 smithi094 ceph-mon[93957]: overall HEALTH_OK 2024-03-24T10:48:52.500 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:52 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:52.500 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:52 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:52.500 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:52 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:48:52.500 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:52 smithi094 ceph-mon[93957]: pgmap v11: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 18 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-24T10:48:54.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:54 smithi096 ceph-mon[84258]: pgmap v12: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 18 KiB/s rd, 0 B/s wr, 12 op/s 2024-03-24T10:48:54.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:54 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:54.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:54 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:54.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:54 smithi094 ceph-mon[93957]: pgmap v12: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 18 KiB/s rd, 0 B/s wr, 12 op/s 2024-03-24T10:48:54.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:54 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:54.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:54 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:56.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:56 smithi096 ceph-mon[84258]: pgmap v13: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:48:56.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:56 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:56.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:56 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:56.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:56 smithi094 ceph-mon[93957]: pgmap v13: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:48:56.760 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:56 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:56.760 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:56 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:56.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:56.978+0000 7f797c06a700 1 -- 172.21.15.94:0/2331331893 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79740fbcd0 msgr2=0x7f79740fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:56.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:56.978+0000 7f797c06a700 1 --2- 172.21.15.94:0/2331331893 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79740fbcd0 0x7f79740fc0e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7970004940 tx=0x7f7970009e30 comp rx=0 tx=0).stop 2024-03-24T10:48:56.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:56.979+0000 7f797c06a700 1 -- 172.21.15.94:0/2331331893 shutdown_connections 2024-03-24T10:48:56.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:56.979+0000 7f797c06a700 1 --2- 172.21.15.94:0/2331331893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79740fcf80 0x7f79740fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:56.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:56.979+0000 7f797c06a700 1 --2- 172.21.15.94:0/2331331893 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79740fbcd0 0x7f79740fc0e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:56.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:56.979+0000 7f797c06a700 1 -- 172.21.15.94:0/2331331893 >> 172.21.15.94:0/2331331893 conn(0x7f79740f7450 msgr2=0x7f79740f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:48:56.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:56.979+0000 7f797c06a700 1 -- 172.21.15.94:0/2331331893 shutdown_connections 2024-03-24T10:48:56.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:56.979+0000 7f797c06a700 1 -- 172.21.15.94:0/2331331893 wait complete. 2024-03-24T10:48:56.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:56.979+0000 7f797c06a700 1 Processor -- start 2024-03-24T10:48:56.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:56.979+0000 7f797c06a700 1 -- start start 2024-03-24T10:48:56.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:56.979+0000 7f797c06a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79740fbcd0 0x7f7974066cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:48:56.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:56.979+0000 7f797c06a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79740fcf80 0x7f7974112730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:48:56.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:56.979+0000 7f797c06a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7974112c70 con 0x7f79740fcf80 2024-03-24T10:48:56.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:56.979+0000 7f797c06a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7974112db0 con 0x7f79740fbcd0 2024-03-24T10:48:56.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:56.980+0000 7f7979605700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79740fcf80 0x7f7974112730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:48:56.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:56.980+0000 7f7979605700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79740fcf80 0x7f7974112730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45100/0 (socket says 172.21.15.94:45100) 2024-03-24T10:48:56.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:56.980+0000 7f7979605700 1 -- 172.21.15.94:0/1379996805 learned_addr learned my addr 172.21.15.94:0/1379996805 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:48:56.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:56.980+0000 7f7979e06700 1 --2- 172.21.15.94:0/1379996805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79740fbcd0 0x7f7974066cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:48:56.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:56.980+0000 7f7979605700 1 -- 172.21.15.94:0/1379996805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79740fbcd0 msgr2=0x7f7974066cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:56.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:56.980+0000 7f7979605700 1 --2- 172.21.15.94:0/1379996805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79740fbcd0 0x7f7974066cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:56.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:56.980+0000 7f7979605700 1 -- 172.21.15.94:0/1379996805 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7970005040 con 0x7f79740fcf80 2024-03-24T10:48:56.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:56.980+0000 7f7979e06700 1 --2- 172.21.15.94:0/1379996805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79740fbcd0 0x7f7974066cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T10:48:56.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:56.980+0000 7f7979605700 1 --2- 172.21.15.94:0/1379996805 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79740fcf80 0x7f7974112730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7964002a50 tx=0x7f79640046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:48:56.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:56.980+0000 7f796a7fc700 1 -- 172.21.15.94:0/1379996805 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7964007cc0 con 0x7f79740fcf80 2024-03-24T10:48:56.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:56.980+0000 7f797c06a700 1 -- 172.21.15.94:0/1379996805 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7974113010 con 0x7f79740fcf80 2024-03-24T10:48:56.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:56.980+0000 7f796a7fc700 1 -- 172.21.15.94:0/1379996805 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f796400f040 con 0x7f79740fcf80 2024-03-24T10:48:56.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:56.980+0000 7f796a7fc700 1 -- 172.21.15.94:0/1379996805 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7964009640 con 0x7f79740fcf80 2024-03-24T10:48:56.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:56.980+0000 7f797c06a700 1 -- 172.21.15.94:0/1379996805 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79741135a0 con 0x7f79740fcf80 2024-03-24T10:48:56.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:56.981+0000 7f797c06a700 1 -- 172.21.15.94:0/1379996805 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f797404c710 con 0x7f79740fcf80 2024-03-24T10:48:56.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:56.982+0000 7f796a7fc700 1 -- 172.21.15.94:0/1379996805 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 35) v1 ==== 91587+0+0 (secure 0 0 0) 0x7f79640097a0 con 0x7f79740fcf80 2024-03-24T10:48:56.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:56.982+0000 7f796a7fc700 1 --2- 172.21.15.94:0/1379996805 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f796006d960 0x7f796006fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:48:56.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:56.982+0000 7f7979e06700 1 --2- 172.21.15.94:0/1379996805 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f796006d960 0x7f796006fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:48:56.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:56.982+0000 7f796a7fc700 1 -- 172.21.15.94:0/1379996805 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f796408ced0 con 0x7f79740fcf80 2024-03-24T10:48:56.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:56.983+0000 7f7979e06700 1 --2- 172.21.15.94:0/1379996805 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f796006d960 0x7f796006fe10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f797000b3b0 tx=0x7f7970009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:48:56.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:56.987+0000 7f796a7fc700 1 -- 172.21.15.94:0/1379996805 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7964059fd0 con 0x7f79740fcf80 2024-03-24T10:48:57.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.200+0000 7f797c06a700 1 -- 172.21.15.94:0/1379996805 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7974101290 con 0x7f796006d960 2024-03-24T10:48:57.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.201+0000 7f796a7fc700 1 -- 172.21.15.94:0/1379996805 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f7974101290 con 0x7f796006d960 2024-03-24T10:48:57.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.203+0000 7f797c06a700 1 -- 172.21.15.94:0/1379996805 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f796006d960 msgr2=0x7f796006fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:57.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.203+0000 7f797c06a700 1 --2- 172.21.15.94:0/1379996805 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f796006d960 0x7f796006fe10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f797000b3b0 tx=0x7f7970009e30 comp rx=0 tx=0).stop 2024-03-24T10:48:57.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.203+0000 7f797c06a700 1 -- 172.21.15.94:0/1379996805 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79740fcf80 msgr2=0x7f7974112730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:57.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.203+0000 7f797c06a700 1 --2- 172.21.15.94:0/1379996805 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79740fcf80 0x7f7974112730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7964002a50 tx=0x7f79640046b0 comp rx=0 tx=0).stop 2024-03-24T10:48:57.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.203+0000 7f797c06a700 1 -- 172.21.15.94:0/1379996805 shutdown_connections 2024-03-24T10:48:57.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.203+0000 7f797c06a700 1 --2- 172.21.15.94:0/1379996805 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f796006d960 0x7f796006fe10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:57.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.203+0000 7f797c06a700 1 --2- 172.21.15.94:0/1379996805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79740fbcd0 0x7f7974066cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:57.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.203+0000 7f797c06a700 1 --2- 172.21.15.94:0/1379996805 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79740fcf80 0x7f7974112730 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:57.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.203+0000 7f797c06a700 1 -- 172.21.15.94:0/1379996805 >> 172.21.15.94:0/1379996805 conn(0x7f79740f7450 msgr2=0x7f79740f8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:48:57.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.204+0000 7f797c06a700 1 -- 172.21.15.94:0/1379996805 shutdown_connections 2024-03-24T10:48:57.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.204+0000 7f797c06a700 1 -- 172.21.15.94:0/1379996805 wait complete. 2024-03-24T10:48:57.213 INFO:teuthology.orchestra.run.smithi094.stdout:true 2024-03-24T10:48:57.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.304+0000 7f96d35a9700 1 -- 172.21.15.94:0/559897249 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96cc0fc720 msgr2=0x7f96cc0fcb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:57.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.304+0000 7f96d35a9700 1 --2- 172.21.15.94:0/559897249 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96cc0fc720 0x7f96cc0fcb90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f96c8004a50 tx=0x7f96c800af40 comp rx=0 tx=0).stop 2024-03-24T10:48:57.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.304+0000 7f96d35a9700 1 -- 172.21.15.94:0/559897249 shutdown_connections 2024-03-24T10:48:57.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.304+0000 7f96d35a9700 1 --2- 172.21.15.94:0/559897249 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96cc0fc720 0x7f96cc0fcb90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:57.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.304+0000 7f96d35a9700 1 --2- 172.21.15.94:0/559897249 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96cc0fb470 0x7f96cc0fb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:57.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.304+0000 7f96d35a9700 1 -- 172.21.15.94:0/559897249 >> 172.21.15.94:0/559897249 conn(0x7f96cc0f6bd0 msgr2=0x7f96cc0f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:48:57.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.304+0000 7f96d35a9700 1 -- 172.21.15.94:0/559897249 shutdown_connections 2024-03-24T10:48:57.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.304+0000 7f96d35a9700 1 -- 172.21.15.94:0/559897249 wait complete. 2024-03-24T10:48:57.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.305+0000 7f96d35a9700 1 Processor -- start 2024-03-24T10:48:57.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.305+0000 7f96d35a9700 1 -- start start 2024-03-24T10:48:57.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.305+0000 7f96d35a9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96cc0fb470 0x7f96cc10b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:48:57.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.305+0000 7f96d35a9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96cc0fc720 0x7f96cc10bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:48:57.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.305+0000 7f96d35a9700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96cc10c350 con 0x7f96cc0fb470 2024-03-24T10:48:57.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.305+0000 7f96d35a9700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96cc110c30 con 0x7f96cc0fc720 2024-03-24T10:48:57.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.305+0000 7f96d1345700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96cc0fb470 0x7f96cc10b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:48:57.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.305+0000 7f96d1345700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96cc0fb470 0x7f96cc10b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45118/0 (socket says 172.21.15.94:45118) 2024-03-24T10:48:57.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.305+0000 7f96d1345700 1 -- 172.21.15.94:0/1114262589 learned_addr learned my addr 172.21.15.94:0/1114262589 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:48:57.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.305+0000 7f96d0b44700 1 --2- 172.21.15.94:0/1114262589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96cc0fc720 0x7f96cc10bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:48:57.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.305+0000 7f96d1345700 1 -- 172.21.15.94:0/1114262589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96cc0fc720 msgr2=0x7f96cc10bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:57.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.305+0000 7f96d1345700 1 --2- 172.21.15.94:0/1114262589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96cc0fc720 0x7f96cc10bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:57.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.305+0000 7f96d1345700 1 -- 172.21.15.94:0/1114262589 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c8005040 con 0x7f96cc0fb470 2024-03-24T10:48:57.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.305+0000 7f96d1345700 1 --2- 172.21.15.94:0/1114262589 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96cc0fb470 0x7f96cc10b780 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f96bc00b790 tx=0x7f96bc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:48:57.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.306+0000 7f96c1ffb700 1 -- 172.21.15.94:0/1114262589 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96bc00f770 con 0x7f96cc0fb470 2024-03-24T10:48:57.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.306+0000 7f96d35a9700 1 -- 172.21.15.94:0/1114262589 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96cc110e90 con 0x7f96cc0fb470 2024-03-24T10:48:57.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.306+0000 7f96c1ffb700 1 -- 172.21.15.94:0/1114262589 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f96bc00fda0 con 0x7f96cc0fb470 2024-03-24T10:48:57.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.306+0000 7f96c1ffb700 1 -- 172.21.15.94:0/1114262589 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96bc00d310 con 0x7f96cc0fb470 2024-03-24T10:48:57.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.306+0000 7f96d35a9700 1 -- 172.21.15.94:0/1114262589 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96cc111370 con 0x7f96cc0fb470 2024-03-24T10:48:57.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.307+0000 7f96d35a9700 1 -- 172.21.15.94:0/1114262589 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96cc04c710 con 0x7f96cc0fb470 2024-03-24T10:48:57.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.309+0000 7f96c1ffb700 1 -- 172.21.15.94:0/1114262589 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 35) v1 ==== 91587+0+0 (secure 0 0 0) 0x7f96bc010420 con 0x7f96cc0fb470 2024-03-24T10:48:57.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.310+0000 7f96c1ffb700 1 --2- 172.21.15.94:0/1114262589 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f96b806fb20 0x7f96b8071fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:48:57.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.310+0000 7f96c1ffb700 1 -- 172.21.15.94:0/1114262589 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f96bc08d660 con 0x7f96cc0fb470 2024-03-24T10:48:57.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.310+0000 7f96d0b44700 1 --2- 172.21.15.94:0/1114262589 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f96b806fb20 0x7f96b8071fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:48:57.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.310+0000 7f96d0b44700 1 --2- 172.21.15.94:0/1114262589 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f96b806fb20 0x7f96b8071fd0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f96c800c3b0 tx=0x7f96c800af40 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:48:57.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.313+0000 7f96c1ffb700 1 -- 172.21.15.94:0/1114262589 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f96bc05a990 con 0x7f96cc0fb470 2024-03-24T10:48:57.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.527+0000 7f96d35a9700 1 -- 172.21.15.94:0/1114262589 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f96cc100960 con 0x7f96b806fb20 2024-03-24T10:48:57.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.528+0000 7f96c1ffb700 1 -- 172.21.15.94:0/1114262589 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f96cc100960 con 0x7f96b806fb20 2024-03-24T10:48:57.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.530+0000 7f96d35a9700 1 -- 172.21.15.94:0/1114262589 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f96b806fb20 msgr2=0x7f96b8071fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:57.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.530+0000 7f96d35a9700 1 --2- 172.21.15.94:0/1114262589 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f96b806fb20 0x7f96b8071fd0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f96c800c3b0 tx=0x7f96c800af40 comp rx=0 tx=0).stop 2024-03-24T10:48:57.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.530+0000 7f96d35a9700 1 -- 172.21.15.94:0/1114262589 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96cc0fb470 msgr2=0x7f96cc10b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:57.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.530+0000 7f96d35a9700 1 --2- 172.21.15.94:0/1114262589 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96cc0fb470 0x7f96cc10b780 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f96bc00b790 tx=0x7f96bc009250 comp rx=0 tx=0).stop 2024-03-24T10:48:57.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.530+0000 7f96d35a9700 1 -- 172.21.15.94:0/1114262589 shutdown_connections 2024-03-24T10:48:57.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.530+0000 7f96d35a9700 1 --2- 172.21.15.94:0/1114262589 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f96b806fb20 0x7f96b8071fd0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:57.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.530+0000 7f96d35a9700 1 --2- 172.21.15.94:0/1114262589 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96cc0fb470 0x7f96cc10b780 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:57.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.530+0000 7f96d35a9700 1 --2- 172.21.15.94:0/1114262589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96cc0fc720 0x7f96cc10bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:57.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.530+0000 7f96d35a9700 1 -- 172.21.15.94:0/1114262589 >> 172.21.15.94:0/1114262589 conn(0x7f96cc0f6bd0 msgr2=0x7f96cc0f8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:48:57.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.530+0000 7f96d35a9700 1 -- 172.21.15.94:0/1114262589 shutdown_connections 2024-03-24T10:48:57.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.530+0000 7f96d35a9700 1 -- 172.21.15.94:0/1114262589 wait complete. 2024-03-24T10:48:57.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.631+0000 7f146d298700 1 -- 172.21.15.94:0/1263423523 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14680fdff0 msgr2=0x7f14680fe460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:57.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.631+0000 7f146d298700 1 --2- 172.21.15.94:0/1263423523 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14680fdff0 0x7f14680fe460 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f145c004940 tx=0x7f145c009e30 comp rx=0 tx=0).stop 2024-03-24T10:48:57.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.631+0000 7f146d298700 1 -- 172.21.15.94:0/1263423523 shutdown_connections 2024-03-24T10:48:57.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.631+0000 7f146d298700 1 --2- 172.21.15.94:0/1263423523 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14680fdff0 0x7f14680fe460 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:57.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.631+0000 7f146d298700 1 --2- 172.21.15.94:0/1263423523 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14680fce80 0x7f14680fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:57.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.631+0000 7f146d298700 1 -- 172.21.15.94:0/1263423523 >> 172.21.15.94:0/1263423523 conn(0x7f14680f8580 msgr2=0x7f14680fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:48:57.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.631+0000 7f146d298700 1 -- 172.21.15.94:0/1263423523 shutdown_connections 2024-03-24T10:48:57.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.631+0000 7f146d298700 1 -- 172.21.15.94:0/1263423523 wait complete. 2024-03-24T10:48:57.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.631+0000 7f146d298700 1 Processor -- start 2024-03-24T10:48:57.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.632+0000 7f146d298700 1 -- start start 2024-03-24T10:48:57.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.632+0000 7f146d298700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14680fce80 0x7f146810d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:48:57.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.632+0000 7f146d298700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14680fdff0 0x7f146810ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:48:57.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.632+0000 7f146d298700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146810e4d0 con 0x7f14680fce80 2024-03-24T10:48:57.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.632+0000 7f146d298700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146810ed20 con 0x7f14680fdff0 2024-03-24T10:48:57.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.632+0000 7f1466ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14680fce80 0x7f146810d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:48:57.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.632+0000 7f1466ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14680fce80 0x7f146810d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45140/0 (socket says 172.21.15.94:45140) 2024-03-24T10:48:57.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.632+0000 7f1466ffd700 1 -- 172.21.15.94:0/1404429246 learned_addr learned my addr 172.21.15.94:0/1404429246 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:48:57.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.632+0000 7f14667fc700 1 --2- 172.21.15.94:0/1404429246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14680fdff0 0x7f146810ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:48:57.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.632+0000 7f1466ffd700 1 -- 172.21.15.94:0/1404429246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14680fdff0 msgr2=0x7f146810ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:57.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.632+0000 7f1466ffd700 1 --2- 172.21.15.94:0/1404429246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14680fdff0 0x7f146810ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:57.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.632+0000 7f1466ffd700 1 -- 172.21.15.94:0/1404429246 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f145c005040 con 0x7f14680fce80 2024-03-24T10:48:57.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.632+0000 7f1466ffd700 1 --2- 172.21.15.94:0/1404429246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14680fce80 0x7f146810d990 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1458002a80 tx=0x7f14580046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:48:57.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.632+0000 7f144f7fe700 1 -- 172.21.15.94:0/1404429246 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1458007cc0 con 0x7f14680fce80 2024-03-24T10:48:57.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.632+0000 7f144f7fe700 1 -- 172.21.15.94:0/1404429246 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f145800f040 con 0x7f14680fce80 2024-03-24T10:48:57.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.632+0000 7f144f7fe700 1 -- 172.21.15.94:0/1404429246 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f145800dac0 con 0x7f14680fce80 2024-03-24T10:48:57.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.632+0000 7f146d298700 1 -- 172.21.15.94:0/1404429246 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f146810f000 con 0x7f14680fce80 2024-03-24T10:48:57.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.632+0000 7f146d298700 1 -- 172.21.15.94:0/1404429246 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f146810f540 con 0x7f14680fce80 2024-03-24T10:48:57.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.633+0000 7f144d7fa700 1 -- 172.21.15.94:0/1404429246 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f146805f5f0 con 0x7f14680fce80 2024-03-24T10:48:57.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.634+0000 7f144f7fe700 1 -- 172.21.15.94:0/1404429246 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 35) v1 ==== 91587+0+0 (secure 0 0 0) 0x7f145800dc20 con 0x7f14680fce80 2024-03-24T10:48:57.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.634+0000 7f144f7fe700 1 --2- 172.21.15.94:0/1404429246 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f145006d900 0x7f145006fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:48:57.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.634+0000 7f14667fc700 1 --2- 172.21.15.94:0/1404429246 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f145006d900 0x7f145006fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:48:57.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.634+0000 7f144f7fe700 1 -- 172.21.15.94:0/1404429246 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f145808cd70 con 0x7f14680fce80 2024-03-24T10:48:57.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.635+0000 7f14667fc700 1 --2- 172.21.15.94:0/1404429246 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f145006d900 0x7f145006fdb0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f145c00b3b0 tx=0x7f145c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:48:57.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.639+0000 7f144f7fe700 1 -- 172.21.15.94:0/1404429246 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1458059ef0 con 0x7f14680fce80 2024-03-24T10:48:57.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.842+0000 7f144d7fa700 1 -- 172.21.15.94:0/1404429246 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f146804bed0 con 0x7f145006d900 2024-03-24T10:48:57.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.853+0000 7f144f7fe700 1 -- 172.21.15.94:0/1404429246 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f146804bed0 con 0x7f145006d900 2024-03-24T10:48:57.853 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T10:48:57.853 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.smithi094 smithi094 *:9093,9094 running (2m) 17s ago 11m 16.7M - 0.25.0 c8568f914cd2 5c583af60554 2024-03-24T10:48:57.853 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi094 smithi094 running (11m) 17s ago 11m 6660k - 17.2.7-726-g9c4e0c34 837124da804f fb5772af46c5 2024-03-24T10:48:57.853 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi096 smithi096 running (9m) 3s ago 9m 6669k - 17.2.7-726-g9c4e0c34 837124da804f 6087ad4e0734 2024-03-24T10:48:57.853 INFO:teuthology.orchestra.run.smithi094.stdout:grafana.smithi094 smithi094 *:3000 running (106s) 17s ago 10m 77.6M - 9.4.7 954c08fa6188 d3d425bf1b7c 2024-03-24T10:48:57.853 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.efiqte smithi094 running (5m) 17s ago 5m 12.9M - 17.2.7-726-g9c4e0c34 837124da804f f0df0a056c21 2024-03-24T10:48:57.853 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.llefgu smithi094 running (5m) 17s ago 5m 109M - 17.2.7-726-g9c4e0c34 837124da804f 52a9771dc8e0 2024-03-24T10:48:57.853 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi096.egbmoq smithi096 running (5m) 3s ago 5m 154M - 17.2.7-726-g9c4e0c34 837124da804f 1cfc930dbab0 2024-03-24T10:48:57.853 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi096.jtebhh smithi096 running (5m) 3s ago 5m 16.1M - 17.2.7-726-g9c4e0c34 837124da804f 4ff3451a6f6e 2024-03-24T10:48:57.854 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi094.yaethu smithi094 *:8443,9283,8765 running (84s) 17s ago 12m 473M - 19.0.0-1855-g21b3bd67 553463a90fd5 d3adab76b18f 2024-03-24T10:48:57.854 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi096.gfonnu smithi096 *:8443,9283,8765 running (45s) 3s ago 9m 448M - 19.0.0-1855-g21b3bd67 553463a90fd5 22937a059eb8 2024-03-24T10:48:57.854 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi094 smithi094 running (31s) 17s ago 12m 29.8M 2048M 19.0.0-1855-g21b3bd67 553463a90fd5 c779d611e29a 2024-03-24T10:48:57.854 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi096 smithi096 running (8s) 3s ago 9m 17.3M 2048M 19.0.0-1855-g21b3bd67 553463a90fd5 ed9bd0bd57f6 2024-03-24T10:48:57.854 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi094 smithi094 *:9100 running (2m) 17s ago 11m 8190k - 1.5.0 0da6a335fe13 ca67b53c64e2 2024-03-24T10:48:57.854 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi096 smithi096 *:9100 running (97s) 3s ago 9m 16.6M - 1.5.0 0da6a335fe13 65137a7a7549 2024-03-24T10:48:57.854 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (8m) 17s ago 8m 248M 1557M 17.2.7-726-g9c4e0c34 837124da804f b664fd3557bd 2024-03-24T10:48:57.854 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi094 running (8m) 17s ago 8m 259M 1557M 17.2.7-726-g9c4e0c34 837124da804f 4e737a80d733 2024-03-24T10:48:57.854 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi094 running (7m) 17s ago 7m 220M 1557M 17.2.7-726-g9c4e0c34 837124da804f cf0aefd20c76 2024-03-24T10:48:57.854 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi096 running (7m) 3s ago 7m 284M 2581M 17.2.7-726-g9c4e0c34 837124da804f f4650d0f6241 2024-03-24T10:48:57.854 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi096 running (7m) 3s ago 7m 257M 2581M 17.2.7-726-g9c4e0c34 837124da804f a63d7c9e705f 2024-03-24T10:48:57.854 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi096 running (6m) 3s ago 6m 242M 2581M 17.2.7-726-g9c4e0c34 837124da804f f56fad2959c3 2024-03-24T10:48:57.854 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.smithi094 smithi094 *:9095 running (50s) 17s ago 10m 41.2M - 2.43.0 a07b618ecd1d 6ce091a0c326 2024-03-24T10:48:57.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.856+0000 7f144d7fa700 1 -- 172.21.15.94:0/1404429246 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f145006d900 msgr2=0x7f145006fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:57.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.856+0000 7f144d7fa700 1 --2- 172.21.15.94:0/1404429246 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f145006d900 0x7f145006fdb0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f145c00b3b0 tx=0x7f145c009e30 comp rx=0 tx=0).stop 2024-03-24T10:48:57.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.856+0000 7f144d7fa700 1 -- 172.21.15.94:0/1404429246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14680fce80 msgr2=0x7f146810d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:57.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.856+0000 7f144d7fa700 1 --2- 172.21.15.94:0/1404429246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14680fce80 0x7f146810d990 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1458002a80 tx=0x7f14580046b0 comp rx=0 tx=0).stop 2024-03-24T10:48:57.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.856+0000 7f144d7fa700 1 -- 172.21.15.94:0/1404429246 shutdown_connections 2024-03-24T10:48:57.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.856+0000 7f144d7fa700 1 --2- 172.21.15.94:0/1404429246 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f145006d900 0x7f145006fdb0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:57.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.856+0000 7f144d7fa700 1 --2- 172.21.15.94:0/1404429246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14680fce80 0x7f146810d990 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:57.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.856+0000 7f144d7fa700 1 --2- 172.21.15.94:0/1404429246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14680fdff0 0x7f146810ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:57.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.856+0000 7f144d7fa700 1 -- 172.21.15.94:0/1404429246 >> 172.21.15.94:0/1404429246 conn(0x7f14680f8580 msgr2=0x7f14680f9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:48:57.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.856+0000 7f144d7fa700 1 -- 172.21.15.94:0/1404429246 shutdown_connections 2024-03-24T10:48:57.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.856+0000 7f144d7fa700 1 -- 172.21.15.94:0/1404429246 wait complete. 2024-03-24T10:48:57.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.955+0000 7f051f820700 1 -- 172.21.15.94:0/1159533121 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05180fcf80 msgr2=0x7f05180fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:57.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.955+0000 7f051f820700 1 --2- 172.21.15.94:0/1159533121 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05180fcf80 0x7f05180fd3f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0508004940 tx=0x7f0508009e30 comp rx=0 tx=0).stop 2024-03-24T10:48:57.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.955+0000 7f051f820700 1 -- 172.21.15.94:0/1159533121 shutdown_connections 2024-03-24T10:48:57.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.955+0000 7f051f820700 1 --2- 172.21.15.94:0/1159533121 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05180fcf80 0x7f05180fd3f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:57.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.955+0000 7f051f820700 1 --2- 172.21.15.94:0/1159533121 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05180fbcd0 0x7f05180fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:57.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.955+0000 7f051f820700 1 -- 172.21.15.94:0/1159533121 >> 172.21.15.94:0/1159533121 conn(0x7f05180f7430 msgr2=0x7f05180f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:48:57.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.955+0000 7f051f820700 1 -- 172.21.15.94:0/1159533121 shutdown_connections 2024-03-24T10:48:57.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.955+0000 7f051f820700 1 -- 172.21.15.94:0/1159533121 wait complete. 2024-03-24T10:48:57.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.955+0000 7f051f820700 1 Processor -- start 2024-03-24T10:48:57.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.955+0000 7f051f820700 1 -- start start 2024-03-24T10:48:57.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.956+0000 7f051f820700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05180fbcd0 0x7f051810fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:48:57.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.956+0000 7f051f820700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05180fcf80 0x7f05181100e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:48:57.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.956+0000 7f051f820700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05181106e0 con 0x7f05180fcf80 2024-03-24T10:48:57.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.956+0000 7f051f820700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0518110f30 con 0x7f05180fbcd0 2024-03-24T10:48:57.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.956+0000 7f051cdbb700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05180fcf80 0x7f05181100e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:48:57.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.956+0000 7f051cdbb700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05180fcf80 0x7f05181100e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45144/0 (socket says 172.21.15.94:45144) 2024-03-24T10:48:57.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.956+0000 7f051cdbb700 1 -- 172.21.15.94:0/2371550418 learned_addr learned my addr 172.21.15.94:0/2371550418 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:48:57.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.956+0000 7f051d5bc700 1 --2- 172.21.15.94:0/2371550418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05180fbcd0 0x7f051810fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:48:57.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.956+0000 7f051cdbb700 1 -- 172.21.15.94:0/2371550418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05180fbcd0 msgr2=0x7f051810fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:57.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.956+0000 7f051cdbb700 1 --2- 172.21.15.94:0/2371550418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05180fbcd0 0x7f051810fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:57.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.956+0000 7f051cdbb700 1 -- 172.21.15.94:0/2371550418 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0508005040 con 0x7f05180fcf80 2024-03-24T10:48:57.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.956+0000 7f051cdbb700 1 --2- 172.21.15.94:0/2371550418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05180fcf80 0x7f05181100e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0508013870 tx=0x7f0508009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:48:57.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.956+0000 7f050dffb700 1 -- 172.21.15.94:0/2371550418 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0508021070 con 0x7f05180fcf80 2024-03-24T10:48:57.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.956+0000 7f050dffb700 1 -- 172.21.15.94:0/2371550418 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0508009430 con 0x7f05180fcf80 2024-03-24T10:48:57.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.956+0000 7f051f820700 1 -- 172.21.15.94:0/2371550418 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05181111b0 con 0x7f05180fcf80 2024-03-24T10:48:57.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.956+0000 7f050dffb700 1 -- 172.21.15.94:0/2371550418 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f050800e070 con 0x7f05180fcf80 2024-03-24T10:48:57.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.957+0000 7f051f820700 1 -- 172.21.15.94:0/2371550418 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05181116f0 con 0x7f05180fcf80 2024-03-24T10:48:57.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.958+0000 7f051f820700 1 -- 172.21.15.94:0/2371550418 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0518102a00 con 0x7f05180fcf80 2024-03-24T10:48:57.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.958+0000 7f050dffb700 1 -- 172.21.15.94:0/2371550418 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 35) v1 ==== 91587+0+0 (secure 0 0 0) 0x7f0508026410 con 0x7f05180fcf80 2024-03-24T10:48:57.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.958+0000 7f050dffb700 1 --2- 172.21.15.94:0/2371550418 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f050406d900 0x7f050406fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:48:57.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.958+0000 7f050dffb700 1 -- 172.21.15.94:0/2371550418 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f050808e270 con 0x7f05180fcf80 2024-03-24T10:48:57.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.958+0000 7f051d5bc700 1 --2- 172.21.15.94:0/2371550418 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f050406d900 0x7f050406fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:48:57.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.959+0000 7f051d5bc700 1 --2- 172.21.15.94:0/2371550418 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f050406d900 0x7f050406fdb0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0510001470 tx=0x7f051000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:48:57.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:57.963+0000 7f050dffb700 1 -- 172.21.15.94:0/2371550418 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f050805b490 con 0x7f05180fcf80 2024-03-24T10:48:58.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.244+0000 7f051f820700 1 -- 172.21.15.94:0/2371550418 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0518111340 con 0x7f05180fcf80 2024-03-24T10:48:58.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.245+0000 7f050dffb700 1 -- 172.21.15.94:0/2371550418 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+736 (secure 0 0 0) 0x7f0518111340 con 0x7f05180fcf80 2024-03-24T10:48:58.245 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-03-24T10:48:58.245 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": { 2024-03-24T10:48:58.245 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 2 2024-03-24T10:48:58.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:48:58.245 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": { 2024-03-24T10:48:58.245 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 2 2024-03-24T10:48:58.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:48:58.245 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": { 2024-03-24T10:48:58.245 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-24T10:48:58.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:48:58.245 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-03-24T10:48:58.245 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-24T10:48:58.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:48:58.246 INFO:teuthology.orchestra.run.smithi094.stdout: "overall": { 2024-03-24T10:48:58.246 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 10, 2024-03-24T10:48:58.246 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 4 2024-03-24T10:48:58.246 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-03-24T10:48:58.246 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-03-24T10:48:58.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.247+0000 7f051f820700 1 -- 172.21.15.94:0/2371550418 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f050406d900 msgr2=0x7f050406fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:58.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.247+0000 7f051f820700 1 --2- 172.21.15.94:0/2371550418 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f050406d900 0x7f050406fdb0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0510001470 tx=0x7f051000b040 comp rx=0 tx=0).stop 2024-03-24T10:48:58.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.247+0000 7f051f820700 1 -- 172.21.15.94:0/2371550418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05180fcf80 msgr2=0x7f05181100e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:58.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.247+0000 7f051f820700 1 --2- 172.21.15.94:0/2371550418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05180fcf80 0x7f05181100e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0508013870 tx=0x7f0508009e30 comp rx=0 tx=0).stop 2024-03-24T10:48:58.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.247+0000 7f051f820700 1 -- 172.21.15.94:0/2371550418 shutdown_connections 2024-03-24T10:48:58.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.247+0000 7f051f820700 1 --2- 172.21.15.94:0/2371550418 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f050406d900 0x7f050406fdb0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:58.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.247+0000 7f051f820700 1 --2- 172.21.15.94:0/2371550418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05180fbcd0 0x7f051810fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:58.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.247+0000 7f051f820700 1 --2- 172.21.15.94:0/2371550418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05180fcf80 0x7f05181100e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:58.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.247+0000 7f051f820700 1 -- 172.21.15.94:0/2371550418 >> 172.21.15.94:0/2371550418 conn(0x7f05180f7430 msgr2=0x7f05180f8a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:48:58.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.247+0000 7f051f820700 1 -- 172.21.15.94:0/2371550418 shutdown_connections 2024-03-24T10:48:58.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.248+0000 7f051f820700 1 -- 172.21.15.94:0/2371550418 wait complete. 2024-03-24T10:48:58.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.350+0000 7f2c0444e700 1 -- 172.21.15.94:0/3093660633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bfc0fed80 msgr2=0x7f2bfc0faec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:58.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.350+0000 7f2c0444e700 1 --2- 172.21.15.94:0/3093660633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bfc0fed80 0x7f2bfc0faec0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0004940 tx=0x7f2bf0009e30 comp rx=0 tx=0).stop 2024-03-24T10:48:58.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.350+0000 7f2c0444e700 1 -- 172.21.15.94:0/3093660633 shutdown_connections 2024-03-24T10:48:58.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.350+0000 7f2c0444e700 1 --2- 172.21.15.94:0/3093660633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bfc0fed80 0x7f2bfc0faec0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:58.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.350+0000 7f2c0444e700 1 --2- 172.21.15.94:0/3093660633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bfc0fe430 0x7f2bfc0fe840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:58.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.350+0000 7f2c0444e700 1 -- 172.21.15.94:0/3093660633 >> 172.21.15.94:0/3093660633 conn(0x7f2bfc0f6bd0 msgr2=0x7f2bfc0f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:48:58.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.350+0000 7f2c0444e700 1 -- 172.21.15.94:0/3093660633 shutdown_connections 2024-03-24T10:48:58.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.350+0000 7f2c0444e700 1 -- 172.21.15.94:0/3093660633 wait complete. 2024-03-24T10:48:58.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.350+0000 7f2c0444e700 1 Processor -- start 2024-03-24T10:48:58.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.351+0000 7f2c0444e700 1 -- start start 2024-03-24T10:48:58.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.351+0000 7f2c0444e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bfc0fe430 0x7f2bfc0fda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:48:58.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.351+0000 7f2c0444e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bfc0fed80 0x7f2bfc0fc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:48:58.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.351+0000 7f2c0444e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bfc0fc630 con 0x7f2bfc0fed80 2024-03-24T10:48:58.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.351+0000 7f2c0444e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bfc0fc7a0 con 0x7f2bfc0fe430 2024-03-24T10:48:58.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.351+0000 7f2c019e9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bfc0fed80 0x7f2bfc0fc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:48:58.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.351+0000 7f2c019e9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bfc0fed80 0x7f2bfc0fc0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45174/0 (socket says 172.21.15.94:45174) 2024-03-24T10:48:58.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.351+0000 7f2c019e9700 1 -- 172.21.15.94:0/2587335667 learned_addr learned my addr 172.21.15.94:0/2587335667 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:48:58.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.351+0000 7f2c021ea700 1 --2- 172.21.15.94:0/2587335667 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bfc0fe430 0x7f2bfc0fda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:48:58.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.351+0000 7f2c019e9700 1 -- 172.21.15.94:0/2587335667 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bfc0fe430 msgr2=0x7f2bfc0fda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:58.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.351+0000 7f2c019e9700 1 --2- 172.21.15.94:0/2587335667 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bfc0fe430 0x7f2bfc0fda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:58.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.351+0000 7f2c019e9700 1 -- 172.21.15.94:0/2587335667 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf0005040 con 0x7f2bfc0fed80 2024-03-24T10:48:58.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.351+0000 7f2c019e9700 1 --2- 172.21.15.94:0/2587335667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bfc0fed80 0x7f2bfc0fc0f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0009150 tx=0x7f2bf0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:48:58.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.351+0000 7f2bfaffd700 1 -- 172.21.15.94:0/2587335667 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2bf000e040 con 0x7f2bfc0fed80 2024-03-24T10:48:58.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.351+0000 7f2bfaffd700 1 -- 172.21.15.94:0/2587335667 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2bf000bcc0 con 0x7f2bfc0fed80 2024-03-24T10:48:58.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.351+0000 7f2c0444e700 1 -- 172.21.15.94:0/2587335667 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bfc0fca20 con 0x7f2bfc0fed80 2024-03-24T10:48:58.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.352+0000 7f2bfaffd700 1 -- 172.21.15.94:0/2587335667 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2bf001c630 con 0x7f2bfc0fed80 2024-03-24T10:48:58.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.352+0000 7f2c0444e700 1 -- 172.21.15.94:0/2587335667 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bfc115a30 con 0x7f2bfc0fed80 2024-03-24T10:48:58.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.352+0000 7f2c0444e700 1 -- 172.21.15.94:0/2587335667 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bfc04c710 con 0x7f2bfc0fed80 2024-03-24T10:48:58.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.353+0000 7f2bfaffd700 1 -- 172.21.15.94:0/2587335667 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 35) v1 ==== 91587+0+0 (secure 0 0 0) 0x7f2bf0013420 con 0x7f2bfc0fed80 2024-03-24T10:48:58.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.353+0000 7f2bfaffd700 1 --2- 172.21.15.94:0/2587335667 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f2be406d910 0x7f2be406fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:48:58.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.353+0000 7f2bfaffd700 1 -- 172.21.15.94:0/2587335667 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f2bf008dd80 con 0x7f2bfc0fed80 2024-03-24T10:48:58.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.353+0000 7f2c021ea700 1 --2- 172.21.15.94:0/2587335667 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f2be406d910 0x7f2be406fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:48:58.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.354+0000 7f2c021ea700 1 --2- 172.21.15.94:0/2587335667 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f2be406d910 0x7f2be406fdc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4006fd0 tx=0x7f2bf4008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:48:58.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.358+0000 7f2bfaffd700 1 -- 172.21.15.94:0/2587335667 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f2bf005afa0 con 0x7f2bfc0fed80 2024-03-24T10:48:58.418 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:58 smithi096 ceph-mon[84258]: from='client.34130 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:48:58.418 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:58 smithi096 ceph-mon[84258]: pgmap v14: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:48:58.418 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:58 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/2371550418' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:48:58.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:58 smithi094 ceph-mon[93957]: from='client.34130 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:48:58.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:58 smithi094 ceph-mon[93957]: pgmap v14: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:48:58.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:58 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/2371550418' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:48:58.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.600+0000 7f2c0444e700 1 -- 172.21.15.94:0/2587335667 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2bfc105960 con 0x7f2bfc0fed80 2024-03-24T10:48:58.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.601+0000 7f2bfaffd700 1 -- 172.21.15.94:0/2587335667 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1903 (secure 0 0 0) 0x7f2bf005ab30 con 0x7f2bfc0fed80 2024-03-24T10:48:58.601 INFO:teuthology.orchestra.run.smithi094.stdout:e18 2024-03-24T10:48:58.601 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T10:48:58.602 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:48:58.602 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-03-24T10:48:58.602 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:48:58.602 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-03-24T10:48:58.602 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-03-24T10:48:58.602 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 18 2024-03-24T10:48:58.602 INFO:teuthology.orchestra.run.smithi094.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-24T10:48:58.602 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-03-24T10:43:39.435738+0000 2024-03-24T10:48:58.602 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-03-24T10:43:58.133901+0000 2024-03-24T10:48:58.602 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-03-24T10:48:58.602 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-03-24T10:48:58.602 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 60 2024-03-24T10:48:58.602 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-03-24T10:48:58.602 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-03-24T10:48:58.603 INFO:teuthology.orchestra.run.smithi094.stdout:max_xattr_size 65536 2024-03-24T10:48:58.603 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-03-24T10:48:58.603 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-03-24T10:48:58.603 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 46 2024-03-24T10:48:58.603 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:48:58.603 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-03-24T10:48:58.603 INFO:teuthology.orchestra.run.smithi094.stdout:in 0 2024-03-24T10:48:58.603 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=24251} 2024-03-24T10:48:58.603 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-03-24T10:48:58.603 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-03-24T10:48:58.603 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-03-24T10:48:58.603 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-03-24T10:48:58.603 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-03-24T10:48:58.603 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-03-24T10:48:58.603 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-03-24T10:48:58.604 INFO:teuthology.orchestra.run.smithi094.stdout:bal_rank_mask -1 2024-03-24T10:48:58.604 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 1 2024-03-24T10:48:58.604 INFO:teuthology.orchestra.run.smithi094.stdout:qdb_cluster leader: 0 members: 2024-03-24T10:48:58.604 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi096.egbmoq{0:24251} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.96:6824/1539752208,v1:172.21.15.96:6825/1539752208] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:48:58.604 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.llefgu{0:14500} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.94:6828/126762980,v1:172.21.15.94:6829/126762980] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:48:58.604 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:48:58.604 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:48:58.604 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-03-24T10:48:58.604 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:48:58.604 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.efiqte{-1:14512} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.94:6826/1636528860,v1:172.21.15.94:6827/1636528860] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:48:58.604 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi096.jtebhh{-1:24261} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.96:6826/1321087930,v1:172.21.15.96:6827/1321087930] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:48:58.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.603+0000 7f2c0444e700 1 -- 172.21.15.94:0/2587335667 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f2be406d910 msgr2=0x7f2be406fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:58.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.603+0000 7f2c0444e700 1 --2- 172.21.15.94:0/2587335667 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f2be406d910 0x7f2be406fdc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4006fd0 tx=0x7f2bf4008040 comp rx=0 tx=0).stop 2024-03-24T10:48:58.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.603+0000 7f2c0444e700 1 -- 172.21.15.94:0/2587335667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bfc0fed80 msgr2=0x7f2bfc0fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:58.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.603+0000 7f2c0444e700 1 --2- 172.21.15.94:0/2587335667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bfc0fed80 0x7f2bfc0fc0f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0009150 tx=0x7f2bf0009e30 comp rx=0 tx=0).stop 2024-03-24T10:48:58.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.603+0000 7f2c0444e700 1 -- 172.21.15.94:0/2587335667 shutdown_connections 2024-03-24T10:48:58.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.603+0000 7f2c0444e700 1 --2- 172.21.15.94:0/2587335667 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f2be406d910 0x7f2be406fdc0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:58.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.603+0000 7f2c0444e700 1 --2- 172.21.15.94:0/2587335667 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bfc0fe430 0x7f2bfc0fda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:58.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.603+0000 7f2c0444e700 1 --2- 172.21.15.94:0/2587335667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bfc0fed80 0x7f2bfc0fc0f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:58.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.603+0000 7f2c0444e700 1 -- 172.21.15.94:0/2587335667 >> 172.21.15.94:0/2587335667 conn(0x7f2bfc0f6bd0 msgr2=0x7f2bfc0f81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:48:58.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.603+0000 7f2c0444e700 1 -- 172.21.15.94:0/2587335667 shutdown_connections 2024-03-24T10:48:58.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.603+0000 7f2c0444e700 1 -- 172.21.15.94:0/2587335667 wait complete. 2024-03-24T10:48:58.606 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 18 2024-03-24T10:48:58.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.706+0000 7f17598d6700 1 -- 172.21.15.94:0/4005354836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17540fe070 msgr2=0x7f17540fe4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:58.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.706+0000 7f17598d6700 1 --2- 172.21.15.94:0/4005354836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17540fe070 0x7f17540fe4e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1748004940 tx=0x7f1748009e30 comp rx=0 tx=0).stop 2024-03-24T10:48:58.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.706+0000 7f17598d6700 1 -- 172.21.15.94:0/4005354836 shutdown_connections 2024-03-24T10:48:58.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.706+0000 7f17598d6700 1 --2- 172.21.15.94:0/4005354836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17540fe070 0x7f17540fe4e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:58.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.706+0000 7f17598d6700 1 --2- 172.21.15.94:0/4005354836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17540fcf00 0x7f17540fd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:58.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.706+0000 7f17598d6700 1 -- 172.21.15.94:0/4005354836 >> 172.21.15.94:0/4005354836 conn(0x7f17540f85a0 msgr2=0x7f17540fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:48:58.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.706+0000 7f17598d6700 1 -- 172.21.15.94:0/4005354836 shutdown_connections 2024-03-24T10:48:58.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.706+0000 7f17598d6700 1 -- 172.21.15.94:0/4005354836 wait complete. 2024-03-24T10:48:58.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.707+0000 7f17598d6700 1 Processor -- start 2024-03-24T10:48:58.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.707+0000 7f17598d6700 1 -- start start 2024-03-24T10:48:58.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.707+0000 7f17598d6700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17540fcf00 0x7f1754071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:48:58.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.707+0000 7f17598d6700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17540fe070 0x7f1754070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:48:58.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.707+0000 7f17598d6700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17540708c0 con 0x7f17540fcf00 2024-03-24T10:48:58.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.707+0000 7f17598d6700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1754070a30 con 0x7f17540fe070 2024-03-24T10:48:58.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.707+0000 7f1752ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17540fcf00 0x7f1754071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:48:58.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.707+0000 7f1752ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17540fcf00 0x7f1754071e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45186/0 (socket says 172.21.15.94:45186) 2024-03-24T10:48:58.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.707+0000 7f1752ffd700 1 -- 172.21.15.94:0/3620870230 learned_addr learned my addr 172.21.15.94:0/3620870230 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:48:58.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.707+0000 7f17527fc700 1 --2- 172.21.15.94:0/3620870230 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17540fe070 0x7f1754070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:48:58.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.707+0000 7f1752ffd700 1 -- 172.21.15.94:0/3620870230 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17540fe070 msgr2=0x7f1754070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:58.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.707+0000 7f1752ffd700 1 --2- 172.21.15.94:0/3620870230 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17540fe070 0x7f1754070310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:58.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.707+0000 7f1752ffd700 1 -- 172.21.15.94:0/3620870230 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1748005040 con 0x7f17540fcf00 2024-03-24T10:48:58.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.707+0000 7f1752ffd700 1 --2- 172.21.15.94:0/3620870230 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17540fcf00 0x7f1754071e40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f174400c8c0 tx=0x7f17440044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:48:58.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.708+0000 7f173b7fe700 1 -- 172.21.15.94:0/3620870230 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f174400a980 con 0x7f17540fcf00 2024-03-24T10:48:58.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.708+0000 7f173b7fe700 1 -- 172.21.15.94:0/3620870230 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f174400ce60 con 0x7f17540fcf00 2024-03-24T10:48:58.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.708+0000 7f17598d6700 1 -- 172.21.15.94:0/3620870230 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1754070d10 con 0x7f17540fcf00 2024-03-24T10:48:58.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.708+0000 7f173b7fe700 1 -- 172.21.15.94:0/3620870230 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17440115e0 con 0x7f17540fcf00 2024-03-24T10:48:58.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.708+0000 7f17598d6700 1 -- 172.21.15.94:0/3620870230 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1754115a80 con 0x7f17540fcf00 2024-03-24T10:48:58.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.709+0000 7f17598d6700 1 -- 172.21.15.94:0/3620870230 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f175405f5f0 con 0x7f17540fcf00 2024-03-24T10:48:58.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.709+0000 7f173b7fe700 1 -- 172.21.15.94:0/3620870230 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 35) v1 ==== 91587+0+0 (secure 0 0 0) 0x7f1744012400 con 0x7f17540fcf00 2024-03-24T10:48:58.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.710+0000 7f173b7fe700 1 --2- 172.21.15.94:0/3620870230 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f173c06d900 0x7f173c06fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:48:58.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.710+0000 7f173b7fe700 1 -- 172.21.15.94:0/3620870230 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f174408d080 con 0x7f17540fcf00 2024-03-24T10:48:58.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.710+0000 7f17527fc700 1 --2- 172.21.15.94:0/3620870230 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f173c06d900 0x7f173c06fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:48:58.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.710+0000 7f17527fc700 1 --2- 172.21.15.94:0/3620870230 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f173c06d900 0x7f173c06fdb0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f174800b3b0 tx=0x7f1748009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:48:58.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.714+0000 7f173b7fe700 1 -- 172.21.15.94:0/3620870230 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1744056880 con 0x7f17540fcf00 2024-03-24T10:48:58.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.935+0000 7f17598d6700 1 -- 172.21.15.94:0/3620870230 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f17541023d0 con 0x7f173c06d900 2024-03-24T10:48:58.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.936+0000 7f173b7fe700 1 -- 172.21.15.94:0/3620870230 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f17541023d0 con 0x7f173c06d900 2024-03-24T10:48:58.937 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-03-24T10:48:58.937 INFO:teuthology.orchestra.run.smithi094.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919", 2024-03-24T10:48:58.937 INFO:teuthology.orchestra.run.smithi094.stdout: "in_progress": true, 2024-03-24T10:48:58.937 INFO:teuthology.orchestra.run.smithi094.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T10:48:58.937 INFO:teuthology.orchestra.run.smithi094.stdout: "services_complete": [ 2024-03-24T10:48:58.937 INFO:teuthology.orchestra.run.smithi094.stdout: "mon", 2024-03-24T10:48:58.937 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr" 2024-03-24T10:48:58.937 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-03-24T10:48:58.937 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": "4/21 daemons upgraded", 2024-03-24T10:48:58.937 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "Currently upgrading mon daemons", 2024-03-24T10:48:58.938 INFO:teuthology.orchestra.run.smithi094.stdout: "is_paused": false 2024-03-24T10:48:58.938 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-03-24T10:48:58.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.938+0000 7f17598d6700 1 -- 172.21.15.94:0/3620870230 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f173c06d900 msgr2=0x7f173c06fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:58.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.938+0000 7f17598d6700 1 --2- 172.21.15.94:0/3620870230 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f173c06d900 0x7f173c06fdb0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f174800b3b0 tx=0x7f1748009e30 comp rx=0 tx=0).stop 2024-03-24T10:48:58.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.938+0000 7f17598d6700 1 -- 172.21.15.94:0/3620870230 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17540fcf00 msgr2=0x7f1754071e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:58.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.938+0000 7f17598d6700 1 --2- 172.21.15.94:0/3620870230 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17540fcf00 0x7f1754071e40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f174400c8c0 tx=0x7f17440044c0 comp rx=0 tx=0).stop 2024-03-24T10:48:58.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.938+0000 7f17598d6700 1 -- 172.21.15.94:0/3620870230 shutdown_connections 2024-03-24T10:48:58.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.938+0000 7f17598d6700 1 --2- 172.21.15.94:0/3620870230 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f173c06d900 0x7f173c06fdb0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:58.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.938+0000 7f17598d6700 1 --2- 172.21.15.94:0/3620870230 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17540fcf00 0x7f1754071e40 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:58.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.938+0000 7f17598d6700 1 --2- 172.21.15.94:0/3620870230 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17540fe070 0x7f1754070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:58.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.938+0000 7f17598d6700 1 -- 172.21.15.94:0/3620870230 >> 172.21.15.94:0/3620870230 conn(0x7f17540f85a0 msgr2=0x7f17540f9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:48:58.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.939+0000 7f17598d6700 1 -- 172.21.15.94:0/3620870230 shutdown_connections 2024-03-24T10:48:58.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:58.939+0000 7f17598d6700 1 -- 172.21.15.94:0/3620870230 wait complete. 2024-03-24T10:48:59.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:59.055+0000 7f3cd2169700 1 -- 172.21.15.94:0/2253227295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ccc0fe400 msgr2=0x7f3ccc0fe810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:59.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:59.055+0000 7f3cd2169700 1 --2- 172.21.15.94:0/2253227295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ccc0fe400 0x7f3ccc0fe810 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc004940 tx=0x7f3cbc009e30 comp rx=0 tx=0).stop 2024-03-24T10:48:59.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:59.055+0000 7f3cd2169700 1 -- 172.21.15.94:0/2253227295 shutdown_connections 2024-03-24T10:48:59.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:59.055+0000 7f3cd2169700 1 --2- 172.21.15.94:0/2253227295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ccc0fed50 0x7f3ccc0fae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:59.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:59.055+0000 7f3cd2169700 1 --2- 172.21.15.94:0/2253227295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ccc0fe400 0x7f3ccc0fe810 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:59.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:59.055+0000 7f3cd2169700 1 -- 172.21.15.94:0/2253227295 >> 172.21.15.94:0/2253227295 conn(0x7f3ccc0f6ba0 msgr2=0x7f3ccc0f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:48:59.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:59.055+0000 7f3cd2169700 1 -- 172.21.15.94:0/2253227295 shutdown_connections 2024-03-24T10:48:59.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:59.055+0000 7f3cd2169700 1 -- 172.21.15.94:0/2253227295 wait complete. 2024-03-24T10:48:59.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:59.056+0000 7f3cd2169700 1 Processor -- start 2024-03-24T10:48:59.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:59.056+0000 7f3cd2169700 1 -- start start 2024-03-24T10:48:59.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:59.056+0000 7f3cd2169700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ccc0fe400 0x7f3ccc10d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:48:59.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:59.056+0000 7f3cd2169700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ccc0fed50 0x7f3ccc10df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:48:59.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:59.056+0000 7f3cd2169700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc10e530 con 0x7f3ccc0fe400 2024-03-24T10:48:59.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:59.056+0000 7f3cd2169700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc10ed80 con 0x7f3ccc0fed50 2024-03-24T10:48:59.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:59.056+0000 7f3ccb7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ccc0fe400 0x7f3ccc10d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:48:59.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:59.057+0000 7f3ccb7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ccc0fe400 0x7f3ccc10d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45198/0 (socket says 172.21.15.94:45198) 2024-03-24T10:48:59.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:59.057+0000 7f3ccb7fe700 1 -- 172.21.15.94:0/1419626204 learned_addr learned my addr 172.21.15.94:0/1419626204 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:48:59.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:59.057+0000 7f3ccaffd700 1 --2- 172.21.15.94:0/1419626204 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ccc0fed50 0x7f3ccc10df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:48:59.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:59.057+0000 7f3ccb7fe700 1 -- 172.21.15.94:0/1419626204 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ccc0fed50 msgr2=0x7f3ccc10df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:59.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:59.057+0000 7f3ccb7fe700 1 --2- 172.21.15.94:0/1419626204 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ccc0fed50 0x7f3ccc10df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:59.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:59.057+0000 7f3ccb7fe700 1 -- 172.21.15.94:0/1419626204 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cbc005040 con 0x7f3ccc0fe400 2024-03-24T10:48:59.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:59.057+0000 7f3ccb7fe700 1 --2- 172.21.15.94:0/1419626204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ccc0fe400 0x7f3ccc10d9f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc00b3b0 tx=0x7f3cbc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:48:59.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:59.057+0000 7f3cb3fff700 1 -- 172.21.15.94:0/1419626204 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cbc00e040 con 0x7f3ccc0fe400 2024-03-24T10:48:59.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:59.057+0000 7f3cb3fff700 1 -- 172.21.15.94:0/1419626204 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3cbc00bcc0 con 0x7f3ccc0fe400 2024-03-24T10:48:59.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:59.057+0000 7f3cd2169700 1 -- 172.21.15.94:0/1419626204 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ccc10f000 con 0x7f3ccc0fe400 2024-03-24T10:48:59.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:59.057+0000 7f3cb3fff700 1 -- 172.21.15.94:0/1419626204 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cbc01c630 con 0x7f3ccc0fe400 2024-03-24T10:48:59.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:59.057+0000 7f3cd2169700 1 -- 172.21.15.94:0/1419626204 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ccc10f540 con 0x7f3ccc0fe400 2024-03-24T10:48:59.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:59.058+0000 7f3cd2169700 1 -- 172.21.15.94:0/1419626204 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ccc05f5f0 con 0x7f3ccc0fe400 2024-03-24T10:48:59.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:59.059+0000 7f3cb3fff700 1 -- 172.21.15.94:0/1419626204 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 35) v1 ==== 91587+0+0 (secure 0 0 0) 0x7f3cbc026440 con 0x7f3ccc0fe400 2024-03-24T10:48:59.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:59.059+0000 7f3cb3fff700 1 --2- 172.21.15.94:0/1419626204 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f3cb406d900 0x7f3cb406fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:48:59.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:59.059+0000 7f3cb3fff700 1 -- 172.21.15.94:0/1419626204 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f3cbc08dee0 con 0x7f3ccc0fe400 2024-03-24T10:48:59.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:59.059+0000 7f3ccaffd700 1 --2- 172.21.15.94:0/1419626204 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f3cb406d900 0x7f3cb406fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:48:59.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:59.060+0000 7f3ccaffd700 1 --2- 172.21.15.94:0/1419626204 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f3cb406d900 0x7f3cb406fdb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0004990 tx=0x7f3cc0009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:48:59.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:59.064+0000 7f3cb3fff700 1 -- 172.21.15.94:0/1419626204 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3cbc05afe0 con 0x7f3ccc0fe400 2024-03-24T10:48:59.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:59.360+0000 7f3cd2169700 1 -- 172.21.15.94:0/1419626204 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3ccc04bed0 con 0x7f3ccc0fe400 2024-03-24T10:48:59.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:59.361+0000 7f3cb3fff700 1 -- 172.21.15.94:0/1419626204 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f3cbc05ab70 con 0x7f3ccc0fe400 2024-03-24T10:48:59.362 INFO:teuthology.orchestra.run.smithi094.stdout:HEALTH_OK 2024-03-24T10:48:59.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:59.363+0000 7f3cd2169700 1 -- 172.21.15.94:0/1419626204 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f3cb406d900 msgr2=0x7f3cb406fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:59.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:59.363+0000 7f3cd2169700 1 --2- 172.21.15.94:0/1419626204 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f3cb406d900 0x7f3cb406fdb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0004990 tx=0x7f3cc0009ee0 comp rx=0 tx=0).stop 2024-03-24T10:48:59.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:59.363+0000 7f3cd2169700 1 -- 172.21.15.94:0/1419626204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ccc0fe400 msgr2=0x7f3ccc10d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:48:59.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:59.363+0000 7f3cd2169700 1 --2- 172.21.15.94:0/1419626204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ccc0fe400 0x7f3ccc10d9f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc00b3b0 tx=0x7f3cbc009e30 comp rx=0 tx=0).stop 2024-03-24T10:48:59.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:59.363+0000 7f3cd2169700 1 -- 172.21.15.94:0/1419626204 shutdown_connections 2024-03-24T10:48:59.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:59.363+0000 7f3cd2169700 1 --2- 172.21.15.94:0/1419626204 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f3cb406d900 0x7f3cb406fdb0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:59.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:59.363+0000 7f3cd2169700 1 --2- 172.21.15.94:0/1419626204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ccc0fe400 0x7f3ccc10d9f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:59.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:59.363+0000 7f3cd2169700 1 --2- 172.21.15.94:0/1419626204 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ccc0fed50 0x7f3ccc10df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:48:59.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:59.363+0000 7f3cd2169700 1 -- 172.21.15.94:0/1419626204 >> 172.21.15.94:0/1419626204 conn(0x7f3ccc0f6ba0 msgr2=0x7f3ccc0f82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:48:59.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:59.363+0000 7f3cd2169700 1 -- 172.21.15.94:0/1419626204 shutdown_connections 2024-03-24T10:48:59.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:48:59.363+0000 7f3cd2169700 1 -- 172.21.15.94:0/1419626204 wait complete. 2024-03-24T10:48:59.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:59 smithi096 ceph-mon[84258]: from='client.34134 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:48:59.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:59 smithi096 ceph-mon[84258]: from='client.34138 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:48:59.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:59 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/2587335667' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T10:48:59.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:59 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:59.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:59 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:59.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:59 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:48:59.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:59 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:48:59.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:59 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:59.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:59 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T10:48:59.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:59 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T10:48:59.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:48:59 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:48:59.635 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:59 smithi094 ceph-mon[93957]: from='client.34134 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:48:59.635 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:59 smithi094 ceph-mon[93957]: from='client.34138 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:48:59.635 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:59 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/2587335667' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T10:48:59.635 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:59 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:59.635 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:59 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:59.635 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:59 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:48:59.635 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:59 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:48:59.635 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:59 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:48:59.636 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:59 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T10:48:59.636 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:59 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T10:48:59.636 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:48:59 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:49:00.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:00 smithi096 ceph-mon[84258]: from='client.34150 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:49:00.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:00 smithi096 ceph-mon[84258]: Reconfiguring mon.smithi094 (monmap changed)... 2024-03-24T10:49:00.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:00 smithi096 ceph-mon[84258]: Reconfiguring daemon mon.smithi094 on smithi094 2024-03-24T10:49:00.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:00 smithi096 ceph-mon[84258]: pgmap v15: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:49:00.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:00 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/1419626204' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T10:49:00.579 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:00 smithi094 ceph-mon[93957]: from='client.34150 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:49:00.579 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:00 smithi094 ceph-mon[93957]: Reconfiguring mon.smithi094 (monmap changed)... 2024-03-24T10:49:00.579 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:00 smithi094 ceph-mon[93957]: Reconfiguring daemon mon.smithi094 on smithi094 2024-03-24T10:49:00.579 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:00 smithi094 ceph-mon[93957]: pgmap v15: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:49:00.579 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:00 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/1419626204' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T10:49:01.888 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:01 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:01.888 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:01 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:01.888 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:01 smithi094 ceph-mon[93957]: Reconfiguring mgr.smithi094.yaethu (monmap changed)... 2024-03-24T10:49:01.888 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:01 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi094.yaethu", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T10:49:01.888 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:01 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T10:49:01.888 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:01 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:49:01.888 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:01 smithi094 ceph-mon[93957]: Reconfiguring daemon mgr.smithi094.yaethu on smithi094 2024-03-24T10:49:01.888 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:01 smithi094 ceph-mon[93957]: pgmap v16: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:49:02.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:01 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:02.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:01 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:02.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:01 smithi096 ceph-mon[84258]: Reconfiguring mgr.smithi094.yaethu (monmap changed)... 2024-03-24T10:49:02.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:01 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi094.yaethu", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T10:49:02.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:01 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T10:49:02.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:01 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:49:02.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:01 smithi096 ceph-mon[84258]: Reconfiguring daemon mgr.smithi094.yaethu on smithi094 2024-03-24T10:49:02.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:01 smithi096 ceph-mon[84258]: pgmap v16: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:49:03.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:03 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:03.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:03 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:03.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:03 smithi094 ceph-mon[93957]: Reconfiguring crash.smithi094 (monmap changed)... 2024-03-24T10:49:03.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:03 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi094", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-24T10:49:03.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:03 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:49:03.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:03 smithi094 ceph-mon[93957]: Reconfiguring daemon crash.smithi094 on smithi094 2024-03-24T10:49:03.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:03 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:03.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:03 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:03.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:03 smithi096 ceph-mon[84258]: Reconfiguring crash.smithi094 (monmap changed)... 2024-03-24T10:49:03.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:03 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi094", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-24T10:49:03.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:03 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:49:03.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:03 smithi096 ceph-mon[84258]: Reconfiguring daemon crash.smithi094 on smithi094 2024-03-24T10:49:04.581 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:04 smithi094 ceph-mon[93957]: pgmap v17: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:49:04.581 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:04 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:04.581 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:04 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:04.581 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:04 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-24T10:49:04.581 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:04 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:49:04.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:04 smithi096 ceph-mon[84258]: pgmap v17: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:49:04.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:04 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:04.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:04 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:04.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:04 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-24T10:49:04.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:04 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:49:05.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:05 smithi094 ceph-mon[93957]: Reconfiguring osd.0 (monmap changed)... 2024-03-24T10:49:05.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:05 smithi094 ceph-mon[93957]: Reconfiguring daemon osd.0 on smithi094 2024-03-24T10:49:05.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:05 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:49:05.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:05 smithi096 ceph-mon[84258]: Reconfiguring osd.0 (monmap changed)... 2024-03-24T10:49:05.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:05 smithi096 ceph-mon[84258]: Reconfiguring daemon osd.0 on smithi094 2024-03-24T10:49:05.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:05 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:49:06.483 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:06 smithi094 ceph-mon[93957]: pgmap v18: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:49:06.484 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:06 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:06.484 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:06 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:06.484 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:06 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-24T10:49:06.484 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:06 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:49:06.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:06 smithi096 ceph-mon[84258]: pgmap v18: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:49:06.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:06 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:06.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:06 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:06.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:06 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-24T10:49:06.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:06 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:49:07.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:07 smithi094 ceph-mon[93957]: Reconfiguring osd.1 (monmap changed)... 2024-03-24T10:49:07.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:07 smithi094 ceph-mon[93957]: Reconfiguring daemon osd.1 on smithi094 2024-03-24T10:49:07.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:07 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:07.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:07 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:07.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:07 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-24T10:49:07.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:07 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:49:07.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:07 smithi096 ceph-mon[84258]: Reconfiguring osd.1 (monmap changed)... 2024-03-24T10:49:07.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:07 smithi096 ceph-mon[84258]: Reconfiguring daemon osd.1 on smithi094 2024-03-24T10:49:07.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:07 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:07.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:07 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:07.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:07 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-24T10:49:07.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:07 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:49:08.447 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:08 smithi094 ceph-mon[93957]: Reconfiguring osd.2 (monmap changed)... 2024-03-24T10:49:08.447 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:08 smithi094 ceph-mon[93957]: Reconfiguring daemon osd.2 on smithi094 2024-03-24T10:49:08.447 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:08 smithi094 ceph-mon[93957]: pgmap v19: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:49:08.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:08 smithi096 ceph-mon[84258]: Reconfiguring osd.2 (monmap changed)... 2024-03-24T10:49:08.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:08 smithi096 ceph-mon[84258]: Reconfiguring daemon osd.2 on smithi094 2024-03-24T10:49:08.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:08 smithi096 ceph-mon[84258]: pgmap v19: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:49:10.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:09 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:10.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:09 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:10.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:09 smithi094 ceph-mon[93957]: Reconfiguring mds.cephfs.smithi094.efiqte (monmap changed)... 2024-03-24T10:49:10.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:09 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi094.efiqte", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T10:49:10.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:09 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:49:10.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:09 smithi094 ceph-mon[93957]: Reconfiguring daemon mds.cephfs.smithi094.efiqte on smithi094 2024-03-24T10:49:10.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:09 smithi094 ceph-mon[93957]: pgmap v20: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:49:10.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:09 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:10.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:09 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:10.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:09 smithi096 ceph-mon[84258]: Reconfiguring mds.cephfs.smithi094.efiqte (monmap changed)... 2024-03-24T10:49:10.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:09 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi094.efiqte", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T10:49:10.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:09 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:49:10.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:09 smithi096 ceph-mon[84258]: Reconfiguring daemon mds.cephfs.smithi094.efiqte on smithi094 2024-03-24T10:49:10.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:09 smithi096 ceph-mon[84258]: pgmap v20: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:49:11.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:11 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:11.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:11 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:11.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:11 smithi094 ceph-mon[93957]: Reconfiguring mds.cephfs.smithi094.llefgu (monmap changed)... 2024-03-24T10:49:11.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:11 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi094.llefgu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T10:49:11.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:11 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:49:11.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:11 smithi094 ceph-mon[93957]: Reconfiguring daemon mds.cephfs.smithi094.llefgu on smithi094 2024-03-24T10:49:11.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:11 smithi094 ceph-mon[93957]: pgmap v21: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:49:12.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:11 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:12.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:11 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:12.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:11 smithi096 ceph-mon[84258]: Reconfiguring mds.cephfs.smithi094.llefgu (monmap changed)... 2024-03-24T10:49:12.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:11 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi094.llefgu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T10:49:12.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:11 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:49:12.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:11 smithi096 ceph-mon[84258]: Reconfiguring daemon mds.cephfs.smithi094.llefgu on smithi094 2024-03-24T10:49:12.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:11 smithi096 ceph-mon[84258]: pgmap v21: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:49:13.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:13 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:13.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:13 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:13.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:13 smithi094 ceph-mon[93957]: Reconfiguring crash.smithi096 (monmap changed)... 2024-03-24T10:49:13.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:13 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi096", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-24T10:49:13.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:13 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:49:13.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:13 smithi094 ceph-mon[93957]: Reconfiguring daemon crash.smithi096 on smithi096 2024-03-24T10:49:13.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:13 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:13.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:13 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:13.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:13 smithi096 ceph-mon[84258]: Reconfiguring crash.smithi096 (monmap changed)... 2024-03-24T10:49:13.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:13 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi096", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-24T10:49:13.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:13 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:49:13.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:13 smithi096 ceph-mon[84258]: Reconfiguring daemon crash.smithi096 on smithi096 2024-03-24T10:49:14.573 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:14 smithi096 ceph-mon[84258]: pgmap v22: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:49:14.573 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:14 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:14.573 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:14 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:14.573 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:14 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi096.gfonnu", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T10:49:14.573 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:14 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T10:49:14.574 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:14 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:49:14.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:14 smithi094 ceph-mon[93957]: pgmap v22: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:49:14.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:14 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:14.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:14 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:14.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:14 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi096.gfonnu", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T10:49:14.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:14 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T10:49:14.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:14 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:49:15.463 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:15 smithi096 ceph-mon[84258]: Reconfiguring mgr.smithi096.gfonnu (monmap changed)... 2024-03-24T10:49:15.463 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:15 smithi096 ceph-mon[84258]: Reconfiguring daemon mgr.smithi096.gfonnu on smithi096 2024-03-24T10:49:15.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:15 smithi094 ceph-mon[93957]: Reconfiguring mgr.smithi096.gfonnu (monmap changed)... 2024-03-24T10:49:15.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:15 smithi094 ceph-mon[93957]: Reconfiguring daemon mgr.smithi096.gfonnu on smithi096 2024-03-24T10:49:16.458 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:16 smithi096 ceph-mon[84258]: pgmap v23: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:49:16.459 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:16 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:16.459 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:16 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:16.459 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:16 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T10:49:16.459 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:16 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T10:49:16.459 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:16 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:49:16.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:16 smithi094 ceph-mon[93957]: pgmap v23: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:49:16.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:16 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:16.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:16 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:16.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:16 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T10:49:16.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:16 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T10:49:16.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:16 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:49:17.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:17 smithi096 ceph-mon[84258]: Reconfiguring mon.smithi096 (monmap changed)... 2024-03-24T10:49:17.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:17 smithi096 ceph-mon[84258]: Reconfiguring daemon mon.smithi096 on smithi096 2024-03-24T10:49:17.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:17 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:17.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:17 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:17.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:17 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-24T10:49:17.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:17 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:49:17.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:17 smithi094 ceph-mon[93957]: Reconfiguring mon.smithi096 (monmap changed)... 2024-03-24T10:49:17.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:17 smithi094 ceph-mon[93957]: Reconfiguring daemon mon.smithi096 on smithi096 2024-03-24T10:49:17.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:17 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:17.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:17 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:17.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:17 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-24T10:49:17.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:17 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:49:18.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:18 smithi096 ceph-mon[84258]: Reconfiguring osd.3 (monmap changed)... 2024-03-24T10:49:18.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:18 smithi096 ceph-mon[84258]: Reconfiguring daemon osd.3 on smithi096 2024-03-24T10:49:18.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:18 smithi096 ceph-mon[84258]: pgmap v24: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:49:18.658 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:18 smithi094 ceph-mon[93957]: Reconfiguring osd.3 (monmap changed)... 2024-03-24T10:49:18.658 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:18 smithi094 ceph-mon[93957]: Reconfiguring daemon osd.3 on smithi096 2024-03-24T10:49:18.658 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:18 smithi094 ceph-mon[93957]: pgmap v24: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:49:20.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:19 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:20.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:19 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:20.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:19 smithi096 ceph-mon[84258]: Reconfiguring osd.4 (monmap changed)... 2024-03-24T10:49:20.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:19 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-24T10:49:20.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:19 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:49:20.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:19 smithi096 ceph-mon[84258]: Reconfiguring daemon osd.4 on smithi096 2024-03-24T10:49:20.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:19 smithi096 ceph-mon[84258]: pgmap v25: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:49:20.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:19 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:20.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:19 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:20.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:19 smithi094 ceph-mon[93957]: Reconfiguring osd.4 (monmap changed)... 2024-03-24T10:49:20.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:19 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-24T10:49:20.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:19 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:49:20.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:19 smithi094 ceph-mon[93957]: Reconfiguring daemon osd.4 on smithi096 2024-03-24T10:49:20.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:19 smithi094 ceph-mon[93957]: pgmap v25: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:49:21.151 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:20 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:49:21.151 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:20 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:21.151 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:20 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:21.151 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:20 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-24T10:49:21.151 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:20 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:49:21.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:20 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:49:21.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:20 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:21.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:20 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:21.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:20 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-24T10:49:21.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:20 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:49:22.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:21 smithi096 ceph-mon[84258]: Reconfiguring osd.5 (monmap changed)... 2024-03-24T10:49:22.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:21 smithi096 ceph-mon[84258]: Reconfiguring daemon osd.5 on smithi096 2024-03-24T10:49:22.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:21 smithi096 ceph-mon[84258]: pgmap v26: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:49:22.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:21 smithi094 ceph-mon[93957]: Reconfiguring osd.5 (monmap changed)... 2024-03-24T10:49:22.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:21 smithi094 ceph-mon[93957]: Reconfiguring daemon osd.5 on smithi096 2024-03-24T10:49:22.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:21 smithi094 ceph-mon[93957]: pgmap v26: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:49:23.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:23 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:23.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:23 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:23.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:23 smithi094 ceph-mon[93957]: Reconfiguring mds.cephfs.smithi096.egbmoq (monmap changed)... 2024-03-24T10:49:23.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:23 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi096.egbmoq", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T10:49:23.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:23 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:49:23.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:23 smithi094 ceph-mon[93957]: Reconfiguring daemon mds.cephfs.smithi096.egbmoq on smithi096 2024-03-24T10:49:23.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:23 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:23.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:23 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:23.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:23 smithi096 ceph-mon[84258]: Reconfiguring mds.cephfs.smithi096.egbmoq (monmap changed)... 2024-03-24T10:49:23.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:23 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi096.egbmoq", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T10:49:23.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:23 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:49:23.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:23 smithi096 ceph-mon[84258]: Reconfiguring daemon mds.cephfs.smithi096.egbmoq on smithi096 2024-03-24T10:49:24.494 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:24 smithi096 ceph-mon[84258]: pgmap v27: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:49:24.494 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:24 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:24.494 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:24 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:24.494 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:24 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi096.jtebhh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T10:49:24.494 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:24 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:49:24.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:24 smithi094 ceph-mon[93957]: pgmap v27: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:49:24.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:24 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:24.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:24 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:24.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:24 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi096.jtebhh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T10:49:24.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:24 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:49:25.374 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:25 smithi096 ceph-mon[84258]: Reconfiguring mds.cephfs.smithi096.jtebhh (monmap changed)... 2024-03-24T10:49:25.374 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:25 smithi096 ceph-mon[84258]: Reconfiguring daemon mds.cephfs.smithi096.jtebhh on smithi096 2024-03-24T10:49:25.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:25 smithi094 ceph-mon[93957]: Reconfiguring mds.cephfs.smithi096.jtebhh (monmap changed)... 2024-03-24T10:49:25.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:25 smithi094 ceph-mon[93957]: Reconfiguring daemon mds.cephfs.smithi096.jtebhh on smithi096 2024-03-24T10:49:26.383 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:26 smithi094 ceph-mon[93957]: pgmap v28: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:49:26.383 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:26 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:26.384 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:26 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:26.384 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:26 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:49:26.384 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:26 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:49:26.384 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:26 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:49:26.384 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:26 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:26.384 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:26 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi094"}]: dispatch 2024-03-24T10:49:26.384 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:26 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi094"}]': finished 2024-03-24T10:49:26.384 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:26 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi096"}]: dispatch 2024-03-24T10:49:26.384 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:26 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi096"}]': finished 2024-03-24T10:49:26.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:26 smithi096 ceph-mon[84258]: pgmap v28: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:49:26.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:26 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:26.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:26 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:26.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:26 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:49:26.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:26 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:49:26.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:26 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:49:26.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:26 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:26.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:26 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi094"}]: dispatch 2024-03-24T10:49:26.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:26 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi094"}]': finished 2024-03-24T10:49:26.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:26 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi096"}]: dispatch 2024-03-24T10:49:26.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:26 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi096"}]': finished 2024-03-24T10:49:27.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:27 smithi094 ceph-mon[93957]: Upgrade: Setting container_image for all mon 2024-03-24T10:49:27.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:27 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:27.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:27 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi094", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-24T10:49:27.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:27 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:49:27.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:27 smithi096 ceph-mon[84258]: Upgrade: Setting container_image for all mon 2024-03-24T10:49:27.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:27 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:27.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:27 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi094", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-24T10:49:27.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:27 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:49:28.477 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:28 smithi096 ceph-mon[84258]: Upgrade: Updating crash.smithi094 (1/2) 2024-03-24T10:49:28.477 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:28 smithi096 ceph-mon[84258]: Deploying daemon crash.smithi094 on smithi094 2024-03-24T10:49:28.477 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:28 smithi096 ceph-mon[84258]: pgmap v29: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:49:28.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:28 smithi094 ceph-mon[93957]: Upgrade: Updating crash.smithi094 (1/2) 2024-03-24T10:49:28.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:28 smithi094 ceph-mon[93957]: Deploying daemon crash.smithi094 on smithi094 2024-03-24T10:49:28.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:28 smithi094 ceph-mon[93957]: pgmap v29: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:49:29.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.470+0000 7f101a618700 1 -- 172.21.15.94:0/914940476 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1014066ba0 msgr2=0x7f10140ffa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:49:29.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.470+0000 7f101a618700 1 --2- 172.21.15.94:0/914940476 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1014066ba0 0x7f10140ffa50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1008004940 tx=0x7f1008009e30 comp rx=0 tx=0).stop 2024-03-24T10:49:29.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.470+0000 7f101a618700 1 -- 172.21.15.94:0/914940476 shutdown_connections 2024-03-24T10:49:29.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.470+0000 7f101a618700 1 --2- 172.21.15.94:0/914940476 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10140fff90 0x7f10141023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:49:29.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.470+0000 7f101a618700 1 --2- 172.21.15.94:0/914940476 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1014066ba0 0x7f10140ffa50 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:49:29.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.470+0000 7f101a618700 1 -- 172.21.15.94:0/914940476 >> 172.21.15.94:0/914940476 conn(0x7f10140f7480 msgr2=0x7f10140f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:49:29.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.470+0000 7f101a618700 1 -- 172.21.15.94:0/914940476 shutdown_connections 2024-03-24T10:49:29.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.470+0000 7f101a618700 1 -- 172.21.15.94:0/914940476 wait complete. 2024-03-24T10:49:29.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.471+0000 7f101a618700 1 Processor -- start 2024-03-24T10:49:29.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.471+0000 7f101a618700 1 -- start start 2024-03-24T10:49:29.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.471+0000 7f101a618700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1014066ba0 0x7f101410daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:49:29.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.471+0000 7f101a618700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10140fff90 0x7f101410dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:49:29.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.471+0000 7f101a618700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101410e670 con 0x7f1014066ba0 2024-03-24T10:49:29.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.471+0000 7f101a618700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101410ee10 con 0x7f10140fff90 2024-03-24T10:49:29.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.471+0000 7f1013fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1014066ba0 0x7f101410daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:49:29.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.471+0000 7f1013fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1014066ba0 0x7f101410daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44474/0 (socket says 172.21.15.94:44474) 2024-03-24T10:49:29.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.471+0000 7f1013fff700 1 -- 172.21.15.94:0/3438559080 learned_addr learned my addr 172.21.15.94:0/3438559080 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:49:29.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.471+0000 7f10137fe700 1 --2- 172.21.15.94:0/3438559080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10140fff90 0x7f101410dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:49:29.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.471+0000 7f1013fff700 1 -- 172.21.15.94:0/3438559080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10140fff90 msgr2=0x7f101410dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:49:29.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.471+0000 7f1013fff700 1 --2- 172.21.15.94:0/3438559080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10140fff90 0x7f101410dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:49:29.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.471+0000 7f1013fff700 1 -- 172.21.15.94:0/3438559080 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1008005040 con 0x7f1014066ba0 2024-03-24T10:49:29.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.472+0000 7f1013fff700 1 --2- 172.21.15.94:0/3438559080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1014066ba0 0x7f101410daa0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1008000c00 tx=0x7f1008009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:49:29.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.472+0000 7f1010ff9700 1 -- 172.21.15.94:0/3438559080 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f100800e040 con 0x7f1014066ba0 2024-03-24T10:49:29.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.472+0000 7f1010ff9700 1 -- 172.21.15.94:0/3438559080 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f100800bbf0 con 0x7f1014066ba0 2024-03-24T10:49:29.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.472+0000 7f1010ff9700 1 -- 172.21.15.94:0/3438559080 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f100801c630 con 0x7f1014066ba0 2024-03-24T10:49:29.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.472+0000 7f101a618700 1 -- 172.21.15.94:0/3438559080 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f101410f090 con 0x7f1014066ba0 2024-03-24T10:49:29.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.472+0000 7f101a618700 1 -- 172.21.15.94:0/3438559080 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f101410f650 con 0x7f1014066ba0 2024-03-24T10:49:29.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.473+0000 7f101a618700 1 -- 172.21.15.94:0/3438559080 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f101405f5f0 con 0x7f1014066ba0 2024-03-24T10:49:29.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.473+0000 7f1010ff9700 1 -- 172.21.15.94:0/3438559080 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 35) v1 ==== 91587+0+0 (secure 0 0 0) 0x7f100800bd60 con 0x7f1014066ba0 2024-03-24T10:49:29.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.474+0000 7f1010ff9700 1 --2- 172.21.15.94:0/3438559080 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f0ffc06d900 0x7f0ffc06fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:49:29.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.474+0000 7f1010ff9700 1 -- 172.21.15.94:0/3438559080 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f100808e410 con 0x7f1014066ba0 2024-03-24T10:49:29.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.474+0000 7f10137fe700 1 --2- 172.21.15.94:0/3438559080 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f0ffc06d900 0x7f0ffc06fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:49:29.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.474+0000 7f10137fe700 1 --2- 172.21.15.94:0/3438559080 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f0ffc06d900 0x7f0ffc06fdb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1004009bf0 tx=0x7f10040091a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:49:29.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.479+0000 7f1010ff9700 1 -- 172.21.15.94:0/3438559080 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f100805b630 con 0x7f1014066ba0 2024-03-24T10:49:29.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.702+0000 7f101a618700 1 -- 172.21.15.94:0/3438559080 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f101410f930 con 0x7f0ffc06d900 2024-03-24T10:49:29.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.703+0000 7f1010ff9700 1 -- 172.21.15.94:0/3438559080 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7f101410f930 con 0x7f0ffc06d900 2024-03-24T10:49:29.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.705+0000 7f101a618700 1 -- 172.21.15.94:0/3438559080 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f0ffc06d900 msgr2=0x7f0ffc06fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:49:29.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.705+0000 7f101a618700 1 --2- 172.21.15.94:0/3438559080 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f0ffc06d900 0x7f0ffc06fdb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1004009bf0 tx=0x7f10040091a0 comp rx=0 tx=0).stop 2024-03-24T10:49:29.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.705+0000 7f101a618700 1 -- 172.21.15.94:0/3438559080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1014066ba0 msgr2=0x7f101410daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:49:29.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.705+0000 7f101a618700 1 --2- 172.21.15.94:0/3438559080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1014066ba0 0x7f101410daa0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1008000c00 tx=0x7f1008009e30 comp rx=0 tx=0).stop 2024-03-24T10:49:29.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.706+0000 7f101a618700 1 -- 172.21.15.94:0/3438559080 shutdown_connections 2024-03-24T10:49:29.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.706+0000 7f101a618700 1 --2- 172.21.15.94:0/3438559080 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f0ffc06d900 0x7f0ffc06fdb0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:49:29.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.706+0000 7f101a618700 1 --2- 172.21.15.94:0/3438559080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1014066ba0 0x7f101410daa0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:49:29.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.706+0000 7f101a618700 1 --2- 172.21.15.94:0/3438559080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10140fff90 0x7f101410dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:49:29.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.706+0000 7f101a618700 1 -- 172.21.15.94:0/3438559080 >> 172.21.15.94:0/3438559080 conn(0x7f10140f7480 msgr2=0x7f10140f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:49:29.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.706+0000 7f101a618700 1 -- 172.21.15.94:0/3438559080 shutdown_connections 2024-03-24T10:49:29.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.706+0000 7f101a618700 1 -- 172.21.15.94:0/3438559080 wait complete. 2024-03-24T10:49:29.718 INFO:teuthology.orchestra.run.smithi094.stdout:true 2024-03-24T10:49:29.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.821+0000 7fc9f0859700 1 -- 172.21.15.94:0/3161677722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9ec0fce70 msgr2=0x7fc9ec0fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:49:29.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.821+0000 7fc9f0859700 1 --2- 172.21.15.94:0/3161677722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9ec0fce70 0x7fc9ec0fd280 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0004940 tx=0x7fc9e0009e30 comp rx=0 tx=0).stop 2024-03-24T10:49:29.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.821+0000 7fc9f0859700 1 -- 172.21.15.94:0/3161677722 shutdown_connections 2024-03-24T10:49:29.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.821+0000 7fc9f0859700 1 --2- 172.21.15.94:0/3161677722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9ec0fdfe0 0x7fc9ec0fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:49:29.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.821+0000 7fc9f0859700 1 --2- 172.21.15.94:0/3161677722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9ec0fce70 0x7fc9ec0fd280 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:49:29.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.821+0000 7fc9f0859700 1 -- 172.21.15.94:0/3161677722 >> 172.21.15.94:0/3161677722 conn(0x7fc9ec0f8550 msgr2=0x7fc9ec0fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:49:29.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.821+0000 7fc9f0859700 1 -- 172.21.15.94:0/3161677722 shutdown_connections 2024-03-24T10:49:29.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.821+0000 7fc9f0859700 1 -- 172.21.15.94:0/3161677722 wait complete. 2024-03-24T10:49:29.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.822+0000 7fc9f0859700 1 Processor -- start 2024-03-24T10:49:29.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.822+0000 7fc9f0859700 1 -- start start 2024-03-24T10:49:29.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.822+0000 7fc9f0859700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9ec0fce70 0x7fc9ec10d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:49:29.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.822+0000 7fc9f0859700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9ec0fdfe0 0x7fc9ec10de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:49:29.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.822+0000 7fc9f0859700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ec10e410 con 0x7fc9ec0fce70 2024-03-24T10:49:29.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.822+0000 7fc9f0859700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ec10e580 con 0x7fc9ec0fdfe0 2024-03-24T10:49:29.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.822+0000 7fc9ea59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9ec0fce70 0x7fc9ec10d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:49:29.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.822+0000 7fc9ea59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9ec0fce70 0x7fc9ec10d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33304/0 (socket says 172.21.15.94:33304) 2024-03-24T10:49:29.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.822+0000 7fc9ea59c700 1 -- 172.21.15.94:0/1544005393 learned_addr learned my addr 172.21.15.94:0/1544005393 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:49:29.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.822+0000 7fc9e9d9b700 1 --2- 172.21.15.94:0/1544005393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9ec0fdfe0 0x7fc9ec10de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:49:29.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.823+0000 7fc9ea59c700 1 -- 172.21.15.94:0/1544005393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9ec0fdfe0 msgr2=0x7fc9ec10de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:49:29.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.823+0000 7fc9ea59c700 1 --2- 172.21.15.94:0/1544005393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9ec0fdfe0 0x7fc9ec10de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:49:29.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.823+0000 7fc9ea59c700 1 -- 172.21.15.94:0/1544005393 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e0005040 con 0x7fc9ec0fce70 2024-03-24T10:49:29.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.823+0000 7fc9ea59c700 1 --2- 172.21.15.94:0/1544005393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9ec0fce70 0x7fc9ec10d8d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc9e000b3b0 tx=0x7fc9e0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:49:29.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.823+0000 7fc9daffd700 1 -- 172.21.15.94:0/1544005393 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9e000e040 con 0x7fc9ec0fce70 2024-03-24T10:49:29.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.823+0000 7fc9daffd700 1 -- 172.21.15.94:0/1544005393 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc9e000bbf0 con 0x7fc9ec0fce70 2024-03-24T10:49:29.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.823+0000 7fc9daffd700 1 -- 172.21.15.94:0/1544005393 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9e001c630 con 0x7fc9ec0fce70 2024-03-24T10:49:29.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.823+0000 7fc9f0859700 1 -- 172.21.15.94:0/1544005393 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9ec112f80 con 0x7fc9ec0fce70 2024-03-24T10:49:29.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.823+0000 7fc9f0859700 1 -- 172.21.15.94:0/1544005393 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9ec1134b0 con 0x7fc9ec0fce70 2024-03-24T10:49:29.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.824+0000 7fc9f0859700 1 -- 172.21.15.94:0/1544005393 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9ec04c710 con 0x7fc9ec0fce70 2024-03-24T10:49:29.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.824+0000 7fc9daffd700 1 -- 172.21.15.94:0/1544005393 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 35) v1 ==== 91587+0+0 (secure 0 0 0) 0x7fc9e000bd60 con 0x7fc9ec0fce70 2024-03-24T10:49:29.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.825+0000 7fc9daffd700 1 --2- 172.21.15.94:0/1544005393 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fc9d406d910 0x7fc9d406fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:49:29.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.825+0000 7fc9e9d9b700 1 --2- 172.21.15.94:0/1544005393 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fc9d406d910 0x7fc9d406fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:49:29.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.825+0000 7fc9daffd700 1 -- 172.21.15.94:0/1544005393 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fc9e008dda0 con 0x7fc9ec0fce70 2024-03-24T10:49:29.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.825+0000 7fc9e9d9b700 1 --2- 172.21.15.94:0/1544005393 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fc9d406d910 0x7fc9d406fdc0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc009fd0 tx=0x7fc9dc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:49:29.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:29.830+0000 7fc9daffd700 1 -- 172.21.15.94:0/1544005393 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc9e005af20 con 0x7fc9ec0fce70 2024-03-24T10:49:30.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.055+0000 7fc9f0859700 1 -- 172.21.15.94:0/1544005393 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc9ec1022b0 con 0x7fc9d406d910 2024-03-24T10:49:30.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.057+0000 7fc9daffd700 1 -- 172.21.15.94:0/1544005393 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7fc9ec1022b0 con 0x7fc9d406d910 2024-03-24T10:49:30.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.059+0000 7fc9f0859700 1 -- 172.21.15.94:0/1544005393 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fc9d406d910 msgr2=0x7fc9d406fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:49:30.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.059+0000 7fc9f0859700 1 --2- 172.21.15.94:0/1544005393 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fc9d406d910 0x7fc9d406fdc0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc009fd0 tx=0x7fc9dc00b040 comp rx=0 tx=0).stop 2024-03-24T10:49:30.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.059+0000 7fc9f0859700 1 -- 172.21.15.94:0/1544005393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9ec0fce70 msgr2=0x7fc9ec10d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:49:30.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.059+0000 7fc9f0859700 1 --2- 172.21.15.94:0/1544005393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9ec0fce70 0x7fc9ec10d8d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc9e000b3b0 tx=0x7fc9e0009e30 comp rx=0 tx=0).stop 2024-03-24T10:49:30.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.059+0000 7fc9f0859700 1 -- 172.21.15.94:0/1544005393 shutdown_connections 2024-03-24T10:49:30.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.059+0000 7fc9f0859700 1 --2- 172.21.15.94:0/1544005393 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fc9d406d910 0x7fc9d406fdc0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:49:30.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.059+0000 7fc9f0859700 1 --2- 172.21.15.94:0/1544005393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9ec0fce70 0x7fc9ec10d8d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:49:30.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.059+0000 7fc9f0859700 1 --2- 172.21.15.94:0/1544005393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9ec0fdfe0 0x7fc9ec10de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:49:30.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.059+0000 7fc9f0859700 1 -- 172.21.15.94:0/1544005393 >> 172.21.15.94:0/1544005393 conn(0x7fc9ec0f8550 msgr2=0x7fc9ec0f9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:49:30.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.059+0000 7fc9f0859700 1 -- 172.21.15.94:0/1544005393 shutdown_connections 2024-03-24T10:49:30.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.059+0000 7fc9f0859700 1 -- 172.21.15.94:0/1544005393 wait complete. 2024-03-24T10:49:30.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.164+0000 7f1b24b7c700 1 -- 172.21.15.94:0/743306702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b20066b70 msgr2=0x7f1b200ffa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:49:30.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.164+0000 7f1b24b7c700 1 --2- 172.21.15.94:0/743306702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b20066b70 0x7f1b200ffa50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1b18009ce0 tx=0x7f1b18004d10 comp rx=0 tx=0).stop 2024-03-24T10:49:30.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.164+0000 7f1b24b7c700 1 -- 172.21.15.94:0/743306702 shutdown_connections 2024-03-24T10:49:30.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.164+0000 7f1b24b7c700 1 --2- 172.21.15.94:0/743306702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b200fff90 0x7f1b201023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:49:30.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.164+0000 7f1b24b7c700 1 --2- 172.21.15.94:0/743306702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b20066b70 0x7f1b200ffa50 secure :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1b18009ce0 tx=0x7f1b18004d10 comp rx=0 tx=0).stop 2024-03-24T10:49:30.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.164+0000 7f1b24b7c700 1 -- 172.21.15.94:0/743306702 >> 172.21.15.94:0/743306702 conn(0x7f1b200f74e0 msgr2=0x7f1b200f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:49:30.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.169+0000 7f1b24b7c700 1 -- 172.21.15.94:0/743306702 shutdown_connections 2024-03-24T10:49:30.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.170+0000 7f1b24b7c700 1 -- 172.21.15.94:0/743306702 wait complete. 2024-03-24T10:49:30.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.171+0000 7f1b24b7c700 1 Processor -- start 2024-03-24T10:49:30.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.173+0000 7f1b24b7c700 1 -- start start 2024-03-24T10:49:30.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.173+0000 7f1b24b7c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b200fff90 0x7f1b200fd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:49:30.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.173+0000 7f1b24b7c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b200fb950 0x7f1b200fbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:49:30.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.173+0000 7f1b1e59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b200fff90 0x7f1b200fd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:49:30.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.173+0000 7f1b1e59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b200fff90 0x7f1b200fd3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33316/0 (socket says 172.21.15.94:33316) 2024-03-24T10:49:30.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.173+0000 7f1b1e59c700 1 -- 172.21.15.94:0/955887625 learned_addr learned my addr 172.21.15.94:0/955887625 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:49:30.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.173+0000 7f1b24b7c700 1 -- 172.21.15.94:0/955887625 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b200fc300 con 0x7f1b200fff90 2024-03-24T10:49:30.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.173+0000 7f1b24b7c700 1 -- 172.21.15.94:0/955887625 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b200fc470 con 0x7f1b200fb950 2024-03-24T10:49:30.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.174+0000 7f1b1e59c700 1 -- 172.21.15.94:0/955887625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b200fb950 msgr2=0x7f1b200fbdc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-24T10:49:30.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.174+0000 7f1b1e59c700 1 --2- 172.21.15.94:0/955887625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b200fb950 0x7f1b200fbdc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:49:30.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.174+0000 7f1b1e59c700 1 -- 172.21.15.94:0/955887625 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b18009940 con 0x7f1b200fff90 2024-03-24T10:49:30.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.174+0000 7f1b1e59c700 1 --2- 172.21.15.94:0/955887625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b200fff90 0x7f1b200fd3e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1b200668b0 tx=0x7f1b18004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:49:30.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.174+0000 7f1b0effd700 1 -- 172.21.15.94:0/955887625 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b1800e030 con 0x7f1b200fff90 2024-03-24T10:49:30.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.174+0000 7f1b0effd700 1 -- 172.21.15.94:0/955887625 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1b1801caf0 con 0x7f1b200fff90 2024-03-24T10:49:30.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.174+0000 7f1b0effd700 1 -- 172.21.15.94:0/955887625 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b18012970 con 0x7f1b200fff90 2024-03-24T10:49:30.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.174+0000 7f1b24b7c700 1 -- 172.21.15.94:0/955887625 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b200fc6f0 con 0x7f1b200fff90 2024-03-24T10:49:30.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.175+0000 7f1b24b7c700 1 -- 172.21.15.94:0/955887625 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b20115f60 con 0x7f1b200fff90 2024-03-24T10:49:30.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.175+0000 7f1b24b7c700 1 -- 172.21.15.94:0/955887625 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b2005f5f0 con 0x7f1b200fff90 2024-03-24T10:49:30.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.176+0000 7f1b0effd700 1 -- 172.21.15.94:0/955887625 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 35) v1 ==== 91587+0+0 (secure 0 0 0) 0x7f1b18019070 con 0x7f1b200fff90 2024-03-24T10:49:30.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.176+0000 7f1b0effd700 1 --2- 172.21.15.94:0/955887625 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1b0806d900 0x7f1b0806fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:49:30.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.177+0000 7f1b0effd700 1 -- 172.21.15.94:0/955887625 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f1b1808ed70 con 0x7f1b200fff90 2024-03-24T10:49:30.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.177+0000 7f1b1dd9b700 1 --2- 172.21.15.94:0/955887625 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1b0806d900 0x7f1b0806fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:49:30.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.177+0000 7f1b1dd9b700 1 --2- 172.21.15.94:0/955887625 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1b0806d900 0x7f1b0806fdb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1b200fcb60 tx=0x7f1b1400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:49:30.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.185+0000 7f1b0effd700 1 -- 172.21.15.94:0/955887625 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1b180583c0 con 0x7f1b200fff90 2024-03-24T10:49:30.409 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:30 smithi094 ceph-mon[93957]: pgmap v30: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:49:30.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.407+0000 7f1b24b7c700 1 -- 172.21.15.94:0/955887625 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1b200f8fd0 con 0x7f1b0806d900 2024-03-24T10:49:30.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.424+0000 7f1b0effd700 1 -- 172.21.15.94:0/955887625 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f1b200f8fd0 con 0x7f1b0806d900 2024-03-24T10:49:30.425 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T10:49:30.425 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.smithi094 smithi094 *:9093,9094 running (2m) 49s ago 11m 16.7M - 0.25.0 c8568f914cd2 5c583af60554 2024-03-24T10:49:30.425 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi094 smithi094 running (11m) 49s ago 11m 6660k - 17.2.7-726-g9c4e0c34 837124da804f fb5772af46c5 2024-03-24T10:49:30.425 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi096 smithi096 running (10m) 36s ago 10m 6669k - 17.2.7-726-g9c4e0c34 837124da804f 6087ad4e0734 2024-03-24T10:49:30.425 INFO:teuthology.orchestra.run.smithi094.stdout:grafana.smithi094 smithi094 *:3000 running (2m) 49s ago 10m 77.6M - 9.4.7 954c08fa6188 d3d425bf1b7c 2024-03-24T10:49:30.425 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.efiqte smithi094 running (5m) 49s ago 5m 12.9M - 17.2.7-726-g9c4e0c34 837124da804f f0df0a056c21 2024-03-24T10:49:30.425 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.llefgu smithi094 running (5m) 49s ago 5m 109M - 17.2.7-726-g9c4e0c34 837124da804f 52a9771dc8e0 2024-03-24T10:49:30.425 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi096.egbmoq smithi096 running (5m) 36s ago 5m 154M - 17.2.7-726-g9c4e0c34 837124da804f 1cfc930dbab0 2024-03-24T10:49:30.425 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi096.jtebhh smithi096 running (5m) 36s ago 5m 16.1M - 17.2.7-726-g9c4e0c34 837124da804f 4ff3451a6f6e 2024-03-24T10:49:30.425 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi094.yaethu smithi094 *:8443,9283,8765 running (117s) 49s ago 13m 473M - 19.0.0-1855-g21b3bd67 553463a90fd5 d3adab76b18f 2024-03-24T10:49:30.426 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi096.gfonnu smithi096 *:8443,9283,8765 running (77s) 36s ago 10m 448M - 19.0.0-1855-g21b3bd67 553463a90fd5 22937a059eb8 2024-03-24T10:49:30.426 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi094 smithi094 running (63s) 49s ago 13m 29.8M 2048M 19.0.0-1855-g21b3bd67 553463a90fd5 c779d611e29a 2024-03-24T10:49:30.426 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi096 smithi096 running (41s) 36s ago 10m 17.3M 2048M 19.0.0-1855-g21b3bd67 553463a90fd5 ed9bd0bd57f6 2024-03-24T10:49:30.426 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi094 smithi094 *:9100 running (2m) 49s ago 11m 8190k - 1.5.0 0da6a335fe13 ca67b53c64e2 2024-03-24T10:49:30.426 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi096 smithi096 *:9100 running (2m) 36s ago 10m 16.6M - 1.5.0 0da6a335fe13 65137a7a7549 2024-03-24T10:49:30.426 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (9m) 49s ago 9m 248M 1557M 17.2.7-726-g9c4e0c34 837124da804f b664fd3557bd 2024-03-24T10:49:30.426 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi094 running (8m) 49s ago 8m 259M 1557M 17.2.7-726-g9c4e0c34 837124da804f 4e737a80d733 2024-03-24T10:49:30.426 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi094 running (8m) 49s ago 8m 220M 1557M 17.2.7-726-g9c4e0c34 837124da804f cf0aefd20c76 2024-03-24T10:49:30.426 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi096 running (8m) 36s ago 8m 284M 2581M 17.2.7-726-g9c4e0c34 837124da804f f4650d0f6241 2024-03-24T10:49:30.426 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi096 running (7m) 36s ago 7m 257M 2581M 17.2.7-726-g9c4e0c34 837124da804f a63d7c9e705f 2024-03-24T10:49:30.426 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi096 running (7m) 36s ago 7m 242M 2581M 17.2.7-726-g9c4e0c34 837124da804f f56fad2959c3 2024-03-24T10:49:30.426 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.smithi094 smithi094 *:9095 running (83s) 49s ago 10m 41.2M - 2.43.0 a07b618ecd1d 6ce091a0c326 2024-03-24T10:49:30.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.426+0000 7f1b24b7c700 1 -- 172.21.15.94:0/955887625 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1b0806d900 msgr2=0x7f1b0806fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:49:30.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.426+0000 7f1b24b7c700 1 --2- 172.21.15.94:0/955887625 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1b0806d900 0x7f1b0806fdb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1b200fcb60 tx=0x7f1b1400b040 comp rx=0 tx=0).stop 2024-03-24T10:49:30.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.426+0000 7f1b24b7c700 1 -- 172.21.15.94:0/955887625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b200fff90 msgr2=0x7f1b200fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:49:30.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.426+0000 7f1b24b7c700 1 --2- 172.21.15.94:0/955887625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b200fff90 0x7f1b200fd3e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1b200668b0 tx=0x7f1b18004d10 comp rx=0 tx=0).stop 2024-03-24T10:49:30.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.426+0000 7f1b24b7c700 1 -- 172.21.15.94:0/955887625 shutdown_connections 2024-03-24T10:49:30.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.426+0000 7f1b24b7c700 1 --2- 172.21.15.94:0/955887625 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1b0806d900 0x7f1b0806fdb0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:49:30.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.426+0000 7f1b24b7c700 1 --2- 172.21.15.94:0/955887625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b200fff90 0x7f1b200fd3e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:49:30.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.426+0000 7f1b24b7c700 1 --2- 172.21.15.94:0/955887625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b200fb950 0x7f1b200fbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:49:30.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.426+0000 7f1b24b7c700 1 -- 172.21.15.94:0/955887625 >> 172.21.15.94:0/955887625 conn(0x7f1b200f74e0 msgr2=0x7f1b200f78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:49:30.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.426+0000 7f1b24b7c700 1 -- 172.21.15.94:0/955887625 shutdown_connections 2024-03-24T10:49:30.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.426+0000 7f1b24b7c700 1 -- 172.21.15.94:0/955887625 wait complete. 2024-03-24T10:49:30.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.533+0000 7fb5dcfc8700 1 -- 172.21.15.94:0/2619614501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5d80fce70 msgr2=0x7fb5d80fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:49:30.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.533+0000 7fb5dcfc8700 1 --2- 172.21.15.94:0/2619614501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5d80fce70 0x7fb5d80fd280 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0004940 tx=0x7fb5d0009e30 comp rx=0 tx=0).stop 2024-03-24T10:49:30.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.533+0000 7fb5dcfc8700 1 -- 172.21.15.94:0/2619614501 shutdown_connections 2024-03-24T10:49:30.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.533+0000 7fb5dcfc8700 1 --2- 172.21.15.94:0/2619614501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5d80fdfe0 0x7fb5d80fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:49:30.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.533+0000 7fb5dcfc8700 1 --2- 172.21.15.94:0/2619614501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5d80fce70 0x7fb5d80fd280 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:49:30.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.533+0000 7fb5dcfc8700 1 -- 172.21.15.94:0/2619614501 >> 172.21.15.94:0/2619614501 conn(0x7fb5d80f8550 msgr2=0x7fb5d80fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:49:30.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.533+0000 7fb5dcfc8700 1 -- 172.21.15.94:0/2619614501 shutdown_connections 2024-03-24T10:49:30.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.533+0000 7fb5dcfc8700 1 -- 172.21.15.94:0/2619614501 wait complete. 2024-03-24T10:49:30.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.534+0000 7fb5dcfc8700 1 Processor -- start 2024-03-24T10:49:30.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.534+0000 7fb5dcfc8700 1 -- start start 2024-03-24T10:49:30.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.534+0000 7fb5dcfc8700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5d80fce70 0x7fb5d810d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:49:30.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.534+0000 7fb5dcfc8700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5d80fdfe0 0x7fb5d810de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:49:30.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.534+0000 7fb5dcfc8700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d810e410 con 0x7fb5d80fce70 2024-03-24T10:49:30.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.534+0000 7fb5dcfc8700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d810e580 con 0x7fb5d80fdfe0 2024-03-24T10:49:30.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.534+0000 7fb5d659c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5d80fce70 0x7fb5d810d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:49:30.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.534+0000 7fb5d659c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5d80fce70 0x7fb5d810d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33320/0 (socket says 172.21.15.94:33320) 2024-03-24T10:49:30.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.534+0000 7fb5d659c700 1 -- 172.21.15.94:0/3743552869 learned_addr learned my addr 172.21.15.94:0/3743552869 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:49:30.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.534+0000 7fb5d5d9b700 1 --2- 172.21.15.94:0/3743552869 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5d80fdfe0 0x7fb5d810de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:49:30.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.534+0000 7fb5d659c700 1 -- 172.21.15.94:0/3743552869 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5d80fdfe0 msgr2=0x7fb5d810de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:49:30.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.534+0000 7fb5d659c700 1 --2- 172.21.15.94:0/3743552869 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5d80fdfe0 0x7fb5d810de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:49:30.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.534+0000 7fb5d659c700 1 -- 172.21.15.94:0/3743552869 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d0005040 con 0x7fb5d80fce70 2024-03-24T10:49:30.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.534+0000 7fb5d659c700 1 --2- 172.21.15.94:0/3743552869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5d80fce70 0x7fb5d810d8d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0013870 tx=0x7fb5d0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:49:30.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.535+0000 7fb5c6ffd700 1 -- 172.21.15.94:0/3743552869 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5d0021070 con 0x7fb5d80fce70 2024-03-24T10:49:30.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.535+0000 7fb5c6ffd700 1 -- 172.21.15.94:0/3743552869 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb5d0009430 con 0x7fb5d80fce70 2024-03-24T10:49:30.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.535+0000 7fb5c6ffd700 1 -- 172.21.15.94:0/3743552869 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5d000e070 con 0x7fb5d80fce70 2024-03-24T10:49:30.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.535+0000 7fb5dcfc8700 1 -- 172.21.15.94:0/3743552869 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5d8112f80 con 0x7fb5d80fce70 2024-03-24T10:49:30.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.535+0000 7fb5dcfc8700 1 -- 172.21.15.94:0/3743552869 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5d8113430 con 0x7fb5d80fce70 2024-03-24T10:49:30.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.536+0000 7fb5dcfc8700 1 -- 172.21.15.94:0/3743552869 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5d804c710 con 0x7fb5d80fce70 2024-03-24T10:49:30.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.536+0000 7fb5c6ffd700 1 -- 172.21.15.94:0/3743552869 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 35) v1 ==== 91587+0+0 (secure 0 0 0) 0x7fb5d001c750 con 0x7fb5d80fce70 2024-03-24T10:49:30.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.536+0000 7fb5c6ffd700 1 --2- 172.21.15.94:0/3743552869 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fb5c006d910 0x7fb5c006fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:49:30.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.537+0000 7fb5c6ffd700 1 -- 172.21.15.94:0/3743552869 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fb5d008e220 con 0x7fb5d80fce70 2024-03-24T10:49:30.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.537+0000 7fb5d5d9b700 1 --2- 172.21.15.94:0/3743552869 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fb5c006d910 0x7fb5c006fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:49:30.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.537+0000 7fb5d5d9b700 1 --2- 172.21.15.94:0/3743552869 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fb5c006d910 0x7fb5c006fdc0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc001470 tx=0x7fb5cc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:49:30.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.541+0000 7fb5c6ffd700 1 -- 172.21.15.94:0/3743552869 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb5d005b440 con 0x7fb5d80fce70 2024-03-24T10:49:30.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:30 smithi096 ceph-mon[84258]: pgmap v30: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:49:30.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.831+0000 7fb5dcfc8700 1 -- 172.21.15.94:0/3743552869 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb5d81136a0 con 0x7fb5d80fce70 2024-03-24T10:49:30.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.832+0000 7fb5c6ffd700 1 -- 172.21.15.94:0/3743552869 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+736 (secure 0 0 0) 0x7fb5d0004490 con 0x7fb5d80fce70 2024-03-24T10:49:30.832 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-03-24T10:49:30.832 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": { 2024-03-24T10:49:30.832 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 2 2024-03-24T10:49:30.832 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:49:30.832 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": { 2024-03-24T10:49:30.832 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 2 2024-03-24T10:49:30.832 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:49:30.832 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": { 2024-03-24T10:49:30.832 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-24T10:49:30.832 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:49:30.832 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-03-24T10:49:30.832 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-24T10:49:30.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:49:30.833 INFO:teuthology.orchestra.run.smithi094.stdout: "overall": { 2024-03-24T10:49:30.833 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 10, 2024-03-24T10:49:30.833 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 4 2024-03-24T10:49:30.833 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-03-24T10:49:30.833 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-03-24T10:49:30.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.834+0000 7fb5dcfc8700 1 -- 172.21.15.94:0/3743552869 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fb5c006d910 msgr2=0x7fb5c006fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:49:30.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.834+0000 7fb5dcfc8700 1 --2- 172.21.15.94:0/3743552869 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fb5c006d910 0x7fb5c006fdc0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc001470 tx=0x7fb5cc00b040 comp rx=0 tx=0).stop 2024-03-24T10:49:30.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.834+0000 7fb5dcfc8700 1 -- 172.21.15.94:0/3743552869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5d80fce70 msgr2=0x7fb5d810d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:49:30.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.834+0000 7fb5dcfc8700 1 --2- 172.21.15.94:0/3743552869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5d80fce70 0x7fb5d810d8d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0013870 tx=0x7fb5d0009e30 comp rx=0 tx=0).stop 2024-03-24T10:49:30.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.834+0000 7fb5dcfc8700 1 -- 172.21.15.94:0/3743552869 shutdown_connections 2024-03-24T10:49:30.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.834+0000 7fb5dcfc8700 1 --2- 172.21.15.94:0/3743552869 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fb5c006d910 0x7fb5c006fdc0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:49:30.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.834+0000 7fb5dcfc8700 1 --2- 172.21.15.94:0/3743552869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5d80fce70 0x7fb5d810d8d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:49:30.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.834+0000 7fb5dcfc8700 1 --2- 172.21.15.94:0/3743552869 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5d80fdfe0 0x7fb5d810de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:49:30.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.834+0000 7fb5dcfc8700 1 -- 172.21.15.94:0/3743552869 >> 172.21.15.94:0/3743552869 conn(0x7fb5d80f8550 msgr2=0x7fb5d80f9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:49:30.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.834+0000 7fb5dcfc8700 1 -- 172.21.15.94:0/3743552869 shutdown_connections 2024-03-24T10:49:30.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.834+0000 7fb5dcfc8700 1 -- 172.21.15.94:0/3743552869 wait complete. 2024-03-24T10:49:30.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.940+0000 7f4c66d72700 1 -- 172.21.15.94:0/2548826617 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c600fcf80 msgr2=0x7f4c600fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:49:30.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.940+0000 7f4c66d72700 1 --2- 172.21.15.94:0/2548826617 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c600fcf80 0x7f4c600fd3f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c004940 tx=0x7f4c4c009e30 comp rx=0 tx=0).stop 2024-03-24T10:49:30.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.940+0000 7f4c66d72700 1 -- 172.21.15.94:0/2548826617 shutdown_connections 2024-03-24T10:49:30.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.940+0000 7f4c66d72700 1 --2- 172.21.15.94:0/2548826617 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c600fcf80 0x7f4c600fd3f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:49:30.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.940+0000 7f4c66d72700 1 --2- 172.21.15.94:0/2548826617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c600fbcd0 0x7f4c600fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:49:30.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.940+0000 7f4c66d72700 1 -- 172.21.15.94:0/2548826617 >> 172.21.15.94:0/2548826617 conn(0x7f4c600f7430 msgr2=0x7f4c600f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:49:30.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.941+0000 7f4c66d72700 1 -- 172.21.15.94:0/2548826617 shutdown_connections 2024-03-24T10:49:30.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.941+0000 7f4c66d72700 1 -- 172.21.15.94:0/2548826617 wait complete. 2024-03-24T10:49:30.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.941+0000 7f4c66d72700 1 Processor -- start 2024-03-24T10:49:30.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.941+0000 7f4c66d72700 1 -- start start 2024-03-24T10:49:30.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.942+0000 7f4c66d72700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c600fbcd0 0x7f4c6010d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:49:30.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.942+0000 7f4c66d72700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c600fcf80 0x7f4c6010dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:49:30.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.942+0000 7f4c66d72700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6010e4c0 con 0x7f4c600fbcd0 2024-03-24T10:49:30.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.942+0000 7f4c66d72700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6010ed10 con 0x7f4c600fcf80 2024-03-24T10:49:30.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.942+0000 7f4c5ffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c600fcf80 0x7f4c6010dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:49:30.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.942+0000 7f4c64b0e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c600fbcd0 0x7f4c6010d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:49:30.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.942+0000 7f4c5ffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c600fcf80 0x7f4c6010dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.94:37638/0 (socket says 172.21.15.94:37638) 2024-03-24T10:49:30.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.942+0000 7f4c5ffff700 1 -- 172.21.15.94:0/4012195884 learned_addr learned my addr 172.21.15.94:0/4012195884 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:49:30.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.942+0000 7f4c5ffff700 1 -- 172.21.15.94:0/4012195884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c600fbcd0 msgr2=0x7f4c6010d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:49:30.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.942+0000 7f4c5ffff700 1 --2- 172.21.15.94:0/4012195884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c600fbcd0 0x7f4c6010d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:49:30.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.942+0000 7f4c5ffff700 1 -- 172.21.15.94:0/4012195884 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c4c005040 con 0x7f4c600fcf80 2024-03-24T10:49:30.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.942+0000 7f4c5ffff700 1 --2- 172.21.15.94:0/4012195884 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c600fcf80 0x7f4c6010dec0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c00b0d0 tx=0x7f4c4c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:49:30.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.943+0000 7f4c5d7fa700 1 -- 172.21.15.94:0/4012195884 <== mon.1 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c4c00e030 con 0x7f4c600fcf80 2024-03-24T10:49:30.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.943+0000 7f4c5d7fa700 1 -- 172.21.15.94:0/4012195884 <== mon.1 v2:172.21.15.96:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4c4c01c400 con 0x7f4c600fcf80 2024-03-24T10:49:30.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.943+0000 7f4c5d7fa700 1 -- 172.21.15.94:0/4012195884 <== mon.1 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c4c00b630 con 0x7f4c600fcf80 2024-03-24T10:49:30.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.943+0000 7f4c66d72700 1 -- 172.21.15.94:0/4012195884 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c6010ef90 con 0x7f4c600fcf80 2024-03-24T10:49:30.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.943+0000 7f4c66d72700 1 -- 172.21.15.94:0/4012195884 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c6010f480 con 0x7f4c600fcf80 2024-03-24T10:49:30.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.944+0000 7f4c66d72700 1 -- 172.21.15.94:0/4012195884 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c60107360 con 0x7f4c600fcf80 2024-03-24T10:49:30.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.944+0000 7f4c5d7fa700 1 -- 172.21.15.94:0/4012195884 <== mon.1 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 35) v1 ==== 91587+0+0 (secure 0 0 0) 0x7f4c4c013af0 con 0x7f4c600fcf80 2024-03-24T10:49:30.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.945+0000 7f4c5d7fa700 1 --2- 172.21.15.94:0/4012195884 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f4c5006d900 0x7f4c5006fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:49:30.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.945+0000 7f4c64b0e700 1 --2- 172.21.15.94:0/4012195884 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f4c5006d900 0x7f4c5006fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:49:30.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.945+0000 7f4c5d7fa700 1 -- 172.21.15.94:0/4012195884 <== mon.1 v2:172.21.15.96:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f4c4c08ea50 con 0x7f4c600fcf80 2024-03-24T10:49:30.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.945+0000 7f4c64b0e700 1 --2- 172.21.15.94:0/4012195884 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f4c5006d900 0x7f4c5006fdb0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4c54009b80 tx=0x7f4c5400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:49:30.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:30.950+0000 7f4c5d7fa700 1 -- 172.21.15.94:0/4012195884 <== mon.1 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4c4c05bbd0 con 0x7f4c600fcf80 2024-03-24T10:49:31.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.188+0000 7f4c66d72700 1 -- 172.21.15.94:0/4012195884 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4c6004bed0 con 0x7f4c600fcf80 2024-03-24T10:49:31.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.189+0000 7f4c5d7fa700 1 -- 172.21.15.94:0/4012195884 <== mon.1 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1903 (secure 0 0 0) 0x7f4c4c004490 con 0x7f4c600fcf80 2024-03-24T10:49:31.189 INFO:teuthology.orchestra.run.smithi094.stdout:e18 2024-03-24T10:49:31.189 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T10:49:31.189 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:49:31.189 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-03-24T10:49:31.189 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:49:31.189 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-03-24T10:49:31.189 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-03-24T10:49:31.190 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 18 2024-03-24T10:49:31.190 INFO:teuthology.orchestra.run.smithi094.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-24T10:49:31.190 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-03-24T10:43:39.435738+0000 2024-03-24T10:49:31.190 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-03-24T10:43:58.133901+0000 2024-03-24T10:49:31.190 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-03-24T10:49:31.190 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-03-24T10:49:31.190 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 60 2024-03-24T10:49:31.190 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-03-24T10:49:31.190 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-03-24T10:49:31.190 INFO:teuthology.orchestra.run.smithi094.stdout:max_xattr_size 65536 2024-03-24T10:49:31.190 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-03-24T10:49:31.191 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-03-24T10:49:31.191 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 46 2024-03-24T10:49:31.191 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:49:31.191 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-03-24T10:49:31.191 INFO:teuthology.orchestra.run.smithi094.stdout:in 0 2024-03-24T10:49:31.191 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=24251} 2024-03-24T10:49:31.191 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-03-24T10:49:31.191 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-03-24T10:49:31.191 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-03-24T10:49:31.191 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-03-24T10:49:31.191 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-03-24T10:49:31.192 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-03-24T10:49:31.192 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-03-24T10:49:31.192 INFO:teuthology.orchestra.run.smithi094.stdout:bal_rank_mask -1 2024-03-24T10:49:31.192 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 1 2024-03-24T10:49:31.192 INFO:teuthology.orchestra.run.smithi094.stdout:qdb_cluster leader: 0 members: 2024-03-24T10:49:31.192 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi096.egbmoq{0:24251} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.96:6824/1539752208,v1:172.21.15.96:6825/1539752208] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:49:31.192 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.llefgu{0:14500} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.94:6828/126762980,v1:172.21.15.94:6829/126762980] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:49:31.192 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:49:31.192 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:49:31.192 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-03-24T10:49:31.192 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:49:31.192 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.efiqte{-1:14512} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.94:6826/1636528860,v1:172.21.15.94:6827/1636528860] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:49:31.193 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi096.jtebhh{-1:24261} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.96:6826/1321087930,v1:172.21.15.96:6827/1321087930] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:49:31.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.191+0000 7f4c66d72700 1 -- 172.21.15.94:0/4012195884 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f4c5006d900 msgr2=0x7f4c5006fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:49:31.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.191+0000 7f4c66d72700 1 --2- 172.21.15.94:0/4012195884 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f4c5006d900 0x7f4c5006fdb0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4c54009b80 tx=0x7f4c5400b040 comp rx=0 tx=0).stop 2024-03-24T10:49:31.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.191+0000 7f4c66d72700 1 -- 172.21.15.94:0/4012195884 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c600fcf80 msgr2=0x7f4c6010dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:49:31.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.191+0000 7f4c66d72700 1 --2- 172.21.15.94:0/4012195884 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c600fcf80 0x7f4c6010dec0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c00b0d0 tx=0x7f4c4c009e30 comp rx=0 tx=0).stop 2024-03-24T10:49:31.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.191+0000 7f4c66d72700 1 -- 172.21.15.94:0/4012195884 shutdown_connections 2024-03-24T10:49:31.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.191+0000 7f4c66d72700 1 --2- 172.21.15.94:0/4012195884 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f4c5006d900 0x7f4c5006fdb0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:49:31.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.191+0000 7f4c66d72700 1 --2- 172.21.15.94:0/4012195884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c600fbcd0 0x7f4c6010d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:49:31.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.191+0000 7f4c66d72700 1 --2- 172.21.15.94:0/4012195884 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c600fcf80 0x7f4c6010dec0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:49:31.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.191+0000 7f4c66d72700 1 -- 172.21.15.94:0/4012195884 >> 172.21.15.94:0/4012195884 conn(0x7f4c600f7430 msgr2=0x7f4c600f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:49:31.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.191+0000 7f4c66d72700 1 -- 172.21.15.94:0/4012195884 shutdown_connections 2024-03-24T10:49:31.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.191+0000 7f4c66d72700 1 -- 172.21.15.94:0/4012195884 wait complete. 2024-03-24T10:49:31.195 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 18 2024-03-24T10:49:31.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.294+0000 7fc0f537a700 1 -- 172.21.15.94:0/2936033508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0f00fcf80 msgr2=0x7fc0f00fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:49:31.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.294+0000 7fc0f537a700 1 --2- 172.21.15.94:0/2936033508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0f00fcf80 0x7fc0f00fd3f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4004940 tx=0x7fc0e4009e30 comp rx=0 tx=0).stop 2024-03-24T10:49:31.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.294+0000 7fc0f537a700 1 -- 172.21.15.94:0/2936033508 shutdown_connections 2024-03-24T10:49:31.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.294+0000 7fc0f537a700 1 --2- 172.21.15.94:0/2936033508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0f00fcf80 0x7fc0f00fd3f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:49:31.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.294+0000 7fc0f537a700 1 --2- 172.21.15.94:0/2936033508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0f00fbcd0 0x7fc0f00fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:49:31.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.294+0000 7fc0f537a700 1 -- 172.21.15.94:0/2936033508 >> 172.21.15.94:0/2936033508 conn(0x7fc0f00f7450 msgr2=0x7fc0f00f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:49:31.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.295+0000 7fc0f537a700 1 -- 172.21.15.94:0/2936033508 shutdown_connections 2024-03-24T10:49:31.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.295+0000 7fc0f537a700 1 -- 172.21.15.94:0/2936033508 wait complete. 2024-03-24T10:49:31.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.295+0000 7fc0f537a700 1 Processor -- start 2024-03-24T10:49:31.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.295+0000 7fc0f537a700 1 -- start start 2024-03-24T10:49:31.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.295+0000 7fc0f537a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0f00fbcd0 0x7fc0f00ffbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:49:31.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.295+0000 7fc0f537a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0f00fcf80 0x7fc0f00fe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:49:31.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.295+0000 7fc0f537a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f01001c0 con 0x7fc0f00fbcd0 2024-03-24T10:49:31.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.295+0000 7fc0f537a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f00fe800 con 0x7fc0f00fcf80 2024-03-24T10:49:31.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.295+0000 7fc0eeffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0f00fbcd0 0x7fc0f00ffbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:49:31.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.295+0000 7fc0eeffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0f00fbcd0 0x7fc0f00ffbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33358/0 (socket says 172.21.15.94:33358) 2024-03-24T10:49:31.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.295+0000 7fc0eeffd700 1 -- 172.21.15.94:0/2021180614 learned_addr learned my addr 172.21.15.94:0/2021180614 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:49:31.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.296+0000 7fc0ee7fc700 1 --2- 172.21.15.94:0/2021180614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0f00fcf80 0x7fc0f00fe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:49:31.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.296+0000 7fc0eeffd700 1 -- 172.21.15.94:0/2021180614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0f00fcf80 msgr2=0x7fc0f00fe2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:49:31.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.296+0000 7fc0eeffd700 1 --2- 172.21.15.94:0/2021180614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0f00fcf80 0x7fc0f00fe2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:49:31.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.296+0000 7fc0eeffd700 1 -- 172.21.15.94:0/2021180614 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e4005040 con 0x7fc0f00fbcd0 2024-03-24T10:49:31.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.296+0000 7fc0eeffd700 1 --2- 172.21.15.94:0/2021180614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0f00fbcd0 0x7fc0f00ffbc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc0e000eb40 tx=0x7fc0e000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:49:31.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.296+0000 7fc0d77fe700 1 -- 172.21.15.94:0/2021180614 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0e0007cc0 con 0x7fc0f00fbcd0 2024-03-24T10:49:31.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.296+0000 7fc0d77fe700 1 -- 172.21.15.94:0/2021180614 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc0e000f040 con 0x7fc0f00fbcd0 2024-03-24T10:49:31.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.296+0000 7fc0d77fe700 1 -- 172.21.15.94:0/2021180614 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0e000ca10 con 0x7fc0f00fbcd0 2024-03-24T10:49:31.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.296+0000 7fc0f537a700 1 -- 172.21.15.94:0/2021180614 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0f00fea80 con 0x7fc0f00fbcd0 2024-03-24T10:49:31.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.296+0000 7fc0f537a700 1 -- 172.21.15.94:0/2021180614 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0f00fefc0 con 0x7fc0f00fbcd0 2024-03-24T10:49:31.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.297+0000 7fc0f537a700 1 -- 172.21.15.94:0/2021180614 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0f004c710 con 0x7fc0f00fbcd0 2024-03-24T10:49:31.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.298+0000 7fc0d77fe700 1 -- 172.21.15.94:0/2021180614 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 35) v1 ==== 91587+0+0 (secure 0 0 0) 0x7fc0e000cb70 con 0x7fc0f00fbcd0 2024-03-24T10:49:31.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.298+0000 7fc0d77fe700 1 --2- 172.21.15.94:0/2021180614 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fc0d806d910 0x7fc0d806fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:49:31.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.298+0000 7fc0d77fe700 1 -- 172.21.15.94:0/2021180614 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fc0e008cca0 con 0x7fc0f00fbcd0 2024-03-24T10:49:31.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.298+0000 7fc0ee7fc700 1 --2- 172.21.15.94:0/2021180614 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fc0d806d910 0x7fc0d806fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:49:31.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.298+0000 7fc0ee7fc700 1 --2- 172.21.15.94:0/2021180614 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fc0d806d910 0x7fc0d806fdc0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4009b30 tx=0x7fc0e4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:49:31.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.303+0000 7fc0d77fe700 1 -- 172.21.15.94:0/2021180614 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc0e0091070 con 0x7fc0f00fbcd0 2024-03-24T10:49:31.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:31 smithi094 ceph-mon[93957]: from='client.34158 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:49:31.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:31 smithi094 ceph-mon[93957]: from='client.34162 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:49:31.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:31 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/3743552869' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:49:31.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:31 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:31.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:31 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:31.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:31 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/4012195884' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T10:49:31.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.520+0000 7fc0f537a700 1 -- 172.21.15.94:0/2021180614 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc0f005e400 con 0x7fc0d806d910 2024-03-24T10:49:31.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.522+0000 7fc0d77fe700 1 -- 172.21.15.94:0/2021180614 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7fc0f005e400 con 0x7fc0d806d910 2024-03-24T10:49:31.521 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-03-24T10:49:31.522 INFO:teuthology.orchestra.run.smithi094.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919", 2024-03-24T10:49:31.522 INFO:teuthology.orchestra.run.smithi094.stdout: "in_progress": true, 2024-03-24T10:49:31.522 INFO:teuthology.orchestra.run.smithi094.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T10:49:31.522 INFO:teuthology.orchestra.run.smithi094.stdout: "services_complete": [ 2024-03-24T10:49:31.522 INFO:teuthology.orchestra.run.smithi094.stdout: "mon", 2024-03-24T10:49:31.522 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr" 2024-03-24T10:49:31.522 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-03-24T10:49:31.522 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": "4/21 daemons upgraded", 2024-03-24T10:49:31.522 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "Currently upgrading crash daemons", 2024-03-24T10:49:31.522 INFO:teuthology.orchestra.run.smithi094.stdout: "is_paused": false 2024-03-24T10:49:31.522 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-03-24T10:49:31.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.524+0000 7fc0f537a700 1 -- 172.21.15.94:0/2021180614 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fc0d806d910 msgr2=0x7fc0d806fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:49:31.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.524+0000 7fc0f537a700 1 --2- 172.21.15.94:0/2021180614 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fc0d806d910 0x7fc0d806fdc0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4009b30 tx=0x7fc0e4009e30 comp rx=0 tx=0).stop 2024-03-24T10:49:31.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.524+0000 7fc0f537a700 1 -- 172.21.15.94:0/2021180614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0f00fbcd0 msgr2=0x7fc0f00ffbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:49:31.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.524+0000 7fc0f537a700 1 --2- 172.21.15.94:0/2021180614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0f00fbcd0 0x7fc0f00ffbc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc0e000eb40 tx=0x7fc0e000b040 comp rx=0 tx=0).stop 2024-03-24T10:49:31.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.524+0000 7fc0f537a700 1 -- 172.21.15.94:0/2021180614 shutdown_connections 2024-03-24T10:49:31.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.524+0000 7fc0f537a700 1 --2- 172.21.15.94:0/2021180614 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fc0d806d910 0x7fc0d806fdc0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:49:31.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.524+0000 7fc0f537a700 1 --2- 172.21.15.94:0/2021180614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0f00fbcd0 0x7fc0f00ffbc0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:49:31.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.524+0000 7fc0f537a700 1 --2- 172.21.15.94:0/2021180614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0f00fcf80 0x7fc0f00fe2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:49:31.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.524+0000 7fc0f537a700 1 -- 172.21.15.94:0/2021180614 >> 172.21.15.94:0/2021180614 conn(0x7fc0f00f7450 msgr2=0x7fc0f00667c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:49:31.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.524+0000 7fc0f537a700 1 -- 172.21.15.94:0/2021180614 shutdown_connections 2024-03-24T10:49:31.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.524+0000 7fc0f537a700 1 -- 172.21.15.94:0/2021180614 wait complete. 2024-03-24T10:49:31.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:31 smithi096 ceph-mon[84258]: from='client.34158 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:49:31.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:31 smithi096 ceph-mon[84258]: from='client.34162 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:49:31.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:31 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/3743552869' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:49:31.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:31 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:31.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:31 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:31.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:31 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/4012195884' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T10:49:31.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.624+0000 7f167a778700 1 -- 172.21.15.94:0/3794347330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16740fed80 msgr2=0x7f16740faec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:49:31.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.624+0000 7f167a778700 1 --2- 172.21.15.94:0/3794347330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16740fed80 0x7f16740faec0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1668004940 tx=0x7f1668009e30 comp rx=0 tx=0).stop 2024-03-24T10:49:31.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.624+0000 7f167a778700 1 -- 172.21.15.94:0/3794347330 shutdown_connections 2024-03-24T10:49:31.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.624+0000 7f167a778700 1 --2- 172.21.15.94:0/3794347330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16740fed80 0x7f16740faec0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:49:31.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.624+0000 7f167a778700 1 --2- 172.21.15.94:0/3794347330 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16740fe430 0x7f16740fe840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:49:31.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.624+0000 7f167a778700 1 -- 172.21.15.94:0/3794347330 >> 172.21.15.94:0/3794347330 conn(0x7f16740f6bd0 msgr2=0x7f16740f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:49:31.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.624+0000 7f167a778700 1 -- 172.21.15.94:0/3794347330 shutdown_connections 2024-03-24T10:49:31.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.624+0000 7f167a778700 1 -- 172.21.15.94:0/3794347330 wait complete. 2024-03-24T10:49:31.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.624+0000 7f167a778700 1 Processor -- start 2024-03-24T10:49:31.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.624+0000 7f167a778700 1 -- start start 2024-03-24T10:49:31.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.625+0000 7f167a778700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16740fe430 0x7f167410d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:49:31.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.625+0000 7f167a778700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16740fed80 0x7f167410ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:49:31.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.625+0000 7f167a778700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167410e3b0 con 0x7f16740fed80 2024-03-24T10:49:31.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.625+0000 7f167a778700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167410e520 con 0x7f16740fe430 2024-03-24T10:49:31.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.625+0000 7f16737fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16740fed80 0x7f167410ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:49:31.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.625+0000 7f16737fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16740fed80 0x7f167410ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33388/0 (socket says 172.21.15.94:33388) 2024-03-24T10:49:31.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.625+0000 7f16737fe700 1 -- 172.21.15.94:0/1154697879 learned_addr learned my addr 172.21.15.94:0/1154697879 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:49:31.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.625+0000 7f1673fff700 1 --2- 172.21.15.94:0/1154697879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16740fe430 0x7f167410d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:49:31.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.625+0000 7f16737fe700 1 -- 172.21.15.94:0/1154697879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16740fe430 msgr2=0x7f167410d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:49:31.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.625+0000 7f16737fe700 1 --2- 172.21.15.94:0/1154697879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16740fe430 0x7f167410d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:49:31.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.625+0000 7f16737fe700 1 -- 172.21.15.94:0/1154697879 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1668005040 con 0x7f16740fed80 2024-03-24T10:49:31.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.625+0000 7f16737fe700 1 --2- 172.21.15.94:0/1154697879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16740fed80 0x7f167410ddb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1668013870 tx=0x7f1668009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:49:31.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.625+0000 7f1670ff9700 1 -- 172.21.15.94:0/1154697879 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1668021070 con 0x7f16740fed80 2024-03-24T10:49:31.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.625+0000 7f1670ff9700 1 -- 172.21.15.94:0/1154697879 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1668009430 con 0x7f16740fed80 2024-03-24T10:49:31.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.625+0000 7f167a778700 1 -- 172.21.15.94:0/1154697879 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1674112f20 con 0x7f16740fed80 2024-03-24T10:49:31.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.625+0000 7f1670ff9700 1 -- 172.21.15.94:0/1154697879 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f166800e070 con 0x7f16740fed80 2024-03-24T10:49:31.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.625+0000 7f167a778700 1 -- 172.21.15.94:0/1154697879 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1674113430 con 0x7f16740fed80 2024-03-24T10:49:31.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.626+0000 7f167a778700 1 -- 172.21.15.94:0/1154697879 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f167404c710 con 0x7f16740fed80 2024-03-24T10:49:31.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.627+0000 7f1670ff9700 1 -- 172.21.15.94:0/1154697879 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 35) v1 ==== 91587+0+0 (secure 0 0 0) 0x7f166801c750 con 0x7f16740fed80 2024-03-24T10:49:31.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.627+0000 7f1670ff9700 1 --2- 172.21.15.94:0/1154697879 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f165c06d8a0 0x7f165c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:49:31.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.627+0000 7f1670ff9700 1 -- 172.21.15.94:0/1154697879 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f166808e2b0 con 0x7f16740fed80 2024-03-24T10:49:31.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.627+0000 7f1673fff700 1 --2- 172.21.15.94:0/1154697879 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f165c06d8a0 0x7f165c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:49:31.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.628+0000 7f1673fff700 1 --2- 172.21.15.94:0/1154697879 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f165c06d8a0 0x7f165c06fd50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1664009b80 tx=0x7f166400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:49:31.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.632+0000 7f1670ff9700 1 -- 172.21.15.94:0/1154697879 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f166805b430 con 0x7f16740fed80 2024-03-24T10:49:31.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.910+0000 7f167a778700 1 -- 172.21.15.94:0/1154697879 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f16741058e0 con 0x7f16740fed80 2024-03-24T10:49:31.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.910+0000 7f1670ff9700 1 -- 172.21.15.94:0/1154697879 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f166805afc0 con 0x7f16740fed80 2024-03-24T10:49:31.911 INFO:teuthology.orchestra.run.smithi094.stdout:HEALTH_OK 2024-03-24T10:49:31.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.913+0000 7f167a778700 1 -- 172.21.15.94:0/1154697879 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f165c06d8a0 msgr2=0x7f165c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:49:31.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.913+0000 7f167a778700 1 --2- 172.21.15.94:0/1154697879 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f165c06d8a0 0x7f165c06fd50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1664009b80 tx=0x7f166400b040 comp rx=0 tx=0).stop 2024-03-24T10:49:31.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.913+0000 7f167a778700 1 -- 172.21.15.94:0/1154697879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16740fed80 msgr2=0x7f167410ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:49:31.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.913+0000 7f167a778700 1 --2- 172.21.15.94:0/1154697879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16740fed80 0x7f167410ddb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1668013870 tx=0x7f1668009e30 comp rx=0 tx=0).stop 2024-03-24T10:49:31.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.913+0000 7f167a778700 1 -- 172.21.15.94:0/1154697879 shutdown_connections 2024-03-24T10:49:31.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.913+0000 7f167a778700 1 --2- 172.21.15.94:0/1154697879 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f165c06d8a0 0x7f165c06fd50 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:49:31.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.913+0000 7f167a778700 1 --2- 172.21.15.94:0/1154697879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16740fe430 0x7f167410d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:49:31.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.913+0000 7f167a778700 1 --2- 172.21.15.94:0/1154697879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16740fed80 0x7f167410ddb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:49:31.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.913+0000 7f167a778700 1 -- 172.21.15.94:0/1154697879 >> 172.21.15.94:0/1154697879 conn(0x7f16740f6bd0 msgr2=0x7f16740f8180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:49:31.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.913+0000 7f167a778700 1 -- 172.21.15.94:0/1154697879 shutdown_connections 2024-03-24T10:49:31.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:49:31.914+0000 7f167a778700 1 -- 172.21.15.94:0/1154697879 wait complete. 2024-03-24T10:49:32.421 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:32 smithi096 ceph-mon[84258]: from='client.34166 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:49:32.422 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:32 smithi096 ceph-mon[84258]: pgmap v31: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:49:32.422 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:32 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/1154697879' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T10:49:32.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:32 smithi094 ceph-mon[93957]: from='client.34166 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:49:32.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:32 smithi094 ceph-mon[93957]: pgmap v31: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:49:32.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:32 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/1154697879' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T10:49:33.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:33 smithi096 ceph-mon[84258]: from='client.34178 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:49:33.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:33 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:33.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:33 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi096", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-24T10:49:33.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:33 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:49:33.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:33 smithi094 ceph-mon[93957]: from='client.34178 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:49:33.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:33 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:33.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:33 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi096", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-24T10:49:33.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:33 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:49:34.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:34 smithi096 ceph-mon[84258]: Upgrade: Updating crash.smithi096 (2/2) 2024-03-24T10:49:34.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:34 smithi096 ceph-mon[84258]: Deploying daemon crash.smithi096 on smithi096 2024-03-24T10:49:34.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:34 smithi096 ceph-mon[84258]: pgmap v32: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:49:34.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:34 smithi094 ceph-mon[93957]: Upgrade: Updating crash.smithi096 (2/2) 2024-03-24T10:49:34.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:34 smithi094 ceph-mon[93957]: Deploying daemon crash.smithi096 on smithi096 2024-03-24T10:49:34.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:34 smithi094 ceph-mon[93957]: pgmap v32: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:49:36.338 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:36 smithi096 ceph-mon[84258]: pgmap v33: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:49:36.338 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:36 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:49:36.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:36 smithi094 ceph-mon[93957]: pgmap v33: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:49:36.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:36 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:49:37.384 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:37 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:37.384 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:37 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:37.384 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:37 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:49:37.385 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:37 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:37.385 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:37 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:37.666 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:37 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:37.666 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:37 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:37.666 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:37 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:49:37.666 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:37 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:37.666 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:37 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:38.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:38 smithi096 ceph-mon[84258]: pgmap v34: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:49:38.581 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:38 smithi094 ceph-mon[93957]: pgmap v34: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:49:40.675 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:40 smithi094 ceph-mon[93957]: pgmap v35: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:49:40.676 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:40 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:40.676 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:40 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:40.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:40 smithi096 ceph-mon[84258]: pgmap v35: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:49:40.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:40 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:40.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:40 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:42.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:41 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:42.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:41 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:42.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:41 smithi096 ceph-mon[84258]: pgmap v36: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:49:42.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:41 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:42.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:41 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:42.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:41 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:42.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:41 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:42.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:41 smithi094 ceph-mon[93957]: pgmap v36: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:49:42.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:41 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:42.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:41 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:43.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:43 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:43.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:43 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:43.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:43 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:43.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:43 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:44.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:44 smithi096 ceph-mon[84258]: pgmap v37: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:49:44.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:44 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:44.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:44 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:44.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:44 smithi094 ceph-mon[93957]: pgmap v37: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:49:44.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:44 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:44.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:44 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:46.676 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:46 smithi094 ceph-mon[93957]: pgmap v38: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:49:46.676 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:46 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:46.676 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:46 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:46.676 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:46 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:49:46.677 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:46 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:49:46.677 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:46 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:46.677 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:46 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:49:46.677 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:46 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:49:46.677 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:46 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:49:46.677 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:46 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:49:46.677 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:46 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:46.677 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:46 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi094"}]: dispatch 2024-03-24T10:49:46.677 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:46 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi094"}]': finished 2024-03-24T10:49:46.677 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:46 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi096"}]: dispatch 2024-03-24T10:49:46.677 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:46 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi096"}]': finished 2024-03-24T10:49:46.677 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:46 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-24T10:49:46.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:46 smithi096 ceph-mon[84258]: pgmap v38: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:49:46.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:46 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:46.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:46 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:46.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:46 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:49:46.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:46 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:49:46.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:46 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:46.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:46 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:49:46.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:46 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:49:46.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:46 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:49:46.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:46 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:49:46.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:46 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:46.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:46 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi094"}]: dispatch 2024-03-24T10:49:46.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:46 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi094"}]': finished 2024-03-24T10:49:46.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:46 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi096"}]: dispatch 2024-03-24T10:49:46.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:46 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi096"}]': finished 2024-03-24T10:49:46.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:46 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-24T10:49:47.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:47 smithi094 ceph-mon[93957]: Upgrade: Setting container_image for all crash 2024-03-24T10:49:47.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:47 smithi094 ceph-mon[93957]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-24T10:49:47.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:47 smithi094 ceph-mon[93957]: Upgrade: osd.0 is safe to restart 2024-03-24T10:49:47.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:47 smithi096 ceph-mon[84258]: Upgrade: Setting container_image for all crash 2024-03-24T10:49:47.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:47 smithi096 ceph-mon[84258]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-24T10:49:47.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:47 smithi096 ceph-mon[84258]: Upgrade: osd.0 is safe to restart 2024-03-24T10:49:48.659 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:48 smithi094 ceph-mon[93957]: pgmap v39: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:49:48.659 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:48 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:48.659 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:48 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-24T10:49:48.659 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:48 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:49:48.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:48 smithi096 ceph-mon[84258]: pgmap v39: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:49:48.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:48 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:48.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:48 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-24T10:49:48.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:48 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:49:49.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:49 smithi094 ceph-mon[93957]: Upgrade: Updating osd.0 2024-03-24T10:49:49.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:49 smithi094 ceph-mon[93957]: Deploying daemon osd.0 on smithi094 2024-03-24T10:49:49.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:49 smithi096 ceph-mon[84258]: Upgrade: Updating osd.0 2024-03-24T10:49:49.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:49 smithi096 ceph-mon[84258]: Deploying daemon osd.0 on smithi094 2024-03-24T10:49:50.689 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:50 smithi094 ceph-mon[93957]: pgmap v40: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-24T10:49:50.689 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:50 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:49:50.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:50 smithi096 ceph-mon[84258]: pgmap v40: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-24T10:49:50.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:50 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:49:50.942 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:49:50 smithi094 systemd[1]: Stopping Ceph osd.0 for 386110d4-e9ca-11ee-95cd-87774f69a715... 2024-03-24T10:49:51.247 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:49:50 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-0[47727]: 2024-03-24T10:49:50.941+0000 7fcc88172700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T10:49:51.247 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:49:50 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-0[47727]: 2024-03-24T10:49:50.941+0000 7fcc88172700 -1 osd.0 49 *** Got signal Terminated *** 2024-03-24T10:49:51.247 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:49:50 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-0[47727]: 2024-03-24T10:49:50.941+0000 7fcc88172700 -1 osd.0 49 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T10:49:51.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:51 smithi094 ceph-mon[93957]: osd.0 marked itself down and dead 2024-03-24T10:49:51.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:51 smithi094 ceph-mon[93957]: pgmap v41: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-24T10:49:51.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:51 smithi096 ceph-mon[84258]: osd.0 marked itself down and dead 2024-03-24T10:49:51.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:51 smithi096 ceph-mon[84258]: pgmap v41: 65 pgs: 65 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-24T10:49:52.247 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:49:51 smithi094 podman[102214]: 2024-03-24 10:49:51.972947099 +0000 UTC m=+1.144528293 container died b664fd3557bd0e22fdc8f556ea069fedd1622aaf7f443f261a46e3033637d5a6 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-9c4e0c3, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319) 2024-03-24T10:49:52.526 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:52 smithi094 ceph-mon[93957]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T10:49:52.526 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:52 smithi094 ceph-mon[93957]: failed to encode map e50 with expected crc 2024-03-24T10:49:52.526 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:52 smithi094 ceph-mon[93957]: osdmap e50: 6 total, 5 up, 6 in 2024-03-24T10:49:52.526 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:49:52 smithi094 podman[102214]: 2024-03-24 10:49:52.266765684 +0000 UTC m=+1.438346919 container cleanup b664fd3557bd0e22fdc8f556ea069fedd1622aaf7f443f261a46e3033637d5a6 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-0, RELEASE=quincy-9c4e0c3, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T10:49:52.526 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:49:52 smithi094 bash[102214]: ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-0 2024-03-24T10:49:52.824 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:49:52 smithi094 podman[102231]: 2024-03-24 10:49:52.52667795 +0000 UTC m=+0.555447091 container remove b664fd3557bd0e22fdc8f556ea069fedd1622aaf7f443f261a46e3033637d5a6 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, RELEASE=quincy-9c4e0c3, ceph=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-24T10:49:52.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:52 smithi096 ceph-mon[84258]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T10:49:52.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:52 smithi096 ceph-mon[84258]: failed to encode map e50 with expected crc 2024-03-24T10:49:52.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:52 smithi096 ceph-mon[84258]: osdmap e50: 6 total, 5 up, 6 in 2024-03-24T10:49:53.121 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:49:52 smithi094 podman[102322]: 2024-03-24 10:49:52.728271146 +0000 UTC m=+0.017965774 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-24T10:49:53.121 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:49:52 smithi094 podman[102322]: 2024-03-24 10:49:52.827570219 +0000 UTC m=+0.117264823 container create 3ae3b569a5682f1975a71e6f77ba27efc038bcb02fe5d2a0f94973c6fecd107c (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T10:49:53.121 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:49:53 smithi094 podman[102322]: 2024-03-24 10:49:53.04696244 +0000 UTC m=+0.336657049 container init 3ae3b569a5682f1975a71e6f77ba27efc038bcb02fe5d2a0f94973c6fecd107c (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-0-deactivate, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T10:49:53.122 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:49:53 smithi094 podman[102322]: 2024-03-24 10:49:53.05027561 +0000 UTC m=+0.339970219 container start 3ae3b569a5682f1975a71e6f77ba27efc038bcb02fe5d2a0f94973c6fecd107c (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-24T10:49:53.122 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:49:53 smithi094 podman[102322]: 2024-03-24 10:49:53.121896674 +0000 UTC m=+0.411591286 container attach 3ae3b569a5682f1975a71e6f77ba27efc038bcb02fe5d2a0f94973c6fecd107c (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-0-deactivate, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0) 2024-03-24T10:49:53.446 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:49:53 smithi094 podman[102322]: 2024-03-24 10:49:53.33992559 +0000 UTC m=+0.629620200 container died 3ae3b569a5682f1975a71e6f77ba27efc038bcb02fe5d2a0f94973c6fecd107c (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-24T10:49:53.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:53 smithi094 ceph-mon[93957]: failed to encode map e51 with expected crc 2024-03-24T10:49:53.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:53 smithi094 ceph-mon[93957]: osdmap e51: 6 total, 5 up, 6 in 2024-03-24T10:49:53.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:53 smithi094 ceph-mon[93957]: failed to encode map e51 with expected crc 2024-03-24T10:49:53.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:53 smithi094 ceph-mon[93957]: failed to encode map e51 with expected crc 2024-03-24T10:49:53.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:53 smithi094 ceph-mon[93957]: failed to encode map e51 with expected crc 2024-03-24T10:49:53.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:53 smithi094 ceph-mon[93957]: failed to encode map e51 with expected crc 2024-03-24T10:49:53.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:53 smithi094 ceph-mon[93957]: pgmap v44: 65 pgs: 9 stale+active+clean, 56 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-03-24T10:49:53.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:53 smithi096 ceph-mon[84258]: failed to encode map e51 with expected crc 2024-03-24T10:49:53.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:53 smithi096 ceph-mon[84258]: osdmap e51: 6 total, 5 up, 6 in 2024-03-24T10:49:53.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:53 smithi096 ceph-mon[84258]: failed to encode map e51 with expected crc 2024-03-24T10:49:53.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:53 smithi096 ceph-mon[84258]: failed to encode map e51 with expected crc 2024-03-24T10:49:53.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:53 smithi096 ceph-mon[84258]: failed to encode map e51 with expected crc 2024-03-24T10:49:53.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:53 smithi096 ceph-mon[84258]: failed to encode map e51 with expected crc 2024-03-24T10:49:53.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:53 smithi096 ceph-mon[84258]: pgmap v44: 65 pgs: 9 stale+active+clean, 56 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-03-24T10:49:54.247 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:49:53 smithi094 podman[102347]: 2024-03-24 10:49:53.876533544 +0000 UTC m=+0.531321021 container remove 3ae3b569a5682f1975a71e6f77ba27efc038bcb02fe5d2a0f94973c6fecd107c (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319) 2024-03-24T10:49:54.248 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:49:53 smithi094 systemd[1]: ceph-386110d4-e9ca-11ee-95cd-87774f69a715@osd.0.service: Deactivated successfully. 2024-03-24T10:49:54.248 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:49:53 smithi094 systemd[1]: Stopped Ceph osd.0 for 386110d4-e9ca-11ee-95cd-87774f69a715. 2024-03-24T10:49:54.248 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:49:53 smithi094 systemd[1]: ceph-386110d4-e9ca-11ee-95cd-87774f69a715@osd.0.service: Consumed 59.933s CPU time. 2024-03-24T10:49:54.248 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:49:54 smithi094 systemd[1]: Starting Ceph osd.0 for 386110d4-e9ca-11ee-95cd-87774f69a715... 2024-03-24T10:49:54.630 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:54 smithi094 ceph-mon[93957]: failed to encode map e51 with expected crc 2024-03-24T10:49:54.630 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:49:54 smithi094 podman[102451]: 2024-03-24 10:49:54.225449368 +0000 UTC m=+0.017825632 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-24T10:49:54.630 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:49:54 smithi094 podman[102451]: 2024-03-24 10:49:54.344668125 +0000 UTC m=+0.137044382 container create a95ed5bff2954cc88b954c76106551c224ffe3c281c55bb2649fe364eddfc822 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-0-activate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-24T10:49:54.630 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:49:54 smithi094 podman[102451]: 2024-03-24 10:49:54.555665503 +0000 UTC m=+0.348041771 container init a95ed5bff2954cc88b954c76106551c224ffe3c281c55bb2649fe364eddfc822 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-0-activate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-24T10:49:54.630 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:49:54 smithi094 podman[102451]: 2024-03-24 10:49:54.558989791 +0000 UTC m=+0.351366054 container start a95ed5bff2954cc88b954c76106551c224ffe3c281c55bb2649fe364eddfc822 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-0-activate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T10:49:54.630 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:49:54 smithi094 podman[102451]: 2024-03-24 10:49:54.630600472 +0000 UTC m=+0.422976741 container attach a95ed5bff2954cc88b954c76106551c224ffe3c281c55bb2649fe364eddfc822 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-0-activate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=) 2024-03-24T10:49:54.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:54 smithi096 ceph-mon[84258]: failed to encode map e51 with expected crc 2024-03-24T10:49:55.351 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:49:55 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-0-activate[102465]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-24T10:49:55.351 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:49:55 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-0-activate[102465]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-24T10:49:55.351 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:49:55 smithi094 bash[102451]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-24T10:49:55.351 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:49:55 smithi094 bash[102451]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-24T10:49:55.351 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:49:55 smithi094 bash[102451]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-24T10:49:55.351 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:49:55 smithi094 bash[102451]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-24T10:49:55.351 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:49:55 smithi094 bash[102451]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-03-24T10:49:55.351 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:49:55 smithi094 bash[102451]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-24T10:49:55.351 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:49:55 smithi094 bash[102451]: --> ceph-volume raw activate successful for osd ID: 0 2024-03-24T10:49:55.351 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:49:55 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-0-activate[102465]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-24T10:49:55.351 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:49:55 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-0-activate[102465]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-24T10:49:55.351 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:49:55 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-0-activate[102465]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-03-24T10:49:55.352 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:49:55 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-0-activate[102465]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-24T10:49:55.352 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:49:55 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-0-activate[102465]: --> ceph-volume raw activate successful for osd ID: 0 2024-03-24T10:49:55.352 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:49:55 smithi094 podman[102451]: 2024-03-24 10:49:55.02841026 +0000 UTC m=+0.820786517 container died a95ed5bff2954cc88b954c76106551c224ffe3c281c55bb2649fe364eddfc822 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-0-activate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True) 2024-03-24T10:49:55.651 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:55 smithi094 ceph-mon[93957]: pgmap v45: 65 pgs: 9 stale+active+clean, 56 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-03-24T10:49:55.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:55 smithi096 ceph-mon[84258]: pgmap v45: 65 pgs: 9 stale+active+clean, 56 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-03-24T10:49:55.979 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:49:55 smithi094 podman[102589]: 2024-03-24 10:49:55.652087912 +0000 UTC m=+0.618694343 container remove a95ed5bff2954cc88b954c76106551c224ffe3c281c55bb2649fe364eddfc822 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-0-activate, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0) 2024-03-24T10:49:55.979 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:49:55 smithi094 podman[102650]: 2024-03-24 10:49:55.768004834 +0000 UTC m=+0.018491233 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-24T10:49:55.979 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:49:55 smithi094 podman[102650]: 2024-03-24 10:49:55.878042795 +0000 UTC m=+0.128529185 container create ec57ede0c9e45d834b70169db96f6d70e295e7936c8a7337eb0f7bbd6b3f6b4e (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319) 2024-03-24T10:49:56.230 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:49:56 smithi094 podman[102650]: 2024-03-24 10:49:56.139019004 +0000 UTC m=+0.389505397 container init ec57ede0c9e45d834b70169db96f6d70e295e7936c8a7337eb0f7bbd6b3f6b4e (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-0, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6) 2024-03-24T10:49:56.230 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:49:56 smithi094 podman[102650]: 2024-03-24 10:49:56.142058022 +0000 UTC m=+0.392544418 container start ec57ede0c9e45d834b70169db96f6d70e295e7936c8a7337eb0f7bbd6b3f6b4e (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-24T10:49:56.497 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:49:56 smithi094 bash[102650]: ec57ede0c9e45d834b70169db96f6d70e295e7936c8a7337eb0f7bbd6b3f6b4e 2024-03-24T10:49:56.497 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:49:56 smithi094 systemd[1]: Started Ceph osd.0 for 386110d4-e9ca-11ee-95cd-87774f69a715. 2024-03-24T10:49:56.997 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:49:56 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-0[102665]: 2024-03-24T10:49:56.687+0000 7ff3d684a7c0 -1 Falling back to public interface 2024-03-24T10:49:57.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:57 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:57.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:57 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:57.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:57 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:49:57.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:57 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:57.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:57 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:49:57.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:57 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:49:58.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:58 smithi096 ceph-mon[84258]: pgmap v46: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 44/219 objects degraded (20.091%) 2024-03-24T10:49:58.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:49:58 smithi096 ceph-mon[84258]: Health check failed: Degraded data redundancy: 44/219 objects degraded (20.091%), 18 pgs degraded (PG_DEGRADED) 2024-03-24T10:49:58.630 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:58 smithi094 ceph-mon[93957]: pgmap v46: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 44/219 objects degraded (20.091%) 2024-03-24T10:49:58.630 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:49:58 smithi094 ceph-mon[93957]: Health check failed: Degraded data redundancy: 44/219 objects degraded (20.091%), 18 pgs degraded (PG_DEGRADED) 2024-03-24T10:50:00.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:00 smithi094 ceph-mon[93957]: pgmap v47: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 44/219 objects degraded (20.091%) 2024-03-24T10:50:00.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:00 smithi094 ceph-mon[93957]: Health detail: HEALTH_WARN 1 osds down; Degraded data redundancy: 44/219 objects degraded (20.091%), 18 pgs degraded 2024-03-24T10:50:00.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:00 smithi094 ceph-mon[93957]: [WRN] OSD_DOWN: 1 osds down 2024-03-24T10:50:00.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:00 smithi094 ceph-mon[93957]: osd.0 (root=default,host=smithi094) is down 2024-03-24T10:50:00.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:00 smithi094 ceph-mon[93957]: [WRN] PG_DEGRADED: Degraded data redundancy: 44/219 objects degraded (20.091%), 18 pgs degraded 2024-03-24T10:50:00.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:00 smithi094 ceph-mon[93957]: pg 1.0 is active+undersized+degraded, acting [3,1] 2024-03-24T10:50:00.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:00 smithi094 ceph-mon[93957]: pg 2.0 is active+undersized+degraded, acting [3,1] 2024-03-24T10:50:00.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:00 smithi094 ceph-mon[93957]: pg 2.1 is active+undersized+degraded, acting [2,1] 2024-03-24T10:50:00.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:00 smithi094 ceph-mon[93957]: pg 2.2 is active+undersized+degraded, acting [5,1] 2024-03-24T10:50:00.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:00 smithi094 ceph-mon[93957]: pg 2.4 is active+undersized+degraded, acting [1,4] 2024-03-24T10:50:00.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:00 smithi094 ceph-mon[93957]: pg 2.5 is active+undersized+degraded, acting [3,4] 2024-03-24T10:50:00.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:00 smithi094 ceph-mon[93957]: pg 2.9 is active+undersized+degraded, acting [1,4] 2024-03-24T10:50:00.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:00 smithi094 ceph-mon[93957]: pg 2.c is active+undersized+degraded, acting [2,3] 2024-03-24T10:50:00.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:00 smithi094 ceph-mon[93957]: pg 2.e is active+undersized+degraded, acting [2,3] 2024-03-24T10:50:00.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:00 smithi094 ceph-mon[93957]: pg 2.f is active+undersized+degraded, acting [4,5] 2024-03-24T10:50:00.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:00 smithi094 ceph-mon[93957]: pg 2.10 is active+undersized+degraded, acting [2,1] 2024-03-24T10:50:00.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:00 smithi094 ceph-mon[93957]: pg 2.13 is active+undersized+degraded, acting [4,2] 2024-03-24T10:50:00.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:00 smithi094 ceph-mon[93957]: pg 2.15 is active+undersized+degraded, acting [1,3] 2024-03-24T10:50:00.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:00 smithi094 ceph-mon[93957]: pg 2.19 is active+undersized+degraded, acting [4,2] 2024-03-24T10:50:00.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:00 smithi094 ceph-mon[93957]: pg 2.1b is active+undersized+degraded, acting [1,5] 2024-03-24T10:50:00.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:00 smithi094 ceph-mon[93957]: pg 2.1d is active+undersized+degraded, acting [3,5] 2024-03-24T10:50:00.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:00 smithi094 ceph-mon[93957]: pg 2.1e is active+undersized+degraded, acting [2,5] 2024-03-24T10:50:00.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:00 smithi094 ceph-mon[93957]: pg 2.1f is active+undersized+degraded, acting [3,4] 2024-03-24T10:50:00.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:00 smithi096 ceph-mon[84258]: pgmap v47: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 44/219 objects degraded (20.091%) 2024-03-24T10:50:00.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:00 smithi096 ceph-mon[84258]: Health detail: HEALTH_WARN 1 osds down; Degraded data redundancy: 44/219 objects degraded (20.091%), 18 pgs degraded 2024-03-24T10:50:00.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:00 smithi096 ceph-mon[84258]: [WRN] OSD_DOWN: 1 osds down 2024-03-24T10:50:00.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:00 smithi096 ceph-mon[84258]: osd.0 (root=default,host=smithi094) is down 2024-03-24T10:50:00.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:00 smithi096 ceph-mon[84258]: [WRN] PG_DEGRADED: Degraded data redundancy: 44/219 objects degraded (20.091%), 18 pgs degraded 2024-03-24T10:50:00.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:00 smithi096 ceph-mon[84258]: pg 1.0 is active+undersized+degraded, acting [3,1] 2024-03-24T10:50:00.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:00 smithi096 ceph-mon[84258]: pg 2.0 is active+undersized+degraded, acting [3,1] 2024-03-24T10:50:00.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:00 smithi096 ceph-mon[84258]: pg 2.1 is active+undersized+degraded, acting [2,1] 2024-03-24T10:50:00.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:00 smithi096 ceph-mon[84258]: pg 2.2 is active+undersized+degraded, acting [5,1] 2024-03-24T10:50:00.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:00 smithi096 ceph-mon[84258]: pg 2.4 is active+undersized+degraded, acting [1,4] 2024-03-24T10:50:00.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:00 smithi096 ceph-mon[84258]: pg 2.5 is active+undersized+degraded, acting [3,4] 2024-03-24T10:50:00.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:00 smithi096 ceph-mon[84258]: pg 2.9 is active+undersized+degraded, acting [1,4] 2024-03-24T10:50:00.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:00 smithi096 ceph-mon[84258]: pg 2.c is active+undersized+degraded, acting [2,3] 2024-03-24T10:50:00.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:00 smithi096 ceph-mon[84258]: pg 2.e is active+undersized+degraded, acting [2,3] 2024-03-24T10:50:00.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:00 smithi096 ceph-mon[84258]: pg 2.f is active+undersized+degraded, acting [4,5] 2024-03-24T10:50:00.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:00 smithi096 ceph-mon[84258]: pg 2.10 is active+undersized+degraded, acting [2,1] 2024-03-24T10:50:00.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:00 smithi096 ceph-mon[84258]: pg 2.13 is active+undersized+degraded, acting [4,2] 2024-03-24T10:50:00.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:00 smithi096 ceph-mon[84258]: pg 2.15 is active+undersized+degraded, acting [1,3] 2024-03-24T10:50:00.577 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:00 smithi096 ceph-mon[84258]: pg 2.19 is active+undersized+degraded, acting [4,2] 2024-03-24T10:50:00.577 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:00 smithi096 ceph-mon[84258]: pg 2.1b is active+undersized+degraded, acting [1,5] 2024-03-24T10:50:00.577 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:00 smithi096 ceph-mon[84258]: pg 2.1d is active+undersized+degraded, acting [3,5] 2024-03-24T10:50:00.577 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:00 smithi096 ceph-mon[84258]: pg 2.1e is active+undersized+degraded, acting [2,5] 2024-03-24T10:50:00.577 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:00 smithi096 ceph-mon[84258]: pg 2.1f is active+undersized+degraded, acting [3,4] 2024-03-24T10:50:01.497 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:50:01 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-0[102665]: 2024-03-24T10:50:01.161+0000 7ff3d684a7c0 -1 osd.0 0 read_superblock omap replica is missing. 2024-03-24T10:50:01.997 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:50:01 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-0[102665]: 2024-03-24T10:50:01.691+0000 7ff3d684a7c0 -1 osd.0 49 log_to_monitors true 2024-03-24T10:50:02.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.021+0000 7f3e5821f700 1 -- 172.21.15.94:0/1538005711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e500fe430 msgr2=0x7f3e500fe840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:02.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.021+0000 7f3e5821f700 1 --2- 172.21.15.94:0/1538005711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e500fe430 0x7f3e500fe840 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c004940 tx=0x7f3e4c009e30 comp rx=0 tx=0).stop 2024-03-24T10:50:02.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.021+0000 7f3e5821f700 1 -- 172.21.15.94:0/1538005711 shutdown_connections 2024-03-24T10:50:02.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.021+0000 7f3e5821f700 1 --2- 172.21.15.94:0/1538005711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e500fed80 0x7f3e500faec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:02.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.021+0000 7f3e5821f700 1 --2- 172.21.15.94:0/1538005711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e500fe430 0x7f3e500fe840 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:02.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.021+0000 7f3e5821f700 1 -- 172.21.15.94:0/1538005711 >> 172.21.15.94:0/1538005711 conn(0x7f3e500f6bd0 msgr2=0x7f3e500f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:50:02.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.021+0000 7f3e5821f700 1 -- 172.21.15.94:0/1538005711 shutdown_connections 2024-03-24T10:50:02.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.021+0000 7f3e5821f700 1 -- 172.21.15.94:0/1538005711 wait complete. 2024-03-24T10:50:02.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.022+0000 7f3e5821f700 1 Processor -- start 2024-03-24T10:50:02.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.022+0000 7f3e5821f700 1 -- start start 2024-03-24T10:50:02.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.022+0000 7f3e5821f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e500fe430 0x7f3e5010fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:50:02.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.022+0000 7f3e5821f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e500fed80 0x7f3e50110080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:50:02.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.022+0000 7f3e5821f700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e50110680 con 0x7f3e500fe430 2024-03-24T10:50:02.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.022+0000 7f3e5821f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e501107f0 con 0x7f3e500fed80 2024-03-24T10:50:02.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.022+0000 7f3e55fbb700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e500fe430 0x7f3e5010fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:50:02.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.022+0000 7f3e55fbb700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e500fe430 0x7f3e5010fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46390/0 (socket says 172.21.15.94:46390) 2024-03-24T10:50:02.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.022+0000 7f3e55fbb700 1 -- 172.21.15.94:0/2666966256 learned_addr learned my addr 172.21.15.94:0/2666966256 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:50:02.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.022+0000 7f3e557ba700 1 --2- 172.21.15.94:0/2666966256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e500fed80 0x7f3e50110080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:50:02.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.022+0000 7f3e55fbb700 1 -- 172.21.15.94:0/2666966256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e500fed80 msgr2=0x7f3e50110080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:02.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.022+0000 7f3e55fbb700 1 --2- 172.21.15.94:0/2666966256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e500fed80 0x7f3e50110080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:02.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.022+0000 7f3e55fbb700 1 -- 172.21.15.94:0/2666966256 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e4c005040 con 0x7f3e500fe430 2024-03-24T10:50:02.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.023+0000 7f3e55fbb700 1 --2- 172.21.15.94:0/2666966256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e500fe430 0x7f3e5010fb40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c00b3b0 tx=0x7f3e4c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:50:02.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.023+0000 7f3e467fc700 1 -- 172.21.15.94:0/2666966256 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e4c00e040 con 0x7f3e500fe430 2024-03-24T10:50:02.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.023+0000 7f3e5821f700 1 -- 172.21.15.94:0/2666966256 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e501151f0 con 0x7f3e500fe430 2024-03-24T10:50:02.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.023+0000 7f3e467fc700 1 -- 172.21.15.94:0/2666966256 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3e4c004290 con 0x7f3e500fe430 2024-03-24T10:50:02.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.023+0000 7f3e467fc700 1 -- 172.21.15.94:0/2666966256 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e4c01c3f0 con 0x7f3e500fe430 2024-03-24T10:50:02.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.023+0000 7f3e5821f700 1 -- 172.21.15.94:0/2666966256 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e501157b0 con 0x7f3e500fe430 2024-03-24T10:50:02.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.024+0000 7f3e5821f700 1 -- 172.21.15.94:0/2666966256 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e50101b70 con 0x7f3e500fe430 2024-03-24T10:50:02.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.024+0000 7f3e467fc700 1 -- 172.21.15.94:0/2666966256 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 35) v1 ==== 91587+0+0 (secure 0 0 0) 0x7f3e4c00bbf0 con 0x7f3e500fe430 2024-03-24T10:50:02.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.025+0000 7f3e467fc700 1 --2- 172.21.15.94:0/2666966256 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f3e3c06d910 0x7f3e3c06fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:50:02.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.025+0000 7f3e467fc700 1 -- 172.21.15.94:0/2666966256 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f3e4c08e4c0 con 0x7f3e500fe430 2024-03-24T10:50:02.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.025+0000 7f3e557ba700 1 --2- 172.21.15.94:0/2666966256 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f3e3c06d910 0x7f3e3c06fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:50:02.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.025+0000 7f3e557ba700 1 --2- 172.21.15.94:0/2666966256 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f3e3c06d910 0x7f3e3c06fdc0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3e40004f40 tx=0x7f3e4000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:50:02.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.030+0000 7f3e467fc700 1 -- 172.21.15.94:0/2666966256 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3e4c05b640 con 0x7f3e500fe430 2024-03-24T10:50:02.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.269+0000 7f3e5821f700 1 -- 172.21.15.94:0/2666966256 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e5002b310 con 0x7f3e3c06d910 2024-03-24T10:50:02.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.271+0000 7f3e467fc700 1 -- 172.21.15.94:0/2666966256 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f3e5002b310 con 0x7f3e3c06d910 2024-03-24T10:50:02.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.273+0000 7f3e5821f700 1 -- 172.21.15.94:0/2666966256 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f3e3c06d910 msgr2=0x7f3e3c06fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:02.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.273+0000 7f3e5821f700 1 --2- 172.21.15.94:0/2666966256 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f3e3c06d910 0x7f3e3c06fdc0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3e40004f40 tx=0x7f3e4000b040 comp rx=0 tx=0).stop 2024-03-24T10:50:02.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.273+0000 7f3e5821f700 1 -- 172.21.15.94:0/2666966256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e500fe430 msgr2=0x7f3e5010fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:02.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.273+0000 7f3e5821f700 1 --2- 172.21.15.94:0/2666966256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e500fe430 0x7f3e5010fb40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c00b3b0 tx=0x7f3e4c009e30 comp rx=0 tx=0).stop 2024-03-24T10:50:02.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.273+0000 7f3e5821f700 1 -- 172.21.15.94:0/2666966256 shutdown_connections 2024-03-24T10:50:02.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.273+0000 7f3e5821f700 1 --2- 172.21.15.94:0/2666966256 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f3e3c06d910 0x7f3e3c06fdc0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:02.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.273+0000 7f3e5821f700 1 --2- 172.21.15.94:0/2666966256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e500fe430 0x7f3e5010fb40 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:02.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.273+0000 7f3e5821f700 1 --2- 172.21.15.94:0/2666966256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e500fed80 0x7f3e50110080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:02.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.273+0000 7f3e5821f700 1 -- 172.21.15.94:0/2666966256 >> 172.21.15.94:0/2666966256 conn(0x7f3e500f6bd0 msgr2=0x7f3e500f8250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:50:02.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.273+0000 7f3e5821f700 1 -- 172.21.15.94:0/2666966256 shutdown_connections 2024-03-24T10:50:02.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.273+0000 7f3e5821f700 1 -- 172.21.15.94:0/2666966256 wait complete. 2024-03-24T10:50:02.285 INFO:teuthology.orchestra.run.smithi094.stdout:true 2024-03-24T10:50:02.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.380+0000 7f1b20b21700 1 -- 172.21.15.94:0/995833032 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b1c0fbcd0 msgr2=0x7f1b1c0fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:02.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.380+0000 7f1b20b21700 1 --2- 172.21.15.94:0/995833032 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b1c0fbcd0 0x7f1b1c0fc0e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c004940 tx=0x7f1b0c009e30 comp rx=0 tx=0).stop 2024-03-24T10:50:02.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.380+0000 7f1b20b21700 1 -- 172.21.15.94:0/995833032 shutdown_connections 2024-03-24T10:50:02.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.380+0000 7f1b20b21700 1 --2- 172.21.15.94:0/995833032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b1c0fcf80 0x7f1b1c0fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:02.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.380+0000 7f1b20b21700 1 --2- 172.21.15.94:0/995833032 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b1c0fbcd0 0x7f1b1c0fc0e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:02.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.380+0000 7f1b20b21700 1 -- 172.21.15.94:0/995833032 >> 172.21.15.94:0/995833032 conn(0x7f1b1c0f7430 msgr2=0x7f1b1c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:50:02.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.380+0000 7f1b20b21700 1 -- 172.21.15.94:0/995833032 shutdown_connections 2024-03-24T10:50:02.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.380+0000 7f1b20b21700 1 -- 172.21.15.94:0/995833032 wait complete. 2024-03-24T10:50:02.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.381+0000 7f1b20b21700 1 Processor -- start 2024-03-24T10:50:02.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.381+0000 7f1b20b21700 1 -- start start 2024-03-24T10:50:02.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.381+0000 7f1b20b21700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b1c0fbcd0 0x7f1b1c10daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:50:02.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.381+0000 7f1b20b21700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b1c0fcf80 0x7f1b1c10dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:50:02.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.381+0000 7f1b20b21700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1c10e5e0 con 0x7f1b1c0fbcd0 2024-03-24T10:50:02.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.381+0000 7f1b20b21700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1c10ed90 con 0x7f1b1c0fcf80 2024-03-24T10:50:02.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.381+0000 7f1b1a59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b1c0fbcd0 0x7f1b1c10daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:50:02.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.381+0000 7f1b1a59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b1c0fbcd0 0x7f1b1c10daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46398/0 (socket says 172.21.15.94:46398) 2024-03-24T10:50:02.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.381+0000 7f1b1a59c700 1 -- 172.21.15.94:0/1519515576 learned_addr learned my addr 172.21.15.94:0/1519515576 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:50:02.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.381+0000 7f1b19d9b700 1 --2- 172.21.15.94:0/1519515576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b1c0fcf80 0x7f1b1c10dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:50:02.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.382+0000 7f1b1a59c700 1 -- 172.21.15.94:0/1519515576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b1c0fcf80 msgr2=0x7f1b1c10dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:02.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.382+0000 7f1b1a59c700 1 --2- 172.21.15.94:0/1519515576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b1c0fcf80 0x7f1b1c10dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:02.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.382+0000 7f1b1a59c700 1 -- 172.21.15.94:0/1519515576 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b0c005040 con 0x7f1b1c0fbcd0 2024-03-24T10:50:02.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.382+0000 7f1b1a59c700 1 --2- 172.21.15.94:0/1519515576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b1c0fbcd0 0x7f1b1c10daa0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c009150 tx=0x7f1b0c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:50:02.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.382+0000 7f1b0affd700 1 -- 172.21.15.94:0/1519515576 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b0c00e040 con 0x7f1b1c0fbcd0 2024-03-24T10:50:02.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.382+0000 7f1b20b21700 1 -- 172.21.15.94:0/1519515576 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b1c10f010 con 0x7f1b1c0fbcd0 2024-03-24T10:50:02.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.382+0000 7f1b20b21700 1 -- 172.21.15.94:0/1519515576 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b1c10f550 con 0x7f1b1c0fbcd0 2024-03-24T10:50:02.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.382+0000 7f1b0affd700 1 -- 172.21.15.94:0/1519515576 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1b0c00bbf0 con 0x7f1b1c0fbcd0 2024-03-24T10:50:02.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.382+0000 7f1b0affd700 1 -- 172.21.15.94:0/1519515576 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b0c01c630 con 0x7f1b1c0fbcd0 2024-03-24T10:50:02.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.383+0000 7f1b20b21700 1 -- 172.21.15.94:0/1519515576 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b1c101b10 con 0x7f1b1c0fbcd0 2024-03-24T10:50:02.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.383+0000 7f1b0affd700 1 -- 172.21.15.94:0/1519515576 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 35) v1 ==== 91587+0+0 (secure 0 0 0) 0x7f1b0c013420 con 0x7f1b1c0fbcd0 2024-03-24T10:50:02.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.384+0000 7f1b0affd700 1 --2- 172.21.15.94:0/1519515576 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1b0406d900 0x7f1b0406fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:50:02.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.384+0000 7f1b0affd700 1 -- 172.21.15.94:0/1519515576 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f1b0c08e510 con 0x7f1b1c0fbcd0 2024-03-24T10:50:02.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.384+0000 7f1b19d9b700 1 --2- 172.21.15.94:0/1519515576 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1b0406d900 0x7f1b0406fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:50:02.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.384+0000 7f1b19d9b700 1 --2- 172.21.15.94:0/1519515576 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1b0406d900 0x7f1b0406fdb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1b100041d0 tx=0x7f1b100043c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:50:02.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.393+0000 7f1b0affd700 1 -- 172.21.15.94:0/1519515576 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1b0c05b690 con 0x7f1b1c0fbcd0 2024-03-24T10:50:02.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:02 smithi096 ceph-mon[84258]: pgmap v48: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 521 B/s rd, 0 op/s; 44/219 objects degraded (20.091%) 2024-03-24T10:50:02.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:02 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:50:02.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:02 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:50:02.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:02 smithi096 ceph-mon[84258]: from='osd.0 [v2:172.21.15.94:6802/1160876432,v1:172.21.15.94:6803/1160876432]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-24T10:50:02.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.652+0000 7f1b20b21700 1 -- 172.21.15.94:0/1519515576 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1b1c104930 con 0x7f1b0406d900 2024-03-24T10:50:02.653 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:02 smithi094 ceph-mon[93957]: pgmap v48: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 196 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 521 B/s rd, 0 op/s; 44/219 objects degraded (20.091%) 2024-03-24T10:50:02.653 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:02 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:50:02.653 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:02 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:50:02.653 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:02 smithi094 ceph-mon[93957]: from='osd.0 [v2:172.21.15.94:6802/1160876432,v1:172.21.15.94:6803/1160876432]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-24T10:50:02.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.654+0000 7f1b0affd700 1 -- 172.21.15.94:0/1519515576 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f1b1c104930 con 0x7f1b0406d900 2024-03-24T10:50:02.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.656+0000 7f1b20b21700 1 -- 172.21.15.94:0/1519515576 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1b0406d900 msgr2=0x7f1b0406fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:02.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.656+0000 7f1b20b21700 1 --2- 172.21.15.94:0/1519515576 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1b0406d900 0x7f1b0406fdb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1b100041d0 tx=0x7f1b100043c0 comp rx=0 tx=0).stop 2024-03-24T10:50:02.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.656+0000 7f1b20b21700 1 -- 172.21.15.94:0/1519515576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b1c0fbcd0 msgr2=0x7f1b1c10daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:02.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.656+0000 7f1b20b21700 1 --2- 172.21.15.94:0/1519515576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b1c0fbcd0 0x7f1b1c10daa0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c009150 tx=0x7f1b0c009e30 comp rx=0 tx=0).stop 2024-03-24T10:50:02.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.656+0000 7f1b20b21700 1 -- 172.21.15.94:0/1519515576 shutdown_connections 2024-03-24T10:50:02.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.656+0000 7f1b20b21700 1 --2- 172.21.15.94:0/1519515576 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1b0406d900 0x7f1b0406fdb0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:02.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.656+0000 7f1b20b21700 1 --2- 172.21.15.94:0/1519515576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b1c0fbcd0 0x7f1b1c10daa0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:02.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.656+0000 7f1b20b21700 1 --2- 172.21.15.94:0/1519515576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b1c0fcf80 0x7f1b1c10dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:02.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.656+0000 7f1b20b21700 1 -- 172.21.15.94:0/1519515576 >> 172.21.15.94:0/1519515576 conn(0x7f1b1c0f7430 msgr2=0x7f1b1c065ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:50:02.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.656+0000 7f1b20b21700 1 -- 172.21.15.94:0/1519515576 shutdown_connections 2024-03-24T10:50:02.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.656+0000 7f1b20b21700 1 -- 172.21.15.94:0/1519515576 wait complete. 2024-03-24T10:50:02.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.782+0000 7f8d803ae700 1 -- 172.21.15.94:0/1395833895 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d780fcf70 msgr2=0x7f8d780fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:02.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.782+0000 7f8d803ae700 1 --2- 172.21.15.94:0/1395833895 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d780fcf70 0x7f8d780fd3e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8d68004940 tx=0x7f8d68009e30 comp rx=0 tx=0).stop 2024-03-24T10:50:02.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.782+0000 7f8d803ae700 1 -- 172.21.15.94:0/1395833895 shutdown_connections 2024-03-24T10:50:02.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.782+0000 7f8d803ae700 1 --2- 172.21.15.94:0/1395833895 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d780fcf70 0x7f8d780fd3e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:02.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.782+0000 7f8d803ae700 1 --2- 172.21.15.94:0/1395833895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8d780fbe00 0x7f8d780fc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:02.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.782+0000 7f8d803ae700 1 -- 172.21.15.94:0/1395833895 >> 172.21.15.94:0/1395833895 conn(0x7f8d780f74e0 msgr2=0x7f8d780f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:50:02.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.782+0000 7f8d803ae700 1 -- 172.21.15.94:0/1395833895 shutdown_connections 2024-03-24T10:50:02.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.782+0000 7f8d803ae700 1 -- 172.21.15.94:0/1395833895 wait complete. 2024-03-24T10:50:02.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.782+0000 7f8d803ae700 1 Processor -- start 2024-03-24T10:50:02.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.782+0000 7f8d803ae700 1 -- start start 2024-03-24T10:50:02.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.783+0000 7f8d803ae700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d780fbe00 0x7f8d7810da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:50:02.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.783+0000 7f8d803ae700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8d780fcf70 0x7f8d7810df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:50:02.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.783+0000 7f8d803ae700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7810e550 con 0x7f8d780fbe00 2024-03-24T10:50:02.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.783+0000 7f8d803ae700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7810eda0 con 0x7f8d780fcf70 2024-03-24T10:50:02.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.783+0000 7f8d7e14a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d780fbe00 0x7f8d7810da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:50:02.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.783+0000 7f8d7e14a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d780fbe00 0x7f8d7810da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46426/0 (socket says 172.21.15.94:46426) 2024-03-24T10:50:02.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.783+0000 7f8d7e14a700 1 -- 172.21.15.94:0/3116025705 learned_addr learned my addr 172.21.15.94:0/3116025705 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:50:02.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.783+0000 7f8d7d949700 1 --2- 172.21.15.94:0/3116025705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8d780fcf70 0x7f8d7810df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:50:02.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.783+0000 7f8d7e14a700 1 -- 172.21.15.94:0/3116025705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8d780fcf70 msgr2=0x7f8d7810df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:02.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.783+0000 7f8d7e14a700 1 --2- 172.21.15.94:0/3116025705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8d780fcf70 0x7f8d7810df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:02.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.783+0000 7f8d7e14a700 1 -- 172.21.15.94:0/3116025705 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d68005040 con 0x7f8d780fbe00 2024-03-24T10:50:02.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.784+0000 7f8d7e14a700 1 --2- 172.21.15.94:0/3116025705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d780fbe00 0x7f8d7810da10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8d7400c830 tx=0x7f8d74004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:50:02.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.784+0000 7f8d6effd700 1 -- 172.21.15.94:0/3116025705 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d740106f0 con 0x7f8d780fbe00 2024-03-24T10:50:02.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.784+0000 7f8d6effd700 1 -- 172.21.15.94:0/3116025705 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d74010d20 con 0x7f8d780fbe00 2024-03-24T10:50:02.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.784+0000 7f8d803ae700 1 -- 172.21.15.94:0/3116025705 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d7810f080 con 0x7f8d780fbe00 2024-03-24T10:50:02.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.784+0000 7f8d6effd700 1 -- 172.21.15.94:0/3116025705 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d740113a0 con 0x7f8d780fbe00 2024-03-24T10:50:02.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.784+0000 7f8d803ae700 1 -- 172.21.15.94:0/3116025705 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d7810f560 con 0x7f8d780fbe00 2024-03-24T10:50:02.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.785+0000 7f8d803ae700 1 -- 172.21.15.94:0/3116025705 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d7805f5f0 con 0x7f8d780fbe00 2024-03-24T10:50:02.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.786+0000 7f8d6effd700 1 -- 172.21.15.94:0/3116025705 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 35) v1 ==== 91587+0+0 (secure 0 0 0) 0x7f8d74012400 con 0x7f8d780fbe00 2024-03-24T10:50:02.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.786+0000 7f8d6effd700 1 --2- 172.21.15.94:0/3116025705 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f8d6406d900 0x7f8d6406fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:50:02.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.787+0000 7f8d6effd700 1 -- 172.21.15.94:0/3116025705 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f8d7408cf70 con 0x7f8d780fbe00 2024-03-24T10:50:02.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.787+0000 7f8d7d949700 1 --2- 172.21.15.94:0/3116025705 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f8d6406d900 0x7f8d6406fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:50:02.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.787+0000 7f8d7d949700 1 --2- 172.21.15.94:0/3116025705 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f8d6406d900 0x7f8d6406fdb0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8d6800b3b0 tx=0x7f8d68009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:50:02.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:02.795+0000 7f8d6effd700 1 -- 172.21.15.94:0/3116025705 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8d740c1280 con 0x7f8d780fbe00 2024-03-24T10:50:03.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.109+0000 7f8d803ae700 1 -- 172.21.15.94:0/3116025705 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8d7810f9d0 con 0x7f8d6406d900 2024-03-24T10:50:03.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.128+0000 7f8d6effd700 1 -- 172.21.15.94:0/3116025705 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f8d7810f9d0 con 0x7f8d6406d900 2024-03-24T10:50:03.129 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T10:50:03.129 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.smithi094 smithi094 *:9093,9094 running (3m) 1s ago 12m 16.8M - 0.25.0 c8568f914cd2 5c583af60554 2024-03-24T10:50:03.129 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi094 smithi094 running (32s) 1s ago 12m 6651k - 19.0.0-1855-g21b3bd67 553463a90fd5 2b0897df571d 2024-03-24T10:50:03.129 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi096 smithi096 running (27s) 23s ago 10m 6669k - 19.0.0-1855-g21b3bd67 553463a90fd5 1759a9ff2805 2024-03-24T10:50:03.129 INFO:teuthology.orchestra.run.smithi094.stdout:grafana.smithi094 smithi094 *:3000 running (2m) 1s ago 11m 77.9M - 9.4.7 954c08fa6188 d3d425bf1b7c 2024-03-24T10:50:03.129 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.efiqte smithi094 running (6m) 1s ago 6m 13.4M - 17.2.7-726-g9c4e0c34 837124da804f f0df0a056c21 2024-03-24T10:50:03.129 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.llefgu smithi094 running (6m) 1s ago 6m 109M - 17.2.7-726-g9c4e0c34 837124da804f 52a9771dc8e0 2024-03-24T10:50:03.129 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi096.egbmoq smithi096 running (6m) 23s ago 6m 154M - 17.2.7-726-g9c4e0c34 837124da804f 1cfc930dbab0 2024-03-24T10:50:03.129 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi096.jtebhh smithi096 running (6m) 23s ago 6m 16.3M - 17.2.7-726-g9c4e0c34 837124da804f 4ff3451a6f6e 2024-03-24T10:50:03.130 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi094.yaethu smithi094 *:8443,9283,8765 running (2m) 1s ago 13m 497M - 19.0.0-1855-g21b3bd67 553463a90fd5 d3adab76b18f 2024-03-24T10:50:03.130 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi096.gfonnu smithi096 *:8443,9283,8765 running (110s) 23s ago 10m 448M - 19.0.0-1855-g21b3bd67 553463a90fd5 22937a059eb8 2024-03-24T10:50:03.130 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi094 smithi094 running (96s) 1s ago 13m 45.2M 2048M 19.0.0-1855-g21b3bd67 553463a90fd5 c779d611e29a 2024-03-24T10:50:03.130 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi096 smithi096 running (73s) 23s ago 10m 31.2M 2048M 19.0.0-1855-g21b3bd67 553463a90fd5 ed9bd0bd57f6 2024-03-24T10:50:03.130 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi094 smithi094 *:9100 running (3m) 1s ago 12m 16.4M - 1.5.0 0da6a335fe13 ca67b53c64e2 2024-03-24T10:50:03.130 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi096 smithi096 *:9100 running (2m) 23s ago 10m 17.0M - 1.5.0 0da6a335fe13 65137a7a7549 2024-03-24T10:50:03.130 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (7s) 1s ago 9m 12.6M 1557M 19.0.0-1855-g21b3bd67 553463a90fd5 ec57ede0c9e4 2024-03-24T10:50:03.130 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi094 running (9m) 1s ago 9m 259M 1557M 17.2.7-726-g9c4e0c34 837124da804f 4e737a80d733 2024-03-24T10:50:03.130 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi094 running (9m) 1s ago 9m 221M 1557M 17.2.7-726-g9c4e0c34 837124da804f cf0aefd20c76 2024-03-24T10:50:03.130 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi096 running (8m) 23s ago 8m 285M 2581M 17.2.7-726-g9c4e0c34 837124da804f f4650d0f6241 2024-03-24T10:50:03.130 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi096 running (8m) 23s ago 8m 258M 2581M 17.2.7-726-g9c4e0c34 837124da804f a63d7c9e705f 2024-03-24T10:50:03.130 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi096 running (7m) 23s ago 7m 243M 2581M 17.2.7-726-g9c4e0c34 837124da804f f56fad2959c3 2024-03-24T10:50:03.131 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.smithi094 smithi094 *:9095 running (116s) 1s ago 11m 51.2M - 2.43.0 a07b618ecd1d 6ce091a0c326 2024-03-24T10:50:03.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.131+0000 7f8d803ae700 1 -- 172.21.15.94:0/3116025705 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f8d6406d900 msgr2=0x7f8d6406fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.131+0000 7f8d803ae700 1 --2- 172.21.15.94:0/3116025705 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f8d6406d900 0x7f8d6406fdb0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8d6800b3b0 tx=0x7f8d68009e30 comp rx=0 tx=0).stop 2024-03-24T10:50:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.131+0000 7f8d803ae700 1 -- 172.21.15.94:0/3116025705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d780fbe00 msgr2=0x7f8d7810da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.131+0000 7f8d803ae700 1 --2- 172.21.15.94:0/3116025705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d780fbe00 0x7f8d7810da10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8d7400c830 tx=0x7f8d74004470 comp rx=0 tx=0).stop 2024-03-24T10:50:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.131+0000 7f8d803ae700 1 -- 172.21.15.94:0/3116025705 shutdown_connections 2024-03-24T10:50:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.131+0000 7f8d803ae700 1 --2- 172.21.15.94:0/3116025705 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f8d6406d900 0x7f8d6406fdb0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.131+0000 7f8d803ae700 1 --2- 172.21.15.94:0/3116025705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d780fbe00 0x7f8d7810da10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.131+0000 7f8d803ae700 1 --2- 172.21.15.94:0/3116025705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8d780fcf70 0x7f8d7810df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.131+0000 7f8d803ae700 1 -- 172.21.15.94:0/3116025705 >> 172.21.15.94:0/3116025705 conn(0x7f8d780f74e0 msgr2=0x7f8d780f8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:50:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.131+0000 7f8d803ae700 1 -- 172.21.15.94:0/3116025705 shutdown_connections 2024-03-24T10:50:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.132+0000 7f8d803ae700 1 -- 172.21.15.94:0/3116025705 wait complete. 2024-03-24T10:50:03.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.256+0000 7ff36d518700 1 -- 172.21.15.94:0/2178918430 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3680fcf80 msgr2=0x7ff3680fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:03.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.256+0000 7ff36d518700 1 --2- 172.21.15.94:0/2178918430 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3680fcf80 0x7ff3680fd3f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff35c004940 tx=0x7ff35c009e30 comp rx=0 tx=0).stop 2024-03-24T10:50:03.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.256+0000 7ff36d518700 1 -- 172.21.15.94:0/2178918430 shutdown_connections 2024-03-24T10:50:03.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.256+0000 7ff36d518700 1 --2- 172.21.15.94:0/2178918430 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3680fcf80 0x7ff3680fd3f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:03.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.256+0000 7ff36d518700 1 --2- 172.21.15.94:0/2178918430 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3680fbcd0 0x7ff3680fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:03.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.256+0000 7ff36d518700 1 -- 172.21.15.94:0/2178918430 >> 172.21.15.94:0/2178918430 conn(0x7ff3680f7430 msgr2=0x7ff3680f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:50:03.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.256+0000 7ff36d518700 1 -- 172.21.15.94:0/2178918430 shutdown_connections 2024-03-24T10:50:03.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.256+0000 7ff36d518700 1 -- 172.21.15.94:0/2178918430 wait complete. 2024-03-24T10:50:03.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.257+0000 7ff36d518700 1 Processor -- start 2024-03-24T10:50:03.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.257+0000 7ff36d518700 1 -- start start 2024-03-24T10:50:03.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.257+0000 7ff36d518700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3680fbcd0 0x7ff36810d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:50:03.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.257+0000 7ff36d518700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3680fcf80 0x7ff36810de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:50:03.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.257+0000 7ff36d518700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff36810e470 con 0x7ff3680fbcd0 2024-03-24T10:50:03.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.257+0000 7ff36d518700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff36810e5e0 con 0x7ff3680fcf80 2024-03-24T10:50:03.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.257+0000 7ff366ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3680fbcd0 0x7ff36810d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:50:03.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.257+0000 7ff366ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3680fbcd0 0x7ff36810d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46446/0 (socket says 172.21.15.94:46446) 2024-03-24T10:50:03.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.257+0000 7ff366ffd700 1 -- 172.21.15.94:0/2249722561 learned_addr learned my addr 172.21.15.94:0/2249722561 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:50:03.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.257+0000 7ff3667fc700 1 --2- 172.21.15.94:0/2249722561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3680fcf80 0x7ff36810de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:50:03.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.258+0000 7ff366ffd700 1 -- 172.21.15.94:0/2249722561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3680fcf80 msgr2=0x7ff36810de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:03.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.258+0000 7ff366ffd700 1 --2- 172.21.15.94:0/2249722561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3680fcf80 0x7ff36810de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:03.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.258+0000 7ff366ffd700 1 -- 172.21.15.94:0/2249722561 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff35c005040 con 0x7ff3680fbcd0 2024-03-24T10:50:03.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.258+0000 7ff3667fc700 1 --2- 172.21.15.94:0/2249722561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3680fcf80 0x7ff36810de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T10:50:03.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.258+0000 7ff366ffd700 1 --2- 172.21.15.94:0/2249722561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3680fbcd0 0x7ff36810d930 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff35800eb40 tx=0x7ff35800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:50:03.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.258+0000 7ff34f7fe700 1 -- 172.21.15.94:0/2249722561 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff358007cc0 con 0x7ff3680fbcd0 2024-03-24T10:50:03.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.258+0000 7ff34f7fe700 1 -- 172.21.15.94:0/2249722561 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff35800f040 con 0x7ff3680fbcd0 2024-03-24T10:50:03.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.258+0000 7ff36d518700 1 -- 172.21.15.94:0/2249722561 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff368112f90 con 0x7ff3680fbcd0 2024-03-24T10:50:03.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.258+0000 7ff34f7fe700 1 -- 172.21.15.94:0/2249722561 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff358009620 con 0x7ff3680fbcd0 2024-03-24T10:50:03.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.259+0000 7ff36d518700 1 -- 172.21.15.94:0/2249722561 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff368113440 con 0x7ff3680fbcd0 2024-03-24T10:50:03.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.260+0000 7ff36d518700 1 -- 172.21.15.94:0/2249722561 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff368101b10 con 0x7ff3680fbcd0 2024-03-24T10:50:03.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.260+0000 7ff34f7fe700 1 -- 172.21.15.94:0/2249722561 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 35) v1 ==== 91587+0+0 (secure 0 0 0) 0x7ff358009780 con 0x7ff3680fbcd0 2024-03-24T10:50:03.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.261+0000 7ff34f7fe700 1 --2- 172.21.15.94:0/2249722561 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7ff35006d910 0x7ff35006fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:50:03.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.261+0000 7ff34f7fe700 1 -- 172.21.15.94:0/2249722561 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6500+0+0 (secure 0 0 0) 0x7ff35808ce30 con 0x7ff3680fbcd0 2024-03-24T10:50:03.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.261+0000 7ff3667fc700 1 --2- 172.21.15.94:0/2249722561 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7ff35006d910 0x7ff35006fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:50:03.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.262+0000 7ff3667fc700 1 --2- 172.21.15.94:0/2249722561 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7ff35006d910 0x7ff35006fdc0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff35c009b30 tx=0x7ff35c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:50:03.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.270+0000 7ff34f7fe700 1 -- 172.21.15.94:0/2249722561 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff358059fe0 con 0x7ff3680fbcd0 2024-03-24T10:50:03.466 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:50:03 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-0[102665]: 2024-03-24T10:50:03.298+0000 7ff3c3c9a700 -1 osd.0 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T10:50:03.466 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:03 smithi094 ceph-mon[93957]: from='client.34190 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:50:03.466 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:03 smithi094 ceph-mon[93957]: from='osd.0 [v2:172.21.15.94:6802/1160876432,v1:172.21.15.94:6803/1160876432]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-24T10:50:03.466 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:03 smithi094 ceph-mon[93957]: osdmap e52: 6 total, 5 up, 6 in 2024-03-24T10:50:03.467 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:03 smithi094 ceph-mon[93957]: from='osd.0 [v2:172.21.15.94:6802/1160876432,v1:172.21.15.94:6803/1160876432]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-03-24T10:50:03.467 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:03 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:50:03.467 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:03 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:50:03.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:03 smithi096 ceph-mon[84258]: from='client.34190 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:50:03.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:03 smithi096 ceph-mon[84258]: from='osd.0 [v2:172.21.15.94:6802/1160876432,v1:172.21.15.94:6803/1160876432]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-24T10:50:03.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:03 smithi096 ceph-mon[84258]: osdmap e52: 6 total, 5 up, 6 in 2024-03-24T10:50:03.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:03 smithi096 ceph-mon[84258]: from='osd.0 [v2:172.21.15.94:6802/1160876432,v1:172.21.15.94:6803/1160876432]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-03-24T10:50:03.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:03 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:50:03.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:03 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:50:03.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.585+0000 7ff36d518700 1 -- 172.21.15.94:0/2249722561 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff3681047c0 con 0x7ff3680fbcd0 2024-03-24T10:50:03.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.586+0000 7ff34f7fe700 1 -- 172.21.15.94:0/2249722561 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+735 (secure 0 0 0) 0x7ff358059b70 con 0x7ff3680fbcd0 2024-03-24T10:50:03.587 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-03-24T10:50:03.587 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": { 2024-03-24T10:50:03.587 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 2 2024-03-24T10:50:03.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:50:03.587 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": { 2024-03-24T10:50:03.587 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 2 2024-03-24T10:50:03.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:50:03.587 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": { 2024-03-24T10:50:03.587 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 5 2024-03-24T10:50:03.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:50:03.587 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-03-24T10:50:03.588 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-24T10:50:03.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:50:03.588 INFO:teuthology.orchestra.run.smithi094.stdout: "overall": { 2024-03-24T10:50:03.588 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 9, 2024-03-24T10:50:03.588 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 4 2024-03-24T10:50:03.588 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-03-24T10:50:03.588 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-03-24T10:50:03.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.588+0000 7ff36d518700 1 -- 172.21.15.94:0/2249722561 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7ff35006d910 msgr2=0x7ff35006fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:03.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.588+0000 7ff36d518700 1 --2- 172.21.15.94:0/2249722561 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7ff35006d910 0x7ff35006fdc0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff35c009b30 tx=0x7ff35c009e30 comp rx=0 tx=0).stop 2024-03-24T10:50:03.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.588+0000 7ff36d518700 1 -- 172.21.15.94:0/2249722561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3680fbcd0 msgr2=0x7ff36810d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:03.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.588+0000 7ff36d518700 1 --2- 172.21.15.94:0/2249722561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3680fbcd0 0x7ff36810d930 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff35800eb40 tx=0x7ff35800b040 comp rx=0 tx=0).stop 2024-03-24T10:50:03.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.588+0000 7ff36d518700 1 -- 172.21.15.94:0/2249722561 shutdown_connections 2024-03-24T10:50:03.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.588+0000 7ff36d518700 1 --2- 172.21.15.94:0/2249722561 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7ff35006d910 0x7ff35006fdc0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:03.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.588+0000 7ff36d518700 1 --2- 172.21.15.94:0/2249722561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3680fbcd0 0x7ff36810d930 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:03.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.588+0000 7ff36d518700 1 --2- 172.21.15.94:0/2249722561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3680fcf80 0x7ff36810de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:03.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.588+0000 7ff36d518700 1 -- 172.21.15.94:0/2249722561 >> 172.21.15.94:0/2249722561 conn(0x7ff3680f7430 msgr2=0x7ff368066720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:50:03.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.588+0000 7ff36d518700 1 -- 172.21.15.94:0/2249722561 shutdown_connections 2024-03-24T10:50:03.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.588+0000 7ff36d518700 1 -- 172.21.15.94:0/2249722561 wait complete. 2024-03-24T10:50:03.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.692+0000 7f41f90db700 1 -- 172.21.15.94:0/1213365113 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41f40fdff0 msgr2=0x7f41f40fe460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:03.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.692+0000 7f41f90db700 1 --2- 172.21.15.94:0/1213365113 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41f40fdff0 0x7f41f40fe460 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f41e4004940 tx=0x7f41e4009e30 comp rx=0 tx=0).stop 2024-03-24T10:50:03.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.692+0000 7f41f90db700 1 -- 172.21.15.94:0/1213365113 shutdown_connections 2024-03-24T10:50:03.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.692+0000 7f41f90db700 1 --2- 172.21.15.94:0/1213365113 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41f40fdff0 0x7f41f40fe460 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:03.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.692+0000 7f41f90db700 1 --2- 172.21.15.94:0/1213365113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41f40fce80 0x7f41f40fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:03.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.692+0000 7f41f90db700 1 -- 172.21.15.94:0/1213365113 >> 172.21.15.94:0/1213365113 conn(0x7f41f40f8560 msgr2=0x7f41f40fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:50:03.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.692+0000 7f41f90db700 1 -- 172.21.15.94:0/1213365113 shutdown_connections 2024-03-24T10:50:03.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.692+0000 7f41f90db700 1 -- 172.21.15.94:0/1213365113 wait complete. 2024-03-24T10:50:03.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.692+0000 7f41f90db700 1 Processor -- start 2024-03-24T10:50:03.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.693+0000 7f41f90db700 1 -- start start 2024-03-24T10:50:03.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.693+0000 7f41f90db700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41f40fce80 0x7f41f410da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:50:03.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.693+0000 7f41f90db700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41f40fdff0 0x7f41f410df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:50:03.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.693+0000 7f41f90db700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f410e550 con 0x7f41f40fdff0 2024-03-24T10:50:03.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.693+0000 7f41f90db700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f410eda0 con 0x7f41f40fce80 2024-03-24T10:50:03.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.693+0000 7f41f259c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41f40fdff0 0x7f41f410df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:50:03.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.693+0000 7f41f259c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41f40fdff0 0x7f41f410df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46462/0 (socket says 172.21.15.94:46462) 2024-03-24T10:50:03.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.693+0000 7f41f259c700 1 -- 172.21.15.94:0/1588749801 learned_addr learned my addr 172.21.15.94:0/1588749801 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:50:03.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.693+0000 7f41f2d9d700 1 --2- 172.21.15.94:0/1588749801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41f40fce80 0x7f41f410da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:50:03.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.693+0000 7f41f259c700 1 -- 172.21.15.94:0/1588749801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41f40fce80 msgr2=0x7f41f410da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:03.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.693+0000 7f41f259c700 1 --2- 172.21.15.94:0/1588749801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41f40fce80 0x7f41f410da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:03.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.693+0000 7f41f259c700 1 -- 172.21.15.94:0/1588749801 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e4005040 con 0x7f41f40fdff0 2024-03-24T10:50:03.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.693+0000 7f41f2d9d700 1 --2- 172.21.15.94:0/1588749801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41f40fce80 0x7f41f410da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T10:50:03.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.693+0000 7f41f259c700 1 --2- 172.21.15.94:0/1588749801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41f40fdff0 0x7f41f410df50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f41e4009150 tx=0x7f41e4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:50:03.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.693+0000 7f41db7fe700 1 -- 172.21.15.94:0/1588749801 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41e400e040 con 0x7f41f40fdff0 2024-03-24T10:50:03.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.694+0000 7f41f90db700 1 -- 172.21.15.94:0/1588749801 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41f410f020 con 0x7f41f40fdff0 2024-03-24T10:50:03.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.693+0000 7f41db7fe700 1 -- 172.21.15.94:0/1588749801 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f41e400bcc0 con 0x7f41f40fdff0 2024-03-24T10:50:03.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.694+0000 7f41db7fe700 1 -- 172.21.15.94:0/1588749801 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41e401c630 con 0x7f41f40fdff0 2024-03-24T10:50:03.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.694+0000 7f41f90db700 1 -- 172.21.15.94:0/1588749801 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41f410f560 con 0x7f41f40fdff0 2024-03-24T10:50:03.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.695+0000 7f41f90db700 1 -- 172.21.15.94:0/1588749801 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41f405f5f0 con 0x7f41f40fdff0 2024-03-24T10:50:03.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.695+0000 7f41db7fe700 1 -- 172.21.15.94:0/1588749801 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 35) v1 ==== 91587+0+0 (secure 0 0 0) 0x7f41e4013420 con 0x7f41f40fdff0 2024-03-24T10:50:03.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.695+0000 7f41db7fe700 1 --2- 172.21.15.94:0/1588749801 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f41dc06d900 0x7f41dc06fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:50:03.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.696+0000 7f41db7fe700 1 -- 172.21.15.94:0/1588749801 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f41e408dd40 con 0x7f41f40fdff0 2024-03-24T10:50:03.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.696+0000 7f41f2d9d700 1 --2- 172.21.15.94:0/1588749801 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f41dc06d900 0x7f41dc06fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:50:03.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.696+0000 7f41f2d9d700 1 --2- 172.21.15.94:0/1588749801 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f41dc06d900 0x7f41dc06fdb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f41e8009fd0 tx=0x7f41e800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:50:03.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.700+0000 7f41db7fe700 1 -- 172.21.15.94:0/1588749801 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f41e405af60 con 0x7f41f40fdff0 2024-03-24T10:50:03.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.947+0000 7f41f90db700 1 -- 172.21.15.94:0/1588749801 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f41f410f780 con 0x7f41f40fdff0 2024-03-24T10:50:03.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.947+0000 7f41db7fe700 1 -- 172.21.15.94:0/1588749801 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1903 (secure 0 0 0) 0x7f41e405aaf0 con 0x7f41f40fdff0 2024-03-24T10:50:03.948 INFO:teuthology.orchestra.run.smithi094.stdout:e18 2024-03-24T10:50:03.948 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T10:50:03.948 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:50:03.948 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-03-24T10:50:03.949 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:50:03.949 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-03-24T10:50:03.949 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-03-24T10:50:03.949 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 18 2024-03-24T10:50:03.949 INFO:teuthology.orchestra.run.smithi094.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-24T10:50:03.949 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-03-24T10:43:39.435738+0000 2024-03-24T10:50:03.949 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-03-24T10:43:58.133901+0000 2024-03-24T10:50:03.949 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-03-24T10:50:03.949 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-03-24T10:50:03.949 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 60 2024-03-24T10:50:03.949 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-03-24T10:50:03.949 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-03-24T10:50:03.949 INFO:teuthology.orchestra.run.smithi094.stdout:max_xattr_size 65536 2024-03-24T10:50:03.949 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-03-24T10:50:03.949 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-03-24T10:50:03.950 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 46 2024-03-24T10:50:03.950 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:50:03.950 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-03-24T10:50:03.950 INFO:teuthology.orchestra.run.smithi094.stdout:in 0 2024-03-24T10:50:03.950 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=24251} 2024-03-24T10:50:03.950 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-03-24T10:50:03.950 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-03-24T10:50:03.950 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-03-24T10:50:03.950 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-03-24T10:50:03.950 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-03-24T10:50:03.950 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-03-24T10:50:03.950 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-03-24T10:50:03.950 INFO:teuthology.orchestra.run.smithi094.stdout:bal_rank_mask -1 2024-03-24T10:50:03.950 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 1 2024-03-24T10:50:03.950 INFO:teuthology.orchestra.run.smithi094.stdout:qdb_cluster leader: 0 members: 2024-03-24T10:50:03.951 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi096.egbmoq{0:24251} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.96:6824/1539752208,v1:172.21.15.96:6825/1539752208] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:50:03.951 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.llefgu{0:14500} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.94:6828/126762980,v1:172.21.15.94:6829/126762980] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:50:03.951 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:50:03.951 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:50:03.951 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-03-24T10:50:03.951 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:50:03.951 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.efiqte{-1:14512} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.94:6826/1636528860,v1:172.21.15.94:6827/1636528860] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:50:03.951 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi096.jtebhh{-1:24261} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.96:6826/1321087930,v1:172.21.15.96:6827/1321087930] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:50:03.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.950+0000 7f41f90db700 1 -- 172.21.15.94:0/1588749801 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f41dc06d900 msgr2=0x7f41dc06fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:03.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.950+0000 7f41f90db700 1 --2- 172.21.15.94:0/1588749801 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f41dc06d900 0x7f41dc06fdb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f41e8009fd0 tx=0x7f41e800b040 comp rx=0 tx=0).stop 2024-03-24T10:50:03.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.950+0000 7f41f90db700 1 -- 172.21.15.94:0/1588749801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41f40fdff0 msgr2=0x7f41f410df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:03.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.950+0000 7f41f90db700 1 --2- 172.21.15.94:0/1588749801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41f40fdff0 0x7f41f410df50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f41e4009150 tx=0x7f41e4009e30 comp rx=0 tx=0).stop 2024-03-24T10:50:03.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.950+0000 7f41f90db700 1 -- 172.21.15.94:0/1588749801 shutdown_connections 2024-03-24T10:50:03.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.950+0000 7f41f90db700 1 --2- 172.21.15.94:0/1588749801 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f41dc06d900 0x7f41dc06fdb0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:03.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.950+0000 7f41f90db700 1 --2- 172.21.15.94:0/1588749801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41f40fce80 0x7f41f410da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:03.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.950+0000 7f41f90db700 1 --2- 172.21.15.94:0/1588749801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41f40fdff0 0x7f41f410df50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:03.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.950+0000 7f41f90db700 1 -- 172.21.15.94:0/1588749801 >> 172.21.15.94:0/1588749801 conn(0x7f41f40f8560 msgr2=0x7f41f40f9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:50:03.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.950+0000 7f41f90db700 1 -- 172.21.15.94:0/1588749801 shutdown_connections 2024-03-24T10:50:03.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:03.950+0000 7f41f90db700 1 -- 172.21.15.94:0/1588749801 wait complete. 2024-03-24T10:50:03.952 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 18 2024-03-24T10:50:04.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.054+0000 7fbfbbdb7700 1 -- 172.21.15.94:0/1778611903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfb40fe050 msgr2=0x7fbfb40fe4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:04.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.054+0000 7fbfbbdb7700 1 --2- 172.21.15.94:0/1778611903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfb40fe050 0x7fbfb40fe4c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4004940 tx=0x7fbfa4009e30 comp rx=0 tx=0).stop 2024-03-24T10:50:04.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.054+0000 7fbfbbdb7700 1 -- 172.21.15.94:0/1778611903 shutdown_connections 2024-03-24T10:50:04.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.054+0000 7fbfbbdb7700 1 --2- 172.21.15.94:0/1778611903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfb40fe050 0x7fbfb40fe4c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:04.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.054+0000 7fbfbbdb7700 1 --2- 172.21.15.94:0/1778611903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbfb40fce50 0x7fbfb40fd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:04.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.054+0000 7fbfbbdb7700 1 -- 172.21.15.94:0/1778611903 >> 172.21.15.94:0/1778611903 conn(0x7fbfb40f8580 msgr2=0x7fbfb40fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:50:04.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.055+0000 7fbfbbdb7700 1 -- 172.21.15.94:0/1778611903 shutdown_connections 2024-03-24T10:50:04.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.055+0000 7fbfbbdb7700 1 -- 172.21.15.94:0/1778611903 wait complete. 2024-03-24T10:50:04.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.055+0000 7fbfbbdb7700 1 Processor -- start 2024-03-24T10:50:04.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.055+0000 7fbfbbdb7700 1 -- start start 2024-03-24T10:50:04.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.055+0000 7fbfbbdb7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbfb40fce50 0x7fbfb4071c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:50:04.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.055+0000 7fbfbbdb7700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfb40fe050 0x7fbfb4070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:50:04.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.055+0000 7fbfbbdb7700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb4072260 con 0x7fbfb40fe050 2024-03-24T10:50:04.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.055+0000 7fbfbbdb7700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb4070850 con 0x7fbfb40fce50 2024-03-24T10:50:04.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.055+0000 7fbfb9352700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfb40fe050 0x7fbfb4070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:50:04.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.055+0000 7fbfb9352700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfb40fe050 0x7fbfb4070310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46482/0 (socket says 172.21.15.94:46482) 2024-03-24T10:50:04.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.055+0000 7fbfb9352700 1 -- 172.21.15.94:0/1984654181 learned_addr learned my addr 172.21.15.94:0/1984654181 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:50:04.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.056+0000 7fbfb9b53700 1 --2- 172.21.15.94:0/1984654181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbfb40fce50 0x7fbfb4071c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:50:04.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.056+0000 7fbfb9352700 1 -- 172.21.15.94:0/1984654181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbfb40fce50 msgr2=0x7fbfb4071c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:04.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.056+0000 7fbfb9352700 1 --2- 172.21.15.94:0/1984654181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbfb40fce50 0x7fbfb4071c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:04.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.056+0000 7fbfb9352700 1 -- 172.21.15.94:0/1984654181 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa4005040 con 0x7fbfb40fe050 2024-03-24T10:50:04.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.056+0000 7fbfb9b53700 1 --2- 172.21.15.94:0/1984654181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbfb40fce50 0x7fbfb4071c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T10:50:04.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.056+0000 7fbfb9352700 1 --2- 172.21.15.94:0/1984654181 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfb40fe050 0x7fbfb4070310 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbfa400b3b0 tx=0x7fbfa4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:50:04.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.056+0000 7fbfaa7fc700 1 -- 172.21.15.94:0/1984654181 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfa400e040 con 0x7fbfb40fe050 2024-03-24T10:50:04.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.056+0000 7fbfbbdb7700 1 -- 172.21.15.94:0/1984654181 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfb4070aa0 con 0x7fbfb40fe050 2024-03-24T10:50:04.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.056+0000 7fbfaa7fc700 1 -- 172.21.15.94:0/1984654181 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbfa4004290 con 0x7fbfb40fe050 2024-03-24T10:50:04.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.056+0000 7fbfaa7fc700 1 -- 172.21.15.94:0/1984654181 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfa401c630 con 0x7fbfb40fe050 2024-03-24T10:50:04.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.056+0000 7fbfbbdb7700 1 -- 172.21.15.94:0/1984654181 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfb4071060 con 0x7fbfb40fe050 2024-03-24T10:50:04.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.057+0000 7fbfbbdb7700 1 -- 172.21.15.94:0/1984654181 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfb404c710 con 0x7fbfb40fe050 2024-03-24T10:50:04.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.057+0000 7fbfaa7fc700 1 -- 172.21.15.94:0/1984654181 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 35) v1 ==== 91587+0+0 (secure 0 0 0) 0x7fbfa400bbf0 con 0x7fbfb40fe050 2024-03-24T10:50:04.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.058+0000 7fbfaa7fc700 1 --2- 172.21.15.94:0/1984654181 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fbfa006d910 0x7fbfa006fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:50:04.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.058+0000 7fbfaa7fc700 1 -- 172.21.15.94:0/1984654181 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fbfa408e640 con 0x7fbfb40fe050 2024-03-24T10:50:04.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.058+0000 7fbfb9b53700 1 --2- 172.21.15.94:0/1984654181 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fbfa006d910 0x7fbfa006fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:50:04.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.058+0000 7fbfb9b53700 1 --2- 172.21.15.94:0/1984654181 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fbfa006d910 0x7fbfa006fdc0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0009fd0 tx=0x7fbfb0009f00 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:50:04.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.062+0000 7fbfaa7fc700 1 -- 172.21.15.94:0/1984654181 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fbfa405b7c0 con 0x7fbfb40fe050 2024-03-24T10:50:04.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.279+0000 7fbfbbdb7700 1 -- 172.21.15.94:0/1984654181 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbfb4102310 con 0x7fbfa006d910 2024-03-24T10:50:04.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.281+0000 7fbfaa7fc700 1 -- 172.21.15.94:0/1984654181 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fbfb4102310 con 0x7fbfa006d910 2024-03-24T10:50:04.281 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-03-24T10:50:04.281 INFO:teuthology.orchestra.run.smithi094.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919", 2024-03-24T10:50:04.282 INFO:teuthology.orchestra.run.smithi094.stdout: "in_progress": true, 2024-03-24T10:50:04.282 INFO:teuthology.orchestra.run.smithi094.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T10:50:04.282 INFO:teuthology.orchestra.run.smithi094.stdout: "services_complete": [ 2024-03-24T10:50:04.282 INFO:teuthology.orchestra.run.smithi094.stdout: "mon", 2024-03-24T10:50:04.282 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-03-24T10:50:04.282 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr" 2024-03-24T10:50:04.282 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-03-24T10:50:04.282 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": "7/21 daemons upgraded", 2024-03-24T10:50:04.282 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "Currently upgrading osd daemons", 2024-03-24T10:50:04.282 INFO:teuthology.orchestra.run.smithi094.stdout: "is_paused": false 2024-03-24T10:50:04.282 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-03-24T10:50:04.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.283+0000 7fbfbbdb7700 1 -- 172.21.15.94:0/1984654181 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fbfa006d910 msgr2=0x7fbfa006fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:04.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.283+0000 7fbfbbdb7700 1 --2- 172.21.15.94:0/1984654181 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fbfa006d910 0x7fbfa006fdc0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0009fd0 tx=0x7fbfb0009f00 comp rx=0 tx=0).stop 2024-03-24T10:50:04.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.283+0000 7fbfbbdb7700 1 -- 172.21.15.94:0/1984654181 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfb40fe050 msgr2=0x7fbfb4070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:04.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.283+0000 7fbfbbdb7700 1 --2- 172.21.15.94:0/1984654181 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfb40fe050 0x7fbfb4070310 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbfa400b3b0 tx=0x7fbfa4009e30 comp rx=0 tx=0).stop 2024-03-24T10:50:04.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.283+0000 7fbfbbdb7700 1 -- 172.21.15.94:0/1984654181 shutdown_connections 2024-03-24T10:50:04.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.283+0000 7fbfbbdb7700 1 --2- 172.21.15.94:0/1984654181 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fbfa006d910 0x7fbfa006fdc0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:04.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.283+0000 7fbfbbdb7700 1 --2- 172.21.15.94:0/1984654181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbfb40fce50 0x7fbfb4071c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:04.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.283+0000 7fbfbbdb7700 1 --2- 172.21.15.94:0/1984654181 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfb40fe050 0x7fbfb4070310 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:04.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.283+0000 7fbfbbdb7700 1 -- 172.21.15.94:0/1984654181 >> 172.21.15.94:0/1984654181 conn(0x7fbfb40f8580 msgr2=0x7fbfb40f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:50:04.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.283+0000 7fbfbbdb7700 1 -- 172.21.15.94:0/1984654181 shutdown_connections 2024-03-24T10:50:04.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.283+0000 7fbfbbdb7700 1 -- 172.21.15.94:0/1984654181 wait complete. 2024-03-24T10:50:04.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.398+0000 7fd2ba5fd700 1 -- 172.21.15.94:0/589728569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2b40fbcd0 msgr2=0x7fd2b40fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:04.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.398+0000 7fd2ba5fd700 1 --2- 172.21.15.94:0/589728569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2b40fbcd0 0x7fd2b40fc0e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4004940 tx=0x7fd2a4009e30 comp rx=0 tx=0).stop 2024-03-24T10:50:04.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.398+0000 7fd2ba5fd700 1 -- 172.21.15.94:0/589728569 shutdown_connections 2024-03-24T10:50:04.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.398+0000 7fd2ba5fd700 1 --2- 172.21.15.94:0/589728569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2b40fcf80 0x7fd2b40fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:04.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.398+0000 7fd2ba5fd700 1 --2- 172.21.15.94:0/589728569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2b40fbcd0 0x7fd2b40fc0e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:04.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.398+0000 7fd2ba5fd700 1 -- 172.21.15.94:0/589728569 >> 172.21.15.94:0/589728569 conn(0x7fd2b40f7430 msgr2=0x7fd2b40f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:50:04.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.398+0000 7fd2ba5fd700 1 -- 172.21.15.94:0/589728569 shutdown_connections 2024-03-24T10:50:04.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.398+0000 7fd2ba5fd700 1 -- 172.21.15.94:0/589728569 wait complete. 2024-03-24T10:50:04.400 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:04 smithi094 ceph-mon[93957]: failed to encode map e52 with expected crc 2024-03-24T10:50:04.400 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:04 smithi094 ceph-mon[93957]: from='client.34194 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:50:04.400 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:04 smithi094 ceph-mon[93957]: from='client.34198 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:50:04.400 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:04 smithi094 ceph-mon[93957]: pgmap v50: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 196 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 44/219 objects degraded (20.091%) 2024-03-24T10:50:04.400 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:04 smithi094 ceph-mon[93957]: from='osd.0 [v2:172.21.15.94:6802/1160876432,v1:172.21.15.94:6803/1160876432]' entity='osd.0' 2024-03-24T10:50:04.400 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:04 smithi094 ceph-mon[93957]: failed to encode map e52 with expected crc 2024-03-24T10:50:04.400 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:04 smithi094 ceph-mon[93957]: failed to encode map e52 with expected crc 2024-03-24T10:50:04.400 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:04 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/2249722561' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:50:04.400 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:04 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/1588749801' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T10:50:04.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.398+0000 7fd2ba5fd700 1 Processor -- start 2024-03-24T10:50:04.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.399+0000 7fd2ba5fd700 1 -- start start 2024-03-24T10:50:04.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.399+0000 7fd2ba5fd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2b40fbcd0 0x7fd2b4110720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:50:04.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.399+0000 7fd2ba5fd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2b40fcf80 0x7fd2b4112c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:50:04.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.399+0000 7fd2ba5fd700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b4066350 con 0x7fd2b40fcf80 2024-03-24T10:50:04.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.399+0000 7fd2ba5fd700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b4066490 con 0x7fd2b40fbcd0 2024-03-24T10:50:04.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.399+0000 7fd2b37fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2b40fcf80 0x7fd2b4112c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:50:04.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.399+0000 7fd2b37fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2b40fcf80 0x7fd2b4112c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46500/0 (socket says 172.21.15.94:46500) 2024-03-24T10:50:04.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.399+0000 7fd2b3fff700 1 --2- 172.21.15.94:0/113346454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2b40fbcd0 0x7fd2b4110720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:50:04.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.399+0000 7fd2b37fe700 1 -- 172.21.15.94:0/113346454 learned_addr learned my addr 172.21.15.94:0/113346454 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:50:04.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.399+0000 7fd2b37fe700 1 -- 172.21.15.94:0/113346454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2b40fbcd0 msgr2=0x7fd2b4110720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:04.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.399+0000 7fd2b37fe700 1 --2- 172.21.15.94:0/113346454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2b40fbcd0 0x7fd2b4110720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:04.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.399+0000 7fd2b37fe700 1 -- 172.21.15.94:0/113346454 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a4005040 con 0x7fd2b40fcf80 2024-03-24T10:50:04.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.399+0000 7fd2b37fe700 1 --2- 172.21.15.94:0/113346454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2b40fcf80 0x7fd2b4112c70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd2a800c8c0 tx=0x7fd2a80044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:50:04.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.400+0000 7fd2b0ff9700 1 -- 172.21.15.94:0/113346454 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2a800a760 con 0x7fd2b40fcf80 2024-03-24T10:50:04.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.400+0000 7fd2b0ff9700 1 -- 172.21.15.94:0/113346454 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd2a800ad90 con 0x7fd2b40fcf80 2024-03-24T10:50:04.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.400+0000 7fd2ba5fd700 1 -- 172.21.15.94:0/113346454 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2b4113210 con 0x7fd2b40fcf80 2024-03-24T10:50:04.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.400+0000 7fd2b0ff9700 1 -- 172.21.15.94:0/113346454 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2a80113a0 con 0x7fd2b40fcf80 2024-03-24T10:50:04.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.400+0000 7fd2ba5fd700 1 -- 172.21.15.94:0/113346454 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2b41137d0 con 0x7fd2b40fcf80 2024-03-24T10:50:04.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.401+0000 7fd2ba5fd700 1 -- 172.21.15.94:0/113346454 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2b405f5f0 con 0x7fd2b40fcf80 2024-03-24T10:50:04.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.401+0000 7fd2b0ff9700 1 -- 172.21.15.94:0/113346454 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 35) v1 ==== 91587+0+0 (secure 0 0 0) 0x7fd2a8012400 con 0x7fd2b40fcf80 2024-03-24T10:50:04.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.402+0000 7fd2b0ff9700 1 --2- 172.21.15.94:0/113346454 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fd29c06d900 0x7fd29c06fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:50:04.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.402+0000 7fd2b0ff9700 1 -- 172.21.15.94:0/113346454 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fd2a808d000 con 0x7fd2b40fcf80 2024-03-24T10:50:04.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.402+0000 7fd2b3fff700 1 --2- 172.21.15.94:0/113346454 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fd29c06d900 0x7fd29c06fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:50:04.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.402+0000 7fd2b3fff700 1 --2- 172.21.15.94:0/113346454 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fd29c06d900 0x7fd29c06fdb0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd2a400b840 tx=0x7fd2a4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:50:04.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.406+0000 7fd2b0ff9700 1 -- 172.21.15.94:0/113346454 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd2a8056800 con 0x7fd2b40fcf80 2024-03-24T10:50:04.578 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:04 smithi096 ceph-mon[84258]: failed to encode map e52 with expected crc 2024-03-24T10:50:04.578 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:04 smithi096 ceph-mon[84258]: from='client.34194 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:50:04.578 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:04 smithi096 ceph-mon[84258]: from='client.34198 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:50:04.578 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:04 smithi096 ceph-mon[84258]: pgmap v50: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 196 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 44/219 objects degraded (20.091%) 2024-03-24T10:50:04.578 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:04 smithi096 ceph-mon[84258]: from='osd.0 [v2:172.21.15.94:6802/1160876432,v1:172.21.15.94:6803/1160876432]' entity='osd.0' 2024-03-24T10:50:04.578 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:04 smithi096 ceph-mon[84258]: failed to encode map e52 with expected crc 2024-03-24T10:50:04.578 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:04 smithi096 ceph-mon[84258]: failed to encode map e52 with expected crc 2024-03-24T10:50:04.578 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:04 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/2249722561' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:50:04.578 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:04 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/1588749801' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T10:50:04.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.715+0000 7fd2ba5fd700 1 -- 172.21.15.94:0/113346454 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd2b404bed0 con 0x7fd2b40fcf80 2024-03-24T10:50:04.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.715+0000 7fd2b0ff9700 1 -- 172.21.15.94:0/113346454 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1183 (secure 0 0 0) 0x7fd2a8059ec0 con 0x7fd2b40fcf80 2024-03-24T10:50:04.716 INFO:teuthology.orchestra.run.smithi094.stdout:HEALTH_WARN Degraded data redundancy: 44/219 objects degraded (20.091%), 18 pgs degraded 2024-03-24T10:50:04.716 INFO:teuthology.orchestra.run.smithi094.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 44/219 objects degraded (20.091%), 18 pgs degraded 2024-03-24T10:50:04.716 INFO:teuthology.orchestra.run.smithi094.stdout: pg 1.0 is active+undersized+degraded, acting [3,1] 2024-03-24T10:50:04.716 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.0 is active+undersized+degraded, acting [3,1] 2024-03-24T10:50:04.717 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.1 is active+undersized+degraded, acting [2,1] 2024-03-24T10:50:04.717 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.2 is active+undersized+degraded, acting [5,1] 2024-03-24T10:50:04.717 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.4 is active+undersized+degraded, acting [1,4] 2024-03-24T10:50:04.717 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.5 is active+undersized+degraded, acting [3,4] 2024-03-24T10:50:04.717 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.9 is active+undersized+degraded, acting [1,4] 2024-03-24T10:50:04.717 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.c is active+undersized+degraded, acting [2,3] 2024-03-24T10:50:04.717 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.e is active+undersized+degraded, acting [2,3] 2024-03-24T10:50:04.717 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.f is active+undersized+degraded, acting [4,5] 2024-03-24T10:50:04.717 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.10 is active+undersized+degraded, acting [2,1] 2024-03-24T10:50:04.717 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.13 is active+undersized+degraded, acting [4,2] 2024-03-24T10:50:04.717 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.15 is active+undersized+degraded, acting [1,3] 2024-03-24T10:50:04.717 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.19 is active+undersized+degraded, acting [4,2] 2024-03-24T10:50:04.717 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.1b is active+undersized+degraded, acting [1,5] 2024-03-24T10:50:04.717 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.1d is active+undersized+degraded, acting [3,5] 2024-03-24T10:50:04.717 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.1e is active+undersized+degraded, acting [2,5] 2024-03-24T10:50:04.717 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.1f is active+undersized+degraded, acting [3,4] 2024-03-24T10:50:04.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.718+0000 7fd2ba5fd700 1 -- 172.21.15.94:0/113346454 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fd29c06d900 msgr2=0x7fd29c06fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:04.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.718+0000 7fd2ba5fd700 1 --2- 172.21.15.94:0/113346454 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fd29c06d900 0x7fd29c06fdb0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd2a400b840 tx=0x7fd2a4009e30 comp rx=0 tx=0).stop 2024-03-24T10:50:04.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.718+0000 7fd2ba5fd700 1 -- 172.21.15.94:0/113346454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2b40fcf80 msgr2=0x7fd2b4112c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:04.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.718+0000 7fd2ba5fd700 1 --2- 172.21.15.94:0/113346454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2b40fcf80 0x7fd2b4112c70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd2a800c8c0 tx=0x7fd2a80044c0 comp rx=0 tx=0).stop 2024-03-24T10:50:04.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.718+0000 7fd2ba5fd700 1 -- 172.21.15.94:0/113346454 shutdown_connections 2024-03-24T10:50:04.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.718+0000 7fd2ba5fd700 1 --2- 172.21.15.94:0/113346454 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fd29c06d900 0x7fd29c06fdb0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:04.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.718+0000 7fd2ba5fd700 1 --2- 172.21.15.94:0/113346454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2b40fbcd0 0x7fd2b4110720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:04.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.718+0000 7fd2ba5fd700 1 --2- 172.21.15.94:0/113346454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2b40fcf80 0x7fd2b4112c70 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:04.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.718+0000 7fd2ba5fd700 1 -- 172.21.15.94:0/113346454 >> 172.21.15.94:0/113346454 conn(0x7fd2b40f7430 msgr2=0x7fd2b40f8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:50:04.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.718+0000 7fd2ba5fd700 1 -- 172.21.15.94:0/113346454 shutdown_connections 2024-03-24T10:50:04.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:04.718+0000 7fd2ba5fd700 1 -- 172.21.15.94:0/113346454 wait complete. 2024-03-24T10:50:05.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:05 smithi096 ceph-mon[84258]: failed to encode map e52 with expected crc 2024-03-24T10:50:05.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:05 smithi096 ceph-mon[84258]: from='client.34210 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:50:05.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:05 smithi096 ceph-mon[84258]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T10:50:05.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:05 smithi096 ceph-mon[84258]: failed to encode map e53 with expected crc 2024-03-24T10:50:05.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:05 smithi096 ceph-mon[84258]: osd.0 [v2:172.21.15.94:6802/1160876432,v1:172.21.15.94:6803/1160876432] boot 2024-03-24T10:50:05.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:05 smithi096 ceph-mon[84258]: osdmap e53: 6 total, 6 up, 6 in 2024-03-24T10:50:05.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:05 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T10:50:05.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:05 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/113346454' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T10:50:05.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:05 smithi096 ceph-mon[84258]: osdmap e54: 6 total, 6 up, 6 in 2024-03-24T10:50:05.600 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:05 smithi094 ceph-mon[93957]: failed to encode map e52 with expected crc 2024-03-24T10:50:05.600 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:05 smithi094 ceph-mon[93957]: from='client.34210 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:50:05.600 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:05 smithi094 ceph-mon[93957]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T10:50:05.600 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:05 smithi094 ceph-mon[93957]: failed to encode map e53 with expected crc 2024-03-24T10:50:05.600 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:05 smithi094 ceph-mon[93957]: osd.0 [v2:172.21.15.94:6802/1160876432,v1:172.21.15.94:6803/1160876432] boot 2024-03-24T10:50:05.600 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:05 smithi094 ceph-mon[93957]: osdmap e53: 6 total, 6 up, 6 in 2024-03-24T10:50:05.600 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:05 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T10:50:05.600 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:05 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/113346454' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T10:50:05.601 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:05 smithi094 ceph-mon[93957]: osdmap e54: 6 total, 6 up, 6 in 2024-03-24T10:50:06.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:06 smithi094 ceph-mon[93957]: pgmap v52: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 196 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 44/219 objects degraded (20.091%) 2024-03-24T10:50:06.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:06 smithi094 ceph-mon[93957]: failed to encode map e54 with expected crc 2024-03-24T10:50:06.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:06 smithi094 ceph-mon[93957]: failed to encode map e54 with expected crc 2024-03-24T10:50:06.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:06 smithi094 ceph-mon[93957]: failed to encode map e54 with expected crc 2024-03-24T10:50:06.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:06 smithi094 ceph-mon[93957]: failed to encode map e54 with expected crc 2024-03-24T10:50:06.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:06 smithi094 ceph-mon[93957]: failed to encode map e54 with expected crc 2024-03-24T10:50:06.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:06 smithi094 ceph-mon[93957]: failed to encode map e54 with expected crc 2024-03-24T10:50:06.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:06 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:50:06.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:06 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:50:06.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:06 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:50:06.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:06 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:50:06.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:06 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:50:06.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:06 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:50:06.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:06 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:50:06.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:06 smithi096 ceph-mon[84258]: pgmap v52: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 196 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 44/219 objects degraded (20.091%) 2024-03-24T10:50:06.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:06 smithi096 ceph-mon[84258]: failed to encode map e54 with expected crc 2024-03-24T10:50:06.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:06 smithi096 ceph-mon[84258]: failed to encode map e54 with expected crc 2024-03-24T10:50:06.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:06 smithi096 ceph-mon[84258]: failed to encode map e54 with expected crc 2024-03-24T10:50:06.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:06 smithi096 ceph-mon[84258]: failed to encode map e54 with expected crc 2024-03-24T10:50:06.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:06 smithi096 ceph-mon[84258]: failed to encode map e54 with expected crc 2024-03-24T10:50:06.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:06 smithi096 ceph-mon[84258]: failed to encode map e54 with expected crc 2024-03-24T10:50:06.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:06 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:50:06.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:06 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:50:06.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:06 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:50:06.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:06 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:50:06.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:06 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:50:06.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:06 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:50:06.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:06 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:50:07.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:07 smithi094 ceph-mon[93957]: mgrmap e36: smithi094.yaethu(active, since 91s), standbys: smithi096.gfonnu 2024-03-24T10:50:07.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:07 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:50:07.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:07 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:50:07.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:07 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:50:07.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:07 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:50:07.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:07 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-24T10:50:07.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:07 smithi096 ceph-mon[84258]: mgrmap e36: smithi094.yaethu(active, since 91s), standbys: smithi096.gfonnu 2024-03-24T10:50:07.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:07 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:50:07.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:07 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:50:07.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:07 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:50:07.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:07 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:50:07.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:07 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-24T10:50:08.659 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:08 smithi094 ceph-mon[93957]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-24T10:50:08.659 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:08 smithi094 ceph-mon[93957]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-03-24T10:50:08.659 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:08 smithi094 ceph-mon[93957]: pgmap v54: 65 pgs: 3 active+undersized+degraded, 6 active+undersized, 56 active+clean; 196 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 3 op/s; 6/219 objects degraded (2.740%) 2024-03-24T10:50:08.659 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:08 smithi094 ceph-mon[93957]: Health check update: Degraded data redundancy: 6/219 objects degraded (2.740%), 3 pgs degraded (PG_DEGRADED) 2024-03-24T10:50:08.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:08 smithi096 ceph-mon[84258]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-24T10:50:08.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:08 smithi096 ceph-mon[84258]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-03-24T10:50:08.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:08 smithi096 ceph-mon[84258]: pgmap v54: 65 pgs: 3 active+undersized+degraded, 6 active+undersized, 56 active+clean; 196 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 3 op/s; 6/219 objects degraded (2.740%) 2024-03-24T10:50:08.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:08 smithi096 ceph-mon[84258]: Health check update: Degraded data redundancy: 6/219 objects degraded (2.740%), 3 pgs degraded (PG_DEGRADED) 2024-03-24T10:50:09.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:09 smithi096 ceph-mon[84258]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 6/219 objects degraded (2.740%), 3 pgs degraded) 2024-03-24T10:50:09.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:09 smithi096 ceph-mon[84258]: Cluster is now healthy 2024-03-24T10:50:09.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:09 smithi094 ceph-mon[93957]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 6/219 objects degraded (2.740%), 3 pgs degraded) 2024-03-24T10:50:09.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:09 smithi094 ceph-mon[93957]: Cluster is now healthy 2024-03-24T10:50:10.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:10 smithi094 ceph-mon[93957]: pgmap v55: 65 pgs: 65 active+clean; 196 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 608 B/s rd, 456 B/s wr, 1 op/s 2024-03-24T10:50:10.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:10 smithi096 ceph-mon[84258]: pgmap v55: 65 pgs: 65 active+clean; 196 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 608 B/s rd, 456 B/s wr, 1 op/s 2024-03-24T10:50:12.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:12 smithi094 ceph-mon[93957]: pgmap v56: 65 pgs: 65 active+clean; 196 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 511 B/s rd, 383 B/s wr, 1 op/s 2024-03-24T10:50:12.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:12 smithi096 ceph-mon[84258]: pgmap v56: 65 pgs: 65 active+clean; 196 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 511 B/s rd, 383 B/s wr, 1 op/s 2024-03-24T10:50:14.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:14 smithi094 ceph-mon[93957]: pgmap v57: 65 pgs: 65 active+clean; 196 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 342 B/s wr, 2 op/s 2024-03-24T10:50:14.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:14 smithi096 ceph-mon[84258]: pgmap v57: 65 pgs: 65 active+clean; 196 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 342 B/s wr, 2 op/s 2024-03-24T10:50:16.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:16 smithi094 ceph-mon[93957]: pgmap v58: 65 pgs: 65 active+clean; 196 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 307 B/s wr, 2 op/s 2024-03-24T10:50:16.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:16 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:50:16.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:16 smithi096 ceph-mon[84258]: pgmap v58: 65 pgs: 65 active+clean; 196 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 307 B/s wr, 2 op/s 2024-03-24T10:50:16.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:16 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:50:18.658 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:18 smithi094 ceph-mon[93957]: pgmap v59: 65 pgs: 65 active+clean; 196 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 256 B/s wr, 2 op/s 2024-03-24T10:50:18.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:18 smithi096 ceph-mon[84258]: pgmap v59: 65 pgs: 65 active+clean; 196 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 256 B/s wr, 2 op/s 2024-03-24T10:50:20.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:20 smithi094 ceph-mon[93957]: pgmap v60: 65 pgs: 65 active+clean; 196 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 938 B/s rd, 1 op/s 2024-03-24T10:50:20.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:20 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:50:20.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:20 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:50:20.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:20 smithi096 ceph-mon[84258]: pgmap v60: 65 pgs: 65 active+clean; 196 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 938 B/s rd, 1 op/s 2024-03-24T10:50:20.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:20 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:50:20.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:20 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:50:22.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:22 smithi094 ceph-mon[93957]: pgmap v61: 65 pgs: 65 active+clean; 196 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 938 B/s rd, 1 op/s 2024-03-24T10:50:22.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:22 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-24T10:50:22.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:22 smithi096 ceph-mon[84258]: pgmap v61: 65 pgs: 65 active+clean; 196 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 938 B/s rd, 1 op/s 2024-03-24T10:50:22.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:22 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-24T10:50:23.627 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:23 smithi094 ceph-mon[93957]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-24T10:50:23.628 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:23 smithi094 ceph-mon[93957]: Upgrade: osd.1 is safe to restart 2024-03-24T10:50:23.628 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:23 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:50:23.628 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:23 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-24T10:50:23.628 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:23 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:50:23.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:23 smithi096 ceph-mon[84258]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-24T10:50:23.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:23 smithi096 ceph-mon[84258]: Upgrade: osd.1 is safe to restart 2024-03-24T10:50:23.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:23 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:50:23.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:23 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-24T10:50:23.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:23 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:50:24.596 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:24 smithi094 ceph-mon[93957]: Upgrade: Updating osd.1 2024-03-24T10:50:24.597 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:24 smithi094 ceph-mon[93957]: Deploying daemon osd.1 on smithi094 2024-03-24T10:50:24.597 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:24 smithi094 ceph-mon[93957]: pgmap v62: 65 pgs: 65 active+clean; 196 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-03-24T10:50:24.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:24 smithi096 ceph-mon[84258]: Upgrade: Updating osd.1 2024-03-24T10:50:24.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:24 smithi096 ceph-mon[84258]: Deploying daemon osd.1 on smithi094 2024-03-24T10:50:24.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:24 smithi096 ceph-mon[84258]: pgmap v62: 65 pgs: 65 active+clean; 196 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-03-24T10:50:25.997 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:25 smithi094 systemd[1]: Stopping Ceph osd.1 for 386110d4-e9ca-11ee-95cd-87774f69a715... 2024-03-24T10:50:26.348 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:26 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-1[55045]: 2024-03-24T10:50:26.075+0000 7f4d1e323700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T10:50:26.349 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:26 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-1[55045]: 2024-03-24T10:50:26.075+0000 7f4d1e323700 -1 osd.1 54 *** Got signal Terminated *** 2024-03-24T10:50:26.349 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:26 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-1[55045]: 2024-03-24T10:50:26.075+0000 7f4d1e323700 -1 osd.1 54 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T10:50:26.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:26 smithi094 ceph-mon[93957]: pgmap v63: 65 pgs: 65 active+clean; 196 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:50:26.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:26 smithi094 ceph-mon[93957]: osd.1 marked itself down and dead 2024-03-24T10:50:26.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:26 smithi096 ceph-mon[84258]: pgmap v63: 65 pgs: 65 active+clean; 196 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:50:26.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:26 smithi096 ceph-mon[84258]: osd.1 marked itself down and dead 2024-03-24T10:50:27.176 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:26 smithi094 podman[107617]: 2024-03-24 10:50:26.923194929 +0000 UTC m=+0.929543475 container died 4e737a80d73313371e43b4733634bae94bed77ffd4c4150bd13e731c12d9644a (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-1, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=quincy-9c4e0c3, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-24T10:50:27.497 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:27 smithi094 podman[107617]: 2024-03-24 10:50:27.276169644 +0000 UTC m=+1.282518180 container cleanup 4e737a80d73313371e43b4733634bae94bed77ffd4c4150bd13e731c12d9644a (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=quincy-9c4e0c3, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-24T10:50:27.497 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:27 smithi094 bash[107617]: ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-1 2024-03-24T10:50:27.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:27 smithi094 ceph-mon[93957]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T10:50:27.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:27 smithi094 ceph-mon[93957]: osdmap e55: 6 total, 5 up, 6 in 2024-03-24T10:50:27.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:27 smithi094 ceph-mon[93957]: failed to encode map e55 with expected crc 2024-03-24T10:50:27.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:27 smithi094 ceph-mon[93957]: failed to encode map e55 with expected crc 2024-03-24T10:50:27.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:27 smithi094 ceph-mon[93957]: failed to encode map e55 with expected crc 2024-03-24T10:50:27.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:27 smithi094 ceph-mon[93957]: failed to encode map e55 with expected crc 2024-03-24T10:50:27.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:27 smithi096 ceph-mon[84258]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T10:50:27.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:27 smithi096 ceph-mon[84258]: osdmap e55: 6 total, 5 up, 6 in 2024-03-24T10:50:27.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:27 smithi096 ceph-mon[84258]: failed to encode map e55 with expected crc 2024-03-24T10:50:27.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:27 smithi096 ceph-mon[84258]: failed to encode map e55 with expected crc 2024-03-24T10:50:27.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:27 smithi096 ceph-mon[84258]: failed to encode map e55 with expected crc 2024-03-24T10:50:27.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:27 smithi096 ceph-mon[84258]: failed to encode map e55 with expected crc 2024-03-24T10:50:27.898 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:27 smithi094 podman[107634]: 2024-03-24 10:50:27.552710539 +0000 UTC m=+0.631072257 container remove 4e737a80d73313371e43b4733634bae94bed77ffd4c4150bd13e731c12d9644a (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-1, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, ceph=True, org.label-schema.schema-version=1.0, RELEASE=quincy-9c4e0c3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-24T10:50:28.189 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:27 smithi094 podman[107722]: 2024-03-24 10:50:27.802001743 +0000 UTC m=+0.017830345 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-24T10:50:28.189 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:27 smithi094 podman[107722]: 2024-03-24 10:50:27.912012603 +0000 UTC m=+0.127841190 container create 19d5c2185eec614c00ca0f49cf3b6ffa82649e91357fc6f03690ebea86fcad88 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T10:50:28.189 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:28 smithi094 podman[107722]: 2024-03-24 10:50:28.097766139 +0000 UTC m=+0.313594727 container init 19d5c2185eec614c00ca0f49cf3b6ffa82649e91357fc6f03690ebea86fcad88 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-1-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, ceph=True) 2024-03-24T10:50:28.189 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:28 smithi094 podman[107722]: 2024-03-24 10:50:28.101006635 +0000 UTC m=+0.316835232 container start 19d5c2185eec614c00ca0f49cf3b6ffa82649e91357fc6f03690ebea86fcad88 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-1-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, io.buildah.version=1.33.5) 2024-03-24T10:50:28.189 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:28 smithi094 podman[107722]: 2024-03-24 10:50:28.189388491 +0000 UTC m=+0.405217098 container attach 19d5c2185eec614c00ca0f49cf3b6ffa82649e91357fc6f03690ebea86fcad88 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T10:50:28.497 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:28 smithi094 podman[107722]: 2024-03-24 10:50:28.39372921 +0000 UTC m=+0.609557793 container died 19d5c2185eec614c00ca0f49cf3b6ffa82649e91357fc6f03690ebea86fcad88 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-1-deactivate, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-24T10:50:28.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:28 smithi094 ceph-mon[93957]: pgmap v65: 65 pgs: 12 stale+active+clean, 53 active+clean; 196 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-03-24T10:50:28.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:28 smithi094 ceph-mon[93957]: failed to encode map e56 with expected crc 2024-03-24T10:50:28.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:28 smithi094 ceph-mon[93957]: osdmap e56: 6 total, 5 up, 6 in 2024-03-24T10:50:28.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:28 smithi094 ceph-mon[93957]: failed to encode map e56 with expected crc 2024-03-24T10:50:28.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:28 smithi094 ceph-mon[93957]: failed to encode map e56 with expected crc 2024-03-24T10:50:28.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:28 smithi094 ceph-mon[93957]: failed to encode map e56 with expected crc 2024-03-24T10:50:28.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:28 smithi094 ceph-mon[93957]: failed to encode map e56 with expected crc 2024-03-24T10:50:28.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:28 smithi094 ceph-mon[93957]: failed to encode map e56 with expected crc 2024-03-24T10:50:28.726 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:28 smithi096 ceph-mon[84258]: pgmap v65: 65 pgs: 12 stale+active+clean, 53 active+clean; 196 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-03-24T10:50:28.726 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:28 smithi096 ceph-mon[84258]: failed to encode map e56 with expected crc 2024-03-24T10:50:28.726 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:28 smithi096 ceph-mon[84258]: osdmap e56: 6 total, 5 up, 6 in 2024-03-24T10:50:28.726 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:28 smithi096 ceph-mon[84258]: failed to encode map e56 with expected crc 2024-03-24T10:50:28.726 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:28 smithi096 ceph-mon[84258]: failed to encode map e56 with expected crc 2024-03-24T10:50:28.727 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:28 smithi096 ceph-mon[84258]: failed to encode map e56 with expected crc 2024-03-24T10:50:28.727 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:28 smithi096 ceph-mon[84258]: failed to encode map e56 with expected crc 2024-03-24T10:50:28.727 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:28 smithi096 ceph-mon[84258]: failed to encode map e56 with expected crc 2024-03-24T10:50:29.411 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:29 smithi094 podman[107747]: 2024-03-24 10:50:29.051724886 +0000 UTC m=+0.652534632 container remove 19d5c2185eec614c00ca0f49cf3b6ffa82649e91357fc6f03690ebea86fcad88 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-24T10:50:29.411 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:29 smithi094 systemd[1]: ceph-386110d4-e9ca-11ee-95cd-87774f69a715@osd.1.service: Deactivated successfully. 2024-03-24T10:50:29.411 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:29 smithi094 systemd[1]: Stopped Ceph osd.1 for 386110d4-e9ca-11ee-95cd-87774f69a715. 2024-03-24T10:50:29.411 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:29 smithi094 systemd[1]: ceph-386110d4-e9ca-11ee-95cd-87774f69a715@osd.1.service: Consumed 1min 5.391s CPU time. 2024-03-24T10:50:29.411 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:29 smithi094 systemd[1]: Starting Ceph osd.1 for 386110d4-e9ca-11ee-95cd-87774f69a715... 2024-03-24T10:50:29.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:29 smithi094 ceph-mon[93957]: Health check failed: Degraded data redundancy: 2/219 objects degraded (0.913%), 2 pgs degraded (PG_DEGRADED) 2024-03-24T10:50:29.748 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:29 smithi094 podman[107851]: 2024-03-24 10:50:29.401671629 +0000 UTC m=+0.017623303 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-24T10:50:29.748 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:29 smithi094 podman[107851]: 2024-03-24 10:50:29.519650292 +0000 UTC m=+0.135601923 container create 14f9373e26c254a2d7d3e6dba18722a3b077b9565ed6146d57ba6b824db97a27 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-1-activate, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-24T10:50:29.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:29 smithi096 ceph-mon[84258]: Health check failed: Degraded data redundancy: 2/219 objects degraded (0.913%), 2 pgs degraded (PG_DEGRADED) 2024-03-24T10:50:30.230 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:29 smithi094 podman[107851]: 2024-03-24 10:50:29.789011121 +0000 UTC m=+0.404962757 container init 14f9373e26c254a2d7d3e6dba18722a3b077b9565ed6146d57ba6b824db97a27 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-1-activate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-24T10:50:30.230 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:29 smithi094 podman[107851]: 2024-03-24 10:50:29.792199594 +0000 UTC m=+0.408151236 container start 14f9373e26c254a2d7d3e6dba18722a3b077b9565ed6146d57ba6b824db97a27 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-1-activate, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-24T10:50:30.231 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:29 smithi094 podman[107851]: 2024-03-24 10:50:29.847076586 +0000 UTC m=+0.463028240 container attach 14f9373e26c254a2d7d3e6dba18722a3b077b9565ed6146d57ba6b824db97a27 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-1-activate, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T10:50:30.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:30 smithi094 ceph-mon[93957]: pgmap v67: 65 pgs: 2 active+undersized+degraded, 9 stale+active+clean, 3 active+undersized, 51 active+clean; 196 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 2/219 objects degraded (0.913%) 2024-03-24T10:50:30.498 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:30 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-1-activate[107866]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-24T10:50:30.498 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:30 smithi094 bash[107851]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-24T10:50:30.498 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:30 smithi094 bash[107851]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-24T10:50:30.498 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:30 smithi094 bash[107851]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-24T10:50:30.498 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:30 smithi094 bash[107851]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-24T10:50:30.498 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:30 smithi094 bash[107851]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-03-24T10:50:30.498 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:30 smithi094 bash[107851]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-24T10:50:30.498 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:30 smithi094 bash[107851]: --> ceph-volume raw activate successful for osd ID: 1 2024-03-24T10:50:30.498 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:30 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-1-activate[107866]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-24T10:50:30.498 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:30 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-1-activate[107866]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-24T10:50:30.499 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:30 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-1-activate[107866]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-24T10:50:30.499 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:30 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-1-activate[107866]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-03-24T10:50:30.499 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:30 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-1-activate[107866]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-24T10:50:30.499 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:30 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-1-activate[107866]: --> ceph-volume raw activate successful for osd ID: 1 2024-03-24T10:50:30.499 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:30 smithi094 podman[107851]: 2024-03-24 10:50:30.255422747 +0000 UTC m=+0.871374381 container died 14f9373e26c254a2d7d3e6dba18722a3b077b9565ed6146d57ba6b824db97a27 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-1-activate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True, io.buildah.version=1.33.5) 2024-03-24T10:50:30.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:30 smithi096 ceph-mon[84258]: pgmap v67: 65 pgs: 2 active+undersized+degraded, 9 stale+active+clean, 3 active+undersized, 51 active+clean; 196 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 2/219 objects degraded (0.913%) 2024-03-24T10:50:30.997 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:30 smithi094 podman[108005]: 2024-03-24 10:50:30.8340578 +0000 UTC m=+0.573417623 container remove 14f9373e26c254a2d7d3e6dba18722a3b077b9565ed6146d57ba6b824db97a27 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-1-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-24T10:50:31.293 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:31 smithi094 podman[108070]: 2024-03-24 10:50:31.043341379 +0000 UTC m=+0.110622328 container create c5b8b2bbf8b54b6935ced95e2a31dc82592fb302a31c78dc0ed63026cc367a80 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2) 2024-03-24T10:50:31.293 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:31 smithi094 podman[108070]: 2024-03-24 10:50:30.950881023 +0000 UTC m=+0.018161943 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-24T10:50:31.293 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:31 smithi094 podman[108070]: 2024-03-24 10:50:31.270531395 +0000 UTC m=+0.337812321 container init c5b8b2bbf8b54b6935ced95e2a31dc82592fb302a31c78dc0ed63026cc367a80 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-1, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319) 2024-03-24T10:50:31.293 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:31 smithi094 podman[108070]: 2024-03-24 10:50:31.273756538 +0000 UTC m=+0.341037457 container start c5b8b2bbf8b54b6935ced95e2a31dc82592fb302a31c78dc0ed63026cc367a80 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-1, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319) 2024-03-24T10:50:31.293 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:31 smithi094 ceph-osd[108089]: -- 172.21.15.94:0/905170067 <== mon.0 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5617affd91e0 con 0x5617affab000 2024-03-24T10:50:31.747 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:31 smithi094 bash[108070]: c5b8b2bbf8b54b6935ced95e2a31dc82592fb302a31c78dc0ed63026cc367a80 2024-03-24T10:50:31.747 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:31 smithi094 systemd[1]: Started Ceph osd.1 for 386110d4-e9ca-11ee-95cd-87774f69a715. 2024-03-24T10:50:32.199 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:31 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-1[108085]: 2024-03-24T10:50:31.826+0000 7f407ae8a7c0 -1 Falling back to public interface 2024-03-24T10:50:32.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:32 smithi094 ceph-mon[93957]: pgmap v68: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 196 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 31/219 objects degraded (14.155%) 2024-03-24T10:50:32.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:32 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:50:32.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:32 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:50:32.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:32 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:50:32.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:32 smithi096 ceph-mon[84258]: pgmap v68: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 196 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 31/219 objects degraded (14.155%) 2024-03-24T10:50:32.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:32 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:50:32.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:32 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:50:32.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:32 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:50:34.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:34 smithi094 ceph-mon[93957]: pgmap v69: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 196 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 31/219 objects degraded (14.155%) 2024-03-24T10:50:34.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:34 smithi096 ceph-mon[84258]: pgmap v69: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 196 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 31/219 objects degraded (14.155%) 2024-03-24T10:50:34.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:34.840+0000 7fa11d383700 1 -- 172.21.15.94:0/1510380052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1180fee10 msgr2=0x7fa1180faf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:34.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:34.840+0000 7fa11d383700 1 --2- 172.21.15.94:0/1510380052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1180fee10 0x7fa1180faf50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa10c004940 tx=0x7fa10c009e30 comp rx=0 tx=0).stop 2024-03-24T10:50:34.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:34.840+0000 7fa11d383700 1 -- 172.21.15.94:0/1510380052 shutdown_connections 2024-03-24T10:50:34.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:34.840+0000 7fa11d383700 1 --2- 172.21.15.94:0/1510380052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1180fee10 0x7fa1180faf50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:34.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:34.841+0000 7fa11d383700 1 --2- 172.21.15.94:0/1510380052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1180fe4c0 0x7fa1180fe8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:34.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:34.841+0000 7fa11d383700 1 -- 172.21.15.94:0/1510380052 >> 172.21.15.94:0/1510380052 conn(0x7fa1180f6ba0 msgr2=0x7fa1180f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:50:34.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:34.841+0000 7fa11d383700 1 -- 172.21.15.94:0/1510380052 shutdown_connections 2024-03-24T10:50:34.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:34.841+0000 7fa11d383700 1 -- 172.21.15.94:0/1510380052 wait complete. 2024-03-24T10:50:34.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:34.841+0000 7fa11d383700 1 Processor -- start 2024-03-24T10:50:34.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:34.842+0000 7fa11d383700 1 -- start start 2024-03-24T10:50:34.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:34.842+0000 7fa11d383700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1180fe4c0 0x7fa11810fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:50:34.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:34.842+0000 7fa11d383700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1180fee10 0x7fa118110180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:50:34.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:34.842+0000 7fa11d383700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa118110780 con 0x7fa1180fee10 2024-03-24T10:50:34.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:34.842+0000 7fa11d383700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa118110fd0 con 0x7fa1180fe4c0 2024-03-24T10:50:34.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:34.842+0000 7fa1167fc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1180fee10 0x7fa118110180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:50:34.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:34.842+0000 7fa1167fc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1180fee10 0x7fa118110180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42126/0 (socket says 172.21.15.94:42126) 2024-03-24T10:50:34.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:34.842+0000 7fa1167fc700 1 -- 172.21.15.94:0/2780516969 learned_addr learned my addr 172.21.15.94:0/2780516969 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:50:34.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:34.842+0000 7fa116ffd700 1 --2- 172.21.15.94:0/2780516969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1180fe4c0 0x7fa11810fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:50:34.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:34.842+0000 7fa1167fc700 1 -- 172.21.15.94:0/2780516969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1180fe4c0 msgr2=0x7fa11810fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:34.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:34.842+0000 7fa1167fc700 1 --2- 172.21.15.94:0/2780516969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1180fe4c0 0x7fa11810fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:34.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:34.842+0000 7fa1167fc700 1 -- 172.21.15.94:0/2780516969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa10c005040 con 0x7fa1180fee10 2024-03-24T10:50:34.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:34.842+0000 7fa116ffd700 1 --2- 172.21.15.94:0/2780516969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1180fe4c0 0x7fa11810fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T10:50:34.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:34.842+0000 7fa1167fc700 1 --2- 172.21.15.94:0/2780516969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1180fee10 0x7fa118110180 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa10c00b3b0 tx=0x7fa10c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:50:34.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:34.843+0000 7fa0ff7fe700 1 -- 172.21.15.94:0/2780516969 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa10c00e040 con 0x7fa1180fee10 2024-03-24T10:50:34.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:34.843+0000 7fa11d383700 1 -- 172.21.15.94:0/2780516969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1181111d0 con 0x7fa1180fee10 2024-03-24T10:50:34.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:34.843+0000 7fa0ff7fe700 1 -- 172.21.15.94:0/2780516969 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa10c004290 con 0x7fa1180fee10 2024-03-24T10:50:34.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:34.843+0000 7fa0ff7fe700 1 -- 172.21.15.94:0/2780516969 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa10c01c630 con 0x7fa1180fee10 2024-03-24T10:50:34.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:34.843+0000 7fa11d383700 1 -- 172.21.15.94:0/2780516969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa118111790 con 0x7fa1180fee10 2024-03-24T10:50:34.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:34.844+0000 7fa0ff7fe700 1 -- 172.21.15.94:0/2780516969 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fa10c00bbf0 con 0x7fa1180fee10 2024-03-24T10:50:34.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:34.845+0000 7fa0ff7fe700 1 --2- 172.21.15.94:0/2780516969 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fa10006d8c0 0x7fa10006fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:50:34.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:34.845+0000 7fa0ff7fe700 1 -- 172.21.15.94:0/2780516969 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fa10c08e330 con 0x7fa1180fee10 2024-03-24T10:50:34.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:34.845+0000 7fa11d383700 1 -- 172.21.15.94:0/2780516969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa118101c00 con 0x7fa1180fee10 2024-03-24T10:50:34.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:34.845+0000 7fa116ffd700 1 --2- 172.21.15.94:0/2780516969 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fa10006d8c0 0x7fa10006fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:50:34.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:34.846+0000 7fa116ffd700 1 --2- 172.21.15.94:0/2780516969 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fa10006d8c0 0x7fa10006fd70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa108009fd0 tx=0x7fa108009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:50:34.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:34.851+0000 7fa0ff7fe700 1 -- 172.21.15.94:0/2780516969 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa10c05b6b0 con 0x7fa1180fee10 2024-03-24T10:50:35.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.105+0000 7fa11d383700 1 -- 172.21.15.94:0/2780516969 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa118111a00 con 0x7fa10006d8c0 2024-03-24T10:50:35.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.106+0000 7fa0ff7fe700 1 -- 172.21.15.94:0/2780516969 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fa118111a00 con 0x7fa10006d8c0 2024-03-24T10:50:35.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.109+0000 7fa11d383700 1 -- 172.21.15.94:0/2780516969 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fa10006d8c0 msgr2=0x7fa10006fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:35.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.109+0000 7fa11d383700 1 --2- 172.21.15.94:0/2780516969 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fa10006d8c0 0x7fa10006fd70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa108009fd0 tx=0x7fa108009ee0 comp rx=0 tx=0).stop 2024-03-24T10:50:35.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.109+0000 7fa11d383700 1 -- 172.21.15.94:0/2780516969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1180fee10 msgr2=0x7fa118110180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:35.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.109+0000 7fa11d383700 1 --2- 172.21.15.94:0/2780516969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1180fee10 0x7fa118110180 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa10c00b3b0 tx=0x7fa10c009e30 comp rx=0 tx=0).stop 2024-03-24T10:50:35.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.109+0000 7fa11d383700 1 -- 172.21.15.94:0/2780516969 shutdown_connections 2024-03-24T10:50:35.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.109+0000 7fa11d383700 1 --2- 172.21.15.94:0/2780516969 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fa10006d8c0 0x7fa10006fd70 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:35.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.109+0000 7fa11d383700 1 --2- 172.21.15.94:0/2780516969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1180fe4c0 0x7fa11810fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:35.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.109+0000 7fa11d383700 1 --2- 172.21.15.94:0/2780516969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1180fee10 0x7fa118110180 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:35.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.109+0000 7fa11d383700 1 -- 172.21.15.94:0/2780516969 >> 172.21.15.94:0/2780516969 conn(0x7fa1180f6ba0 msgr2=0x7fa1180f81d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:50:35.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.109+0000 7fa11d383700 1 -- 172.21.15.94:0/2780516969 shutdown_connections 2024-03-24T10:50:35.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.109+0000 7fa11d383700 1 -- 172.21.15.94:0/2780516969 wait complete. 2024-03-24T10:50:35.121 INFO:teuthology.orchestra.run.smithi094.stdout:true 2024-03-24T10:50:35.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.219+0000 7fbe2100e700 1 -- 172.21.15.94:0/2675614751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe1c071e40 msgr2=0x7fbe1c070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:35.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.219+0000 7fbe2100e700 1 --2- 172.21.15.94:0/2675614751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe1c071e40 0x7fbe1c070350 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c004940 tx=0x7fbe0c009e30 comp rx=0 tx=0).stop 2024-03-24T10:50:35.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.220+0000 7fbe2100e700 1 -- 172.21.15.94:0/2675614751 shutdown_connections 2024-03-24T10:50:35.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.220+0000 7fbe2100e700 1 --2- 172.21.15.94:0/2675614751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe1c070890 0x7fbe1c070d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:35.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.220+0000 7fbe2100e700 1 --2- 172.21.15.94:0/2675614751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe1c071e40 0x7fbe1c070350 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:35.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.220+0000 7fbe2100e700 1 -- 172.21.15.94:0/2675614751 >> 172.21.15.94:0/2675614751 conn(0x7fbe1c0f8560 msgr2=0x7fbe1c0fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:50:35.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.220+0000 7fbe2100e700 1 -- 172.21.15.94:0/2675614751 shutdown_connections 2024-03-24T10:50:35.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.220+0000 7fbe2100e700 1 -- 172.21.15.94:0/2675614751 wait complete. 2024-03-24T10:50:35.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.220+0000 7fbe2100e700 1 Processor -- start 2024-03-24T10:50:35.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.220+0000 7fbe2100e700 1 -- start start 2024-03-24T10:50:35.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.220+0000 7fbe2100e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe1c070890 0x7fbe1c10fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:50:35.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.220+0000 7fbe2100e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe1c071e40 0x7fbe1c110260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:50:35.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.220+0000 7fbe2100e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1c110860 con 0x7fbe1c071e40 2024-03-24T10:50:35.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.220+0000 7fbe2100e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1c110ff0 con 0x7fbe1c070890 2024-03-24T10:50:35.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.221+0000 7fbe1a59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe1c071e40 0x7fbe1c110260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:50:35.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.221+0000 7fbe1a59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe1c071e40 0x7fbe1c110260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42150/0 (socket says 172.21.15.94:42150) 2024-03-24T10:50:35.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.221+0000 7fbe1a59c700 1 -- 172.21.15.94:0/1476352182 learned_addr learned my addr 172.21.15.94:0/1476352182 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:50:35.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.221+0000 7fbe1ad9d700 1 --2- 172.21.15.94:0/1476352182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe1c070890 0x7fbe1c10fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:50:35.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.221+0000 7fbe1a59c700 1 -- 172.21.15.94:0/1476352182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe1c070890 msgr2=0x7fbe1c10fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:35.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.221+0000 7fbe1a59c700 1 --2- 172.21.15.94:0/1476352182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe1c070890 0x7fbe1c10fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:35.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.221+0000 7fbe1a59c700 1 -- 172.21.15.94:0/1476352182 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe0c005040 con 0x7fbe1c071e40 2024-03-24T10:50:35.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.221+0000 7fbe1a59c700 1 --2- 172.21.15.94:0/1476352182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe1c071e40 0x7fbe1c110260 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbe1000c8c0 tx=0x7fbe100044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:50:35.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.221+0000 7fbe037fe700 1 -- 172.21.15.94:0/1476352182 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe1000a980 con 0x7fbe1c071e40 2024-03-24T10:50:35.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.221+0000 7fbe037fe700 1 -- 172.21.15.94:0/1476352182 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe1000ce60 con 0x7fbe1c071e40 2024-03-24T10:50:35.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.221+0000 7fbe2100e700 1 -- 172.21.15.94:0/1476352182 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe1c1112d0 con 0x7fbe1c071e40 2024-03-24T10:50:35.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.221+0000 7fbe037fe700 1 -- 172.21.15.94:0/1476352182 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe100115e0 con 0x7fbe1c071e40 2024-03-24T10:50:35.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.221+0000 7fbe2100e700 1 -- 172.21.15.94:0/1476352182 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe1c111810 con 0x7fbe1c071e40 2024-03-24T10:50:35.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.222+0000 7fbe2100e700 1 -- 172.21.15.94:0/1476352182 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe1c05f5f0 con 0x7fbe1c071e40 2024-03-24T10:50:35.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.223+0000 7fbe037fe700 1 -- 172.21.15.94:0/1476352182 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fbe1000aae0 con 0x7fbe1c071e40 2024-03-24T10:50:35.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.223+0000 7fbe037fe700 1 --2- 172.21.15.94:0/1476352182 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fbe0406d7e0 0x7fbe0406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:50:35.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.223+0000 7fbe1ad9d700 1 --2- 172.21.15.94:0/1476352182 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fbe0406d7e0 0x7fbe0406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:50:35.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.223+0000 7fbe037fe700 1 -- 172.21.15.94:0/1476352182 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fbe1008cfb0 con 0x7fbe1c071e40 2024-03-24T10:50:35.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.223+0000 7fbe1ad9d700 1 --2- 172.21.15.94:0/1476352182 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fbe0406d7e0 0x7fbe0406fc90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c00b970 tx=0x7fbe0c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:50:35.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.227+0000 7fbe037fe700 1 -- 172.21.15.94:0/1476352182 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fbe1005a350 con 0x7fbe1c071e40 2024-03-24T10:50:35.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.457+0000 7fbe2100e700 1 -- 172.21.15.94:0/1476352182 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe1c111b30 con 0x7fbe0406d7e0 2024-03-24T10:50:35.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.458+0000 7fbe037fe700 1 -- 172.21.15.94:0/1476352182 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fbe1c111b30 con 0x7fbe0406d7e0 2024-03-24T10:50:35.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.460+0000 7fbe2100e700 1 -- 172.21.15.94:0/1476352182 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fbe0406d7e0 msgr2=0x7fbe0406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:35.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.460+0000 7fbe2100e700 1 --2- 172.21.15.94:0/1476352182 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fbe0406d7e0 0x7fbe0406fc90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c00b970 tx=0x7fbe0c009e30 comp rx=0 tx=0).stop 2024-03-24T10:50:35.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.460+0000 7fbe2100e700 1 -- 172.21.15.94:0/1476352182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe1c071e40 msgr2=0x7fbe1c110260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:35.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.460+0000 7fbe2100e700 1 --2- 172.21.15.94:0/1476352182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe1c071e40 0x7fbe1c110260 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbe1000c8c0 tx=0x7fbe100044c0 comp rx=0 tx=0).stop 2024-03-24T10:50:35.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.460+0000 7fbe2100e700 1 -- 172.21.15.94:0/1476352182 shutdown_connections 2024-03-24T10:50:35.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.460+0000 7fbe2100e700 1 --2- 172.21.15.94:0/1476352182 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fbe0406d7e0 0x7fbe0406fc90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:35.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.460+0000 7fbe2100e700 1 --2- 172.21.15.94:0/1476352182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe1c070890 0x7fbe1c10fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:35.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.460+0000 7fbe2100e700 1 --2- 172.21.15.94:0/1476352182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe1c071e40 0x7fbe1c110260 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:35.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.460+0000 7fbe2100e700 1 -- 172.21.15.94:0/1476352182 >> 172.21.15.94:0/1476352182 conn(0x7fbe1c0f8560 msgr2=0x7fbe1c0f9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:50:35.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.460+0000 7fbe2100e700 1 -- 172.21.15.94:0/1476352182 shutdown_connections 2024-03-24T10:50:35.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.460+0000 7fbe2100e700 1 -- 172.21.15.94:0/1476352182 wait complete. 2024-03-24T10:50:35.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.568+0000 7f775b850700 1 -- 172.21.15.94:0/880364036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77540fbd00 msgr2=0x7f77540fc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:35.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.568+0000 7f775b850700 1 --2- 172.21.15.94:0/880364036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77540fbd00 0x7f77540fc110 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7750004940 tx=0x7f7750009e30 comp rx=0 tx=0).stop 2024-03-24T10:50:35.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.568+0000 7f775b850700 1 -- 172.21.15.94:0/880364036 shutdown_connections 2024-03-24T10:50:35.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.568+0000 7f775b850700 1 --2- 172.21.15.94:0/880364036 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77540fcfb0 0x7f77540fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:35.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.568+0000 7f775b850700 1 --2- 172.21.15.94:0/880364036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77540fbd00 0x7f77540fc110 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:35.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.568+0000 7f775b850700 1 -- 172.21.15.94:0/880364036 >> 172.21.15.94:0/880364036 conn(0x7f77540f7480 msgr2=0x7f77540f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:50:35.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.568+0000 7f775b850700 1 -- 172.21.15.94:0/880364036 shutdown_connections 2024-03-24T10:50:35.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.568+0000 7f775b850700 1 -- 172.21.15.94:0/880364036 wait complete. 2024-03-24T10:50:35.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.569+0000 7f775b850700 1 Processor -- start 2024-03-24T10:50:35.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.569+0000 7f775b850700 1 -- start start 2024-03-24T10:50:35.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.569+0000 7f775b850700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77540fbd00 0x7f7754066d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:50:35.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.569+0000 7f775b850700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77540fcfb0 0x7f77541126f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:50:35.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.569+0000 7f775b850700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7754112c30 con 0x7f77540fbd00 2024-03-24T10:50:35.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.569+0000 7f775b850700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7754112d70 con 0x7f77540fcfb0 2024-03-24T10:50:35.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.569+0000 7f77595ec700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77540fbd00 0x7f7754066d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:50:35.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.569+0000 7f77595ec700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77540fbd00 0x7f7754066d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42174/0 (socket says 172.21.15.94:42174) 2024-03-24T10:50:35.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.569+0000 7f77595ec700 1 -- 172.21.15.94:0/1153344538 learned_addr learned my addr 172.21.15.94:0/1153344538 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:50:35.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.570+0000 7f7758deb700 1 --2- 172.21.15.94:0/1153344538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77540fcfb0 0x7f77541126f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:50:35.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.570+0000 7f77595ec700 1 -- 172.21.15.94:0/1153344538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77540fcfb0 msgr2=0x7f77541126f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:35.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.570+0000 7f77595ec700 1 --2- 172.21.15.94:0/1153344538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77540fcfb0 0x7f77541126f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:35.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.570+0000 7f77595ec700 1 -- 172.21.15.94:0/1153344538 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7750005040 con 0x7f77540fbd00 2024-03-24T10:50:35.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.570+0000 7f77595ec700 1 --2- 172.21.15.94:0/1153344538 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77540fbd00 0x7f7754066d10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7750013870 tx=0x7f7750009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:50:35.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.570+0000 7f7749ffb700 1 -- 172.21.15.94:0/1153344538 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f775000e070 con 0x7f77540fbd00 2024-03-24T10:50:35.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.570+0000 7f775b850700 1 -- 172.21.15.94:0/1153344538 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7754112f70 con 0x7f77540fbd00 2024-03-24T10:50:35.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.570+0000 7f7749ffb700 1 -- 172.21.15.94:0/1153344538 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7750013de0 con 0x7f77540fbd00 2024-03-24T10:50:35.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.570+0000 7f7749ffb700 1 -- 172.21.15.94:0/1153344538 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f775001cea0 con 0x7f77540fbd00 2024-03-24T10:50:35.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.571+0000 7f775b850700 1 -- 172.21.15.94:0/1153344538 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77541133d0 con 0x7f77540fbd00 2024-03-24T10:50:35.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.572+0000 7f775b850700 1 -- 172.21.15.94:0/1153344538 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f775404c710 con 0x7f77540fbd00 2024-03-24T10:50:35.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.572+0000 7f7749ffb700 1 -- 172.21.15.94:0/1153344538 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f775001c690 con 0x7f77540fbd00 2024-03-24T10:50:35.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.573+0000 7f7749ffb700 1 --2- 172.21.15.94:0/1153344538 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f774006d7e0 0x7f774006fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:50:35.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.573+0000 7f7758deb700 1 --2- 172.21.15.94:0/1153344538 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f774006d7e0 0x7f774006fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:50:35.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.573+0000 7f7749ffb700 1 -- 172.21.15.94:0/1153344538 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f775008e510 con 0x7f77540fbd00 2024-03-24T10:50:35.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.574+0000 7f7758deb700 1 --2- 172.21.15.94:0/1153344538 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f774006d7e0 0x7f774006fc90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7744009b80 tx=0x7f774400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:50:35.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.578+0000 7f7749ffb700 1 -- 172.21.15.94:0/1153344538 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f775005b950 con 0x7f77540fbd00 2024-03-24T10:50:35.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.813+0000 7f775b850700 1 -- 172.21.15.94:0/1153344538 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7754113640 con 0x7f774006d7e0 2024-03-24T10:50:35.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.822+0000 7f7749ffb700 1 -- 172.21.15.94:0/1153344538 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f7754113640 con 0x7f774006d7e0 2024-03-24T10:50:35.822 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T10:50:35.822 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.smithi094 smithi094 *:9093,9094 running (3m) 34s ago 12m 16.8M - 0.25.0 c8568f914cd2 5c583af60554 2024-03-24T10:50:35.822 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi094 smithi094 running (65s) 34s ago 13m 6651k - 19.0.0-1855-g21b3bd67 553463a90fd5 2b0897df571d 2024-03-24T10:50:35.823 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi096 smithi096 running (59s) 56s ago 11m 6669k - 19.0.0-1855-g21b3bd67 553463a90fd5 1759a9ff2805 2024-03-24T10:50:35.823 INFO:teuthology.orchestra.run.smithi094.stdout:grafana.smithi094 smithi094 *:3000 running (3m) 34s ago 11m 77.9M - 9.4.7 954c08fa6188 d3d425bf1b7c 2024-03-24T10:50:35.823 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.efiqte smithi094 running (6m) 34s ago 6m 13.4M - 17.2.7-726-g9c4e0c34 837124da804f f0df0a056c21 2024-03-24T10:50:35.823 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.llefgu smithi094 running (6m) 34s ago 6m 109M - 17.2.7-726-g9c4e0c34 837124da804f 52a9771dc8e0 2024-03-24T10:50:35.823 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi096.egbmoq smithi096 running (6m) 56s ago 6m 154M - 17.2.7-726-g9c4e0c34 837124da804f 1cfc930dbab0 2024-03-24T10:50:35.823 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi096.jtebhh smithi096 running (6m) 56s ago 6m 16.3M - 17.2.7-726-g9c4e0c34 837124da804f 4ff3451a6f6e 2024-03-24T10:50:35.823 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi094.yaethu smithi094 *:8443,9283,8765 running (3m) 34s ago 14m 497M - 19.0.0-1855-g21b3bd67 553463a90fd5 d3adab76b18f 2024-03-24T10:50:35.823 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi096.gfonnu smithi096 *:8443,9283,8765 running (2m) 56s ago 11m 448M - 19.0.0-1855-g21b3bd67 553463a90fd5 22937a059eb8 2024-03-24T10:50:35.823 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi094 smithi094 running (2m) 34s ago 14m 45.2M 2048M 19.0.0-1855-g21b3bd67 553463a90fd5 c779d611e29a 2024-03-24T10:50:35.823 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi096 smithi096 running (106s) 56s ago 11m 31.2M 2048M 19.0.0-1855-g21b3bd67 553463a90fd5 ed9bd0bd57f6 2024-03-24T10:50:35.823 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi094 smithi094 *:9100 running (3m) 34s ago 12m 16.4M - 1.5.0 0da6a335fe13 ca67b53c64e2 2024-03-24T10:50:35.823 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi096 smithi096 *:9100 running (3m) 56s ago 11m 17.0M - 1.5.0 0da6a335fe13 65137a7a7549 2024-03-24T10:50:35.823 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (40s) 34s ago 10m 12.6M 1557M 19.0.0-1855-g21b3bd67 553463a90fd5 ec57ede0c9e4 2024-03-24T10:50:35.823 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi094 starting - - - 1557M 2024-03-24T10:50:35.823 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi094 running (9m) 34s ago 9m 221M 1557M 17.2.7-726-g9c4e0c34 837124da804f cf0aefd20c76 2024-03-24T10:50:35.824 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi096 running (9m) 56s ago 9m 285M 2581M 17.2.7-726-g9c4e0c34 837124da804f f4650d0f6241 2024-03-24T10:50:35.824 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi096 running (8m) 56s ago 8m 258M 2581M 17.2.7-726-g9c4e0c34 837124da804f a63d7c9e705f 2024-03-24T10:50:35.824 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi096 running (8m) 56s ago 8m 243M 2581M 17.2.7-726-g9c4e0c34 837124da804f f56fad2959c3 2024-03-24T10:50:35.824 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.smithi094 smithi094 *:9095 running (2m) 34s ago 11m 51.2M - 2.43.0 a07b618ecd1d 6ce091a0c326 2024-03-24T10:50:35.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.825+0000 7f775b850700 1 -- 172.21.15.94:0/1153344538 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f774006d7e0 msgr2=0x7f774006fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:35.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.825+0000 7f775b850700 1 --2- 172.21.15.94:0/1153344538 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f774006d7e0 0x7f774006fc90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7744009b80 tx=0x7f774400b040 comp rx=0 tx=0).stop 2024-03-24T10:50:35.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.825+0000 7f775b850700 1 -- 172.21.15.94:0/1153344538 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77540fbd00 msgr2=0x7f7754066d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:35.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.825+0000 7f775b850700 1 --2- 172.21.15.94:0/1153344538 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77540fbd00 0x7f7754066d10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7750013870 tx=0x7f7750009e30 comp rx=0 tx=0).stop 2024-03-24T10:50:35.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.825+0000 7f775b850700 1 -- 172.21.15.94:0/1153344538 shutdown_connections 2024-03-24T10:50:35.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.825+0000 7f775b850700 1 --2- 172.21.15.94:0/1153344538 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f774006d7e0 0x7f774006fc90 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:35.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.825+0000 7f775b850700 1 --2- 172.21.15.94:0/1153344538 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77540fbd00 0x7f7754066d10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:35.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.825+0000 7f775b850700 1 --2- 172.21.15.94:0/1153344538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77540fcfb0 0x7f77541126f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:35.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.825+0000 7f775b850700 1 -- 172.21.15.94:0/1153344538 >> 172.21.15.94:0/1153344538 conn(0x7f77540f7480 msgr2=0x7f77540f8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:50:35.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.825+0000 7f775b850700 1 -- 172.21.15.94:0/1153344538 shutdown_connections 2024-03-24T10:50:35.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.825+0000 7f775b850700 1 -- 172.21.15.94:0/1153344538 wait complete. 2024-03-24T10:50:35.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.926+0000 7f3475442700 1 -- 172.21.15.94:0/3918615408 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34700fcf80 msgr2=0x7f34700fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:35.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.926+0000 7f3475442700 1 --2- 172.21.15.94:0/3918615408 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34700fcf80 0x7f34700fd3f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3464004940 tx=0x7f3464009e30 comp rx=0 tx=0).stop 2024-03-24T10:50:35.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.926+0000 7f3475442700 1 -- 172.21.15.94:0/3918615408 shutdown_connections 2024-03-24T10:50:35.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.926+0000 7f3475442700 1 --2- 172.21.15.94:0/3918615408 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34700fcf80 0x7f34700fd3f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:35.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.926+0000 7f3475442700 1 --2- 172.21.15.94:0/3918615408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34700fbcd0 0x7f34700fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:35.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.926+0000 7f3475442700 1 -- 172.21.15.94:0/3918615408 >> 172.21.15.94:0/3918615408 conn(0x7f34700f7450 msgr2=0x7f34700f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:50:35.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.926+0000 7f3475442700 1 -- 172.21.15.94:0/3918615408 shutdown_connections 2024-03-24T10:50:35.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.927+0000 7f3475442700 1 -- 172.21.15.94:0/3918615408 wait complete. 2024-03-24T10:50:35.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.927+0000 7f3475442700 1 Processor -- start 2024-03-24T10:50:35.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.927+0000 7f3475442700 1 -- start start 2024-03-24T10:50:35.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.927+0000 7f3475442700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34700fbcd0 0x7f347010c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:50:35.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.927+0000 7f3475442700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34700fcf80 0x7f347010cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:50:35.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.927+0000 7f3475442700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347010d350 con 0x7f34700fbcd0 2024-03-24T10:50:35.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.927+0000 7f3475442700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347010d4c0 con 0x7f34700fcf80 2024-03-24T10:50:35.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.927+0000 7f346effd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34700fbcd0 0x7f347010c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:50:35.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.927+0000 7f346effd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34700fbcd0 0x7f347010c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42204/0 (socket says 172.21.15.94:42204) 2024-03-24T10:50:35.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.927+0000 7f346effd700 1 -- 172.21.15.94:0/3502598092 learned_addr learned my addr 172.21.15.94:0/3502598092 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:50:35.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.928+0000 7f346e7fc700 1 --2- 172.21.15.94:0/3502598092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34700fcf80 0x7f347010cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:50:35.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.928+0000 7f346effd700 1 -- 172.21.15.94:0/3502598092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34700fcf80 msgr2=0x7f347010cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:35.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.928+0000 7f346effd700 1 --2- 172.21.15.94:0/3502598092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34700fcf80 0x7f347010cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:35.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.928+0000 7f346effd700 1 -- 172.21.15.94:0/3502598092 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3464005040 con 0x7f34700fbcd0 2024-03-24T10:50:35.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.928+0000 7f346effd700 1 --2- 172.21.15.94:0/3502598092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34700fbcd0 0x7f347010c810 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f346000fce0 tx=0x7f34600046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:50:35.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.928+0000 7f34577fe700 1 -- 172.21.15.94:0/3502598092 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f346000fe90 con 0x7f34700fbcd0 2024-03-24T10:50:35.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.928+0000 7f34577fe700 1 -- 172.21.15.94:0/3502598092 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f34600090d0 con 0x7f34700fbcd0 2024-03-24T10:50:35.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.928+0000 7f3475442700 1 -- 172.21.15.94:0/3502598092 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34700662d0 con 0x7f34700fbcd0 2024-03-24T10:50:35.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.928+0000 7f34577fe700 1 -- 172.21.15.94:0/3502598092 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34600126a0 con 0x7f34700fbcd0 2024-03-24T10:50:35.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.928+0000 7f3475442700 1 -- 172.21.15.94:0/3502598092 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3470066890 con 0x7f34700fbcd0 2024-03-24T10:50:35.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.929+0000 7f3475442700 1 -- 172.21.15.94:0/3502598092 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f347004c710 con 0x7f34700fbcd0 2024-03-24T10:50:35.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.930+0000 7f34577fe700 1 -- 172.21.15.94:0/3502598092 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f3460009240 con 0x7f34700fbcd0 2024-03-24T10:50:35.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.930+0000 7f34577fe700 1 --2- 172.21.15.94:0/3502598092 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f345806d7e0 0x7f345806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:50:35.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.930+0000 7f34577fe700 1 -- 172.21.15.94:0/3502598092 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f346008f6f0 con 0x7f34700fbcd0 2024-03-24T10:50:35.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.930+0000 7f346e7fc700 1 --2- 172.21.15.94:0/3502598092 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f345806d7e0 0x7f345806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:50:35.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.930+0000 7f346e7fc700 1 --2- 172.21.15.94:0/3502598092 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f345806d7e0 0x7f345806fc90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f346400b970 tx=0x7f3464009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:50:35.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:35.935+0000 7f34577fe700 1 -- 172.21.15.94:0/3502598092 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f346005ca10 con 0x7f34700fbcd0 2024-03-24T10:50:36.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.234+0000 7f3475442700 1 -- 172.21.15.94:0/3502598092 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3470066cc0 con 0x7f34700fbcd0 2024-03-24T10:50:36.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.234+0000 7f34577fe700 1 -- 172.21.15.94:0/3502598092 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f346005c5a0 con 0x7f34700fbcd0 2024-03-24T10:50:36.234 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-03-24T10:50:36.234 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": { 2024-03-24T10:50:36.234 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 2 2024-03-24T10:50:36.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:50:36.234 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": { 2024-03-24T10:50:36.234 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 2 2024-03-24T10:50:36.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:50:36.234 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": { 2024-03-24T10:50:36.235 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4, 2024-03-24T10:50:36.235 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 1 2024-03-24T10:50:36.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:50:36.235 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-03-24T10:50:36.235 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-24T10:50:36.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:50:36.235 INFO:teuthology.orchestra.run.smithi094.stdout: "overall": { 2024-03-24T10:50:36.235 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 8, 2024-03-24T10:50:36.235 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 5 2024-03-24T10:50:36.235 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-03-24T10:50:36.235 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-03-24T10:50:36.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.236+0000 7f3475442700 1 -- 172.21.15.94:0/3502598092 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f345806d7e0 msgr2=0x7f345806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:36.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.236+0000 7f3475442700 1 --2- 172.21.15.94:0/3502598092 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f345806d7e0 0x7f345806fc90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f346400b970 tx=0x7f3464009e30 comp rx=0 tx=0).stop 2024-03-24T10:50:36.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.236+0000 7f3475442700 1 -- 172.21.15.94:0/3502598092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34700fbcd0 msgr2=0x7f347010c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:36.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.236+0000 7f3475442700 1 --2- 172.21.15.94:0/3502598092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34700fbcd0 0x7f347010c810 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f346000fce0 tx=0x7f34600046b0 comp rx=0 tx=0).stop 2024-03-24T10:50:36.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.236+0000 7f3475442700 1 -- 172.21.15.94:0/3502598092 shutdown_connections 2024-03-24T10:50:36.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.236+0000 7f3475442700 1 --2- 172.21.15.94:0/3502598092 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f345806d7e0 0x7f345806fc90 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:36.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.236+0000 7f3475442700 1 --2- 172.21.15.94:0/3502598092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34700fbcd0 0x7f347010c810 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:36.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.236+0000 7f3475442700 1 --2- 172.21.15.94:0/3502598092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34700fcf80 0x7f347010cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:36.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.236+0000 7f3475442700 1 -- 172.21.15.94:0/3502598092 >> 172.21.15.94:0/3502598092 conn(0x7f34700f7450 msgr2=0x7f34700f8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:50:36.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.237+0000 7f3475442700 1 -- 172.21.15.94:0/3502598092 shutdown_connections 2024-03-24T10:50:36.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.237+0000 7f3475442700 1 -- 172.21.15.94:0/3502598092 wait complete. 2024-03-24T10:50:36.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.343+0000 7f8304da2700 1 -- 172.21.15.94:0/518786985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83000fe240 msgr2=0x7f83000fe650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:36.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.343+0000 7f8304da2700 1 --2- 172.21.15.94:0/518786985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83000fe240 0x7f83000fe650 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f82f0004940 tx=0x7f82f0009e30 comp rx=0 tx=0).stop 2024-03-24T10:50:36.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.343+0000 7f8304da2700 1 -- 172.21.15.94:0/518786985 shutdown_connections 2024-03-24T10:50:36.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.343+0000 7f8304da2700 1 --2- 172.21.15.94:0/518786985 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83000feb90 0x7f83000facd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:36.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.343+0000 7f8304da2700 1 --2- 172.21.15.94:0/518786985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83000fe240 0x7f83000fe650 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:36.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.343+0000 7f8304da2700 1 -- 172.21.15.94:0/518786985 >> 172.21.15.94:0/518786985 conn(0x7f83000f6a00 msgr2=0x7f83000f8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:50:36.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.343+0000 7f8304da2700 1 -- 172.21.15.94:0/518786985 shutdown_connections 2024-03-24T10:50:36.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.343+0000 7f8304da2700 1 -- 172.21.15.94:0/518786985 wait complete. 2024-03-24T10:50:36.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.344+0000 7f8304da2700 1 Processor -- start 2024-03-24T10:50:36.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.344+0000 7f8304da2700 1 -- start start 2024-03-24T10:50:36.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.344+0000 7f8304da2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83000fe240 0x7f83000fda30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:50:36.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.344+0000 7f8304da2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83000feb90 0x7f83000fbf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:50:36.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.344+0000 7f8304da2700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83000fc4b0 con 0x7f83000fe240 2024-03-24T10:50:36.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.344+0000 7f8304da2700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83000fc620 con 0x7f83000feb90 2024-03-24T10:50:36.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.344+0000 7f82ff7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83000fe240 0x7f83000fda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:50:36.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.344+0000 7f82ff7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83000fe240 0x7f83000fda30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42226/0 (socket says 172.21.15.94:42226) 2024-03-24T10:50:36.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.344+0000 7f82ff7fe700 1 -- 172.21.15.94:0/3617863356 learned_addr learned my addr 172.21.15.94:0/3617863356 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:50:36.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.345+0000 7f82ff7fe700 1 -- 172.21.15.94:0/3617863356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83000feb90 msgr2=0x7f83000fbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:36.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.345+0000 7f82ff7fe700 1 --2- 172.21.15.94:0/3617863356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83000feb90 0x7f83000fbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:36.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.345+0000 7f82ff7fe700 1 -- 172.21.15.94:0/3617863356 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f0005040 con 0x7f83000fe240 2024-03-24T10:50:36.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.345+0000 7f82ff7fe700 1 --2- 172.21.15.94:0/3617863356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83000fe240 0x7f83000fda30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f82f000b3b0 tx=0x7f82f0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:50:36.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.345+0000 7f82e7fff700 1 -- 172.21.15.94:0/3617863356 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82f000e040 con 0x7f83000fe240 2024-03-24T10:50:36.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.345+0000 7f82e7fff700 1 -- 172.21.15.94:0/3617863356 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f82f0004290 con 0x7f83000fe240 2024-03-24T10:50:36.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.345+0000 7f8304da2700 1 -- 172.21.15.94:0/3617863356 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83000fc8a0 con 0x7f83000fe240 2024-03-24T10:50:36.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.345+0000 7f82e7fff700 1 -- 172.21.15.94:0/3617863356 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82f001c630 con 0x7f83000fe240 2024-03-24T10:50:36.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.345+0000 7f8304da2700 1 -- 172.21.15.94:0/3617863356 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8300115890 con 0x7f83000fe240 2024-03-24T10:50:36.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.346+0000 7f8304da2700 1 -- 172.21.15.94:0/3617863356 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f830005f5f0 con 0x7f83000fe240 2024-03-24T10:50:36.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.347+0000 7f82e7fff700 1 -- 172.21.15.94:0/3617863356 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f82f0013420 con 0x7f83000fe240 2024-03-24T10:50:36.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.347+0000 7f82e7fff700 1 --2- 172.21.15.94:0/3617863356 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f82e806d7f0 0x7f82e806fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:50:36.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.347+0000 7f82e7fff700 1 -- 172.21.15.94:0/3617863356 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f82f008e390 con 0x7f83000fe240 2024-03-24T10:50:36.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.347+0000 7f82feffd700 1 --2- 172.21.15.94:0/3617863356 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f82e806d7f0 0x7f82e806fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:50:36.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.348+0000 7f82feffd700 1 --2- 172.21.15.94:0/3617863356 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f82e806d7f0 0x7f82e806fca0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f82f4009fd0 tx=0x7f82f4009e10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:50:36.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.352+0000 7f82e7fff700 1 -- 172.21.15.94:0/3617863356 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f82f0057c00 con 0x7f83000fe240 2024-03-24T10:50:36.740 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:36 smithi094 ceph-mon[93957]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:50:36.740 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:36 smithi094 ceph-mon[93957]: pgmap v70: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 196 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 31/219 objects degraded (14.155%) 2024-03-24T10:50:36.740 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:36 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:50:36.740 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:36 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:50:36.740 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:36 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/3502598092' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:50:36.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.739+0000 7f8304da2700 1 -- 172.21.15.94:0/3617863356 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8300115b50 con 0x7f83000fe240 2024-03-24T10:50:36.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.740+0000 7f82e7fff700 1 -- 172.21.15.94:0/3617863356 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1903 (secure 0 0 0) 0x7f82f005b2c0 con 0x7f83000fe240 2024-03-24T10:50:36.741 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:36 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-1[108085]: 2024-03-24T10:50:36.405+0000 7f407ae8a7c0 -1 osd.1 0 read_superblock omap replica is missing. 2024-03-24T10:50:36.741 INFO:teuthology.orchestra.run.smithi094.stdout:e18 2024-03-24T10:50:36.741 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T10:50:36.741 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:50:36.741 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-03-24T10:50:36.741 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:50:36.741 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-03-24T10:50:36.741 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-03-24T10:50:36.742 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 18 2024-03-24T10:50:36.742 INFO:teuthology.orchestra.run.smithi094.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-24T10:50:36.742 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-03-24T10:43:39.435738+0000 2024-03-24T10:50:36.742 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-03-24T10:43:58.133901+0000 2024-03-24T10:50:36.742 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-03-24T10:50:36.742 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-03-24T10:50:36.742 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 60 2024-03-24T10:50:36.742 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-03-24T10:50:36.742 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-03-24T10:50:36.742 INFO:teuthology.orchestra.run.smithi094.stdout:max_xattr_size 65536 2024-03-24T10:50:36.742 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-03-24T10:50:36.742 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-03-24T10:50:36.742 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 46 2024-03-24T10:50:36.742 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:50:36.743 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-03-24T10:50:36.743 INFO:teuthology.orchestra.run.smithi094.stdout:in 0 2024-03-24T10:50:36.743 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=24251} 2024-03-24T10:50:36.743 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-03-24T10:50:36.743 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-03-24T10:50:36.743 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-03-24T10:50:36.743 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-03-24T10:50:36.743 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-03-24T10:50:36.743 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-03-24T10:50:36.743 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-03-24T10:50:36.743 INFO:teuthology.orchestra.run.smithi094.stdout:bal_rank_mask -1 2024-03-24T10:50:36.743 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 1 2024-03-24T10:50:36.743 INFO:teuthology.orchestra.run.smithi094.stdout:qdb_cluster leader: 0 members: 2024-03-24T10:50:36.743 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi096.egbmoq{0:24251} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.96:6824/1539752208,v1:172.21.15.96:6825/1539752208] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:50:36.743 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.llefgu{0:14500} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.94:6828/126762980,v1:172.21.15.94:6829/126762980] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:50:36.744 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:50:36.744 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:50:36.744 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-03-24T10:50:36.744 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:50:36.744 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.efiqte{-1:14512} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.94:6826/1636528860,v1:172.21.15.94:6827/1636528860] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:50:36.744 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi096.jtebhh{-1:24261} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.96:6826/1321087930,v1:172.21.15.96:6827/1321087930] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:50:36.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.743+0000 7f8304da2700 1 -- 172.21.15.94:0/3617863356 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f82e806d7f0 msgr2=0x7f82e806fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:36.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.743+0000 7f8304da2700 1 --2- 172.21.15.94:0/3617863356 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f82e806d7f0 0x7f82e806fca0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f82f4009fd0 tx=0x7f82f4009e10 comp rx=0 tx=0).stop 2024-03-24T10:50:36.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.743+0000 7f8304da2700 1 -- 172.21.15.94:0/3617863356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83000fe240 msgr2=0x7f83000fda30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:36.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.743+0000 7f8304da2700 1 --2- 172.21.15.94:0/3617863356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83000fe240 0x7f83000fda30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f82f000b3b0 tx=0x7f82f0009e30 comp rx=0 tx=0).stop 2024-03-24T10:50:36.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.743+0000 7f8304da2700 1 -- 172.21.15.94:0/3617863356 shutdown_connections 2024-03-24T10:50:36.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.743+0000 7f8304da2700 1 --2- 172.21.15.94:0/3617863356 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f82e806d7f0 0x7f82e806fca0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:36.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.743+0000 7f8304da2700 1 --2- 172.21.15.94:0/3617863356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83000fe240 0x7f83000fda30 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f82f000b3b0 tx=0x7f82f0009e30 comp rx=0 tx=0).stop 2024-03-24T10:50:36.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.743+0000 7f8304da2700 1 --2- 172.21.15.94:0/3617863356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83000feb90 0x7f83000fbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:36.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.743+0000 7f8304da2700 1 -- 172.21.15.94:0/3617863356 >> 172.21.15.94:0/3617863356 conn(0x7f83000f6a00 msgr2=0x7f83000f80c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:50:36.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.744+0000 7f8304da2700 1 -- 172.21.15.94:0/3617863356 shutdown_connections 2024-03-24T10:50:36.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.744+0000 7f8304da2700 1 -- 172.21.15.94:0/3617863356 wait complete. 2024-03-24T10:50:36.746 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 18 2024-03-24T10:50:36.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:36 smithi096 ceph-mon[84258]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:50:36.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:36 smithi096 ceph-mon[84258]: pgmap v70: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 196 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 31/219 objects degraded (14.155%) 2024-03-24T10:50:36.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:36 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:50:36.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:36 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:50:36.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:36 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/3502598092' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:50:36.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.883+0000 7f9bf9030700 1 -- 172.21.15.94:0/2815602311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9bf40fdc50 msgr2=0x7f9bf40fc060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:36.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.883+0000 7f9bf9030700 1 --2- 172.21.15.94:0/2815602311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9bf40fdc50 0x7f9bf40fc060 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9be8004940 tx=0x7f9be8009e30 comp rx=0 tx=0).stop 2024-03-24T10:50:36.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.883+0000 7f9bf9030700 1 -- 172.21.15.94:0/2815602311 shutdown_connections 2024-03-24T10:50:36.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.883+0000 7f9bf9030700 1 --2- 172.21.15.94:0/2815602311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9bf40fdc50 0x7f9bf40fc060 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:36.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.883+0000 7f9bf9030700 1 --2- 172.21.15.94:0/2815602311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bf40fd300 0x7f9bf40fd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:36.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.883+0000 7f9bf9030700 1 -- 172.21.15.94:0/2815602311 >> 172.21.15.94:0/2815602311 conn(0x7f9bf40f6be0 msgr2=0x7f9bf40f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:50:36.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.883+0000 7f9bf9030700 1 -- 172.21.15.94:0/2815602311 shutdown_connections 2024-03-24T10:50:36.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.883+0000 7f9bf9030700 1 -- 172.21.15.94:0/2815602311 wait complete. 2024-03-24T10:50:36.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.884+0000 7f9bf9030700 1 Processor -- start 2024-03-24T10:50:36.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.884+0000 7f9bf9030700 1 -- start start 2024-03-24T10:50:36.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.884+0000 7f9bf9030700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9bf40fd300 0x7f9bf40fb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:50:36.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.884+0000 7f9bf9030700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bf40fdc50 0x7f9bf4112730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:50:36.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.884+0000 7f9bf9030700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf4112c70 con 0x7f9bf40fd300 2024-03-24T10:50:36.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.884+0000 7f9bf9030700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf4112db0 con 0x7f9bf40fdc50 2024-03-24T10:50:36.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.884+0000 7f9bf2d9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9bf40fd300 0x7f9bf40fb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:50:36.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.885+0000 7f9bf2d9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9bf40fd300 0x7f9bf40fb9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42244/0 (socket says 172.21.15.94:42244) 2024-03-24T10:50:36.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.885+0000 7f9bf2d9d700 1 -- 172.21.15.94:0/186826564 learned_addr learned my addr 172.21.15.94:0/186826564 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:50:36.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.885+0000 7f9bf259c700 1 --2- 172.21.15.94:0/186826564 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bf40fdc50 0x7f9bf4112730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:50:36.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.885+0000 7f9bf2d9d700 1 -- 172.21.15.94:0/186826564 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bf40fdc50 msgr2=0x7f9bf4112730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:36.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.885+0000 7f9bf2d9d700 1 --2- 172.21.15.94:0/186826564 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bf40fdc50 0x7f9bf4112730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:36.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.885+0000 7f9bf2d9d700 1 -- 172.21.15.94:0/186826564 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be8005040 con 0x7f9bf40fd300 2024-03-24T10:50:36.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.885+0000 7f9bf259c700 1 --2- 172.21.15.94:0/186826564 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bf40fdc50 0x7f9bf4112730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T10:50:36.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.885+0000 7f9bf2d9d700 1 --2- 172.21.15.94:0/186826564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9bf40fd300 0x7f9bf40fb9b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9be4002a50 tx=0x7f9be40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:50:36.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.886+0000 7f9bdb7fe700 1 -- 172.21.15.94:0/186826564 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9be4007cc0 con 0x7f9bf40fd300 2024-03-24T10:50:36.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.886+0000 7f9bf9030700 1 -- 172.21.15.94:0/186826564 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf4112fb0 con 0x7f9bf40fd300 2024-03-24T10:50:36.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.886+0000 7f9bdb7fe700 1 -- 172.21.15.94:0/186826564 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9be400f040 con 0x7f9bf40fd300 2024-03-24T10:50:36.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.886+0000 7f9bf9030700 1 -- 172.21.15.94:0/186826564 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bf4113540 con 0x7f9bf40fd300 2024-03-24T10:50:36.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.886+0000 7f9bdb7fe700 1 -- 172.21.15.94:0/186826564 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9be400d9f0 con 0x7f9bf40fd300 2024-03-24T10:50:36.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.887+0000 7f9bf9030700 1 -- 172.21.15.94:0/186826564 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bf404c710 con 0x7f9bf40fd300 2024-03-24T10:50:36.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.888+0000 7f9bdb7fe700 1 -- 172.21.15.94:0/186826564 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f9be400db50 con 0x7f9bf40fd300 2024-03-24T10:50:36.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.889+0000 7f9bdb7fe700 1 --2- 172.21.15.94:0/186826564 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f9bdc06d850 0x7f9bdc06fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:50:36.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.889+0000 7f9bdb7fe700 1 -- 172.21.15.94:0/186826564 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f9be4007e20 con 0x7f9bf40fd300 2024-03-24T10:50:36.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.889+0000 7f9bf259c700 1 --2- 172.21.15.94:0/186826564 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f9bdc06d850 0x7f9bdc06fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:50:36.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.889+0000 7f9bf259c700 1 --2- 172.21.15.94:0/186826564 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f9bdc06d850 0x7f9bdc06fd00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9be8004890 tx=0x7f9be8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:50:36.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:36.895+0000 7f9bdb7fe700 1 -- 172.21.15.94:0/186826564 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9be4056b50 con 0x7f9bf40fd300 2024-03-24T10:50:37.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.145+0000 7f9bf9030700 1 -- 172.21.15.94:0/186826564 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9bf4101310 con 0x7f9bdc06d850 2024-03-24T10:50:37.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.146+0000 7f9bdb7fe700 1 -- 172.21.15.94:0/186826564 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f9bf4101310 con 0x7f9bdc06d850 2024-03-24T10:50:37.146 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-03-24T10:50:37.146 INFO:teuthology.orchestra.run.smithi094.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919", 2024-03-24T10:50:37.146 INFO:teuthology.orchestra.run.smithi094.stdout: "in_progress": true, 2024-03-24T10:50:37.146 INFO:teuthology.orchestra.run.smithi094.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T10:50:37.147 INFO:teuthology.orchestra.run.smithi094.stdout: "services_complete": [ 2024-03-24T10:50:37.147 INFO:teuthology.orchestra.run.smithi094.stdout: "mon", 2024-03-24T10:50:37.147 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-03-24T10:50:37.147 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr" 2024-03-24T10:50:37.147 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-03-24T10:50:37.147 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": "8/21 daemons upgraded", 2024-03-24T10:50:37.147 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "Currently upgrading osd daemons", 2024-03-24T10:50:37.147 INFO:teuthology.orchestra.run.smithi094.stdout: "is_paused": false 2024-03-24T10:50:37.147 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-03-24T10:50:37.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.148+0000 7f9bf9030700 1 -- 172.21.15.94:0/186826564 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f9bdc06d850 msgr2=0x7f9bdc06fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:37.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.148+0000 7f9bf9030700 1 --2- 172.21.15.94:0/186826564 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f9bdc06d850 0x7f9bdc06fd00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9be8004890 tx=0x7f9be8009e30 comp rx=0 tx=0).stop 2024-03-24T10:50:37.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.148+0000 7f9bf9030700 1 -- 172.21.15.94:0/186826564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9bf40fd300 msgr2=0x7f9bf40fb9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:37.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.148+0000 7f9bf9030700 1 --2- 172.21.15.94:0/186826564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9bf40fd300 0x7f9bf40fb9b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9be4002a50 tx=0x7f9be40046b0 comp rx=0 tx=0).stop 2024-03-24T10:50:37.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.149+0000 7f9bf9030700 1 -- 172.21.15.94:0/186826564 shutdown_connections 2024-03-24T10:50:37.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.149+0000 7f9bf9030700 1 --2- 172.21.15.94:0/186826564 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f9bdc06d850 0x7f9bdc06fd00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:37.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.149+0000 7f9bf9030700 1 --2- 172.21.15.94:0/186826564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9bf40fd300 0x7f9bf40fb9b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:37.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.149+0000 7f9bf9030700 1 --2- 172.21.15.94:0/186826564 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bf40fdc50 0x7f9bf4112730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:37.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.149+0000 7f9bf9030700 1 -- 172.21.15.94:0/186826564 >> 172.21.15.94:0/186826564 conn(0x7f9bf40f6be0 msgr2=0x7f9bf40f8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:50:37.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.149+0000 7f9bf9030700 1 -- 172.21.15.94:0/186826564 shutdown_connections 2024-03-24T10:50:37.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.149+0000 7f9bf9030700 1 -- 172.21.15.94:0/186826564 wait complete. 2024-03-24T10:50:37.247 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:36 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-1[108085]: 2024-03-24T10:50:36.992+0000 7f407ae8a7c0 -1 osd.1 54 log_to_monitors true 2024-03-24T10:50:37.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.272+0000 7f6003b72700 1 -- 172.21.15.94:0/2374316416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ffc0fce40 msgr2=0x7f5ffc0fd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:37.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.272+0000 7f6003b72700 1 --2- 172.21.15.94:0/2374316416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ffc0fce40 0x7f5ffc0fd250 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8004940 tx=0x7f5ff8009e30 comp rx=0 tx=0).stop 2024-03-24T10:50:37.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.272+0000 7f6003b72700 1 -- 172.21.15.94:0/2374316416 shutdown_connections 2024-03-24T10:50:37.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.272+0000 7f6003b72700 1 --2- 172.21.15.94:0/2374316416 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ffc0fe040 0x7f5ffc0fe4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:37.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.272+0000 7f6003b72700 1 --2- 172.21.15.94:0/2374316416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ffc0fce40 0x7f5ffc0fd250 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:37.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.272+0000 7f6003b72700 1 -- 172.21.15.94:0/2374316416 >> 172.21.15.94:0/2374316416 conn(0x7f5ffc0f8550 msgr2=0x7f5ffc0fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:50:37.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.272+0000 7f6003b72700 1 -- 172.21.15.94:0/2374316416 shutdown_connections 2024-03-24T10:50:37.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.272+0000 7f6003b72700 1 -- 172.21.15.94:0/2374316416 wait complete. 2024-03-24T10:50:37.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.273+0000 7f6003b72700 1 Processor -- start 2024-03-24T10:50:37.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.273+0000 7f6003b72700 1 -- start start 2024-03-24T10:50:37.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.273+0000 7f6003b72700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ffc0fce40 0x7f5ffc114040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:50:37.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.273+0000 7f6003b72700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ffc0fe040 0x7f5ffc114580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:50:37.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.273+0000 7f6003b72700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ffc114b80 con 0x7f5ffc0fe040 2024-03-24T10:50:37.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.273+0000 7f6003b72700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ffc1153c0 con 0x7f5ffc0fce40 2024-03-24T10:50:37.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.273+0000 7f600110d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ffc0fe040 0x7f5ffc114580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:50:37.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.273+0000 7f600110d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ffc0fe040 0x7f5ffc114580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42262/0 (socket says 172.21.15.94:42262) 2024-03-24T10:50:37.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.273+0000 7f600110d700 1 -- 172.21.15.94:0/510715808 learned_addr learned my addr 172.21.15.94:0/510715808 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:50:37.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.273+0000 7f600190e700 1 --2- 172.21.15.94:0/510715808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ffc0fce40 0x7f5ffc114040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:50:37.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.273+0000 7f600110d700 1 -- 172.21.15.94:0/510715808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ffc0fce40 msgr2=0x7f5ffc114040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:37.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.273+0000 7f600110d700 1 --2- 172.21.15.94:0/510715808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ffc0fce40 0x7f5ffc114040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:37.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.273+0000 7f600110d700 1 -- 172.21.15.94:0/510715808 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff8005040 con 0x7f5ffc0fe040 2024-03-24T10:50:37.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.274+0000 7f600110d700 1 --2- 172.21.15.94:0/510715808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ffc0fe040 0x7f5ffc114580 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5fec002a50 tx=0x7f5fec0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:50:37.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.274+0000 7f5ff27fc700 1 -- 172.21.15.94:0/510715808 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5fec007cc0 con 0x7f5ffc0fe040 2024-03-24T10:50:37.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.274+0000 7f6003b72700 1 -- 172.21.15.94:0/510715808 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ffc1156a0 con 0x7f5ffc0fe040 2024-03-24T10:50:37.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.274+0000 7f6003b72700 1 -- 172.21.15.94:0/510715808 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ffc115be0 con 0x7f5ffc0fe040 2024-03-24T10:50:37.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.274+0000 7f5ff27fc700 1 -- 172.21.15.94:0/510715808 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5fec00f040 con 0x7f5ffc0fe040 2024-03-24T10:50:37.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.274+0000 7f5ff27fc700 1 -- 172.21.15.94:0/510715808 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5fec00da30 con 0x7f5ffc0fe040 2024-03-24T10:50:37.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.275+0000 7f6003b72700 1 -- 172.21.15.94:0/510715808 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fe00052e0 con 0x7f5ffc0fe040 2024-03-24T10:50:37.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.275+0000 7f5ff27fc700 1 -- 172.21.15.94:0/510715808 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f5fec00dc70 con 0x7f5ffc0fe040 2024-03-24T10:50:37.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.276+0000 7f5ff27fc700 1 --2- 172.21.15.94:0/510715808 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f5fe806d7e0 0x7f5fe806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:50:37.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.276+0000 7f5ff27fc700 1 -- 172.21.15.94:0/510715808 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f5fec08cbe0 con 0x7f5ffc0fe040 2024-03-24T10:50:37.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.276+0000 7f600190e700 1 --2- 172.21.15.94:0/510715808 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f5fe806d7e0 0x7f5fe806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:50:37.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.276+0000 7f600190e700 1 --2- 172.21.15.94:0/510715808 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f5fe806d7e0 0x7f5fe806fc90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8004890 tx=0x7f5ff8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:50:37.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.282+0000 7f5ff27fc700 1 -- 172.21.15.94:0/510715808 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5fec05afd0 con 0x7f5ffc0fe040 2024-03-24T10:50:37.592 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:37 smithi094 ceph-mon[93957]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:50:37.592 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:37 smithi094 ceph-mon[93957]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:50:37.593 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:37 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:50:37.593 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:37 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:50:37.593 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:37 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/3617863356' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T10:50:37.593 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:37 smithi094 ceph-mon[93957]: Health check update: Degraded data redundancy: 31/219 objects degraded (14.155%), 13 pgs degraded (PG_DEGRADED) 2024-03-24T10:50:37.593 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:37 smithi094 ceph-mon[93957]: from='osd.1 [v2:172.21.15.94:6810/3303603595,v1:172.21.15.94:6811/3303603595]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-24T10:50:37.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.592+0000 7f6003b72700 1 -- 172.21.15.94:0/510715808 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5fe0005590 con 0x7f5ffc0fe040 2024-03-24T10:50:37.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.593+0000 7f5ff27fc700 1 -- 172.21.15.94:0/510715808 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+992 (secure 0 0 0) 0x7f5fec05ab60 con 0x7f5ffc0fe040 2024-03-24T10:50:37.594 INFO:teuthology.orchestra.run.smithi094.stdout:HEALTH_WARN 1 osds down; Degraded data redundancy: 31/219 objects degraded (14.155%), 13 pgs degraded 2024-03-24T10:50:37.594 INFO:teuthology.orchestra.run.smithi094.stdout:[WRN] OSD_DOWN: 1 osds down 2024-03-24T10:50:37.594 INFO:teuthology.orchestra.run.smithi094.stdout: osd.1 (root=default,host=smithi094) is down 2024-03-24T10:50:37.594 INFO:teuthology.orchestra.run.smithi094.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 31/219 objects degraded (14.155%), 13 pgs degraded 2024-03-24T10:50:37.594 INFO:teuthology.orchestra.run.smithi094.stdout: pg 1.0 is active+undersized+degraded, acting [3,0] 2024-03-24T10:50:37.595 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.0 is active+undersized+degraded, acting [3,0] 2024-03-24T10:50:37.595 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.1 is active+undersized+degraded, acting [2,0] 2024-03-24T10:50:37.595 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.2 is active+undersized+degraded, acting [5,0] 2024-03-24T10:50:37.595 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.3 is active+undersized+degraded, acting [5,2] 2024-03-24T10:50:37.595 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.4 is active+undersized+degraded, acting [0,4] 2024-03-24T10:50:37.595 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.6 is active+undersized+degraded, acting [3,4] 2024-03-24T10:50:37.595 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.9 is active+undersized+degraded, acting [4,0] 2024-03-24T10:50:37.595 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.a is active+undersized+degraded, acting [4,3] 2024-03-24T10:50:37.595 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.d is active+undersized+degraded, acting [3,2] 2024-03-24T10:50:37.595 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.10 is active+undersized+degraded, acting [2,0] 2024-03-24T10:50:37.595 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.15 is active+undersized+degraded, acting [3,0] 2024-03-24T10:50:37.596 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.1b is active+undersized+degraded, acting [0,5] 2024-03-24T10:50:37.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.596+0000 7f6003b72700 1 -- 172.21.15.94:0/510715808 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f5fe806d7e0 msgr2=0x7f5fe806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:37.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.596+0000 7f6003b72700 1 --2- 172.21.15.94:0/510715808 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f5fe806d7e0 0x7f5fe806fc90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8004890 tx=0x7f5ff8009e30 comp rx=0 tx=0).stop 2024-03-24T10:50:37.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.596+0000 7f6003b72700 1 -- 172.21.15.94:0/510715808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ffc0fe040 msgr2=0x7f5ffc114580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:50:37.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.596+0000 7f6003b72700 1 --2- 172.21.15.94:0/510715808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ffc0fe040 0x7f5ffc114580 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5fec002a50 tx=0x7f5fec0046b0 comp rx=0 tx=0).stop 2024-03-24T10:50:37.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.596+0000 7f6003b72700 1 -- 172.21.15.94:0/510715808 shutdown_connections 2024-03-24T10:50:37.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.596+0000 7f6003b72700 1 --2- 172.21.15.94:0/510715808 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f5fe806d7e0 0x7f5fe806fc90 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:37.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.596+0000 7f6003b72700 1 --2- 172.21.15.94:0/510715808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ffc0fce40 0x7f5ffc114040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:37.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.596+0000 7f6003b72700 1 --2- 172.21.15.94:0/510715808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ffc0fe040 0x7f5ffc114580 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:50:37.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.596+0000 7f6003b72700 1 -- 172.21.15.94:0/510715808 >> 172.21.15.94:0/510715808 conn(0x7f5ffc0f8550 msgr2=0x7f5ffc0f9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:50:37.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.596+0000 7f6003b72700 1 -- 172.21.15.94:0/510715808 shutdown_connections 2024-03-24T10:50:37.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:50:37.596+0000 7f6003b72700 1 -- 172.21.15.94:0/510715808 wait complete. 2024-03-24T10:50:37.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:37 smithi096 ceph-mon[84258]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:50:37.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:37 smithi096 ceph-mon[84258]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:50:37.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:37 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:50:37.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:37 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:50:37.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:37 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/3617863356' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T10:50:37.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:37 smithi096 ceph-mon[84258]: Health check update: Degraded data redundancy: 31/219 objects degraded (14.155%), 13 pgs degraded (PG_DEGRADED) 2024-03-24T10:50:37.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:37 smithi096 ceph-mon[84258]: from='osd.1 [v2:172.21.15.94:6810/3303603595,v1:172.21.15.94:6811/3303603595]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-24T10:50:38.584 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:50:38 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-1[108085]: 2024-03-24T10:50:38.273+0000 7f40682da700 -1 osd.1 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T10:50:38.584 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:38 smithi094 ceph-mon[93957]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:50:38.584 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:38 smithi094 ceph-mon[93957]: pgmap v71: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 196 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 31/219 objects degraded (14.155%) 2024-03-24T10:50:38.584 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:38 smithi094 ceph-mon[93957]: from='osd.1 [v2:172.21.15.94:6810/3303603595,v1:172.21.15.94:6811/3303603595]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-24T10:50:38.584 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:38 smithi094 ceph-mon[93957]: osdmap e57: 6 total, 5 up, 6 in 2024-03-24T10:50:38.584 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:38 smithi094 ceph-mon[93957]: from='osd.1 [v2:172.21.15.94:6810/3303603595,v1:172.21.15.94:6811/3303603595]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-03-24T10:50:38.584 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:38 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/510715808' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T10:50:38.584 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:38 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:50:38.584 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:38 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:50:38.584 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:38 smithi094 ceph-mon[93957]: failed to encode map e57 with expected crc 2024-03-24T10:50:38.585 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:38 smithi094 ceph-mon[93957]: from='osd.1 [v2:172.21.15.94:6810/3303603595,v1:172.21.15.94:6811/3303603595]' entity='osd.1' 2024-03-24T10:50:38.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:38 smithi096 ceph-mon[84258]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:50:38.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:38 smithi096 ceph-mon[84258]: pgmap v71: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 196 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 31/219 objects degraded (14.155%) 2024-03-24T10:50:38.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:38 smithi096 ceph-mon[84258]: from='osd.1 [v2:172.21.15.94:6810/3303603595,v1:172.21.15.94:6811/3303603595]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-24T10:50:38.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:38 smithi096 ceph-mon[84258]: osdmap e57: 6 total, 5 up, 6 in 2024-03-24T10:50:38.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:38 smithi096 ceph-mon[84258]: from='osd.1 [v2:172.21.15.94:6810/3303603595,v1:172.21.15.94:6811/3303603595]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-03-24T10:50:38.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:38 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/510715808' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T10:50:38.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:38 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:50:38.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:38 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:50:38.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:38 smithi096 ceph-mon[84258]: failed to encode map e57 with expected crc 2024-03-24T10:50:38.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:38 smithi096 ceph-mon[84258]: from='osd.1 [v2:172.21.15.94:6810/3303603595,v1:172.21.15.94:6811/3303603595]' entity='osd.1' 2024-03-24T10:50:39.540 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:39 smithi094 ceph-mon[93957]: failed to encode map e57 with expected crc 2024-03-24T10:50:39.540 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:39 smithi094 ceph-mon[93957]: failed to encode map e57 with expected crc 2024-03-24T10:50:39.541 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:39 smithi094 ceph-mon[93957]: failed to encode map e57 with expected crc 2024-03-24T10:50:39.541 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:39 smithi094 ceph-mon[93957]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T10:50:39.541 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:39 smithi094 ceph-mon[93957]: osd.1 [v2:172.21.15.94:6810/3303603595,v1:172.21.15.94:6811/3303603595] boot 2024-03-24T10:50:39.541 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:39 smithi094 ceph-mon[93957]: osdmap e58: 6 total, 6 up, 6 in 2024-03-24T10:50:39.541 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:39 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T10:50:39.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:39 smithi096 ceph-mon[84258]: failed to encode map e57 with expected crc 2024-03-24T10:50:39.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:39 smithi096 ceph-mon[84258]: failed to encode map e57 with expected crc 2024-03-24T10:50:39.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:39 smithi096 ceph-mon[84258]: failed to encode map e57 with expected crc 2024-03-24T10:50:39.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:39 smithi096 ceph-mon[84258]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T10:50:39.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:39 smithi096 ceph-mon[84258]: osd.1 [v2:172.21.15.94:6810/3303603595,v1:172.21.15.94:6811/3303603595] boot 2024-03-24T10:50:39.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:39 smithi096 ceph-mon[84258]: osdmap e58: 6 total, 6 up, 6 in 2024-03-24T10:50:39.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:39 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T10:50:40.676 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:40 smithi094 ceph-mon[93957]: failed to encode map e58 with expected crc 2024-03-24T10:50:40.676 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:40 smithi094 ceph-mon[93957]: pgmap v74: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 196 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 31/219 objects degraded (14.155%) 2024-03-24T10:50:40.676 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:40 smithi094 ceph-mon[93957]: osdmap e59: 6 total, 6 up, 6 in 2024-03-24T10:50:40.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:40 smithi096 ceph-mon[84258]: failed to encode map e58 with expected crc 2024-03-24T10:50:40.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:40 smithi096 ceph-mon[84258]: pgmap v74: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 196 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 31/219 objects degraded (14.155%) 2024-03-24T10:50:40.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:40 smithi096 ceph-mon[84258]: osdmap e59: 6 total, 6 up, 6 in 2024-03-24T10:50:41.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:41 smithi094 ceph-mon[93957]: failed to encode map e59 with expected crc 2024-03-24T10:50:41.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:41 smithi094 ceph-mon[93957]: failed to encode map e59 with expected crc 2024-03-24T10:50:41.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:41 smithi094 ceph-mon[93957]: failed to encode map e59 with expected crc 2024-03-24T10:50:41.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:41 smithi094 ceph-mon[93957]: failed to encode map e59 with expected crc 2024-03-24T10:50:41.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:41 smithi094 ceph-mon[93957]: failed to encode map e59 with expected crc 2024-03-24T10:50:41.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:41 smithi094 ceph-mon[93957]: failed to encode map e59 with expected crc 2024-03-24T10:50:41.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:41 smithi094 ceph-mon[93957]: failed to encode map e59 with expected crc 2024-03-24T10:50:41.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:41 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:50:41.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:41 smithi096 ceph-mon[84258]: failed to encode map e59 with expected crc 2024-03-24T10:50:41.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:41 smithi096 ceph-mon[84258]: failed to encode map e59 with expected crc 2024-03-24T10:50:41.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:41 smithi096 ceph-mon[84258]: failed to encode map e59 with expected crc 2024-03-24T10:50:41.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:41 smithi096 ceph-mon[84258]: failed to encode map e59 with expected crc 2024-03-24T10:50:41.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:41 smithi096 ceph-mon[84258]: failed to encode map e59 with expected crc 2024-03-24T10:50:41.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:41 smithi096 ceph-mon[84258]: failed to encode map e59 with expected crc 2024-03-24T10:50:41.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:41 smithi096 ceph-mon[84258]: failed to encode map e59 with expected crc 2024-03-24T10:50:41.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:41 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:50:42.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:42 smithi094 ceph-mon[93957]: pgmap v76: 65 pgs: 7 active+undersized+degraded, 7 active+undersized, 51 active+clean; 196 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s; 16/219 objects degraded (7.306%) 2024-03-24T10:50:42.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:42 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:50:42.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:42 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:50:42.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:42 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:50:42.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:42 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:50:42.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:42 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:50:42.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:42 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:50:42.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:42 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:50:42.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:42 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:50:42.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:42 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-24T10:50:42.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:42 smithi094 ceph-mon[93957]: Health check update: Degraded data redundancy: 16/219 objects degraded (7.306%), 7 pgs degraded (PG_DEGRADED) 2024-03-24T10:50:42.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:42 smithi096 ceph-mon[84258]: pgmap v76: 65 pgs: 7 active+undersized+degraded, 7 active+undersized, 51 active+clean; 196 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s; 16/219 objects degraded (7.306%) 2024-03-24T10:50:42.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:42 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:50:42.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:42 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:50:42.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:42 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:50:42.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:42 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:50:42.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:42 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:50:42.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:42 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:50:42.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:42 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:50:42.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:42 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:50:42.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:42 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-24T10:50:42.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:42 smithi096 ceph-mon[84258]: Health check update: Degraded data redundancy: 16/219 objects degraded (7.306%), 7 pgs degraded (PG_DEGRADED) 2024-03-24T10:50:43.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:43 smithi094 ceph-mon[93957]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-24T10:50:43.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:43 smithi094 ceph-mon[93957]: Upgrade: unsafe to stop osd(s) at this time (3 PGs are or would become offline) 2024-03-24T10:50:43.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:43 smithi096 ceph-mon[84258]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-24T10:50:43.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:43 smithi096 ceph-mon[84258]: Upgrade: unsafe to stop osd(s) at this time (3 PGs are or would become offline) 2024-03-24T10:50:44.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:44 smithi094 ceph-mon[93957]: pgmap v77: 65 pgs: 65 active+clean; 196 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 682 B/s rd, 1 op/s 2024-03-24T10:50:44.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:44 smithi094 ceph-mon[93957]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 16/219 objects degraded (7.306%), 7 pgs degraded) 2024-03-24T10:50:44.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:44 smithi094 ceph-mon[93957]: Cluster is now healthy 2024-03-24T10:50:44.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:44 smithi096 ceph-mon[84258]: pgmap v77: 65 pgs: 65 active+clean; 196 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 682 B/s rd, 1 op/s 2024-03-24T10:50:44.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:44 smithi096 ceph-mon[84258]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 16/219 objects degraded (7.306%), 7 pgs degraded) 2024-03-24T10:50:44.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:44 smithi096 ceph-mon[84258]: Cluster is now healthy 2024-03-24T10:50:46.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:46 smithi094 ceph-mon[93957]: pgmap v78: 65 pgs: 65 active+clean; 196 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 531 B/s rd, 0 op/s 2024-03-24T10:50:46.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:46 smithi096 ceph-mon[84258]: pgmap v78: 65 pgs: 65 active+clean; 196 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 531 B/s rd, 0 op/s 2024-03-24T10:50:48.657 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:48 smithi094 ceph-mon[93957]: pgmap v79: 65 pgs: 65 active+clean; 196 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-03-24T10:50:48.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:48 smithi096 ceph-mon[84258]: pgmap v79: 65 pgs: 65 active+clean; 196 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-03-24T10:50:50.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:50 smithi094 ceph-mon[93957]: pgmap v80: 65 pgs: 65 active+clean; 196 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 921 B/s rd, 1 op/s 2024-03-24T10:50:50.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:50 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:50:50.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:50 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:50:50.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:50 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:50:50.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:50 smithi096 ceph-mon[84258]: pgmap v80: 65 pgs: 65 active+clean; 196 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 921 B/s rd, 1 op/s 2024-03-24T10:50:50.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:50 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:50:50.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:50 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:50:50.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:50 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:50:52.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:52 smithi094 ceph-mon[93957]: pgmap v81: 65 pgs: 65 active+clean; 196 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-03-24T10:50:52.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:52 smithi096 ceph-mon[84258]: pgmap v81: 65 pgs: 65 active+clean; 196 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-03-24T10:50:54.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:54 smithi094 ceph-mon[93957]: pgmap v82: 65 pgs: 65 active+clean; 196 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:50:54.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:54 smithi096 ceph-mon[84258]: pgmap v82: 65 pgs: 65 active+clean; 196 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:50:56.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:56 smithi094 ceph-mon[93957]: pgmap v83: 65 pgs: 65 active+clean; 196 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:50:56.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:56 smithi096 ceph-mon[84258]: pgmap v83: 65 pgs: 65 active+clean; 196 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:50:57.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:57 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-24T10:50:57.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:57 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-24T10:50:58.626 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:58 smithi094 ceph-mon[93957]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-24T10:50:58.627 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:58 smithi094 ceph-mon[93957]: Upgrade: osd.2 is safe to restart 2024-03-24T10:50:58.627 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:58 smithi094 ceph-mon[93957]: pgmap v84: 65 pgs: 65 active+clean; 196 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:50:58.627 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:50:58.627 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-24T10:50:58.627 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:50:58.726 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:58 smithi096 ceph-mon[84258]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-24T10:50:58.726 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:58 smithi096 ceph-mon[84258]: Upgrade: osd.2 is safe to restart 2024-03-24T10:50:58.727 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:58 smithi096 ceph-mon[84258]: pgmap v84: 65 pgs: 65 active+clean; 196 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:50:58.727 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:50:58.727 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-24T10:50:58.727 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:50:59.544 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:59 smithi094 ceph-mon[93957]: Upgrade: Updating osd.2 2024-03-24T10:50:59.547 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:50:59 smithi094 ceph-mon[93957]: Deploying daemon osd.2 on smithi094 2024-03-24T10:50:59.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:59 smithi096 ceph-mon[84258]: Upgrade: Updating osd.2 2024-03-24T10:50:59.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:50:59 smithi096 ceph-mon[84258]: Deploying daemon osd.2 on smithi094 2024-03-24T10:51:00.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:00 smithi094 ceph-mon[93957]: pgmap v85: 65 pgs: 65 active+clean; 196 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:51:00.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:00 smithi096 ceph-mon[84258]: pgmap v85: 65 pgs: 65 active+clean; 196 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:51:01.248 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:01 smithi094 systemd[1]: Stopping Ceph osd.2 for 386110d4-e9ca-11ee-95cd-87774f69a715... 2024-03-24T10:51:01.747 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:01 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-2[62439]: 2024-03-24T10:51:01.396+0000 7fc7b04a6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T10:51:01.748 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:01 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-2[62439]: 2024-03-24T10:51:01.396+0000 7fc7b04a6700 -1 osd.2 59 *** Got signal Terminated *** 2024-03-24T10:51:01.748 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:01 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-2[62439]: 2024-03-24T10:51:01.396+0000 7fc7b04a6700 -1 osd.2 59 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T10:51:02.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:02 smithi094 ceph-mon[93957]: pgmap v86: 65 pgs: 65 active+clean; 196 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:51:02.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:02 smithi094 ceph-mon[93957]: osd.2 marked itself down and dead 2024-03-24T10:51:02.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:02 smithi096 ceph-mon[84258]: pgmap v86: 65 pgs: 65 active+clean; 196 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:51:02.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:02 smithi096 ceph-mon[84258]: osd.2 marked itself down and dead 2024-03-24T10:51:03.140 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:02 smithi094 podman[113284]: 2024-03-24 10:51:02.849687152 +0000 UTC m=+1.577548104 container died cf0aefd20c763317dc044f999ce362f5241b911ec64d7c73daed5e4d4686683e (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=quincy-9c4e0c3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-24T10:51:03.490 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:03 smithi094 ceph-mon[93957]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T10:51:03.491 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:03 smithi094 ceph-mon[93957]: osdmap e60: 6 total, 5 up, 6 in 2024-03-24T10:51:03.491 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:03 smithi094 ceph-mon[93957]: failed to encode map e60 with expected crc 2024-03-24T10:51:03.491 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:03 smithi094 ceph-mon[93957]: failed to encode map e60 with expected crc 2024-03-24T10:51:03.491 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:03 smithi094 ceph-mon[93957]: failed to encode map e60 with expected crc 2024-03-24T10:51:03.491 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:03 smithi094 ceph-mon[93957]: failed to encode map e60 with expected crc 2024-03-24T10:51:03.491 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:03 smithi094 podman[113284]: 2024-03-24 10:51:03.247821655 +0000 UTC m=+1.975682598 container cleanup cf0aefd20c763317dc044f999ce362f5241b911ec64d7c73daed5e4d4686683e (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-9c4e0c3, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-03-24T10:51:03.491 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:03 smithi094 bash[113284]: ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-2 2024-03-24T10:51:03.747 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:03 smithi094 podman[113301]: 2024-03-24 10:51:03.490978869 +0000 UTC m=+0.632455749 container remove cf0aefd20c763317dc044f999ce362f5241b911ec64d7c73daed5e4d4686683e (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=quincy-9c4e0c3, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T10:51:03.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:03 smithi096 ceph-mon[84258]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T10:51:03.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:03 smithi096 ceph-mon[84258]: osdmap e60: 6 total, 5 up, 6 in 2024-03-24T10:51:03.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:03 smithi096 ceph-mon[84258]: failed to encode map e60 with expected crc 2024-03-24T10:51:03.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:03 smithi096 ceph-mon[84258]: failed to encode map e60 with expected crc 2024-03-24T10:51:03.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:03 smithi096 ceph-mon[84258]: failed to encode map e60 with expected crc 2024-03-24T10:51:03.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:03 smithi096 ceph-mon[84258]: failed to encode map e60 with expected crc 2024-03-24T10:51:04.118 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:03 smithi094 podman[113389]: 2024-03-24 10:51:03.800219027 +0000 UTC m=+0.103825645 container create 6f99cf7917afdf6665a476e659b5ed6dc23779f2a791b5c7be8b5d572104b08e (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-2-deactivate, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-24T10:51:04.118 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:03 smithi094 podman[113389]: 2024-03-24 10:51:03.715746429 +0000 UTC m=+0.019353007 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-24T10:51:04.119 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:04 smithi094 podman[113389]: 2024-03-24 10:51:04.045159791 +0000 UTC m=+0.348766363 container init 6f99cf7917afdf6665a476e659b5ed6dc23779f2a791b5c7be8b5d572104b08e (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-24T10:51:04.119 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:04 smithi094 podman[113389]: 2024-03-24 10:51:04.048247533 +0000 UTC m=+0.351854114 container start 6f99cf7917afdf6665a476e659b5ed6dc23779f2a791b5c7be8b5d572104b08e (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240319) 2024-03-24T10:51:04.119 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:04 smithi094 podman[113389]: 2024-03-24 10:51:04.119041464 +0000 UTC m=+0.422648051 container attach 6f99cf7917afdf6665a476e659b5ed6dc23779f2a791b5c7be8b5d572104b08e (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6) 2024-03-24T10:51:04.380 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:04 smithi094 podman[113389]: 2024-03-24 10:51:04.338275802 +0000 UTC m=+0.641882388 container died 6f99cf7917afdf6665a476e659b5ed6dc23779f2a791b5c7be8b5d572104b08e (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T10:51:04.663 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:04 smithi094 ceph-mon[93957]: pgmap v88: 65 pgs: 15 peering, 2 stale+active+clean, 48 active+clean; 196 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:51:04.664 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:04 smithi094 ceph-mon[93957]: Health check failed: Reduced data availability: 6 pgs peering (PG_AVAILABILITY) 2024-03-24T10:51:04.664 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:04 smithi094 ceph-mon[93957]: osdmap e61: 6 total, 5 up, 6 in 2024-03-24T10:51:04.664 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:04 smithi094 ceph-mon[93957]: failed to encode map e61 with expected crc 2024-03-24T10:51:04.664 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:04 smithi094 ceph-mon[93957]: failed to encode map e61 with expected crc 2024-03-24T10:51:04.664 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:04 smithi094 ceph-mon[93957]: failed to encode map e61 with expected crc 2024-03-24T10:51:04.664 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:04 smithi094 ceph-mon[93957]: failed to encode map e61 with expected crc 2024-03-24T10:51:04.664 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:04 smithi094 ceph-mon[93957]: failed to encode map e61 with expected crc 2024-03-24T10:51:04.664 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:04 smithi094 ceph-mon[93957]: failed to encode map e61 with expected crc 2024-03-24T10:51:04.664 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:04 smithi094 ceph-mon[93957]: failed to encode map e61 with expected crc 2024-03-24T10:51:04.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:04 smithi096 ceph-mon[84258]: pgmap v88: 65 pgs: 15 peering, 2 stale+active+clean, 48 active+clean; 196 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:51:04.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:04 smithi096 ceph-mon[84258]: Health check failed: Reduced data availability: 6 pgs peering (PG_AVAILABILITY) 2024-03-24T10:51:04.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:04 smithi096 ceph-mon[84258]: osdmap e61: 6 total, 5 up, 6 in 2024-03-24T10:51:04.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:04 smithi096 ceph-mon[84258]: failed to encode map e61 with expected crc 2024-03-24T10:51:04.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:04 smithi096 ceph-mon[84258]: failed to encode map e61 with expected crc 2024-03-24T10:51:04.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:04 smithi096 ceph-mon[84258]: failed to encode map e61 with expected crc 2024-03-24T10:51:04.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:04 smithi096 ceph-mon[84258]: failed to encode map e61 with expected crc 2024-03-24T10:51:04.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:04 smithi096 ceph-mon[84258]: failed to encode map e61 with expected crc 2024-03-24T10:51:04.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:04 smithi096 ceph-mon[84258]: failed to encode map e61 with expected crc 2024-03-24T10:51:04.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:04 smithi096 ceph-mon[84258]: failed to encode map e61 with expected crc 2024-03-24T10:51:04.919 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:04 smithi094 podman[113415]: 2024-03-24 10:51:04.87274668 +0000 UTC m=+0.528903115 container remove 6f99cf7917afdf6665a476e659b5ed6dc23779f2a791b5c7be8b5d572104b08e (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T10:51:04.920 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:04 smithi094 systemd[1]: ceph-386110d4-e9ca-11ee-95cd-87774f69a715@osd.2.service: Deactivated successfully. 2024-03-24T10:51:04.920 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:04 smithi094 systemd[1]: Stopped Ceph osd.2 for 386110d4-e9ca-11ee-95cd-87774f69a715. 2024-03-24T10:51:04.920 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:04 smithi094 systemd[1]: ceph-386110d4-e9ca-11ee-95cd-87774f69a715@osd.2.service: Consumed 55.305s CPU time. 2024-03-24T10:51:05.247 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:05 smithi094 systemd[1]: Starting Ceph osd.2 for 386110d4-e9ca-11ee-95cd-87774f69a715... 2024-03-24T10:51:05.626 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:05 smithi094 podman[113517]: 2024-03-24 10:51:05.245789737 +0000 UTC m=+0.019155344 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-24T10:51:05.626 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:05 smithi094 podman[113517]: 2024-03-24 10:51:05.348906931 +0000 UTC m=+0.122272567 container create 280278aa512200180c77b6d5f1e17c0ad2968e12a115793558a580527115dae5 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-2-activate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6) 2024-03-24T10:51:05.626 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:05 smithi094 podman[113517]: 2024-03-24 10:51:05.56883225 +0000 UTC m=+0.342197853 container init 280278aa512200180c77b6d5f1e17c0ad2968e12a115793558a580527115dae5 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-2-activate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-24T10:51:05.626 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:05 smithi094 podman[113517]: 2024-03-24 10:51:05.572201845 +0000 UTC m=+0.345567448 container start 280278aa512200180c77b6d5f1e17c0ad2968e12a115793558a580527115dae5 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-2-activate, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-24T10:51:05.626 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:05 smithi094 podman[113517]: 2024-03-24 10:51:05.626521427 +0000 UTC m=+0.399887024 container attach 280278aa512200180c77b6d5f1e17c0ad2968e12a115793558a580527115dae5 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-2-activate, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-24T10:51:06.296 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:06 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-2-activate[113532]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-24T10:51:06.296 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:06 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-2-activate[113532]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-24T10:51:06.296 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:06 smithi094 bash[113517]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-24T10:51:06.296 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:06 smithi094 bash[113517]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-24T10:51:06.296 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:06 smithi094 bash[113517]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-24T10:51:06.296 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:06 smithi094 bash[113517]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-24T10:51:06.296 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:06 smithi094 bash[113517]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-03-24T10:51:06.296 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:06 smithi094 bash[113517]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-24T10:51:06.296 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:06 smithi094 bash[113517]: --> ceph-volume raw activate successful for osd ID: 2 2024-03-24T10:51:06.297 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:06 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-2-activate[113532]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-24T10:51:06.297 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:06 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-2-activate[113532]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-24T10:51:06.297 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:06 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-2-activate[113532]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-03-24T10:51:06.297 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:06 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-2-activate[113532]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-24T10:51:06.297 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:06 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-2-activate[113532]: --> ceph-volume raw activate successful for osd ID: 2 2024-03-24T10:51:06.297 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:06 smithi094 podman[113517]: 2024-03-24 10:51:06.033369557 +0000 UTC m=+0.806735169 container died 280278aa512200180c77b6d5f1e17c0ad2968e12a115793558a580527115dae5 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-2-activate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-24T10:51:06.604 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:06 smithi094 ceph-mon[93957]: pgmap v90: 65 pgs: 15 peering, 2 stale+active+clean, 48 active+clean; 196 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:51:06.605 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:06 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:51:06.605 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:06 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:51:06.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:06 smithi096 ceph-mon[84258]: pgmap v90: 65 pgs: 15 peering, 2 stale+active+clean, 48 active+clean; 196 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:51:06.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:06 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:51:06.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:06 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:51:06.862 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:06 smithi094 podman[113666]: 2024-03-24 10:51:06.605309443 +0000 UTC m=+0.566871165 container remove 280278aa512200180c77b6d5f1e17c0ad2968e12a115793558a580527115dae5 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-2-activate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6) 2024-03-24T10:51:06.862 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:06 smithi094 podman[113731]: 2024-03-24 10:51:06.789743779 +0000 UTC m=+0.087414481 container create c961dcdc746feeda53bc276575966b3a3d633503f742b4a744b289741956888a (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-24T10:51:06.862 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:06 smithi094 podman[113731]: 2024-03-24 10:51:06.720116163 +0000 UTC m=+0.017786855 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-24T10:51:07.247 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:07 smithi094 podman[113731]: 2024-03-24 10:51:07.008562987 +0000 UTC m=+0.306233689 container init c961dcdc746feeda53bc276575966b3a3d633503f742b4a744b289741956888a (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-2, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-24T10:51:07.247 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:07 smithi094 podman[113731]: 2024-03-24 10:51:07.012341675 +0000 UTC m=+0.310012376 container start c961dcdc746feeda53bc276575966b3a3d633503f742b4a744b289741956888a (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-2, org.label-schema.build-date=20240319, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-24T10:51:07.247 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:07 smithi094 ceph-osd[113750]: -- 172.21.15.94:0/1060530203 <== mon.0 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560d4ddd91e0 con 0x560d4ddab000 2024-03-24T10:51:07.248 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:07 smithi094 bash[113731]: c961dcdc746feeda53bc276575966b3a3d633503f742b4a744b289741956888a 2024-03-24T10:51:07.248 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:07 smithi094 systemd[1]: Started Ceph osd.2 for 386110d4-e9ca-11ee-95cd-87774f69a715. 2024-03-24T10:51:07.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:07.722+0000 7f4a6fe47700 1 -- 172.21.15.94:0/880697769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a68066ef0 msgr2=0x7f4a680ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:07.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:07.722+0000 7f4a6fe47700 1 --2- 172.21.15.94:0/880697769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a68066ef0 0x7f4a680ff9c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4a64004940 tx=0x7f4a64009e30 comp rx=0 tx=0).stop 2024-03-24T10:51:07.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:07.722+0000 7f4a6fe47700 1 -- 172.21.15.94:0/880697769 shutdown_connections 2024-03-24T10:51:07.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:07.722+0000 7f4a6fe47700 1 --2- 172.21.15.94:0/880697769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a680fff00 0x7f4a68102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:07.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:07.722+0000 7f4a6fe47700 1 --2- 172.21.15.94:0/880697769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a68066ef0 0x7f4a680ff9c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:07.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:07.722+0000 7f4a6fe47700 1 -- 172.21.15.94:0/880697769 >> 172.21.15.94:0/880697769 conn(0x7f4a680f7430 msgr2=0x7f4a680f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:51:07.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:07.722+0000 7f4a6fe47700 1 -- 172.21.15.94:0/880697769 shutdown_connections 2024-03-24T10:51:07.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:07.722+0000 7f4a6fe47700 1 -- 172.21.15.94:0/880697769 wait complete. 2024-03-24T10:51:07.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:07.723+0000 7f4a6fe47700 1 Processor -- start 2024-03-24T10:51:07.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:07.723+0000 7f4a6fe47700 1 -- start start 2024-03-24T10:51:07.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:07.723+0000 7f4a6fe47700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a68066ef0 0x7f4a6810d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:51:07.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:07.723+0000 7f4a6fe47700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a680fff00 0x7f4a6810de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:51:07.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:07.723+0000 7f4a6fe47700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6810e440 con 0x7f4a68066ef0 2024-03-24T10:51:07.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:07.723+0000 7f4a6fe47700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6810e5b0 con 0x7f4a680fff00 2024-03-24T10:51:07.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:07.723+0000 7f4a6dbe3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a68066ef0 0x7f4a6810d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:51:07.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:07.723+0000 7f4a6dbe3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a68066ef0 0x7f4a6810d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39798/0 (socket says 172.21.15.94:39798) 2024-03-24T10:51:07.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:07.723+0000 7f4a6d3e2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a680fff00 0x7f4a6810de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:51:07.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:07.723+0000 7f4a6dbe3700 1 -- 172.21.15.94:0/2030743698 learned_addr learned my addr 172.21.15.94:0/2030743698 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:51:07.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:07.723+0000 7f4a6d3e2700 1 -- 172.21.15.94:0/2030743698 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a68066ef0 msgr2=0x7f4a6810d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:07.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:07.723+0000 7f4a6d3e2700 1 --2- 172.21.15.94:0/2030743698 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a68066ef0 0x7f4a6810d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:07.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:07.723+0000 7f4a6d3e2700 1 -- 172.21.15.94:0/2030743698 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a64005040 con 0x7f4a680fff00 2024-03-24T10:51:07.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:07.723+0000 7f4a6dbe3700 1 --2- 172.21.15.94:0/2030743698 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a68066ef0 0x7f4a6810d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T10:51:07.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:07.724+0000 7f4a6d3e2700 1 --2- 172.21.15.94:0/2030743698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a680fff00 0x7f4a6810de40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4a58009fd0 tx=0x7f4a5800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:51:07.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:07.724+0000 7f4a5e7fc700 1 -- 172.21.15.94:0/2030743698 <== mon.1 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a58004e90 con 0x7f4a680fff00 2024-03-24T10:51:07.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:07.724+0000 7f4a5e7fc700 1 -- 172.21.15.94:0/2030743698 <== mon.1 v2:172.21.15.96:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4a5800f040 con 0x7f4a680fff00 2024-03-24T10:51:07.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:07.724+0000 7f4a6fe47700 1 -- 172.21.15.94:0/2030743698 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a68112fb0 con 0x7f4a680fff00 2024-03-24T10:51:07.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:07.724+0000 7f4a5e7fc700 1 -- 172.21.15.94:0/2030743698 <== mon.1 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a5800ca50 con 0x7f4a680fff00 2024-03-24T10:51:07.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:07.724+0000 7f4a6fe47700 1 -- 172.21.15.94:0/2030743698 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a681134c0 con 0x7f4a680fff00 2024-03-24T10:51:07.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:07.725+0000 7f4a6fe47700 1 -- 172.21.15.94:0/2030743698 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a6804c710 con 0x7f4a680fff00 2024-03-24T10:51:07.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:07.725+0000 7f4a5e7fc700 1 -- 172.21.15.94:0/2030743698 <== mon.1 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f4a5800cbb0 con 0x7f4a680fff00 2024-03-24T10:51:07.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:07.726+0000 7f4a5e7fc700 1 --2- 172.21.15.94:0/2030743698 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f4a5406d7e0 0x7f4a5406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:51:07.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:07.726+0000 7f4a5e7fc700 1 -- 172.21.15.94:0/2030743698 <== mon.1 v2:172.21.15.96:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f4a5808c8c0 con 0x7f4a680fff00 2024-03-24T10:51:07.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:07.726+0000 7f4a6dbe3700 1 --2- 172.21.15.94:0/2030743698 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f4a5406d7e0 0x7f4a5406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:51:07.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:07.726+0000 7f4a6dbe3700 1 --2- 172.21.15.94:0/2030743698 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f4a5406d7e0 0x7f4a5406fc90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4a6400b0d0 tx=0x7f4a64009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:51:07.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:07.731+0000 7f4a5e7fc700 1 -- 172.21.15.94:0/2030743698 <== mon.1 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4a58059d20 con 0x7f4a680fff00 2024-03-24T10:51:07.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:07.977+0000 7f4a6fe47700 1 -- 172.21.15.94:0/2030743698 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a680f8f70 con 0x7f4a5406d7e0 2024-03-24T10:51:07.978 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:07 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-2[113746]: 2024-03-24T10:51:07.554+0000 7f07d3d2c7c0 -1 Falling back to public interface 2024-03-24T10:51:07.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:07.978+0000 7f4a5e7fc700 1 -- 172.21.15.94:0/2030743698 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f4a680f8f70 con 0x7f4a5406d7e0 2024-03-24T10:51:07.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:07.980+0000 7f4a6fe47700 1 -- 172.21.15.94:0/2030743698 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f4a5406d7e0 msgr2=0x7f4a5406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:07.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:07.980+0000 7f4a6fe47700 1 --2- 172.21.15.94:0/2030743698 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f4a5406d7e0 0x7f4a5406fc90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4a6400b0d0 tx=0x7f4a64009e30 comp rx=0 tx=0).stop 2024-03-24T10:51:07.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:07.980+0000 7f4a6fe47700 1 -- 172.21.15.94:0/2030743698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a680fff00 msgr2=0x7f4a6810de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:07.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:07.980+0000 7f4a6fe47700 1 --2- 172.21.15.94:0/2030743698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a680fff00 0x7f4a6810de40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4a58009fd0 tx=0x7f4a5800b040 comp rx=0 tx=0).stop 2024-03-24T10:51:07.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:07.980+0000 7f4a6fe47700 1 -- 172.21.15.94:0/2030743698 shutdown_connections 2024-03-24T10:51:07.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:07.980+0000 7f4a6fe47700 1 --2- 172.21.15.94:0/2030743698 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f4a5406d7e0 0x7f4a5406fc90 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:07.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:07.980+0000 7f4a6fe47700 1 --2- 172.21.15.94:0/2030743698 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a68066ef0 0x7f4a6810d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:07.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:07.980+0000 7f4a6fe47700 1 --2- 172.21.15.94:0/2030743698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a680fff00 0x7f4a6810de40 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:07.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:07.980+0000 7f4a6fe47700 1 -- 172.21.15.94:0/2030743698 >> 172.21.15.94:0/2030743698 conn(0x7f4a680f7430 msgr2=0x7f4a680f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:51:07.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:07.981+0000 7f4a6fe47700 1 -- 172.21.15.94:0/2030743698 shutdown_connections 2024-03-24T10:51:07.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:07.981+0000 7f4a6fe47700 1 -- 172.21.15.94:0/2030743698 wait complete. 2024-03-24T10:51:07.993 INFO:teuthology.orchestra.run.smithi094.stdout:true 2024-03-24T10:51:08.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.088+0000 7f7bc8bb5700 1 -- 172.21.15.94:0/1753422846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7bc40fbd00 msgr2=0x7f7bc40fc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:08.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.088+0000 7f7bc8bb5700 1 --2- 172.21.15.94:0/1753422846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7bc40fbd00 0x7f7bc40fc110 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4004940 tx=0x7f7bb4009e30 comp rx=0 tx=0).stop 2024-03-24T10:51:08.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.088+0000 7f7bc8bb5700 1 -- 172.21.15.94:0/1753422846 shutdown_connections 2024-03-24T10:51:08.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.088+0000 7f7bc8bb5700 1 --2- 172.21.15.94:0/1753422846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7bc40fcfb0 0x7f7bc40fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:08.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.088+0000 7f7bc8bb5700 1 --2- 172.21.15.94:0/1753422846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7bc40fbd00 0x7f7bc40fc110 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:08.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.088+0000 7f7bc8bb5700 1 -- 172.21.15.94:0/1753422846 >> 172.21.15.94:0/1753422846 conn(0x7f7bc40f7480 msgr2=0x7f7bc40f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:51:08.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.088+0000 7f7bc8bb5700 1 -- 172.21.15.94:0/1753422846 shutdown_connections 2024-03-24T10:51:08.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.088+0000 7f7bc8bb5700 1 -- 172.21.15.94:0/1753422846 wait complete. 2024-03-24T10:51:08.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.089+0000 7f7bc8bb5700 1 Processor -- start 2024-03-24T10:51:08.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.089+0000 7f7bc8bb5700 1 -- start start 2024-03-24T10:51:08.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.089+0000 7f7bc8bb5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7bc40fbd00 0x7f7bc410c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:51:08.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.089+0000 7f7bc8bb5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7bc40fcfb0 0x7f7bc410cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:51:08.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.089+0000 7f7bc8bb5700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc410d380 con 0x7f7bc40fbd00 2024-03-24T10:51:08.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.089+0000 7f7bc8bb5700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc410d4f0 con 0x7f7bc40fcfb0 2024-03-24T10:51:08.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.089+0000 7f7bc259c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7bc40fbd00 0x7f7bc410c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:51:08.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.089+0000 7f7bc259c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7bc40fbd00 0x7f7bc410c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39812/0 (socket says 172.21.15.94:39812) 2024-03-24T10:51:08.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.089+0000 7f7bc259c700 1 -- 172.21.15.94:0/2639571285 learned_addr learned my addr 172.21.15.94:0/2639571285 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:51:08.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.089+0000 7f7bc1d9b700 1 --2- 172.21.15.94:0/2639571285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7bc40fcfb0 0x7f7bc410cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:51:08.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.089+0000 7f7bc259c700 1 -- 172.21.15.94:0/2639571285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7bc40fcfb0 msgr2=0x7f7bc410cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:08.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.089+0000 7f7bc259c700 1 --2- 172.21.15.94:0/2639571285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7bc40fcfb0 0x7f7bc410cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:08.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.089+0000 7f7bc259c700 1 -- 172.21.15.94:0/2639571285 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb4005040 con 0x7f7bc40fbd00 2024-03-24T10:51:08.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.090+0000 7f7bc259c700 1 --2- 172.21.15.94:0/2639571285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7bc40fbd00 0x7f7bc410c840 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7bb400b3b0 tx=0x7f7bb4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:51:08.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.090+0000 7f7bb2ffd700 1 -- 172.21.15.94:0/2639571285 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bb400e040 con 0x7f7bc40fbd00 2024-03-24T10:51:08.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.090+0000 7f7bb2ffd700 1 -- 172.21.15.94:0/2639571285 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7bb400bbf0 con 0x7f7bc40fbd00 2024-03-24T10:51:08.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.090+0000 7f7bb2ffd700 1 -- 172.21.15.94:0/2639571285 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bb401c630 con 0x7f7bc40fbd00 2024-03-24T10:51:08.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.090+0000 7f7bc8bb5700 1 -- 172.21.15.94:0/2639571285 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bc4066290 con 0x7f7bc40fbd00 2024-03-24T10:51:08.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.090+0000 7f7bc8bb5700 1 -- 172.21.15.94:0/2639571285 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bc40667d0 con 0x7f7bc40fbd00 2024-03-24T10:51:08.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.091+0000 7f7bc8bb5700 1 -- 172.21.15.94:0/2639571285 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bc404c710 con 0x7f7bc40fbd00 2024-03-24T10:51:08.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.092+0000 7f7bb2ffd700 1 -- 172.21.15.94:0/2639571285 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f7bb400bd60 con 0x7f7bc40fbd00 2024-03-24T10:51:08.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.092+0000 7f7bb2ffd700 1 --2- 172.21.15.94:0/2639571285 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f7bac06d7e0 0x7f7bac06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:51:08.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.092+0000 7f7bb2ffd700 1 -- 172.21.15.94:0/2639571285 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f7bb408da00 con 0x7f7bc40fbd00 2024-03-24T10:51:08.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.092+0000 7f7bc1d9b700 1 --2- 172.21.15.94:0/2639571285 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f7bac06d7e0 0x7f7bac06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:51:08.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.093+0000 7f7bc1d9b700 1 --2- 172.21.15.94:0/2639571285 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f7bac06d7e0 0x7f7bac06fc90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8004f40 tx=0x7f7bb800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:51:08.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.098+0000 7f7bb2ffd700 1 -- 172.21.15.94:0/2639571285 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7bb405ae40 con 0x7f7bc40fbd00 2024-03-24T10:51:08.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:08 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:51:08.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:08 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:51:08.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:08 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:51:08.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:08 smithi094 ceph-mon[93957]: pgmap v91: 65 pgs: 5 active+undersized+degraded, 15 peering, 7 active+undersized, 38 active+clean; 196 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 9/219 objects degraded (4.110%) 2024-03-24T10:51:08.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.328+0000 7f7bc8bb5700 1 -- 172.21.15.94:0/2639571285 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7bc4066fc0 con 0x7f7bac06d7e0 2024-03-24T10:51:08.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.329+0000 7f7bb2ffd700 1 -- 172.21.15.94:0/2639571285 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f7bc4066fc0 con 0x7f7bac06d7e0 2024-03-24T10:51:08.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.331+0000 7f7bc8bb5700 1 -- 172.21.15.94:0/2639571285 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f7bac06d7e0 msgr2=0x7f7bac06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:08.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.332+0000 7f7bc8bb5700 1 --2- 172.21.15.94:0/2639571285 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f7bac06d7e0 0x7f7bac06fc90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8004f40 tx=0x7f7bb800b040 comp rx=0 tx=0).stop 2024-03-24T10:51:08.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.332+0000 7f7bc8bb5700 1 -- 172.21.15.94:0/2639571285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7bc40fbd00 msgr2=0x7f7bc410c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:08.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.332+0000 7f7bc8bb5700 1 --2- 172.21.15.94:0/2639571285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7bc40fbd00 0x7f7bc410c840 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7bb400b3b0 tx=0x7f7bb4009e30 comp rx=0 tx=0).stop 2024-03-24T10:51:08.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.332+0000 7f7bc8bb5700 1 -- 172.21.15.94:0/2639571285 shutdown_connections 2024-03-24T10:51:08.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.332+0000 7f7bc8bb5700 1 --2- 172.21.15.94:0/2639571285 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f7bac06d7e0 0x7f7bac06fc90 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:08.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.332+0000 7f7bc8bb5700 1 --2- 172.21.15.94:0/2639571285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7bc40fbd00 0x7f7bc410c840 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:08.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.332+0000 7f7bc8bb5700 1 --2- 172.21.15.94:0/2639571285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7bc40fcfb0 0x7f7bc410cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:08.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.332+0000 7f7bc8bb5700 1 -- 172.21.15.94:0/2639571285 >> 172.21.15.94:0/2639571285 conn(0x7f7bc40f7480 msgr2=0x7f7bc40f8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:51:08.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.332+0000 7f7bc8bb5700 1 -- 172.21.15.94:0/2639571285 shutdown_connections 2024-03-24T10:51:08.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.332+0000 7f7bc8bb5700 1 -- 172.21.15.94:0/2639571285 wait complete. 2024-03-24T10:51:08.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.445+0000 7fc4b36b1700 1 -- 172.21.15.94:0/1180644756 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4ac0fe400 msgr2=0x7fc4ac0fe810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:08.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.445+0000 7fc4b36b1700 1 --2- 172.21.15.94:0/1180644756 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4ac0fe400 0x7fc4ac0fe810 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8004940 tx=0x7fc4a8009e30 comp rx=0 tx=0).stop 2024-03-24T10:51:08.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.445+0000 7fc4b36b1700 1 -- 172.21.15.94:0/1180644756 shutdown_connections 2024-03-24T10:51:08.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.445+0000 7fc4b36b1700 1 --2- 172.21.15.94:0/1180644756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4ac0fed50 0x7fc4ac0fae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:08.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.445+0000 7fc4b36b1700 1 --2- 172.21.15.94:0/1180644756 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4ac0fe400 0x7fc4ac0fe810 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:08.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.445+0000 7fc4b36b1700 1 -- 172.21.15.94:0/1180644756 >> 172.21.15.94:0/1180644756 conn(0x7fc4ac0f6ba0 msgr2=0x7fc4ac0f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:51:08.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.445+0000 7fc4b36b1700 1 -- 172.21.15.94:0/1180644756 shutdown_connections 2024-03-24T10:51:08.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.446+0000 7fc4b36b1700 1 -- 172.21.15.94:0/1180644756 wait complete. 2024-03-24T10:51:08.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.446+0000 7fc4b36b1700 1 Processor -- start 2024-03-24T10:51:08.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.446+0000 7fc4b36b1700 1 -- start start 2024-03-24T10:51:08.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.446+0000 7fc4b36b1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4ac0fe400 0x7fc4ac10d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:51:08.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.446+0000 7fc4b36b1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4ac0fed50 0x7fc4ac10df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:51:08.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.446+0000 7fc4b36b1700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ac10e530 con 0x7fc4ac0fed50 2024-03-24T10:51:08.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.446+0000 7fc4b36b1700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ac10ed80 con 0x7fc4ac0fe400 2024-03-24T10:51:08.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.446+0000 7fc4b0c4c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4ac0fed50 0x7fc4ac10df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:51:08.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.446+0000 7fc4b0c4c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4ac0fed50 0x7fc4ac10df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39824/0 (socket says 172.21.15.94:39824) 2024-03-24T10:51:08.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.446+0000 7fc4b0c4c700 1 -- 172.21.15.94:0/4270143301 learned_addr learned my addr 172.21.15.94:0/4270143301 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:51:08.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.446+0000 7fc4b144d700 1 --2- 172.21.15.94:0/4270143301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4ac0fe400 0x7fc4ac10d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:51:08.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.447+0000 7fc4b0c4c700 1 -- 172.21.15.94:0/4270143301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4ac0fe400 msgr2=0x7fc4ac10d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:08.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.447+0000 7fc4b0c4c700 1 --2- 172.21.15.94:0/4270143301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4ac0fe400 0x7fc4ac10d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:08.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.447+0000 7fc4b0c4c700 1 -- 172.21.15.94:0/4270143301 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4a8005040 con 0x7fc4ac0fed50 2024-03-24T10:51:08.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.447+0000 7fc4b0c4c700 1 --2- 172.21.15.94:0/4270143301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4ac0fed50 0x7fc4ac10df30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc49c002a50 tx=0x7fc49c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:51:08.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.447+0000 7fc4a1ffb700 1 -- 172.21.15.94:0/4270143301 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc49c007cc0 con 0x7fc4ac0fed50 2024-03-24T10:51:08.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.447+0000 7fc4a1ffb700 1 -- 172.21.15.94:0/4270143301 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc49c00f040 con 0x7fc4ac0fed50 2024-03-24T10:51:08.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.447+0000 7fc4b36b1700 1 -- 172.21.15.94:0/4270143301 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4ac10f060 con 0x7fc4ac0fed50 2024-03-24T10:51:08.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.447+0000 7fc4a1ffb700 1 -- 172.21.15.94:0/4270143301 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc49c00da30 con 0x7fc4ac0fed50 2024-03-24T10:51:08.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.448+0000 7fc4b36b1700 1 -- 172.21.15.94:0/4270143301 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4ac10f5a0 con 0x7fc4ac0fed50 2024-03-24T10:51:08.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.449+0000 7fc4a1ffb700 1 -- 172.21.15.94:0/4270143301 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fc49c00db90 con 0x7fc4ac0fed50 2024-03-24T10:51:08.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.449+0000 7fc4a1ffb700 1 --2- 172.21.15.94:0/4270143301 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fc49806d7e0 0x7fc49806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:51:08.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.449+0000 7fc4b36b1700 1 -- 172.21.15.94:0/4270143301 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4ac05f5f0 con 0x7fc4ac0fed50 2024-03-24T10:51:08.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.450+0000 7fc4a1ffb700 1 -- 172.21.15.94:0/4270143301 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fc49c08cbb0 con 0x7fc4ac0fed50 2024-03-24T10:51:08.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.450+0000 7fc4b144d700 1 --2- 172.21.15.94:0/4270143301 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fc49806d7e0 0x7fc49806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:51:08.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.451+0000 7fc4b144d700 1 --2- 172.21.15.94:0/4270143301 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fc49806d7e0 0x7fc49806fc90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc4a800b3b0 tx=0x7fc4a8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:51:08.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.456+0000 7fc4a1ffb700 1 -- 172.21.15.94:0/4270143301 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc49c059f50 con 0x7fc4ac0fed50 2024-03-24T10:51:08.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:08 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:51:08.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:08 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:51:08.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:08 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:51:08.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:08 smithi096 ceph-mon[84258]: pgmap v91: 65 pgs: 5 active+undersized+degraded, 15 peering, 7 active+undersized, 38 active+clean; 196 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 9/219 objects degraded (4.110%) 2024-03-24T10:51:08.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.698+0000 7fc4b36b1700 1 -- 172.21.15.94:0/4270143301 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4ac10f920 con 0x7fc49806d7e0 2024-03-24T10:51:08.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.708+0000 7fc4a1ffb700 1 -- 172.21.15.94:0/4270143301 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fc4ac10f920 con 0x7fc49806d7e0 2024-03-24T10:51:08.709 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T10:51:08.710 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.smithi094 smithi094 *:9093,9094 running (4m) 32s ago 13m 25.7M - 0.25.0 c8568f914cd2 5c583af60554 2024-03-24T10:51:08.710 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi094 smithi094 running (98s) 32s ago 13m 6651k - 19.0.0-1855-g21b3bd67 553463a90fd5 2b0897df571d 2024-03-24T10:51:08.710 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi096 smithi096 running (92s) 88s ago 11m 6669k - 19.0.0-1855-g21b3bd67 553463a90fd5 1759a9ff2805 2024-03-24T10:51:08.710 INFO:teuthology.orchestra.run.smithi094.stdout:grafana.smithi094 smithi094 *:3000 running (3m) 32s ago 12m 76.2M - 9.4.7 954c08fa6188 d3d425bf1b7c 2024-03-24T10:51:08.710 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.efiqte smithi094 running (7m) 32s ago 7m 13.5M - 17.2.7-726-g9c4e0c34 837124da804f f0df0a056c21 2024-03-24T10:51:08.710 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.llefgu smithi094 running (7m) 32s ago 7m 109M - 17.2.7-726-g9c4e0c34 837124da804f 52a9771dc8e0 2024-03-24T10:51:08.710 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi096.egbmoq smithi096 running (7m) 88s ago 7m 154M - 17.2.7-726-g9c4e0c34 837124da804f 1cfc930dbab0 2024-03-24T10:51:08.710 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi096.jtebhh smithi096 running (7m) 88s ago 7m 16.3M - 17.2.7-726-g9c4e0c34 837124da804f 4ff3451a6f6e 2024-03-24T10:51:08.710 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi094.yaethu smithi094 *:8443,9283,8765 running (3m) 32s ago 14m 499M - 19.0.0-1855-g21b3bd67 553463a90fd5 d3adab76b18f 2024-03-24T10:51:08.710 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi096.gfonnu smithi096 *:8443,9283,8765 running (2m) 88s ago 11m 448M - 19.0.0-1855-g21b3bd67 553463a90fd5 22937a059eb8 2024-03-24T10:51:08.710 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi094 smithi094 running (2m) 32s ago 14m 47.3M 2048M 19.0.0-1855-g21b3bd67 553463a90fd5 c779d611e29a 2024-03-24T10:51:08.710 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi096 smithi096 running (2m) 88s ago 11m 31.2M 2048M 19.0.0-1855-g21b3bd67 553463a90fd5 ed9bd0bd57f6 2024-03-24T10:51:08.710 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi094 smithi094 *:9100 running (4m) 32s ago 13m 16.4M - 1.5.0 0da6a335fe13 ca67b53c64e2 2024-03-24T10:51:08.710 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi096 smithi096 *:9100 running (3m) 88s ago 11m 17.0M - 1.5.0 0da6a335fe13 65137a7a7549 2024-03-24T10:51:08.711 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (72s) 32s ago 11m 98.5M 1557M 19.0.0-1855-g21b3bd67 553463a90fd5 ec57ede0c9e4 2024-03-24T10:51:08.711 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi094 running (37s) 32s ago 10m 14.9M 1557M 19.0.0-1855-g21b3bd67 553463a90fd5 c5b8b2bbf8b5 2024-03-24T10:51:08.711 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi094 starting - - - 1557M 2024-03-24T10:51:08.711 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi096 running (9m) 88s ago 9m 285M 2581M 17.2.7-726-g9c4e0c34 837124da804f f4650d0f6241 2024-03-24T10:51:08.711 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi096 running (9m) 88s ago 9m 258M 2581M 17.2.7-726-g9c4e0c34 837124da804f a63d7c9e705f 2024-03-24T10:51:08.711 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi096 running (8m) 88s ago 8m 243M 2581M 17.2.7-726-g9c4e0c34 837124da804f f56fad2959c3 2024-03-24T10:51:08.711 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.smithi094 smithi094 *:9095 running (3m) 32s ago 12m 53.4M - 2.43.0 a07b618ecd1d 6ce091a0c326 2024-03-24T10:51:08.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.711+0000 7fc4b36b1700 1 -- 172.21.15.94:0/4270143301 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fc49806d7e0 msgr2=0x7fc49806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:08.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.711+0000 7fc4b36b1700 1 --2- 172.21.15.94:0/4270143301 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fc49806d7e0 0x7fc49806fc90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc4a800b3b0 tx=0x7fc4a8009e30 comp rx=0 tx=0).stop 2024-03-24T10:51:08.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.711+0000 7fc4b36b1700 1 -- 172.21.15.94:0/4270143301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4ac0fed50 msgr2=0x7fc4ac10df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:08.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.711+0000 7fc4b36b1700 1 --2- 172.21.15.94:0/4270143301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4ac0fed50 0x7fc4ac10df30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc49c002a50 tx=0x7fc49c0046b0 comp rx=0 tx=0).stop 2024-03-24T10:51:08.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.711+0000 7fc4b36b1700 1 -- 172.21.15.94:0/4270143301 shutdown_connections 2024-03-24T10:51:08.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.711+0000 7fc4b36b1700 1 --2- 172.21.15.94:0/4270143301 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fc49806d7e0 0x7fc49806fc90 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:08.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.711+0000 7fc4b36b1700 1 --2- 172.21.15.94:0/4270143301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4ac0fe400 0x7fc4ac10d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:08.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.711+0000 7fc4b36b1700 1 --2- 172.21.15.94:0/4270143301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4ac0fed50 0x7fc4ac10df30 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:08.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.711+0000 7fc4b36b1700 1 -- 172.21.15.94:0/4270143301 >> 172.21.15.94:0/4270143301 conn(0x7fc4ac0f6ba0 msgr2=0x7fc4ac0f82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:51:08.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.711+0000 7fc4b36b1700 1 -- 172.21.15.94:0/4270143301 shutdown_connections 2024-03-24T10:51:08.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.711+0000 7fc4b36b1700 1 -- 172.21.15.94:0/4270143301 wait complete. 2024-03-24T10:51:08.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.823+0000 7f293707d700 1 -- 172.21.15.94:0/3937838570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29300fcf80 msgr2=0x7f29300fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:08.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.823+0000 7f293707d700 1 --2- 172.21.15.94:0/3937838570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29300fcf80 0x7f29300fd3f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f291c004940 tx=0x7f291c009e30 comp rx=0 tx=0).stop 2024-03-24T10:51:08.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.823+0000 7f293707d700 1 -- 172.21.15.94:0/3937838570 shutdown_connections 2024-03-24T10:51:08.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.823+0000 7f293707d700 1 --2- 172.21.15.94:0/3937838570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29300fcf80 0x7f29300fd3f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:08.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.823+0000 7f293707d700 1 --2- 172.21.15.94:0/3937838570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f29300fbcd0 0x7f29300fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:08.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.823+0000 7f293707d700 1 -- 172.21.15.94:0/3937838570 >> 172.21.15.94:0/3937838570 conn(0x7f29300f7430 msgr2=0x7f29300f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:51:08.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.823+0000 7f293707d700 1 -- 172.21.15.94:0/3937838570 shutdown_connections 2024-03-24T10:51:08.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.823+0000 7f293707d700 1 -- 172.21.15.94:0/3937838570 wait complete. 2024-03-24T10:51:08.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.823+0000 7f293707d700 1 Processor -- start 2024-03-24T10:51:08.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.823+0000 7f293707d700 1 -- start start 2024-03-24T10:51:08.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.824+0000 7f293707d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f29300fbcd0 0x7f293010da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:51:08.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.824+0000 7f293707d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29300fcf80 0x7f293010dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:51:08.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.824+0000 7f293707d700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f293010e5d0 con 0x7f29300fcf80 2024-03-24T10:51:08.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.824+0000 7f293707d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f293010edd0 con 0x7f29300fbcd0 2024-03-24T10:51:08.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.824+0000 7f292ffff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29300fcf80 0x7f293010dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:51:08.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.824+0000 7f292ffff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29300fcf80 0x7f293010dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39840/0 (socket says 172.21.15.94:39840) 2024-03-24T10:51:08.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.824+0000 7f292ffff700 1 -- 172.21.15.94:0/365207387 learned_addr learned my addr 172.21.15.94:0/365207387 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:51:08.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.824+0000 7f2934e19700 1 --2- 172.21.15.94:0/365207387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f29300fbcd0 0x7f293010da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:51:08.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.824+0000 7f292ffff700 1 -- 172.21.15.94:0/365207387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f29300fbcd0 msgr2=0x7f293010da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:08.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.824+0000 7f292ffff700 1 --2- 172.21.15.94:0/365207387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f29300fbcd0 0x7f293010da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:08.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.824+0000 7f292ffff700 1 -- 172.21.15.94:0/365207387 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f291c005040 con 0x7f29300fcf80 2024-03-24T10:51:08.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.824+0000 7f292ffff700 1 --2- 172.21.15.94:0/365207387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29300fcf80 0x7f293010dfd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f291c009150 tx=0x7f291c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:51:08.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.825+0000 7f292d7fa700 1 -- 172.21.15.94:0/365207387 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f291c00e040 con 0x7f29300fcf80 2024-03-24T10:51:08.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.825+0000 7f293707d700 1 -- 172.21.15.94:0/365207387 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f293010f050 con 0x7f29300fcf80 2024-03-24T10:51:08.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.825+0000 7f292d7fa700 1 -- 172.21.15.94:0/365207387 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f291c00bcc0 con 0x7f29300fcf80 2024-03-24T10:51:08.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.825+0000 7f292d7fa700 1 -- 172.21.15.94:0/365207387 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f291c01c630 con 0x7f29300fcf80 2024-03-24T10:51:08.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.825+0000 7f293707d700 1 -- 172.21.15.94:0/365207387 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f293010f590 con 0x7f29300fcf80 2024-03-24T10:51:08.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.826+0000 7f293707d700 1 -- 172.21.15.94:0/365207387 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2930101b10 con 0x7f29300fcf80 2024-03-24T10:51:08.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.826+0000 7f292d7fa700 1 -- 172.21.15.94:0/365207387 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f291c013420 con 0x7f29300fcf80 2024-03-24T10:51:08.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.827+0000 7f292d7fa700 1 --2- 172.21.15.94:0/365207387 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f292006d7e0 0x7f292006fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:51:08.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.827+0000 7f292d7fa700 1 -- 172.21.15.94:0/365207387 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f291c08dc00 con 0x7f29300fcf80 2024-03-24T10:51:08.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.827+0000 7f2934e19700 1 --2- 172.21.15.94:0/365207387 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f292006d7e0 0x7f292006fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:51:08.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.827+0000 7f2934e19700 1 --2- 172.21.15.94:0/365207387 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f292006d7e0 0x7f292006fc90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2924007fd0 tx=0x7f2924009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:51:08.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:08.831+0000 7f292d7fa700 1 -- 172.21.15.94:0/365207387 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f291c093020 con 0x7f29300fcf80 2024-03-24T10:51:09.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.120+0000 7f293707d700 1 -- 172.21.15.94:0/365207387 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f293005f5f0 con 0x7f29300fcf80 2024-03-24T10:51:09.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.121+0000 7f292d7fa700 1 -- 172.21.15.94:0/365207387 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f291c05afa0 con 0x7f29300fcf80 2024-03-24T10:51:09.122 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-03-24T10:51:09.122 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": { 2024-03-24T10:51:09.122 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 2 2024-03-24T10:51:09.122 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:51:09.122 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": { 2024-03-24T10:51:09.122 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 2 2024-03-24T10:51:09.122 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:51:09.122 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": { 2024-03-24T10:51:09.122 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 3, 2024-03-24T10:51:09.122 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 2 2024-03-24T10:51:09.123 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:51:09.123 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-03-24T10:51:09.123 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-24T10:51:09.123 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:51:09.123 INFO:teuthology.orchestra.run.smithi094.stdout: "overall": { 2024-03-24T10:51:09.123 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 7, 2024-03-24T10:51:09.123 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 6 2024-03-24T10:51:09.123 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-03-24T10:51:09.123 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-03-24T10:51:09.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.124+0000 7f293707d700 1 -- 172.21.15.94:0/365207387 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f292006d7e0 msgr2=0x7f292006fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:09.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.124+0000 7f293707d700 1 --2- 172.21.15.94:0/365207387 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f292006d7e0 0x7f292006fc90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2924007fd0 tx=0x7f2924009040 comp rx=0 tx=0).stop 2024-03-24T10:51:09.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.124+0000 7f293707d700 1 -- 172.21.15.94:0/365207387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29300fcf80 msgr2=0x7f293010dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:09.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.124+0000 7f293707d700 1 --2- 172.21.15.94:0/365207387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29300fcf80 0x7f293010dfd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f291c009150 tx=0x7f291c009e30 comp rx=0 tx=0).stop 2024-03-24T10:51:09.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.124+0000 7f293707d700 1 -- 172.21.15.94:0/365207387 shutdown_connections 2024-03-24T10:51:09.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.124+0000 7f293707d700 1 --2- 172.21.15.94:0/365207387 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f292006d7e0 0x7f292006fc90 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:09.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.124+0000 7f293707d700 1 --2- 172.21.15.94:0/365207387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f29300fbcd0 0x7f293010da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:09.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.124+0000 7f293707d700 1 --2- 172.21.15.94:0/365207387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29300fcf80 0x7f293010dfd0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:09.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.124+0000 7f293707d700 1 -- 172.21.15.94:0/365207387 >> 172.21.15.94:0/365207387 conn(0x7f29300f7430 msgr2=0x7f2930065b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:51:09.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.124+0000 7f293707d700 1 -- 172.21.15.94:0/365207387 shutdown_connections 2024-03-24T10:51:09.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.124+0000 7f293707d700 1 -- 172.21.15.94:0/365207387 wait complete. 2024-03-24T10:51:09.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.237+0000 7fadf0bef700 1 -- 172.21.15.94:0/3478325317 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadec0fe070 msgr2=0x7fadec0fe4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:09.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.237+0000 7fadf0bef700 1 --2- 172.21.15.94:0/3478325317 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadec0fe070 0x7fadec0fe4e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fade0004940 tx=0x7fade0009e30 comp rx=0 tx=0).stop 2024-03-24T10:51:09.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.238+0000 7fadf0bef700 1 -- 172.21.15.94:0/3478325317 shutdown_connections 2024-03-24T10:51:09.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.238+0000 7fadf0bef700 1 --2- 172.21.15.94:0/3478325317 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadec0fe070 0x7fadec0fe4e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:09.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.238+0000 7fadf0bef700 1 --2- 172.21.15.94:0/3478325317 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fadec0fcf00 0x7fadec0fd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:09.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.238+0000 7fadf0bef700 1 -- 172.21.15.94:0/3478325317 >> 172.21.15.94:0/3478325317 conn(0x7fadec0f85a0 msgr2=0x7fadec0fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:51:09.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.238+0000 7fadf0bef700 1 -- 172.21.15.94:0/3478325317 shutdown_connections 2024-03-24T10:51:09.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.238+0000 7fadf0bef700 1 -- 172.21.15.94:0/3478325317 wait complete. 2024-03-24T10:51:09.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.238+0000 7fadf0bef700 1 Processor -- start 2024-03-24T10:51:09.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.238+0000 7fadf0bef700 1 -- start start 2024-03-24T10:51:09.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.239+0000 7fadf0bef700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadec0fcf00 0x7fadec10d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:51:09.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.239+0000 7fadf0bef700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fadec0fe070 0x7fadec10df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:51:09.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.239+0000 7fadf0bef700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadec10e510 con 0x7fadec0fcf00 2024-03-24T10:51:09.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.239+0000 7fadf0bef700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadec10ed60 con 0x7fadec0fe070 2024-03-24T10:51:09.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.239+0000 7fadea59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadec0fcf00 0x7fadec10d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:51:09.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.239+0000 7fadea59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadec0fcf00 0x7fadec10d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39868/0 (socket says 172.21.15.94:39868) 2024-03-24T10:51:09.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.239+0000 7fadea59c700 1 -- 172.21.15.94:0/3925232500 learned_addr learned my addr 172.21.15.94:0/3925232500 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:51:09.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.239+0000 7fade9d9b700 1 --2- 172.21.15.94:0/3925232500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fadec0fe070 0x7fadec10df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:51:09.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.239+0000 7fadea59c700 1 -- 172.21.15.94:0/3925232500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fadec0fe070 msgr2=0x7fadec10df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:09.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.239+0000 7fadea59c700 1 --2- 172.21.15.94:0/3925232500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fadec0fe070 0x7fadec10df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:09.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.239+0000 7fadea59c700 1 -- 172.21.15.94:0/3925232500 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade0005040 con 0x7fadec0fcf00 2024-03-24T10:51:09.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.239+0000 7fade9d9b700 1 --2- 172.21.15.94:0/3925232500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fadec0fe070 0x7fadec10df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T10:51:09.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.239+0000 7fadea59c700 1 --2- 172.21.15.94:0/3925232500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadec0fcf00 0x7fadec10d9d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7faddc00eb40 tx=0x7faddc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:51:09.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.240+0000 7faddaffd700 1 -- 172.21.15.94:0/3925232500 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faddc007cc0 con 0x7fadec0fcf00 2024-03-24T10:51:09.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.240+0000 7fadf0bef700 1 -- 172.21.15.94:0/3925232500 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadec10efe0 con 0x7fadec0fcf00 2024-03-24T10:51:09.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.240+0000 7faddaffd700 1 -- 172.21.15.94:0/3925232500 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faddc00f040 con 0x7fadec0fcf00 2024-03-24T10:51:09.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.240+0000 7faddaffd700 1 -- 172.21.15.94:0/3925232500 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faddc009620 con 0x7fadec0fcf00 2024-03-24T10:51:09.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.240+0000 7fadf0bef700 1 -- 172.21.15.94:0/3925232500 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadec10f520 con 0x7fadec0fcf00 2024-03-24T10:51:09.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.241+0000 7fadf0bef700 1 -- 172.21.15.94:0/3925232500 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadec05f5f0 con 0x7fadec0fcf00 2024-03-24T10:51:09.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.242+0000 7faddaffd700 1 -- 172.21.15.94:0/3925232500 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7faddc009780 con 0x7fadec0fcf00 2024-03-24T10:51:09.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.242+0000 7faddaffd700 1 --2- 172.21.15.94:0/3925232500 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fadd406d7e0 0x7fadd406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:51:09.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.242+0000 7faddaffd700 1 -- 172.21.15.94:0/3925232500 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6500+0+0 (secure 0 0 0) 0x7faddc08cbb0 con 0x7fadec0fcf00 2024-03-24T10:51:09.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.242+0000 7fade9d9b700 1 --2- 172.21.15.94:0/3925232500 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fadd406d7e0 0x7fadd406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:51:09.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.243+0000 7fade9d9b700 1 --2- 172.21.15.94:0/3925232500 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fadd406d7e0 0x7fadd406fc90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fade0009b30 tx=0x7fade0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:51:09.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.251+0000 7faddaffd700 1 -- 172.21.15.94:0/3925232500 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7faddc093080 con 0x7fadec0fcf00 2024-03-24T10:51:09.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:09 smithi094 ceph-mon[93957]: from='client.44193 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:51:09.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:09 smithi094 ceph-mon[93957]: Health check failed: Degraded data redundancy: 9/219 objects degraded (4.110%), 5 pgs degraded (PG_DEGRADED) 2024-03-24T10:51:09.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:09 smithi094 ceph-mon[93957]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:51:09.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:09 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/365207387' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:51:09.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.501+0000 7fadf0bef700 1 -- 172.21.15.94:0/3925232500 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fadec04bed0 con 0x7fadec0fcf00 2024-03-24T10:51:09.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.502+0000 7faddaffd700 1 -- 172.21.15.94:0/3925232500 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1903 (secure 0 0 0) 0x7faddc060070 con 0x7fadec0fcf00 2024-03-24T10:51:09.503 INFO:teuthology.orchestra.run.smithi094.stdout:e18 2024-03-24T10:51:09.503 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T10:51:09.503 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:51:09.503 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-03-24T10:51:09.503 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:51:09.504 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-03-24T10:51:09.504 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-03-24T10:51:09.504 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 18 2024-03-24T10:51:09.504 INFO:teuthology.orchestra.run.smithi094.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-24T10:51:09.504 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-03-24T10:43:39.435738+0000 2024-03-24T10:51:09.504 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-03-24T10:43:58.133901+0000 2024-03-24T10:51:09.504 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-03-24T10:51:09.504 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-03-24T10:51:09.504 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 60 2024-03-24T10:51:09.504 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-03-24T10:51:09.504 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-03-24T10:51:09.504 INFO:teuthology.orchestra.run.smithi094.stdout:max_xattr_size 65536 2024-03-24T10:51:09.505 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-03-24T10:51:09.505 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-03-24T10:51:09.505 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 46 2024-03-24T10:51:09.505 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:51:09.505 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-03-24T10:51:09.505 INFO:teuthology.orchestra.run.smithi094.stdout:in 0 2024-03-24T10:51:09.505 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=24251} 2024-03-24T10:51:09.505 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-03-24T10:51:09.505 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-03-24T10:51:09.505 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-03-24T10:51:09.505 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-03-24T10:51:09.506 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-03-24T10:51:09.506 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-03-24T10:51:09.506 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-03-24T10:51:09.506 INFO:teuthology.orchestra.run.smithi094.stdout:bal_rank_mask -1 2024-03-24T10:51:09.506 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 1 2024-03-24T10:51:09.506 INFO:teuthology.orchestra.run.smithi094.stdout:qdb_cluster leader: 0 members: 2024-03-24T10:51:09.506 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi096.egbmoq{0:24251} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.96:6824/1539752208,v1:172.21.15.96:6825/1539752208] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:51:09.506 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.llefgu{0:14500} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.94:6828/126762980,v1:172.21.15.94:6829/126762980] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:51:09.506 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:51:09.506 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:51:09.506 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-03-24T10:51:09.507 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:51:09.507 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.efiqte{-1:14512} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.94:6826/1636528860,v1:172.21.15.94:6827/1636528860] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:51:09.507 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi096.jtebhh{-1:24261} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.96:6826/1321087930,v1:172.21.15.96:6827/1321087930] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:51:09.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.505+0000 7fadf0bef700 1 -- 172.21.15.94:0/3925232500 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fadd406d7e0 msgr2=0x7fadd406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:09.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.505+0000 7fadf0bef700 1 --2- 172.21.15.94:0/3925232500 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fadd406d7e0 0x7fadd406fc90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fade0009b30 tx=0x7fade0009e30 comp rx=0 tx=0).stop 2024-03-24T10:51:09.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.505+0000 7fadf0bef700 1 -- 172.21.15.94:0/3925232500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadec0fcf00 msgr2=0x7fadec10d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:09.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.505+0000 7fadf0bef700 1 --2- 172.21.15.94:0/3925232500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadec0fcf00 0x7fadec10d9d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7faddc00eb40 tx=0x7faddc00b040 comp rx=0 tx=0).stop 2024-03-24T10:51:09.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.505+0000 7fadf0bef700 1 -- 172.21.15.94:0/3925232500 shutdown_connections 2024-03-24T10:51:09.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.505+0000 7fadf0bef700 1 --2- 172.21.15.94:0/3925232500 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fadd406d7e0 0x7fadd406fc90 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:09.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.505+0000 7fadf0bef700 1 --2- 172.21.15.94:0/3925232500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadec0fcf00 0x7fadec10d9d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:09.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.505+0000 7fadf0bef700 1 --2- 172.21.15.94:0/3925232500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fadec0fe070 0x7fadec10df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:09.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.505+0000 7fadf0bef700 1 -- 172.21.15.94:0/3925232500 >> 172.21.15.94:0/3925232500 conn(0x7fadec0f85a0 msgr2=0x7fadec0f9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:51:09.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.505+0000 7fadf0bef700 1 -- 172.21.15.94:0/3925232500 shutdown_connections 2024-03-24T10:51:09.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.505+0000 7fadf0bef700 1 -- 172.21.15.94:0/3925232500 wait complete. 2024-03-24T10:51:09.509 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 18 2024-03-24T10:51:09.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:09 smithi096 ceph-mon[84258]: from='client.44193 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:51:09.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:09 smithi096 ceph-mon[84258]: Health check failed: Degraded data redundancy: 9/219 objects degraded (4.110%), 5 pgs degraded (PG_DEGRADED) 2024-03-24T10:51:09.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:09 smithi096 ceph-mon[84258]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:51:09.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:09 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/365207387' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:51:09.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.617+0000 7f39346a8700 1 -- 172.21.15.94:0/82923533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f392c066260 msgr2=0x7f392c0ffa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:09.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.617+0000 7f39346a8700 1 --2- 172.21.15.94:0/82923533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f392c066260 0x7f392c0ffa40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3928004940 tx=0x7f3928009e30 comp rx=0 tx=0).stop 2024-03-24T10:51:09.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.617+0000 7f39346a8700 1 -- 172.21.15.94:0/82923533 shutdown_connections 2024-03-24T10:51:09.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.617+0000 7f39346a8700 1 --2- 172.21.15.94:0/82923533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f392c0fff80 0x7f392c102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:09.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.617+0000 7f39346a8700 1 --2- 172.21.15.94:0/82923533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f392c066260 0x7f392c0ffa40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:09.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.617+0000 7f39346a8700 1 -- 172.21.15.94:0/82923533 >> 172.21.15.94:0/82923533 conn(0x7f392c0f7430 msgr2=0x7f392c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:51:09.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.617+0000 7f39346a8700 1 -- 172.21.15.94:0/82923533 shutdown_connections 2024-03-24T10:51:09.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.617+0000 7f39346a8700 1 -- 172.21.15.94:0/82923533 wait complete. 2024-03-24T10:51:09.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.618+0000 7f39346a8700 1 Processor -- start 2024-03-24T10:51:09.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.618+0000 7f39346a8700 1 -- start start 2024-03-24T10:51:09.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.618+0000 7f39346a8700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f392c066260 0x7f392c10da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:51:09.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.618+0000 7f39346a8700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f392c0fff80 0x7f392c10df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:51:09.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.618+0000 7f39346a8700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f392c10e580 con 0x7f392c066260 2024-03-24T10:51:09.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.618+0000 7f39346a8700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f392c10ed80 con 0x7f392c0fff80 2024-03-24T10:51:09.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.618+0000 7f3932444700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f392c066260 0x7f392c10da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:51:09.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.618+0000 7f3932444700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f392c066260 0x7f392c10da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47910/0 (socket says 172.21.15.94:47910) 2024-03-24T10:51:09.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.618+0000 7f3932444700 1 -- 172.21.15.94:0/626079450 learned_addr learned my addr 172.21.15.94:0/626079450 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:51:09.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.618+0000 7f3931c43700 1 --2- 172.21.15.94:0/626079450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f392c0fff80 0x7f392c10df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:51:09.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.618+0000 7f3932444700 1 -- 172.21.15.94:0/626079450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f392c0fff80 msgr2=0x7f392c10df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:09.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.618+0000 7f3932444700 1 --2- 172.21.15.94:0/626079450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f392c0fff80 0x7f392c10df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:09.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.618+0000 7f3932444700 1 -- 172.21.15.94:0/626079450 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3928005040 con 0x7f392c066260 2024-03-24T10:51:09.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.619+0000 7f3932444700 1 --2- 172.21.15.94:0/626079450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f392c066260 0x7f392c10da40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3928000c00 tx=0x7f3928009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:51:09.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.619+0000 7f3922ffd700 1 -- 172.21.15.94:0/626079450 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f392800e030 con 0x7f392c066260 2024-03-24T10:51:09.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.619+0000 7f3922ffd700 1 -- 172.21.15.94:0/626079450 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f392801c400 con 0x7f392c066260 2024-03-24T10:51:09.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.619+0000 7f39346a8700 1 -- 172.21.15.94:0/626079450 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f392c10f000 con 0x7f392c066260 2024-03-24T10:51:09.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.619+0000 7f3922ffd700 1 -- 172.21.15.94:0/626079450 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f392800b650 con 0x7f392c066260 2024-03-24T10:51:09.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.619+0000 7f39346a8700 1 -- 172.21.15.94:0/626079450 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f392c10f540 con 0x7f392c066260 2024-03-24T10:51:09.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.620+0000 7f39346a8700 1 -- 172.21.15.94:0/626079450 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f392c05f5f0 con 0x7f392c066260 2024-03-24T10:51:09.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.620+0000 7f3922ffd700 1 -- 172.21.15.94:0/626079450 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f392801ca90 con 0x7f392c066260 2024-03-24T10:51:09.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.621+0000 7f3922ffd700 1 --2- 172.21.15.94:0/626079450 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f391806d7e0 0x7f391806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:51:09.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.621+0000 7f3922ffd700 1 -- 172.21.15.94:0/626079450 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f392808e8c0 con 0x7f392c066260 2024-03-24T10:51:09.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.621+0000 7f3931c43700 1 --2- 172.21.15.94:0/626079450 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f391806d7e0 0x7f391806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:51:09.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.621+0000 7f3931c43700 1 --2- 172.21.15.94:0/626079450 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f391806d7e0 0x7f391806fc90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f391c004600 tx=0x7f391c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:51:09.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.625+0000 7f3922ffd700 1 -- 172.21.15.94:0/626079450 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f392805bc60 con 0x7f392c066260 2024-03-24T10:51:09.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.849+0000 7f39346a8700 1 -- 172.21.15.94:0/626079450 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f392c10f890 con 0x7f391806d7e0 2024-03-24T10:51:09.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.850+0000 7f3922ffd700 1 -- 172.21.15.94:0/626079450 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f392c10f890 con 0x7f391806d7e0 2024-03-24T10:51:09.851 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-03-24T10:51:09.851 INFO:teuthology.orchestra.run.smithi094.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919", 2024-03-24T10:51:09.851 INFO:teuthology.orchestra.run.smithi094.stdout: "in_progress": true, 2024-03-24T10:51:09.851 INFO:teuthology.orchestra.run.smithi094.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T10:51:09.851 INFO:teuthology.orchestra.run.smithi094.stdout: "services_complete": [ 2024-03-24T10:51:09.851 INFO:teuthology.orchestra.run.smithi094.stdout: "mon", 2024-03-24T10:51:09.852 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-03-24T10:51:09.852 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr" 2024-03-24T10:51:09.852 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-03-24T10:51:09.852 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": "8/21 daemons upgraded", 2024-03-24T10:51:09.852 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "Currently upgrading osd daemons", 2024-03-24T10:51:09.852 INFO:teuthology.orchestra.run.smithi094.stdout: "is_paused": false 2024-03-24T10:51:09.852 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-03-24T10:51:09.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.853+0000 7f39346a8700 1 -- 172.21.15.94:0/626079450 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f391806d7e0 msgr2=0x7f391806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:09.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.853+0000 7f39346a8700 1 --2- 172.21.15.94:0/626079450 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f391806d7e0 0x7f391806fc90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f391c004600 tx=0x7f391c00b040 comp rx=0 tx=0).stop 2024-03-24T10:51:09.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.853+0000 7f39346a8700 1 -- 172.21.15.94:0/626079450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f392c066260 msgr2=0x7f392c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:09.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.853+0000 7f39346a8700 1 --2- 172.21.15.94:0/626079450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f392c066260 0x7f392c10da40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3928000c00 tx=0x7f3928009e30 comp rx=0 tx=0).stop 2024-03-24T10:51:09.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.853+0000 7f39346a8700 1 -- 172.21.15.94:0/626079450 shutdown_connections 2024-03-24T10:51:09.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.853+0000 7f39346a8700 1 --2- 172.21.15.94:0/626079450 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f391806d7e0 0x7f391806fc90 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:09.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.853+0000 7f39346a8700 1 --2- 172.21.15.94:0/626079450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f392c066260 0x7f392c10da40 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:09.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.853+0000 7f39346a8700 1 --2- 172.21.15.94:0/626079450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f392c0fff80 0x7f392c10df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:09.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.853+0000 7f39346a8700 1 -- 172.21.15.94:0/626079450 >> 172.21.15.94:0/626079450 conn(0x7f392c0f7430 msgr2=0x7f392c0f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:51:09.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.853+0000 7f39346a8700 1 -- 172.21.15.94:0/626079450 shutdown_connections 2024-03-24T10:51:09.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.853+0000 7f39346a8700 1 -- 172.21.15.94:0/626079450 wait complete. 2024-03-24T10:51:09.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.955+0000 7f5f3ef95700 1 -- 172.21.15.94:0/1285759973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f380fce40 msgr2=0x7f5f380fd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:09.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.955+0000 7f5f3ef95700 1 --2- 172.21.15.94:0/1285759973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f380fce40 0x7f5f380fd250 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c004940 tx=0x7f5f2c009e30 comp rx=0 tx=0).stop 2024-03-24T10:51:09.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.955+0000 7f5f3ef95700 1 -- 172.21.15.94:0/1285759973 shutdown_connections 2024-03-24T10:51:09.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.955+0000 7f5f3ef95700 1 --2- 172.21.15.94:0/1285759973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f380fe040 0x7f5f380fe4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:09.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.955+0000 7f5f3ef95700 1 --2- 172.21.15.94:0/1285759973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f380fce40 0x7f5f380fd250 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:09.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.955+0000 7f5f3ef95700 1 -- 172.21.15.94:0/1285759973 >> 172.21.15.94:0/1285759973 conn(0x7f5f380f8550 msgr2=0x7f5f380fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:51:09.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.955+0000 7f5f3ef95700 1 -- 172.21.15.94:0/1285759973 shutdown_connections 2024-03-24T10:51:09.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.955+0000 7f5f3ef95700 1 -- 172.21.15.94:0/1285759973 wait complete. 2024-03-24T10:51:09.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.955+0000 7f5f3ef95700 1 Processor -- start 2024-03-24T10:51:09.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.955+0000 7f5f3ef95700 1 -- start start 2024-03-24T10:51:09.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.956+0000 7f5f3ef95700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f380fce40 0x7f5f3810da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:51:09.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.956+0000 7f5f3ef95700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f380fe040 0x7f5f3810dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:51:09.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.956+0000 7f5f3ef95700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3810e5a0 con 0x7f5f380fce40 2024-03-24T10:51:09.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.956+0000 7f5f3ef95700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3810edf0 con 0x7f5f380fe040 2024-03-24T10:51:09.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.956+0000 7f5f3cd31700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f380fce40 0x7f5f3810da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:51:09.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.956+0000 7f5f3cd31700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f380fce40 0x7f5f3810da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47926/0 (socket says 172.21.15.94:47926) 2024-03-24T10:51:09.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.956+0000 7f5f3cd31700 1 -- 172.21.15.94:0/2379603020 learned_addr learned my addr 172.21.15.94:0/2379603020 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:51:09.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.956+0000 7f5f37fff700 1 --2- 172.21.15.94:0/2379603020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f380fe040 0x7f5f3810dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:51:09.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.956+0000 7f5f3cd31700 1 -- 172.21.15.94:0/2379603020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f380fe040 msgr2=0x7f5f3810dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:09.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.956+0000 7f5f3cd31700 1 --2- 172.21.15.94:0/2379603020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f380fe040 0x7f5f3810dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:09.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.956+0000 7f5f3cd31700 1 -- 172.21.15.94:0/2379603020 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f2c005040 con 0x7f5f380fce40 2024-03-24T10:51:09.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.956+0000 7f5f3cd31700 1 --2- 172.21.15.94:0/2379603020 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f380fce40 0x7f5f3810da60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c00b3b0 tx=0x7f5f2c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:51:09.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.956+0000 7f5f357fa700 1 -- 172.21.15.94:0/2379603020 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f2c00e040 con 0x7f5f380fce40 2024-03-24T10:51:09.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.956+0000 7f5f357fa700 1 -- 172.21.15.94:0/2379603020 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5f2c00bcc0 con 0x7f5f380fce40 2024-03-24T10:51:09.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.956+0000 7f5f3ef95700 1 -- 172.21.15.94:0/2379603020 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f3810f070 con 0x7f5f380fce40 2024-03-24T10:51:09.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.957+0000 7f5f357fa700 1 -- 172.21.15.94:0/2379603020 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f2c01c630 con 0x7f5f380fce40 2024-03-24T10:51:09.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.957+0000 7f5f3ef95700 1 -- 172.21.15.94:0/2379603020 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f3810f5b0 con 0x7f5f380fce40 2024-03-24T10:51:09.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.958+0000 7f5f3ef95700 1 -- 172.21.15.94:0/2379603020 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f3805f5f0 con 0x7f5f380fce40 2024-03-24T10:51:09.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.958+0000 7f5f357fa700 1 -- 172.21.15.94:0/2379603020 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f5f2c0139f0 con 0x7f5f380fce40 2024-03-24T10:51:09.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.959+0000 7f5f357fa700 1 --2- 172.21.15.94:0/2379603020 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f5f2806d7e0 0x7f5f2806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:51:09.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.959+0000 7f5f357fa700 1 -- 172.21.15.94:0/2379603020 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f5f2c08dbf0 con 0x7f5f380fce40 2024-03-24T10:51:09.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.959+0000 7f5f37fff700 1 --2- 172.21.15.94:0/2379603020 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f5f2806d7e0 0x7f5f2806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:51:09.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.959+0000 7f5f37fff700 1 --2- 172.21.15.94:0/2379603020 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f5f2806d7e0 0x7f5f2806fc90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5f24004d60 tx=0x7f5f2400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:51:09.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:09.963+0000 7f5f357fa700 1 -- 172.21.15.94:0/2379603020 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5f2c05af90 con 0x7f5f380fce40 2024-03-24T10:51:10.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:10.304+0000 7f5f3ef95700 1 -- 172.21.15.94:0/2379603020 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5f3810f890 con 0x7f5f380fce40 2024-03-24T10:51:10.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:10.304+0000 7f5f357fa700 1 -- 172.21.15.94:0/2379603020 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+940 (secure 0 0 0) 0x7f5f2c05ab20 con 0x7f5f380fce40 2024-03-24T10:51:10.305 INFO:teuthology.orchestra.run.smithi094.stdout:HEALTH_WARN 1 osds down; Degraded data redundancy: 30/219 objects degraded (13.699%), 12 pgs degraded 2024-03-24T10:51:10.305 INFO:teuthology.orchestra.run.smithi094.stdout:[WRN] OSD_DOWN: 1 osds down 2024-03-24T10:51:10.305 INFO:teuthology.orchestra.run.smithi094.stdout: osd.2 (root=default,host=smithi094) is down 2024-03-24T10:51:10.306 INFO:teuthology.orchestra.run.smithi094.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 30/219 objects degraded (13.699%), 12 pgs degraded 2024-03-24T10:51:10.306 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.1 is active+undersized+degraded, acting [1,0] 2024-03-24T10:51:10.306 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.3 is active+undersized+degraded, acting [5,1] 2024-03-24T10:51:10.306 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.7 is active+undersized+degraded, acting [3,4] 2024-03-24T10:51:10.306 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.c is active+undersized+degraded, acting [3,0] 2024-03-24T10:51:10.306 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.d is active+undersized+degraded, acting [1,3] 2024-03-24T10:51:10.306 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.e is active+undersized+degraded, acting [0,3] 2024-03-24T10:51:10.306 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.10 is active+undersized+degraded, acting [1,0] 2024-03-24T10:51:10.306 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.13 is active+undersized+degraded, acting [0,4] 2024-03-24T10:51:10.306 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.16 is active+undersized+degraded, acting [5,3] 2024-03-24T10:51:10.306 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.19 is active+undersized+degraded, acting [0,4] 2024-03-24T10:51:10.306 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.1c is active+undersized+degraded, acting [4,5] 2024-03-24T10:51:10.306 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.1e is active+undersized+degraded, acting [0,5] 2024-03-24T10:51:10.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:10.306+0000 7f5f3ef95700 1 -- 172.21.15.94:0/2379603020 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f5f2806d7e0 msgr2=0x7f5f2806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:10.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:10.306+0000 7f5f3ef95700 1 --2- 172.21.15.94:0/2379603020 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f5f2806d7e0 0x7f5f2806fc90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5f24004d60 tx=0x7f5f2400b040 comp rx=0 tx=0).stop 2024-03-24T10:51:10.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:10.306+0000 7f5f3ef95700 1 -- 172.21.15.94:0/2379603020 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f380fce40 msgr2=0x7f5f3810da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:10.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:10.306+0000 7f5f3ef95700 1 --2- 172.21.15.94:0/2379603020 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f380fce40 0x7f5f3810da60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c00b3b0 tx=0x7f5f2c009e30 comp rx=0 tx=0).stop 2024-03-24T10:51:10.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:10.307+0000 7f5f3ef95700 1 -- 172.21.15.94:0/2379603020 shutdown_connections 2024-03-24T10:51:10.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:10.307+0000 7f5f3ef95700 1 --2- 172.21.15.94:0/2379603020 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f5f2806d7e0 0x7f5f2806fc90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:10.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:10.307+0000 7f5f3ef95700 1 --2- 172.21.15.94:0/2379603020 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f380fce40 0x7f5f3810da60 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:10.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:10.307+0000 7f5f3ef95700 1 --2- 172.21.15.94:0/2379603020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f380fe040 0x7f5f3810dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:10.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:10.307+0000 7f5f3ef95700 1 -- 172.21.15.94:0/2379603020 >> 172.21.15.94:0/2379603020 conn(0x7f5f380f8550 msgr2=0x7f5f380f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:51:10.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:10.307+0000 7f5f3ef95700 1 -- 172.21.15.94:0/2379603020 shutdown_connections 2024-03-24T10:51:10.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:10.307+0000 7f5f3ef95700 1 -- 172.21.15.94:0/2379603020 wait complete. 2024-03-24T10:51:10.450 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:10 smithi094 ceph-mon[93957]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:51:10.450 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:10 smithi094 ceph-mon[93957]: pgmap v92: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 196 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 30/219 objects degraded (13.699%) 2024-03-24T10:51:10.450 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:10 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/3925232500' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T10:51:10.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:10 smithi096 ceph-mon[84258]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:51:10.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:10 smithi096 ceph-mon[84258]: pgmap v92: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 196 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 30/219 objects degraded (13.699%) 2024-03-24T10:51:10.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:10 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/3925232500' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T10:51:11.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:11 smithi094 ceph-mon[93957]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:51:11.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:11 smithi094 ceph-mon[93957]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 7 pgs peering) 2024-03-24T10:51:11.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:11 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/2379603020' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T10:51:11.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:11 smithi096 ceph-mon[84258]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:51:11.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:11 smithi096 ceph-mon[84258]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 7 pgs peering) 2024-03-24T10:51:11.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:11 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/2379603020' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T10:51:12.160 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:11 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-2[113746]: 2024-03-24T10:51:11.782+0000 7f07d3d2c7c0 -1 osd.2 0 read_superblock omap replica is missing. 2024-03-24T10:51:12.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:12 smithi094 ceph-mon[93957]: pgmap v93: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 196 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 30/219 objects degraded (13.699%) 2024-03-24T10:51:12.497 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:12 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-2[113746]: 2024-03-24T10:51:12.223+0000 7f07d3d2c7c0 -1 osd.2 59 log_to_monitors true 2024-03-24T10:51:12.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:12 smithi096 ceph-mon[84258]: pgmap v93: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 196 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 30/219 objects degraded (13.699%) 2024-03-24T10:51:13.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:13 smithi094 ceph-mon[93957]: from='osd.2 [v2:172.21.15.94:6818/3153715097,v1:172.21.15.94:6819/3153715097]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-24T10:51:13.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:13 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:51:13.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:13 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:51:13.498 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:51:13 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-2[113746]: 2024-03-24T10:51:13.183+0000 7f07c5184700 -1 osd.2 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T10:51:13.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:13 smithi096 ceph-mon[84258]: from='osd.2 [v2:172.21.15.94:6818/3153715097,v1:172.21.15.94:6819/3153715097]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-24T10:51:13.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:13 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:51:13.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:13 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:51:14.350 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:14 smithi094 ceph-mon[93957]: from='osd.2 [v2:172.21.15.94:6818/3153715097,v1:172.21.15.94:6819/3153715097]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-24T10:51:14.350 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:14 smithi094 ceph-mon[93957]: osdmap e62: 6 total, 5 up, 6 in 2024-03-24T10:51:14.350 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:14 smithi094 ceph-mon[93957]: from='osd.2 [v2:172.21.15.94:6818/3153715097,v1:172.21.15.94:6819/3153715097]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-03-24T10:51:14.350 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:14 smithi094 ceph-mon[93957]: pgmap v95: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 196 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 2 op/s; 30/219 objects degraded (13.699%) 2024-03-24T10:51:14.350 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:14 smithi094 ceph-mon[93957]: failed to encode map e62 with expected crc 2024-03-24T10:51:14.350 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:14 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:51:14.351 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:14 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:51:14.351 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:14 smithi094 ceph-mon[93957]: failed to encode map e62 with expected crc 2024-03-24T10:51:14.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:14 smithi096 ceph-mon[84258]: from='osd.2 [v2:172.21.15.94:6818/3153715097,v1:172.21.15.94:6819/3153715097]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-24T10:51:14.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:14 smithi096 ceph-mon[84258]: osdmap e62: 6 total, 5 up, 6 in 2024-03-24T10:51:14.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:14 smithi096 ceph-mon[84258]: from='osd.2 [v2:172.21.15.94:6818/3153715097,v1:172.21.15.94:6819/3153715097]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-03-24T10:51:14.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:14 smithi096 ceph-mon[84258]: pgmap v95: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 196 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 2 op/s; 30/219 objects degraded (13.699%) 2024-03-24T10:51:14.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:14 smithi096 ceph-mon[84258]: failed to encode map e62 with expected crc 2024-03-24T10:51:14.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:14 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:51:14.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:14 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:51:14.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:14 smithi096 ceph-mon[84258]: failed to encode map e62 with expected crc 2024-03-24T10:51:15.246 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:15 smithi094 ceph-mon[93957]: failed to encode map e62 with expected crc 2024-03-24T10:51:15.246 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:15 smithi094 ceph-mon[93957]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T10:51:15.246 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:15 smithi094 ceph-mon[93957]: failed to encode map e63 with expected crc 2024-03-24T10:51:15.246 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:15 smithi094 ceph-mon[93957]: failed to encode map e63 with expected crc 2024-03-24T10:51:15.246 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:15 smithi094 ceph-mon[93957]: failed to encode map e63 with expected crc 2024-03-24T10:51:15.246 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:15 smithi094 ceph-mon[93957]: osd.2 [v2:172.21.15.94:6818/3153715097,v1:172.21.15.94:6819/3153715097] boot 2024-03-24T10:51:15.246 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:15 smithi094 ceph-mon[93957]: osdmap e63: 6 total, 6 up, 6 in 2024-03-24T10:51:15.246 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:15 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T10:51:15.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:15 smithi096 ceph-mon[84258]: failed to encode map e62 with expected crc 2024-03-24T10:51:15.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:15 smithi096 ceph-mon[84258]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T10:51:15.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:15 smithi096 ceph-mon[84258]: failed to encode map e63 with expected crc 2024-03-24T10:51:15.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:15 smithi096 ceph-mon[84258]: failed to encode map e63 with expected crc 2024-03-24T10:51:15.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:15 smithi096 ceph-mon[84258]: failed to encode map e63 with expected crc 2024-03-24T10:51:15.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:15 smithi096 ceph-mon[84258]: osd.2 [v2:172.21.15.94:6818/3153715097,v1:172.21.15.94:6819/3153715097] boot 2024-03-24T10:51:15.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:15 smithi096 ceph-mon[84258]: osdmap e63: 6 total, 6 up, 6 in 2024-03-24T10:51:15.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:15 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T10:51:16.427 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:16 smithi094 ceph-mon[93957]: failed to encode map e64 with expected crc 2024-03-24T10:51:16.427 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:16 smithi094 ceph-mon[93957]: osdmap e64: 6 total, 6 up, 6 in 2024-03-24T10:51:16.428 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:16 smithi094 ceph-mon[93957]: failed to encode map e64 with expected crc 2024-03-24T10:51:16.428 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:16 smithi094 ceph-mon[93957]: failed to encode map e64 with expected crc 2024-03-24T10:51:16.428 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:16 smithi094 ceph-mon[93957]: failed to encode map e64 with expected crc 2024-03-24T10:51:16.428 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:16 smithi094 ceph-mon[93957]: failed to encode map e64 with expected crc 2024-03-24T10:51:16.428 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:16 smithi094 ceph-mon[93957]: failed to encode map e64 with expected crc 2024-03-24T10:51:16.428 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:16 smithi094 ceph-mon[93957]: failed to encode map e64 with expected crc 2024-03-24T10:51:16.428 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:16 smithi094 ceph-mon[93957]: failed to encode map e64 with expected crc 2024-03-24T10:51:16.428 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:16 smithi094 ceph-mon[93957]: pgmap v98: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 196 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 30/219 objects degraded (13.699%) 2024-03-24T10:51:16.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:16 smithi096 ceph-mon[84258]: failed to encode map e64 with expected crc 2024-03-24T10:51:16.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:16 smithi096 ceph-mon[84258]: osdmap e64: 6 total, 6 up, 6 in 2024-03-24T10:51:16.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:16 smithi096 ceph-mon[84258]: failed to encode map e64 with expected crc 2024-03-24T10:51:16.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:16 smithi096 ceph-mon[84258]: failed to encode map e64 with expected crc 2024-03-24T10:51:16.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:16 smithi096 ceph-mon[84258]: failed to encode map e64 with expected crc 2024-03-24T10:51:16.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:16 smithi096 ceph-mon[84258]: failed to encode map e64 with expected crc 2024-03-24T10:51:16.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:16 smithi096 ceph-mon[84258]: failed to encode map e64 with expected crc 2024-03-24T10:51:16.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:16 smithi096 ceph-mon[84258]: failed to encode map e64 with expected crc 2024-03-24T10:51:16.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:16 smithi096 ceph-mon[84258]: failed to encode map e64 with expected crc 2024-03-24T10:51:16.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:16 smithi096 ceph-mon[84258]: pgmap v98: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 196 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 30/219 objects degraded (13.699%) 2024-03-24T10:51:18.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:17 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:51:18.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:17 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:51:18.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:17 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:51:18.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:17 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:51:18.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:17 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:51:18.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:17 smithi096 ceph-mon[84258]: Health check update: Degraded data redundancy: 30/219 objects degraded (13.699%), 12 pgs degraded (PG_DEGRADED) 2024-03-24T10:51:18.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:17 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:51:18.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:17 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:51:18.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:17 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:51:18.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:17 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:51:18.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:17 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-24T10:51:18.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:17 smithi096 ceph-mon[84258]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-24T10:51:18.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:17 smithi096 ceph-mon[84258]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-03-24T10:51:18.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:17 smithi096 ceph-mon[84258]: pgmap v99: 65 pgs: 8 active+undersized+degraded, 9 active+undersized, 48 active+clean; 196 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 22/219 objects degraded (10.046%) 2024-03-24T10:51:18.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:17 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:51:18.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:17 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:51:18.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:17 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:51:18.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:17 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:51:18.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:17 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:51:18.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:17 smithi094 ceph-mon[93957]: Health check update: Degraded data redundancy: 30/219 objects degraded (13.699%), 12 pgs degraded (PG_DEGRADED) 2024-03-24T10:51:18.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:17 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:51:18.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:17 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:51:18.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:17 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:51:18.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:17 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:51:18.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:17 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-24T10:51:18.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:17 smithi094 ceph-mon[93957]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-24T10:51:18.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:17 smithi094 ceph-mon[93957]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-03-24T10:51:18.249 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:17 smithi094 ceph-mon[93957]: pgmap v99: 65 pgs: 8 active+undersized+degraded, 9 active+undersized, 48 active+clean; 196 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 22/219 objects degraded (10.046%) 2024-03-24T10:51:19.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:19 smithi094 ceph-mon[93957]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 22/219 objects degraded (10.046%), 8 pgs degraded) 2024-03-24T10:51:19.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:19 smithi094 ceph-mon[93957]: Cluster is now healthy 2024-03-24T10:51:19.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:19 smithi096 ceph-mon[84258]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 22/219 objects degraded (10.046%), 8 pgs degraded) 2024-03-24T10:51:19.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:19 smithi096 ceph-mon[84258]: Cluster is now healthy 2024-03-24T10:51:20.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:20 smithi094 ceph-mon[93957]: pgmap v100: 65 pgs: 65 active+clean; 196 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:51:20.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:20 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:51:20.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:20 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:51:20.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:20 smithi096 ceph-mon[84258]: pgmap v100: 65 pgs: 65 active+clean; 196 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-24T10:51:20.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:20 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:51:20.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:20 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:51:22.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:22 smithi094 ceph-mon[93957]: pgmap v101: 65 pgs: 65 active+clean; 196 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-24T10:51:22.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:22 smithi096 ceph-mon[84258]: pgmap v101: 65 pgs: 65 active+clean; 196 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-24T10:51:24.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:24 smithi094 ceph-mon[93957]: pgmap v102: 65 pgs: 65 active+clean; 196 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 op/s 2024-03-24T10:51:24.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:24 smithi096 ceph-mon[84258]: pgmap v102: 65 pgs: 65 active+clean; 196 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 op/s 2024-03-24T10:51:26.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:26 smithi094 ceph-mon[93957]: pgmap v103: 65 pgs: 65 active+clean; 196 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1012 B/s rd, 0 op/s 2024-03-24T10:51:26.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:26 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:51:26.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:26 smithi096 ceph-mon[84258]: pgmap v103: 65 pgs: 65 active+clean; 196 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1012 B/s rd, 0 op/s 2024-03-24T10:51:26.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:26 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:51:28.627 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:28 smithi096 ceph-mon[84258]: pgmap v104: 65 pgs: 65 active+clean; 196 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T10:51:28.658 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:28 smithi094 ceph-mon[93957]: pgmap v104: 65 pgs: 65 active+clean; 196 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T10:51:30.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:30 smithi094 ceph-mon[93957]: pgmap v105: 65 pgs: 65 active+clean; 196 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:51:30.812 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:30 smithi096 ceph-mon[84258]: pgmap v105: 65 pgs: 65 active+clean; 196 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:51:32.672 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:32 smithi096 ceph-mon[84258]: pgmap v106: 65 pgs: 65 active+clean; 196 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:51:32.672 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:32 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-24T10:51:32.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:32 smithi094 ceph-mon[93957]: pgmap v106: 65 pgs: 65 active+clean; 196 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:51:32.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:32 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-24T10:51:33.627 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:33 smithi096 ceph-mon[84258]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-24T10:51:33.627 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:33 smithi096 ceph-mon[84258]: Upgrade: osd.3 is safe to restart 2024-03-24T10:51:33.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:33 smithi094 ceph-mon[93957]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-24T10:51:33.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:33 smithi094 ceph-mon[93957]: Upgrade: osd.3 is safe to restart 2024-03-24T10:51:34.531 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:34 smithi096 ceph-mon[84258]: pgmap v107: 65 pgs: 65 active+clean; 196 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-24T10:51:34.531 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:34 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:51:34.531 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:34 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-24T10:51:34.531 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:34 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:51:34.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:34 smithi094 ceph-mon[93957]: pgmap v107: 65 pgs: 65 active+clean; 196 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-24T10:51:34.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:34 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:51:34.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:34 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-24T10:51:34.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:34 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:51:35.688 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:35 smithi096 ceph-mon[84258]: Upgrade: Updating osd.3 2024-03-24T10:51:35.688 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:35 smithi096 ceph-mon[84258]: Deploying daemon osd.3 on smithi096 2024-03-24T10:51:35.688 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:35 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:51:35.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:35 smithi094 ceph-mon[93957]: Upgrade: Updating osd.3 2024-03-24T10:51:35.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:35 smithi094 ceph-mon[93957]: Deploying daemon osd.3 on smithi096 2024-03-24T10:51:35.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:35 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:51:36.649 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:36 smithi094 ceph-mon[93957]: pgmap v108: 65 pgs: 65 active+clean; 196 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:51:36.741 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:36 smithi096 ceph-mon[84258]: pgmap v108: 65 pgs: 65 active+clean; 196 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:51:37.038 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:36 smithi096 systemd[1]: Stopping Ceph osd.3 for 386110d4-e9ca-11ee-95cd-87774f69a715... 2024-03-24T10:51:37.325 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:37 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3[43669]: 2024-03-24T10:51:37.037+0000 7ff8d8073700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T10:51:37.325 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:37 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3[43669]: 2024-03-24T10:51:37.037+0000 7ff8d8073700 -1 osd.3 64 *** Got signal Terminated *** 2024-03-24T10:51:37.325 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:37 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3[43669]: 2024-03-24T10:51:37.037+0000 7ff8d8073700 -1 osd.3 64 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T10:51:37.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:37 smithi094 ceph-mon[93957]: osd.3 marked itself down and dead 2024-03-24T10:51:37.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:37 smithi096 ceph-mon[84258]: osd.3 marked itself down and dead 2024-03-24T10:51:38.154 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:37 smithi096 podman[93109]: 2024-03-24 10:51:37.836506536 +0000 UTC m=+0.930700079 container died f4650d0f624167a5430ec81d9323c5c5bc12b4e2763f53b7d23af4ffde4817a3 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-9c4e0c3, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux ) 2024-03-24T10:51:38.575 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:38 smithi096 podman[93109]: 2024-03-24 10:51:38.279605629 +0000 UTC m=+1.373799183 container cleanup f4650d0f624167a5430ec81d9323c5c5bc12b4e2763f53b7d23af4ffde4817a3 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_CLEAN=True, RELEASE=quincy-9c4e0c3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-03-24T10:51:38.575 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:38 smithi096 bash[93109]: ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3 2024-03-24T10:51:38.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:38 smithi096 ceph-mon[84258]: pgmap v109: 65 pgs: 65 active+clean; 196 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:51:38.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:38 smithi096 ceph-mon[84258]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T10:51:38.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:38 smithi096 ceph-mon[84258]: failed to encode map e65 with expected crc 2024-03-24T10:51:38.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:38 smithi096 ceph-mon[84258]: osdmap e65: 6 total, 5 up, 6 in 2024-03-24T10:51:38.658 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:38 smithi094 ceph-mon[93957]: pgmap v109: 65 pgs: 65 active+clean; 196 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:51:38.658 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:38 smithi094 ceph-mon[93957]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T10:51:38.658 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:38 smithi094 ceph-mon[93957]: failed to encode map e65 with expected crc 2024-03-24T10:51:38.658 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:38 smithi094 ceph-mon[93957]: osdmap e65: 6 total, 5 up, 6 in 2024-03-24T10:51:38.900 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:38 smithi096 podman[93127]: 2024-03-24 10:51:38.589072204 +0000 UTC m=+0.754050652 container remove f4650d0f624167a5430ec81d9323c5c5bc12b4e2763f53b7d23af4ffde4817a3 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-9c4e0c3, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-24T10:51:39.209 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:38 smithi096 podman[93221]: 2024-03-24 10:51:38.803803172 +0000 UTC m=+0.017650690 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-24T10:51:39.209 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:38 smithi096 podman[93221]: 2024-03-24 10:51:38.948489915 +0000 UTC m=+0.162337434 container create 65e19550387532daab6005645656612321e476742ae47c2045794dfa1d9b7e0b (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True) 2024-03-24T10:51:39.209 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:39 smithi096 podman[93221]: 2024-03-24 10:51:39.134521148 +0000 UTC m=+0.348368674 container init 65e19550387532daab6005645656612321e476742ae47c2045794dfa1d9b7e0b (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True) 2024-03-24T10:51:39.209 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:39 smithi096 podman[93221]: 2024-03-24 10:51:39.137803828 +0000 UTC m=+0.351651349 container start 65e19550387532daab6005645656612321e476742ae47c2045794dfa1d9b7e0b (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T10:51:39.209 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:39 smithi096 podman[93221]: 2024-03-24 10:51:39.208976479 +0000 UTC m=+0.422824012 container attach 65e19550387532daab6005645656612321e476742ae47c2045794dfa1d9b7e0b (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-24T10:51:39.575 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:39 smithi096 podman[93221]: 2024-03-24 10:51:39.424635053 +0000 UTC m=+0.638482567 container died 65e19550387532daab6005645656612321e476742ae47c2045794dfa1d9b7e0b (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS) 2024-03-24T10:51:39.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:39 smithi096 ceph-mon[84258]: failed to encode map e66 with expected crc 2024-03-24T10:51:39.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:39 smithi096 ceph-mon[84258]: failed to encode map e66 with expected crc 2024-03-24T10:51:39.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:39 smithi096 ceph-mon[84258]: osdmap e66: 6 total, 5 up, 6 in 2024-03-24T10:51:39.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:39 smithi096 ceph-mon[84258]: failed to encode map e66 with expected crc 2024-03-24T10:51:39.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:39 smithi096 ceph-mon[84258]: failed to encode map e66 with expected crc 2024-03-24T10:51:39.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:39 smithi096 ceph-mon[84258]: failed to encode map e66 with expected crc 2024-03-24T10:51:39.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:39 smithi096 ceph-mon[84258]: failed to encode map e66 with expected crc 2024-03-24T10:51:39.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:39 smithi094 ceph-mon[93957]: failed to encode map e66 with expected crc 2024-03-24T10:51:39.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:39 smithi094 ceph-mon[93957]: failed to encode map e66 with expected crc 2024-03-24T10:51:39.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:39 smithi094 ceph-mon[93957]: osdmap e66: 6 total, 5 up, 6 in 2024-03-24T10:51:39.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:39 smithi094 ceph-mon[93957]: failed to encode map e66 with expected crc 2024-03-24T10:51:39.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:39 smithi094 ceph-mon[93957]: failed to encode map e66 with expected crc 2024-03-24T10:51:39.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:39 smithi094 ceph-mon[93957]: failed to encode map e66 with expected crc 2024-03-24T10:51:39.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:39 smithi094 ceph-mon[93957]: failed to encode map e66 with expected crc 2024-03-24T10:51:40.325 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:40 smithi096 podman[93247]: 2024-03-24 10:51:40.012916086 +0000 UTC m=+0.583406205 container remove 65e19550387532daab6005645656612321e476742ae47c2045794dfa1d9b7e0b (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T10:51:40.325 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:40 smithi096 systemd[1]: ceph-386110d4-e9ca-11ee-95cd-87774f69a715@osd.3.service: Deactivated successfully. 2024-03-24T10:51:40.325 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:40 smithi096 systemd[1]: Stopped Ceph osd.3 for 386110d4-e9ca-11ee-95cd-87774f69a715. 2024-03-24T10:51:40.325 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:40 smithi096 systemd[1]: ceph-386110d4-e9ca-11ee-95cd-87774f69a715@osd.3.service: Consumed 1min 14.481s CPU time. 2024-03-24T10:51:40.326 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:40 smithi096 systemd[1]: Starting Ceph osd.3 for 386110d4-e9ca-11ee-95cd-87774f69a715... 2024-03-24T10:51:40.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.421+0000 7fdcc5e34700 1 -- 172.21.15.94:0/1334896855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcc00fcf80 msgr2=0x7fdcc00fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:40.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.421+0000 7fdcc5e34700 1 --2- 172.21.15.94:0/1334896855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcc00fcf80 0x7fdcc00fd3f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4004940 tx=0x7fdcb4009e30 comp rx=0 tx=0).stop 2024-03-24T10:51:40.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.421+0000 7fdcc5e34700 1 -- 172.21.15.94:0/1334896855 shutdown_connections 2024-03-24T10:51:40.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.421+0000 7fdcc5e34700 1 --2- 172.21.15.94:0/1334896855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcc00fcf80 0x7fdcc00fd3f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:40.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.421+0000 7fdcc5e34700 1 --2- 172.21.15.94:0/1334896855 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcc00fbcd0 0x7fdcc00fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:40.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.421+0000 7fdcc5e34700 1 -- 172.21.15.94:0/1334896855 >> 172.21.15.94:0/1334896855 conn(0x7fdcc00f7450 msgr2=0x7fdcc00f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:51:40.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.421+0000 7fdcc5e34700 1 -- 172.21.15.94:0/1334896855 shutdown_connections 2024-03-24T10:51:40.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.421+0000 7fdcc5e34700 1 -- 172.21.15.94:0/1334896855 wait complete. 2024-03-24T10:51:40.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.421+0000 7fdcc5e34700 1 Processor -- start 2024-03-24T10:51:40.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.422+0000 7fdcc5e34700 1 -- start start 2024-03-24T10:51:40.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.422+0000 7fdcc5e34700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcc00fbcd0 0x7fdcc010d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:51:40.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.422+0000 7fdcc5e34700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcc00fcf80 0x7fdcc010ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:51:40.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.422+0000 7fdcc5e34700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc010e3f0 con 0x7fdcc00fcf80 2024-03-24T10:51:40.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.422+0000 7fdcc5e34700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc010e560 con 0x7fdcc00fbcd0 2024-03-24T10:51:40.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.422+0000 7fdcbeffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcc00fcf80 0x7fdcc010ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:51:40.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.422+0000 7fdcbeffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcc00fcf80 0x7fdcc010ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56492/0 (socket says 172.21.15.94:56492) 2024-03-24T10:51:40.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.422+0000 7fdcbeffd700 1 -- 172.21.15.94:0/3668333522 learned_addr learned my addr 172.21.15.94:0/3668333522 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:51:40.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.422+0000 7fdcbf7fe700 1 --2- 172.21.15.94:0/3668333522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcc00fbcd0 0x7fdcc010d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:51:40.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.422+0000 7fdcbeffd700 1 -- 172.21.15.94:0/3668333522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcc00fbcd0 msgr2=0x7fdcc010d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:40.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.422+0000 7fdcbeffd700 1 --2- 172.21.15.94:0/3668333522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcc00fbcd0 0x7fdcc010d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:40.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.422+0000 7fdcbeffd700 1 -- 172.21.15.94:0/3668333522 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb4005040 con 0x7fdcc00fcf80 2024-03-24T10:51:40.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.423+0000 7fdcbeffd700 1 --2- 172.21.15.94:0/3668333522 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcc00fcf80 0x7fdcc010ddf0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4013870 tx=0x7fdcb4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:51:40.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.423+0000 7fdca7fff700 1 -- 172.21.15.94:0/3668333522 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdcb4021070 con 0x7fdcc00fcf80 2024-03-24T10:51:40.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.423+0000 7fdcc5e34700 1 -- 172.21.15.94:0/3668333522 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcc0112f60 con 0x7fdcc00fcf80 2024-03-24T10:51:40.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.423+0000 7fdca7fff700 1 -- 172.21.15.94:0/3668333522 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdcb4009430 con 0x7fdcc00fcf80 2024-03-24T10:51:40.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.423+0000 7fdca7fff700 1 -- 172.21.15.94:0/3668333522 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdcb400e070 con 0x7fdcc00fcf80 2024-03-24T10:51:40.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.423+0000 7fdcc5e34700 1 -- 172.21.15.94:0/3668333522 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcc0113470 con 0x7fdcc00fcf80 2024-03-24T10:51:40.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.424+0000 7fdcc5e34700 1 -- 172.21.15.94:0/3668333522 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcc0101b10 con 0x7fdcc00fcf80 2024-03-24T10:51:40.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.424+0000 7fdca7fff700 1 -- 172.21.15.94:0/3668333522 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fdcb401c750 con 0x7fdcc00fcf80 2024-03-24T10:51:40.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.425+0000 7fdca7fff700 1 --2- 172.21.15.94:0/3668333522 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fdca806d780 0x7fdca806fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:51:40.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.425+0000 7fdca7fff700 1 -- 172.21.15.94:0/3668333522 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fdcb408e020 con 0x7fdcc00fcf80 2024-03-24T10:51:40.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.425+0000 7fdcbf7fe700 1 --2- 172.21.15.94:0/3668333522 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fdca806d780 0x7fdca806fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:51:40.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.425+0000 7fdcbf7fe700 1 --2- 172.21.15.94:0/3668333522 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fdca806d780 0x7fdca806fc30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0009b80 tx=0x7fdcb000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:51:40.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.430+0000 7fdca7fff700 1 -- 172.21.15.94:0/3668333522 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fdcb405b3c0 con 0x7fdcc00fcf80 2024-03-24T10:51:40.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.643+0000 7fdcc5e34700 1 -- 172.21.15.94:0/3668333522 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdcc002b450 con 0x7fdca806d780 2024-03-24T10:51:40.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.644+0000 7fdca7fff700 1 -- 172.21.15.94:0/3668333522 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fdcc002b450 con 0x7fdca806d780 2024-03-24T10:51:40.644 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:40 smithi094 ceph-mon[93957]: pgmap v112: 65 pgs: 20 peering, 10 stale+active+clean, 35 active+clean; 196 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-03-24T10:51:40.644 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:40 smithi094 ceph-mon[93957]: Health check failed: Reduced data availability: 6 pgs peering (PG_AVAILABILITY) 2024-03-24T10:51:40.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.646+0000 7fdcc5e34700 1 -- 172.21.15.94:0/3668333522 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fdca806d780 msgr2=0x7fdca806fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:40.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.646+0000 7fdcc5e34700 1 --2- 172.21.15.94:0/3668333522 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fdca806d780 0x7fdca806fc30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0009b80 tx=0x7fdcb000b040 comp rx=0 tx=0).stop 2024-03-24T10:51:40.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.647+0000 7fdcc5e34700 1 -- 172.21.15.94:0/3668333522 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcc00fcf80 msgr2=0x7fdcc010ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:40.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.647+0000 7fdcc5e34700 1 --2- 172.21.15.94:0/3668333522 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcc00fcf80 0x7fdcc010ddf0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4013870 tx=0x7fdcb4009e30 comp rx=0 tx=0).stop 2024-03-24T10:51:40.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.647+0000 7fdcc5e34700 1 -- 172.21.15.94:0/3668333522 shutdown_connections 2024-03-24T10:51:40.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.647+0000 7fdcc5e34700 1 --2- 172.21.15.94:0/3668333522 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fdca806d780 0x7fdca806fc30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:40.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.647+0000 7fdcc5e34700 1 --2- 172.21.15.94:0/3668333522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcc00fbcd0 0x7fdcc010d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:40.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.647+0000 7fdcc5e34700 1 --2- 172.21.15.94:0/3668333522 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcc00fcf80 0x7fdcc010ddf0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:40.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.647+0000 7fdcc5e34700 1 -- 172.21.15.94:0/3668333522 >> 172.21.15.94:0/3668333522 conn(0x7fdcc00f7450 msgr2=0x7fdcc00f8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:51:40.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.647+0000 7fdcc5e34700 1 -- 172.21.15.94:0/3668333522 shutdown_connections 2024-03-24T10:51:40.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.647+0000 7fdcc5e34700 1 -- 172.21.15.94:0/3668333522 wait complete. 2024-03-24T10:51:40.657 INFO:teuthology.orchestra.run.smithi094.stdout:true 2024-03-24T10:51:40.682 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:40 smithi096 ceph-mon[84258]: pgmap v112: 65 pgs: 20 peering, 10 stale+active+clean, 35 active+clean; 196 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-03-24T10:51:40.682 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:40 smithi096 ceph-mon[84258]: Health check failed: Reduced data availability: 6 pgs peering (PG_AVAILABILITY) 2024-03-24T10:51:40.683 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:40 smithi096 podman[93352]: 2024-03-24 10:51:40.447412583 +0000 UTC m=+0.112264024 container create b514e1e2494a050a6af890401595344bf5000ff18f41bc51de5a6e7fd083bc48 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3-activate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2) 2024-03-24T10:51:40.683 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:40 smithi096 podman[93352]: 2024-03-24 10:51:40.353266849 +0000 UTC m=+0.018118291 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-24T10:51:40.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.748+0000 7fb07dc1b700 1 -- 172.21.15.94:0/3711228699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0780fe050 msgr2=0x7fb0780fe4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:40.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.748+0000 7fb07dc1b700 1 --2- 172.21.15.94:0/3711228699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0780fe050 0x7fb0780fe4c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb06c004940 tx=0x7fb06c009e30 comp rx=0 tx=0).stop 2024-03-24T10:51:40.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.748+0000 7fb07dc1b700 1 -- 172.21.15.94:0/3711228699 shutdown_connections 2024-03-24T10:51:40.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.748+0000 7fb07dc1b700 1 --2- 172.21.15.94:0/3711228699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0780fe050 0x7fb0780fe4c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:40.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.748+0000 7fb07dc1b700 1 --2- 172.21.15.94:0/3711228699 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0780fce50 0x7fb0780fd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:40.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.748+0000 7fb07dc1b700 1 -- 172.21.15.94:0/3711228699 >> 172.21.15.94:0/3711228699 conn(0x7fb0780f8580 msgr2=0x7fb0780fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:51:40.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.748+0000 7fb07dc1b700 1 -- 172.21.15.94:0/3711228699 shutdown_connections 2024-03-24T10:51:40.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.748+0000 7fb07dc1b700 1 -- 172.21.15.94:0/3711228699 wait complete. 2024-03-24T10:51:40.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.749+0000 7fb07dc1b700 1 Processor -- start 2024-03-24T10:51:40.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.749+0000 7fb07dc1b700 1 -- start start 2024-03-24T10:51:40.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.749+0000 7fb07dc1b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0780fce50 0x7fb078071c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:51:40.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.749+0000 7fb07dc1b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0780fe050 0x7fb078070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:51:40.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.749+0000 7fb07dc1b700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb078072260 con 0x7fb0780fe050 2024-03-24T10:51:40.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.749+0000 7fb07dc1b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb078070850 con 0x7fb0780fce50 2024-03-24T10:51:40.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.749+0000 7fb076ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0780fe050 0x7fb078070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:51:40.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.749+0000 7fb076ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0780fe050 0x7fb078070310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56500/0 (socket says 172.21.15.94:56500) 2024-03-24T10:51:40.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.749+0000 7fb076ffd700 1 -- 172.21.15.94:0/3297870118 learned_addr learned my addr 172.21.15.94:0/3297870118 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:51:40.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.749+0000 7fb0777fe700 1 --2- 172.21.15.94:0/3297870118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0780fce50 0x7fb078071c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:51:40.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.750+0000 7fb076ffd700 1 -- 172.21.15.94:0/3297870118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0780fce50 msgr2=0x7fb078071c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:40.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.750+0000 7fb076ffd700 1 --2- 172.21.15.94:0/3297870118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0780fce50 0x7fb078071c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:40.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.750+0000 7fb076ffd700 1 -- 172.21.15.94:0/3297870118 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb06c005040 con 0x7fb0780fe050 2024-03-24T10:51:40.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.750+0000 7fb076ffd700 1 --2- 172.21.15.94:0/3297870118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0780fe050 0x7fb078070310 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb06c009150 tx=0x7fb06c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:51:40.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.750+0000 7fb05ffff700 1 -- 172.21.15.94:0/3297870118 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb06c00e040 con 0x7fb0780fe050 2024-03-24T10:51:40.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.750+0000 7fb05ffff700 1 -- 172.21.15.94:0/3297870118 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb06c00bbf0 con 0x7fb0780fe050 2024-03-24T10:51:40.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.750+0000 7fb05ffff700 1 -- 172.21.15.94:0/3297870118 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb06c01c630 con 0x7fb0780fe050 2024-03-24T10:51:40.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.750+0000 7fb07dc1b700 1 -- 172.21.15.94:0/3297870118 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb078070aa0 con 0x7fb0780fe050 2024-03-24T10:51:40.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.750+0000 7fb07dc1b700 1 -- 172.21.15.94:0/3297870118 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb078071060 con 0x7fb0780fe050 2024-03-24T10:51:40.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.751+0000 7fb07dc1b700 1 -- 172.21.15.94:0/3297870118 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb07804c710 con 0x7fb0780fe050 2024-03-24T10:51:40.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.751+0000 7fb05ffff700 1 -- 172.21.15.94:0/3297870118 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fb06c00bd60 con 0x7fb0780fe050 2024-03-24T10:51:40.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.752+0000 7fb05ffff700 1 --2- 172.21.15.94:0/3297870118 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fb06006d7e0 0x7fb06006fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:51:40.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.752+0000 7fb05ffff700 1 -- 172.21.15.94:0/3297870118 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fb06c08e240 con 0x7fb0780fe050 2024-03-24T10:51:40.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.752+0000 7fb0777fe700 1 --2- 172.21.15.94:0/3297870118 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fb06006d7e0 0x7fb06006fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:51:40.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.752+0000 7fb0777fe700 1 --2- 172.21.15.94:0/3297870118 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fb06006d7e0 0x7fb06006fc90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb0780fdbc0 tx=0x7fb06800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:51:40.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.756+0000 7fb05ffff700 1 -- 172.21.15.94:0/3297870118 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb06c05b5e0 con 0x7fb0780fe050 2024-03-24T10:51:40.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.968+0000 7fb07dc1b700 1 -- 172.21.15.94:0/3297870118 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb078102310 con 0x7fb06006d7e0 2024-03-24T10:51:40.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.969+0000 7fb05ffff700 1 -- 172.21.15.94:0/3297870118 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fb078102310 con 0x7fb06006d7e0 2024-03-24T10:51:40.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.971+0000 7fb07dc1b700 1 -- 172.21.15.94:0/3297870118 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fb06006d7e0 msgr2=0x7fb06006fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:40.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.971+0000 7fb07dc1b700 1 --2- 172.21.15.94:0/3297870118 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fb06006d7e0 0x7fb06006fc90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb0780fdbc0 tx=0x7fb06800b040 comp rx=0 tx=0).stop 2024-03-24T10:51:40.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.971+0000 7fb07dc1b700 1 -- 172.21.15.94:0/3297870118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0780fe050 msgr2=0x7fb078070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:40.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.971+0000 7fb07dc1b700 1 --2- 172.21.15.94:0/3297870118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0780fe050 0x7fb078070310 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb06c009150 tx=0x7fb06c009e30 comp rx=0 tx=0).stop 2024-03-24T10:51:40.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.971+0000 7fb07dc1b700 1 -- 172.21.15.94:0/3297870118 shutdown_connections 2024-03-24T10:51:40.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.971+0000 7fb07dc1b700 1 --2- 172.21.15.94:0/3297870118 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fb06006d7e0 0x7fb06006fc90 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:40.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.971+0000 7fb07dc1b700 1 --2- 172.21.15.94:0/3297870118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0780fce50 0x7fb078071c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:40.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.971+0000 7fb07dc1b700 1 --2- 172.21.15.94:0/3297870118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0780fe050 0x7fb078070310 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:40.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.971+0000 7fb07dc1b700 1 -- 172.21.15.94:0/3297870118 >> 172.21.15.94:0/3297870118 conn(0x7fb0780f8580 msgr2=0x7fb0780f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:51:40.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.972+0000 7fb07dc1b700 1 -- 172.21.15.94:0/3297870118 shutdown_connections 2024-03-24T10:51:40.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:40.972+0000 7fb07dc1b700 1 -- 172.21.15.94:0/3297870118 wait complete. 2024-03-24T10:51:41.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.072+0000 7fddd85a7700 1 -- 172.21.15.94:0/443334415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddd00fbcd0 msgr2=0x7fddd00fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:41.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.072+0000 7fddd85a7700 1 --2- 172.21.15.94:0/443334415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddd00fbcd0 0x7fddd00fc0e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fddc0004940 tx=0x7fddc0009e30 comp rx=0 tx=0).stop 2024-03-24T10:51:41.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.072+0000 7fddd85a7700 1 -- 172.21.15.94:0/443334415 shutdown_connections 2024-03-24T10:51:41.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.072+0000 7fddd85a7700 1 --2- 172.21.15.94:0/443334415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddd00fcf80 0x7fddd00fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:41.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.072+0000 7fddd85a7700 1 --2- 172.21.15.94:0/443334415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddd00fbcd0 0x7fddd00fc0e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:41.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.072+0000 7fddd85a7700 1 -- 172.21.15.94:0/443334415 >> 172.21.15.94:0/443334415 conn(0x7fddd00f7430 msgr2=0x7fddd00f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:51:41.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.072+0000 7fddd85a7700 1 -- 172.21.15.94:0/443334415 shutdown_connections 2024-03-24T10:51:41.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.072+0000 7fddd85a7700 1 -- 172.21.15.94:0/443334415 wait complete. 2024-03-24T10:51:41.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.073+0000 7fddd85a7700 1 Processor -- start 2024-03-24T10:51:41.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.073+0000 7fddd85a7700 1 -- start start 2024-03-24T10:51:41.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.073+0000 7fddd85a7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddd00fbcd0 0x7fddd010c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:51:41.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.073+0000 7fddd85a7700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddd00fcf80 0x7fddd010cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:51:41.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.073+0000 7fddd85a7700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd010d3b0 con 0x7fddd00fcf80 2024-03-24T10:51:41.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.073+0000 7fddd85a7700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd010dc00 con 0x7fddd00fbcd0 2024-03-24T10:51:41.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.073+0000 7fddd5b42700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddd00fcf80 0x7fddd010cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:51:41.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.073+0000 7fddd5b42700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddd00fcf80 0x7fddd010cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56522/0 (socket says 172.21.15.94:56522) 2024-03-24T10:51:41.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.073+0000 7fddd5b42700 1 -- 172.21.15.94:0/4145004982 learned_addr learned my addr 172.21.15.94:0/4145004982 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:51:41.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.073+0000 7fddd6343700 1 --2- 172.21.15.94:0/4145004982 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddd00fbcd0 0x7fddd010c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:51:41.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.073+0000 7fddd5b42700 1 -- 172.21.15.94:0/4145004982 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddd00fbcd0 msgr2=0x7fddd010c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:41.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.073+0000 7fddd5b42700 1 --2- 172.21.15.94:0/4145004982 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddd00fbcd0 0x7fddd010c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:41.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.073+0000 7fddd5b42700 1 -- 172.21.15.94:0/4145004982 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc0005040 con 0x7fddd00fcf80 2024-03-24T10:51:41.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.073+0000 7fddd5b42700 1 --2- 172.21.15.94:0/4145004982 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddd00fcf80 0x7fddd010cdb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fddcc00eb40 tx=0x7fddcc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:51:41.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.074+0000 7fddc6ffd700 1 -- 172.21.15.94:0/4145004982 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fddcc007cc0 con 0x7fddd00fcf80 2024-03-24T10:51:41.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.074+0000 7fddc6ffd700 1 -- 172.21.15.94:0/4145004982 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fddcc00f040 con 0x7fddd00fcf80 2024-03-24T10:51:41.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.074+0000 7fddd85a7700 1 -- 172.21.15.94:0/4145004982 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddd010de80 con 0x7fddd00fcf80 2024-03-24T10:51:41.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.074+0000 7fddc6ffd700 1 -- 172.21.15.94:0/4145004982 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fddcc00ca10 con 0x7fddd00fcf80 2024-03-24T10:51:41.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.074+0000 7fddd85a7700 1 -- 172.21.15.94:0/4145004982 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddd010e3c0 con 0x7fddd00fcf80 2024-03-24T10:51:41.076 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:40 smithi096 podman[93352]: 2024-03-24 10:51:40.683378872 +0000 UTC m=+0.348230325 container init b514e1e2494a050a6af890401595344bf5000ff18f41bc51de5a6e7fd083bc48 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3-activate, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T10:51:41.076 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:40 smithi096 podman[93352]: 2024-03-24 10:51:40.686798051 +0000 UTC m=+0.351649498 container start b514e1e2494a050a6af890401595344bf5000ff18f41bc51de5a6e7fd083bc48 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-24T10:51:41.076 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:40 smithi096 podman[93352]: 2024-03-24 10:51:40.766392834 +0000 UTC m=+0.431244290 container attach b514e1e2494a050a6af890401595344bf5000ff18f41bc51de5a6e7fd083bc48 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-24T10:51:41.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.075+0000 7fddd85a7700 1 -- 172.21.15.94:0/4145004982 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddd005f5f0 con 0x7fddd00fcf80 2024-03-24T10:51:41.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.075+0000 7fddc6ffd700 1 -- 172.21.15.94:0/4145004982 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fddcc00cc80 con 0x7fddd00fcf80 2024-03-24T10:51:41.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.076+0000 7fddc6ffd700 1 --2- 172.21.15.94:0/4145004982 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fddbc06d7e0 0x7fddbc06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:51:41.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.076+0000 7fddd6343700 1 --2- 172.21.15.94:0/4145004982 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fddbc06d7e0 0x7fddbc06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:51:41.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.076+0000 7fddc6ffd700 1 -- 172.21.15.94:0/4145004982 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fddcc08cac0 con 0x7fddd00fcf80 2024-03-24T10:51:41.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.076+0000 7fddd6343700 1 --2- 172.21.15.94:0/4145004982 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fddbc06d7e0 0x7fddbc06fc90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fddc0009b30 tx=0x7fddc0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:51:41.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.080+0000 7fddc6ffd700 1 -- 172.21.15.94:0/4145004982 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fddcc059e90 con 0x7fddd00fcf80 2024-03-24T10:51:41.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.286+0000 7fddd85a7700 1 -- 172.21.15.94:0/4145004982 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fddd010e5e0 con 0x7fddbc06d7e0 2024-03-24T10:51:41.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.296+0000 7fddc6ffd700 1 -- 172.21.15.94:0/4145004982 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fddd010e5e0 con 0x7fddbc06d7e0 2024-03-24T10:51:41.295 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T10:51:41.295 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.smithi094 smithi094 *:9093,9094 running (4m) 28s ago 13m 24.5M - 0.25.0 c8568f914cd2 5c583af60554 2024-03-24T10:51:41.296 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi094 smithi094 running (2m) 28s ago 14m 6651k - 19.0.0-1855-g21b3bd67 553463a90fd5 2b0897df571d 2024-03-24T10:51:41.296 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi096 smithi096 running (2m) 2m ago 12m 6669k - 19.0.0-1855-g21b3bd67 553463a90fd5 1759a9ff2805 2024-03-24T10:51:41.296 INFO:teuthology.orchestra.run.smithi094.stdout:grafana.smithi094 smithi094 *:3000 running (4m) 28s ago 12m 78.5M - 9.4.7 954c08fa6188 d3d425bf1b7c 2024-03-24T10:51:41.296 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.efiqte smithi094 running (7m) 28s ago 7m 13.7M - 17.2.7-726-g9c4e0c34 837124da804f f0df0a056c21 2024-03-24T10:51:41.296 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.llefgu smithi094 running (7m) 28s ago 7m 109M - 17.2.7-726-g9c4e0c34 837124da804f 52a9771dc8e0 2024-03-24T10:51:41.296 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi096.egbmoq smithi096 running (7m) 2m ago 7m 154M - 17.2.7-726-g9c4e0c34 837124da804f 1cfc930dbab0 2024-03-24T10:51:41.296 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi096.jtebhh smithi096 running (7m) 2m ago 7m 16.3M - 17.2.7-726-g9c4e0c34 837124da804f 4ff3451a6f6e 2024-03-24T10:51:41.296 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi094.yaethu smithi094 *:8443,9283,8765 running (4m) 28s ago 15m 500M - 19.0.0-1855-g21b3bd67 553463a90fd5 d3adab76b18f 2024-03-24T10:51:41.296 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi096.gfonnu smithi096 *:8443,9283,8765 running (3m) 2m ago 12m 448M - 19.0.0-1855-g21b3bd67 553463a90fd5 22937a059eb8 2024-03-24T10:51:41.296 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi094 smithi094 running (3m) 28s ago 15m 48.8M 2048M 19.0.0-1855-g21b3bd67 553463a90fd5 c779d611e29a 2024-03-24T10:51:41.296 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi096 smithi096 running (2m) 2m ago 12m 31.2M 2048M 19.0.0-1855-g21b3bd67 553463a90fd5 ed9bd0bd57f6 2024-03-24T10:51:41.296 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi094 smithi094 *:9100 running (4m) 28s ago 14m 18.7M - 1.5.0 0da6a335fe13 ca67b53c64e2 2024-03-24T10:51:41.296 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi096 smithi096 *:9100 running (4m) 2m ago 12m 17.0M - 1.5.0 0da6a335fe13 65137a7a7549 2024-03-24T10:51:41.296 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (105s) 28s ago 11m 100M 1557M 19.0.0-1855-g21b3bd67 553463a90fd5 ec57ede0c9e4 2024-03-24T10:51:41.296 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi094 running (70s) 28s ago 11m 106M 1557M 19.0.0-1855-g21b3bd67 553463a90fd5 c5b8b2bbf8b5 2024-03-24T10:51:41.296 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi094 running (34s) 28s ago 10m 14.9M 1557M 19.0.0-1855-g21b3bd67 553463a90fd5 c961dcdc746f 2024-03-24T10:51:41.296 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi096 running (10m) 2m ago 10m 285M 2581M 17.2.7-726-g9c4e0c34 837124da804f f4650d0f6241 2024-03-24T10:51:41.297 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi096 running (9m) 2m ago 9m 258M 2581M 17.2.7-726-g9c4e0c34 837124da804f a63d7c9e705f 2024-03-24T10:51:41.297 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi096 running (9m) 2m ago 9m 243M 2581M 17.2.7-726-g9c4e0c34 837124da804f f56fad2959c3 2024-03-24T10:51:41.297 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.smithi094 smithi094 *:9095 running (3m) 28s ago 12m 55.8M - 2.43.0 a07b618ecd1d 6ce091a0c326 2024-03-24T10:51:41.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.298+0000 7fddd85a7700 1 -- 172.21.15.94:0/4145004982 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fddbc06d7e0 msgr2=0x7fddbc06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:41.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.298+0000 7fddd85a7700 1 --2- 172.21.15.94:0/4145004982 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fddbc06d7e0 0x7fddbc06fc90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fddc0009b30 tx=0x7fddc0009e30 comp rx=0 tx=0).stop 2024-03-24T10:51:41.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.298+0000 7fddd85a7700 1 -- 172.21.15.94:0/4145004982 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddd00fcf80 msgr2=0x7fddd010cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:41.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.298+0000 7fddd85a7700 1 --2- 172.21.15.94:0/4145004982 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddd00fcf80 0x7fddd010cdb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fddcc00eb40 tx=0x7fddcc00b040 comp rx=0 tx=0).stop 2024-03-24T10:51:41.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.298+0000 7fddd85a7700 1 -- 172.21.15.94:0/4145004982 shutdown_connections 2024-03-24T10:51:41.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.298+0000 7fddd85a7700 1 --2- 172.21.15.94:0/4145004982 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fddbc06d7e0 0x7fddbc06fc90 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:41.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.298+0000 7fddd85a7700 1 --2- 172.21.15.94:0/4145004982 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddd00fbcd0 0x7fddd010c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:41.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.298+0000 7fddd85a7700 1 --2- 172.21.15.94:0/4145004982 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddd00fcf80 0x7fddd010cdb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:41.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.298+0000 7fddd85a7700 1 -- 172.21.15.94:0/4145004982 >> 172.21.15.94:0/4145004982 conn(0x7fddd00f7430 msgr2=0x7fddd00f8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:51:41.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.298+0000 7fddd85a7700 1 -- 172.21.15.94:0/4145004982 shutdown_connections 2024-03-24T10:51:41.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.298+0000 7fddd85a7700 1 -- 172.21.15.94:0/4145004982 wait complete. 2024-03-24T10:51:41.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.398+0000 7fdebb562700 1 -- 172.21.15.94:0/275499188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdeb40fce80 msgr2=0x7fdeb40fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:41.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.398+0000 7fdebb562700 1 --2- 172.21.15.94:0/275499188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdeb40fce80 0x7fdeb40fd290 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0004940 tx=0x7fdeb0009e30 comp rx=0 tx=0).stop 2024-03-24T10:51:41.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.398+0000 7fdebb562700 1 -- 172.21.15.94:0/275499188 shutdown_connections 2024-03-24T10:51:41.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.398+0000 7fdebb562700 1 --2- 172.21.15.94:0/275499188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdeb40fdff0 0x7fdeb40fe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:41.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.398+0000 7fdebb562700 1 --2- 172.21.15.94:0/275499188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdeb40fce80 0x7fdeb40fd290 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:41.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.398+0000 7fdebb562700 1 -- 172.21.15.94:0/275499188 >> 172.21.15.94:0/275499188 conn(0x7fdeb40f8560 msgr2=0x7fdeb40fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:51:41.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.398+0000 7fdebb562700 1 -- 172.21.15.94:0/275499188 shutdown_connections 2024-03-24T10:51:41.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.398+0000 7fdebb562700 1 -- 172.21.15.94:0/275499188 wait complete. 2024-03-24T10:51:41.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.398+0000 7fdebb562700 1 Processor -- start 2024-03-24T10:51:41.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.399+0000 7fdebb562700 1 -- start start 2024-03-24T10:51:41.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.399+0000 7fdebb562700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdeb40fce80 0x7fdeb410d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:51:41.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.399+0000 7fdebb562700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdeb40fdff0 0x7fdeb410def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:51:41.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.399+0000 7fdebb562700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb410e4f0 con 0x7fdeb40fce80 2024-03-24T10:51:41.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.399+0000 7fdebb562700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb410ed40 con 0x7fdeb40fdff0 2024-03-24T10:51:41.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.399+0000 7fdeb92fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdeb40fce80 0x7fdeb410d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:51:41.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.399+0000 7fdeb92fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdeb40fce80 0x7fdeb410d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56546/0 (socket says 172.21.15.94:56546) 2024-03-24T10:51:41.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.399+0000 7fdeb92fe700 1 -- 172.21.15.94:0/2484975341 learned_addr learned my addr 172.21.15.94:0/2484975341 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:51:41.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.399+0000 7fdeb8afd700 1 --2- 172.21.15.94:0/2484975341 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdeb40fdff0 0x7fdeb410def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:51:41.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.399+0000 7fdeb92fe700 1 -- 172.21.15.94:0/2484975341 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdeb40fdff0 msgr2=0x7fdeb410def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:41.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.399+0000 7fdeb92fe700 1 --2- 172.21.15.94:0/2484975341 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdeb40fdff0 0x7fdeb410def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:41.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.399+0000 7fdeb92fe700 1 -- 172.21.15.94:0/2484975341 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeb0005040 con 0x7fdeb40fce80 2024-03-24T10:51:41.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.399+0000 7fdeb92fe700 1 --2- 172.21.15.94:0/2484975341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdeb40fce80 0x7fdeb410d9b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0013870 tx=0x7fdeb0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:51:41.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.399+0000 7fdea9ffb700 1 -- 172.21.15.94:0/2484975341 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdeb000e070 con 0x7fdeb40fce80 2024-03-24T10:51:41.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.399+0000 7fdebb562700 1 -- 172.21.15.94:0/2484975341 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdeb410efc0 con 0x7fdeb40fce80 2024-03-24T10:51:41.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.399+0000 7fdea9ffb700 1 -- 172.21.15.94:0/2484975341 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdeb0013de0 con 0x7fdeb40fce80 2024-03-24T10:51:41.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.399+0000 7fdea9ffb700 1 -- 172.21.15.94:0/2484975341 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdeb001cea0 con 0x7fdeb40fce80 2024-03-24T10:51:41.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.400+0000 7fdebb562700 1 -- 172.21.15.94:0/2484975341 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeb410f4a0 con 0x7fdeb40fce80 2024-03-24T10:51:41.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.400+0000 7fdebb562700 1 -- 172.21.15.94:0/2484975341 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdeb405f5f0 con 0x7fdeb40fce80 2024-03-24T10:51:41.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.401+0000 7fdea9ffb700 1 -- 172.21.15.94:0/2484975341 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fdeb001c690 con 0x7fdeb40fce80 2024-03-24T10:51:41.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.401+0000 7fdea9ffb700 1 --2- 172.21.15.94:0/2484975341 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fdea006d7f0 0x7fdea006fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:51:41.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.402+0000 7fdea9ffb700 1 -- 172.21.15.94:0/2484975341 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fdeb008e620 con 0x7fdeb40fce80 2024-03-24T10:51:41.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.402+0000 7fdeb8afd700 1 --2- 172.21.15.94:0/2484975341 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fdea006d7f0 0x7fdea006fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:51:41.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.402+0000 7fdeb8afd700 1 --2- 172.21.15.94:0/2484975341 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fdea006d7f0 0x7fdea006fca0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdea4009b80 tx=0x7fdea400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:51:41.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.406+0000 7fdea9ffb700 1 -- 172.21.15.94:0/2484975341 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fdeb005b9c0 con 0x7fdeb40fce80 2024-03-24T10:51:41.446 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:41 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3-activate[93366]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-24T10:51:41.446 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:41 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3-activate[93366]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-24T10:51:41.446 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:41 smithi096 bash[93352]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-24T10:51:41.446 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:41 smithi096 bash[93352]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-24T10:51:41.446 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:41 smithi096 bash[93352]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-24T10:51:41.446 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:41 smithi096 bash[93352]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-24T10:51:41.446 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:41 smithi096 bash[93352]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-03-24T10:51:41.447 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:41 smithi096 bash[93352]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-24T10:51:41.447 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:41 smithi096 bash[93352]: --> ceph-volume raw activate successful for osd ID: 3 2024-03-24T10:51:41.447 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:41 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3-activate[93366]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-24T10:51:41.447 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:41 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3-activate[93366]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-24T10:51:41.447 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:41 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3-activate[93366]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-03-24T10:51:41.447 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:41 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3-activate[93366]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-24T10:51:41.447 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:41 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3-activate[93366]: --> ceph-volume raw activate successful for osd ID: 3 2024-03-24T10:51:41.447 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:41 smithi096 podman[93352]: 2024-03-24 10:51:41.153840496 +0000 UTC m=+0.818691938 container died b514e1e2494a050a6af890401595344bf5000ff18f41bc51de5a6e7fd083bc48 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3-activate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.schema-version=1.0) 2024-03-24T10:51:41.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.684+0000 7fdebb562700 1 -- 172.21.15.94:0/2484975341 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fdeb410f6c0 con 0x7fdeb40fce80 2024-03-24T10:51:41.684 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:41 smithi094 ceph-mon[93957]: from='client.34282 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:51:41.685 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:41 smithi094 ceph-mon[93957]: from='client.34286 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:51:41.685 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:41 smithi094 ceph-mon[93957]: from='client.34290 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:51:41.685 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:41 smithi094 ceph-mon[93957]: pgmap v113: 65 pgs: 7 active+undersized+degraded, 20 peering, 11 active+undersized, 27 active+clean; 196 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 23/219 objects degraded (10.502%) 2024-03-24T10:51:41.685 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:41 smithi094 ceph-mon[93957]: Health check failed: Degraded data redundancy: 23/219 objects degraded (10.502%), 7 pgs degraded (PG_DEGRADED) 2024-03-24T10:51:41.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.685+0000 7fdea9ffb700 1 -- 172.21.15.94:0/2484975341 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7fdeb0004490 con 0x7fdeb40fce80 2024-03-24T10:51:41.685 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-03-24T10:51:41.685 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": { 2024-03-24T10:51:41.686 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 2 2024-03-24T10:51:41.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:51:41.686 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": { 2024-03-24T10:51:41.686 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 2 2024-03-24T10:51:41.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:51:41.686 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": { 2024-03-24T10:51:41.686 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2, 2024-03-24T10:51:41.686 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 3 2024-03-24T10:51:41.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:51:41.686 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-03-24T10:51:41.686 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-24T10:51:41.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:51:41.686 INFO:teuthology.orchestra.run.smithi094.stdout: "overall": { 2024-03-24T10:51:41.687 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6, 2024-03-24T10:51:41.687 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 7 2024-03-24T10:51:41.687 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-03-24T10:51:41.687 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-03-24T10:51:41.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.688+0000 7fdebb562700 1 -- 172.21.15.94:0/2484975341 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fdea006d7f0 msgr2=0x7fdea006fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:41.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.688+0000 7fdebb562700 1 --2- 172.21.15.94:0/2484975341 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fdea006d7f0 0x7fdea006fca0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdea4009b80 tx=0x7fdea400b040 comp rx=0 tx=0).stop 2024-03-24T10:51:41.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.688+0000 7fdebb562700 1 -- 172.21.15.94:0/2484975341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdeb40fce80 msgr2=0x7fdeb410d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:41.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.688+0000 7fdebb562700 1 --2- 172.21.15.94:0/2484975341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdeb40fce80 0x7fdeb410d9b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0013870 tx=0x7fdeb0009e30 comp rx=0 tx=0).stop 2024-03-24T10:51:41.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.688+0000 7fdebb562700 1 -- 172.21.15.94:0/2484975341 shutdown_connections 2024-03-24T10:51:41.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.688+0000 7fdebb562700 1 --2- 172.21.15.94:0/2484975341 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fdea006d7f0 0x7fdea006fca0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:41.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.688+0000 7fdebb562700 1 --2- 172.21.15.94:0/2484975341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdeb40fce80 0x7fdeb410d9b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:41.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.688+0000 7fdebb562700 1 --2- 172.21.15.94:0/2484975341 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdeb40fdff0 0x7fdeb410def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:41.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.688+0000 7fdebb562700 1 -- 172.21.15.94:0/2484975341 >> 172.21.15.94:0/2484975341 conn(0x7fdeb40f8560 msgr2=0x7fdeb40f9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:51:41.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.688+0000 7fdebb562700 1 -- 172.21.15.94:0/2484975341 shutdown_connections 2024-03-24T10:51:41.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.688+0000 7fdebb562700 1 -- 172.21.15.94:0/2484975341 wait complete. 2024-03-24T10:51:41.769 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:41 smithi096 ceph-mon[84258]: from='client.34282 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:51:41.769 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:41 smithi096 ceph-mon[84258]: from='client.34286 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:51:41.769 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:41 smithi096 ceph-mon[84258]: from='client.34290 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:51:41.769 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:41 smithi096 ceph-mon[84258]: pgmap v113: 65 pgs: 7 active+undersized+degraded, 20 peering, 11 active+undersized, 27 active+clean; 196 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 23/219 objects degraded (10.502%) 2024-03-24T10:51:41.770 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:41 smithi096 ceph-mon[84258]: Health check failed: Degraded data redundancy: 23/219 objects degraded (10.502%), 7 pgs degraded (PG_DEGRADED) 2024-03-24T10:51:41.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.789+0000 7f9c8ba9e700 1 -- 172.21.15.94:0/1042898759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c840fce70 msgr2=0x7f9c840fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:41.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.789+0000 7f9c8ba9e700 1 --2- 172.21.15.94:0/1042898759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c840fce70 0x7f9c840fd280 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9c74004940 tx=0x7f9c74009e30 comp rx=0 tx=0).stop 2024-03-24T10:51:41.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.789+0000 7f9c8ba9e700 1 -- 172.21.15.94:0/1042898759 shutdown_connections 2024-03-24T10:51:41.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.789+0000 7f9c8ba9e700 1 --2- 172.21.15.94:0/1042898759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c840fdfe0 0x7f9c840fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:41.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.789+0000 7f9c8ba9e700 1 --2- 172.21.15.94:0/1042898759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c840fce70 0x7f9c840fd280 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:41.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.789+0000 7f9c8ba9e700 1 -- 172.21.15.94:0/1042898759 >> 172.21.15.94:0/1042898759 conn(0x7f9c840f8550 msgr2=0x7f9c840fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:51:41.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.789+0000 7f9c8ba9e700 1 -- 172.21.15.94:0/1042898759 shutdown_connections 2024-03-24T10:51:41.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.789+0000 7f9c8ba9e700 1 -- 172.21.15.94:0/1042898759 wait complete. 2024-03-24T10:51:41.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.790+0000 7f9c8ba9e700 1 Processor -- start 2024-03-24T10:51:41.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.790+0000 7f9c8ba9e700 1 -- start start 2024-03-24T10:51:41.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.790+0000 7f9c8ba9e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c840fce70 0x7f9c8410d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:51:41.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.790+0000 7f9c8ba9e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c840fdfe0 0x7f9c8410de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:51:41.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.790+0000 7f9c8ba9e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8410e510 con 0x7f9c840fce70 2024-03-24T10:51:41.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.790+0000 7f9c8ba9e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c84112eb0 con 0x7f9c840fdfe0 2024-03-24T10:51:41.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.790+0000 7f9c8983a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c840fce70 0x7f9c8410d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:51:41.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.790+0000 7f9c8983a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c840fce70 0x7f9c8410d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56552/0 (socket says 172.21.15.94:56552) 2024-03-24T10:51:41.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.790+0000 7f9c8983a700 1 -- 172.21.15.94:0/547483972 learned_addr learned my addr 172.21.15.94:0/547483972 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:51:41.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.790+0000 7f9c89039700 1 --2- 172.21.15.94:0/547483972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c840fdfe0 0x7f9c8410de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:51:41.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.790+0000 7f9c8983a700 1 -- 172.21.15.94:0/547483972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c840fdfe0 msgr2=0x7f9c8410de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:41.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.790+0000 7f9c8983a700 1 --2- 172.21.15.94:0/547483972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c840fdfe0 0x7f9c8410de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:41.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.790+0000 7f9c8983a700 1 -- 172.21.15.94:0/547483972 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c74005040 con 0x7f9c840fce70 2024-03-24T10:51:41.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.791+0000 7f9c8983a700 1 --2- 172.21.15.94:0/547483972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c840fce70 0x7f9c8410d940 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9c7400b3b0 tx=0x7f9c74009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:51:41.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.791+0000 7f9c7a7fc700 1 -- 172.21.15.94:0/547483972 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c7400e040 con 0x7f9c840fce70 2024-03-24T10:51:41.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.791+0000 7f9c8ba9e700 1 -- 172.21.15.94:0/547483972 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c841130b0 con 0x7f9c840fce70 2024-03-24T10:51:41.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.791+0000 7f9c7a7fc700 1 -- 172.21.15.94:0/547483972 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9c74004290 con 0x7f9c840fce70 2024-03-24T10:51:41.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.791+0000 7f9c7a7fc700 1 -- 172.21.15.94:0/547483972 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c7401c3f0 con 0x7f9c840fce70 2024-03-24T10:51:41.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.791+0000 7f9c8ba9e700 1 -- 172.21.15.94:0/547483972 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c84113620 con 0x7f9c840fce70 2024-03-24T10:51:41.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.792+0000 7f9c8ba9e700 1 -- 172.21.15.94:0/547483972 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c8404c710 con 0x7f9c840fce70 2024-03-24T10:51:41.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.792+0000 7f9c7a7fc700 1 -- 172.21.15.94:0/547483972 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f9c7400bbf0 con 0x7f9c840fce70 2024-03-24T10:51:41.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.793+0000 7f9c7a7fc700 1 --2- 172.21.15.94:0/547483972 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f9c7006d7e0 0x7f9c7006fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:51:41.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.793+0000 7f9c89039700 1 --2- 172.21.15.94:0/547483972 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f9c7006d7e0 0x7f9c7006fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:51:41.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.793+0000 7f9c7a7fc700 1 -- 172.21.15.94:0/547483972 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f9c7408e220 con 0x7f9c840fce70 2024-03-24T10:51:41.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.793+0000 7f9c89039700 1 --2- 172.21.15.94:0/547483972 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f9c7006d7e0 0x7f9c7006fc90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9c80009bf0 tx=0x7f9c800091a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:51:41.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:41.797+0000 7f9c7a7fc700 1 -- 172.21.15.94:0/547483972 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9c7405b5c0 con 0x7f9c840fce70 2024-03-24T10:51:42.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.035+0000 7f9c8ba9e700 1 -- 172.21.15.94:0/547483972 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9c84113a20 con 0x7f9c840fce70 2024-03-24T10:51:42.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.035+0000 7f9c7a7fc700 1 -- 172.21.15.94:0/547483972 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1903 (secure 0 0 0) 0x7f9c7405b150 con 0x7f9c840fce70 2024-03-24T10:51:42.035 INFO:teuthology.orchestra.run.smithi094.stdout:e18 2024-03-24T10:51:42.035 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T10:51:42.035 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:51:42.035 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-03-24T10:51:42.036 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:51:42.036 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-03-24T10:51:42.036 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-03-24T10:51:42.036 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 18 2024-03-24T10:51:42.036 INFO:teuthology.orchestra.run.smithi094.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-24T10:51:42.036 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-03-24T10:43:39.435738+0000 2024-03-24T10:51:42.036 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-03-24T10:43:58.133901+0000 2024-03-24T10:51:42.036 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-03-24T10:51:42.036 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-03-24T10:51:42.036 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 60 2024-03-24T10:51:42.036 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-03-24T10:51:42.036 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-03-24T10:51:42.036 INFO:teuthology.orchestra.run.smithi094.stdout:max_xattr_size 65536 2024-03-24T10:51:42.036 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-03-24T10:51:42.036 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-03-24T10:51:42.036 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 46 2024-03-24T10:51:42.036 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:51:42.037 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-03-24T10:51:42.037 INFO:teuthology.orchestra.run.smithi094.stdout:in 0 2024-03-24T10:51:42.037 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=24251} 2024-03-24T10:51:42.037 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-03-24T10:51:42.037 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-03-24T10:51:42.037 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-03-24T10:51:42.037 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-03-24T10:51:42.037 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-03-24T10:51:42.037 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-03-24T10:51:42.037 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-03-24T10:51:42.037 INFO:teuthology.orchestra.run.smithi094.stdout:bal_rank_mask -1 2024-03-24T10:51:42.037 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 1 2024-03-24T10:51:42.037 INFO:teuthology.orchestra.run.smithi094.stdout:qdb_cluster leader: 0 members: 2024-03-24T10:51:42.037 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi096.egbmoq{0:24251} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.96:6824/1539752208,v1:172.21.15.96:6825/1539752208] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:51:42.037 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.llefgu{0:14500} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.94:6828/126762980,v1:172.21.15.94:6829/126762980] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:51:42.038 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:51:42.038 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:51:42.038 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-03-24T10:51:42.038 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:51:42.038 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.efiqte{-1:14512} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.94:6826/1636528860,v1:172.21.15.94:6827/1636528860] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:51:42.038 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi096.jtebhh{-1:24261} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.96:6826/1321087930,v1:172.21.15.96:6827/1321087930] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:51:42.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.038+0000 7f9c8ba9e700 1 -- 172.21.15.94:0/547483972 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f9c7006d7e0 msgr2=0x7f9c7006fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:42.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.038+0000 7f9c8ba9e700 1 --2- 172.21.15.94:0/547483972 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f9c7006d7e0 0x7f9c7006fc90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9c80009bf0 tx=0x7f9c800091a0 comp rx=0 tx=0).stop 2024-03-24T10:51:42.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.038+0000 7f9c8ba9e700 1 -- 172.21.15.94:0/547483972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c840fce70 msgr2=0x7f9c8410d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:42.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.038+0000 7f9c8ba9e700 1 --2- 172.21.15.94:0/547483972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c840fce70 0x7f9c8410d940 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9c7400b3b0 tx=0x7f9c74009e30 comp rx=0 tx=0).stop 2024-03-24T10:51:42.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.038+0000 7f9c8ba9e700 1 -- 172.21.15.94:0/547483972 shutdown_connections 2024-03-24T10:51:42.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.038+0000 7f9c8ba9e700 1 --2- 172.21.15.94:0/547483972 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f9c7006d7e0 0x7f9c7006fc90 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:42.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.038+0000 7f9c8ba9e700 1 --2- 172.21.15.94:0/547483972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c840fce70 0x7f9c8410d940 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:42.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.038+0000 7f9c8ba9e700 1 --2- 172.21.15.94:0/547483972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c840fdfe0 0x7f9c8410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:42.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.038+0000 7f9c8ba9e700 1 -- 172.21.15.94:0/547483972 >> 172.21.15.94:0/547483972 conn(0x7f9c840f8550 msgr2=0x7f9c840f9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:51:42.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.038+0000 7f9c8ba9e700 1 -- 172.21.15.94:0/547483972 shutdown_connections 2024-03-24T10:51:42.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.038+0000 7f9c8ba9e700 1 -- 172.21.15.94:0/547483972 wait complete. 2024-03-24T10:51:42.039 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 18 2024-03-24T10:51:42.067 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:41 smithi096 podman[93500]: 2024-03-24 10:51:41.770161726 +0000 UTC m=+0.611511836 container remove b514e1e2494a050a6af890401595344bf5000ff18f41bc51de5a6e7fd083bc48 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3-activate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-24T10:51:42.067 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:41 smithi096 podman[93560]: 2024-03-24 10:51:41.9711789 +0000 UTC m=+0.103004256 container create 6323fdc6e6ca1dd1cce9a0c443c9b490aaf4f251f346523b9bfb0b96cb0dfd6f (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-24T10:51:42.067 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:41 smithi096 podman[93560]: 2024-03-24 10:51:41.885768172 +0000 UTC m=+0.017593541 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-24T10:51:42.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.139+0000 7f86197d2700 1 -- 172.21.15.94:0/3576808037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86140fcee0 msgr2=0x7f86140fd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:42.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.139+0000 7f86197d2700 1 --2- 172.21.15.94:0/3576808037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86140fcee0 0x7f86140fd310 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8604004940 tx=0x7f8604009e30 comp rx=0 tx=0).stop 2024-03-24T10:51:42.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.139+0000 7f86197d2700 1 -- 172.21.15.94:0/3576808037 shutdown_connections 2024-03-24T10:51:42.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.139+0000 7f86197d2700 1 --2- 172.21.15.94:0/3576808037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86140fe0d0 0x7f86140fe540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:42.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.139+0000 7f86197d2700 1 --2- 172.21.15.94:0/3576808037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86140fcee0 0x7f86140fd310 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:42.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.139+0000 7f86197d2700 1 -- 172.21.15.94:0/3576808037 >> 172.21.15.94:0/3576808037 conn(0x7f86140f8560 msgr2=0x7f86140fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:51:42.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.139+0000 7f86197d2700 1 -- 172.21.15.94:0/3576808037 shutdown_connections 2024-03-24T10:51:42.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.139+0000 7f86197d2700 1 -- 172.21.15.94:0/3576808037 wait complete. 2024-03-24T10:51:42.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.139+0000 7f86197d2700 1 Processor -- start 2024-03-24T10:51:42.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.140+0000 7f86197d2700 1 -- start start 2024-03-24T10:51:42.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.140+0000 7f86197d2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86140fcee0 0x7f8614071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:51:42.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.140+0000 7f86197d2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86140fe0d0 0x7f8614070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:51:42.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.140+0000 7f86197d2700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86140708c0 con 0x7f86140fcee0 2024-03-24T10:51:42.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.140+0000 7f86197d2700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8614070a30 con 0x7f86140fe0d0 2024-03-24T10:51:42.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.140+0000 7f8612ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86140fcee0 0x7f8614071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:51:42.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.140+0000 7f8612ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86140fcee0 0x7f8614071e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56568/0 (socket says 172.21.15.94:56568) 2024-03-24T10:51:42.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.140+0000 7f8612ffd700 1 -- 172.21.15.94:0/391543726 learned_addr learned my addr 172.21.15.94:0/391543726 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:51:42.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.140+0000 7f86127fc700 1 --2- 172.21.15.94:0/391543726 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86140fe0d0 0x7f8614070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:51:42.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.140+0000 7f8612ffd700 1 -- 172.21.15.94:0/391543726 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86140fe0d0 msgr2=0x7f8614070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:42.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.140+0000 7f8612ffd700 1 --2- 172.21.15.94:0/391543726 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86140fe0d0 0x7f8614070310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:42.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.140+0000 7f8612ffd700 1 -- 172.21.15.94:0/391543726 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8604005040 con 0x7f86140fcee0 2024-03-24T10:51:42.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.140+0000 7f8612ffd700 1 --2- 172.21.15.94:0/391543726 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86140fcee0 0x7f8614071e40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f860400b3b0 tx=0x7f8604009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:51:42.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.140+0000 7f85fb7fe700 1 -- 172.21.15.94:0/391543726 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f860400e040 con 0x7f86140fcee0 2024-03-24T10:51:42.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.140+0000 7f85fb7fe700 1 -- 172.21.15.94:0/391543726 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f860400bcc0 con 0x7f86140fcee0 2024-03-24T10:51:42.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.140+0000 7f85fb7fe700 1 -- 172.21.15.94:0/391543726 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f860401c630 con 0x7f86140fcee0 2024-03-24T10:51:42.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.140+0000 7f86197d2700 1 -- 172.21.15.94:0/391543726 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8614070cb0 con 0x7f86140fcee0 2024-03-24T10:51:42.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.140+0000 7f86197d2700 1 -- 172.21.15.94:0/391543726 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8614115a90 con 0x7f86140fcee0 2024-03-24T10:51:42.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.141+0000 7f86197d2700 1 -- 172.21.15.94:0/391543726 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f861405f5f0 con 0x7f86140fcee0 2024-03-24T10:51:42.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.142+0000 7f85fb7fe700 1 -- 172.21.15.94:0/391543726 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f86040139f0 con 0x7f86140fcee0 2024-03-24T10:51:42.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.142+0000 7f85fb7fe700 1 --2- 172.21.15.94:0/391543726 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f85fc06d7e0 0x7f85fc06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:51:42.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.142+0000 7f85fb7fe700 1 -- 172.21.15.94:0/391543726 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f860408daf0 con 0x7f86140fcee0 2024-03-24T10:51:42.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.142+0000 7f86127fc700 1 --2- 172.21.15.94:0/391543726 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f85fc06d7e0 0x7f85fc06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:51:42.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.143+0000 7f86127fc700 1 --2- 172.21.15.94:0/391543726 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f85fc06d7e0 0x7f85fc06fc90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f86080041d0 tx=0x7f8608009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:51:42.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.147+0000 7f85fb7fe700 1 -- 172.21.15.94:0/391543726 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8604057400 con 0x7f86140fcee0 2024-03-24T10:51:42.325 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:42 smithi096 podman[93560]: 2024-03-24 10:51:42.156452929 +0000 UTC m=+0.288278296 container init 6323fdc6e6ca1dd1cce9a0c443c9b490aaf4f251f346523b9bfb0b96cb0dfd6f (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T10:51:42.325 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:42 smithi096 podman[93560]: 2024-03-24 10:51:42.159440432 +0000 UTC m=+0.291265788 container start 6323fdc6e6ca1dd1cce9a0c443c9b490aaf4f251f346523b9bfb0b96cb0dfd6f (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True) 2024-03-24T10:51:42.325 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:42 smithi096 ceph-osd[93580]: -- 172.21.15.96:0/262068211 <== mon.1 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564293da71e0 con 0x564293d78c00 2024-03-24T10:51:42.325 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:42 smithi096 bash[93560]: 6323fdc6e6ca1dd1cce9a0c443c9b490aaf4f251f346523b9bfb0b96cb0dfd6f 2024-03-24T10:51:42.325 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:42 smithi096 systemd[1]: Started Ceph osd.3 for 386110d4-e9ca-11ee-95cd-87774f69a715. 2024-03-24T10:51:42.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.361+0000 7f86197d2700 1 -- 172.21.15.94:0/391543726 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f86141024e0 con 0x7f85fc06d7e0 2024-03-24T10:51:42.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.362+0000 7f85fb7fe700 1 -- 172.21.15.94:0/391543726 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f86141024e0 con 0x7f85fc06d7e0 2024-03-24T10:51:42.362 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-03-24T10:51:42.362 INFO:teuthology.orchestra.run.smithi094.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919", 2024-03-24T10:51:42.362 INFO:teuthology.orchestra.run.smithi094.stdout: "in_progress": true, 2024-03-24T10:51:42.362 INFO:teuthology.orchestra.run.smithi094.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T10:51:42.362 INFO:teuthology.orchestra.run.smithi094.stdout: "services_complete": [ 2024-03-24T10:51:42.362 INFO:teuthology.orchestra.run.smithi094.stdout: "mon", 2024-03-24T10:51:42.362 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-03-24T10:51:42.362 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr" 2024-03-24T10:51:42.362 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-03-24T10:51:42.363 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": "9/21 daemons upgraded", 2024-03-24T10:51:42.363 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "Currently upgrading osd daemons", 2024-03-24T10:51:42.363 INFO:teuthology.orchestra.run.smithi094.stdout: "is_paused": false 2024-03-24T10:51:42.363 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-03-24T10:51:42.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.364+0000 7f86197d2700 1 -- 172.21.15.94:0/391543726 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f85fc06d7e0 msgr2=0x7f85fc06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:42.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.364+0000 7f86197d2700 1 --2- 172.21.15.94:0/391543726 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f85fc06d7e0 0x7f85fc06fc90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f86080041d0 tx=0x7f8608009040 comp rx=0 tx=0).stop 2024-03-24T10:51:42.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.364+0000 7f86197d2700 1 -- 172.21.15.94:0/391543726 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86140fcee0 msgr2=0x7f8614071e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:42.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.364+0000 7f86197d2700 1 --2- 172.21.15.94:0/391543726 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86140fcee0 0x7f8614071e40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f860400b3b0 tx=0x7f8604009e30 comp rx=0 tx=0).stop 2024-03-24T10:51:42.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.364+0000 7f86197d2700 1 -- 172.21.15.94:0/391543726 shutdown_connections 2024-03-24T10:51:42.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.364+0000 7f86197d2700 1 --2- 172.21.15.94:0/391543726 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f85fc06d7e0 0x7f85fc06fc90 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:42.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.364+0000 7f86197d2700 1 --2- 172.21.15.94:0/391543726 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86140fcee0 0x7f8614071e40 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:42.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.364+0000 7f86197d2700 1 --2- 172.21.15.94:0/391543726 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86140fe0d0 0x7f8614070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:42.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.364+0000 7f86197d2700 1 -- 172.21.15.94:0/391543726 >> 172.21.15.94:0/391543726 conn(0x7f86140f8560 msgr2=0x7f86140f9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:51:42.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.365+0000 7f86197d2700 1 -- 172.21.15.94:0/391543726 shutdown_connections 2024-03-24T10:51:42.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.365+0000 7f86197d2700 1 -- 172.21.15.94:0/391543726 wait complete. 2024-03-24T10:51:42.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.467+0000 7fbe33b21700 1 -- 172.21.15.94:0/1156748977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe2c066220 msgr2=0x7fbe2c0ffa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:42.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.467+0000 7fbe33b21700 1 --2- 172.21.15.94:0/1156748977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe2c066220 0x7fbe2c0ffa80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbe28009ce0 tx=0x7fbe28004d10 comp rx=0 tx=0).stop 2024-03-24T10:51:42.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.467+0000 7fbe33b21700 1 -- 172.21.15.94:0/1156748977 shutdown_connections 2024-03-24T10:51:42.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.467+0000 7fbe33b21700 1 --2- 172.21.15.94:0/1156748977 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe2c0fffc0 0x7fbe2c1023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:42.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.467+0000 7fbe33b21700 1 --2- 172.21.15.94:0/1156748977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe2c066220 0x7fbe2c0ffa80 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:42.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.467+0000 7fbe33b21700 1 -- 172.21.15.94:0/1156748977 >> 172.21.15.94:0/1156748977 conn(0x7fbe2c0f7430 msgr2=0x7fbe2c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:51:42.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.467+0000 7fbe33b21700 1 -- 172.21.15.94:0/1156748977 shutdown_connections 2024-03-24T10:51:42.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.467+0000 7fbe33b21700 1 -- 172.21.15.94:0/1156748977 wait complete. 2024-03-24T10:51:42.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.467+0000 7fbe33b21700 1 Processor -- start 2024-03-24T10:51:42.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.467+0000 7fbe33b21700 1 -- start start 2024-03-24T10:51:42.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.468+0000 7fbe33b21700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe2c066220 0x7fbe2c10dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:51:42.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.468+0000 7fbe33b21700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe2c0fffc0 0x7fbe2c10e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:51:42.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.468+0000 7fbe33b21700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2c10e720 con 0x7fbe2c0fffc0 2024-03-24T10:51:42.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.468+0000 7fbe33b21700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2c10ef20 con 0x7fbe2c066220 2024-03-24T10:51:42.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.468+0000 7fbe310bc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe2c0fffc0 0x7fbe2c10e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:51:42.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.468+0000 7fbe310bc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe2c0fffc0 0x7fbe2c10e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56584/0 (socket says 172.21.15.94:56584) 2024-03-24T10:51:42.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.468+0000 7fbe310bc700 1 -- 172.21.15.94:0/1999797232 learned_addr learned my addr 172.21.15.94:0/1999797232 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:51:42.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.468+0000 7fbe318bd700 1 --2- 172.21.15.94:0/1999797232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe2c066220 0x7fbe2c10dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:51:42.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.468+0000 7fbe310bc700 1 -- 172.21.15.94:0/1999797232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe2c066220 msgr2=0x7fbe2c10dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:42.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.468+0000 7fbe310bc700 1 --2- 172.21.15.94:0/1999797232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe2c066220 0x7fbe2c10dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:42.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.468+0000 7fbe310bc700 1 -- 172.21.15.94:0/1999797232 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe28009940 con 0x7fbe2c0fffc0 2024-03-24T10:51:42.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.468+0000 7fbe310bc700 1 --2- 172.21.15.94:0/1999797232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe2c0fffc0 0x7fbe2c10e120 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c002a50 tx=0x7fbe1c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:51:42.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.468+0000 7fbe227fc700 1 -- 172.21.15.94:0/1999797232 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe1c007cc0 con 0x7fbe2c0fffc0 2024-03-24T10:51:42.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.468+0000 7fbe227fc700 1 -- 172.21.15.94:0/1999797232 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe1c00f040 con 0x7fbe2c0fffc0 2024-03-24T10:51:42.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.468+0000 7fbe33b21700 1 -- 172.21.15.94:0/1999797232 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe2c10f200 con 0x7fbe2c0fffc0 2024-03-24T10:51:42.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.468+0000 7fbe227fc700 1 -- 172.21.15.94:0/1999797232 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe1c00db30 con 0x7fbe2c0fffc0 2024-03-24T10:51:42.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.468+0000 7fbe33b21700 1 -- 172.21.15.94:0/1999797232 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe2c10f740 con 0x7fbe2c0fffc0 2024-03-24T10:51:42.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.469+0000 7fbe33b21700 1 -- 172.21.15.94:0/1999797232 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe2c05f5f0 con 0x7fbe2c0fffc0 2024-03-24T10:51:42.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.470+0000 7fbe227fc700 1 -- 172.21.15.94:0/1999797232 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fbe1c00dc90 con 0x7fbe2c0fffc0 2024-03-24T10:51:42.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.470+0000 7fbe227fc700 1 --2- 172.21.15.94:0/1999797232 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fbe1806d7e0 0x7fbe1806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:51:42.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.471+0000 7fbe227fc700 1 -- 172.21.15.94:0/1999797232 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fbe1c08cbc0 con 0x7fbe2c0fffc0 2024-03-24T10:51:42.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.471+0000 7fbe318bd700 1 --2- 172.21.15.94:0/1999797232 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fbe1806d7e0 0x7fbe1806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:51:42.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.471+0000 7fbe318bd700 1 --2- 172.21.15.94:0/1999797232 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fbe1806d7e0 0x7fbe1806fc90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbe28009470 tx=0x7fbe28004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:51:42.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.475+0000 7fbe227fc700 1 -- 172.21.15.94:0/1999797232 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fbe1c059f60 con 0x7fbe2c0fffc0 2024-03-24T10:51:42.699 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:42 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/2484975341' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:51:42.699 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:42 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/547483972' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T10:51:42.699 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:42 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:51:42.699 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:42 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:51:42.699 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:42 smithi096 ceph-mon[84258]: from='client.34302 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:51:42.699 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:42 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:51:42.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:42 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/2484975341' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:51:42.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:42 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/547483972' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T10:51:42.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:42 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:51:42.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:42 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:51:42.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:42 smithi094 ceph-mon[93957]: from='client.34302 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:51:42.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:42 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:51:42.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.752+0000 7fbe33b21700 1 -- 172.21.15.94:0/1999797232 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fbe2c04bed0 con 0x7fbe2c0fffc0 2024-03-24T10:51:42.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.753+0000 7fbe227fc700 1 -- 172.21.15.94:0/1999797232 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1232 (secure 0 0 0) 0x7fbe1c059af0 con 0x7fbe2c0fffc0 2024-03-24T10:51:42.753 INFO:teuthology.orchestra.run.smithi094.stdout:HEALTH_WARN 1 osds down; Reduced data availability: 6 pgs peering; Degraded data redundancy: 23/219 objects degraded (10.502%), 7 pgs degraded 2024-03-24T10:51:42.753 INFO:teuthology.orchestra.run.smithi094.stdout:[WRN] OSD_DOWN: 1 osds down 2024-03-24T10:51:42.753 INFO:teuthology.orchestra.run.smithi094.stdout: osd.3 (root=default,host=smithi096) is down 2024-03-24T10:51:42.753 INFO:teuthology.orchestra.run.smithi094.stdout:[WRN] PG_AVAILABILITY: Reduced data availability: 6 pgs peering 2024-03-24T10:51:42.753 INFO:teuthology.orchestra.run.smithi094.stdout: pg 1.0 is stuck peering for 3m, current state peering, last acting [0,1] 2024-03-24T10:51:42.753 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.0 is stuck peering for 3m, current state peering, last acting [1,0] 2024-03-24T10:51:42.753 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.5 is stuck peering for 3m, current state peering, last acting [0,4] 2024-03-24T10:51:42.753 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.12 is stuck peering for 3m, current state peering, last acting [1,0] 2024-03-24T10:51:42.753 INFO:teuthology.orchestra.run.smithi094.stdout: pg 3.d is stuck peering for 3m, current state peering, last acting [2,4] 2024-03-24T10:51:42.753 INFO:teuthology.orchestra.run.smithi094.stdout: pg 3.15 is stuck peering for 3m, current state peering, last acting [0,4] 2024-03-24T10:51:42.753 INFO:teuthology.orchestra.run.smithi094.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 23/219 objects degraded (10.502%), 7 pgs degraded 2024-03-24T10:51:42.754 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.7 is active+undersized+degraded, acting [4,2] 2024-03-24T10:51:42.754 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.b is active+undersized+degraded, acting [4,5] 2024-03-24T10:51:42.754 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.14 is active+undersized+degraded, acting [4,5] 2024-03-24T10:51:42.754 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.16 is active+undersized+degraded, acting [5,2] 2024-03-24T10:51:42.754 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.18 is active+undersized+degraded, acting [5,4] 2024-03-24T10:51:42.754 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.1a is active+undersized+degraded, acting [4,5] 2024-03-24T10:51:42.754 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.1d is active+undersized+degraded, acting [5,0] 2024-03-24T10:51:42.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.755+0000 7fbe33b21700 1 -- 172.21.15.94:0/1999797232 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fbe1806d7e0 msgr2=0x7fbe1806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:42.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.755+0000 7fbe33b21700 1 --2- 172.21.15.94:0/1999797232 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fbe1806d7e0 0x7fbe1806fc90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbe28009470 tx=0x7fbe28004d10 comp rx=0 tx=0).stop 2024-03-24T10:51:42.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.755+0000 7fbe33b21700 1 -- 172.21.15.94:0/1999797232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe2c0fffc0 msgr2=0x7fbe2c10e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:51:42.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.755+0000 7fbe33b21700 1 --2- 172.21.15.94:0/1999797232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe2c0fffc0 0x7fbe2c10e120 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c002a50 tx=0x7fbe1c0046b0 comp rx=0 tx=0).stop 2024-03-24T10:51:42.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.755+0000 7fbe33b21700 1 -- 172.21.15.94:0/1999797232 shutdown_connections 2024-03-24T10:51:42.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.755+0000 7fbe33b21700 1 --2- 172.21.15.94:0/1999797232 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fbe1806d7e0 0x7fbe1806fc90 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:42.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.755+0000 7fbe33b21700 1 --2- 172.21.15.94:0/1999797232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe2c066220 0x7fbe2c10dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:42.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.755+0000 7fbe33b21700 1 --2- 172.21.15.94:0/1999797232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe2c0fffc0 0x7fbe2c10e120 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:51:42.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.755+0000 7fbe33b21700 1 -- 172.21.15.94:0/1999797232 >> 172.21.15.94:0/1999797232 conn(0x7fbe2c0f7430 msgr2=0x7fbe2c0f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:51:42.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.755+0000 7fbe33b21700 1 -- 172.21.15.94:0/1999797232 shutdown_connections 2024-03-24T10:51:42.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:51:42.755+0000 7fbe33b21700 1 -- 172.21.15.94:0/1999797232 wait complete. 2024-03-24T10:51:43.075 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:42 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3[93576]: 2024-03-24T10:51:42.698+0000 7f1ff9eda7c0 -1 Falling back to public interface 2024-03-24T10:51:43.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:43 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/1999797232' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T10:51:43.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:43 smithi094 ceph-mon[93957]: pgmap v114: 65 pgs: 17 active+undersized+degraded, 21 active+undersized, 27 active+clean; 196 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 48/219 objects degraded (21.918%) 2024-03-24T10:51:43.811 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:43 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/1999797232' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T10:51:43.811 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:43 smithi096 ceph-mon[84258]: pgmap v114: 65 pgs: 17 active+undersized+degraded, 21 active+undersized, 27 active+clean; 196 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 48/219 objects degraded (21.918%) 2024-03-24T10:51:44.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:44 smithi094 ceph-mon[93957]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 6 pgs peering) 2024-03-24T10:51:44.755 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:44 smithi096 ceph-mon[84258]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 6 pgs peering) 2024-03-24T10:51:46.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:46 smithi096 ceph-mon[84258]: pgmap v115: 65 pgs: 17 active+undersized+degraded, 21 active+undersized, 27 active+clean; 196 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 48/219 objects degraded (21.918%) 2024-03-24T10:51:46.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:46 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:51:46.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:46 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:51:46.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:46 smithi094 ceph-mon[93957]: pgmap v115: 65 pgs: 17 active+undersized+degraded, 21 active+undersized, 27 active+clean; 196 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 48/219 objects degraded (21.918%) 2024-03-24T10:51:46.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:46 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:51:46.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:46 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:51:47.575 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:47 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3[93576]: 2024-03-24T10:51:47.304+0000 7f1ff9eda7c0 -1 osd.3 0 read_superblock omap replica is missing. 2024-03-24T10:51:47.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:47 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:51:47.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:47 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:51:47.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:47 smithi096 ceph-mon[84258]: Health check update: Degraded data redundancy: 48/219 objects degraded (21.918%), 17 pgs degraded (PG_DEGRADED) 2024-03-24T10:51:47.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:47 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:51:47.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:47 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:51:47.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:47 smithi094 ceph-mon[93957]: Health check update: Degraded data redundancy: 48/219 objects degraded (21.918%), 17 pgs degraded (PG_DEGRADED) 2024-03-24T10:51:48.139 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:47 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3[93576]: 2024-03-24T10:51:47.889+0000 7f1ff9eda7c0 -1 osd.3 64 log_to_monitors true 2024-03-24T10:51:48.411 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:48 smithi096 ceph-mon[84258]: pgmap v116: 65 pgs: 17 active+undersized+degraded, 21 active+undersized, 27 active+clean; 196 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 48/219 objects degraded (21.918%) 2024-03-24T10:51:48.412 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:48 smithi096 ceph-mon[84258]: from='osd.3 [v2:172.21.15.96:6800/2693214290,v1:172.21.15.96:6801/2693214290]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-24T10:51:48.412 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:48 smithi096 ceph-mon[84258]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-24T10:51:48.658 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:48 smithi094 ceph-mon[93957]: pgmap v116: 65 pgs: 17 active+undersized+degraded, 21 active+undersized, 27 active+clean; 196 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 48/219 objects degraded (21.918%) 2024-03-24T10:51:48.658 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:48 smithi094 ceph-mon[93957]: from='osd.3 [v2:172.21.15.96:6800/2693214290,v1:172.21.15.96:6801/2693214290]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-24T10:51:48.658 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:48 smithi094 ceph-mon[93957]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-24T10:51:48.718 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:51:48 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3[93576]: 2024-03-24T10:51:48.411+0000 7f1feb332700 -1 osd.3 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T10:51:49.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:49 smithi096 ceph-mon[84258]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-24T10:51:49.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:49 smithi096 ceph-mon[84258]: osdmap e67: 6 total, 5 up, 6 in 2024-03-24T10:51:49.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:49 smithi096 ceph-mon[84258]: from='osd.3 [v2:172.21.15.96:6800/2693214290,v1:172.21.15.96:6801/2693214290]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-03-24T10:51:49.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:49 smithi096 ceph-mon[84258]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-03-24T10:51:49.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:49 smithi096 ceph-mon[84258]: failed to encode map e67 with expected crc 2024-03-24T10:51:49.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:49 smithi094 ceph-mon[93957]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-24T10:51:49.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:49 smithi094 ceph-mon[93957]: osdmap e67: 6 total, 5 up, 6 in 2024-03-24T10:51:49.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:49 smithi094 ceph-mon[93957]: from='osd.3 [v2:172.21.15.96:6800/2693214290,v1:172.21.15.96:6801/2693214290]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-03-24T10:51:49.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:49 smithi094 ceph-mon[93957]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-03-24T10:51:49.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:49 smithi094 ceph-mon[93957]: failed to encode map e67 with expected crc 2024-03-24T10:51:50.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:50 smithi094 ceph-mon[93957]: failed to encode map e67 with expected crc 2024-03-24T10:51:50.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:50 smithi094 ceph-mon[93957]: pgmap v118: 65 pgs: 17 active+undersized+degraded, 21 active+undersized, 27 active+clean; 196 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 48/219 objects degraded (21.918%) 2024-03-24T10:51:50.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:50 smithi094 ceph-mon[93957]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T10:51:50.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:50 smithi094 ceph-mon[93957]: failed to encode map e68 with expected crc 2024-03-24T10:51:50.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:50 smithi094 ceph-mon[93957]: osd.3 [v2:172.21.15.96:6800/2693214290,v1:172.21.15.96:6801/2693214290] boot 2024-03-24T10:51:50.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:50 smithi094 ceph-mon[93957]: osdmap e68: 6 total, 6 up, 6 in 2024-03-24T10:51:50.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:50 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T10:51:50.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:50 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:51:50.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:50 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:51:50.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:50 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:51:50.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:50 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:51:50.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:50 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:51:50.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:50 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:51:50.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:50 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:51:50.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:50 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:51:50.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:50 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:51:50.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:50 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-24T10:51:50.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:50 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:51:50.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:50 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:51:50.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:50 smithi096 ceph-mon[84258]: failed to encode map e67 with expected crc 2024-03-24T10:51:50.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:50 smithi096 ceph-mon[84258]: pgmap v118: 65 pgs: 17 active+undersized+degraded, 21 active+undersized, 27 active+clean; 196 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 48/219 objects degraded (21.918%) 2024-03-24T10:51:50.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:50 smithi096 ceph-mon[84258]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T10:51:50.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:50 smithi096 ceph-mon[84258]: failed to encode map e68 with expected crc 2024-03-24T10:51:50.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:50 smithi096 ceph-mon[84258]: osd.3 [v2:172.21.15.96:6800/2693214290,v1:172.21.15.96:6801/2693214290] boot 2024-03-24T10:51:50.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:50 smithi096 ceph-mon[84258]: osdmap e68: 6 total, 6 up, 6 in 2024-03-24T10:51:50.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:50 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T10:51:50.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:50 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:51:50.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:50 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:51:50.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:50 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:51:50.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:50 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:51:50.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:50 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:51:50.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:50 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:51:50.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:50 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:51:50.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:50 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:51:50.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:50 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:51:50.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:50 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-24T10:51:50.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:50 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:51:50.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:50 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:51:51.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:51 smithi094 ceph-mon[93957]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-24T10:51:51.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:51 smithi094 ceph-mon[93957]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-03-24T10:51:51.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:51 smithi094 ceph-mon[93957]: failed to encode map e69 with expected crc 2024-03-24T10:51:51.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:51 smithi094 ceph-mon[93957]: osdmap e69: 6 total, 6 up, 6 in 2024-03-24T10:51:51.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:51 smithi094 ceph-mon[93957]: failed to encode map e69 with expected crc 2024-03-24T10:51:51.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:51 smithi094 ceph-mon[93957]: failed to encode map e69 with expected crc 2024-03-24T10:51:51.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:51 smithi094 ceph-mon[93957]: failed to encode map e69 with expected crc 2024-03-24T10:51:51.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:51 smithi096 ceph-mon[84258]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-24T10:51:51.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:51 smithi096 ceph-mon[84258]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-03-24T10:51:51.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:51 smithi096 ceph-mon[84258]: failed to encode map e69 with expected crc 2024-03-24T10:51:51.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:51 smithi096 ceph-mon[84258]: osdmap e69: 6 total, 6 up, 6 in 2024-03-24T10:51:51.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:51 smithi096 ceph-mon[84258]: failed to encode map e69 with expected crc 2024-03-24T10:51:51.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:51 smithi096 ceph-mon[84258]: failed to encode map e69 with expected crc 2024-03-24T10:51:51.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:51 smithi096 ceph-mon[84258]: failed to encode map e69 with expected crc 2024-03-24T10:51:52.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:52 smithi094 ceph-mon[93957]: pgmap v121: 65 pgs: 15 active+undersized+degraded, 15 active+undersized, 35 active+clean; 196 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 41/219 objects degraded (18.721%) 2024-03-24T10:51:52.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:52 smithi094 ceph-mon[93957]: Health check update: Degraded data redundancy: 41/219 objects degraded (18.721%), 15 pgs degraded (PG_DEGRADED) 2024-03-24T10:51:52.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:52 smithi096 ceph-mon[84258]: pgmap v121: 65 pgs: 15 active+undersized+degraded, 15 active+undersized, 35 active+clean; 196 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 41/219 objects degraded (18.721%) 2024-03-24T10:51:52.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:52 smithi096 ceph-mon[84258]: Health check update: Degraded data redundancy: 41/219 objects degraded (18.721%), 15 pgs degraded (PG_DEGRADED) 2024-03-24T10:51:54.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:54 smithi094 ceph-mon[93957]: pgmap v122: 65 pgs: 8 active+undersized+degraded, 8 active+undersized, 49 active+clean; 196 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 23/219 objects degraded (10.502%) 2024-03-24T10:51:54.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:54 smithi096 ceph-mon[84258]: pgmap v122: 65 pgs: 8 active+undersized+degraded, 8 active+undersized, 49 active+clean; 196 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 23/219 objects degraded (10.502%) 2024-03-24T10:51:55.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:55 smithi094 ceph-mon[93957]: pgmap v123: 65 pgs: 65 active+clean; 196 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 742 B/s rd, 1 op/s 2024-03-24T10:51:55.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:55 smithi094 ceph-mon[93957]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 23/219 objects degraded (10.502%), 8 pgs degraded) 2024-03-24T10:51:55.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:55 smithi094 ceph-mon[93957]: Cluster is now healthy 2024-03-24T10:51:55.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:55 smithi096 ceph-mon[84258]: pgmap v123: 65 pgs: 65 active+clean; 196 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 742 B/s rd, 1 op/s 2024-03-24T10:51:55.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:55 smithi096 ceph-mon[84258]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 23/219 objects degraded (10.502%), 8 pgs degraded) 2024-03-24T10:51:55.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:55 smithi096 ceph-mon[84258]: Cluster is now healthy 2024-03-24T10:51:58.659 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:51:58 smithi094 ceph-mon[93957]: pgmap v124: 65 pgs: 65 active+clean; 196 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-03-24T10:51:58.798 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:51:58 smithi096 ceph-mon[84258]: pgmap v124: 65 pgs: 65 active+clean; 196 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-03-24T10:52:00.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:00 smithi094 ceph-mon[93957]: pgmap v125: 65 pgs: 65 active+clean; 196 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-24T10:52:00.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:00 smithi096 ceph-mon[84258]: pgmap v125: 65 pgs: 65 active+clean; 196 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-24T10:52:01.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:01 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:01.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:01 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:02.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:02 smithi094 ceph-mon[93957]: pgmap v126: 65 pgs: 65 active+clean; 196 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 940 B/s rd, 1 op/s 2024-03-24T10:52:02.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:02 smithi096 ceph-mon[84258]: pgmap v126: 65 pgs: 65 active+clean; 196 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 940 B/s rd, 1 op/s 2024-03-24T10:52:04.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:04 smithi094 ceph-mon[93957]: pgmap v127: 65 pgs: 65 active+clean; 196 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:52:04.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:04 smithi096 ceph-mon[84258]: pgmap v127: 65 pgs: 65 active+clean; 196 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:52:05.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:05 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-24T10:52:05.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:05 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:05.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:05 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:52:05.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:05 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-24T10:52:05.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:05 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:05.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:05 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:52:06.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:06 smithi096 ceph-mon[84258]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-24T10:52:06.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:06 smithi096 ceph-mon[84258]: Upgrade: osd.4 is safe to restart 2024-03-24T10:52:06.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:06 smithi096 ceph-mon[84258]: pgmap v128: 65 pgs: 65 active+clean; 196 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:52:06.648 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:06 smithi094 ceph-mon[93957]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-24T10:52:06.649 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:06 smithi094 ceph-mon[93957]: Upgrade: osd.4 is safe to restart 2024-03-24T10:52:06.649 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:06 smithi094 ceph-mon[93957]: pgmap v128: 65 pgs: 65 active+clean; 196 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:52:07.927 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:07 smithi094 ceph-mon[93957]: Upgrade: Updating osd.4 2024-03-24T10:52:07.927 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:07 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:07.927 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:07 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-24T10:52:07.927 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:07 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:52:07.927 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:07 smithi094 ceph-mon[93957]: Deploying daemon osd.4 on smithi096 2024-03-24T10:52:07.927 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:07 smithi094 ceph-mon[93957]: pgmap v129: 65 pgs: 65 active+clean; 196 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:52:08.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:07 smithi096 ceph-mon[84258]: Upgrade: Updating osd.4 2024-03-24T10:52:08.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:07 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:08.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:07 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-24T10:52:08.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:07 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:52:08.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:07 smithi096 ceph-mon[84258]: Deploying daemon osd.4 on smithi096 2024-03-24T10:52:08.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:07 smithi096 ceph-mon[84258]: pgmap v129: 65 pgs: 65 active+clean; 196 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:52:10.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:10 smithi096 ceph-mon[84258]: pgmap v130: 65 pgs: 65 active+clean; 196 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:52:10.576 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:10 smithi096 systemd[1]: Stopping Ceph osd.4 for 386110d4-e9ca-11ee-95cd-87774f69a715... 2024-03-24T10:52:10.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:10 smithi094 ceph-mon[93957]: pgmap v130: 65 pgs: 65 active+clean; 196 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:52:11.075 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:10 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4[50398]: 2024-03-24T10:52:10.578+0000 7f5193c73700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T10:52:11.075 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:10 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4[50398]: 2024-03-24T10:52:10.578+0000 7f5193c73700 -1 osd.4 69 *** Got signal Terminated *** 2024-03-24T10:52:11.075 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:10 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4[50398]: 2024-03-24T10:52:10.578+0000 7f5193c73700 -1 osd.4 69 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T10:52:11.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:11 smithi094 ceph-mon[93957]: osd.4 marked itself down and dead 2024-03-24T10:52:11.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:11 smithi096 ceph-mon[84258]: osd.4 marked itself down and dead 2024-03-24T10:52:12.180 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:11 smithi096 podman[98147]: 2024-03-24 10:52:11.907600801 +0000 UTC m=+1.449617785 container died a63d7c9e705f902476ae6cd78fd5e6d60f9e80cbbd1952930b70715113da9099 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-9c4e0c3, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-24T10:52:12.571 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:12 smithi096 ceph-mon[84258]: pgmap v131: 65 pgs: 65 active+clean; 196 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:52:12.571 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:12 smithi096 ceph-mon[84258]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T10:52:12.571 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:12 smithi096 ceph-mon[84258]: failed to encode map e70 with expected crc 2024-03-24T10:52:12.571 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:12 smithi096 ceph-mon[84258]: osdmap e70: 6 total, 5 up, 6 in 2024-03-24T10:52:12.572 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:12 smithi096 podman[98147]: 2024-03-24 10:52:12.278795271 +0000 UTC m=+1.820812248 container cleanup a63d7c9e705f902476ae6cd78fd5e6d60f9e80cbbd1952930b70715113da9099 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=quincy-9c4e0c3, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-24T10:52:12.572 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:12 smithi096 bash[98147]: ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4 2024-03-24T10:52:12.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:12 smithi094 ceph-mon[93957]: pgmap v131: 65 pgs: 65 active+clean; 196 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:52:12.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:12 smithi094 ceph-mon[93957]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T10:52:12.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:12 smithi094 ceph-mon[93957]: failed to encode map e70 with expected crc 2024-03-24T10:52:12.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:12 smithi094 ceph-mon[93957]: osdmap e70: 6 total, 5 up, 6 in 2024-03-24T10:52:12.825 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:12 smithi096 podman[98163]: 2024-03-24 10:52:12.571702175 +0000 UTC m=+0.659165702 container remove a63d7c9e705f902476ae6cd78fd5e6d60f9e80cbbd1952930b70715113da9099 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-9c4e0c3, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-24T10:52:12.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:12.870+0000 7f6c37699700 1 -- 172.21.15.94:0/1566495012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c300fce70 msgr2=0x7f6c300fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:12.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:12.870+0000 7f6c37699700 1 --2- 172.21.15.94:0/1566495012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c300fce70 0x7f6c300fd280 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c004880 tx=0x7f6c2c009e30 comp rx=0 tx=0).stop 2024-03-24T10:52:12.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:12.870+0000 7f6c37699700 1 -- 172.21.15.94:0/1566495012 shutdown_connections 2024-03-24T10:52:12.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:12.870+0000 7f6c37699700 1 --2- 172.21.15.94:0/1566495012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c300fdfe0 0x7f6c300fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:12.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:12.870+0000 7f6c37699700 1 --2- 172.21.15.94:0/1566495012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c300fce70 0x7f6c300fd280 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:12.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:12.870+0000 7f6c37699700 1 -- 172.21.15.94:0/1566495012 >> 172.21.15.94:0/1566495012 conn(0x7f6c300f8550 msgr2=0x7f6c300fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:52:12.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:12.870+0000 7f6c37699700 1 -- 172.21.15.94:0/1566495012 shutdown_connections 2024-03-24T10:52:12.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:12.870+0000 7f6c37699700 1 -- 172.21.15.94:0/1566495012 wait complete. 2024-03-24T10:52:12.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:12.871+0000 7f6c37699700 1 Processor -- start 2024-03-24T10:52:12.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:12.871+0000 7f6c37699700 1 -- start start 2024-03-24T10:52:12.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:12.871+0000 7f6c37699700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c300fce70 0x7f6c30071c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:52:12.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:12.871+0000 7f6c37699700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c300fdfe0 0x7f6c30070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:52:12.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:12.871+0000 7f6c37699700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c30072260 con 0x7f6c300fdfe0 2024-03-24T10:52:12.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:12.871+0000 7f6c37699700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c30070850 con 0x7f6c300fce70 2024-03-24T10:52:12.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:12.871+0000 7f6c34c34700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c300fdfe0 0x7f6c30070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:52:12.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:12.871+0000 7f6c34c34700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c300fdfe0 0x7f6c30070310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35386/0 (socket says 172.21.15.94:35386) 2024-03-24T10:52:12.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:12.871+0000 7f6c34c34700 1 -- 172.21.15.94:0/266193613 learned_addr learned my addr 172.21.15.94:0/266193613 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:52:12.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:12.871+0000 7f6c35435700 1 --2- 172.21.15.94:0/266193613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c300fce70 0x7f6c30071c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:52:12.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:12.871+0000 7f6c34c34700 1 -- 172.21.15.94:0/266193613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c300fce70 msgr2=0x7f6c30071c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:12.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:12.871+0000 7f6c34c34700 1 --2- 172.21.15.94:0/266193613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c300fce70 0x7f6c30071c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:12.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:12.871+0000 7f6c34c34700 1 -- 172.21.15.94:0/266193613 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c2c005040 con 0x7f6c300fdfe0 2024-03-24T10:52:12.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:12.872+0000 7f6c34c34700 1 --2- 172.21.15.94:0/266193613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c300fdfe0 0x7f6c30070310 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6c200041d0 tx=0x7f6c2000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:52:12.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:12.872+0000 7f6c25ffb700 1 -- 172.21.15.94:0/266193613 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c20007cc0 con 0x7f6c300fdfe0 2024-03-24T10:52:12.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:12.872+0000 7f6c37699700 1 -- 172.21.15.94:0/266193613 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c30070aa0 con 0x7f6c300fdfe0 2024-03-24T10:52:12.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:12.872+0000 7f6c25ffb700 1 -- 172.21.15.94:0/266193613 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6c2000f040 con 0x7f6c300fdfe0 2024-03-24T10:52:12.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:12.872+0000 7f6c25ffb700 1 -- 172.21.15.94:0/266193613 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c2000ca50 con 0x7f6c300fdfe0 2024-03-24T10:52:12.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:12.872+0000 7f6c37699700 1 -- 172.21.15.94:0/266193613 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c30071060 con 0x7f6c300fdfe0 2024-03-24T10:52:12.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:12.873+0000 7f6c37699700 1 -- 172.21.15.94:0/266193613 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c3004c710 con 0x7f6c300fdfe0 2024-03-24T10:52:12.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:12.873+0000 7f6c25ffb700 1 -- 172.21.15.94:0/266193613 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f6c2000ccc0 con 0x7f6c300fdfe0 2024-03-24T10:52:12.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:12.874+0000 7f6c25ffb700 1 --2- 172.21.15.94:0/266193613 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f6c1c06d7e0 0x7f6c1c06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:52:12.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:12.874+0000 7f6c25ffb700 1 -- 172.21.15.94:0/266193613 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f6c2008c950 con 0x7f6c300fdfe0 2024-03-24T10:52:12.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:12.874+0000 7f6c35435700 1 --2- 172.21.15.94:0/266193613 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f6c1c06d7e0 0x7f6c1c06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:52:12.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:12.874+0000 7f6c35435700 1 --2- 172.21.15.94:0/266193613 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f6c1c06d7e0 0x7f6c1c06fc90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c000c00 tx=0x7f6c2c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:52:12.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:12.878+0000 7f6c25ffb700 1 -- 172.21.15.94:0/266193613 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6c20059d20 con 0x7f6c300fdfe0 2024-03-24T10:52:13.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.090+0000 7f6c37699700 1 -- 172.21.15.94:0/266193613 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c30102340 con 0x7f6c1c06d7e0 2024-03-24T10:52:13.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.091+0000 7f6c25ffb700 1 -- 172.21.15.94:0/266193613 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f6c30102340 con 0x7f6c1c06d7e0 2024-03-24T10:52:13.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.093+0000 7f6c37699700 1 -- 172.21.15.94:0/266193613 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f6c1c06d7e0 msgr2=0x7f6c1c06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:13.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.093+0000 7f6c37699700 1 --2- 172.21.15.94:0/266193613 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f6c1c06d7e0 0x7f6c1c06fc90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c000c00 tx=0x7f6c2c009e30 comp rx=0 tx=0).stop 2024-03-24T10:52:13.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.093+0000 7f6c37699700 1 -- 172.21.15.94:0/266193613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c300fdfe0 msgr2=0x7f6c30070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:13.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.093+0000 7f6c37699700 1 --2- 172.21.15.94:0/266193613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c300fdfe0 0x7f6c30070310 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6c200041d0 tx=0x7f6c2000b040 comp rx=0 tx=0).stop 2024-03-24T10:52:13.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.093+0000 7f6c37699700 1 -- 172.21.15.94:0/266193613 shutdown_connections 2024-03-24T10:52:13.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.093+0000 7f6c37699700 1 --2- 172.21.15.94:0/266193613 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f6c1c06d7e0 0x7f6c1c06fc90 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:13.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.093+0000 7f6c37699700 1 --2- 172.21.15.94:0/266193613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c300fce70 0x7f6c30071c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:13.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.093+0000 7f6c37699700 1 --2- 172.21.15.94:0/266193613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c300fdfe0 0x7f6c30070310 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:13.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.093+0000 7f6c37699700 1 -- 172.21.15.94:0/266193613 >> 172.21.15.94:0/266193613 conn(0x7f6c300f8550 msgr2=0x7f6c300f9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:52:13.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.093+0000 7f6c37699700 1 -- 172.21.15.94:0/266193613 shutdown_connections 2024-03-24T10:52:13.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.093+0000 7f6c37699700 1 -- 172.21.15.94:0/266193613 wait complete. 2024-03-24T10:52:13.104 INFO:teuthology.orchestra.run.smithi094.stdout:true 2024-03-24T10:52:13.124 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:12 smithi096 podman[98256]: 2024-03-24 10:52:12.765274289 +0000 UTC m=+0.017980387 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-24T10:52:13.125 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:12 smithi096 podman[98256]: 2024-03-24 10:52:12.87262905 +0000 UTC m=+0.125335135 container create 90b01b43196975a3790d04daefad53fc2179e9c367482ccf10af97a45a2f42fc (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6) 2024-03-24T10:52:13.125 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:13 smithi096 podman[98256]: 2024-03-24 10:52:13.125538737 +0000 UTC m=+0.378244832 container init 90b01b43196975a3790d04daefad53fc2179e9c367482ccf10af97a45a2f42fc (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-03-24T10:52:13.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.195+0000 7fb2abc6e700 1 -- 172.21.15.94:0/421641904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2a40fbcd0 msgr2=0x7fb2a40fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:13.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.195+0000 7fb2abc6e700 1 --2- 172.21.15.94:0/421641904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2a40fbcd0 0x7fb2a40fc0e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0009ce0 tx=0x7fb2a0004d10 comp rx=0 tx=0).stop 2024-03-24T10:52:13.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.195+0000 7fb2abc6e700 1 -- 172.21.15.94:0/421641904 shutdown_connections 2024-03-24T10:52:13.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.195+0000 7fb2abc6e700 1 --2- 172.21.15.94:0/421641904 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2a40fcf80 0x7fb2a40fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:13.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.195+0000 7fb2abc6e700 1 --2- 172.21.15.94:0/421641904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2a40fbcd0 0x7fb2a40fc0e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:13.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.195+0000 7fb2abc6e700 1 -- 172.21.15.94:0/421641904 >> 172.21.15.94:0/421641904 conn(0x7fb2a40f7430 msgr2=0x7fb2a40f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:52:13.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.195+0000 7fb2abc6e700 1 -- 172.21.15.94:0/421641904 shutdown_connections 2024-03-24T10:52:13.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.195+0000 7fb2abc6e700 1 -- 172.21.15.94:0/421641904 wait complete. 2024-03-24T10:52:13.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.196+0000 7fb2abc6e700 1 Processor -- start 2024-03-24T10:52:13.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.196+0000 7fb2abc6e700 1 -- start start 2024-03-24T10:52:13.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.196+0000 7fb2abc6e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2a40fbcd0 0x7fb2a4110890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:52:13.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.196+0000 7fb2abc6e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2a40fcf80 0x7fb2a4112de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:52:13.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.196+0000 7fb2abc6e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a4066350 con 0x7fb2a40fcf80 2024-03-24T10:52:13.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.196+0000 7fb2abc6e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a4066490 con 0x7fb2a40fbcd0 2024-03-24T10:52:13.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.196+0000 7fb2a9209700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2a40fcf80 0x7fb2a4112de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:52:13.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.196+0000 7fb2a9a0a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2a40fbcd0 0x7fb2a4110890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:52:13.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.196+0000 7fb2a9a0a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2a40fbcd0 0x7fb2a4110890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.94:34918/0 (socket says 172.21.15.94:34918) 2024-03-24T10:52:13.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.196+0000 7fb2a9a0a700 1 -- 172.21.15.94:0/2457521788 learned_addr learned my addr 172.21.15.94:0/2457521788 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:52:13.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.196+0000 7fb2a9209700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2a40fcf80 0x7fb2a4112de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35400/0 (socket says 172.21.15.94:35400) 2024-03-24T10:52:13.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.196+0000 7fb2a9209700 1 -- 172.21.15.94:0/2457521788 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2a40fbcd0 msgr2=0x7fb2a4110890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:13.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.196+0000 7fb2a9209700 1 --2- 172.21.15.94:0/2457521788 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2a40fbcd0 0x7fb2a4110890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:13.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.196+0000 7fb2a9209700 1 -- 172.21.15.94:0/2457521788 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a0009940 con 0x7fb2a40fcf80 2024-03-24T10:52:13.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.196+0000 7fb2a9a0a700 1 --2- 172.21.15.94:0/2457521788 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2a40fbcd0 0x7fb2a4110890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T10:52:13.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.197+0000 7fb2a9209700 1 --2- 172.21.15.94:0/2457521788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2a40fcf80 0x7fb2a4112de0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb294002820 tx=0x7fb294004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:52:13.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.197+0000 7fb29a7fc700 1 -- 172.21.15.94:0/2457521788 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb294007cc0 con 0x7fb2a40fcf80 2024-03-24T10:52:13.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.197+0000 7fb29a7fc700 1 -- 172.21.15.94:0/2457521788 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb29400f040 con 0x7fb2a40fcf80 2024-03-24T10:52:13.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.197+0000 7fb2abc6e700 1 -- 172.21.15.94:0/2457521788 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2a4113380 con 0x7fb2a40fcf80 2024-03-24T10:52:13.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.197+0000 7fb2abc6e700 1 -- 172.21.15.94:0/2457521788 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2a4113940 con 0x7fb2a40fcf80 2024-03-24T10:52:13.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.197+0000 7fb29a7fc700 1 -- 172.21.15.94:0/2457521788 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb294013940 con 0x7fb2a40fcf80 2024-03-24T10:52:13.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.198+0000 7fb2abc6e700 1 -- 172.21.15.94:0/2457521788 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2a405f5f0 con 0x7fb2a40fcf80 2024-03-24T10:52:13.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.198+0000 7fb29a7fc700 1 -- 172.21.15.94:0/2457521788 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fb294013b40 con 0x7fb2a40fcf80 2024-03-24T10:52:13.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.199+0000 7fb29a7fc700 1 --2- 172.21.15.94:0/2457521788 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fb29006d7e0 0x7fb29006fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:52:13.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.199+0000 7fb29a7fc700 1 -- 172.21.15.94:0/2457521788 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fb29408cc40 con 0x7fb2a40fcf80 2024-03-24T10:52:13.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.199+0000 7fb2a9a0a700 1 --2- 172.21.15.94:0/2457521788 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fb29006d7e0 0x7fb29006fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:52:13.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.199+0000 7fb2a9a0a700 1 --2- 172.21.15.94:0/2457521788 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fb29006d7e0 0x7fb29006fc90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb2a000b740 tx=0x7fb2a0004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:52:13.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.203+0000 7fb29a7fc700 1 -- 172.21.15.94:0/2457521788 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb2940564a0 con 0x7fb2a40fcf80 2024-03-24T10:52:13.420 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:13 smithi096 ceph-mon[84258]: failed to encode map e71 with expected crc 2024-03-24T10:52:13.420 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:13 smithi096 ceph-mon[84258]: osdmap e71: 6 total, 5 up, 6 in 2024-03-24T10:52:13.421 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:13 smithi096 ceph-mon[84258]: failed to encode map e71 with expected crc 2024-03-24T10:52:13.421 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:13 smithi096 ceph-mon[84258]: failed to encode map e71 with expected crc 2024-03-24T10:52:13.421 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:13 smithi096 ceph-mon[84258]: failed to encode map e71 with expected crc 2024-03-24T10:52:13.421 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:13 smithi096 podman[98256]: 2024-03-24 10:52:13.128757622 +0000 UTC m=+0.381463713 container start 90b01b43196975a3790d04daefad53fc2179e9c367482ccf10af97a45a2f42fc (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T10:52:13.421 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:13 smithi096 podman[98256]: 2024-03-24 10:52:13.225429606 +0000 UTC m=+0.478135703 container attach 90b01b43196975a3790d04daefad53fc2179e9c367482ccf10af97a45a2f42fc (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-24T10:52:13.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.425+0000 7fb2abc6e700 1 -- 172.21.15.94:0/2457521788 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb2a4101290 con 0x7fb29006d7e0 2024-03-24T10:52:13.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.426+0000 7fb29a7fc700 1 -- 172.21.15.94:0/2457521788 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fb2a4101290 con 0x7fb29006d7e0 2024-03-24T10:52:13.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.428+0000 7fb2abc6e700 1 -- 172.21.15.94:0/2457521788 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fb29006d7e0 msgr2=0x7fb29006fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:13.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.428+0000 7fb2abc6e700 1 --2- 172.21.15.94:0/2457521788 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fb29006d7e0 0x7fb29006fc90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb2a000b740 tx=0x7fb2a0004d10 comp rx=0 tx=0).stop 2024-03-24T10:52:13.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.428+0000 7fb2abc6e700 1 -- 172.21.15.94:0/2457521788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2a40fcf80 msgr2=0x7fb2a4112de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:13.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.428+0000 7fb2abc6e700 1 --2- 172.21.15.94:0/2457521788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2a40fcf80 0x7fb2a4112de0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb294002820 tx=0x7fb294004570 comp rx=0 tx=0).stop 2024-03-24T10:52:13.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.428+0000 7fb2abc6e700 1 -- 172.21.15.94:0/2457521788 shutdown_connections 2024-03-24T10:52:13.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.428+0000 7fb2abc6e700 1 --2- 172.21.15.94:0/2457521788 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fb29006d7e0 0x7fb29006fc90 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:13.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.428+0000 7fb2abc6e700 1 --2- 172.21.15.94:0/2457521788 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2a40fbcd0 0x7fb2a4110890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:13.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.428+0000 7fb2abc6e700 1 --2- 172.21.15.94:0/2457521788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2a40fcf80 0x7fb2a4112de0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:13.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.428+0000 7fb2abc6e700 1 -- 172.21.15.94:0/2457521788 >> 172.21.15.94:0/2457521788 conn(0x7fb2a40f7430 msgr2=0x7fb2a40f8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:52:13.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.428+0000 7fb2abc6e700 1 -- 172.21.15.94:0/2457521788 shutdown_connections 2024-03-24T10:52:13.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.428+0000 7fb2abc6e700 1 -- 172.21.15.94:0/2457521788 wait complete. 2024-03-24T10:52:13.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:13 smithi094 ceph-mon[93957]: failed to encode map e71 with expected crc 2024-03-24T10:52:13.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:13 smithi094 ceph-mon[93957]: osdmap e71: 6 total, 5 up, 6 in 2024-03-24T10:52:13.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:13 smithi094 ceph-mon[93957]: failed to encode map e71 with expected crc 2024-03-24T10:52:13.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:13 smithi094 ceph-mon[93957]: failed to encode map e71 with expected crc 2024-03-24T10:52:13.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:13 smithi094 ceph-mon[93957]: failed to encode map e71 with expected crc 2024-03-24T10:52:13.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.529+0000 7f24bfddf700 1 -- 172.21.15.94:0/1074283949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24b80fe4f0 msgr2=0x7f24b80fe900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:13.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.529+0000 7f24bfddf700 1 --2- 172.21.15.94:0/1074283949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24b80fe4f0 0x7f24b80fe900 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f24b4004940 tx=0x7f24b4009e30 comp rx=0 tx=0).stop 2024-03-24T10:52:13.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.529+0000 7f24bfddf700 1 -- 172.21.15.94:0/1074283949 shutdown_connections 2024-03-24T10:52:13.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.529+0000 7f24bfddf700 1 --2- 172.21.15.94:0/1074283949 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24b80fee40 0x7f24b80faf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:13.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.529+0000 7f24bfddf700 1 --2- 172.21.15.94:0/1074283949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24b80fe4f0 0x7f24b80fe900 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:13.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.529+0000 7f24bfddf700 1 -- 172.21.15.94:0/1074283949 >> 172.21.15.94:0/1074283949 conn(0x7f24b80f6bd0 msgr2=0x7f24b80f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:52:13.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.530+0000 7f24bfddf700 1 -- 172.21.15.94:0/1074283949 shutdown_connections 2024-03-24T10:52:13.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.530+0000 7f24bfddf700 1 -- 172.21.15.94:0/1074283949 wait complete. 2024-03-24T10:52:13.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.530+0000 7f24bfddf700 1 Processor -- start 2024-03-24T10:52:13.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.530+0000 7f24bfddf700 1 -- start start 2024-03-24T10:52:13.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.530+0000 7f24bfddf700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24b80fe4f0 0x7f24b810d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:52:13.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.530+0000 7f24bfddf700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24b80fee40 0x7f24b810de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:52:13.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.530+0000 7f24bfddf700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b810e490 con 0x7f24b80fe4f0 2024-03-24T10:52:13.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.530+0000 7f24bfddf700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b810e600 con 0x7f24b80fee40 2024-03-24T10:52:13.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.530+0000 7f24bdb7b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24b80fe4f0 0x7f24b810d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:52:13.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.530+0000 7f24bdb7b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24b80fe4f0 0x7f24b810d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35404/0 (socket says 172.21.15.94:35404) 2024-03-24T10:52:13.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.530+0000 7f24bdb7b700 1 -- 172.21.15.94:0/1363804660 learned_addr learned my addr 172.21.15.94:0/1363804660 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:52:13.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.530+0000 7f24bd37a700 1 --2- 172.21.15.94:0/1363804660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24b80fee40 0x7f24b810de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:52:13.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.531+0000 7f24bdb7b700 1 -- 172.21.15.94:0/1363804660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24b80fee40 msgr2=0x7f24b810de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:13.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.531+0000 7f24bdb7b700 1 --2- 172.21.15.94:0/1363804660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24b80fee40 0x7f24b810de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:13.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.531+0000 7f24bdb7b700 1 -- 172.21.15.94:0/1363804660 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b4005040 con 0x7f24b80fe4f0 2024-03-24T10:52:13.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.531+0000 7f24bd37a700 1 --2- 172.21.15.94:0/1363804660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24b80fee40 0x7f24b810de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T10:52:13.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.531+0000 7f24bdb7b700 1 --2- 172.21.15.94:0/1363804660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24b80fe4f0 0x7f24b810d950 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f24b4009150 tx=0x7f24b4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:52:13.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.531+0000 7f24ae7fc700 1 -- 172.21.15.94:0/1363804660 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24b400e040 con 0x7f24b80fe4f0 2024-03-24T10:52:13.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.531+0000 7f24bfddf700 1 -- 172.21.15.94:0/1363804660 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24b8113000 con 0x7f24b80fe4f0 2024-03-24T10:52:13.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.531+0000 7f24ae7fc700 1 -- 172.21.15.94:0/1363804660 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f24b4004290 con 0x7f24b80fe4f0 2024-03-24T10:52:13.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.531+0000 7f24ae7fc700 1 -- 172.21.15.94:0/1363804660 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24b401c630 con 0x7f24b80fe4f0 2024-03-24T10:52:13.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.531+0000 7f24bfddf700 1 -- 172.21.15.94:0/1363804660 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24b81135c0 con 0x7f24b80fe4f0 2024-03-24T10:52:13.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.532+0000 7f24bfddf700 1 -- 172.21.15.94:0/1363804660 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24b804c710 con 0x7f24b80fe4f0 2024-03-24T10:52:13.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.533+0000 7f24ae7fc700 1 -- 172.21.15.94:0/1363804660 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f24b400bbf0 con 0x7f24b80fe4f0 2024-03-24T10:52:13.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.533+0000 7f24ae7fc700 1 --2- 172.21.15.94:0/1363804660 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f24a406d7e0 0x7f24a406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:52:13.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.533+0000 7f24bd37a700 1 --2- 172.21.15.94:0/1363804660 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f24a406d7e0 0x7f24a406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:52:13.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.533+0000 7f24ae7fc700 1 -- 172.21.15.94:0/1363804660 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f24b408daf0 con 0x7f24b80fe4f0 2024-03-24T10:52:13.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.533+0000 7f24bd37a700 1 --2- 172.21.15.94:0/1363804660 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f24a406d7e0 0x7f24a406fc90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f24a8010520 tx=0x7f24a800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:52:13.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.538+0000 7f24ae7fc700 1 -- 172.21.15.94:0/1363804660 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f24b405ae90 con 0x7f24b80fe4f0 2024-03-24T10:52:13.736 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:13 smithi096 podman[98256]: 2024-03-24 10:52:13.421380868 +0000 UTC m=+0.674086959 container died 90b01b43196975a3790d04daefad53fc2179e9c367482ccf10af97a45a2f42fc (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-24T10:52:13.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.741+0000 7f24bfddf700 1 -- 172.21.15.94:0/1363804660 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f24b81025c0 con 0x7f24a406d7e0 2024-03-24T10:52:13.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.750+0000 7f24ae7fc700 1 -- 172.21.15.94:0/1363804660 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f24b81025c0 con 0x7f24a406d7e0 2024-03-24T10:52:13.751 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T10:52:13.751 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.smithi094 smithi094 *:9093,9094 running (5m) 61s ago 14m 24.5M - 0.25.0 c8568f914cd2 5c583af60554 2024-03-24T10:52:13.751 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi094 smithi094 running (2m) 61s ago 14m 6651k - 19.0.0-1855-g21b3bd67 553463a90fd5 2b0897df571d 2024-03-24T10:52:13.752 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi096 smithi096 running (2m) 28s ago 13m 6669k - 19.0.0-1855-g21b3bd67 553463a90fd5 1759a9ff2805 2024-03-24T10:52:13.752 INFO:teuthology.orchestra.run.smithi094.stdout:grafana.smithi094 smithi094 *:3000 running (5m) 61s ago 13m 78.5M - 9.4.7 954c08fa6188 d3d425bf1b7c 2024-03-24T10:52:13.752 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.efiqte smithi094 running (8m) 61s ago 8m 13.7M - 17.2.7-726-g9c4e0c34 837124da804f f0df0a056c21 2024-03-24T10:52:13.752 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.llefgu smithi094 running (8m) 61s ago 8m 109M - 17.2.7-726-g9c4e0c34 837124da804f 52a9771dc8e0 2024-03-24T10:52:13.752 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi096.egbmoq smithi096 running (8m) 28s ago 8m 154M - 17.2.7-726-g9c4e0c34 837124da804f 1cfc930dbab0 2024-03-24T10:52:13.752 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi096.jtebhh smithi096 running (8m) 28s ago 8m 16.8M - 17.2.7-726-g9c4e0c34 837124da804f 4ff3451a6f6e 2024-03-24T10:52:13.752 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi094.yaethu smithi094 *:8443,9283,8765 running (4m) 61s ago 15m 500M - 19.0.0-1855-g21b3bd67 553463a90fd5 d3adab76b18f 2024-03-24T10:52:13.752 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi096.gfonnu smithi096 *:8443,9283,8765 running (4m) 28s ago 12m 449M - 19.0.0-1855-g21b3bd67 553463a90fd5 22937a059eb8 2024-03-24T10:52:13.752 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi094 smithi094 running (3m) 61s ago 16m 48.8M 2048M 19.0.0-1855-g21b3bd67 553463a90fd5 c779d611e29a 2024-03-24T10:52:13.752 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi096 smithi096 running (3m) 28s ago 12m 35.4M 2048M 19.0.0-1855-g21b3bd67 553463a90fd5 ed9bd0bd57f6 2024-03-24T10:52:13.752 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi094 smithi094 *:9100 running (5m) 61s ago 14m 18.7M - 1.5.0 0da6a335fe13 ca67b53c64e2 2024-03-24T10:52:13.752 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi096 smithi096 *:9100 running (4m) 28s ago 12m 16.7M - 1.5.0 0da6a335fe13 65137a7a7549 2024-03-24T10:52:13.752 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (2m) 61s ago 12m 100M 1557M 19.0.0-1855-g21b3bd67 553463a90fd5 ec57ede0c9e4 2024-03-24T10:52:13.752 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi094 running (102s) 61s ago 11m 106M 1557M 19.0.0-1855-g21b3bd67 553463a90fd5 c5b8b2bbf8b5 2024-03-24T10:52:13.752 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi094 running (67s) 61s ago 11m 14.9M 1557M 19.0.0-1855-g21b3bd67 553463a90fd5 c961dcdc746f 2024-03-24T10:52:13.753 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi096 running (31s) 28s ago 10m 12.6M 2581M 19.0.0-1855-g21b3bd67 553463a90fd5 6323fdc6e6ca 2024-03-24T10:52:13.753 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi096 running (10m) 28s ago 10m 258M 2581M 17.2.7-726-g9c4e0c34 837124da804f a63d7c9e705f 2024-03-24T10:52:13.753 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi096 running (9m) 28s ago 9m 241M 2581M 17.2.7-726-g9c4e0c34 837124da804f f56fad2959c3 2024-03-24T10:52:13.753 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.smithi094 smithi094 *:9095 running (4m) 61s ago 13m 55.8M - 2.43.0 a07b618ecd1d 6ce091a0c326 2024-03-24T10:52:13.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.752+0000 7f24bfddf700 1 -- 172.21.15.94:0/1363804660 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f24a406d7e0 msgr2=0x7f24a406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:13.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.752+0000 7f24bfddf700 1 --2- 172.21.15.94:0/1363804660 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f24a406d7e0 0x7f24a406fc90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f24a8010520 tx=0x7f24a800b040 comp rx=0 tx=0).stop 2024-03-24T10:52:13.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.753+0000 7f24bfddf700 1 -- 172.21.15.94:0/1363804660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24b80fe4f0 msgr2=0x7f24b810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:13.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.753+0000 7f24bfddf700 1 --2- 172.21.15.94:0/1363804660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24b80fe4f0 0x7f24b810d950 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f24b4009150 tx=0x7f24b4009e30 comp rx=0 tx=0).stop 2024-03-24T10:52:13.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.753+0000 7f24bfddf700 1 -- 172.21.15.94:0/1363804660 shutdown_connections 2024-03-24T10:52:13.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.753+0000 7f24bfddf700 1 --2- 172.21.15.94:0/1363804660 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f24a406d7e0 0x7f24a406fc90 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:13.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.753+0000 7f24bfddf700 1 --2- 172.21.15.94:0/1363804660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24b80fe4f0 0x7f24b810d950 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:13.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.753+0000 7f24bfddf700 1 --2- 172.21.15.94:0/1363804660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24b80fee40 0x7f24b810de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:13.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.753+0000 7f24bfddf700 1 -- 172.21.15.94:0/1363804660 >> 172.21.15.94:0/1363804660 conn(0x7f24b80f6bd0 msgr2=0x7f24b80f8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:52:13.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.753+0000 7f24bfddf700 1 -- 172.21.15.94:0/1363804660 shutdown_connections 2024-03-24T10:52:13.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.753+0000 7f24bfddf700 1 -- 172.21.15.94:0/1363804660 wait complete. 2024-03-24T10:52:13.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.853+0000 7f48f50e6700 1 -- 172.21.15.94:0/3469893665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48f0066b70 msgr2=0x7f48f00ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:13.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.853+0000 7f48f50e6700 1 --2- 172.21.15.94:0/3469893665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48f0066b70 0x7f48f00ff9c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f48e0004940 tx=0x7f48e0009e30 comp rx=0 tx=0).stop 2024-03-24T10:52:13.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.853+0000 7f48f50e6700 1 -- 172.21.15.94:0/3469893665 shutdown_connections 2024-03-24T10:52:13.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.853+0000 7f48f50e6700 1 --2- 172.21.15.94:0/3469893665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48f00fff00 0x7f48f0102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:13.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.853+0000 7f48f50e6700 1 --2- 172.21.15.94:0/3469893665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48f0066b70 0x7f48f00ff9c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:13.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.853+0000 7f48f50e6700 1 -- 172.21.15.94:0/3469893665 >> 172.21.15.94:0/3469893665 conn(0x7f48f00f7410 msgr2=0x7f48f00f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:52:13.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.853+0000 7f48f50e6700 1 -- 172.21.15.94:0/3469893665 shutdown_connections 2024-03-24T10:52:13.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.853+0000 7f48f50e6700 1 -- 172.21.15.94:0/3469893665 wait complete. 2024-03-24T10:52:13.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.853+0000 7f48f50e6700 1 Processor -- start 2024-03-24T10:52:13.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.853+0000 7f48f50e6700 1 -- start start 2024-03-24T10:52:13.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.853+0000 7f48f50e6700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48f0066b70 0x7f48f010da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:52:13.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.854+0000 7f48f50e6700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48f00fff00 0x7f48f010df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:52:13.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.854+0000 7f48f50e6700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f010e580 con 0x7f48f0066b70 2024-03-24T10:52:13.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.854+0000 7f48f50e6700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f010edd0 con 0x7f48f00fff00 2024-03-24T10:52:13.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.854+0000 7f48eed9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48f0066b70 0x7f48f010da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:52:13.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.854+0000 7f48eed9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48f0066b70 0x7f48f010da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35430/0 (socket says 172.21.15.94:35430) 2024-03-24T10:52:13.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.854+0000 7f48eed9d700 1 -- 172.21.15.94:0/362444714 learned_addr learned my addr 172.21.15.94:0/362444714 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:52:13.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.854+0000 7f48e7fff700 1 --2- 172.21.15.94:0/362444714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48f00fff00 0x7f48f010df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:52:13.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.854+0000 7f48eed9d700 1 -- 172.21.15.94:0/362444714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48f00fff00 msgr2=0x7f48f010df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:13.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.854+0000 7f48eed9d700 1 --2- 172.21.15.94:0/362444714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48f00fff00 0x7f48f010df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:13.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.854+0000 7f48eed9d700 1 -- 172.21.15.94:0/362444714 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e0005040 con 0x7f48f0066b70 2024-03-24T10:52:13.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.854+0000 7f48eed9d700 1 --2- 172.21.15.94:0/362444714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48f0066b70 0x7f48f010da40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f48e0000c00 tx=0x7f48e0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:52:13.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.854+0000 7f48e77fe700 1 -- 172.21.15.94:0/362444714 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48e000e040 con 0x7f48f0066b70 2024-03-24T10:52:13.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.854+0000 7f48e77fe700 1 -- 172.21.15.94:0/362444714 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f48e000bbf0 con 0x7f48f0066b70 2024-03-24T10:52:13.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.854+0000 7f48f50e6700 1 -- 172.21.15.94:0/362444714 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48f010f050 con 0x7f48f0066b70 2024-03-24T10:52:13.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.854+0000 7f48e77fe700 1 -- 172.21.15.94:0/362444714 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48e001c630 con 0x7f48f0066b70 2024-03-24T10:52:13.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.854+0000 7f48f50e6700 1 -- 172.21.15.94:0/362444714 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48f010f590 con 0x7f48f0066b70 2024-03-24T10:52:13.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.856+0000 7f48f50e6700 1 -- 172.21.15.94:0/362444714 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48f005f5f0 con 0x7f48f0066b70 2024-03-24T10:52:13.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.856+0000 7f48e77fe700 1 -- 172.21.15.94:0/362444714 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f48e001c790 con 0x7f48f0066b70 2024-03-24T10:52:13.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.856+0000 7f48e77fe700 1 --2- 172.21.15.94:0/362444714 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f48dc06d7e0 0x7f48dc06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:52:13.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.856+0000 7f48e77fe700 1 -- 172.21.15.94:0/362444714 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f48e008e110 con 0x7f48f0066b70 2024-03-24T10:52:13.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.857+0000 7f48e7fff700 1 --2- 172.21.15.94:0/362444714 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f48dc06d7e0 0x7f48dc06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:52:13.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.857+0000 7f48e7fff700 1 --2- 172.21.15.94:0/362444714 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f48dc06d7e0 0x7f48dc06fc90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f48d8007fd0 tx=0x7f48d8009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:52:13.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:13.862+0000 7f48e77fe700 1 -- 172.21.15.94:0/362444714 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f48e005b710 con 0x7f48f0066b70 2024-03-24T10:52:14.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.139+0000 7f48f50e6700 1 -- 172.21.15.94:0/362444714 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f48f010f8c0 con 0x7f48f0066b70 2024-03-24T10:52:14.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.139+0000 7f48e77fe700 1 -- 172.21.15.94:0/362444714 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f48e005b2a0 con 0x7f48f0066b70 2024-03-24T10:52:14.140 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-03-24T10:52:14.140 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": { 2024-03-24T10:52:14.140 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 2 2024-03-24T10:52:14.140 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:52:14.140 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": { 2024-03-24T10:52:14.140 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 2 2024-03-24T10:52:14.140 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:52:14.141 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": { 2024-03-24T10:52:14.141 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 1, 2024-03-24T10:52:14.141 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 4 2024-03-24T10:52:14.141 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:52:14.141 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-03-24T10:52:14.141 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-24T10:52:14.141 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:52:14.141 INFO:teuthology.orchestra.run.smithi094.stdout: "overall": { 2024-03-24T10:52:14.141 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 5, 2024-03-24T10:52:14.141 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 8 2024-03-24T10:52:14.141 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-03-24T10:52:14.141 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-03-24T10:52:14.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.142+0000 7f48f50e6700 1 -- 172.21.15.94:0/362444714 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f48dc06d7e0 msgr2=0x7f48dc06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:14.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.142+0000 7f48f50e6700 1 --2- 172.21.15.94:0/362444714 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f48dc06d7e0 0x7f48dc06fc90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f48d8007fd0 tx=0x7f48d8009040 comp rx=0 tx=0).stop 2024-03-24T10:52:14.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.142+0000 7f48f50e6700 1 -- 172.21.15.94:0/362444714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48f0066b70 msgr2=0x7f48f010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:14.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.142+0000 7f48f50e6700 1 --2- 172.21.15.94:0/362444714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48f0066b70 0x7f48f010da40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f48e0000c00 tx=0x7f48e0009e30 comp rx=0 tx=0).stop 2024-03-24T10:52:14.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.142+0000 7f48f50e6700 1 -- 172.21.15.94:0/362444714 shutdown_connections 2024-03-24T10:52:14.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.142+0000 7f48f50e6700 1 --2- 172.21.15.94:0/362444714 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f48dc06d7e0 0x7f48dc06fc90 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:14.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.142+0000 7f48f50e6700 1 --2- 172.21.15.94:0/362444714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48f0066b70 0x7f48f010da40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:14.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.142+0000 7f48f50e6700 1 --2- 172.21.15.94:0/362444714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48f00fff00 0x7f48f010df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:14.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.142+0000 7f48f50e6700 1 -- 172.21.15.94:0/362444714 >> 172.21.15.94:0/362444714 conn(0x7f48f00f7410 msgr2=0x7f48f00f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:52:14.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.142+0000 7f48f50e6700 1 -- 172.21.15.94:0/362444714 shutdown_connections 2024-03-24T10:52:14.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.142+0000 7f48f50e6700 1 -- 172.21.15.94:0/362444714 wait complete. 2024-03-24T10:52:14.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.241+0000 7f13ce1c0700 1 -- 172.21.15.94:0/449588195 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13c80fce70 msgr2=0x7f13c80fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:14.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.241+0000 7f13ce1c0700 1 --2- 172.21.15.94:0/449588195 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13c80fce70 0x7f13c80fd280 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f13c0009ce0 tx=0x7f13c0004d10 comp rx=0 tx=0).stop 2024-03-24T10:52:14.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.241+0000 7f13ce1c0700 1 -- 172.21.15.94:0/449588195 shutdown_connections 2024-03-24T10:52:14.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.241+0000 7f13ce1c0700 1 --2- 172.21.15.94:0/449588195 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13c80fdfe0 0x7f13c80fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:14.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.241+0000 7f13ce1c0700 1 --2- 172.21.15.94:0/449588195 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13c80fce70 0x7f13c80fd280 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:14.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.241+0000 7f13ce1c0700 1 -- 172.21.15.94:0/449588195 >> 172.21.15.94:0/449588195 conn(0x7f13c80f8550 msgr2=0x7f13c80fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:52:14.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.241+0000 7f13ce1c0700 1 -- 172.21.15.94:0/449588195 shutdown_connections 2024-03-24T10:52:14.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.241+0000 7f13ce1c0700 1 -- 172.21.15.94:0/449588195 wait complete. 2024-03-24T10:52:14.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.242+0000 7f13ce1c0700 1 Processor -- start 2024-03-24T10:52:14.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.242+0000 7f13ce1c0700 1 -- start start 2024-03-24T10:52:14.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.242+0000 7f13ce1c0700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13c80fce70 0x7f13c8071c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:52:14.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.242+0000 7f13ce1c0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13c80fdfe0 0x7f13c8070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:52:14.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.242+0000 7f13ce1c0700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c8072260 con 0x7f13c80fdfe0 2024-03-24T10:52:14.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.242+0000 7f13ce1c0700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c8070850 con 0x7f13c80fce70 2024-03-24T10:52:14.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.242+0000 7f13c6ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13c80fdfe0 0x7f13c8070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:52:14.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.242+0000 7f13c6ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13c80fdfe0 0x7f13c8070310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35446/0 (socket says 172.21.15.94:35446) 2024-03-24T10:52:14.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.242+0000 7f13c6ffd700 1 -- 172.21.15.94:0/2539298716 learned_addr learned my addr 172.21.15.94:0/2539298716 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:52:14.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.242+0000 7f13c77fe700 1 --2- 172.21.15.94:0/2539298716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13c80fce70 0x7f13c8071c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:52:14.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.242+0000 7f13c6ffd700 1 -- 172.21.15.94:0/2539298716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13c80fce70 msgr2=0x7f13c8071c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:14.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.242+0000 7f13c6ffd700 1 --2- 172.21.15.94:0/2539298716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13c80fce70 0x7f13c8071c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:14.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.242+0000 7f13c6ffd700 1 -- 172.21.15.94:0/2539298716 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c0009940 con 0x7f13c80fdfe0 2024-03-24T10:52:14.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.242+0000 7f13c6ffd700 1 --2- 172.21.15.94:0/2539298716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13c80fdfe0 0x7f13c8070310 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f13b0002a50 tx=0x7f13b00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:52:14.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.243+0000 7f13bffff700 1 -- 172.21.15.94:0/2539298716 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13b0007cc0 con 0x7f13c80fdfe0 2024-03-24T10:52:14.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.243+0000 7f13bffff700 1 -- 172.21.15.94:0/2539298716 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f13b000f040 con 0x7f13c80fdfe0 2024-03-24T10:52:14.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.243+0000 7f13ce1c0700 1 -- 172.21.15.94:0/2539298716 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13c8070b00 con 0x7f13c80fdfe0 2024-03-24T10:52:14.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.243+0000 7f13bffff700 1 -- 172.21.15.94:0/2539298716 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13b000da30 con 0x7f13c80fdfe0 2024-03-24T10:52:14.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.243+0000 7f13ce1c0700 1 -- 172.21.15.94:0/2539298716 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13c80710c0 con 0x7f13c80fdfe0 2024-03-24T10:52:14.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.244+0000 7f13ce1c0700 1 -- 172.21.15.94:0/2539298716 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13c804c710 con 0x7f13c80fdfe0 2024-03-24T10:52:14.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.244+0000 7f13bffff700 1 -- 172.21.15.94:0/2539298716 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f13b000db90 con 0x7f13c80fdfe0 2024-03-24T10:52:14.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.245+0000 7f13bffff700 1 --2- 172.21.15.94:0/2539298716 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f13b406d7e0 0x7f13b406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:52:14.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.245+0000 7f13bffff700 1 -- 172.21.15.94:0/2539298716 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f13b008ccc0 con 0x7f13c80fdfe0 2024-03-24T10:52:14.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.245+0000 7f13c77fe700 1 --2- 172.21.15.94:0/2539298716 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f13b406d7e0 0x7f13b406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:52:14.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.245+0000 7f13c77fe700 1 --2- 172.21.15.94:0/2539298716 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f13b406d7e0 0x7f13b406fc90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f13c0015040 tx=0x7f13c0004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:52:14.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.249+0000 7f13bffff700 1 -- 172.21.15.94:0/2539298716 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f13b0059fe0 con 0x7f13c80fdfe0 2024-03-24T10:52:14.325 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:13 smithi096 podman[98282]: 2024-03-24 10:52:13.995431002 +0000 UTC m=+0.563998587 container remove 90b01b43196975a3790d04daefad53fc2179e9c367482ccf10af97a45a2f42fc (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T10:52:14.325 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:14 smithi096 systemd[1]: ceph-386110d4-e9ca-11ee-95cd-87774f69a715@osd.4.service: Deactivated successfully. 2024-03-24T10:52:14.325 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:14 smithi096 systemd[1]: Stopped Ceph osd.4 for 386110d4-e9ca-11ee-95cd-87774f69a715. 2024-03-24T10:52:14.326 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:14 smithi096 systemd[1]: ceph-386110d4-e9ca-11ee-95cd-87774f69a715@osd.4.service: Consumed 1min 2.183s CPU time. 2024-03-24T10:52:14.326 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:14 smithi096 systemd[1]: Starting Ceph osd.4 for 386110d4-e9ca-11ee-95cd-87774f69a715... 2024-03-24T10:52:14.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.490+0000 7f13ce1c0700 1 -- 172.21.15.94:0/2539298716 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f13c81058d0 con 0x7f13c80fdfe0 2024-03-24T10:52:14.491 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:14 smithi094 ceph-mon[93957]: from='client.34312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:52:14.491 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:14 smithi094 ceph-mon[93957]: pgmap v134: 65 pgs: 8 active+undersized+degraded, 2 stale+active+clean, 10 active+undersized, 45 active+clean; 196 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 14/219 objects degraded (6.393%) 2024-03-24T10:52:14.491 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:14 smithi094 ceph-mon[93957]: Health check failed: Degraded data redundancy: 14/219 objects degraded (6.393%), 8 pgs degraded (PG_DEGRADED) 2024-03-24T10:52:14.491 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:14 smithi094 ceph-mon[93957]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:52:14.491 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:14 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/362444714' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:52:14.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.490+0000 7f13bffff700 1 -- 172.21.15.94:0/2539298716 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1903 (secure 0 0 0) 0x7f13b0059b70 con 0x7f13c80fdfe0 2024-03-24T10:52:14.492 INFO:teuthology.orchestra.run.smithi094.stdout:e18 2024-03-24T10:52:14.492 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T10:52:14.492 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:52:14.492 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-03-24T10:52:14.492 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:52:14.492 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-03-24T10:52:14.492 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-03-24T10:52:14.492 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 18 2024-03-24T10:52:14.492 INFO:teuthology.orchestra.run.smithi094.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-24T10:52:14.492 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-03-24T10:43:39.435738+0000 2024-03-24T10:52:14.492 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-03-24T10:43:58.133901+0000 2024-03-24T10:52:14.492 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-03-24T10:52:14.492 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-03-24T10:52:14.493 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 60 2024-03-24T10:52:14.493 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-03-24T10:52:14.493 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-03-24T10:52:14.493 INFO:teuthology.orchestra.run.smithi094.stdout:max_xattr_size 65536 2024-03-24T10:52:14.493 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-03-24T10:52:14.493 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-03-24T10:52:14.493 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 46 2024-03-24T10:52:14.493 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:52:14.493 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-03-24T10:52:14.493 INFO:teuthology.orchestra.run.smithi094.stdout:in 0 2024-03-24T10:52:14.493 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=24251} 2024-03-24T10:52:14.493 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-03-24T10:52:14.493 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-03-24T10:52:14.493 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-03-24T10:52:14.493 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-03-24T10:52:14.494 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-03-24T10:52:14.494 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-03-24T10:52:14.494 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-03-24T10:52:14.494 INFO:teuthology.orchestra.run.smithi094.stdout:bal_rank_mask -1 2024-03-24T10:52:14.494 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 1 2024-03-24T10:52:14.494 INFO:teuthology.orchestra.run.smithi094.stdout:qdb_cluster leader: 0 members: 2024-03-24T10:52:14.494 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi096.egbmoq{0:24251} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.96:6824/1539752208,v1:172.21.15.96:6825/1539752208] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:52:14.494 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.llefgu{0:14500} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.94:6828/126762980,v1:172.21.15.94:6829/126762980] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:52:14.494 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:52:14.494 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:52:14.494 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-03-24T10:52:14.494 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:52:14.494 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.efiqte{-1:14512} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.94:6826/1636528860,v1:172.21.15.94:6827/1636528860] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:52:14.494 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi096.jtebhh{-1:24261} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.96:6826/1321087930,v1:172.21.15.96:6827/1321087930] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:52:14.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.492+0000 7f13ce1c0700 1 -- 172.21.15.94:0/2539298716 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f13b406d7e0 msgr2=0x7f13b406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:14.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.492+0000 7f13ce1c0700 1 --2- 172.21.15.94:0/2539298716 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f13b406d7e0 0x7f13b406fc90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f13c0015040 tx=0x7f13c0004d10 comp rx=0 tx=0).stop 2024-03-24T10:52:14.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.492+0000 7f13ce1c0700 1 -- 172.21.15.94:0/2539298716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13c80fdfe0 msgr2=0x7f13c8070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:14.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.492+0000 7f13ce1c0700 1 --2- 172.21.15.94:0/2539298716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13c80fdfe0 0x7f13c8070310 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f13b0002a50 tx=0x7f13b00046b0 comp rx=0 tx=0).stop 2024-03-24T10:52:14.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.492+0000 7f13ce1c0700 1 -- 172.21.15.94:0/2539298716 shutdown_connections 2024-03-24T10:52:14.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.492+0000 7f13ce1c0700 1 --2- 172.21.15.94:0/2539298716 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f13b406d7e0 0x7f13b406fc90 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:14.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.492+0000 7f13ce1c0700 1 --2- 172.21.15.94:0/2539298716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13c80fce70 0x7f13c8071c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:14.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.492+0000 7f13ce1c0700 1 --2- 172.21.15.94:0/2539298716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13c80fdfe0 0x7f13c8070310 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:14.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.492+0000 7f13ce1c0700 1 -- 172.21.15.94:0/2539298716 >> 172.21.15.94:0/2539298716 conn(0x7f13c80f8550 msgr2=0x7f13c80f9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:52:14.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.492+0000 7f13ce1c0700 1 -- 172.21.15.94:0/2539298716 shutdown_connections 2024-03-24T10:52:14.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.493+0000 7f13ce1c0700 1 -- 172.21.15.94:0/2539298716 wait complete. 2024-03-24T10:52:14.496 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 18 2024-03-24T10:52:14.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.594+0000 7fe90626a700 1 -- 172.21.15.94:0/1797606941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9000fdfe0 msgr2=0x7fe9000fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:14.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.594+0000 7fe90626a700 1 --2- 172.21.15.94:0/1797606941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9000fdfe0 0x7fe9000fe450 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0004940 tx=0x7fe8f0009e30 comp rx=0 tx=0).stop 2024-03-24T10:52:14.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.594+0000 7fe90626a700 1 -- 172.21.15.94:0/1797606941 shutdown_connections 2024-03-24T10:52:14.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.594+0000 7fe90626a700 1 --2- 172.21.15.94:0/1797606941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9000fdfe0 0x7fe9000fe450 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:14.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.594+0000 7fe90626a700 1 --2- 172.21.15.94:0/1797606941 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9000fce70 0x7fe9000fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:14.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.594+0000 7fe90626a700 1 -- 172.21.15.94:0/1797606941 >> 172.21.15.94:0/1797606941 conn(0x7fe9000f8550 msgr2=0x7fe9000fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:52:14.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.594+0000 7fe90626a700 1 -- 172.21.15.94:0/1797606941 shutdown_connections 2024-03-24T10:52:14.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.594+0000 7fe90626a700 1 -- 172.21.15.94:0/1797606941 wait complete. 2024-03-24T10:52:14.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.595+0000 7fe90626a700 1 Processor -- start 2024-03-24T10:52:14.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.595+0000 7fe90626a700 1 -- start start 2024-03-24T10:52:14.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.595+0000 7fe90626a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9000fce70 0x7fe90010d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:52:14.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.595+0000 7fe90626a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9000fdfe0 0x7fe90010ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:52:14.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.595+0000 7fe90626a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90010e3d0 con 0x7fe9000fdfe0 2024-03-24T10:52:14.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.595+0000 7fe90626a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90010e540 con 0x7fe9000fce70 2024-03-24T10:52:14.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.595+0000 7fe8ff7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9000fdfe0 0x7fe90010ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:52:14.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.595+0000 7fe8fffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9000fce70 0x7fe90010d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:52:14.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.595+0000 7fe8ff7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9000fdfe0 0x7fe90010ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35464/0 (socket says 172.21.15.94:35464) 2024-03-24T10:52:14.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.595+0000 7fe8ff7fe700 1 -- 172.21.15.94:0/1122435035 learned_addr learned my addr 172.21.15.94:0/1122435035 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:52:14.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.596+0000 7fe8fffff700 1 -- 172.21.15.94:0/1122435035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9000fdfe0 msgr2=0x7fe90010ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:14.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.596+0000 7fe8fffff700 1 --2- 172.21.15.94:0/1122435035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9000fdfe0 0x7fe90010ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:14.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.596+0000 7fe8fffff700 1 -- 172.21.15.94:0/1122435035 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f0005040 con 0x7fe9000fce70 2024-03-24T10:52:14.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.596+0000 7fe8fffff700 1 --2- 172.21.15.94:0/1122435035 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9000fce70 0x7fe90010d890 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4002a50 tx=0x7fe8f4004600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:52:14.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.596+0000 7fe8fcff9700 1 -- 172.21.15.94:0/1122435035 <== mon.1 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8f4007cc0 con 0x7fe9000fce70 2024-03-24T10:52:14.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.596+0000 7fe8fcff9700 1 -- 172.21.15.94:0/1122435035 <== mon.1 v2:172.21.15.96:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe8f400f040 con 0x7fe9000fce70 2024-03-24T10:52:14.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.596+0000 7fe90626a700 1 -- 172.21.15.94:0/1122435035 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe900112fa0 con 0x7fe9000fce70 2024-03-24T10:52:14.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.596+0000 7fe8fcff9700 1 -- 172.21.15.94:0/1122435035 <== mon.1 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8f4009530 con 0x7fe9000fce70 2024-03-24T10:52:14.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.596+0000 7fe90626a700 1 -- 172.21.15.94:0/1122435035 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9001134d0 con 0x7fe9000fce70 2024-03-24T10:52:14.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.597+0000 7fe90626a700 1 -- 172.21.15.94:0/1122435035 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe90004c710 con 0x7fe9000fce70 2024-03-24T10:52:14.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.597+0000 7fe8fcff9700 1 -- 172.21.15.94:0/1122435035 <== mon.1 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fe8f40097f0 con 0x7fe9000fce70 2024-03-24T10:52:14.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.598+0000 7fe8fcff9700 1 --2- 172.21.15.94:0/1122435035 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fe8e806d7e0 0x7fe8e806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:52:14.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.598+0000 7fe8ff7fe700 1 --2- 172.21.15.94:0/1122435035 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fe8e806d7e0 0x7fe8e806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:52:14.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.598+0000 7fe8fcff9700 1 -- 172.21.15.94:0/1122435035 <== mon.1 v2:172.21.15.96:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fe8f408cc40 con 0x7fe9000fce70 2024-03-24T10:52:14.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.598+0000 7fe8ff7fe700 1 --2- 172.21.15.94:0/1122435035 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fe8e806d7e0 0x7fe8e806fc90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0004f90 tx=0x7fe8f0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:52:14.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.603+0000 7fe8fcff9700 1 -- 172.21.15.94:0/1122435035 <== mon.1 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fe8f4059f00 con 0x7fe9000fce70 2024-03-24T10:52:14.666 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:14 smithi096 ceph-mon[84258]: from='client.34312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:52:14.666 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:14 smithi096 ceph-mon[84258]: pgmap v134: 65 pgs: 8 active+undersized+degraded, 2 stale+active+clean, 10 active+undersized, 45 active+clean; 196 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 14/219 objects degraded (6.393%) 2024-03-24T10:52:14.666 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:14 smithi096 ceph-mon[84258]: Health check failed: Degraded data redundancy: 14/219 objects degraded (6.393%), 8 pgs degraded (PG_DEGRADED) 2024-03-24T10:52:14.666 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:14 smithi096 ceph-mon[84258]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:52:14.666 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:14 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/362444714' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:52:14.666 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:14 smithi096 podman[98382]: 2024-03-24 10:52:14.341565125 +0000 UTC m=+0.017539980 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-24T10:52:14.666 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:14 smithi096 podman[98382]: 2024-03-24 10:52:14.455120374 +0000 UTC m=+0.131095269 container create e14dd53a53c28a0790f92a050b615b40faaa854261778fc444003082d57ee130 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4-activate, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True) 2024-03-24T10:52:14.666 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:14 smithi096 podman[98382]: 2024-03-24 10:52:14.666482148 +0000 UTC m=+0.342457001 container init e14dd53a53c28a0790f92a050b615b40faaa854261778fc444003082d57ee130 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4-activate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, ceph=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T10:52:14.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.816+0000 7fe90626a700 1 -- 172.21.15.94:0/1122435035 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe9001022b0 con 0x7fe8e806d7e0 2024-03-24T10:52:14.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.818+0000 7fe8fcff9700 1 -- 172.21.15.94:0/1122435035 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fe9001022b0 con 0x7fe8e806d7e0 2024-03-24T10:52:14.818 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-03-24T10:52:14.819 INFO:teuthology.orchestra.run.smithi094.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919", 2024-03-24T10:52:14.819 INFO:teuthology.orchestra.run.smithi094.stdout: "in_progress": true, 2024-03-24T10:52:14.819 INFO:teuthology.orchestra.run.smithi094.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T10:52:14.819 INFO:teuthology.orchestra.run.smithi094.stdout: "services_complete": [ 2024-03-24T10:52:14.819 INFO:teuthology.orchestra.run.smithi094.stdout: "mon", 2024-03-24T10:52:14.819 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-03-24T10:52:14.819 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr" 2024-03-24T10:52:14.819 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-03-24T10:52:14.819 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": "10/21 daemons upgraded", 2024-03-24T10:52:14.819 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "Currently upgrading osd daemons", 2024-03-24T10:52:14.819 INFO:teuthology.orchestra.run.smithi094.stdout: "is_paused": false 2024-03-24T10:52:14.819 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-03-24T10:52:14.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.820+0000 7fe90626a700 1 -- 172.21.15.94:0/1122435035 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fe8e806d7e0 msgr2=0x7fe8e806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:14.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.820+0000 7fe90626a700 1 --2- 172.21.15.94:0/1122435035 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fe8e806d7e0 0x7fe8e806fc90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0004f90 tx=0x7fe8f0009e30 comp rx=0 tx=0).stop 2024-03-24T10:52:14.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.820+0000 7fe90626a700 1 -- 172.21.15.94:0/1122435035 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9000fce70 msgr2=0x7fe90010d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:14.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.820+0000 7fe90626a700 1 --2- 172.21.15.94:0/1122435035 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9000fce70 0x7fe90010d890 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4002a50 tx=0x7fe8f4004600 comp rx=0 tx=0).stop 2024-03-24T10:52:14.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.820+0000 7fe90626a700 1 -- 172.21.15.94:0/1122435035 shutdown_connections 2024-03-24T10:52:14.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.820+0000 7fe90626a700 1 --2- 172.21.15.94:0/1122435035 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fe8e806d7e0 0x7fe8e806fc90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:14.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.820+0000 7fe90626a700 1 --2- 172.21.15.94:0/1122435035 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9000fce70 0x7fe90010d890 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:14.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.820+0000 7fe90626a700 1 --2- 172.21.15.94:0/1122435035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9000fdfe0 0x7fe90010ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:14.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.820+0000 7fe90626a700 1 -- 172.21.15.94:0/1122435035 >> 172.21.15.94:0/1122435035 conn(0x7fe9000f8550 msgr2=0x7fe9000f9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:52:14.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.820+0000 7fe90626a700 1 -- 172.21.15.94:0/1122435035 shutdown_connections 2024-03-24T10:52:14.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.820+0000 7fe90626a700 1 -- 172.21.15.94:0/1122435035 wait complete. 2024-03-24T10:52:14.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.920+0000 7f3c3b533700 1 -- 172.21.15.94:0/1948318067 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c340fcf90 msgr2=0x7f3c340fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:14.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.920+0000 7f3c3b533700 1 --2- 172.21.15.94:0/1948318067 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c340fcf90 0x7f3c340fd400 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3c24004940 tx=0x7f3c24009e30 comp rx=0 tx=0).stop 2024-03-24T10:52:14.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.920+0000 7f3c3b533700 1 -- 172.21.15.94:0/1948318067 shutdown_connections 2024-03-24T10:52:14.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.920+0000 7f3c3b533700 1 --2- 172.21.15.94:0/1948318067 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c340fcf90 0x7f3c340fd400 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:14.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.920+0000 7f3c3b533700 1 --2- 172.21.15.94:0/1948318067 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c340fbce0 0x7f3c340fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:14.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.920+0000 7f3c3b533700 1 -- 172.21.15.94:0/1948318067 >> 172.21.15.94:0/1948318067 conn(0x7f3c340f7440 msgr2=0x7f3c340f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:52:14.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.920+0000 7f3c3b533700 1 -- 172.21.15.94:0/1948318067 shutdown_connections 2024-03-24T10:52:14.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.920+0000 7f3c3b533700 1 -- 172.21.15.94:0/1948318067 wait complete. 2024-03-24T10:52:14.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.921+0000 7f3c3b533700 1 Processor -- start 2024-03-24T10:52:14.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.921+0000 7f3c3b533700 1 -- start start 2024-03-24T10:52:14.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.921+0000 7f3c3b533700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c340fbce0 0x7f3c340ffc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:52:14.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.921+0000 7f3c3b533700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c340fcf90 0x7f3c340fe2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:52:14.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.921+0000 7f3c3b533700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c34100220 con 0x7f3c340fcf90 2024-03-24T10:52:14.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.921+0000 7f3c3b533700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c340fe810 con 0x7f3c340fbce0 2024-03-24T10:52:14.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.921+0000 7f3c38ace700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c340fcf90 0x7f3c340fe2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:52:14.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.921+0000 7f3c38ace700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c340fcf90 0x7f3c340fe2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35476/0 (socket says 172.21.15.94:35476) 2024-03-24T10:52:14.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.921+0000 7f3c38ace700 1 -- 172.21.15.94:0/949790777 learned_addr learned my addr 172.21.15.94:0/949790777 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:52:14.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.921+0000 7f3c392cf700 1 --2- 172.21.15.94:0/949790777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c340fbce0 0x7f3c340ffc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:52:14.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.921+0000 7f3c38ace700 1 -- 172.21.15.94:0/949790777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c340fbce0 msgr2=0x7f3c340ffc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:14.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.921+0000 7f3c38ace700 1 --2- 172.21.15.94:0/949790777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c340fbce0 0x7f3c340ffc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:14.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.922+0000 7f3c38ace700 1 -- 172.21.15.94:0/949790777 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c24005040 con 0x7f3c340fcf90 2024-03-24T10:52:14.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.922+0000 7f3c38ace700 1 --2- 172.21.15.94:0/949790777 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c340fcf90 0x7f3c340fe2d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3c2400b3b0 tx=0x7f3c24009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:52:14.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.922+0000 7f3c29ffb700 1 -- 172.21.15.94:0/949790777 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c2400e040 con 0x7f3c340fcf90 2024-03-24T10:52:14.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.922+0000 7f3c29ffb700 1 -- 172.21.15.94:0/949790777 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3c2400bbf0 con 0x7f3c340fcf90 2024-03-24T10:52:14.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.922+0000 7f3c29ffb700 1 -- 172.21.15.94:0/949790777 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c2401c630 con 0x7f3c340fcf90 2024-03-24T10:52:14.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.922+0000 7f3c3b533700 1 -- 172.21.15.94:0/949790777 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c340fea60 con 0x7f3c340fcf90 2024-03-24T10:52:14.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.922+0000 7f3c3b533700 1 -- 172.21.15.94:0/949790777 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c340fefd0 con 0x7f3c340fcf90 2024-03-24T10:52:14.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.923+0000 7f3c3b533700 1 -- 172.21.15.94:0/949790777 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c34101b20 con 0x7f3c340fcf90 2024-03-24T10:52:14.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.923+0000 7f3c29ffb700 1 -- 172.21.15.94:0/949790777 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f3c2400bd60 con 0x7f3c340fcf90 2024-03-24T10:52:14.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.924+0000 7f3c29ffb700 1 --2- 172.21.15.94:0/949790777 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f3c2006d7e0 0x7f3c2006fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:52:14.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.924+0000 7f3c29ffb700 1 -- 172.21.15.94:0/949790777 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f3c2408daf0 con 0x7f3c340fcf90 2024-03-24T10:52:14.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.924+0000 7f3c392cf700 1 --2- 172.21.15.94:0/949790777 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f3c2006d7e0 0x7f3c2006fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:52:14.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.924+0000 7f3c392cf700 1 --2- 172.21.15.94:0/949790777 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f3c2006d7e0 0x7f3c2006fc90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3c30004f40 tx=0x7f3c3000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:52:14.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:14.929+0000 7f3c29ffb700 1 -- 172.21.15.94:0/949790777 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3c2405ae90 con 0x7f3c340fcf90 2024-03-24T10:52:15.075 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:14 smithi096 podman[98382]: 2024-03-24 10:52:14.670132334 +0000 UTC m=+0.346107192 container start e14dd53a53c28a0790f92a050b615b40faaa854261778fc444003082d57ee130 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T10:52:15.075 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:14 smithi096 podman[98382]: 2024-03-24 10:52:14.791129881 +0000 UTC m=+0.467104734 container attach e14dd53a53c28a0790f92a050b615b40faaa854261778fc444003082d57ee130 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4-activate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD) 2024-03-24T10:52:15.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:15.205+0000 7f3c3b533700 1 -- 172.21.15.94:0/949790777 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3c3404c710 con 0x7f3c340fcf90 2024-03-24T10:52:15.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:15.205+0000 7f3c29ffb700 1 -- 172.21.15.94:0/949790777 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+713 (secure 0 0 0) 0x7f3c2405aa20 con 0x7f3c340fcf90 2024-03-24T10:52:15.206 INFO:teuthology.orchestra.run.smithi094.stdout:HEALTH_WARN 1 osds down; Degraded data redundancy: 14/219 objects degraded (6.393%), 8 pgs degraded 2024-03-24T10:52:15.206 INFO:teuthology.orchestra.run.smithi094.stdout:[WRN] OSD_DOWN: 1 osds down 2024-03-24T10:52:15.206 INFO:teuthology.orchestra.run.smithi094.stdout: osd.4 (root=default,host=smithi096) is down 2024-03-24T10:52:15.206 INFO:teuthology.orchestra.run.smithi094.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 14/219 objects degraded (6.393%), 8 pgs degraded 2024-03-24T10:52:15.206 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.4 is active+undersized+degraded, acting [1,0] 2024-03-24T10:52:15.206 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.6 is active+undersized+degraded, acting [1,3] 2024-03-24T10:52:15.207 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.9 is active+undersized+degraded, acting [1,0] 2024-03-24T10:52:15.207 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.a is active+undersized+degraded, acting [1,3] 2024-03-24T10:52:15.207 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.f is active+undersized+degraded, acting [0,5] 2024-03-24T10:52:15.207 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.13 is active+undersized+degraded, acting [0,2] 2024-03-24T10:52:15.207 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.19 is active+undersized+degraded, acting [0,2] 2024-03-24T10:52:15.207 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.1f is active+undersized+degraded, acting [0,3] 2024-03-24T10:52:15.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:15.207+0000 7f3c3b533700 1 -- 172.21.15.94:0/949790777 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f3c2006d7e0 msgr2=0x7f3c2006fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:15.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:15.207+0000 7f3c3b533700 1 --2- 172.21.15.94:0/949790777 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f3c2006d7e0 0x7f3c2006fc90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3c30004f40 tx=0x7f3c3000b040 comp rx=0 tx=0).stop 2024-03-24T10:52:15.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:15.207+0000 7f3c3b533700 1 -- 172.21.15.94:0/949790777 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c340fcf90 msgr2=0x7f3c340fe2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:15.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:15.207+0000 7f3c3b533700 1 --2- 172.21.15.94:0/949790777 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c340fcf90 0x7f3c340fe2d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3c2400b3b0 tx=0x7f3c24009e30 comp rx=0 tx=0).stop 2024-03-24T10:52:15.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:15.207+0000 7f3c3b533700 1 -- 172.21.15.94:0/949790777 shutdown_connections 2024-03-24T10:52:15.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:15.207+0000 7f3c3b533700 1 --2- 172.21.15.94:0/949790777 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f3c2006d7e0 0x7f3c2006fc90 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:15.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:15.207+0000 7f3c3b533700 1 --2- 172.21.15.94:0/949790777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c340fbce0 0x7f3c340ffc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:15.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:15.207+0000 7f3c3b533700 1 --2- 172.21.15.94:0/949790777 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c340fcf90 0x7f3c340fe2d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:15.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:15.207+0000 7f3c3b533700 1 -- 172.21.15.94:0/949790777 >> 172.21.15.94:0/949790777 conn(0x7f3c340f7440 msgr2=0x7f3c34066850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:52:15.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:15.207+0000 7f3c3b533700 1 -- 172.21.15.94:0/949790777 shutdown_connections 2024-03-24T10:52:15.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:15.207+0000 7f3c3b533700 1 -- 172.21.15.94:0/949790777 wait complete. 2024-03-24T10:52:15.366 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:15 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4-activate[98397]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-24T10:52:15.367 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:15 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4-activate[98397]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-24T10:52:15.367 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:15 smithi096 bash[98382]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-24T10:52:15.367 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:15 smithi096 bash[98382]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-24T10:52:15.367 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:15 smithi096 bash[98382]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-24T10:52:15.367 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:15 smithi096 bash[98382]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-24T10:52:15.367 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:15 smithi096 bash[98382]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-03-24T10:52:15.367 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:15 smithi096 bash[98382]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-24T10:52:15.367 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:15 smithi096 bash[98382]: --> ceph-volume raw activate successful for osd ID: 4 2024-03-24T10:52:15.367 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:15 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4-activate[98397]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-24T10:52:15.367 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:15 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4-activate[98397]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-24T10:52:15.367 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:15 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4-activate[98397]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-03-24T10:52:15.367 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:15 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4-activate[98397]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-24T10:52:15.367 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:15 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4-activate[98397]: --> ceph-volume raw activate successful for osd ID: 4 2024-03-24T10:52:15.367 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:15 smithi096 podman[98382]: 2024-03-24 10:52:15.135834868 +0000 UTC m=+0.811809715 container died e14dd53a53c28a0790f92a050b615b40faaa854261778fc444003082d57ee130 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4-activate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-24T10:52:15.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:15 smithi094 ceph-mon[93957]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:52:15.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:15 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/2539298716' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T10:52:15.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:15 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/949790777' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T10:52:15.810 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:15 smithi096 ceph-mon[84258]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:52:15.811 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:15 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/2539298716' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T10:52:15.811 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:15 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/949790777' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T10:52:16.075 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:15 smithi096 podman[98527]: 2024-03-24 10:52:15.811364706 +0000 UTC m=+0.668491118 container remove e14dd53a53c28a0790f92a050b615b40faaa854261778fc444003082d57ee130 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4-activate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-24T10:52:16.075 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:16 smithi096 podman[98593]: 2024-03-24 10:52:15.933573744 +0000 UTC m=+0.018272976 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-24T10:52:16.075 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:16 smithi096 podman[98593]: 2024-03-24 10:52:16.045544657 +0000 UTC m=+0.130243889 container create 3e2536bcb263e8614ec90c43a6ef4b6036d4baaf782ca8ebcf2d1bdd13cf00aa (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T10:52:16.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:16 smithi096 ceph-mon[84258]: from='client.44263 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:52:16.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:16 smithi096 ceph-mon[84258]: pgmap v135: 65 pgs: 13 active+undersized+degraded, 1 stale+active+clean, 16 active+undersized, 35 active+clean; 196 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 32/219 objects degraded (14.612%) 2024-03-24T10:52:16.576 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:16 smithi096 podman[98593]: 2024-03-24 10:52:16.264583543 +0000 UTC m=+0.349282782 container init 3e2536bcb263e8614ec90c43a6ef4b6036d4baaf782ca8ebcf2d1bdd13cf00aa (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-24T10:52:16.576 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:16 smithi096 podman[98593]: 2024-03-24 10:52:16.267565747 +0000 UTC m=+0.352264979 container start 3e2536bcb263e8614ec90c43a6ef4b6036d4baaf782ca8ebcf2d1bdd13cf00aa (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-24T10:52:16.576 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:16 smithi096 ceph-osd[98613]: -- 172.21.15.96:0/274918648 <== mon.1 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5634638471e0 con 0x563463818c00 2024-03-24T10:52:16.576 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:16 smithi096 bash[98593]: 3e2536bcb263e8614ec90c43a6ef4b6036d4baaf782ca8ebcf2d1bdd13cf00aa 2024-03-24T10:52:16.576 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:16 smithi096 systemd[1]: Started Ceph osd.4 for 386110d4-e9ca-11ee-95cd-87774f69a715. 2024-03-24T10:52:16.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:16 smithi094 ceph-mon[93957]: from='client.44263 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:52:16.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:16 smithi094 ceph-mon[93957]: pgmap v135: 65 pgs: 13 active+undersized+degraded, 1 stale+active+clean, 16 active+undersized, 35 active+clean; 196 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 32/219 objects degraded (14.612%) 2024-03-24T10:52:17.075 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:16 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4[98609]: 2024-03-24T10:52:16.808+0000 7f0ae916d7c0 -1 Falling back to public interface 2024-03-24T10:52:17.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:17 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:17.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:17 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:17.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:17 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:52:17.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:17 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:17.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:17 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:17.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:17 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:52:18.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:18 smithi094 ceph-mon[93957]: pgmap v136: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 196 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 36/219 objects degraded (16.438%) 2024-03-24T10:52:18.755 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:18 smithi096 ceph-mon[84258]: pgmap v136: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 196 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 36/219 objects degraded (16.438%) 2024-03-24T10:52:20.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:20 smithi094 ceph-mon[93957]: pgmap v137: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 196 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 36/219 objects degraded (16.438%) 2024-03-24T10:52:20.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:20 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:20.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:20 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:20.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:20 smithi096 ceph-mon[84258]: pgmap v137: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 196 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 36/219 objects degraded (16.438%) 2024-03-24T10:52:20.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:20 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:20.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:20 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:21.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:21 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:21.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:21 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:52:21.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:21 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:21.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:21 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:21.575 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:21 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4[98609]: 2024-03-24T10:52:21.382+0000 7f0ae916d7c0 -1 osd.4 0 read_superblock omap replica is missing. 2024-03-24T10:52:21.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:21 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:21.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:21 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:52:21.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:21 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:21.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:21 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:22.215 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:21 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4[98609]: 2024-03-24T10:52:21.952+0000 7f0ae916d7c0 -1 osd.4 69 log_to_monitors true 2024-03-24T10:52:22.544 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:22 smithi096 ceph-mon[84258]: pgmap v138: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 196 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 514 B/s rd, 1 op/s; 36/219 objects degraded (16.438%) 2024-03-24T10:52:22.544 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:22 smithi096 ceph-mon[84258]: Health check update: Degraded data redundancy: 36/219 objects degraded (16.438%), 15 pgs degraded (PG_DEGRADED) 2024-03-24T10:52:22.544 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:22 smithi096 ceph-mon[84258]: from='osd.4 [v2:172.21.15.96:6808/3911504706,v1:172.21.15.96:6809/3911504706]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-24T10:52:22.544 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:22 smithi096 ceph-mon[84258]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-24T10:52:22.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:22 smithi094 ceph-mon[93957]: pgmap v138: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 196 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 514 B/s rd, 1 op/s; 36/219 objects degraded (16.438%) 2024-03-24T10:52:22.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:22 smithi094 ceph-mon[93957]: Health check update: Degraded data redundancy: 36/219 objects degraded (16.438%), 15 pgs degraded (PG_DEGRADED) 2024-03-24T10:52:22.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:22 smithi094 ceph-mon[93957]: from='osd.4 [v2:172.21.15.96:6808/3911504706,v1:172.21.15.96:6809/3911504706]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-24T10:52:22.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:22 smithi094 ceph-mon[93957]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-24T10:52:23.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:23 smithi096 ceph-mon[84258]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-24T10:52:23.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:23 smithi096 ceph-mon[84258]: failed to encode map e72 with expected crc 2024-03-24T10:52:23.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:23 smithi096 ceph-mon[84258]: osdmap e72: 6 total, 5 up, 6 in 2024-03-24T10:52:23.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:23 smithi096 ceph-mon[84258]: from='osd.4 [v2:172.21.15.96:6808/3911504706,v1:172.21.15.96:6809/3911504706]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-03-24T10:52:23.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:23 smithi096 ceph-mon[84258]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-03-24T10:52:23.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:23 smithi096 ceph-mon[84258]: from='osd.4 ' entity='osd.4' 2024-03-24T10:52:23.576 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:52:23 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4[98609]: 2024-03-24T10:52:23.291+0000 7f0ad65bd700 -1 osd.4 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T10:52:23.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:23 smithi094 ceph-mon[93957]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-24T10:52:23.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:23 smithi094 ceph-mon[93957]: failed to encode map e72 with expected crc 2024-03-24T10:52:23.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:23 smithi094 ceph-mon[93957]: osdmap e72: 6 total, 5 up, 6 in 2024-03-24T10:52:23.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:23 smithi094 ceph-mon[93957]: from='osd.4 [v2:172.21.15.96:6808/3911504706,v1:172.21.15.96:6809/3911504706]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-03-24T10:52:23.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:23 smithi094 ceph-mon[93957]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-03-24T10:52:23.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:23 smithi094 ceph-mon[93957]: from='osd.4 ' entity='osd.4' 2024-03-24T10:52:24.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:24 smithi094 ceph-mon[93957]: pgmap v140: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 196 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 511 B/s rd, 1 op/s; 36/219 objects degraded (16.438%) 2024-03-24T10:52:24.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:24 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:24.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:24 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:24.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:24 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:52:24.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:24 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:52:24.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:24 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:24.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:24 smithi094 ceph-mon[93957]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T10:52:24.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:24 smithi094 ceph-mon[93957]: osd.4 [v2:172.21.15.96:6808/3911504706,v1:172.21.15.96:6809/3911504706] boot 2024-03-24T10:52:24.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:24 smithi094 ceph-mon[93957]: osdmap e73: 6 total, 6 up, 6 in 2024-03-24T10:52:24.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:24 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T10:52:24.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:24 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:52:24.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:24 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:52:24.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:24 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:52:24.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:24 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:52:24.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:24 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-24T10:52:24.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:24 smithi096 ceph-mon[84258]: pgmap v140: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 196 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 511 B/s rd, 1 op/s; 36/219 objects degraded (16.438%) 2024-03-24T10:52:24.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:24 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:24.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:24 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:24.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:24 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:52:24.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:24 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:52:24.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:24 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:24.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:24 smithi096 ceph-mon[84258]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T10:52:24.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:24 smithi096 ceph-mon[84258]: osd.4 [v2:172.21.15.96:6808/3911504706,v1:172.21.15.96:6809/3911504706] boot 2024-03-24T10:52:24.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:24 smithi096 ceph-mon[84258]: osdmap e73: 6 total, 6 up, 6 in 2024-03-24T10:52:24.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:24 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T10:52:24.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:24 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:52:24.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:24 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:52:24.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:24 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:52:24.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:24 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:52:24.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:24 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-24T10:52:25.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:25 smithi094 ceph-mon[93957]: failed to encode map e73 with expected crc 2024-03-24T10:52:25.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:25 smithi094 ceph-mon[93957]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-24T10:52:25.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:25 smithi094 ceph-mon[93957]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-03-24T10:52:25.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:25 smithi094 ceph-mon[93957]: osdmap e74: 6 total, 6 up, 6 in 2024-03-24T10:52:25.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:25 smithi096 ceph-mon[84258]: failed to encode map e73 with expected crc 2024-03-24T10:52:25.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:25 smithi096 ceph-mon[84258]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-24T10:52:25.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:25 smithi096 ceph-mon[84258]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-03-24T10:52:25.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:25 smithi096 ceph-mon[84258]: osdmap e74: 6 total, 6 up, 6 in 2024-03-24T10:52:26.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:26 smithi094 ceph-mon[93957]: failed to encode map e74 with expected crc 2024-03-24T10:52:26.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:26 smithi094 ceph-mon[93957]: failed to encode map e74 with expected crc 2024-03-24T10:52:26.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:26 smithi094 ceph-mon[93957]: pgmap v143: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 196 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 36/219 objects degraded (16.438%) 2024-03-24T10:52:26.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:26 smithi096 ceph-mon[84258]: failed to encode map e74 with expected crc 2024-03-24T10:52:26.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:26 smithi096 ceph-mon[84258]: failed to encode map e74 with expected crc 2024-03-24T10:52:26.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:26 smithi096 ceph-mon[84258]: pgmap v143: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 196 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 36/219 objects degraded (16.438%) 2024-03-24T10:52:27.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:27 smithi094 ceph-mon[93957]: pgmap v144: 65 pgs: 14 active+undersized+degraded, 16 active+undersized, 35 active+clean; 196 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 33/219 objects degraded (15.068%) 2024-03-24T10:52:27.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:27 smithi094 ceph-mon[93957]: Health check update: Degraded data redundancy: 33/219 objects degraded (15.068%), 14 pgs degraded (PG_DEGRADED) 2024-03-24T10:52:27.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:27 smithi096 ceph-mon[84258]: pgmap v144: 65 pgs: 14 active+undersized+degraded, 16 active+undersized, 35 active+clean; 196 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 33/219 objects degraded (15.068%) 2024-03-24T10:52:27.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:27 smithi096 ceph-mon[84258]: Health check update: Degraded data redundancy: 33/219 objects degraded (15.068%), 14 pgs degraded (PG_DEGRADED) 2024-03-24T10:52:29.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:29 smithi094 ceph-mon[93957]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 33/219 objects degraded (15.068%), 14 pgs degraded) 2024-03-24T10:52:29.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:29 smithi094 ceph-mon[93957]: Cluster is now healthy 2024-03-24T10:52:29.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:29 smithi096 ceph-mon[84258]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 33/219 objects degraded (15.068%), 14 pgs degraded) 2024-03-24T10:52:29.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:29 smithi096 ceph-mon[84258]: Cluster is now healthy 2024-03-24T10:52:30.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:30 smithi094 ceph-mon[93957]: pgmap v145: 65 pgs: 65 active+clean; 196 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:52:30.813 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:30 smithi096 ceph-mon[84258]: pgmap v145: 65 pgs: 65 active+clean; 196 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:52:32.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:32 smithi094 ceph-mon[93957]: pgmap v146: 65 pgs: 65 active+clean; 196 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-03-24T10:52:32.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:32 smithi096 ceph-mon[84258]: pgmap v146: 65 pgs: 65 active+clean; 196 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-03-24T10:52:34.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:34 smithi094 ceph-mon[93957]: pgmap v147: 65 pgs: 65 active+clean; 196 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 340 B/s rd, 1 op/s 2024-03-24T10:52:34.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:34 smithi096 ceph-mon[84258]: pgmap v147: 65 pgs: 65 active+clean; 196 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 340 B/s rd, 1 op/s 2024-03-24T10:52:36.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:36 smithi094 ceph-mon[93957]: pgmap v148: 65 pgs: 65 active+clean; 196 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 818 B/s rd, 1 op/s 2024-03-24T10:52:36.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:36 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:36.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:36 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:52:36.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:36 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:36.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:36 smithi096 ceph-mon[84258]: pgmap v148: 65 pgs: 65 active+clean; 196 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 818 B/s rd, 1 op/s 2024-03-24T10:52:36.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:36 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:36.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:36 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:52:36.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:36 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:38.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:38 smithi094 ceph-mon[93957]: pgmap v149: 65 pgs: 65 active+clean; 196 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 682 B/s rd, 1 op/s 2024-03-24T10:52:38.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:38 smithi096 ceph-mon[84258]: pgmap v149: 65 pgs: 65 active+clean; 196 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 682 B/s rd, 1 op/s 2024-03-24T10:52:40.535 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:40 smithi096 ceph-mon[84258]: pgmap v150: 65 pgs: 65 active+clean; 196 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-03-24T10:52:40.535 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:40 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-24T10:52:40.535 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:40 smithi096 ceph-mon[84258]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-24T10:52:40.535 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:40 smithi096 ceph-mon[84258]: Upgrade: osd.5 is safe to restart 2024-03-24T10:52:40.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:40 smithi094 ceph-mon[93957]: pgmap v150: 65 pgs: 65 active+clean; 196 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-03-24T10:52:40.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:40 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-24T10:52:40.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:40 smithi094 ceph-mon[93957]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-24T10:52:40.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:40 smithi094 ceph-mon[93957]: Upgrade: osd.5 is safe to restart 2024-03-24T10:52:42.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:42 smithi096 ceph-mon[84258]: Upgrade: Updating osd.5 2024-03-24T10:52:42.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:42 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:42.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:42 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-24T10:52:42.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:42 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:52:42.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:42 smithi096 ceph-mon[84258]: Deploying daemon osd.5 on smithi096 2024-03-24T10:52:42.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:42 smithi096 ceph-mon[84258]: pgmap v151: 65 pgs: 65 active+clean; 196 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:52:42.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:42 smithi094 ceph-mon[93957]: Upgrade: Updating osd.5 2024-03-24T10:52:42.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:42 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:42.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:42 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-24T10:52:42.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:42 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:52:42.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:42 smithi094 ceph-mon[93957]: Deploying daemon osd.5 on smithi096 2024-03-24T10:52:42.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:42 smithi094 ceph-mon[93957]: pgmap v151: 65 pgs: 65 active+clean; 196 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:52:44.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:44 smithi096 ceph-mon[84258]: pgmap v152: 65 pgs: 65 active+clean; 196 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-24T10:52:44.576 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:44 smithi096 systemd[1]: Stopping Ceph osd.5 for 386110d4-e9ca-11ee-95cd-87774f69a715... 2024-03-24T10:52:44.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:44 smithi094 ceph-mon[93957]: pgmap v152: 65 pgs: 65 active+clean; 196 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-24T10:52:45.075 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:44 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5[57301]: 2024-03-24T10:52:44.603+0000 7f2d17193700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T10:52:45.075 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:44 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5[57301]: 2024-03-24T10:52:44.603+0000 7f2d17193700 -1 osd.5 74 *** Got signal Terminated *** 2024-03-24T10:52:45.075 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:44 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5[57301]: 2024-03-24T10:52:44.603+0000 7f2d17193700 -1 osd.5 74 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T10:52:45.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.322+0000 7f68b68ad700 1 -- 172.21.15.94:0/1333427056 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b00fce70 msgr2=0x7f68b00fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:45.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.322+0000 7f68b68ad700 1 --2- 172.21.15.94:0/1333427056 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b00fce70 0x7f68b00fd280 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f68a4004940 tx=0x7f68a4009e30 comp rx=0 tx=0).stop 2024-03-24T10:52:45.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.322+0000 7f68b68ad700 1 -- 172.21.15.94:0/1333427056 shutdown_connections 2024-03-24T10:52:45.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.322+0000 7f68b68ad700 1 --2- 172.21.15.94:0/1333427056 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68b00fdfe0 0x7f68b00fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:45.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.322+0000 7f68b68ad700 1 --2- 172.21.15.94:0/1333427056 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b00fce70 0x7f68b00fd280 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:45.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.322+0000 7f68b68ad700 1 -- 172.21.15.94:0/1333427056 >> 172.21.15.94:0/1333427056 conn(0x7f68b00f8550 msgr2=0x7f68b00fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:52:45.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.322+0000 7f68b68ad700 1 -- 172.21.15.94:0/1333427056 shutdown_connections 2024-03-24T10:52:45.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.322+0000 7f68b68ad700 1 -- 172.21.15.94:0/1333427056 wait complete. 2024-03-24T10:52:45.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.323+0000 7f68b68ad700 1 Processor -- start 2024-03-24T10:52:45.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.323+0000 7f68b68ad700 1 -- start start 2024-03-24T10:52:45.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.323+0000 7f68b68ad700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b00fce70 0x7f68b010d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:52:45.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.323+0000 7f68b68ad700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68b00fdfe0 0x7f68b010de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:52:45.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.323+0000 7f68b68ad700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b010e490 con 0x7f68b00fce70 2024-03-24T10:52:45.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.323+0000 7f68b68ad700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b010e600 con 0x7f68b00fdfe0 2024-03-24T10:52:45.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.323+0000 7f68affff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b00fce70 0x7f68b010d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:52:45.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.323+0000 7f68affff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b00fce70 0x7f68b010d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53946/0 (socket says 172.21.15.94:53946) 2024-03-24T10:52:45.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.323+0000 7f68affff700 1 -- 172.21.15.94:0/12108909 learned_addr learned my addr 172.21.15.94:0/12108909 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:52:45.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.323+0000 7f68af7fe700 1 --2- 172.21.15.94:0/12108909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68b00fdfe0 0x7f68b010de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:52:45.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.324+0000 7f68affff700 1 -- 172.21.15.94:0/12108909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68b00fdfe0 msgr2=0x7f68b010de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:45.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.324+0000 7f68affff700 1 --2- 172.21.15.94:0/12108909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68b00fdfe0 0x7f68b010de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:45.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.324+0000 7f68affff700 1 -- 172.21.15.94:0/12108909 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a4005040 con 0x7f68b00fce70 2024-03-24T10:52:45.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.324+0000 7f68af7fe700 1 --2- 172.21.15.94:0/12108909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68b00fdfe0 0x7f68b010de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T10:52:45.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.324+0000 7f68affff700 1 --2- 172.21.15.94:0/12108909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b00fce70 0x7f68b010d950 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f68a400b3b0 tx=0x7f68a4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:52:45.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.324+0000 7f68acff9700 1 -- 172.21.15.94:0/12108909 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68a400e040 con 0x7f68b00fce70 2024-03-24T10:52:45.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.324+0000 7f68acff9700 1 -- 172.21.15.94:0/12108909 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f68a400bcc0 con 0x7f68b00fce70 2024-03-24T10:52:45.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.324+0000 7f68b68ad700 1 -- 172.21.15.94:0/12108909 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68b0113000 con 0x7f68b00fce70 2024-03-24T10:52:45.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.324+0000 7f68acff9700 1 -- 172.21.15.94:0/12108909 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68a401c630 con 0x7f68b00fce70 2024-03-24T10:52:45.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.324+0000 7f68b68ad700 1 -- 172.21.15.94:0/12108909 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68b01135c0 con 0x7f68b00fce70 2024-03-24T10:52:45.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.325+0000 7f68b68ad700 1 -- 172.21.15.94:0/12108909 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68b004c710 con 0x7f68b00fce70 2024-03-24T10:52:45.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.326+0000 7f68acff9700 1 -- 172.21.15.94:0/12108909 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f68a4013980 con 0x7f68b00fce70 2024-03-24T10:52:45.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.326+0000 7f68acff9700 1 --2- 172.21.15.94:0/12108909 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f689806d7e0 0x7f689806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:52:45.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.326+0000 7f68acff9700 1 -- 172.21.15.94:0/12108909 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f68a408dc10 con 0x7f68b00fce70 2024-03-24T10:52:45.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.326+0000 7f68af7fe700 1 --2- 172.21.15.94:0/12108909 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f689806d7e0 0x7f689806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:52:45.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.326+0000 7f68af7fe700 1 --2- 172.21.15.94:0/12108909 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f689806d7e0 0x7f689806fc90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f68a00041d0 tx=0x7f68a0009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:52:45.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.331+0000 7f68acff9700 1 -- 172.21.15.94:0/12108909 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f68a405af30 con 0x7f68b00fce70 2024-03-24T10:52:45.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.554+0000 7f68b68ad700 1 -- 172.21.15.94:0/12108909 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f68b0102340 con 0x7f689806d7e0 2024-03-24T10:52:45.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.555+0000 7f68acff9700 1 -- 172.21.15.94:0/12108909 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f68b0102340 con 0x7f689806d7e0 2024-03-24T10:52:45.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.557+0000 7f68b68ad700 1 -- 172.21.15.94:0/12108909 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f689806d7e0 msgr2=0x7f689806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:45.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.557+0000 7f68b68ad700 1 --2- 172.21.15.94:0/12108909 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f689806d7e0 0x7f689806fc90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f68a00041d0 tx=0x7f68a0009040 comp rx=0 tx=0).stop 2024-03-24T10:52:45.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.557+0000 7f68b68ad700 1 -- 172.21.15.94:0/12108909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b00fce70 msgr2=0x7f68b010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:45.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.557+0000 7f68b68ad700 1 --2- 172.21.15.94:0/12108909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b00fce70 0x7f68b010d950 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f68a400b3b0 tx=0x7f68a4009e30 comp rx=0 tx=0).stop 2024-03-24T10:52:45.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.557+0000 7f68b68ad700 1 -- 172.21.15.94:0/12108909 shutdown_connections 2024-03-24T10:52:45.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.557+0000 7f68b68ad700 1 --2- 172.21.15.94:0/12108909 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f689806d7e0 0x7f689806fc90 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:45.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.557+0000 7f68b68ad700 1 --2- 172.21.15.94:0/12108909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b00fce70 0x7f68b010d950 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:45.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.557+0000 7f68b68ad700 1 --2- 172.21.15.94:0/12108909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68b00fdfe0 0x7f68b010de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:45.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.557+0000 7f68b68ad700 1 -- 172.21.15.94:0/12108909 >> 172.21.15.94:0/12108909 conn(0x7f68b00f8550 msgr2=0x7f68b00f9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:52:45.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.557+0000 7f68b68ad700 1 -- 172.21.15.94:0/12108909 shutdown_connections 2024-03-24T10:52:45.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.557+0000 7f68b68ad700 1 -- 172.21.15.94:0/12108909 wait complete. 2024-03-24T10:52:45.567 INFO:teuthology.orchestra.run.smithi094.stdout:true 2024-03-24T10:52:45.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.659+0000 7f00932ca700 1 -- 172.21.15.94:0/1455542508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f008c0fcf80 msgr2=0x7f008c0fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:45.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.659+0000 7f00932ca700 1 --2- 172.21.15.94:0/1455542508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f008c0fcf80 0x7f008c0fd3f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0088004940 tx=0x7f0088009e30 comp rx=0 tx=0).stop 2024-03-24T10:52:45.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.659+0000 7f00932ca700 1 -- 172.21.15.94:0/1455542508 shutdown_connections 2024-03-24T10:52:45.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.659+0000 7f00932ca700 1 --2- 172.21.15.94:0/1455542508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f008c0fcf80 0x7f008c0fd3f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:45.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.659+0000 7f00932ca700 1 --2- 172.21.15.94:0/1455542508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f008c0fbcd0 0x7f008c0fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:45.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.659+0000 7f00932ca700 1 -- 172.21.15.94:0/1455542508 >> 172.21.15.94:0/1455542508 conn(0x7f008c0f7430 msgr2=0x7f008c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:52:45.659 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:45 smithi094 ceph-mon[93957]: osd.5 marked itself down and dead 2024-03-24T10:52:45.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.659+0000 7f00932ca700 1 -- 172.21.15.94:0/1455542508 shutdown_connections 2024-03-24T10:52:45.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.659+0000 7f00932ca700 1 -- 172.21.15.94:0/1455542508 wait complete. 2024-03-24T10:52:45.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.660+0000 7f00932ca700 1 Processor -- start 2024-03-24T10:52:45.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.660+0000 7f00932ca700 1 -- start start 2024-03-24T10:52:45.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.660+0000 7f00932ca700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f008c0fbcd0 0x7f008c110700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:52:45.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.660+0000 7f00932ca700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f008c0fcf80 0x7f008c112c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:52:45.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.660+0000 7f00932ca700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008c066370 con 0x7f008c0fcf80 2024-03-24T10:52:45.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.660+0000 7f00932ca700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008c0664b0 con 0x7f008c0fbcd0 2024-03-24T10:52:45.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.660+0000 7f0090865700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f008c0fcf80 0x7f008c112c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:52:45.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.660+0000 7f0090865700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f008c0fcf80 0x7f008c112c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53964/0 (socket says 172.21.15.94:53964) 2024-03-24T10:52:45.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.660+0000 7f0090865700 1 -- 172.21.15.94:0/937740555 learned_addr learned my addr 172.21.15.94:0/937740555 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:52:45.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.660+0000 7f0090865700 1 -- 172.21.15.94:0/937740555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f008c0fbcd0 msgr2=0x7f008c110700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:45.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.660+0000 7f0090865700 1 --2- 172.21.15.94:0/937740555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f008c0fbcd0 0x7f008c110700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:45.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.660+0000 7f0090865700 1 -- 172.21.15.94:0/937740555 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0088005040 con 0x7f008c0fcf80 2024-03-24T10:52:45.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.660+0000 7f0090865700 1 --2- 172.21.15.94:0/937740555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f008c0fcf80 0x7f008c112c50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0088009150 tx=0x7f0088009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:52:45.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.660+0000 7f0081ffb700 1 -- 172.21.15.94:0/937740555 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f008800e040 con 0x7f008c0fcf80 2024-03-24T10:52:45.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.660+0000 7f0081ffb700 1 -- 172.21.15.94:0/937740555 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f008800bcc0 con 0x7f008c0fcf80 2024-03-24T10:52:45.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.660+0000 7f0081ffb700 1 -- 172.21.15.94:0/937740555 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f008801c630 con 0x7f008c0fcf80 2024-03-24T10:52:45.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.660+0000 7f00932ca700 1 -- 172.21.15.94:0/937740555 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f008c1131f0 con 0x7f008c0fcf80 2024-03-24T10:52:45.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.661+0000 7f00932ca700 1 -- 172.21.15.94:0/937740555 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f008c113780 con 0x7f008c0fcf80 2024-03-24T10:52:45.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.662+0000 7f00932ca700 1 -- 172.21.15.94:0/937740555 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f008c05f5f0 con 0x7f008c0fcf80 2024-03-24T10:52:45.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.662+0000 7f0081ffb700 1 -- 172.21.15.94:0/937740555 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f0088013420 con 0x7f008c0fcf80 2024-03-24T10:52:45.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.662+0000 7f0081ffb700 1 --2- 172.21.15.94:0/937740555 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f007806d7e0 0x7f007806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:52:45.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.663+0000 7f0081ffb700 1 -- 172.21.15.94:0/937740555 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f008808db00 con 0x7f008c0fcf80 2024-03-24T10:52:45.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.663+0000 7f0091066700 1 --2- 172.21.15.94:0/937740555 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f007806d7e0 0x7f007806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:52:45.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.663+0000 7f0091066700 1 --2- 172.21.15.94:0/937740555 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f007806d7e0 0x7f007806fc90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f008c065ff0 tx=0x7f007c009ce0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:52:45.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.667+0000 7f0081ffb700 1 -- 172.21.15.94:0/937740555 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f0088057410 con 0x7f008c0fcf80 2024-03-24T10:52:45.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:45 smithi096 ceph-mon[84258]: osd.5 marked itself down and dead 2024-03-24T10:52:45.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.880+0000 7f00932ca700 1 -- 172.21.15.94:0/937740555 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f008c101360 con 0x7f007806d7e0 2024-03-24T10:52:45.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.881+0000 7f0081ffb700 1 -- 172.21.15.94:0/937740555 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f008c101360 con 0x7f007806d7e0 2024-03-24T10:52:45.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.883+0000 7f00932ca700 1 -- 172.21.15.94:0/937740555 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f007806d7e0 msgr2=0x7f007806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:45.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.883+0000 7f00932ca700 1 --2- 172.21.15.94:0/937740555 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f007806d7e0 0x7f007806fc90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f008c065ff0 tx=0x7f007c009ce0 comp rx=0 tx=0).stop 2024-03-24T10:52:45.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.883+0000 7f00932ca700 1 -- 172.21.15.94:0/937740555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f008c0fcf80 msgr2=0x7f008c112c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:45.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.883+0000 7f00932ca700 1 --2- 172.21.15.94:0/937740555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f008c0fcf80 0x7f008c112c50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0088009150 tx=0x7f0088009e30 comp rx=0 tx=0).stop 2024-03-24T10:52:45.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.883+0000 7f00932ca700 1 -- 172.21.15.94:0/937740555 shutdown_connections 2024-03-24T10:52:45.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.883+0000 7f00932ca700 1 --2- 172.21.15.94:0/937740555 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f007806d7e0 0x7f007806fc90 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:45.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.883+0000 7f00932ca700 1 --2- 172.21.15.94:0/937740555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f008c0fbcd0 0x7f008c110700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:45.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.883+0000 7f00932ca700 1 --2- 172.21.15.94:0/937740555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f008c0fcf80 0x7f008c112c50 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:45.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.883+0000 7f00932ca700 1 -- 172.21.15.94:0/937740555 >> 172.21.15.94:0/937740555 conn(0x7f008c0f7430 msgr2=0x7f008c0f8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:52:45.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.883+0000 7f00932ca700 1 -- 172.21.15.94:0/937740555 shutdown_connections 2024-03-24T10:52:45.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.883+0000 7f00932ca700 1 -- 172.21.15.94:0/937740555 wait complete. 2024-03-24T10:52:45.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.983+0000 7f6a5f2db700 1 -- 172.21.15.94:0/2943085352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a580fff00 msgr2=0x7f6a58102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:45.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.983+0000 7f6a5f2db700 1 --2- 172.21.15.94:0/2943085352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a580fff00 0x7f6a58102320 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6a48004940 tx=0x7f6a48009e30 comp rx=0 tx=0).stop 2024-03-24T10:52:45.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.983+0000 7f6a5f2db700 1 -- 172.21.15.94:0/2943085352 shutdown_connections 2024-03-24T10:52:45.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.983+0000 7f6a5f2db700 1 --2- 172.21.15.94:0/2943085352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a580fff00 0x7f6a58102320 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:45.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.983+0000 7f6a5f2db700 1 --2- 172.21.15.94:0/2943085352 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a58066f50 0x7f6a580ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:45.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.983+0000 7f6a5f2db700 1 -- 172.21.15.94:0/2943085352 >> 172.21.15.94:0/2943085352 conn(0x7f6a580f7450 msgr2=0x7f6a580f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:52:45.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.983+0000 7f6a5f2db700 1 -- 172.21.15.94:0/2943085352 shutdown_connections 2024-03-24T10:52:45.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.983+0000 7f6a5f2db700 1 -- 172.21.15.94:0/2943085352 wait complete. 2024-03-24T10:52:45.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.984+0000 7f6a5f2db700 1 Processor -- start 2024-03-24T10:52:45.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.984+0000 7f6a5f2db700 1 -- start start 2024-03-24T10:52:45.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.984+0000 7f6a5f2db700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a58066f50 0x7f6a5810d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:52:45.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.984+0000 7f6a5f2db700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a580fff00 0x7f6a5810de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:52:45.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.984+0000 7f6a5f2db700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5810e480 con 0x7f6a580fff00 2024-03-24T10:52:45.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.984+0000 7f6a5f2db700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5810e5f0 con 0x7f6a58066f50 2024-03-24T10:52:45.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.984+0000 7f6a5c876700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a580fff00 0x7f6a5810de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:52:45.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.984+0000 7f6a5c876700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a580fff00 0x7f6a5810de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53990/0 (socket says 172.21.15.94:53990) 2024-03-24T10:52:45.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.984+0000 7f6a5c876700 1 -- 172.21.15.94:0/143813269 learned_addr learned my addr 172.21.15.94:0/143813269 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:52:45.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.984+0000 7f6a5d077700 1 --2- 172.21.15.94:0/143813269 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a58066f50 0x7f6a5810d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:52:45.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.984+0000 7f6a5c876700 1 -- 172.21.15.94:0/143813269 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a58066f50 msgr2=0x7f6a5810d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:45.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.984+0000 7f6a5c876700 1 --2- 172.21.15.94:0/143813269 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a58066f50 0x7f6a5810d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:45.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.984+0000 7f6a5c876700 1 -- 172.21.15.94:0/143813269 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a48005040 con 0x7f6a580fff00 2024-03-24T10:52:45.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.985+0000 7f6a5c876700 1 --2- 172.21.15.94:0/143813269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a580fff00 0x7f6a5810de80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6a4800b3b0 tx=0x7f6a48009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:52:45.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.985+0000 7f6a4dffb700 1 -- 172.21.15.94:0/143813269 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a4800e040 con 0x7f6a580fff00 2024-03-24T10:52:45.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.985+0000 7f6a4dffb700 1 -- 172.21.15.94:0/143813269 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6a4800bbf0 con 0x7f6a580fff00 2024-03-24T10:52:45.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.985+0000 7f6a5f2db700 1 -- 172.21.15.94:0/143813269 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a58112ff0 con 0x7f6a580fff00 2024-03-24T10:52:45.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.985+0000 7f6a4dffb700 1 -- 172.21.15.94:0/143813269 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a4801c630 con 0x7f6a580fff00 2024-03-24T10:52:45.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.985+0000 7f6a5f2db700 1 -- 172.21.15.94:0/143813269 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a58113580 con 0x7f6a580fff00 2024-03-24T10:52:45.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.986+0000 7f6a5f2db700 1 -- 172.21.15.94:0/143813269 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a5804c710 con 0x7f6a580fff00 2024-03-24T10:52:45.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.987+0000 7f6a4dffb700 1 -- 172.21.15.94:0/143813269 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f6a4800bd60 con 0x7f6a580fff00 2024-03-24T10:52:45.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.987+0000 7f6a4dffb700 1 --2- 172.21.15.94:0/143813269 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f6a4406d7e0 0x7f6a4406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:52:45.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.988+0000 7f6a4dffb700 1 -- 172.21.15.94:0/143813269 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f6a4808d9e0 con 0x7f6a580fff00 2024-03-24T10:52:45.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.988+0000 7f6a5d077700 1 --2- 172.21.15.94:0/143813269 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f6a4406d7e0 0x7f6a4406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:52:45.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.988+0000 7f6a5d077700 1 --2- 172.21.15.94:0/143813269 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f6a4406d7e0 0x7f6a4406fc90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6a54007fd0 tx=0x7f6a54009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:52:45.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:45.992+0000 7f6a4dffb700 1 -- 172.21.15.94:0/143813269 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6a4805ae20 con 0x7f6a580fff00 2024-03-24T10:52:46.130 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:45 smithi096 podman[103048]: 2024-03-24 10:52:45.831232947 +0000 UTC m=+1.341817665 container died f56fad2959c3e00063f4dad21785b00f5e6c6e6973556f3f79ef149ff9a7a483 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5, RELEASE=quincy-9c4e0c3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-24T10:52:46.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.194+0000 7f6a5f2db700 1 -- 172.21.15.94:0/143813269 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a580f8f40 con 0x7f6a4406d7e0 2024-03-24T10:52:46.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.204+0000 7f6a4dffb700 1 -- 172.21.15.94:0/143813269 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f6a580f8f40 con 0x7f6a4406d7e0 2024-03-24T10:52:46.204 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T10:52:46.204 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.smithi094 smithi094 *:9093,9094 running (5m) 93s ago 14m 24.5M - 0.25.0 c8568f914cd2 5c583af60554 2024-03-24T10:52:46.205 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi094 smithi094 running (3m) 93s ago 15m 6651k - 19.0.0-1855-g21b3bd67 553463a90fd5 2b0897df571d 2024-03-24T10:52:46.205 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi096 smithi096 running (3m) 26s ago 13m 6669k - 19.0.0-1855-g21b3bd67 553463a90fd5 1759a9ff2805 2024-03-24T10:52:46.205 INFO:teuthology.orchestra.run.smithi094.stdout:grafana.smithi094 smithi094 *:3000 running (5m) 93s ago 14m 78.5M - 9.4.7 954c08fa6188 d3d425bf1b7c 2024-03-24T10:52:46.205 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.efiqte smithi094 running (9m) 93s ago 9m 13.7M - 17.2.7-726-g9c4e0c34 837124da804f f0df0a056c21 2024-03-24T10:52:46.205 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.llefgu smithi094 running (8m) 93s ago 8m 109M - 17.2.7-726-g9c4e0c34 837124da804f 52a9771dc8e0 2024-03-24T10:52:46.205 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi096.egbmoq smithi096 running (8m) 26s ago 8m 154M - 17.2.7-726-g9c4e0c34 837124da804f 1cfc930dbab0 2024-03-24T10:52:46.205 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi096.jtebhh smithi096 running (8m) 26s ago 8m 17.0M - 17.2.7-726-g9c4e0c34 837124da804f 4ff3451a6f6e 2024-03-24T10:52:46.205 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi094.yaethu smithi094 *:8443,9283,8765 running (5m) 93s ago 16m 500M - 19.0.0-1855-g21b3bd67 553463a90fd5 d3adab76b18f 2024-03-24T10:52:46.205 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi096.gfonnu smithi096 *:8443,9283,8765 running (4m) 26s ago 13m 449M - 19.0.0-1855-g21b3bd67 553463a90fd5 22937a059eb8 2024-03-24T10:52:46.205 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi094 smithi094 running (4m) 93s ago 16m 48.8M 2048M 19.0.0-1855-g21b3bd67 553463a90fd5 c779d611e29a 2024-03-24T10:52:46.205 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi096 smithi096 running (3m) 26s ago 13m 37.4M 2048M 19.0.0-1855-g21b3bd67 553463a90fd5 ed9bd0bd57f6 2024-03-24T10:52:46.205 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi094 smithi094 *:9100 running (5m) 93s ago 15m 18.7M - 1.5.0 0da6a335fe13 ca67b53c64e2 2024-03-24T10:52:46.205 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi096 smithi096 *:9100 running (5m) 26s ago 13m 17.2M - 1.5.0 0da6a335fe13 65137a7a7549 2024-03-24T10:52:46.205 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (2m) 93s ago 12m 100M 1557M 19.0.0-1855-g21b3bd67 553463a90fd5 ec57ede0c9e4 2024-03-24T10:52:46.205 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi094 running (2m) 93s ago 12m 106M 1557M 19.0.0-1855-g21b3bd67 553463a90fd5 c5b8b2bbf8b5 2024-03-24T10:52:46.206 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi094 running (99s) 93s ago 11m 14.9M 1557M 19.0.0-1855-g21b3bd67 553463a90fd5 c961dcdc746f 2024-03-24T10:52:46.206 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi096 running (64s) 26s ago 11m 104M 2581M 19.0.0-1855-g21b3bd67 553463a90fd5 6323fdc6e6ca 2024-03-24T10:52:46.206 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi096 running (30s) 26s ago 10m 12.5M 2581M 19.0.0-1855-g21b3bd67 553463a90fd5 3e2536bcb263 2024-03-24T10:52:46.206 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi096 running (10m) 26s ago 10m 242M 2581M 17.2.7-726-g9c4e0c34 837124da804f f56fad2959c3 2024-03-24T10:52:46.206 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.smithi094 smithi094 *:9095 running (4m) 93s ago 13m 55.8M - 2.43.0 a07b618ecd1d 6ce091a0c326 2024-03-24T10:52:46.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.207+0000 7f6a5f2db700 1 -- 172.21.15.94:0/143813269 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f6a4406d7e0 msgr2=0x7f6a4406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:46.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.207+0000 7f6a5f2db700 1 --2- 172.21.15.94:0/143813269 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f6a4406d7e0 0x7f6a4406fc90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6a54007fd0 tx=0x7f6a54009040 comp rx=0 tx=0).stop 2024-03-24T10:52:46.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.207+0000 7f6a5f2db700 1 -- 172.21.15.94:0/143813269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a580fff00 msgr2=0x7f6a5810de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:46.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.207+0000 7f6a5f2db700 1 --2- 172.21.15.94:0/143813269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a580fff00 0x7f6a5810de80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6a4800b3b0 tx=0x7f6a48009e30 comp rx=0 tx=0).stop 2024-03-24T10:52:46.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.207+0000 7f6a5f2db700 1 -- 172.21.15.94:0/143813269 shutdown_connections 2024-03-24T10:52:46.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.207+0000 7f6a5f2db700 1 --2- 172.21.15.94:0/143813269 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f6a4406d7e0 0x7f6a4406fc90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:46.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.207+0000 7f6a5f2db700 1 --2- 172.21.15.94:0/143813269 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a58066f50 0x7f6a5810d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:46.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.207+0000 7f6a5f2db700 1 --2- 172.21.15.94:0/143813269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a580fff00 0x7f6a5810de80 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:46.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.207+0000 7f6a5f2db700 1 -- 172.21.15.94:0/143813269 >> 172.21.15.94:0/143813269 conn(0x7f6a580f7450 msgr2=0x7f6a580f7820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:52:46.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.207+0000 7f6a5f2db700 1 -- 172.21.15.94:0/143813269 shutdown_connections 2024-03-24T10:52:46.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.207+0000 7f6a5f2db700 1 -- 172.21.15.94:0/143813269 wait complete. 2024-03-24T10:52:46.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.308+0000 7f6275018700 1 -- 172.21.15.94:0/1125208266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62700fcef0 msgr2=0x7f62700fd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:46.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.308+0000 7f6275018700 1 --2- 172.21.15.94:0/1125208266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62700fcef0 0x7f62700fd300 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6268004940 tx=0x7f6268009e30 comp rx=0 tx=0).stop 2024-03-24T10:52:46.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.308+0000 7f6275018700 1 -- 172.21.15.94:0/1125208266 shutdown_connections 2024-03-24T10:52:46.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.308+0000 7f6275018700 1 --2- 172.21.15.94:0/1125208266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62700fe060 0x7f62700fe4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:46.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.308+0000 7f6275018700 1 --2- 172.21.15.94:0/1125208266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62700fcef0 0x7f62700fd300 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:46.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.308+0000 7f6275018700 1 -- 172.21.15.94:0/1125208266 >> 172.21.15.94:0/1125208266 conn(0x7f62700f85b0 msgr2=0x7f62700fa9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:52:46.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.308+0000 7f6275018700 1 -- 172.21.15.94:0/1125208266 shutdown_connections 2024-03-24T10:52:46.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.308+0000 7f6275018700 1 -- 172.21.15.94:0/1125208266 wait complete. 2024-03-24T10:52:46.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.308+0000 7f6275018700 1 Processor -- start 2024-03-24T10:52:46.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.308+0000 7f6275018700 1 -- start start 2024-03-24T10:52:46.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.309+0000 7f6275018700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62700fcef0 0x7f627010da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:52:46.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.309+0000 7f6275018700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62700fe060 0x7f627010dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:52:46.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.309+0000 7f6275018700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627010e5c0 con 0x7f62700fe060 2024-03-24T10:52:46.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.309+0000 7f6275018700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627010edc0 con 0x7f62700fcef0 2024-03-24T10:52:46.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.309+0000 7f626e59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62700fe060 0x7f627010dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:52:46.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.309+0000 7f626e59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62700fe060 0x7f627010dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54006/0 (socket says 172.21.15.94:54006) 2024-03-24T10:52:46.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.309+0000 7f626e59c700 1 -- 172.21.15.94:0/3316950563 learned_addr learned my addr 172.21.15.94:0/3316950563 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:52:46.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.309+0000 7f626ed9d700 1 --2- 172.21.15.94:0/3316950563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62700fcef0 0x7f627010da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:52:46.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.309+0000 7f626e59c700 1 -- 172.21.15.94:0/3316950563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62700fcef0 msgr2=0x7f627010da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:46.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.309+0000 7f626e59c700 1 --2- 172.21.15.94:0/3316950563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62700fcef0 0x7f627010da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:46.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.309+0000 7f626e59c700 1 -- 172.21.15.94:0/3316950563 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6268005040 con 0x7f62700fe060 2024-03-24T10:52:46.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.309+0000 7f626e59c700 1 --2- 172.21.15.94:0/3316950563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62700fe060 0x7f627010dfc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6264001d80 tx=0x7f6264009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:52:46.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.309+0000 7f62577fe700 1 -- 172.21.15.94:0/3316950563 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6264007d10 con 0x7f62700fe060 2024-03-24T10:52:46.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.309+0000 7f62577fe700 1 -- 172.21.15.94:0/3316950563 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6264001e80 con 0x7f62700fe060 2024-03-24T10:52:46.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.309+0000 7f6275018700 1 -- 172.21.15.94:0/3316950563 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f627010f0a0 con 0x7f62700fe060 2024-03-24T10:52:46.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.309+0000 7f62577fe700 1 -- 172.21.15.94:0/3316950563 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6264014640 con 0x7f62700fe060 2024-03-24T10:52:46.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.309+0000 7f6275018700 1 -- 172.21.15.94:0/3316950563 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f627010f5e0 con 0x7f62700fe060 2024-03-24T10:52:46.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.310+0000 7f6275018700 1 -- 172.21.15.94:0/3316950563 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f627005f5f0 con 0x7f62700fe060 2024-03-24T10:52:46.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.311+0000 7f62577fe700 1 -- 172.21.15.94:0/3316950563 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f626400e960 con 0x7f62700fe060 2024-03-24T10:52:46.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.311+0000 7f62577fe700 1 --2- 172.21.15.94:0/3316950563 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f625806d7e0 0x7f625806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:52:46.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.311+0000 7f62577fe700 1 -- 172.21.15.94:0/3316950563 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f626408ccd0 con 0x7f62700fe060 2024-03-24T10:52:46.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.311+0000 7f626ed9d700 1 --2- 172.21.15.94:0/3316950563 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f625806d7e0 0x7f625806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:52:46.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.312+0000 7f626ed9d700 1 --2- 172.21.15.94:0/3316950563 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f625806d7e0 0x7f625806fc90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f626800b970 tx=0x7f6268009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:52:46.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.316+0000 7f62577fe700 1 -- 172.21.15.94:0/3316950563 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f626405a0a0 con 0x7f62700fe060 2024-03-24T10:52:46.497 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:46 smithi096 ceph-mon[84258]: pgmap v153: 65 pgs: 65 active+clean; 196 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:52:46.497 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:46 smithi096 ceph-mon[84258]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T10:52:46.497 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:46 smithi096 ceph-mon[84258]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-03-24T10:52:46.497 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:46 smithi096 ceph-mon[84258]: osdmap e75: 6 total, 5 up, 6 in 2024-03-24T10:52:46.497 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:46 smithi096 podman[103048]: 2024-03-24 10:52:46.229547782 +0000 UTC m=+1.740132508 container cleanup f56fad2959c3e00063f4dad21785b00f5e6c6e6973556f3f79ef149ff9a7a483 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=quincy-9c4e0c3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319) 2024-03-24T10:52:46.497 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:46 smithi096 bash[103048]: ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5 2024-03-24T10:52:46.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.610+0000 7f6275018700 1 -- 172.21.15.94:0/3316950563 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f62701058e0 con 0x7f62700fe060 2024-03-24T10:52:46.610 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:46 smithi094 ceph-mon[93957]: pgmap v153: 65 pgs: 65 active+clean; 196 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:52:46.610 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:46 smithi094 ceph-mon[93957]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T10:52:46.610 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:46 smithi094 ceph-mon[93957]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-03-24T10:52:46.610 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:46 smithi094 ceph-mon[93957]: osdmap e75: 6 total, 5 up, 6 in 2024-03-24T10:52:46.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.610+0000 7f62577fe700 1 -- 172.21.15.94:0/3316950563 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+732 (secure 0 0 0) 0x7f62640097e0 con 0x7f62700fe060 2024-03-24T10:52:46.611 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-03-24T10:52:46.611 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": { 2024-03-24T10:52:46.611 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 2 2024-03-24T10:52:46.611 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:52:46.611 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": { 2024-03-24T10:52:46.611 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 2 2024-03-24T10:52:46.611 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:52:46.611 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": { 2024-03-24T10:52:46.611 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 5 2024-03-24T10:52:46.611 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:52:46.611 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-03-24T10:52:46.611 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-24T10:52:46.612 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:52:46.612 INFO:teuthology.orchestra.run.smithi094.stdout: "overall": { 2024-03-24T10:52:46.612 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4, 2024-03-24T10:52:46.612 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 9 2024-03-24T10:52:46.612 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-03-24T10:52:46.612 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-03-24T10:52:46.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.612+0000 7f6275018700 1 -- 172.21.15.94:0/3316950563 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f625806d7e0 msgr2=0x7f625806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:46.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.612+0000 7f6275018700 1 --2- 172.21.15.94:0/3316950563 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f625806d7e0 0x7f625806fc90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f626800b970 tx=0x7f6268009e30 comp rx=0 tx=0).stop 2024-03-24T10:52:46.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.612+0000 7f6275018700 1 -- 172.21.15.94:0/3316950563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62700fe060 msgr2=0x7f627010dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:46.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.612+0000 7f6275018700 1 --2- 172.21.15.94:0/3316950563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62700fe060 0x7f627010dfc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6264001d80 tx=0x7f6264009ec0 comp rx=0 tx=0).stop 2024-03-24T10:52:46.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.612+0000 7f6275018700 1 -- 172.21.15.94:0/3316950563 shutdown_connections 2024-03-24T10:52:46.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.612+0000 7f6275018700 1 --2- 172.21.15.94:0/3316950563 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f625806d7e0 0x7f625806fc90 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:46.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.612+0000 7f6275018700 1 --2- 172.21.15.94:0/3316950563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62700fcef0 0x7f627010da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:46.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.612+0000 7f6275018700 1 --2- 172.21.15.94:0/3316950563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62700fe060 0x7f627010dfc0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:46.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.612+0000 7f6275018700 1 -- 172.21.15.94:0/3316950563 >> 172.21.15.94:0/3316950563 conn(0x7f62700f85b0 msgr2=0x7f62700f9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:52:46.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.612+0000 7f6275018700 1 -- 172.21.15.94:0/3316950563 shutdown_connections 2024-03-24T10:52:46.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.612+0000 7f6275018700 1 -- 172.21.15.94:0/3316950563 wait complete. 2024-03-24T10:52:46.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.712+0000 7fc57edd9700 1 -- 172.21.15.94:0/2350115395 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc5780fff10 msgr2=0x7fc578102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:46.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.712+0000 7fc57edd9700 1 --2- 172.21.15.94:0/2350115395 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc5780fff10 0x7fc578102330 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc564004940 tx=0x7fc564009e30 comp rx=0 tx=0).stop 2024-03-24T10:52:46.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.712+0000 7fc57edd9700 1 -- 172.21.15.94:0/2350115395 shutdown_connections 2024-03-24T10:52:46.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.712+0000 7fc57edd9700 1 --2- 172.21.15.94:0/2350115395 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc5780fff10 0x7fc578102330 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:46.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.712+0000 7fc57edd9700 1 --2- 172.21.15.94:0/2350115395 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc578066fd0 0x7fc5780ff9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:46.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.712+0000 7fc57edd9700 1 -- 172.21.15.94:0/2350115395 >> 172.21.15.94:0/2350115395 conn(0x7fc5780f7440 msgr2=0x7fc5780f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:52:46.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.713+0000 7fc57edd9700 1 -- 172.21.15.94:0/2350115395 shutdown_connections 2024-03-24T10:52:46.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.713+0000 7fc57edd9700 1 -- 172.21.15.94:0/2350115395 wait complete. 2024-03-24T10:52:46.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.713+0000 7fc57edd9700 1 Processor -- start 2024-03-24T10:52:46.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.713+0000 7fc57edd9700 1 -- start start 2024-03-24T10:52:46.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.713+0000 7fc57edd9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc578066fd0 0x7fc57810d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:52:46.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.713+0000 7fc57edd9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5780fff10 0x7fc57810dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:52:46.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.713+0000 7fc57edd9700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc57810e4a0 con 0x7fc578066fd0 2024-03-24T10:52:46.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.713+0000 7fc57edd9700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc57810e610 con 0x7fc5780fff10 2024-03-24T10:52:46.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.713+0000 7fc57cb75700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc578066fd0 0x7fc57810d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:52:46.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.713+0000 7fc57cb75700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc578066fd0 0x7fc57810d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54018/0 (socket says 172.21.15.94:54018) 2024-03-24T10:52:46.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.714+0000 7fc57cb75700 1 -- 172.21.15.94:0/3652549382 learned_addr learned my addr 172.21.15.94:0/3652549382 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:52:46.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.714+0000 7fc577fff700 1 --2- 172.21.15.94:0/3652549382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5780fff10 0x7fc57810dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:52:46.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.714+0000 7fc57cb75700 1 -- 172.21.15.94:0/3652549382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5780fff10 msgr2=0x7fc57810dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:46.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.714+0000 7fc57cb75700 1 --2- 172.21.15.94:0/3652549382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5780fff10 0x7fc57810dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:46.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.714+0000 7fc57cb75700 1 -- 172.21.15.94:0/3652549382 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc564005040 con 0x7fc578066fd0 2024-03-24T10:52:46.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.714+0000 7fc57cb75700 1 --2- 172.21.15.94:0/3652549382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc578066fd0 0x7fc57810d960 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc56c002a50 tx=0x7fc56c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:52:46.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.714+0000 7fc5757fa700 1 -- 172.21.15.94:0/3652549382 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc56c007cc0 con 0x7fc578066fd0 2024-03-24T10:52:46.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.714+0000 7fc5757fa700 1 -- 172.21.15.94:0/3652549382 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc56c00f040 con 0x7fc578066fd0 2024-03-24T10:52:46.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.714+0000 7fc57edd9700 1 -- 172.21.15.94:0/3652549382 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc578113070 con 0x7fc578066fd0 2024-03-24T10:52:46.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.714+0000 7fc5757fa700 1 -- 172.21.15.94:0/3652549382 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc56c00d9c0 con 0x7fc578066fd0 2024-03-24T10:52:46.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.714+0000 7fc57edd9700 1 -- 172.21.15.94:0/3652549382 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc578113630 con 0x7fc578066fd0 2024-03-24T10:52:46.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.715+0000 7fc57edd9700 1 -- 172.21.15.94:0/3652549382 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc57804c710 con 0x7fc578066fd0 2024-03-24T10:52:46.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.716+0000 7fc5757fa700 1 -- 172.21.15.94:0/3652549382 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fc56c009730 con 0x7fc578066fd0 2024-03-24T10:52:46.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.716+0000 7fc5757fa700 1 --2- 172.21.15.94:0/3652549382 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fc56806d7e0 0x7fc56806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:52:46.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.716+0000 7fc577fff700 1 --2- 172.21.15.94:0/3652549382 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fc56806d7e0 0x7fc56806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:52:46.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.716+0000 7fc5757fa700 1 -- 172.21.15.94:0/3652549382 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fc56c08cc70 con 0x7fc578066fd0 2024-03-24T10:52:46.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.716+0000 7fc577fff700 1 --2- 172.21.15.94:0/3652549382 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fc56806d7e0 0x7fc56806fc90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc56400b970 tx=0x7fc564009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:52:46.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.721+0000 7fc5757fa700 1 -- 172.21.15.94:0/3652549382 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc56c059f90 con 0x7fc578066fd0 2024-03-24T10:52:46.825 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:46 smithi096 podman[103066]: 2024-03-24 10:52:46.497560518 +0000 UTC m=+0.656999027 container remove f56fad2959c3e00063f4dad21785b00f5e6c6e6973556f3f79ef149ff9a7a483 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=quincy-9c4e0c3, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-24T10:52:46.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.959+0000 7fc57edd9700 1 -- 172.21.15.94:0/3652549382 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc578113a70 con 0x7fc578066fd0 2024-03-24T10:52:46.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.960+0000 7fc5757fa700 1 -- 172.21.15.94:0/3652549382 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1903 (secure 0 0 0) 0x7fc56c059b20 con 0x7fc578066fd0 2024-03-24T10:52:46.960 INFO:teuthology.orchestra.run.smithi094.stdout:e18 2024-03-24T10:52:46.960 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T10:52:46.960 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:52:46.961 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-03-24T10:52:46.961 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:52:46.961 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-03-24T10:52:46.961 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-03-24T10:52:46.961 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 18 2024-03-24T10:52:46.961 INFO:teuthology.orchestra.run.smithi094.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-24T10:52:46.962 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-03-24T10:43:39.435738+0000 2024-03-24T10:52:46.962 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-03-24T10:43:58.133901+0000 2024-03-24T10:52:46.962 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-03-24T10:52:46.962 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-03-24T10:52:46.963 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 60 2024-03-24T10:52:46.963 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-03-24T10:52:46.963 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-03-24T10:52:46.963 INFO:teuthology.orchestra.run.smithi094.stdout:max_xattr_size 65536 2024-03-24T10:52:46.963 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-03-24T10:52:46.963 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-03-24T10:52:46.964 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 46 2024-03-24T10:52:46.964 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:52:46.964 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-03-24T10:52:46.964 INFO:teuthology.orchestra.run.smithi094.stdout:in 0 2024-03-24T10:52:46.964 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=24251} 2024-03-24T10:52:46.964 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-03-24T10:52:46.965 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-03-24T10:52:46.965 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-03-24T10:52:46.965 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-03-24T10:52:46.965 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-03-24T10:52:46.965 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-03-24T10:52:46.966 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-03-24T10:52:46.966 INFO:teuthology.orchestra.run.smithi094.stdout:bal_rank_mask -1 2024-03-24T10:52:46.966 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 1 2024-03-24T10:52:46.966 INFO:teuthology.orchestra.run.smithi094.stdout:qdb_cluster leader: 0 members: 2024-03-24T10:52:46.966 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi096.egbmoq{0:24251} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.96:6824/1539752208,v1:172.21.15.96:6825/1539752208] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:52:46.966 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.llefgu{0:14500} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.94:6828/126762980,v1:172.21.15.94:6829/126762980] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:52:46.967 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:52:46.967 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:52:46.967 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-03-24T10:52:46.967 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:52:46.967 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.efiqte{-1:14512} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.94:6826/1636528860,v1:172.21.15.94:6827/1636528860] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:52:46.967 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi096.jtebhh{-1:24261} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.96:6826/1321087930,v1:172.21.15.96:6827/1321087930] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:52:46.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.963+0000 7fc57edd9700 1 -- 172.21.15.94:0/3652549382 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fc56806d7e0 msgr2=0x7fc56806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:46.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.963+0000 7fc57edd9700 1 --2- 172.21.15.94:0/3652549382 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fc56806d7e0 0x7fc56806fc90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc56400b970 tx=0x7fc564009e30 comp rx=0 tx=0).stop 2024-03-24T10:52:46.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.963+0000 7fc57edd9700 1 -- 172.21.15.94:0/3652549382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc578066fd0 msgr2=0x7fc57810d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:46.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.963+0000 7fc57edd9700 1 --2- 172.21.15.94:0/3652549382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc578066fd0 0x7fc57810d960 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc56c002a50 tx=0x7fc56c004600 comp rx=0 tx=0).stop 2024-03-24T10:52:46.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.963+0000 7fc57edd9700 1 -- 172.21.15.94:0/3652549382 shutdown_connections 2024-03-24T10:52:46.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.963+0000 7fc57edd9700 1 --2- 172.21.15.94:0/3652549382 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fc56806d7e0 0x7fc56806fc90 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:46.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.963+0000 7fc57edd9700 1 --2- 172.21.15.94:0/3652549382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc578066fd0 0x7fc57810d960 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:46.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.963+0000 7fc57edd9700 1 --2- 172.21.15.94:0/3652549382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5780fff10 0x7fc57810dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:46.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.963+0000 7fc57edd9700 1 -- 172.21.15.94:0/3652549382 >> 172.21.15.94:0/3652549382 conn(0x7fc5780f7440 msgr2=0x7fc5780f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:52:46.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.963+0000 7fc57edd9700 1 -- 172.21.15.94:0/3652549382 shutdown_connections 2024-03-24T10:52:46.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:46.963+0000 7fc57edd9700 1 -- 172.21.15.94:0/3652549382 wait complete. 2024-03-24T10:52:46.971 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 18 2024-03-24T10:52:47.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.064+0000 7fa92fde1700 1 -- 172.21.15.94:0/1712170203 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9280fe0d0 msgr2=0x7fa9280fe540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:47.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.064+0000 7fa92fde1700 1 --2- 172.21.15.94:0/1712170203 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9280fe0d0 0x7fa9280fe540 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa924004940 tx=0x7fa924009e30 comp rx=0 tx=0).stop 2024-03-24T10:52:47.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.064+0000 7fa92fde1700 1 -- 172.21.15.94:0/1712170203 shutdown_connections 2024-03-24T10:52:47.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.064+0000 7fa92fde1700 1 --2- 172.21.15.94:0/1712170203 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9280fe0d0 0x7fa9280fe540 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:47.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.064+0000 7fa92fde1700 1 --2- 172.21.15.94:0/1712170203 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9280fcee0 0x7fa9280fd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:47.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.064+0000 7fa92fde1700 1 -- 172.21.15.94:0/1712170203 >> 172.21.15.94:0/1712170203 conn(0x7fa9280f8560 msgr2=0x7fa9280fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:52:47.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.064+0000 7fa92fde1700 1 -- 172.21.15.94:0/1712170203 shutdown_connections 2024-03-24T10:52:47.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.064+0000 7fa92fde1700 1 -- 172.21.15.94:0/1712170203 wait complete. 2024-03-24T10:52:47.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.065+0000 7fa92fde1700 1 Processor -- start 2024-03-24T10:52:47.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.065+0000 7fa92fde1700 1 -- start start 2024-03-24T10:52:47.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.065+0000 7fa92fde1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9280fcee0 0x7fa92810fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:52:47.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.065+0000 7fa92fde1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9280fe0d0 0x7fa928110250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:52:47.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.065+0000 7fa92fde1700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa928110850 con 0x7fa9280fcee0 2024-03-24T10:52:47.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.065+0000 7fa92fde1700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa928110f90 con 0x7fa9280fe0d0 2024-03-24T10:52:47.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.065+0000 7fa92db7d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9280fcee0 0x7fa92810fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:52:47.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.065+0000 7fa92db7d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9280fcee0 0x7fa92810fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54024/0 (socket says 172.21.15.94:54024) 2024-03-24T10:52:47.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.065+0000 7fa92db7d700 1 -- 172.21.15.94:0/718327320 learned_addr learned my addr 172.21.15.94:0/718327320 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:52:47.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.065+0000 7fa92d37c700 1 --2- 172.21.15.94:0/718327320 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9280fe0d0 0x7fa928110250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:52:47.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.065+0000 7fa92db7d700 1 -- 172.21.15.94:0/718327320 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9280fe0d0 msgr2=0x7fa928110250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:47.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.065+0000 7fa92db7d700 1 --2- 172.21.15.94:0/718327320 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9280fe0d0 0x7fa928110250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:47.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.065+0000 7fa92db7d700 1 -- 172.21.15.94:0/718327320 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa924005040 con 0x7fa9280fcee0 2024-03-24T10:52:47.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.066+0000 7fa92db7d700 1 --2- 172.21.15.94:0/718327320 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9280fcee0 0x7fa92810fd10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa918002a80 tx=0x7fa918004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:52:47.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.066+0000 7fa91e7fc700 1 -- 172.21.15.94:0/718327320 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa918007cc0 con 0x7fa9280fcee0 2024-03-24T10:52:47.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.066+0000 7fa91e7fc700 1 -- 172.21.15.94:0/718327320 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa91800f040 con 0x7fa9280fcee0 2024-03-24T10:52:47.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.066+0000 7fa92fde1700 1 -- 172.21.15.94:0/718327320 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa928111270 con 0x7fa9280fcee0 2024-03-24T10:52:47.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.066+0000 7fa91e7fc700 1 -- 172.21.15.94:0/718327320 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa91800da30 con 0x7fa9280fcee0 2024-03-24T10:52:47.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.066+0000 7fa92fde1700 1 -- 172.21.15.94:0/718327320 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa928111830 con 0x7fa9280fcee0 2024-03-24T10:52:47.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.067+0000 7fa92fde1700 1 -- 172.21.15.94:0/718327320 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa928101a60 con 0x7fa9280fcee0 2024-03-24T10:52:47.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.067+0000 7fa91e7fc700 1 -- 172.21.15.94:0/718327320 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fa91800db90 con 0x7fa9280fcee0 2024-03-24T10:52:47.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.067+0000 7fa91e7fc700 1 --2- 172.21.15.94:0/718327320 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fa91406d7e0 0x7fa91406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:52:47.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.068+0000 7fa91e7fc700 1 -- 172.21.15.94:0/718327320 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fa91808cbe0 con 0x7fa9280fcee0 2024-03-24T10:52:47.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.068+0000 7fa92d37c700 1 --2- 172.21.15.94:0/718327320 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fa91406d7e0 0x7fa91406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:52:47.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.068+0000 7fa92d37c700 1 --2- 172.21.15.94:0/718327320 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fa91406d7e0 0x7fa91406fc90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa92400b970 tx=0x7fa924009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:52:47.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.072+0000 7fa91e7fc700 1 -- 172.21.15.94:0/718327320 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa918059f80 con 0x7fa9280fcee0 2024-03-24T10:52:47.125 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:46 smithi096 podman[103159]: 2024-03-24 10:52:46.831690822 +0000 UTC m=+0.108882912 container create d78ec15df4acae1658b28b27d085478f8d05f5c6429ae8ebb29791d3710975bd (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T10:52:47.125 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:46 smithi096 podman[103159]: 2024-03-24 10:52:46.740173584 +0000 UTC m=+0.017365647 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-24T10:52:47.126 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:47 smithi096 podman[103159]: 2024-03-24 10:52:47.126190269 +0000 UTC m=+0.403382326 container init d78ec15df4acae1658b28b27d085478f8d05f5c6429ae8ebb29791d3710975bd (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-24T10:52:47.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.288+0000 7fa92fde1700 1 -- 172.21.15.94:0/718327320 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa928111aa0 con 0x7fa91406d7e0 2024-03-24T10:52:47.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.290+0000 7fa91e7fc700 1 -- 172.21.15.94:0/718327320 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fa928111aa0 con 0x7fa91406d7e0 2024-03-24T10:52:47.290 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-03-24T10:52:47.290 INFO:teuthology.orchestra.run.smithi094.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919", 2024-03-24T10:52:47.290 INFO:teuthology.orchestra.run.smithi094.stdout: "in_progress": true, 2024-03-24T10:52:47.290 INFO:teuthology.orchestra.run.smithi094.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T10:52:47.290 INFO:teuthology.orchestra.run.smithi094.stdout: "services_complete": [ 2024-03-24T10:52:47.290 INFO:teuthology.orchestra.run.smithi094.stdout: "mon", 2024-03-24T10:52:47.290 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-03-24T10:52:47.290 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr" 2024-03-24T10:52:47.290 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-03-24T10:52:47.290 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": "11/21 daemons upgraded", 2024-03-24T10:52:47.290 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "Currently upgrading osd daemons", 2024-03-24T10:52:47.290 INFO:teuthology.orchestra.run.smithi094.stdout: "is_paused": false 2024-03-24T10:52:47.290 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-03-24T10:52:47.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.292+0000 7fa92fde1700 1 -- 172.21.15.94:0/718327320 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fa91406d7e0 msgr2=0x7fa91406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:47.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.292+0000 7fa92fde1700 1 --2- 172.21.15.94:0/718327320 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fa91406d7e0 0x7fa91406fc90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa92400b970 tx=0x7fa924009e30 comp rx=0 tx=0).stop 2024-03-24T10:52:47.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.292+0000 7fa92fde1700 1 -- 172.21.15.94:0/718327320 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9280fcee0 msgr2=0x7fa92810fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:47.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.292+0000 7fa92fde1700 1 --2- 172.21.15.94:0/718327320 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9280fcee0 0x7fa92810fd10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa918002a80 tx=0x7fa918004600 comp rx=0 tx=0).stop 2024-03-24T10:52:47.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.292+0000 7fa92fde1700 1 -- 172.21.15.94:0/718327320 shutdown_connections 2024-03-24T10:52:47.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.292+0000 7fa92fde1700 1 --2- 172.21.15.94:0/718327320 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fa91406d7e0 0x7fa91406fc90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:47.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.292+0000 7fa92fde1700 1 --2- 172.21.15.94:0/718327320 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9280fcee0 0x7fa92810fd10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:47.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.292+0000 7fa92fde1700 1 --2- 172.21.15.94:0/718327320 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9280fe0d0 0x7fa928110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:47.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.292+0000 7fa92fde1700 1 -- 172.21.15.94:0/718327320 >> 172.21.15.94:0/718327320 conn(0x7fa9280f8560 msgr2=0x7fa9280f9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:52:47.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.292+0000 7fa92fde1700 1 -- 172.21.15.94:0/718327320 shutdown_connections 2024-03-24T10:52:47.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.292+0000 7fa92fde1700 1 -- 172.21.15.94:0/718327320 wait complete. 2024-03-24T10:52:47.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.394+0000 7f19ddb59700 1 -- 172.21.15.94:0/1408980012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19d80fed80 msgr2=0x7f19d80faec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:47.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.394+0000 7f19ddb59700 1 --2- 172.21.15.94:0/1408980012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19d80fed80 0x7f19d80faec0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f19c0004940 tx=0x7f19c0009e30 comp rx=0 tx=0).stop 2024-03-24T10:52:47.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.394+0000 7f19ddb59700 1 -- 172.21.15.94:0/1408980012 shutdown_connections 2024-03-24T10:52:47.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.394+0000 7f19ddb59700 1 --2- 172.21.15.94:0/1408980012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19d80fed80 0x7f19d80faec0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:47.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.394+0000 7f19ddb59700 1 --2- 172.21.15.94:0/1408980012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19d80fe430 0x7f19d80fe840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:47.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.394+0000 7f19ddb59700 1 -- 172.21.15.94:0/1408980012 >> 172.21.15.94:0/1408980012 conn(0x7f19d80f6bd0 msgr2=0x7f19d80f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:52:47.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.394+0000 7f19ddb59700 1 -- 172.21.15.94:0/1408980012 shutdown_connections 2024-03-24T10:52:47.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.394+0000 7f19ddb59700 1 -- 172.21.15.94:0/1408980012 wait complete. 2024-03-24T10:52:47.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.394+0000 7f19ddb59700 1 Processor -- start 2024-03-24T10:52:47.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.395+0000 7f19ddb59700 1 -- start start 2024-03-24T10:52:47.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.395+0000 7f19ddb59700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19d80fe430 0x7f19d810d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:52:47.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.395+0000 7f19ddb59700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19d80fed80 0x7f19d810de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:52:47.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.395+0000 7f19ddb59700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d810e440 con 0x7f19d80fe430 2024-03-24T10:52:47.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.395+0000 7f19ddb59700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d810e5b0 con 0x7f19d80fed80 2024-03-24T10:52:47.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.395+0000 7f19d77fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19d80fe430 0x7f19d810d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:52:47.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.395+0000 7f19d77fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19d80fe430 0x7f19d810d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54040/0 (socket says 172.21.15.94:54040) 2024-03-24T10:52:47.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.395+0000 7f19d77fe700 1 -- 172.21.15.94:0/3935981852 learned_addr learned my addr 172.21.15.94:0/3935981852 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:52:47.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.395+0000 7f19cffff700 1 --2- 172.21.15.94:0/3935981852 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19d80fed80 0x7f19d810de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:52:47.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.395+0000 7f19d77fe700 1 -- 172.21.15.94:0/3935981852 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19d80fed80 msgr2=0x7f19d810de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:47.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.395+0000 7f19d77fe700 1 --2- 172.21.15.94:0/3935981852 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19d80fed80 0x7f19d810de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:47.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.395+0000 7f19d77fe700 1 -- 172.21.15.94:0/3935981852 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c0005040 con 0x7f19d80fe430 2024-03-24T10:52:47.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.395+0000 7f19d77fe700 1 --2- 172.21.15.94:0/3935981852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19d80fe430 0x7f19d810d900 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f19c8002a50 tx=0x7f19c80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:52:47.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.395+0000 7f19d4ff9700 1 -- 172.21.15.94:0/3935981852 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19c8007cc0 con 0x7f19d80fe430 2024-03-24T10:52:47.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.395+0000 7f19d4ff9700 1 -- 172.21.15.94:0/3935981852 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f19c800f040 con 0x7f19d80fe430 2024-03-24T10:52:47.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.395+0000 7f19ddb59700 1 -- 172.21.15.94:0/3935981852 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19d8112fb0 con 0x7f19d80fe430 2024-03-24T10:52:47.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.396+0000 7f19d4ff9700 1 -- 172.21.15.94:0/3935981852 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19c800da30 con 0x7f19d80fe430 2024-03-24T10:52:47.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.396+0000 7f19ddb59700 1 -- 172.21.15.94:0/3935981852 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19d8113540 con 0x7f19d80fe430 2024-03-24T10:52:47.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.397+0000 7f19ddb59700 1 -- 172.21.15.94:0/3935981852 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19d804c710 con 0x7f19d80fe430 2024-03-24T10:52:47.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.397+0000 7f19d4ff9700 1 -- 172.21.15.94:0/3935981852 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f19c800db90 con 0x7f19d80fe430 2024-03-24T10:52:47.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.397+0000 7f19d4ff9700 1 --2- 172.21.15.94:0/3935981852 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f19c406d7e0 0x7f19c406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:52:47.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.397+0000 7f19d4ff9700 1 -- 172.21.15.94:0/3935981852 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f19c808cd10 con 0x7f19d80fe430 2024-03-24T10:52:47.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.397+0000 7f19cffff700 1 --2- 172.21.15.94:0/3935981852 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f19c406d7e0 0x7f19c406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:52:47.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.398+0000 7f19cffff700 1 --2- 172.21.15.94:0/3935981852 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f19c406d7e0 0x7f19c406fc90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f19c000b970 tx=0x7f19c0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:52:47.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.402+0000 7f19d4ff9700 1 -- 172.21.15.94:0/3935981852 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f19c8059f90 con 0x7f19d80fe430 2024-03-24T10:52:47.422 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:47 smithi096 podman[103159]: 2024-03-24 10:52:47.129602922 +0000 UTC m=+0.406794970 container start d78ec15df4acae1658b28b27d085478f8d05f5c6429ae8ebb29791d3710975bd (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T10:52:47.422 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:47 smithi096 podman[103159]: 2024-03-24 10:52:47.225805942 +0000 UTC m=+0.502997999 container attach d78ec15df4acae1658b28b27d085478f8d05f5c6429ae8ebb29791d3710975bd (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5-deactivate, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-24T10:52:47.422 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:47 smithi096 ceph-mon[84258]: from='client.34342 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:52:47.422 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:47 smithi096 ceph-mon[84258]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:52:47.423 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:47 smithi096 ceph-mon[84258]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:52:47.423 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:47 smithi096 ceph-mon[84258]: osdmap e76: 6 total, 5 up, 6 in 2024-03-24T10:52:47.423 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:47 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/3316950563' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:52:47.423 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:47 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/3652549382' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T10:52:47.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.681+0000 7f19ddb59700 1 -- 172.21.15.94:0/3935981852 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f19d8113730 con 0x7f19d80fe430 2024-03-24T10:52:47.681 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:47 smithi094 ceph-mon[93957]: from='client.34342 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:52:47.681 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:47 smithi094 ceph-mon[93957]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:52:47.681 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:47 smithi094 ceph-mon[93957]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:52:47.681 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:47 smithi094 ceph-mon[93957]: osdmap e76: 6 total, 5 up, 6 in 2024-03-24T10:52:47.681 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:47 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/3316950563' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:52:47.681 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:47 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/3652549382' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T10:52:47.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.681+0000 7f19d4ff9700 1 -- 172.21.15.94:0/3935981852 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+337 (secure 0 0 0) 0x7f19c8059b20 con 0x7f19d80fe430 2024-03-24T10:52:47.682 INFO:teuthology.orchestra.run.smithi094.stdout:HEALTH_WARN 1 osds down; all OSDs are running squid or later but require_osd_release < squid 2024-03-24T10:52:47.682 INFO:teuthology.orchestra.run.smithi094.stdout:[WRN] OSD_DOWN: 1 osds down 2024-03-24T10:52:47.682 INFO:teuthology.orchestra.run.smithi094.stdout: osd.5 (root=default,host=smithi096) is down 2024-03-24T10:52:47.682 INFO:teuthology.orchestra.run.smithi094.stdout:[WRN] OSD_UPGRADE_FINISHED: all OSDs are running squid or later but require_osd_release < squid 2024-03-24T10:52:47.682 INFO:teuthology.orchestra.run.smithi094.stdout: all OSDs are running squid or later but require_osd_release < squid 2024-03-24T10:52:47.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.684+0000 7f19ddb59700 1 -- 172.21.15.94:0/3935981852 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f19c406d7e0 msgr2=0x7f19c406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:47.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.684+0000 7f19ddb59700 1 --2- 172.21.15.94:0/3935981852 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f19c406d7e0 0x7f19c406fc90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f19c000b970 tx=0x7f19c0009e30 comp rx=0 tx=0).stop 2024-03-24T10:52:47.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.684+0000 7f19ddb59700 1 -- 172.21.15.94:0/3935981852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19d80fe430 msgr2=0x7f19d810d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:52:47.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.684+0000 7f19ddb59700 1 --2- 172.21.15.94:0/3935981852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19d80fe430 0x7f19d810d900 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f19c8002a50 tx=0x7f19c80046b0 comp rx=0 tx=0).stop 2024-03-24T10:52:47.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.684+0000 7f19ddb59700 1 -- 172.21.15.94:0/3935981852 shutdown_connections 2024-03-24T10:52:47.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.684+0000 7f19ddb59700 1 --2- 172.21.15.94:0/3935981852 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f19c406d7e0 0x7f19c406fc90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:47.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.684+0000 7f19ddb59700 1 --2- 172.21.15.94:0/3935981852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19d80fe430 0x7f19d810d900 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:47.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.684+0000 7f19ddb59700 1 --2- 172.21.15.94:0/3935981852 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19d80fed80 0x7f19d810de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:52:47.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.684+0000 7f19ddb59700 1 -- 172.21.15.94:0/3935981852 >> 172.21.15.94:0/3935981852 conn(0x7f19d80f6bd0 msgr2=0x7f19d80f8210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:52:47.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.684+0000 7f19ddb59700 1 -- 172.21.15.94:0/3935981852 shutdown_connections 2024-03-24T10:52:47.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:52:47.684+0000 7f19ddb59700 1 -- 172.21.15.94:0/3935981852 wait complete. 2024-03-24T10:52:47.770 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:47 smithi096 podman[103159]: 2024-03-24 10:52:47.423076612 +0000 UTC m=+0.700268664 container died d78ec15df4acae1658b28b27d085478f8d05f5c6429ae8ebb29791d3710975bd (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-24T10:52:48.325 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:48 smithi096 podman[103185]: 2024-03-24 10:52:48.037904595 +0000 UTC m=+0.606039578 container remove d78ec15df4acae1658b28b27d085478f8d05f5c6429ae8ebb29791d3710975bd (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.build-date=20240319) 2024-03-24T10:52:48.325 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:48 smithi096 systemd[1]: ceph-386110d4-e9ca-11ee-95cd-87774f69a715@osd.5.service: Deactivated successfully. 2024-03-24T10:52:48.326 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:48 smithi096 systemd[1]: Stopped Ceph osd.5 for 386110d4-e9ca-11ee-95cd-87774f69a715. 2024-03-24T10:52:48.326 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:48 smithi096 systemd[1]: ceph-386110d4-e9ca-11ee-95cd-87774f69a715@osd.5.service: Consumed 1min 3.894s CPU time. 2024-03-24T10:52:48.326 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:48 smithi096 systemd[1]: Starting Ceph osd.5 for 386110d4-e9ca-11ee-95cd-87774f69a715... 2024-03-24T10:52:48.635 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:48 smithi096 ceph-mon[84258]: from='client.34362 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:52:48.635 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:48 smithi096 ceph-mon[84258]: pgmap v156: 65 pgs: 13 stale+active+clean, 52 active+clean; 196 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-03-24T10:52:48.636 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:48 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/3935981852' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T10:52:48.636 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:48 smithi096 podman[103290]: 2024-03-24 10:52:48.398196349 +0000 UTC m=+0.019301801 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-24T10:52:48.636 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:48 smithi096 podman[103290]: 2024-03-24 10:52:48.530763673 +0000 UTC m=+0.151869108 container create 55d5b2755fa0c9e3d6ee95ed7956d8625fc16604e224c7b6a9dc3eafb14cc4cf (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5-activate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True) 2024-03-24T10:52:48.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:48 smithi094 ceph-mon[93957]: from='client.34362 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:52:48.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:48 smithi094 ceph-mon[93957]: pgmap v156: 65 pgs: 13 stale+active+clean, 52 active+clean; 196 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-03-24T10:52:48.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:48 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/3935981852' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T10:52:49.075 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:48 smithi096 podman[103290]: 2024-03-24 10:52:48.733145434 +0000 UTC m=+0.354250889 container init 55d5b2755fa0c9e3d6ee95ed7956d8625fc16604e224c7b6a9dc3eafb14cc4cf (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5-activate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_CLEAN=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T10:52:49.075 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:48 smithi096 podman[103290]: 2024-03-24 10:52:48.736186257 +0000 UTC m=+0.357291693 container start 55d5b2755fa0c9e3d6ee95ed7956d8625fc16604e224c7b6a9dc3eafb14cc4cf (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5-activate, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-24T10:52:49.075 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:48 smithi096 podman[103290]: 2024-03-24 10:52:48.816295791 +0000 UTC m=+0.437401244 container attach 55d5b2755fa0c9e3d6ee95ed7956d8625fc16604e224c7b6a9dc3eafb14cc4cf (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5-activate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-24T10:52:49.563 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:49 smithi096 ceph-mon[84258]: Health check failed: Degraded data redundancy: 30/219 objects degraded (13.699%), 12 pgs degraded (PG_DEGRADED) 2024-03-24T10:52:49.563 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:49 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5-activate[103305]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-24T10:52:49.563 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:49 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5-activate[103305]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-24T10:52:49.563 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:49 smithi096 bash[103290]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-24T10:52:49.563 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:49 smithi096 bash[103290]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-24T10:52:49.563 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:49 smithi096 bash[103290]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-24T10:52:49.564 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:49 smithi096 bash[103290]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-24T10:52:49.564 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:49 smithi096 bash[103290]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-03-24T10:52:49.564 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:49 smithi096 bash[103290]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-24T10:52:49.564 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:49 smithi096 bash[103290]: --> ceph-volume raw activate successful for osd ID: 5 2024-03-24T10:52:49.564 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:49 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5-activate[103305]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-24T10:52:49.564 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:49 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5-activate[103305]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-24T10:52:49.564 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:49 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5-activate[103305]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-03-24T10:52:49.564 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:49 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5-activate[103305]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-24T10:52:49.564 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:49 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5-activate[103305]: --> ceph-volume raw activate successful for osd ID: 5 2024-03-24T10:52:49.564 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:49 smithi096 podman[103290]: 2024-03-24 10:52:49.2011103 +0000 UTC m=+0.822215736 container died 55d5b2755fa0c9e3d6ee95ed7956d8625fc16604e224c7b6a9dc3eafb14cc4cf (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5-activate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-24T10:52:49.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:49 smithi094 ceph-mon[93957]: Health check failed: Degraded data redundancy: 30/219 objects degraded (13.699%), 12 pgs degraded (PG_DEGRADED) 2024-03-24T10:52:50.152 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:49 smithi096 podman[103440]: 2024-03-24 10:52:49.863773518 +0000 UTC m=+0.652262376 container remove 55d5b2755fa0c9e3d6ee95ed7956d8625fc16604e224c7b6a9dc3eafb14cc4cf (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5-activate, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T10:52:50.152 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:50 smithi096 podman[103503]: 2024-03-24 10:52:50.073003855 +0000 UTC m=+0.114755084 container create 2f77f1fa20e22543f6889f1475e5d16a1dd370623b224be8f94d243655f60b7d (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-24T10:52:50.152 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:50 smithi096 podman[103503]: 2024-03-24 10:52:49.976481264 +0000 UTC m=+0.018232501 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-24T10:52:50.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:50 smithi096 ceph-mon[84258]: pgmap v157: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 196 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 30/219 objects degraded (13.699%) 2024-03-24T10:52:50.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:50 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:50.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:50 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:52:50.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:50 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:50.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:50 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:50.576 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:50 smithi096 podman[103503]: 2024-03-24 10:52:50.258429325 +0000 UTC m=+0.300180551 container init 2f77f1fa20e22543f6889f1475e5d16a1dd370623b224be8f94d243655f60b7d (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-24T10:52:50.576 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:50 smithi096 podman[103503]: 2024-03-24 10:52:50.261354332 +0000 UTC m=+0.303105559 container start 2f77f1fa20e22543f6889f1475e5d16a1dd370623b224be8f94d243655f60b7d (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T10:52:50.576 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:50 smithi096 ceph-osd[103522]: -- 172.21.15.96:0/27212737 <== mon.1 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560d781bd1e0 con 0x560d7818ec00 2024-03-24T10:52:50.576 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:50 smithi096 bash[103503]: 2f77f1fa20e22543f6889f1475e5d16a1dd370623b224be8f94d243655f60b7d 2024-03-24T10:52:50.576 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:50 smithi096 systemd[1]: Started Ceph osd.5 for 386110d4-e9ca-11ee-95cd-87774f69a715. 2024-03-24T10:52:50.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:50 smithi094 ceph-mon[93957]: pgmap v157: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 196 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 30/219 objects degraded (13.699%) 2024-03-24T10:52:50.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:50 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:50.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:50 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:52:50.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:50 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:50.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:50 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:51.075 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:50 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5[103518]: 2024-03-24T10:52:50.809+0000 7f1099c217c0 -1 Falling back to public interface 2024-03-24T10:52:51.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:51 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:52:51.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:51 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:52:52.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:52 smithi094 ceph-mon[93957]: pgmap v158: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 196 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 30/219 objects degraded (13.699%) 2024-03-24T10:52:52.824 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:52 smithi096 ceph-mon[84258]: pgmap v158: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 196 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 30/219 objects degraded (13.699%) 2024-03-24T10:52:54.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:54 smithi096 ceph-mon[84258]: pgmap v159: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 196 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 30/219 objects degraded (13.699%) 2024-03-24T10:52:54.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:54 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:54.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:54 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:54.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:54 smithi094 ceph-mon[93957]: pgmap v159: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 196 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 30/219 objects degraded (13.699%) 2024-03-24T10:52:54.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:54 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:54.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:54 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:55.326 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:55 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5[103518]: 2024-03-24T10:52:55.039+0000 7f1099c217c0 -1 osd.5 0 read_superblock omap replica is missing. 2024-03-24T10:52:55.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:55 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:55.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:55 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:55.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:55 smithi094 ceph-mon[93957]: pgmap v160: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 196 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 514 B/s rd, 1 op/s; 30/219 objects degraded (13.699%) 2024-03-24T10:52:55.825 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:55 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5[103518]: 2024-03-24T10:52:55.504+0000 7f1099c217c0 -1 osd.5 74 log_to_monitors true 2024-03-24T10:52:55.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:55 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:55.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:55 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:55.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:55 smithi096 ceph-mon[84258]: pgmap v160: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 196 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 514 B/s rd, 1 op/s; 30/219 objects degraded (13.699%) 2024-03-24T10:52:56.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:56 smithi096 ceph-mon[84258]: from='osd.5 [v2:172.21.15.96:6816/853588375,v1:172.21.15.96:6817/853588375]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-24T10:52:56.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:56 smithi096 ceph-mon[84258]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-24T10:52:56.575 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:52:56 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5[103518]: 2024-03-24T10:52:56.486+0000 7f108b079700 -1 osd.5 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T10:52:56.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:56 smithi094 ceph-mon[93957]: from='osd.5 [v2:172.21.15.96:6816/853588375,v1:172.21.15.96:6817/853588375]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-24T10:52:56.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:56 smithi094 ceph-mon[93957]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-24T10:52:57.731 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:57 smithi096 ceph-mon[84258]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-24T10:52:57.731 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:57 smithi096 ceph-mon[84258]: osdmap e77: 6 total, 5 up, 6 in 2024-03-24T10:52:57.732 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:57 smithi096 ceph-mon[84258]: from='osd.5 [v2:172.21.15.96:6816/853588375,v1:172.21.15.96:6817/853588375]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-03-24T10:52:57.732 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:57 smithi096 ceph-mon[84258]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-03-24T10:52:57.732 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:57 smithi096 ceph-mon[84258]: pgmap v162: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 511 B/s rd, 1 op/s; 30/219 objects degraded (13.699%) 2024-03-24T10:52:57.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:57 smithi094 ceph-mon[93957]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-24T10:52:57.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:57 smithi094 ceph-mon[93957]: osdmap e77: 6 total, 5 up, 6 in 2024-03-24T10:52:57.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:57 smithi094 ceph-mon[93957]: from='osd.5 [v2:172.21.15.96:6816/853588375,v1:172.21.15.96:6817/853588375]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-03-24T10:52:57.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:57 smithi094 ceph-mon[93957]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-03-24T10:52:57.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:57 smithi094 ceph-mon[93957]: pgmap v162: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 511 B/s rd, 1 op/s; 30/219 objects degraded (13.699%) 2024-03-24T10:52:58.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:58 smithi094 ceph-mon[93957]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T10:52:58.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:58 smithi094 ceph-mon[93957]: osd.5 [v2:172.21.15.96:6816/853588375,v1:172.21.15.96:6817/853588375] boot 2024-03-24T10:52:58.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:58 smithi094 ceph-mon[93957]: osdmap e78: 6 total, 6 up, 6 in 2024-03-24T10:52:58.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T10:52:58.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:58.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:58.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:52:58.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:52:58.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:58.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:52:58.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:52:58.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:52:58.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:52:58.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:52:58.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:58 smithi094 ceph-mon[93957]: Upgrade: Setting container_image for all osd 2024-03-24T10:52:58.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:58.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-03-24T10:52:58.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-03-24T10:52:58.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-03-24T10:52:58.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-03-24T10:52:58.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-03-24T10:52:58.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-03-24T10:52:58.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-03-24T10:52:58.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-03-24T10:52:58.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-03-24T10:52:58.750 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-03-24T10:52:58.750 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-03-24T10:52:58.750 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-03-24T10:52:58.750 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:58 smithi094 ceph-mon[93957]: Upgrade: Setting require_osd_release to 19 squid 2024-03-24T10:52:58.750 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-03-24T10:52:58.750 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:58 smithi094 ceph-mon[93957]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-03-24T10:52:58.750 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-03-24T10:52:58.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:58 smithi096 ceph-mon[84258]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T10:52:58.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:58 smithi096 ceph-mon[84258]: osd.5 [v2:172.21.15.96:6816/853588375,v1:172.21.15.96:6817/853588375] boot 2024-03-24T10:52:58.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:58 smithi096 ceph-mon[84258]: osdmap e78: 6 total, 6 up, 6 in 2024-03-24T10:52:58.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T10:52:58.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:58.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:58.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:52:58.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:52:58.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:58.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:52:58.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:52:58.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:52:58.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:52:58.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:52:58.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:58 smithi096 ceph-mon[84258]: Upgrade: Setting container_image for all osd 2024-03-24T10:52:58.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:58.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-03-24T10:52:58.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-03-24T10:52:58.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-03-24T10:52:58.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-03-24T10:52:58.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-03-24T10:52:58.828 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-03-24T10:52:58.828 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-03-24T10:52:58.828 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-03-24T10:52:58.828 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-03-24T10:52:58.828 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-03-24T10:52:58.828 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-03-24T10:52:58.828 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-03-24T10:52:58.828 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:58 smithi096 ceph-mon[84258]: Upgrade: Setting require_osd_release to 19 squid 2024-03-24T10:52:58.828 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-03-24T10:52:58.828 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:58 smithi096 ceph-mon[84258]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-03-24T10:52:58.828 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-03-24T10:52:59.731 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:59 smithi096 ceph-mon[84258]: osdmap e79: 6 total, 6 up, 6 in 2024-03-24T10:52:59.731 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:59 smithi096 ceph-mon[84258]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-03-24T10:52:59.731 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:59 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:59.731 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:59 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-03-24T10:52:59.731 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:52:59 smithi096 ceph-mon[84258]: pgmap v165: 65 pgs: 12 peering, 6 active+undersized+degraded, 10 active+undersized, 37 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s; 12/219 objects degraded (5.479%) 2024-03-24T10:52:59.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:59 smithi094 ceph-mon[93957]: osdmap e79: 6 total, 6 up, 6 in 2024-03-24T10:52:59.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:59 smithi094 ceph-mon[93957]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-03-24T10:52:59.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:59 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:52:59.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:59 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-03-24T10:52:59.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:52:59 smithi094 ceph-mon[93957]: pgmap v165: 65 pgs: 12 peering, 6 active+undersized+degraded, 10 active+undersized, 37 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s; 12/219 objects degraded (5.479%) 2024-03-24T10:53:00.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:00 smithi094 ceph-mon[93957]: osdmap e80: 6 total, 6 up, 6 in 2024-03-24T10:53:00.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:00 smithi094 ceph-mon[93957]: Health check update: Degraded data redundancy: 12/219 objects degraded (5.479%), 6 pgs degraded (PG_DEGRADED) 2024-03-24T10:53:00.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:00 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-03-24T10:53:00.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:00 smithi094 ceph-mon[93957]: fsmap cephfs:1 {0=cephfs.smithi096.egbmoq=up:active} 2 up:standby 2024-03-24T10:53:00.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:00 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:53:00.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:00 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:53:00.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:00 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:53:00.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:00 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:00.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:00 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:53:00.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:00 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:53:00.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:00 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:53:00.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:00 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:53:00.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:00 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:53:00.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:00 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi094.efiqte"]}]: dispatch 2024-03-24T10:53:00.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:00 smithi094 ceph-mon[93957]: Upgrade: It appears safe to stop mds.cephfs.smithi094.efiqte 2024-03-24T10:53:00.754 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:00 smithi096 ceph-mon[84258]: osdmap e80: 6 total, 6 up, 6 in 2024-03-24T10:53:00.754 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:00 smithi096 ceph-mon[84258]: Health check update: Degraded data redundancy: 12/219 objects degraded (5.479%), 6 pgs degraded (PG_DEGRADED) 2024-03-24T10:53:00.754 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:00 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-03-24T10:53:00.754 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:00 smithi096 ceph-mon[84258]: fsmap cephfs:1 {0=cephfs.smithi096.egbmoq=up:active} 2 up:standby 2024-03-24T10:53:00.754 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:00 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:53:00.754 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:00 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:53:00.754 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:00 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:53:00.754 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:00 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:00.754 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:00 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:53:00.754 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:00 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:53:00.755 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:00 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:53:00.755 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:00 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:53:00.755 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:00 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:53:00.755 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:00 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi094.efiqte"]}]: dispatch 2024-03-24T10:53:00.755 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:00 smithi096 ceph-mon[84258]: Upgrade: It appears safe to stop mds.cephfs.smithi094.efiqte 2024-03-24T10:53:01.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:01 smithi096 ceph-mon[84258]: mds.? [v2:172.21.15.94:6828/3011883830,v1:172.21.15.94:6829/3011883830] up:boot 2024-03-24T10:53:01.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:01 smithi096 ceph-mon[84258]: fsmap cephfs:1 {0=cephfs.smithi096.egbmoq=up:active} 3 up:standby 2024-03-24T10:53:01.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:01 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.llefgu"}]: dispatch 2024-03-24T10:53:01.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:01 smithi096 ceph-mon[84258]: Upgrade: Updating mds.cephfs.smithi094.efiqte 2024-03-24T10:53:01.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:01 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:01.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:01 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi094.efiqte", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T10:53:01.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:01 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:53:01.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:01 smithi096 ceph-mon[84258]: Deploying daemon mds.cephfs.smithi094.efiqte on smithi094 2024-03-24T10:53:01.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:01 smithi096 ceph-mon[84258]: pgmap v167: 65 pgs: 12 peering, 6 active+undersized+degraded, 10 active+undersized, 37 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 634 B/s rd, 1 op/s; 12/219 objects degraded (5.479%) 2024-03-24T10:53:01.834 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:01 smithi094 ceph-mon[93957]: mds.? [v2:172.21.15.94:6828/3011883830,v1:172.21.15.94:6829/3011883830] up:boot 2024-03-24T10:53:01.834 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:01 smithi094 ceph-mon[93957]: fsmap cephfs:1 {0=cephfs.smithi096.egbmoq=up:active} 3 up:standby 2024-03-24T10:53:01.834 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:01 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.llefgu"}]: dispatch 2024-03-24T10:53:01.834 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:01 smithi094 ceph-mon[93957]: Upgrade: Updating mds.cephfs.smithi094.efiqte 2024-03-24T10:53:01.834 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:01 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:01.834 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:01 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi094.efiqte", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T10:53:01.834 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:01 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:53:01.835 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:01 smithi094 ceph-mon[93957]: Deploying daemon mds.cephfs.smithi094.efiqte on smithi094 2024-03-24T10:53:01.835 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:01 smithi094 ceph-mon[93957]: pgmap v167: 65 pgs: 12 peering, 6 active+undersized+degraded, 10 active+undersized, 37 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 634 B/s rd, 1 op/s; 12/219 objects degraded (5.479%) 2024-03-24T10:53:03.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:03 smithi094 ceph-mon[93957]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 12/219 objects degraded (5.479%), 6 pgs degraded) 2024-03-24T10:53:03.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:03 smithi094 ceph-mon[93957]: Cluster is now healthy 2024-03-24T10:53:03.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:03 smithi096 ceph-mon[84258]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 12/219 objects degraded (5.479%), 6 pgs degraded) 2024-03-24T10:53:03.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:03 smithi096 ceph-mon[84258]: Cluster is now healthy 2024-03-24T10:53:04.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:04 smithi094 ceph-mon[93957]: pgmap v168: 65 pgs: 7 peering, 58 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 511 B/s rd, 1 op/s 2024-03-24T10:53:04.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:04 smithi096 ceph-mon[84258]: pgmap v168: 65 pgs: 7 peering, 58 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 511 B/s rd, 1 op/s 2024-03-24T10:53:05.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:05 smithi094 ceph-mon[93957]: osdmap e81: 6 total, 6 up, 6 in 2024-03-24T10:53:05.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:05 smithi094 ceph-mon[93957]: fsmap cephfs:1 {0=cephfs.smithi096.egbmoq=up:active} 2 up:standby 2024-03-24T10:53:05.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:05 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:05.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:05 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:53:05.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:05 smithi096 ceph-mon[84258]: osdmap e81: 6 total, 6 up, 6 in 2024-03-24T10:53:05.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:05 smithi096 ceph-mon[84258]: fsmap cephfs:1 {0=cephfs.smithi096.egbmoq=up:active} 2 up:standby 2024-03-24T10:53:05.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:05 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:05.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:05 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:53:06.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:06 smithi094 ceph-mon[93957]: pgmap v170: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 149 B/s rd, 0 op/s 2024-03-24T10:53:06.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:06 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.efiqte"}]: dispatch 2024-03-24T10:53:06.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:06 smithi096 ceph-mon[84258]: pgmap v170: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 149 B/s rd, 0 op/s 2024-03-24T10:53:06.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:06 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.efiqte"}]: dispatch 2024-03-24T10:53:08.741 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:08 smithi094 ceph-mon[93957]: pgmap v171: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 127 B/s rd, 0 op/s 2024-03-24T10:53:08.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:08 smithi096 ceph-mon[84258]: pgmap v171: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 127 B/s rd, 0 op/s 2024-03-24T10:53:09.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:09 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:09.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:09 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:09.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:09 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:53:09.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:09 smithi094 ceph-mon[93957]: pgmap v172: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 104 B/s rd, 0 op/s 2024-03-24T10:53:09.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:09 smithi094 ceph-mon[93957]: mds.? [v2:172.21.15.94:6826/2043632450,v1:172.21.15.94:6827/2043632450] up:boot 2024-03-24T10:53:09.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:09 smithi094 ceph-mon[93957]: fsmap cephfs:1 {0=cephfs.smithi096.egbmoq=up:active} 3 up:standby 2024-03-24T10:53:09.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:09 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.efiqte"}]: dispatch 2024-03-24T10:53:10.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:09 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:10.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:09 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:10.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:09 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:53:10.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:09 smithi096 ceph-mon[84258]: pgmap v172: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 104 B/s rd, 0 op/s 2024-03-24T10:53:10.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:09 smithi096 ceph-mon[84258]: mds.? [v2:172.21.15.94:6826/2043632450,v1:172.21.15.94:6827/2043632450] up:boot 2024-03-24T10:53:10.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:09 smithi096 ceph-mon[84258]: fsmap cephfs:1 {0=cephfs.smithi096.egbmoq=up:active} 3 up:standby 2024-03-24T10:53:10.326 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:09 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.efiqte"}]: dispatch 2024-03-24T10:53:11.719 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:11 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:11.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:11 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:12.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:12 smithi094 ceph-mon[93957]: pgmap v173: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 102 B/s rd, 0 op/s 2024-03-24T10:53:12.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:12 smithi096 ceph-mon[84258]: pgmap v173: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 102 B/s rd, 0 op/s 2024-03-24T10:53:14.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:14 smithi094 ceph-mon[93957]: pgmap v174: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 102 B/s rd, 0 op/s 2024-03-24T10:53:14.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:14 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:14.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:14 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:14.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:14 smithi096 ceph-mon[84258]: pgmap v174: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 102 B/s rd, 0 op/s 2024-03-24T10:53:14.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:14 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:14.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:14 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:15.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:15 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:15.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:15 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:15.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:15 smithi094 ceph-mon[93957]: pgmap v175: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail 2024-03-24T10:53:16.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:15 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:16.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:15 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:16.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:15 smithi096 ceph-mon[84258]: pgmap v175: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail 2024-03-24T10:53:17.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:17.790+0000 7f632210c700 1 -- 172.21.15.94:0/116745620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f631c0fcf00 msgr2=0x7f631c0fd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:17.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:17.790+0000 7f632210c700 1 --2- 172.21.15.94:0/116745620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f631c0fcf00 0x7f631c0fd310 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f630c004940 tx=0x7f630c009e30 comp rx=0 tx=0).stop 2024-03-24T10:53:17.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:17.790+0000 7f632210c700 1 -- 172.21.15.94:0/116745620 shutdown_connections 2024-03-24T10:53:17.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:17.790+0000 7f632210c700 1 --2- 172.21.15.94:0/116745620 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f631c0fe070 0x7f631c0fe4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:17.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:17.790+0000 7f632210c700 1 --2- 172.21.15.94:0/116745620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f631c0fcf00 0x7f631c0fd310 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:17.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:17.790+0000 7f632210c700 1 -- 172.21.15.94:0/116745620 >> 172.21.15.94:0/116745620 conn(0x7f631c0f85a0 msgr2=0x7f631c0fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:53:17.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:17.790+0000 7f632210c700 1 -- 172.21.15.94:0/116745620 shutdown_connections 2024-03-24T10:53:17.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:17.790+0000 7f632210c700 1 -- 172.21.15.94:0/116745620 wait complete. 2024-03-24T10:53:17.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:17.790+0000 7f632210c700 1 Processor -- start 2024-03-24T10:53:17.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:17.791+0000 7f632210c700 1 -- start start 2024-03-24T10:53:17.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:17.791+0000 7f632210c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f631c0fcf00 0x7f631c10da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:53:17.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:17.791+0000 7f632210c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f631c0fe070 0x7f631c10dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:53:17.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:17.791+0000 7f632210c700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631c10e5d0 con 0x7f631c0fcf00 2024-03-24T10:53:17.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:17.791+0000 7f632210c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631c10edd0 con 0x7f631c0fe070 2024-03-24T10:53:17.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:17.791+0000 7f631b7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f631c0fcf00 0x7f631c10da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:53:17.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:17.791+0000 7f631b7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f631c0fcf00 0x7f631c10da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35934/0 (socket says 172.21.15.94:35934) 2024-03-24T10:53:17.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:17.791+0000 7f631b7fe700 1 -- 172.21.15.94:0/4111580053 learned_addr learned my addr 172.21.15.94:0/4111580053 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:53:17.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:17.791+0000 7f6312ffd700 1 --2- 172.21.15.94:0/4111580053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f631c0fe070 0x7f631c10dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:53:17.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:17.791+0000 7f631b7fe700 1 -- 172.21.15.94:0/4111580053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f631c0fe070 msgr2=0x7f631c10dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:17.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:17.791+0000 7f631b7fe700 1 --2- 172.21.15.94:0/4111580053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f631c0fe070 0x7f631c10dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:17.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:17.791+0000 7f631b7fe700 1 -- 172.21.15.94:0/4111580053 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f630c005040 con 0x7f631c0fcf00 2024-03-24T10:53:17.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:17.791+0000 7f631b7fe700 1 --2- 172.21.15.94:0/4111580053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f631c0fcf00 0x7f631c10da90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f630c009150 tx=0x7f630c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:53:17.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:17.791+0000 7f6318ff9700 1 -- 172.21.15.94:0/4111580053 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f630c00e040 con 0x7f631c0fcf00 2024-03-24T10:53:17.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:17.791+0000 7f632210c700 1 -- 172.21.15.94:0/4111580053 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f631c10f050 con 0x7f631c0fcf00 2024-03-24T10:53:17.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:17.791+0000 7f6318ff9700 1 -- 172.21.15.94:0/4111580053 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f630c00bcc0 con 0x7f631c0fcf00 2024-03-24T10:53:17.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:17.791+0000 7f6318ff9700 1 -- 172.21.15.94:0/4111580053 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f630c01c630 con 0x7f631c0fcf00 2024-03-24T10:53:17.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:17.792+0000 7f632210c700 1 -- 172.21.15.94:0/4111580053 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f631c10f590 con 0x7f631c0fcf00 2024-03-24T10:53:17.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:17.793+0000 7f632210c700 1 -- 172.21.15.94:0/4111580053 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f631c05f5f0 con 0x7f631c0fcf00 2024-03-24T10:53:17.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:17.793+0000 7f6318ff9700 1 -- 172.21.15.94:0/4111580053 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f630c013420 con 0x7f631c0fcf00 2024-03-24T10:53:17.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:17.793+0000 7f6318ff9700 1 --2- 172.21.15.94:0/4111580053 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f630806d7e0 0x7f630806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:53:17.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:17.794+0000 7f6312ffd700 1 --2- 172.21.15.94:0/4111580053 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f630806d7e0 0x7f630806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:53:17.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:17.794+0000 7f6318ff9700 1 -- 172.21.15.94:0/4111580053 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f630c08dbd0 con 0x7f631c0fcf00 2024-03-24T10:53:17.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:17.794+0000 7f6312ffd700 1 --2- 172.21.15.94:0/4111580053 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f630806d7e0 0x7f630806fc90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6304007fd0 tx=0x7f6304009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:53:17.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:17.798+0000 7f6318ff9700 1 -- 172.21.15.94:0/4111580053 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f630c05af70 con 0x7f631c0fcf00 2024-03-24T10:53:18.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.014+0000 7f632210c700 1 -- 172.21.15.94:0/4111580053 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f631c10f820 con 0x7f630806d7e0 2024-03-24T10:53:18.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.015+0000 7f6318ff9700 1 -- 172.21.15.94:0/4111580053 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f631c10f820 con 0x7f630806d7e0 2024-03-24T10:53:18.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.017+0000 7f632210c700 1 -- 172.21.15.94:0/4111580053 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f630806d7e0 msgr2=0x7f630806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:18.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.017+0000 7f632210c700 1 --2- 172.21.15.94:0/4111580053 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f630806d7e0 0x7f630806fc90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6304007fd0 tx=0x7f6304009040 comp rx=0 tx=0).stop 2024-03-24T10:53:18.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.017+0000 7f632210c700 1 -- 172.21.15.94:0/4111580053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f631c0fcf00 msgr2=0x7f631c10da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:18.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.017+0000 7f632210c700 1 --2- 172.21.15.94:0/4111580053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f631c0fcf00 0x7f631c10da90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f630c009150 tx=0x7f630c009e30 comp rx=0 tx=0).stop 2024-03-24T10:53:18.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.017+0000 7f632210c700 1 -- 172.21.15.94:0/4111580053 shutdown_connections 2024-03-24T10:53:18.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.017+0000 7f632210c700 1 --2- 172.21.15.94:0/4111580053 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f630806d7e0 0x7f630806fc90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:18.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.017+0000 7f632210c700 1 --2- 172.21.15.94:0/4111580053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f631c0fcf00 0x7f631c10da90 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:18.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.017+0000 7f632210c700 1 --2- 172.21.15.94:0/4111580053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f631c0fe070 0x7f631c10dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:18.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.017+0000 7f632210c700 1 -- 172.21.15.94:0/4111580053 >> 172.21.15.94:0/4111580053 conn(0x7f631c0f85a0 msgr2=0x7f631c0f9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:53:18.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.017+0000 7f632210c700 1 -- 172.21.15.94:0/4111580053 shutdown_connections 2024-03-24T10:53:18.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.017+0000 7f632210c700 1 -- 172.21.15.94:0/4111580053 wait complete. 2024-03-24T10:53:18.028 INFO:teuthology.orchestra.run.smithi094.stdout:true 2024-03-24T10:53:18.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.125+0000 7f4a96dc5700 1 -- 172.21.15.94:0/3738683841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a900fcf80 msgr2=0x7f4a900fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:18.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.125+0000 7f4a96dc5700 1 --2- 172.21.15.94:0/3738683841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a900fcf80 0x7f4a900fd3f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c004940 tx=0x7f4a7c009e30 comp rx=0 tx=0).stop 2024-03-24T10:53:18.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.125+0000 7f4a96dc5700 1 -- 172.21.15.94:0/3738683841 shutdown_connections 2024-03-24T10:53:18.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.125+0000 7f4a96dc5700 1 --2- 172.21.15.94:0/3738683841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a900fcf80 0x7f4a900fd3f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:18.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.125+0000 7f4a96dc5700 1 --2- 172.21.15.94:0/3738683841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a900fbcd0 0x7f4a900fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:18.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.125+0000 7f4a96dc5700 1 -- 172.21.15.94:0/3738683841 >> 172.21.15.94:0/3738683841 conn(0x7f4a900f7430 msgr2=0x7f4a900f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:53:18.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.125+0000 7f4a96dc5700 1 -- 172.21.15.94:0/3738683841 shutdown_connections 2024-03-24T10:53:18.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.125+0000 7f4a96dc5700 1 -- 172.21.15.94:0/3738683841 wait complete. 2024-03-24T10:53:18.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.125+0000 7f4a96dc5700 1 Processor -- start 2024-03-24T10:53:18.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.125+0000 7f4a96dc5700 1 -- start start 2024-03-24T10:53:18.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.125+0000 7f4a96dc5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a900fbcd0 0x7f4a90110720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:53:18.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.126+0000 7f4a96dc5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a900fcf80 0x7f4a90112c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:53:18.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.126+0000 7f4a96dc5700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a90066350 con 0x7f4a900fbcd0 2024-03-24T10:53:18.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.126+0000 7f4a96dc5700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a90066490 con 0x7f4a900fcf80 2024-03-24T10:53:18.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.126+0000 7f4a94b61700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a900fbcd0 0x7f4a90110720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:53:18.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.126+0000 7f4a94b61700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a900fbcd0 0x7f4a90110720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35954/0 (socket says 172.21.15.94:35954) 2024-03-24T10:53:18.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.126+0000 7f4a94b61700 1 -- 172.21.15.94:0/2045501440 learned_addr learned my addr 172.21.15.94:0/2045501440 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:53:18.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.126+0000 7f4a8ffff700 1 --2- 172.21.15.94:0/2045501440 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a900fcf80 0x7f4a90112c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:53:18.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.126+0000 7f4a94b61700 1 -- 172.21.15.94:0/2045501440 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a900fcf80 msgr2=0x7f4a90112c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:18.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.126+0000 7f4a94b61700 1 --2- 172.21.15.94:0/2045501440 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a900fcf80 0x7f4a90112c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:18.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.126+0000 7f4a94b61700 1 -- 172.21.15.94:0/2045501440 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a7c005040 con 0x7f4a900fbcd0 2024-03-24T10:53:18.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.126+0000 7f4a94b61700 1 --2- 172.21.15.94:0/2045501440 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a900fbcd0 0x7f4a90110720 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4a8400c8c0 tx=0x7f4a840044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:53:18.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.126+0000 7f4a8d7fa700 1 -- 172.21.15.94:0/2045501440 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a8400a760 con 0x7f4a900fbcd0 2024-03-24T10:53:18.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.126+0000 7f4a96dc5700 1 -- 172.21.15.94:0/2045501440 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a90113210 con 0x7f4a900fbcd0 2024-03-24T10:53:18.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.126+0000 7f4a8d7fa700 1 -- 172.21.15.94:0/2045501440 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4a8400ad90 con 0x7f4a900fbcd0 2024-03-24T10:53:18.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.127+0000 7f4a8d7fa700 1 -- 172.21.15.94:0/2045501440 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a840113a0 con 0x7f4a900fbcd0 2024-03-24T10:53:18.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.127+0000 7f4a96dc5700 1 -- 172.21.15.94:0/2045501440 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a901137d0 con 0x7f4a900fbcd0 2024-03-24T10:53:18.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.127+0000 7f4a96dc5700 1 -- 172.21.15.94:0/2045501440 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a9005f5f0 con 0x7f4a900fbcd0 2024-03-24T10:53:18.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.128+0000 7f4a8d7fa700 1 -- 172.21.15.94:0/2045501440 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f4a84012400 con 0x7f4a900fbcd0 2024-03-24T10:53:18.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.128+0000 7f4a8d7fa700 1 --2- 172.21.15.94:0/2045501440 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f4a8006d7e0 0x7f4a8006fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:53:18.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.129+0000 7f4a8d7fa700 1 -- 172.21.15.94:0/2045501440 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f4a8408ce80 con 0x7f4a900fbcd0 2024-03-24T10:53:18.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.129+0000 7f4a8ffff700 1 --2- 172.21.15.94:0/2045501440 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f4a8006d7e0 0x7f4a8006fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:53:18.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.129+0000 7f4a8ffff700 1 --2- 172.21.15.94:0/2045501440 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f4a8006d7e0 0x7f4a8006fc90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c00b970 tx=0x7f4a7c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:53:18.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.134+0000 7f4a8d7fa700 1 -- 172.21.15.94:0/2045501440 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4a84056790 con 0x7f4a900fbcd0 2024-03-24T10:53:18.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.361+0000 7f4a96dc5700 1 -- 172.21.15.94:0/2045501440 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a90101360 con 0x7f4a8006d7e0 2024-03-24T10:53:18.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.363+0000 7f4a8d7fa700 1 -- 172.21.15.94:0/2045501440 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4a90101360 con 0x7f4a8006d7e0 2024-03-24T10:53:18.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.365+0000 7f4a96dc5700 1 -- 172.21.15.94:0/2045501440 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f4a8006d7e0 msgr2=0x7f4a8006fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:18.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.365+0000 7f4a96dc5700 1 --2- 172.21.15.94:0/2045501440 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f4a8006d7e0 0x7f4a8006fc90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c00b970 tx=0x7f4a7c009e30 comp rx=0 tx=0).stop 2024-03-24T10:53:18.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.365+0000 7f4a96dc5700 1 -- 172.21.15.94:0/2045501440 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a900fbcd0 msgr2=0x7f4a90110720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:18.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.365+0000 7f4a96dc5700 1 --2- 172.21.15.94:0/2045501440 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a900fbcd0 0x7f4a90110720 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4a8400c8c0 tx=0x7f4a840044c0 comp rx=0 tx=0).stop 2024-03-24T10:53:18.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.366+0000 7f4a96dc5700 1 -- 172.21.15.94:0/2045501440 shutdown_connections 2024-03-24T10:53:18.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.366+0000 7f4a96dc5700 1 --2- 172.21.15.94:0/2045501440 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f4a8006d7e0 0x7f4a8006fc90 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:18.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.366+0000 7f4a96dc5700 1 --2- 172.21.15.94:0/2045501440 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a900fbcd0 0x7f4a90110720 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:18.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.366+0000 7f4a96dc5700 1 --2- 172.21.15.94:0/2045501440 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a900fcf80 0x7f4a90112c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:18.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.366+0000 7f4a96dc5700 1 -- 172.21.15.94:0/2045501440 >> 172.21.15.94:0/2045501440 conn(0x7f4a900f7430 msgr2=0x7f4a900f8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:53:18.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.366+0000 7f4a96dc5700 1 -- 172.21.15.94:0/2045501440 shutdown_connections 2024-03-24T10:53:18.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.366+0000 7f4a96dc5700 1 -- 172.21.15.94:0/2045501440 wait complete. 2024-03-24T10:53:18.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.474+0000 7f66b5ce1700 1 -- 172.21.15.94:0/3908566884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66b00fcf90 msgr2=0x7f66b00fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:18.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.474+0000 7f66b5ce1700 1 --2- 172.21.15.94:0/3908566884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66b00fcf90 0x7f66b00fd400 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6698004940 tx=0x7f6698009e30 comp rx=0 tx=0).stop 2024-03-24T10:53:18.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.474+0000 7f66b5ce1700 1 -- 172.21.15.94:0/3908566884 shutdown_connections 2024-03-24T10:53:18.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.474+0000 7f66b5ce1700 1 --2- 172.21.15.94:0/3908566884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66b00fcf90 0x7f66b00fd400 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:18.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.474+0000 7f66b5ce1700 1 --2- 172.21.15.94:0/3908566884 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66b00fbce0 0x7f66b00fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:18.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.474+0000 7f66b5ce1700 1 -- 172.21.15.94:0/3908566884 >> 172.21.15.94:0/3908566884 conn(0x7f66b00f7440 msgr2=0x7f66b00f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:53:18.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.474+0000 7f66b5ce1700 1 -- 172.21.15.94:0/3908566884 shutdown_connections 2024-03-24T10:53:18.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.474+0000 7f66b5ce1700 1 -- 172.21.15.94:0/3908566884 wait complete. 2024-03-24T10:53:18.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.475+0000 7f66b5ce1700 1 Processor -- start 2024-03-24T10:53:18.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.475+0000 7f66b5ce1700 1 -- start start 2024-03-24T10:53:18.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.475+0000 7f66b5ce1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66b00fbce0 0x7f66b010fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:53:18.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.475+0000 7f66b5ce1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66b00fcf90 0x7f66b0110250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:53:18.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.475+0000 7f66b5ce1700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b0110850 con 0x7f66b00fcf90 2024-03-24T10:53:18.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.475+0000 7f66b5ce1700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b0110f90 con 0x7f66b00fbce0 2024-03-24T10:53:18.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.475+0000 7f66a7fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66b00fcf90 0x7f66b0110250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:53:18.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.475+0000 7f66a7fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66b00fcf90 0x7f66b0110250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35968/0 (socket says 172.21.15.94:35968) 2024-03-24T10:53:18.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.475+0000 7f66a7fff700 1 -- 172.21.15.94:0/1467237650 learned_addr learned my addr 172.21.15.94:0/1467237650 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:53:18.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.475+0000 7f66af7fe700 1 --2- 172.21.15.94:0/1467237650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66b00fbce0 0x7f66b010fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:53:18.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.475+0000 7f66a7fff700 1 -- 172.21.15.94:0/1467237650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66b00fbce0 msgr2=0x7f66b010fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:18.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.475+0000 7f66a7fff700 1 --2- 172.21.15.94:0/1467237650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66b00fbce0 0x7f66b010fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:18.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.475+0000 7f66a7fff700 1 -- 172.21.15.94:0/1467237650 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6698005040 con 0x7f66b00fcf90 2024-03-24T10:53:18.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.475+0000 7f66a7fff700 1 --2- 172.21.15.94:0/1467237650 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66b00fcf90 0x7f66b0110250 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f669800b3b0 tx=0x7f6698009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:53:18.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.476+0000 7f66acff9700 1 -- 172.21.15.94:0/1467237650 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f669800e040 con 0x7f66b00fcf90 2024-03-24T10:53:18.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.476+0000 7f66b5ce1700 1 -- 172.21.15.94:0/1467237650 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66b0111210 con 0x7f66b00fcf90 2024-03-24T10:53:18.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.476+0000 7f66acff9700 1 -- 172.21.15.94:0/1467237650 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f669800bcc0 con 0x7f66b00fcf90 2024-03-24T10:53:18.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.476+0000 7f66acff9700 1 -- 172.21.15.94:0/1467237650 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f669801c630 con 0x7f66b00fcf90 2024-03-24T10:53:18.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.476+0000 7f66b5ce1700 1 -- 172.21.15.94:0/1467237650 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66b01117d0 con 0x7f66b00fcf90 2024-03-24T10:53:18.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.477+0000 7f66b5ce1700 1 -- 172.21.15.94:0/1467237650 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66b0101920 con 0x7f66b00fcf90 2024-03-24T10:53:18.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.477+0000 7f66acff9700 1 -- 172.21.15.94:0/1467237650 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f6698013980 con 0x7f66b00fcf90 2024-03-24T10:53:18.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.478+0000 7f66acff9700 1 --2- 172.21.15.94:0/1467237650 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f669c06d7e0 0x7f669c06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:53:18.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.478+0000 7f66acff9700 1 -- 172.21.15.94:0/1467237650 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f669808dc10 con 0x7f66b00fcf90 2024-03-24T10:53:18.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.478+0000 7f66af7fe700 1 --2- 172.21.15.94:0/1467237650 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f669c06d7e0 0x7f669c06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:53:18.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.478+0000 7f66af7fe700 1 --2- 172.21.15.94:0/1467237650 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f669c06d7e0 0x7f669c06fc90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f66a0007fd0 tx=0x7f66a0009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:53:18.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.482+0000 7f66acff9700 1 -- 172.21.15.94:0/1467237650 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f669805af30 con 0x7f66b00fcf90 2024-03-24T10:53:18.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.696+0000 7f66b5ce1700 1 -- 172.21.15.94:0/1467237650 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f66b01119f0 con 0x7f669c06d7e0 2024-03-24T10:53:18.697 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:18 smithi094 ceph-mon[93957]: pgmap v176: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail 2024-03-24T10:53:18.697 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:18 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:18.697 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:18 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:18.697 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:18 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:53:18.698 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:18 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:53:18.698 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:18 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:18.698 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:18 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:53:18.698 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:18 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:53:18.698 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:18 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:53:18.698 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:18 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:53:18.698 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:18 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:53:18.698 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:18 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi094.llefgu"]}]: dispatch 2024-03-24T10:53:18.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.707+0000 7f66acff9700 1 -- 172.21.15.94:0/1467237650 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f66b01119f0 con 0x7f669c06d7e0 2024-03-24T10:53:18.708 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T10:53:18.708 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.smithi094 smithi094 *:9093,9094 running (6m) 5s ago 15m 24.4M - 0.25.0 c8568f914cd2 5c583af60554 2024-03-24T10:53:18.708 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi094 smithi094 running (3m) 5s ago 15m 6651k - 19.0.0-1855-g21b3bd67 553463a90fd5 2b0897df571d 2024-03-24T10:53:18.708 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi096 smithi096 running (3m) 25s ago 14m 6669k - 19.0.0-1855-g21b3bd67 553463a90fd5 1759a9ff2805 2024-03-24T10:53:18.708 INFO:teuthology.orchestra.run.smithi094.stdout:grafana.smithi094 smithi094 *:3000 running (6m) 5s ago 14m 84.6M - 9.4.7 954c08fa6188 d3d425bf1b7c 2024-03-24T10:53:18.708 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.efiqte smithi094 running (10s) 5s ago 9m 13.8M - 19.0.0-1855-g21b3bd67 553463a90fd5 732c47b17971 2024-03-24T10:53:18.708 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.llefgu smithi094 running (9m) 5s ago 9m 10.4M - 17.2.7-726-g9c4e0c34 837124da804f 52a9771dc8e0 2024-03-24T10:53:18.708 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi096.egbmoq smithi096 running (9m) 25s ago 9m 154M - 17.2.7-726-g9c4e0c34 837124da804f 1cfc930dbab0 2024-03-24T10:53:18.709 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi096.jtebhh smithi096 running (9m) 25s ago 9m 17.1M - 17.2.7-726-g9c4e0c34 837124da804f 4ff3451a6f6e 2024-03-24T10:53:18.709 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi094.yaethu smithi094 *:8443,9283,8765 running (5m) 5s ago 16m 505M - 19.0.0-1855-g21b3bd67 553463a90fd5 d3adab76b18f 2024-03-24T10:53:18.709 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi096.gfonnu smithi096 *:8443,9283,8765 running (5m) 25s ago 13m 450M - 19.0.0-1855-g21b3bd67 553463a90fd5 22937a059eb8 2024-03-24T10:53:18.709 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi094 smithi094 running (4m) 5s ago 17m 51.0M 2048M 19.0.0-1855-g21b3bd67 553463a90fd5 c779d611e29a 2024-03-24T10:53:18.709 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi096 smithi096 running (4m) 25s ago 13m 37.8M 2048M 19.0.0-1855-g21b3bd67 553463a90fd5 ed9bd0bd57f6 2024-03-24T10:53:18.709 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi094 smithi094 *:9100 running (6m) 5s ago 15m 18.5M - 1.5.0 0da6a335fe13 ca67b53c64e2 2024-03-24T10:53:18.709 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi096 smithi096 *:9100 running (5m) 25s ago 14m 16.7M - 1.5.0 0da6a335fe13 65137a7a7549 2024-03-24T10:53:18.709 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (3m) 5s ago 13m 103M 1557M 19.0.0-1855-g21b3bd67 553463a90fd5 ec57ede0c9e4 2024-03-24T10:53:18.709 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi094 running (2m) 5s ago 12m 111M 1557M 19.0.0-1855-g21b3bd67 553463a90fd5 c5b8b2bbf8b5 2024-03-24T10:53:18.709 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi094 running (2m) 5s ago 12m 92.3M 1557M 19.0.0-1855-g21b3bd67 553463a90fd5 c961dcdc746f 2024-03-24T10:53:18.709 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi096 running (96s) 25s ago 11m 105M 2581M 19.0.0-1855-g21b3bd67 553463a90fd5 6323fdc6e6ca 2024-03-24T10:53:18.709 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi096 running (62s) 25s ago 11m 106M 2581M 19.0.0-1855-g21b3bd67 553463a90fd5 3e2536bcb263 2024-03-24T10:53:18.709 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi096 running (28s) 25s ago 11m 14.8M 2581M 19.0.0-1855-g21b3bd67 553463a90fd5 2f77f1fa20e2 2024-03-24T10:53:18.709 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.smithi094 smithi094 *:9095 running (5m) 5s ago 14m 58.9M - 2.43.0 a07b618ecd1d 6ce091a0c326 2024-03-24T10:53:18.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.709+0000 7f66b5ce1700 1 -- 172.21.15.94:0/1467237650 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f669c06d7e0 msgr2=0x7f669c06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:18.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.709+0000 7f66b5ce1700 1 --2- 172.21.15.94:0/1467237650 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f669c06d7e0 0x7f669c06fc90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f66a0007fd0 tx=0x7f66a0009040 comp rx=0 tx=0).stop 2024-03-24T10:53:18.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.709+0000 7f66b5ce1700 1 -- 172.21.15.94:0/1467237650 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66b00fcf90 msgr2=0x7f66b0110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:18.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.709+0000 7f66b5ce1700 1 --2- 172.21.15.94:0/1467237650 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66b00fcf90 0x7f66b0110250 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f669800b3b0 tx=0x7f6698009e30 comp rx=0 tx=0).stop 2024-03-24T10:53:18.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.709+0000 7f66b5ce1700 1 -- 172.21.15.94:0/1467237650 shutdown_connections 2024-03-24T10:53:18.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.709+0000 7f66b5ce1700 1 --2- 172.21.15.94:0/1467237650 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f669c06d7e0 0x7f669c06fc90 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:18.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.709+0000 7f66b5ce1700 1 --2- 172.21.15.94:0/1467237650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66b00fbce0 0x7f66b010fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:18.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.709+0000 7f66b5ce1700 1 --2- 172.21.15.94:0/1467237650 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66b00fcf90 0x7f66b0110250 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:18.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.709+0000 7f66b5ce1700 1 -- 172.21.15.94:0/1467237650 >> 172.21.15.94:0/1467237650 conn(0x7f66b00f7440 msgr2=0x7f66b00664c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:53:18.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.709+0000 7f66b5ce1700 1 -- 172.21.15.94:0/1467237650 shutdown_connections 2024-03-24T10:53:18.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.709+0000 7f66b5ce1700 1 -- 172.21.15.94:0/1467237650 wait complete. 2024-03-24T10:53:18.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.819+0000 7f16a78e9700 1 -- 172.21.15.94:0/2283636695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16a00fce70 msgr2=0x7f16a00fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:18.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.819+0000 7f16a78e9700 1 --2- 172.21.15.94:0/2283636695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16a00fce70 0x7f16a00fd280 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1690004940 tx=0x7f1690009e30 comp rx=0 tx=0).stop 2024-03-24T10:53:18.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.819+0000 7f16a78e9700 1 -- 172.21.15.94:0/2283636695 shutdown_connections 2024-03-24T10:53:18.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.819+0000 7f16a78e9700 1 --2- 172.21.15.94:0/2283636695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16a00fdfe0 0x7f16a00fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:18.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.819+0000 7f16a78e9700 1 --2- 172.21.15.94:0/2283636695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16a00fce70 0x7f16a00fd280 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:18.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.819+0000 7f16a78e9700 1 -- 172.21.15.94:0/2283636695 >> 172.21.15.94:0/2283636695 conn(0x7f16a00f8550 msgr2=0x7f16a00fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:53:18.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.819+0000 7f16a78e9700 1 -- 172.21.15.94:0/2283636695 shutdown_connections 2024-03-24T10:53:18.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.819+0000 7f16a78e9700 1 -- 172.21.15.94:0/2283636695 wait complete. 2024-03-24T10:53:18.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.820+0000 7f16a78e9700 1 Processor -- start 2024-03-24T10:53:18.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.820+0000 7f16a78e9700 1 -- start start 2024-03-24T10:53:18.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.820+0000 7f16a78e9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16a00fce70 0x7f16a0071c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:53:18.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.820+0000 7f16a78e9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16a00fdfe0 0x7f16a0070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:53:18.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.820+0000 7f16a78e9700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a0070850 con 0x7f16a00fdfe0 2024-03-24T10:53:18.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.820+0000 7f16a78e9700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a00709c0 con 0x7f16a00fce70 2024-03-24T10:53:18.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.820+0000 7f16a4e84700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16a00fdfe0 0x7f16a0070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:53:18.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.820+0000 7f16a4e84700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16a00fdfe0 0x7f16a0070310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35994/0 (socket says 172.21.15.94:35994) 2024-03-24T10:53:18.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.820+0000 7f16a4e84700 1 -- 172.21.15.94:0/789835880 learned_addr learned my addr 172.21.15.94:0/789835880 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:53:18.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.820+0000 7f16a5685700 1 --2- 172.21.15.94:0/789835880 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16a00fce70 0x7f16a0071c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:53:18.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.821+0000 7f16a4e84700 1 -- 172.21.15.94:0/789835880 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16a00fce70 msgr2=0x7f16a0071c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:18.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.821+0000 7f16a4e84700 1 --2- 172.21.15.94:0/789835880 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16a00fce70 0x7f16a0071c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:18.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.821+0000 7f16a4e84700 1 -- 172.21.15.94:0/789835880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1690005040 con 0x7f16a00fdfe0 2024-03-24T10:53:18.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.821+0000 7f16a4e84700 1 --2- 172.21.15.94:0/789835880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16a00fdfe0 0x7f16a0070310 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f169c00b790 tx=0x7f169c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:53:18.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.821+0000 7f1695ffb700 1 -- 172.21.15.94:0/789835880 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f169c00f6f0 con 0x7f16a00fdfe0 2024-03-24T10:53:18.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.821+0000 7f16a78e9700 1 -- 172.21.15.94:0/789835880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16a0070ca0 con 0x7f16a00fdfe0 2024-03-24T10:53:18.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.821+0000 7f1695ffb700 1 -- 172.21.15.94:0/789835880 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f169c00fd20 con 0x7f16a00fdfe0 2024-03-24T10:53:18.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.821+0000 7f1695ffb700 1 -- 172.21.15.94:0/789835880 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f169c0103a0 con 0x7f16a00fdfe0 2024-03-24T10:53:18.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.821+0000 7f16a78e9700 1 -- 172.21.15.94:0/789835880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16a0115a50 con 0x7f16a00fdfe0 2024-03-24T10:53:18.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.822+0000 7f16a78e9700 1 -- 172.21.15.94:0/789835880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16a004c710 con 0x7f16a00fdfe0 2024-03-24T10:53:18.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.822+0000 7f1695ffb700 1 -- 172.21.15.94:0/789835880 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f169c011400 con 0x7f16a00fdfe0 2024-03-24T10:53:18.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.823+0000 7f1695ffb700 1 --2- 172.21.15.94:0/789835880 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f168c06d7e0 0x7f168c06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:53:18.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.823+0000 7f1695ffb700 1 -- 172.21.15.94:0/789835880 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f169c08cd10 con 0x7f16a00fdfe0 2024-03-24T10:53:18.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.823+0000 7f16a5685700 1 --2- 172.21.15.94:0/789835880 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f168c06d7e0 0x7f168c06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:53:18.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.823+0000 7f16a5685700 1 --2- 172.21.15.94:0/789835880 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f168c06d7e0 0x7f168c06fc90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f169000b970 tx=0x7f1690009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:53:18.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:18 smithi096 ceph-mon[84258]: pgmap v176: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail 2024-03-24T10:53:18.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:18 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:18.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:18 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:18.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:18 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:53:18.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:18 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:53:18.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:18 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:18.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:18 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:53:18.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:18 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:53:18.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:18 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:53:18.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:18 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:53:18.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:18 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:53:18.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:18 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi094.llefgu"]}]: dispatch 2024-03-24T10:53:18.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:18.827+0000 7f1695ffb700 1 -- 172.21.15.94:0/789835880 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f169c05a0b0 con 0x7f16a00fdfe0 2024-03-24T10:53:19.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.118+0000 7f16a78e9700 1 -- 172.21.15.94:0/789835880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f16a0105950 con 0x7f16a00fdfe0 2024-03-24T10:53:19.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.119+0000 7f1695ffb700 1 -- 172.21.15.94:0/789835880 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f169c059c40 con 0x7f16a00fdfe0 2024-03-24T10:53:19.120 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-03-24T10:53:19.120 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": { 2024-03-24T10:53:19.120 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 2 2024-03-24T10:53:19.120 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:53:19.120 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": { 2024-03-24T10:53:19.120 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 2 2024-03-24T10:53:19.120 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:53:19.120 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": { 2024-03-24T10:53:19.120 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 6 2024-03-24T10:53:19.120 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:53:19.120 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-03-24T10:53:19.120 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 3, 2024-03-24T10:53:19.120 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 1 2024-03-24T10:53:19.120 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:53:19.120 INFO:teuthology.orchestra.run.smithi094.stdout: "overall": { 2024-03-24T10:53:19.120 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 3, 2024-03-24T10:53:19.121 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 11 2024-03-24T10:53:19.121 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-03-24T10:53:19.121 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-03-24T10:53:19.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.121+0000 7f16a78e9700 1 -- 172.21.15.94:0/789835880 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f168c06d7e0 msgr2=0x7f168c06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:19.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.121+0000 7f16a78e9700 1 --2- 172.21.15.94:0/789835880 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f168c06d7e0 0x7f168c06fc90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f169000b970 tx=0x7f1690009e30 comp rx=0 tx=0).stop 2024-03-24T10:53:19.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.121+0000 7f16a78e9700 1 -- 172.21.15.94:0/789835880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16a00fdfe0 msgr2=0x7f16a0070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:19.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.121+0000 7f16a78e9700 1 --2- 172.21.15.94:0/789835880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16a00fdfe0 0x7f16a0070310 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f169c00b790 tx=0x7f169c009250 comp rx=0 tx=0).stop 2024-03-24T10:53:19.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.121+0000 7f16a78e9700 1 -- 172.21.15.94:0/789835880 shutdown_connections 2024-03-24T10:53:19.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.121+0000 7f16a78e9700 1 --2- 172.21.15.94:0/789835880 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f168c06d7e0 0x7f168c06fc90 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:19.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.121+0000 7f16a78e9700 1 --2- 172.21.15.94:0/789835880 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16a00fce70 0x7f16a0071c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:19.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.121+0000 7f16a78e9700 1 --2- 172.21.15.94:0/789835880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16a00fdfe0 0x7f16a0070310 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:19.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.121+0000 7f16a78e9700 1 -- 172.21.15.94:0/789835880 >> 172.21.15.94:0/789835880 conn(0x7f16a00f8550 msgr2=0x7f16a00f9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:53:19.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.121+0000 7f16a78e9700 1 -- 172.21.15.94:0/789835880 shutdown_connections 2024-03-24T10:53:19.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.121+0000 7f16a78e9700 1 -- 172.21.15.94:0/789835880 wait complete. 2024-03-24T10:53:19.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.226+0000 7fba7c24c700 1 -- 172.21.15.94:0/149357702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba740fdfe0 msgr2=0x7fba740fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:19.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.226+0000 7fba7c24c700 1 --2- 172.21.15.94:0/149357702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba740fdfe0 0x7fba740fe450 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fba64004940 tx=0x7fba64009e30 comp rx=0 tx=0).stop 2024-03-24T10:53:19.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.226+0000 7fba7c24c700 1 -- 172.21.15.94:0/149357702 shutdown_connections 2024-03-24T10:53:19.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.226+0000 7fba7c24c700 1 --2- 172.21.15.94:0/149357702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba740fdfe0 0x7fba740fe450 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:19.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.226+0000 7fba7c24c700 1 --2- 172.21.15.94:0/149357702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba740fce70 0x7fba740fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:19.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.226+0000 7fba7c24c700 1 -- 172.21.15.94:0/149357702 >> 172.21.15.94:0/149357702 conn(0x7fba740f8550 msgr2=0x7fba740fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:53:19.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.226+0000 7fba7c24c700 1 -- 172.21.15.94:0/149357702 shutdown_connections 2024-03-24T10:53:19.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.226+0000 7fba7c24c700 1 -- 172.21.15.94:0/149357702 wait complete. 2024-03-24T10:53:19.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.226+0000 7fba7c24c700 1 Processor -- start 2024-03-24T10:53:19.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.226+0000 7fba7c24c700 1 -- start start 2024-03-24T10:53:19.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.227+0000 7fba7c24c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba740fce70 0x7fba74071c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:53:19.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.227+0000 7fba7c24c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba740fdfe0 0x7fba74070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:53:19.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.227+0000 7fba7c24c700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba74070850 con 0x7fba740fce70 2024-03-24T10:53:19.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.227+0000 7fba7c24c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba740709c0 con 0x7fba740fdfe0 2024-03-24T10:53:19.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.227+0000 7fba79fe8700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba740fce70 0x7fba74071c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:53:19.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.227+0000 7fba79fe8700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba740fce70 0x7fba74071c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36012/0 (socket says 172.21.15.94:36012) 2024-03-24T10:53:19.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.227+0000 7fba79fe8700 1 -- 172.21.15.94:0/66162083 learned_addr learned my addr 172.21.15.94:0/66162083 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:53:19.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.227+0000 7fba79fe8700 1 -- 172.21.15.94:0/66162083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba740fdfe0 msgr2=0x7fba74070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:19.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.227+0000 7fba79fe8700 1 --2- 172.21.15.94:0/66162083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba740fdfe0 0x7fba74070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:19.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.227+0000 7fba79fe8700 1 -- 172.21.15.94:0/66162083 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba64005040 con 0x7fba740fce70 2024-03-24T10:53:19.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.227+0000 7fba79fe8700 1 --2- 172.21.15.94:0/66162083 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba740fce70 0x7fba74071c60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fba7000b790 tx=0x7fba70009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:53:19.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.227+0000 7fba6a7fc700 1 -- 172.21.15.94:0/66162083 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba7000f770 con 0x7fba740fce70 2024-03-24T10:53:19.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.227+0000 7fba6a7fc700 1 -- 172.21.15.94:0/66162083 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fba7000fda0 con 0x7fba740fce70 2024-03-24T10:53:19.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.227+0000 7fba7c24c700 1 -- 172.21.15.94:0/66162083 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba74070ca0 con 0x7fba740fce70 2024-03-24T10:53:19.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.227+0000 7fba6a7fc700 1 -- 172.21.15.94:0/66162083 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba7000d310 con 0x7fba740fce70 2024-03-24T10:53:19.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.227+0000 7fba7c24c700 1 -- 172.21.15.94:0/66162083 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba74115a20 con 0x7fba740fce70 2024-03-24T10:53:19.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.228+0000 7fba7c24c700 1 -- 172.21.15.94:0/66162083 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba7404c710 con 0x7fba740fce70 2024-03-24T10:53:19.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.229+0000 7fba6a7fc700 1 -- 172.21.15.94:0/66162083 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fba7000f8d0 con 0x7fba740fce70 2024-03-24T10:53:19.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.229+0000 7fba6a7fc700 1 --2- 172.21.15.94:0/66162083 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fba6006d7e0 0x7fba6006fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:53:19.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.229+0000 7fba6a7fc700 1 -- 172.21.15.94:0/66162083 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fba7008ce70 con 0x7fba740fce70 2024-03-24T10:53:19.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.229+0000 7fba797e7700 1 --2- 172.21.15.94:0/66162083 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fba6006d7e0 0x7fba6006fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:53:19.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.230+0000 7fba797e7700 1 --2- 172.21.15.94:0/66162083 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fba6006d7e0 0x7fba6006fc90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fba64004f90 tx=0x7fba64009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:53:19.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.234+0000 7fba6a7fc700 1 -- 172.21.15.94:0/66162083 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fba7005a2b0 con 0x7fba740fce70 2024-03-24T10:53:19.480 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:19 smithi094 ceph-mon[93957]: from='client.34378 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:53:19.480 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:19 smithi094 ceph-mon[93957]: Detected new or changed devices on smithi094 2024-03-24T10:53:19.480 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:19 smithi094 ceph-mon[93957]: Upgrade: It appears safe to stop mds.cephfs.smithi094.llefgu 2024-03-24T10:53:19.480 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:19 smithi094 ceph-mon[93957]: from='client.34382 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:53:19.480 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:19 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/789835880' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:53:19.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.479+0000 7fba7c24c700 1 -- 172.21.15.94:0/66162083 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fba74115dc0 con 0x7fba740fce70 2024-03-24T10:53:19.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.479+0000 7fba6a7fc700 1 -- 172.21.15.94:0/66162083 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1887 (secure 0 0 0) 0x7fba70020250 con 0x7fba740fce70 2024-03-24T10:53:19.481 INFO:teuthology.orchestra.run.smithi094.stdout:e22 2024-03-24T10:53:19.481 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T10:53:19.481 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:53:19.481 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-03-24T10:53:19.481 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:53:19.481 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-03-24T10:53:19.481 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-03-24T10:53:19.481 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 19 2024-03-24T10:53:19.481 INFO:teuthology.orchestra.run.smithi094.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-24T10:53:19.481 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-03-24T10:43:39.435738+0000 2024-03-24T10:53:19.482 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-03-24T10:52:59.495272+0000 2024-03-24T10:53:19.482 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-03-24T10:53:19.482 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-03-24T10:53:19.482 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 60 2024-03-24T10:53:19.482 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-03-24T10:53:19.482 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-03-24T10:53:19.482 INFO:teuthology.orchestra.run.smithi094.stdout:max_xattr_size 65536 2024-03-24T10:53:19.482 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-03-24T10:53:19.482 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-03-24T10:53:19.482 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 0 2024-03-24T10:53:19.482 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:53:19.482 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-03-24T10:53:19.482 INFO:teuthology.orchestra.run.smithi094.stdout:in 0 2024-03-24T10:53:19.482 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=24251} 2024-03-24T10:53:19.482 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-03-24T10:53:19.483 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-03-24T10:53:19.483 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-03-24T10:53:19.483 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-03-24T10:53:19.483 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-03-24T10:53:19.483 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-03-24T10:53:19.483 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-03-24T10:53:19.483 INFO:teuthology.orchestra.run.smithi094.stdout:bal_rank_mask -1 2024-03-24T10:53:19.483 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 1 2024-03-24T10:53:19.483 INFO:teuthology.orchestra.run.smithi094.stdout:qdb_cluster leader: 24251 members: 24251 2024-03-24T10:53:19.483 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi096.egbmoq{0:24251} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.96:6824/1539752208,v1:172.21.15.96:6825/1539752208] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:53:19.483 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:53:19.483 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:53:19.483 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-03-24T10:53:19.483 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:53:19.483 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi096.jtebhh{-1:24261} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.96:6826/1321087930,v1:172.21.15.96:6827/1321087930] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:53:19.483 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.llefgu{-1:34370} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.94:6828/3011883830,v1:172.21.15.94:6829/3011883830] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:53:19.483 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.efiqte{-1:34374} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.94:6826/2043632450,v1:172.21.15.94:6827/2043632450] compat {c=[1],r=[1],i=[1fff]}] 2024-03-24T10:53:19.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.481+0000 7fba7c24c700 1 -- 172.21.15.94:0/66162083 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fba6006d7e0 msgr2=0x7fba6006fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:19.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.481+0000 7fba7c24c700 1 --2- 172.21.15.94:0/66162083 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fba6006d7e0 0x7fba6006fc90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fba64004f90 tx=0x7fba64009e30 comp rx=0 tx=0).stop 2024-03-24T10:53:19.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.481+0000 7fba7c24c700 1 -- 172.21.15.94:0/66162083 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba740fce70 msgr2=0x7fba74071c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:19.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.481+0000 7fba7c24c700 1 --2- 172.21.15.94:0/66162083 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba740fce70 0x7fba74071c60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fba7000b790 tx=0x7fba70009250 comp rx=0 tx=0).stop 2024-03-24T10:53:19.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.481+0000 7fba7c24c700 1 -- 172.21.15.94:0/66162083 shutdown_connections 2024-03-24T10:53:19.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.481+0000 7fba7c24c700 1 --2- 172.21.15.94:0/66162083 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fba6006d7e0 0x7fba6006fc90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:19.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.481+0000 7fba7c24c700 1 --2- 172.21.15.94:0/66162083 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba740fce70 0x7fba74071c60 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:19.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.481+0000 7fba7c24c700 1 --2- 172.21.15.94:0/66162083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba740fdfe0 0x7fba74070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:19.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.481+0000 7fba7c24c700 1 -- 172.21.15.94:0/66162083 >> 172.21.15.94:0/66162083 conn(0x7fba740f8550 msgr2=0x7fba740f9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:53:19.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.481+0000 7fba7c24c700 1 -- 172.21.15.94:0/66162083 shutdown_connections 2024-03-24T10:53:19.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.481+0000 7fba7c24c700 1 -- 172.21.15.94:0/66162083 wait complete. 2024-03-24T10:53:19.485 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 22 2024-03-24T10:53:19.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.583+0000 7f0629342700 1 -- 172.21.15.94:0/2705232291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06240fed80 msgr2=0x7f06240faec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:19.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.583+0000 7f0629342700 1 --2- 172.21.15.94:0/2705232291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06240fed80 0x7f06240faec0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0618004940 tx=0x7f0618009e30 comp rx=0 tx=0).stop 2024-03-24T10:53:19.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.583+0000 7f0629342700 1 -- 172.21.15.94:0/2705232291 shutdown_connections 2024-03-24T10:53:19.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.583+0000 7f0629342700 1 --2- 172.21.15.94:0/2705232291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06240fed80 0x7f06240faec0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:19.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.583+0000 7f0629342700 1 --2- 172.21.15.94:0/2705232291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06240fe430 0x7f06240fe840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:19.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.583+0000 7f0629342700 1 -- 172.21.15.94:0/2705232291 >> 172.21.15.94:0/2705232291 conn(0x7f06240f6bd0 msgr2=0x7f06240f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:53:19.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.583+0000 7f0629342700 1 -- 172.21.15.94:0/2705232291 shutdown_connections 2024-03-24T10:53:19.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.583+0000 7f0629342700 1 -- 172.21.15.94:0/2705232291 wait complete. 2024-03-24T10:53:19.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.584+0000 7f0629342700 1 Processor -- start 2024-03-24T10:53:19.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.584+0000 7f0629342700 1 -- start start 2024-03-24T10:53:19.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.584+0000 7f0629342700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06240fe430 0x7f062410d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:53:19.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.584+0000 7f0629342700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06240fed80 0x7f062410de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:53:19.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.584+0000 7f0629342700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062410e440 con 0x7f06240fe430 2024-03-24T10:53:19.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.584+0000 7f0629342700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062410e5b0 con 0x7f06240fed80 2024-03-24T10:53:19.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.584+0000 7f0622ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06240fe430 0x7f062410d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:53:19.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.584+0000 7f0622ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06240fe430 0x7f062410d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36024/0 (socket says 172.21.15.94:36024) 2024-03-24T10:53:19.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.584+0000 7f0622ffd700 1 -- 172.21.15.94:0/1163515979 learned_addr learned my addr 172.21.15.94:0/1163515979 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:53:19.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.585+0000 7f06227fc700 1 --2- 172.21.15.94:0/1163515979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06240fed80 0x7f062410de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:53:19.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.585+0000 7f0622ffd700 1 -- 172.21.15.94:0/1163515979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06240fed80 msgr2=0x7f062410de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:19.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.585+0000 7f0622ffd700 1 --2- 172.21.15.94:0/1163515979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06240fed80 0x7f062410de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:19.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.585+0000 7f0622ffd700 1 -- 172.21.15.94:0/1163515979 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0618005040 con 0x7f06240fe430 2024-03-24T10:53:19.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.585+0000 7f06227fc700 1 --2- 172.21.15.94:0/1163515979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06240fed80 0x7f062410de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T10:53:19.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.585+0000 7f0622ffd700 1 --2- 172.21.15.94:0/1163515979 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06240fe430 0x7f062410d900 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0614002a50 tx=0x7f06140046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:53:19.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.585+0000 7f060b7fe700 1 -- 172.21.15.94:0/1163515979 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0614007cc0 con 0x7f06240fe430 2024-03-24T10:53:19.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.585+0000 7f060b7fe700 1 -- 172.21.15.94:0/1163515979 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f061400f040 con 0x7f06240fe430 2024-03-24T10:53:19.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.585+0000 7f0629342700 1 -- 172.21.15.94:0/1163515979 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0624112fb0 con 0x7f06240fe430 2024-03-24T10:53:19.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.585+0000 7f060b7fe700 1 -- 172.21.15.94:0/1163515979 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0614009640 con 0x7f06240fe430 2024-03-24T10:53:19.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.585+0000 7f0629342700 1 -- 172.21.15.94:0/1163515979 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0624113540 con 0x7f06240fe430 2024-03-24T10:53:19.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.586+0000 7f0629342700 1 -- 172.21.15.94:0/1163515979 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f062404c710 con 0x7f06240fe430 2024-03-24T10:53:19.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.586+0000 7f060b7fe700 1 -- 172.21.15.94:0/1163515979 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f06140097c0 con 0x7f06240fe430 2024-03-24T10:53:19.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.587+0000 7f060b7fe700 1 --2- 172.21.15.94:0/1163515979 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f060c06d7e0 0x7f060c06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:53:19.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.587+0000 7f060b7fe700 1 -- 172.21.15.94:0/1163515979 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f061408cca0 con 0x7f06240fe430 2024-03-24T10:53:19.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.587+0000 7f06227fc700 1 --2- 172.21.15.94:0/1163515979 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f060c06d7e0 0x7f060c06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:53:19.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.587+0000 7f06227fc700 1 --2- 172.21.15.94:0/1163515979 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f060c06d7e0 0x7f060c06fc90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f061800b3b0 tx=0x7f0618009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:53:19.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.591+0000 7f060b7fe700 1 -- 172.21.15.94:0/1163515979 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f0614091070 con 0x7f06240fe430 2024-03-24T10:53:19.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.814+0000 7f0629342700 1 -- 172.21.15.94:0/1163515979 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0624113760 con 0x7f060c06d7e0 2024-03-24T10:53:19.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.815+0000 7f060b7fe700 1 -- 172.21.15.94:0/1163515979 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f0624113760 con 0x7f060c06d7e0 2024-03-24T10:53:19.816 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-03-24T10:53:19.816 INFO:teuthology.orchestra.run.smithi094.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919", 2024-03-24T10:53:19.816 INFO:teuthology.orchestra.run.smithi094.stdout: "in_progress": true, 2024-03-24T10:53:19.816 INFO:teuthology.orchestra.run.smithi094.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T10:53:19.816 INFO:teuthology.orchestra.run.smithi094.stdout: "services_complete": [ 2024-03-24T10:53:19.816 INFO:teuthology.orchestra.run.smithi094.stdout: "mon", 2024-03-24T10:53:19.816 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-03-24T10:53:19.816 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr", 2024-03-24T10:53:19.816 INFO:teuthology.orchestra.run.smithi094.stdout: "osd" 2024-03-24T10:53:19.816 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-03-24T10:53:19.816 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": "13/21 daemons upgraded", 2024-03-24T10:53:19.816 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "Currently upgrading mds daemons", 2024-03-24T10:53:19.816 INFO:teuthology.orchestra.run.smithi094.stdout: "is_paused": false 2024-03-24T10:53:19.816 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-03-24T10:53:19.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.817+0000 7f0629342700 1 -- 172.21.15.94:0/1163515979 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f060c06d7e0 msgr2=0x7f060c06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:19.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.817+0000 7f0629342700 1 --2- 172.21.15.94:0/1163515979 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f060c06d7e0 0x7f060c06fc90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f061800b3b0 tx=0x7f0618009e30 comp rx=0 tx=0).stop 2024-03-24T10:53:19.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.817+0000 7f0629342700 1 -- 172.21.15.94:0/1163515979 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06240fe430 msgr2=0x7f062410d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:19.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.817+0000 7f0629342700 1 --2- 172.21.15.94:0/1163515979 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06240fe430 0x7f062410d900 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0614002a50 tx=0x7f06140046b0 comp rx=0 tx=0).stop 2024-03-24T10:53:19.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.817+0000 7f0629342700 1 -- 172.21.15.94:0/1163515979 shutdown_connections 2024-03-24T10:53:19.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.817+0000 7f0629342700 1 --2- 172.21.15.94:0/1163515979 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f060c06d7e0 0x7f060c06fc90 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:19.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.817+0000 7f0629342700 1 --2- 172.21.15.94:0/1163515979 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06240fe430 0x7f062410d900 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:19.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.817+0000 7f0629342700 1 --2- 172.21.15.94:0/1163515979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06240fed80 0x7f062410de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:19.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.817+0000 7f0629342700 1 -- 172.21.15.94:0/1163515979 >> 172.21.15.94:0/1163515979 conn(0x7f06240f6bd0 msgr2=0x7f06240f8210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:53:19.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.817+0000 7f0629342700 1 -- 172.21.15.94:0/1163515979 shutdown_connections 2024-03-24T10:53:19.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.817+0000 7f0629342700 1 -- 172.21.15.94:0/1163515979 wait complete. 2024-03-24T10:53:19.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:19 smithi096 ceph-mon[84258]: from='client.34378 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:53:19.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:19 smithi096 ceph-mon[84258]: Detected new or changed devices on smithi094 2024-03-24T10:53:19.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:19 smithi096 ceph-mon[84258]: Upgrade: It appears safe to stop mds.cephfs.smithi094.llefgu 2024-03-24T10:53:19.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:19 smithi096 ceph-mon[84258]: from='client.34382 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:53:19.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:19 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/789835880' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:53:19.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.923+0000 7fbf28f70700 1 -- 172.21.15.94:0/3405143079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf240fcf80 msgr2=0x7fbf240fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:19.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.923+0000 7fbf28f70700 1 --2- 172.21.15.94:0/3405143079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf240fcf80 0x7fbf240fd3f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbf18004940 tx=0x7fbf18009e30 comp rx=0 tx=0).stop 2024-03-24T10:53:19.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.923+0000 7fbf28f70700 1 -- 172.21.15.94:0/3405143079 shutdown_connections 2024-03-24T10:53:19.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.924+0000 7fbf28f70700 1 --2- 172.21.15.94:0/3405143079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf240fcf80 0x7fbf240fd3f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:19.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.924+0000 7fbf28f70700 1 --2- 172.21.15.94:0/3405143079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf240fbcd0 0x7fbf240fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:19.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.924+0000 7fbf28f70700 1 -- 172.21.15.94:0/3405143079 >> 172.21.15.94:0/3405143079 conn(0x7fbf240f7430 msgr2=0x7fbf240f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:53:19.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.924+0000 7fbf28f70700 1 -- 172.21.15.94:0/3405143079 shutdown_connections 2024-03-24T10:53:19.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.924+0000 7fbf28f70700 1 -- 172.21.15.94:0/3405143079 wait complete. 2024-03-24T10:53:19.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.924+0000 7fbf28f70700 1 Processor -- start 2024-03-24T10:53:19.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.924+0000 7fbf28f70700 1 -- start start 2024-03-24T10:53:19.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.924+0000 7fbf28f70700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf240fbcd0 0x7fbf241106d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:53:19.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.924+0000 7fbf28f70700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf240fcf80 0x7fbf24112c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:53:19.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.924+0000 7fbf28f70700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf24066440 con 0x7fbf240fbcd0 2024-03-24T10:53:19.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.924+0000 7fbf28f70700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf24066580 con 0x7fbf240fcf80 2024-03-24T10:53:19.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.924+0000 7fbf2259c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf240fbcd0 0x7fbf241106d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:53:19.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.924+0000 7fbf2259c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf240fbcd0 0x7fbf241106d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55124/0 (socket says 172.21.15.94:55124) 2024-03-24T10:53:19.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.924+0000 7fbf2259c700 1 -- 172.21.15.94:0/2578031216 learned_addr learned my addr 172.21.15.94:0/2578031216 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:53:19.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.925+0000 7fbf21d9b700 1 --2- 172.21.15.94:0/2578031216 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf240fcf80 0x7fbf24112c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:53:19.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.925+0000 7fbf2259c700 1 -- 172.21.15.94:0/2578031216 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf240fcf80 msgr2=0x7fbf24112c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:19.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.925+0000 7fbf2259c700 1 --2- 172.21.15.94:0/2578031216 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf240fcf80 0x7fbf24112c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:19.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.925+0000 7fbf2259c700 1 -- 172.21.15.94:0/2578031216 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf18005040 con 0x7fbf240fbcd0 2024-03-24T10:53:19.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.925+0000 7fbf21d9b700 1 --2- 172.21.15.94:0/2578031216 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf240fcf80 0x7fbf24112c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T10:53:19.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.925+0000 7fbf2259c700 1 --2- 172.21.15.94:0/2578031216 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf240fbcd0 0x7fbf241106d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbf1400b790 tx=0x7fbf14009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:53:19.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.925+0000 7fbf12ffd700 1 -- 172.21.15.94:0/2578031216 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf1400f770 con 0x7fbf240fbcd0 2024-03-24T10:53:19.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.925+0000 7fbf28f70700 1 -- 172.21.15.94:0/2578031216 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf241132b0 con 0x7fbf240fbcd0 2024-03-24T10:53:19.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.925+0000 7fbf12ffd700 1 -- 172.21.15.94:0/2578031216 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbf1400fda0 con 0x7fbf240fbcd0 2024-03-24T10:53:19.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.925+0000 7fbf12ffd700 1 -- 172.21.15.94:0/2578031216 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf1400d310 con 0x7fbf240fbcd0 2024-03-24T10:53:19.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.925+0000 7fbf28f70700 1 -- 172.21.15.94:0/2578031216 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf24113870 con 0x7fbf240fbcd0 2024-03-24T10:53:19.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.926+0000 7fbf28f70700 1 -- 172.21.15.94:0/2578031216 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf2405f5f0 con 0x7fbf240fbcd0 2024-03-24T10:53:19.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.927+0000 7fbf12ffd700 1 -- 172.21.15.94:0/2578031216 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fbf14010420 con 0x7fbf240fbcd0 2024-03-24T10:53:19.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.927+0000 7fbf12ffd700 1 --2- 172.21.15.94:0/2578031216 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fbf0c06d7e0 0x7fbf0c06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:53:19.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.928+0000 7fbf12ffd700 1 -- 172.21.15.94:0/2578031216 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fbf1408cf30 con 0x7fbf240fbcd0 2024-03-24T10:53:19.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.928+0000 7fbf21d9b700 1 --2- 172.21.15.94:0/2578031216 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fbf0c06d7e0 0x7fbf0c06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:53:19.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.928+0000 7fbf21d9b700 1 --2- 172.21.15.94:0/2578031216 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fbf0c06d7e0 0x7fbf0c06fc90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbf1800b3b0 tx=0x7fbf18009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:53:19.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:19.932+0000 7fbf12ffd700 1 -- 172.21.15.94:0/2578031216 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fbf140c1280 con 0x7fbf240fbcd0 2024-03-24T10:53:20.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:20.226+0000 7fbf28f70700 1 -- 172.21.15.94:0/2578031216 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fbf2404bed0 con 0x7fbf240fbcd0 2024-03-24T10:53:20.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:20.226+0000 7fbf12ffd700 1 -- 172.21.15.94:0/2578031216 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fbf140567a0 con 0x7fbf240fbcd0 2024-03-24T10:53:20.227 INFO:teuthology.orchestra.run.smithi094.stdout:HEALTH_OK 2024-03-24T10:53:20.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:20.229+0000 7fbf28f70700 1 -- 172.21.15.94:0/2578031216 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fbf0c06d7e0 msgr2=0x7fbf0c06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:20.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:20.229+0000 7fbf28f70700 1 --2- 172.21.15.94:0/2578031216 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fbf0c06d7e0 0x7fbf0c06fc90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbf1800b3b0 tx=0x7fbf18009e30 comp rx=0 tx=0).stop 2024-03-24T10:53:20.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:20.229+0000 7fbf28f70700 1 -- 172.21.15.94:0/2578031216 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf240fbcd0 msgr2=0x7fbf241106d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:20.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:20.229+0000 7fbf28f70700 1 --2- 172.21.15.94:0/2578031216 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf240fbcd0 0x7fbf241106d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbf1400b790 tx=0x7fbf14009250 comp rx=0 tx=0).stop 2024-03-24T10:53:20.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:20.229+0000 7fbf28f70700 1 -- 172.21.15.94:0/2578031216 shutdown_connections 2024-03-24T10:53:20.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:20.229+0000 7fbf28f70700 1 --2- 172.21.15.94:0/2578031216 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fbf0c06d7e0 0x7fbf0c06fc90 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:20.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:20.229+0000 7fbf28f70700 1 --2- 172.21.15.94:0/2578031216 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf240fbcd0 0x7fbf241106d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:20.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:20.229+0000 7fbf28f70700 1 --2- 172.21.15.94:0/2578031216 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf240fcf80 0x7fbf24112c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:20.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:20.229+0000 7fbf28f70700 1 -- 172.21.15.94:0/2578031216 >> 172.21.15.94:0/2578031216 conn(0x7fbf240f7430 msgr2=0x7fbf240f8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:53:20.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:20.230+0000 7fbf28f70700 1 -- 172.21.15.94:0/2578031216 shutdown_connections 2024-03-24T10:53:20.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:20.230+0000 7fbf28f70700 1 -- 172.21.15.94:0/2578031216 wait complete. 2024-03-24T10:53:20.480 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:20 smithi094 ceph-mon[93957]: from='client.34386 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:53:20.480 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:20 smithi094 ceph-mon[93957]: pgmap v177: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail 2024-03-24T10:53:20.480 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:20 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/66162083' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T10:53:20.480 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:20 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:20.480 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:20 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi094.llefgu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T10:53:20.480 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:20 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:53:20.480 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:20 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/2578031216' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T10:53:20.480 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:20 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:53:20.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:20 smithi096 ceph-mon[84258]: from='client.34386 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:53:20.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:20 smithi096 ceph-mon[84258]: pgmap v177: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail 2024-03-24T10:53:20.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:20 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/66162083' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T10:53:20.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:20 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:20.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:20 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi094.llefgu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T10:53:20.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:20 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:53:20.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:20 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/2578031216' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T10:53:20.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:20 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:53:21.564 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:21 smithi094 ceph-mon[93957]: Upgrade: Updating mds.cephfs.smithi094.llefgu 2024-03-24T10:53:21.564 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:21 smithi094 ceph-mon[93957]: Deploying daemon mds.cephfs.smithi094.llefgu on smithi094 2024-03-24T10:53:21.564 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:21 smithi094 ceph-mon[93957]: from='client.34398 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:53:21.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:21 smithi096 ceph-mon[84258]: Upgrade: Updating mds.cephfs.smithi094.llefgu 2024-03-24T10:53:21.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:21 smithi096 ceph-mon[84258]: Deploying daemon mds.cephfs.smithi094.llefgu on smithi094 2024-03-24T10:53:21.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:21 smithi096 ceph-mon[84258]: from='client.34398 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:53:22.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:22 smithi094 ceph-mon[93957]: pgmap v178: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail 2024-03-24T10:53:22.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:22 smithi096 ceph-mon[84258]: pgmap v178: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail 2024-03-24T10:53:23.677 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:23 smithi094 ceph-mon[93957]: osdmap e82: 6 total, 6 up, 6 in 2024-03-24T10:53:23.677 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:23 smithi094 ceph-mon[93957]: fsmap cephfs:1 {0=cephfs.smithi096.egbmoq=up:active} 2 up:standby 2024-03-24T10:53:23.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:23 smithi096 ceph-mon[84258]: osdmap e82: 6 total, 6 up, 6 in 2024-03-24T10:53:23.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:23 smithi096 ceph-mon[84258]: fsmap cephfs:1 {0=cephfs.smithi096.egbmoq=up:active} 2 up:standby 2024-03-24T10:53:24.583 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:24 smithi094 ceph-mon[93957]: pgmap v180: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail 2024-03-24T10:53:24.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:24 smithi096 ceph-mon[84258]: pgmap v180: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail 2024-03-24T10:53:26.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:26 smithi094 ceph-mon[93957]: pgmap v181: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail 2024-03-24T10:53:26.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:26 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:26.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:26 smithi094 ceph-mon[93957]: mds.? [v2:172.21.15.94:6828/2396893131,v1:172.21.15.94:6829/2396893131] up:boot 2024-03-24T10:53:26.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:26 smithi094 ceph-mon[93957]: fsmap cephfs:1 {0=cephfs.smithi096.egbmoq=up:active} 3 up:standby 2024-03-24T10:53:26.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:26 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.llefgu"}]: dispatch 2024-03-24T10:53:26.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:26 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:26.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:26 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:53:26.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:26 smithi096 ceph-mon[84258]: pgmap v181: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail 2024-03-24T10:53:26.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:26 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:26.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:26 smithi096 ceph-mon[84258]: mds.? [v2:172.21.15.94:6828/2396893131,v1:172.21.15.94:6829/2396893131] up:boot 2024-03-24T10:53:26.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:26 smithi096 ceph-mon[84258]: fsmap cephfs:1 {0=cephfs.smithi096.egbmoq=up:active} 3 up:standby 2024-03-24T10:53:26.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:26 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.llefgu"}]: dispatch 2024-03-24T10:53:26.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:26 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:26.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:26 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:53:28.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:28 smithi094 ceph-mon[93957]: pgmap v182: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail 2024-03-24T10:53:28.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:28 smithi096 ceph-mon[84258]: pgmap v182: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail 2024-03-24T10:53:30.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:30 smithi094 ceph-mon[93957]: pgmap v183: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail 2024-03-24T10:53:30.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:30 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:30.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:30 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:30.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:30 smithi096 ceph-mon[84258]: pgmap v183: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail 2024-03-24T10:53:30.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:30 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:30.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:30 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:31.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:31 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:31.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:31 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:31.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:31 smithi094 ceph-mon[93957]: pgmap v184: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail 2024-03-24T10:53:32.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:31 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:32.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:31 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:32.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:31 smithi096 ceph-mon[84258]: pgmap v184: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail 2024-03-24T10:53:34.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:34 smithi094 ceph-mon[93957]: pgmap v185: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail 2024-03-24T10:53:34.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:34 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:34.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:34 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:34.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:34 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:53:34.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:34 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:53:34.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:34 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:34.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:34 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:53:34.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:34 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:53:34.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:34 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:53:34.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:34 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:53:34.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:34 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:53:34.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:34 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi096.egbmoq"]}]: dispatch 2024-03-24T10:53:34.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:34 smithi096 ceph-mon[84258]: pgmap v185: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail 2024-03-24T10:53:34.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:34 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:34.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:34 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:34.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:34 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:53:34.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:34 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:53:34.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:34 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:34.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:34 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:53:34.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:34 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:53:34.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:34 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:53:34.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:34 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:53:34.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:34 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:53:34.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:34 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi096.egbmoq"]}]: dispatch 2024-03-24T10:53:35.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:35 smithi096 ceph-mon[84258]: Upgrade: It appears safe to stop mds.cephfs.smithi096.egbmoq 2024-03-24T10:53:35.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:35 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:53:35.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:35 smithi094 ceph-mon[93957]: Upgrade: It appears safe to stop mds.cephfs.smithi096.egbmoq 2024-03-24T10:53:35.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:35 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:53:36.649 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:36 smithi094 ceph-mon[93957]: pgmap v186: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail 2024-03-24T10:53:36.649 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:36 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:36.649 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:36 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi096.egbmoq", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T10:53:36.649 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:36 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:53:36.703 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:36 smithi096 ceph-mon[84258]: pgmap v186: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail 2024-03-24T10:53:36.703 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:36 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:36.703 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:36 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi096.egbmoq", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T10:53:36.703 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:36 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:53:37.515 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:37 smithi096 ceph-mon[84258]: Upgrade: Updating mds.cephfs.smithi096.egbmoq 2024-03-24T10:53:37.515 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:37 smithi096 ceph-mon[84258]: Deploying daemon mds.cephfs.smithi096.egbmoq on smithi096 2024-03-24T10:53:37.665 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:37 smithi094 ceph-mon[93957]: Upgrade: Updating mds.cephfs.smithi096.egbmoq 2024-03-24T10:53:37.665 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:37 smithi094 ceph-mon[93957]: Deploying daemon mds.cephfs.smithi096.egbmoq on smithi096 2024-03-24T10:53:38.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:38 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-mon-smithi094[93953]: 2024-03-24T10:53:38.029+0000 7f0fa10ce700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-24T10:53:38.325 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:53:38 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3[93576]: 2024-03-24T10:53:38.047+0000 7f1fd730a700 -1 bdev(0x564294b52000 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 2 2024-03-24T10:53:38.325 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:53:38 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3[93576]: 2024-03-24T10:53:38.047+0000 7f1fd0afd700 -1 bdev(0x564294b52000 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 3 2024-03-24T10:53:38.325 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:53:38 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3[93576]: 2024-03-24T10:53:38.049+0000 7f1fd1aff700 -1 bdev(0x564294b52000 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 12 2024-03-24T10:53:38.325 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:53:38 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3[93576]: 2024-03-24T10:53:38.050+0000 7f1fd2300700 -1 bdev(0x564294b52000 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 16 2024-03-24T10:53:38.325 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:53:38 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3[93576]: 2024-03-24T10:53:38.051+0000 7f1fd830c700 -1 bdev(0x564294b52000 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 18 2024-03-24T10:53:38.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:38 smithi094 ceph-mon[93957]: pgmap v187: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail 2024-03-24T10:53:38.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:38 smithi094 ceph-mon[93957]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-24T10:53:38.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:38 smithi094 ceph-mon[93957]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-24T10:53:38.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:38 smithi094 ceph-mon[93957]: osdmap e83: 6 total, 6 up, 6 in 2024-03-24T10:53:38.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:38 smithi094 ceph-mon[93957]: Standby daemon mds.cephfs.smithi096.jtebhh assigned to filesystem cephfs as rank 0 2024-03-24T10:53:38.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:38 smithi094 ceph-mon[93957]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-24T10:53:38.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:38 smithi094 ceph-mon[93957]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-03-24T10:53:38.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:38 smithi094 ceph-mon[93957]: fsmap cephfs:1/1 {0=cephfs.smithi096.jtebhh=up:replay} 2 up:standby 2024-03-24T10:53:38.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:38 smithi096 ceph-mon[84258]: pgmap v187: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail 2024-03-24T10:53:38.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:38 smithi096 ceph-mon[84258]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-24T10:53:38.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:38 smithi096 ceph-mon[84258]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-24T10:53:38.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:38 smithi096 ceph-mon[84258]: osdmap e83: 6 total, 6 up, 6 in 2024-03-24T10:53:38.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:38 smithi096 ceph-mon[84258]: Standby daemon mds.cephfs.smithi096.jtebhh assigned to filesystem cephfs as rank 0 2024-03-24T10:53:38.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:38 smithi096 ceph-mon[84258]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-24T10:53:38.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:38 smithi096 ceph-mon[84258]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-03-24T10:53:38.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:38 smithi096 ceph-mon[84258]: fsmap cephfs:1/1 {0=cephfs.smithi096.jtebhh=up:replay} 2 up:standby 2024-03-24T10:53:40.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:40 smithi094 ceph-mon[93957]: pgmap v189: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 2.8 MiB/s rd, 1 op/s 2024-03-24T10:53:40.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:40 smithi096 ceph-mon[84258]: pgmap v189: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 2.8 MiB/s rd, 1 op/s 2024-03-24T10:53:42.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:42 smithi094 ceph-mon[93957]: pgmap v190: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 2.8 MiB/s rd, 1 op/s 2024-03-24T10:53:42.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:42 smithi096 ceph-mon[84258]: pgmap v190: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 2.8 MiB/s rd, 1 op/s 2024-03-24T10:53:44.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:44 smithi096 ceph-mon[84258]: pgmap v191: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 12 MiB/s rd, 4 op/s 2024-03-24T10:53:44.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:44 smithi094 ceph-mon[93957]: pgmap v191: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 12 MiB/s rd, 4 op/s 2024-03-24T10:53:46.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:46 smithi096 ceph-mon[84258]: pgmap v192: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-03-24T10:53:46.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:46 smithi094 ceph-mon[93957]: pgmap v192: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-03-24T10:53:48.679 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:48 smithi096 ceph-mon[84258]: pgmap v193: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-03-24T10:53:48.679 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:48 smithi096 ceph-mon[84258]: reconnect by client.14550 v1:192.168.0.1:0/449160647 after 0.000999977 2024-03-24T10:53:48.679 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:48 smithi096 ceph-mon[84258]: reconnect by client.14556 v1:172.21.15.96:0/2944380590 after 0.000999977 2024-03-24T10:53:48.679 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:48 smithi096 ceph-mon[84258]: mds.? [v2:172.21.15.96:6826/1321087930,v1:172.21.15.96:6827/1321087930] up:reconnect 2024-03-24T10:53:48.679 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:48 smithi096 ceph-mon[84258]: fsmap cephfs:1/1 {0=cephfs.smithi096.jtebhh=up:reconnect} 2 up:standby 2024-03-24T10:53:48.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:48 smithi094 ceph-mon[93957]: pgmap v193: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-03-24T10:53:48.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:48 smithi094 ceph-mon[93957]: reconnect by client.14550 v1:192.168.0.1:0/449160647 after 0.000999977 2024-03-24T10:53:48.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:48 smithi094 ceph-mon[93957]: reconnect by client.14556 v1:172.21.15.96:0/2944380590 after 0.000999977 2024-03-24T10:53:48.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:48 smithi094 ceph-mon[93957]: mds.? [v2:172.21.15.96:6826/1321087930,v1:172.21.15.96:6827/1321087930] up:reconnect 2024-03-24T10:53:48.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:48 smithi094 ceph-mon[93957]: fsmap cephfs:1/1 {0=cephfs.smithi096.jtebhh=up:reconnect} 2 up:standby 2024-03-24T10:53:49.644 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:49 smithi096 ceph-mon[84258]: mds.? [v2:172.21.15.96:6826/1321087930,v1:172.21.15.96:6827/1321087930] up:rejoin 2024-03-24T10:53:49.644 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:49 smithi096 ceph-mon[84258]: fsmap cephfs:1/1 {0=cephfs.smithi096.jtebhh=up:rejoin} 2 up:standby 2024-03-24T10:53:49.644 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:49 smithi096 ceph-mon[84258]: daemon mds.cephfs.smithi096.jtebhh is now active in filesystem cephfs as rank 0 2024-03-24T10:53:49.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:49 smithi094 ceph-mon[93957]: mds.? [v2:172.21.15.96:6826/1321087930,v1:172.21.15.96:6827/1321087930] up:rejoin 2024-03-24T10:53:49.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:49 smithi094 ceph-mon[93957]: fsmap cephfs:1/1 {0=cephfs.smithi096.jtebhh=up:rejoin} 2 up:standby 2024-03-24T10:53:49.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:49 smithi094 ceph-mon[93957]: daemon mds.cephfs.smithi096.jtebhh is now active in filesystem cephfs as rank 0 2024-03-24T10:53:50.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.346+0000 7f48d9d16700 1 -- 172.21.15.94:0/3872630319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48d40fff60 msgr2=0x7f48d4102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:50.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.346+0000 7f48d9d16700 1 --2- 172.21.15.94:0/3872630319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48d40fff60 0x7f48d4102340 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f48c8004940 tx=0x7f48c8009e30 comp rx=0 tx=0).stop 2024-03-24T10:53:50.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.346+0000 7f48d9d16700 1 -- 172.21.15.94:0/3872630319 shutdown_connections 2024-03-24T10:53:50.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.346+0000 7f48d9d16700 1 --2- 172.21.15.94:0/3872630319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48d40fff60 0x7f48d4102340 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:50.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.346+0000 7f48d9d16700 1 --2- 172.21.15.94:0/3872630319 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48d4066280 0x7f48d40ffa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:50.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.346+0000 7f48d9d16700 1 -- 172.21.15.94:0/3872630319 >> 172.21.15.94:0/3872630319 conn(0x7f48d40f7430 msgr2=0x7f48d40f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:53:50.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.346+0000 7f48d9d16700 1 -- 172.21.15.94:0/3872630319 shutdown_connections 2024-03-24T10:53:50.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.346+0000 7f48d9d16700 1 -- 172.21.15.94:0/3872630319 wait complete. 2024-03-24T10:53:50.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.347+0000 7f48d9d16700 1 Processor -- start 2024-03-24T10:53:50.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.347+0000 7f48d9d16700 1 -- start start 2024-03-24T10:53:50.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.347+0000 7f48d9d16700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48d4066280 0x7f48d40fd450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:53:50.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.347+0000 7f48d9d16700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48d40fff60 0x7f48d40fb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:53:50.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.347+0000 7f48d9d16700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d40fbed0 con 0x7f48d4066280 2024-03-24T10:53:50.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.347+0000 7f48d9d16700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d40fc040 con 0x7f48d40fff60 2024-03-24T10:53:50.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.348+0000 7f48d37fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48d4066280 0x7f48d40fd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:53:50.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.348+0000 7f48d37fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48d4066280 0x7f48d40fd450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47362/0 (socket says 172.21.15.94:47362) 2024-03-24T10:53:50.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.348+0000 7f48d37fe700 1 -- 172.21.15.94:0/2250480891 learned_addr learned my addr 172.21.15.94:0/2250480891 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:53:50.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.348+0000 7f48d2ffd700 1 --2- 172.21.15.94:0/2250480891 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48d40fff60 0x7f48d40fb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:53:50.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.348+0000 7f48d2ffd700 1 -- 172.21.15.94:0/2250480891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48d4066280 msgr2=0x7f48d40fd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:50.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.348+0000 7f48d2ffd700 1 --2- 172.21.15.94:0/2250480891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48d4066280 0x7f48d40fd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:50.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.348+0000 7f48d2ffd700 1 -- 172.21.15.94:0/2250480891 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c8005040 con 0x7f48d40fff60 2024-03-24T10:53:50.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.348+0000 7f48d37fe700 1 --2- 172.21.15.94:0/2250480891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48d4066280 0x7f48d40fd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T10:53:50.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.348+0000 7f48d2ffd700 1 --2- 172.21.15.94:0/2250480891 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48d40fff60 0x7f48d40fb920 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f48c8000c00 tx=0x7f48c8009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:53:50.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.348+0000 7f48bbfff700 1 -- 172.21.15.94:0/2250480891 <== mon.1 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48c800e030 con 0x7f48d40fff60 2024-03-24T10:53:50.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.348+0000 7f48d9d16700 1 -- 172.21.15.94:0/2250480891 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48d40fc2c0 con 0x7f48d40fff60 2024-03-24T10:53:50.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.348+0000 7f48bbfff700 1 -- 172.21.15.94:0/2250480891 <== mon.1 v2:172.21.15.96:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f48c801c400 con 0x7f48d40fff60 2024-03-24T10:53:50.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.348+0000 7f48bbfff700 1 -- 172.21.15.94:0/2250480891 <== mon.1 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48c800b650 con 0x7f48d40fff60 2024-03-24T10:53:50.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.348+0000 7f48d9d16700 1 -- 172.21.15.94:0/2250480891 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48d41159e0 con 0x7f48d40fff60 2024-03-24T10:53:50.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.349+0000 7f48d9d16700 1 -- 172.21.15.94:0/2250480891 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48d405f5f0 con 0x7f48d40fff60 2024-03-24T10:53:50.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.350+0000 7f48bbfff700 1 -- 172.21.15.94:0/2250480891 <== mon.1 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f48c801ca90 con 0x7f48d40fff60 2024-03-24T10:53:50.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.350+0000 7f48bbfff700 1 --2- 172.21.15.94:0/2250480891 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f48bc06d7e0 0x7f48bc06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:53:50.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.350+0000 7f48bbfff700 1 -- 172.21.15.94:0/2250480891 <== mon.1 v2:172.21.15.96:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f48c808e7c0 con 0x7f48d40fff60 2024-03-24T10:53:50.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.350+0000 7f48d37fe700 1 --2- 172.21.15.94:0/2250480891 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f48bc06d7e0 0x7f48bc06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:53:50.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.351+0000 7f48d37fe700 1 --2- 172.21.15.94:0/2250480891 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f48bc06d7e0 0x7f48bc06fc90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f48d40fd1e0 tx=0x7f48c400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:53:50.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.357+0000 7f48bbfff700 1 -- 172.21.15.94:0/2250480891 <== mon.1 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f48c8058040 con 0x7f48d40fff60 2024-03-24T10:53:50.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.575+0000 7f48d9d16700 1 -- 172.21.15.94:0/2250480891 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f48d40f8f70 con 0x7f48bc06d7e0 2024-03-24T10:53:50.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.576+0000 7f48bbfff700 1 -- 172.21.15.94:0/2250480891 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f48d40f8f70 con 0x7f48bc06d7e0 2024-03-24T10:53:50.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.578+0000 7f48d9d16700 1 -- 172.21.15.94:0/2250480891 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f48bc06d7e0 msgr2=0x7f48bc06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:50.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.578+0000 7f48d9d16700 1 --2- 172.21.15.94:0/2250480891 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f48bc06d7e0 0x7f48bc06fc90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f48d40fd1e0 tx=0x7f48c400b040 comp rx=0 tx=0).stop 2024-03-24T10:53:50.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.578+0000 7f48d9d16700 1 -- 172.21.15.94:0/2250480891 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48d40fff60 msgr2=0x7f48d40fb920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:50.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.578+0000 7f48d9d16700 1 --2- 172.21.15.94:0/2250480891 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48d40fff60 0x7f48d40fb920 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f48c8000c00 tx=0x7f48c8009e30 comp rx=0 tx=0).stop 2024-03-24T10:53:50.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.578+0000 7f48d9d16700 1 -- 172.21.15.94:0/2250480891 shutdown_connections 2024-03-24T10:53:50.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.578+0000 7f48d9d16700 1 --2- 172.21.15.94:0/2250480891 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f48bc06d7e0 0x7f48bc06fc90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:50.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.578+0000 7f48d9d16700 1 --2- 172.21.15.94:0/2250480891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48d4066280 0x7f48d40fd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:50.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.578+0000 7f48d9d16700 1 --2- 172.21.15.94:0/2250480891 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48d40fff60 0x7f48d40fb920 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:50.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.578+0000 7f48d9d16700 1 -- 172.21.15.94:0/2250480891 >> 172.21.15.94:0/2250480891 conn(0x7f48d40f7430 msgr2=0x7f48d40f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:53:50.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.578+0000 7f48d9d16700 1 -- 172.21.15.94:0/2250480891 shutdown_connections 2024-03-24T10:53:50.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.578+0000 7f48d9d16700 1 -- 172.21.15.94:0/2250480891 wait complete. 2024-03-24T10:53:50.589 INFO:teuthology.orchestra.run.smithi094.stdout:true 2024-03-24T10:53:50.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.681+0000 7f4fd5f92700 1 -- 172.21.15.94:0/2284072547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fd00fff80 msgr2=0x7f4fd0102360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:50.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.681+0000 7f4fd5f92700 1 --2- 172.21.15.94:0/2284072547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fd00fff80 0x7f4fd0102360 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8004940 tx=0x7f4fb8009e30 comp rx=0 tx=0).stop 2024-03-24T10:53:50.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.681+0000 7f4fd5f92700 1 -- 172.21.15.94:0/2284072547 shutdown_connections 2024-03-24T10:53:50.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.681+0000 7f4fd5f92700 1 --2- 172.21.15.94:0/2284072547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fd00fff80 0x7f4fd0102360 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:50.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.681+0000 7f4fd5f92700 1 --2- 172.21.15.94:0/2284072547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4fd0066260 0x7f4fd00ffa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:50.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.681+0000 7f4fd5f92700 1 -- 172.21.15.94:0/2284072547 >> 172.21.15.94:0/2284072547 conn(0x7f4fd00f7430 msgr2=0x7f4fd00f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:53:50.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.681+0000 7f4fd5f92700 1 -- 172.21.15.94:0/2284072547 shutdown_connections 2024-03-24T10:53:50.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.681+0000 7f4fd5f92700 1 -- 172.21.15.94:0/2284072547 wait complete. 2024-03-24T10:53:50.682 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:50 smithi094 ceph-mon[93957]: pgmap v194: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 90 B/s wr, 5 op/s 2024-03-24T10:53:50.682 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:50 smithi094 ceph-mon[93957]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-24T10:53:50.682 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:50 smithi094 ceph-mon[93957]: Cluster is now healthy 2024-03-24T10:53:50.682 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:50 smithi094 ceph-mon[93957]: mds.? [v2:172.21.15.96:6826/1321087930,v1:172.21.15.96:6827/1321087930] up:active 2024-03-24T10:53:50.682 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:50 smithi094 ceph-mon[93957]: fsmap cephfs:1 {0=cephfs.smithi096.jtebhh=up:active} 2 up:standby 2024-03-24T10:53:50.682 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:50 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:50.682 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:50 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:50.682 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:50 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:53:50.682 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:50 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:53:50.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.681+0000 7f4fd5f92700 1 Processor -- start 2024-03-24T10:53:50.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.682+0000 7f4fd5f92700 1 -- start start 2024-03-24T10:53:50.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.682+0000 7f4fd5f92700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fd0066260 0x7f4fd010da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:53:50.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.682+0000 7f4fd5f92700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4fd00fff80 0x7f4fd010df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:53:50.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.682+0000 7f4fd5f92700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd010e570 con 0x7f4fd0066260 2024-03-24T10:53:50.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.682+0000 7f4fd5f92700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd010edc0 con 0x7f4fd00fff80 2024-03-24T10:53:50.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.682+0000 7f4fcf7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fd0066260 0x7f4fd010da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:53:50.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.682+0000 7f4fcf7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fd0066260 0x7f4fd010da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47378/0 (socket says 172.21.15.94:47378) 2024-03-24T10:53:50.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.682+0000 7f4fcf7fe700 1 -- 172.21.15.94:0/3688439789 learned_addr learned my addr 172.21.15.94:0/3688439789 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:53:50.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.682+0000 7f4fceffd700 1 --2- 172.21.15.94:0/3688439789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4fd00fff80 0x7f4fd010df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:53:50.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.682+0000 7f4fcf7fe700 1 -- 172.21.15.94:0/3688439789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4fd00fff80 msgr2=0x7f4fd010df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:50.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.682+0000 7f4fcf7fe700 1 --2- 172.21.15.94:0/3688439789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4fd00fff80 0x7f4fd010df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:50.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.682+0000 7f4fcf7fe700 1 -- 172.21.15.94:0/3688439789 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb8005040 con 0x7f4fd0066260 2024-03-24T10:53:50.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.682+0000 7f4fcf7fe700 1 --2- 172.21.15.94:0/3688439789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fd0066260 0x7f4fd010da30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8001ce0 tx=0x7f4fc8009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:53:50.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.682+0000 7f4fc7fff700 1 -- 172.21.15.94:0/3688439789 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fc8007b30 con 0x7f4fd0066260 2024-03-24T10:53:50.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.683+0000 7f4fd5f92700 1 -- 172.21.15.94:0/3688439789 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fd010f0a0 con 0x7f4fd0066260 2024-03-24T10:53:50.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.683+0000 7f4fc7fff700 1 -- 172.21.15.94:0/3688439789 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4fc800f040 con 0x7f4fd0066260 2024-03-24T10:53:50.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.683+0000 7f4fc7fff700 1 -- 172.21.15.94:0/3688439789 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fc80096d0 con 0x7f4fd0066260 2024-03-24T10:53:50.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.683+0000 7f4fd5f92700 1 -- 172.21.15.94:0/3688439789 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fd010f5e0 con 0x7f4fd0066260 2024-03-24T10:53:50.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.683+0000 7f4fd5f92700 1 -- 172.21.15.94:0/3688439789 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fd005f5f0 con 0x7f4fd0066260 2024-03-24T10:53:50.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.684+0000 7f4fc7fff700 1 -- 172.21.15.94:0/3688439789 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f4fc8007c90 con 0x7f4fd0066260 2024-03-24T10:53:50.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.684+0000 7f4fc7fff700 1 --2- 172.21.15.94:0/3688439789 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f4fbc06d7e0 0x7f4fbc06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:53:50.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.685+0000 7f4fceffd700 1 --2- 172.21.15.94:0/3688439789 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f4fbc06d7e0 0x7f4fbc06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:53:50.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.685+0000 7f4fc7fff700 1 -- 172.21.15.94:0/3688439789 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f4fc808c910 con 0x7f4fd0066260 2024-03-24T10:53:50.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.685+0000 7f4fceffd700 1 --2- 172.21.15.94:0/3688439789 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f4fbc06d7e0 0x7f4fbc06fc90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4fb800b3b0 tx=0x7f4fb8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:53:50.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.689+0000 7f4fc7fff700 1 -- 172.21.15.94:0/3688439789 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4fc8059c90 con 0x7f4fd0066260 2024-03-24T10:53:50.759 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:50 smithi096 ceph-mon[84258]: pgmap v194: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 90 B/s wr, 5 op/s 2024-03-24T10:53:50.759 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:50 smithi096 ceph-mon[84258]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-24T10:53:50.759 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:50 smithi096 ceph-mon[84258]: Cluster is now healthy 2024-03-24T10:53:50.759 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:50 smithi096 ceph-mon[84258]: mds.? [v2:172.21.15.96:6826/1321087930,v1:172.21.15.96:6827/1321087930] up:active 2024-03-24T10:53:50.759 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:50 smithi096 ceph-mon[84258]: fsmap cephfs:1 {0=cephfs.smithi096.jtebhh=up:active} 2 up:standby 2024-03-24T10:53:50.759 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:50 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:50.759 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:50 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:50.760 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:50 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:53:50.760 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:50 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:53:50.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.900+0000 7f4fd5f92700 1 -- 172.21.15.94:0/3688439789 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4fd010f970 con 0x7f4fbc06d7e0 2024-03-24T10:53:50.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.902+0000 7f4fc7fff700 1 -- 172.21.15.94:0/3688439789 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4fd010f970 con 0x7f4fbc06d7e0 2024-03-24T10:53:50.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.904+0000 7f4fd5f92700 1 -- 172.21.15.94:0/3688439789 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f4fbc06d7e0 msgr2=0x7f4fbc06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:50.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.904+0000 7f4fd5f92700 1 --2- 172.21.15.94:0/3688439789 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f4fbc06d7e0 0x7f4fbc06fc90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4fb800b3b0 tx=0x7f4fb8009e30 comp rx=0 tx=0).stop 2024-03-24T10:53:50.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.904+0000 7f4fd5f92700 1 -- 172.21.15.94:0/3688439789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fd0066260 msgr2=0x7f4fd010da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:50.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.904+0000 7f4fd5f92700 1 --2- 172.21.15.94:0/3688439789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fd0066260 0x7f4fd010da30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8001ce0 tx=0x7f4fc8009e60 comp rx=0 tx=0).stop 2024-03-24T10:53:50.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.904+0000 7f4fd5f92700 1 -- 172.21.15.94:0/3688439789 shutdown_connections 2024-03-24T10:53:50.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.904+0000 7f4fd5f92700 1 --2- 172.21.15.94:0/3688439789 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f4fbc06d7e0 0x7f4fbc06fc90 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:50.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.904+0000 7f4fd5f92700 1 --2- 172.21.15.94:0/3688439789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fd0066260 0x7f4fd010da30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:50.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.904+0000 7f4fd5f92700 1 --2- 172.21.15.94:0/3688439789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4fd00fff80 0x7f4fd010df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:50.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.904+0000 7f4fd5f92700 1 -- 172.21.15.94:0/3688439789 >> 172.21.15.94:0/3688439789 conn(0x7f4fd00f7430 msgr2=0x7f4fd00f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:53:50.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.904+0000 7f4fd5f92700 1 -- 172.21.15.94:0/3688439789 shutdown_connections 2024-03-24T10:53:50.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:50.904+0000 7f4fd5f92700 1 -- 172.21.15.94:0/3688439789 wait complete. 2024-03-24T10:53:51.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.005+0000 7f5df099d700 1 -- 172.21.15.94:0/951892905 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dec0fb480 msgr2=0x7f5dec0fb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:51.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.005+0000 7f5df099d700 1 --2- 172.21.15.94:0/951892905 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dec0fb480 0x7f5dec0fb890 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc004940 tx=0x7f5ddc009e30 comp rx=0 tx=0).stop 2024-03-24T10:53:51.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.005+0000 7f5df099d700 1 -- 172.21.15.94:0/951892905 shutdown_connections 2024-03-24T10:53:51.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.005+0000 7f5df099d700 1 --2- 172.21.15.94:0/951892905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5dec0fc730 0x7f5dec0fcba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:51.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.005+0000 7f5df099d700 1 --2- 172.21.15.94:0/951892905 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dec0fb480 0x7f5dec0fb890 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:51.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.005+0000 7f5df099d700 1 -- 172.21.15.94:0/951892905 >> 172.21.15.94:0/951892905 conn(0x7f5dec0f6be0 msgr2=0x7f5dec0f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:53:51.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.005+0000 7f5df099d700 1 -- 172.21.15.94:0/951892905 shutdown_connections 2024-03-24T10:53:51.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.005+0000 7f5df099d700 1 -- 172.21.15.94:0/951892905 wait complete. 2024-03-24T10:53:51.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.005+0000 7f5df099d700 1 Processor -- start 2024-03-24T10:53:51.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.005+0000 7f5df099d700 1 -- start start 2024-03-24T10:53:51.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.005+0000 7f5df099d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5dec0fb480 0x7f5dec10d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:53:51.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.005+0000 7f5df099d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dec0fc730 0x7f5dec10de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:53:51.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.005+0000 7f5df099d700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dec10e4e0 con 0x7f5dec0fc730 2024-03-24T10:53:51.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.005+0000 7f5df099d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dec112e80 con 0x7f5dec0fb480 2024-03-24T10:53:51.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.006+0000 7f5de9d9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dec0fc730 0x7f5dec10de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:53:51.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.006+0000 7f5de9d9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dec0fc730 0x7f5dec10de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47400/0 (socket says 172.21.15.94:47400) 2024-03-24T10:53:51.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.006+0000 7f5de9d9b700 1 -- 172.21.15.94:0/1286807947 learned_addr learned my addr 172.21.15.94:0/1286807947 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:53:51.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.006+0000 7f5dea59c700 1 --2- 172.21.15.94:0/1286807947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5dec0fb480 0x7f5dec10d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:53:51.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.006+0000 7f5de9d9b700 1 -- 172.21.15.94:0/1286807947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5dec0fb480 msgr2=0x7f5dec10d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:51.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.006+0000 7f5de9d9b700 1 --2- 172.21.15.94:0/1286807947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5dec0fb480 0x7f5dec10d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:51.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.006+0000 7f5de9d9b700 1 -- 172.21.15.94:0/1286807947 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ddc005040 con 0x7f5dec0fc730 2024-03-24T10:53:51.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.006+0000 7f5de9d9b700 1 --2- 172.21.15.94:0/1286807947 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dec0fc730 0x7f5dec10de50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5de000b790 tx=0x7f5de0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:53:51.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.006+0000 7f5ddaffd700 1 -- 172.21.15.94:0/1286807947 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5de000f6f0 con 0x7f5dec0fc730 2024-03-24T10:53:51.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.006+0000 7f5ddaffd700 1 -- 172.21.15.94:0/1286807947 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5de000fd20 con 0x7f5dec0fc730 2024-03-24T10:53:51.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.006+0000 7f5df099d700 1 -- 172.21.15.94:0/1286807947 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dec1130e0 con 0x7f5dec0fc730 2024-03-24T10:53:51.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.006+0000 7f5ddaffd700 1 -- 172.21.15.94:0/1286807947 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5de00103a0 con 0x7f5dec0fc730 2024-03-24T10:53:51.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.006+0000 7f5df099d700 1 -- 172.21.15.94:0/1286807947 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dec113650 con 0x7f5dec0fc730 2024-03-24T10:53:51.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.007+0000 7f5df099d700 1 -- 172.21.15.94:0/1286807947 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dec04c710 con 0x7f5dec0fc730 2024-03-24T10:53:51.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.008+0000 7f5ddaffd700 1 -- 172.21.15.94:0/1286807947 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f5de0011400 con 0x7f5dec0fc730 2024-03-24T10:53:51.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.008+0000 7f5ddaffd700 1 --2- 172.21.15.94:0/1286807947 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f5dd406d7e0 0x7f5dd406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:53:51.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.008+0000 7f5dea59c700 1 --2- 172.21.15.94:0/1286807947 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f5dd406d7e0 0x7f5dd406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:53:51.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.008+0000 7f5ddaffd700 1 -- 172.21.15.94:0/1286807947 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5de008cc20 con 0x7f5dec0fc730 2024-03-24T10:53:51.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.009+0000 7f5dea59c700 1 --2- 172.21.15.94:0/1286807947 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f5dd406d7e0 0x7f5dd406fc90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc00b970 tx=0x7f5ddc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:53:51.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.013+0000 7f5ddaffd700 1 -- 172.21.15.94:0/1286807947 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5de005a010 con 0x7f5dec0fc730 2024-03-24T10:53:51.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.220+0000 7f5df099d700 1 -- 172.21.15.94:0/1286807947 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5dec0f83e0 con 0x7f5dd406d7e0 2024-03-24T10:53:51.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.229+0000 7f5ddaffd700 1 -- 172.21.15.94:0/1286807947 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f5dec0f83e0 con 0x7f5dd406d7e0 2024-03-24T10:53:51.229 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T10:53:51.229 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.smithi094 smithi094 *:9093,9094 running (6m) 21s ago 16m 24.4M - 0.25.0 c8568f914cd2 5c583af60554 2024-03-24T10:53:51.229 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi094 smithi094 running (4m) 21s ago 16m 6651k - 19.0.0-1855-g21b3bd67 553463a90fd5 2b0897df571d 2024-03-24T10:53:51.229 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi096 smithi096 running (4m) 57s ago 14m 6669k - 19.0.0-1855-g21b3bd67 553463a90fd5 1759a9ff2805 2024-03-24T10:53:51.229 INFO:teuthology.orchestra.run.smithi094.stdout:grafana.smithi094 smithi094 *:3000 running (6m) 21s ago 15m 84.8M - 9.4.7 954c08fa6188 d3d425bf1b7c 2024-03-24T10:53:51.230 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.efiqte smithi094 running (42s) 21s ago 10m 15.1M - 19.0.0-1855-g21b3bd67 553463a90fd5 732c47b17971 2024-03-24T10:53:51.230 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.llefgu smithi094 running (26s) 21s ago 10m 14.8M - 19.0.0-1855-g21b3bd67 553463a90fd5 6e06f8a6dde0 2024-03-24T10:53:51.230 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi096.egbmoq smithi096 starting - - - - 2024-03-24T10:53:51.230 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi096.jtebhh smithi096 running (9m) 57s ago 9m 17.1M - 17.2.7-726-g9c4e0c34 837124da804f 4ff3451a6f6e 2024-03-24T10:53:51.230 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi094.yaethu smithi094 *:8443,9283,8765 running (6m) 21s ago 17m 506M - 19.0.0-1855-g21b3bd67 553463a90fd5 d3adab76b18f 2024-03-24T10:53:51.230 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi096.gfonnu smithi096 *:8443,9283,8765 running (5m) 57s ago 14m 450M - 19.0.0-1855-g21b3bd67 553463a90fd5 22937a059eb8 2024-03-24T10:53:51.230 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi094 smithi094 running (5m) 21s ago 17m 52.8M 2048M 19.0.0-1855-g21b3bd67 553463a90fd5 c779d611e29a 2024-03-24T10:53:51.230 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi096 smithi096 running (5m) 57s ago 14m 37.8M 2048M 19.0.0-1855-g21b3bd67 553463a90fd5 ed9bd0bd57f6 2024-03-24T10:53:51.230 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi094 smithi094 *:9100 running (7m) 21s ago 16m 18.6M - 1.5.0 0da6a335fe13 ca67b53c64e2 2024-03-24T10:53:51.230 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi096 smithi096 *:9100 running (6m) 57s ago 14m 16.7M - 1.5.0 0da6a335fe13 65137a7a7549 2024-03-24T10:53:51.230 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (3m) 21s ago 13m 103M 1557M 19.0.0-1855-g21b3bd67 553463a90fd5 ec57ede0c9e4 2024-03-24T10:53:51.230 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi094 running (3m) 21s ago 13m 111M 1557M 19.0.0-1855-g21b3bd67 553463a90fd5 c5b8b2bbf8b5 2024-03-24T10:53:51.231 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi094 running (2m) 21s ago 12m 92.4M 1557M 19.0.0-1855-g21b3bd67 553463a90fd5 c961dcdc746f 2024-03-24T10:53:51.231 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi096 running (2m) 57s ago 12m 105M 2581M 19.0.0-1855-g21b3bd67 553463a90fd5 6323fdc6e6ca 2024-03-24T10:53:51.231 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi096 running (95s) 57s ago 11m 106M 2581M 19.0.0-1855-g21b3bd67 553463a90fd5 3e2536bcb263 2024-03-24T10:53:51.231 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi096 running (61s) 57s ago 11m 14.8M 2581M 19.0.0-1855-g21b3bd67 553463a90fd5 2f77f1fa20e2 2024-03-24T10:53:51.231 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.smithi094 smithi094 *:9095 running (5m) 21s ago 14m 60.7M - 2.43.0 a07b618ecd1d 6ce091a0c326 2024-03-24T10:53:51.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.231+0000 7f5df099d700 1 -- 172.21.15.94:0/1286807947 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f5dd406d7e0 msgr2=0x7f5dd406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:51.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.231+0000 7f5df099d700 1 --2- 172.21.15.94:0/1286807947 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f5dd406d7e0 0x7f5dd406fc90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc00b970 tx=0x7f5ddc009e30 comp rx=0 tx=0).stop 2024-03-24T10:53:51.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.231+0000 7f5df099d700 1 -- 172.21.15.94:0/1286807947 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dec0fc730 msgr2=0x7f5dec10de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:51.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.231+0000 7f5df099d700 1 --2- 172.21.15.94:0/1286807947 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dec0fc730 0x7f5dec10de50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5de000b790 tx=0x7f5de0009250 comp rx=0 tx=0).stop 2024-03-24T10:53:51.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.231+0000 7f5df099d700 1 -- 172.21.15.94:0/1286807947 shutdown_connections 2024-03-24T10:53:51.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.231+0000 7f5df099d700 1 --2- 172.21.15.94:0/1286807947 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f5dd406d7e0 0x7f5dd406fc90 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:51.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.231+0000 7f5df099d700 1 --2- 172.21.15.94:0/1286807947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5dec0fb480 0x7f5dec10d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:51.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.231+0000 7f5df099d700 1 --2- 172.21.15.94:0/1286807947 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dec0fc730 0x7f5dec10de50 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:51.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.231+0000 7f5df099d700 1 -- 172.21.15.94:0/1286807947 >> 172.21.15.94:0/1286807947 conn(0x7f5dec0f6be0 msgr2=0x7f5dec0f7cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:53:51.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.232+0000 7f5df099d700 1 -- 172.21.15.94:0/1286807947 shutdown_connections 2024-03-24T10:53:51.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.232+0000 7f5df099d700 1 -- 172.21.15.94:0/1286807947 wait complete. 2024-03-24T10:53:51.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.331+0000 7f23f4715700 1 -- 172.21.15.94:0/4206307342 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23ec0fed90 msgr2=0x7f23ec0faed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:51.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.331+0000 7f23f4715700 1 --2- 172.21.15.94:0/4206307342 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23ec0fed90 0x7f23ec0faed0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f23dc004940 tx=0x7f23dc009e30 comp rx=0 tx=0).stop 2024-03-24T10:53:51.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.331+0000 7f23f4715700 1 -- 172.21.15.94:0/4206307342 shutdown_connections 2024-03-24T10:53:51.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.331+0000 7f23f4715700 1 --2- 172.21.15.94:0/4206307342 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23ec0fed90 0x7f23ec0faed0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:51.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.331+0000 7f23f4715700 1 --2- 172.21.15.94:0/4206307342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23ec0fe440 0x7f23ec0fe850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:51.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.331+0000 7f23f4715700 1 -- 172.21.15.94:0/4206307342 >> 172.21.15.94:0/4206307342 conn(0x7f23ec0f6be0 msgr2=0x7f23ec0f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:53:51.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.331+0000 7f23f4715700 1 -- 172.21.15.94:0/4206307342 shutdown_connections 2024-03-24T10:53:51.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.331+0000 7f23f4715700 1 -- 172.21.15.94:0/4206307342 wait complete. 2024-03-24T10:53:51.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.331+0000 7f23f4715700 1 Processor -- start 2024-03-24T10:53:51.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.332+0000 7f23f4715700 1 -- start start 2024-03-24T10:53:51.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.332+0000 7f23f4715700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23ec0fe440 0x7f23ec10d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:53:51.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.332+0000 7f23f4715700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23ec0fed90 0x7f23ec10dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:53:51.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.332+0000 7f23f4715700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23ec10e4a0 con 0x7f23ec0fed90 2024-03-24T10:53:51.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.332+0000 7f23f4715700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23ec10e610 con 0x7f23ec0fe440 2024-03-24T10:53:51.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.332+0000 7f23f1cb0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23ec0fed90 0x7f23ec10dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:53:51.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.332+0000 7f23f1cb0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23ec0fed90 0x7f23ec10dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47414/0 (socket says 172.21.15.94:47414) 2024-03-24T10:53:51.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.332+0000 7f23f1cb0700 1 -- 172.21.15.94:0/2440460241 learned_addr learned my addr 172.21.15.94:0/2440460241 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:53:51.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.332+0000 7f23f24b1700 1 --2- 172.21.15.94:0/2440460241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23ec0fe440 0x7f23ec10d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:53:51.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.332+0000 7f23f1cb0700 1 -- 172.21.15.94:0/2440460241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23ec0fe440 msgr2=0x7f23ec10d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:51.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.332+0000 7f23f1cb0700 1 --2- 172.21.15.94:0/2440460241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23ec0fe440 0x7f23ec10d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:51.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.332+0000 7f23f1cb0700 1 -- 172.21.15.94:0/2440460241 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23dc005040 con 0x7f23ec0fed90 2024-03-24T10:53:51.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.332+0000 7f23f1cb0700 1 --2- 172.21.15.94:0/2440460241 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23ec0fed90 0x7f23ec10dea0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f23dc00b970 tx=0x7f23dc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:53:51.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.333+0000 7f23e2ffd700 1 -- 172.21.15.94:0/2440460241 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23dc00e040 con 0x7f23ec0fed90 2024-03-24T10:53:51.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.333+0000 7f23e2ffd700 1 -- 172.21.15.94:0/2440460241 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f23dc00bbf0 con 0x7f23ec0fed90 2024-03-24T10:53:51.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.333+0000 7f23f4715700 1 -- 172.21.15.94:0/2440460241 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23ec113010 con 0x7f23ec0fed90 2024-03-24T10:53:51.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.333+0000 7f23e2ffd700 1 -- 172.21.15.94:0/2440460241 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23dc01c630 con 0x7f23ec0fed90 2024-03-24T10:53:51.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.333+0000 7f23f4715700 1 -- 172.21.15.94:0/2440460241 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23ec1135d0 con 0x7f23ec0fed90 2024-03-24T10:53:51.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.334+0000 7f23f4715700 1 -- 172.21.15.94:0/2440460241 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23ec04c710 con 0x7f23ec0fed90 2024-03-24T10:53:51.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.334+0000 7f23e2ffd700 1 -- 172.21.15.94:0/2440460241 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f23dc00bd60 con 0x7f23ec0fed90 2024-03-24T10:53:51.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.335+0000 7f23e2ffd700 1 --2- 172.21.15.94:0/2440460241 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f23d806d7e0 0x7f23d806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:53:51.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.335+0000 7f23f24b1700 1 --2- 172.21.15.94:0/2440460241 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f23d806d7e0 0x7f23d806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:53:51.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.335+0000 7f23e2ffd700 1 -- 172.21.15.94:0/2440460241 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f23dc08e310 con 0x7f23ec0fed90 2024-03-24T10:53:51.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.335+0000 7f23f24b1700 1 --2- 172.21.15.94:0/2440460241 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f23d806d7e0 0x7f23d806fc90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f23e8004f60 tx=0x7f23e800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:53:51.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.339+0000 7f23e2ffd700 1 -- 172.21.15.94:0/2440460241 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f23dc05b600 con 0x7f23ec0fed90 2024-03-24T10:53:51.525 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:51 smithi096 ceph-mon[84258]: mds.? [v2:172.21.15.96:6824/2694461318,v1:172.21.15.96:6825/2694461318] up:boot 2024-03-24T10:53:51.526 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:51 smithi096 ceph-mon[84258]: fsmap cephfs:1 {0=cephfs.smithi096.jtebhh=up:active} 3 up:standby 2024-03-24T10:53:51.526 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:51 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi096.egbmoq"}]: dispatch 2024-03-24T10:53:51.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.617+0000 7f23f4715700 1 -- 172.21.15.94:0/2440460241 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f23ec105910 con 0x7f23ec0fed90 2024-03-24T10:53:51.617 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:51 smithi094 ceph-mon[93957]: mds.? [v2:172.21.15.96:6824/2694461318,v1:172.21.15.96:6825/2694461318] up:boot 2024-03-24T10:53:51.618 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:51 smithi094 ceph-mon[93957]: fsmap cephfs:1 {0=cephfs.smithi096.jtebhh=up:active} 3 up:standby 2024-03-24T10:53:51.618 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:51 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi096.egbmoq"}]: dispatch 2024-03-24T10:53:51.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.618+0000 7f23e2ffd700 1 -- 172.21.15.94:0/2440460241 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f23dc05b190 con 0x7f23ec0fed90 2024-03-24T10:53:51.618 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-03-24T10:53:51.618 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": { 2024-03-24T10:53:51.618 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 2 2024-03-24T10:53:51.618 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:53:51.618 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": { 2024-03-24T10:53:51.618 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 2 2024-03-24T10:53:51.619 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:53:51.619 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": { 2024-03-24T10:53:51.619 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 6 2024-03-24T10:53:51.619 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:53:51.619 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-03-24T10:53:51.619 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 1, 2024-03-24T10:53:51.619 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 3 2024-03-24T10:53:51.619 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:53:51.619 INFO:teuthology.orchestra.run.smithi094.stdout: "overall": { 2024-03-24T10:53:51.619 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 1, 2024-03-24T10:53:51.619 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 13 2024-03-24T10:53:51.619 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-03-24T10:53:51.619 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-03-24T10:53:51.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.620+0000 7f23f4715700 1 -- 172.21.15.94:0/2440460241 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f23d806d7e0 msgr2=0x7f23d806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:51.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.620+0000 7f23f4715700 1 --2- 172.21.15.94:0/2440460241 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f23d806d7e0 0x7f23d806fc90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f23e8004f60 tx=0x7f23e800b040 comp rx=0 tx=0).stop 2024-03-24T10:53:51.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.620+0000 7f23f4715700 1 -- 172.21.15.94:0/2440460241 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23ec0fed90 msgr2=0x7f23ec10dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:51.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.620+0000 7f23f4715700 1 --2- 172.21.15.94:0/2440460241 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23ec0fed90 0x7f23ec10dea0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f23dc00b970 tx=0x7f23dc009e30 comp rx=0 tx=0).stop 2024-03-24T10:53:51.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.620+0000 7f23f4715700 1 -- 172.21.15.94:0/2440460241 shutdown_connections 2024-03-24T10:53:51.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.620+0000 7f23f4715700 1 --2- 172.21.15.94:0/2440460241 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f23d806d7e0 0x7f23d806fc90 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:51.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.620+0000 7f23f4715700 1 --2- 172.21.15.94:0/2440460241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23ec0fe440 0x7f23ec10d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:51.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.620+0000 7f23f4715700 1 --2- 172.21.15.94:0/2440460241 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23ec0fed90 0x7f23ec10dea0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:51.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.620+0000 7f23f4715700 1 -- 172.21.15.94:0/2440460241 >> 172.21.15.94:0/2440460241 conn(0x7f23ec0f6be0 msgr2=0x7f23ec0f82a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:53:51.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.620+0000 7f23f4715700 1 -- 172.21.15.94:0/2440460241 shutdown_connections 2024-03-24T10:53:51.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.620+0000 7f23f4715700 1 -- 172.21.15.94:0/2440460241 wait complete. 2024-03-24T10:53:51.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.721+0000 7ffa28777700 1 -- 172.21.15.94:0/2216577247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa200fbe00 msgr2=0x7ffa200fc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:51.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.721+0000 7ffa28777700 1 --2- 172.21.15.94:0/2216577247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa200fbe00 0x7ffa200fc210 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ffa10004940 tx=0x7ffa10009e30 comp rx=0 tx=0).stop 2024-03-24T10:53:51.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.721+0000 7ffa28777700 1 -- 172.21.15.94:0/2216577247 shutdown_connections 2024-03-24T10:53:51.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.721+0000 7ffa28777700 1 --2- 172.21.15.94:0/2216577247 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa200fcf70 0x7ffa200fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:51.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.721+0000 7ffa28777700 1 --2- 172.21.15.94:0/2216577247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa200fbe00 0x7ffa200fc210 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:51.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.721+0000 7ffa28777700 1 -- 172.21.15.94:0/2216577247 >> 172.21.15.94:0/2216577247 conn(0x7ffa200f74e0 msgr2=0x7ffa200f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:53:51.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.721+0000 7ffa28777700 1 -- 172.21.15.94:0/2216577247 shutdown_connections 2024-03-24T10:53:51.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.721+0000 7ffa28777700 1 -- 172.21.15.94:0/2216577247 wait complete. 2024-03-24T10:53:51.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.722+0000 7ffa28777700 1 Processor -- start 2024-03-24T10:53:51.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.722+0000 7ffa28777700 1 -- start start 2024-03-24T10:53:51.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.722+0000 7ffa28777700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa200fbe00 0x7ffa20110750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:53:51.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.722+0000 7ffa28777700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa200fcf70 0x7ffa20112ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:53:51.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.722+0000 7ffa28777700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa20066be0 con 0x7ffa200fbe00 2024-03-24T10:53:51.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.722+0000 7ffa28777700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa20066d20 con 0x7ffa200fcf70 2024-03-24T10:53:51.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.722+0000 7ffa26513700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa200fbe00 0x7ffa20110750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:53:51.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.722+0000 7ffa26513700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa200fbe00 0x7ffa20110750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47426/0 (socket says 172.21.15.94:47426) 2024-03-24T10:53:51.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.722+0000 7ffa26513700 1 -- 172.21.15.94:0/4157786329 learned_addr learned my addr 172.21.15.94:0/4157786329 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:53:51.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.722+0000 7ffa25d12700 1 --2- 172.21.15.94:0/4157786329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa200fcf70 0x7ffa20112ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:53:51.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.722+0000 7ffa26513700 1 -- 172.21.15.94:0/4157786329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa200fcf70 msgr2=0x7ffa20112ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:51.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.722+0000 7ffa26513700 1 --2- 172.21.15.94:0/4157786329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa200fcf70 0x7ffa20112ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:51.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.722+0000 7ffa26513700 1 -- 172.21.15.94:0/4157786329 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa10005040 con 0x7ffa200fbe00 2024-03-24T10:53:51.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.722+0000 7ffa26513700 1 --2- 172.21.15.94:0/4157786329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa200fbe00 0x7ffa20110750 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ffa10009150 tx=0x7ffa10009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:53:51.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.722+0000 7ffa16ffd700 1 -- 172.21.15.94:0/4157786329 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa1000e040 con 0x7ffa200fbe00 2024-03-24T10:53:51.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.723+0000 7ffa16ffd700 1 -- 172.21.15.94:0/4157786329 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffa10004290 con 0x7ffa200fbe00 2024-03-24T10:53:51.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.723+0000 7ffa28777700 1 -- 172.21.15.94:0/4157786329 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa201131e0 con 0x7ffa200fbe00 2024-03-24T10:53:51.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.723+0000 7ffa16ffd700 1 -- 172.21.15.94:0/4157786329 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa1001c630 con 0x7ffa200fbe00 2024-03-24T10:53:51.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.723+0000 7ffa28777700 1 -- 172.21.15.94:0/4157786329 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa201137a0 con 0x7ffa200fbe00 2024-03-24T10:53:51.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.723+0000 7ffa28777700 1 -- 172.21.15.94:0/4157786329 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa2005f5f0 con 0x7ffa200fbe00 2024-03-24T10:53:51.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.724+0000 7ffa16ffd700 1 -- 172.21.15.94:0/4157786329 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7ffa1000bbf0 con 0x7ffa200fbe00 2024-03-24T10:53:51.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.724+0000 7ffa16ffd700 1 --2- 172.21.15.94:0/4157786329 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7ffa0c06d7e0 0x7ffa0c06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:53:51.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.724+0000 7ffa16ffd700 1 -- 172.21.15.94:0/4157786329 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ffa1008db20 con 0x7ffa200fbe00 2024-03-24T10:53:51.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.724+0000 7ffa25d12700 1 --2- 172.21.15.94:0/4157786329 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7ffa0c06d7e0 0x7ffa0c06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:53:51.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.725+0000 7ffa25d12700 1 --2- 172.21.15.94:0/4157786329 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7ffa0c06d7e0 0x7ffa0c06fc90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c007fd0 tx=0x7ffa1c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:53:51.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.729+0000 7ffa16ffd700 1 -- 172.21.15.94:0/4157786329 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ffa10057340 con 0x7ffa200fbe00 2024-03-24T10:53:51.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.965+0000 7ffa28777700 1 -- 172.21.15.94:0/4157786329 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ffa20113a80 con 0x7ffa200fbe00 2024-03-24T10:53:51.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.965+0000 7ffa16ffd700 1 -- 172.21.15.94:0/4157786329 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1892 (secure 0 0 0) 0x7ffa1005aa00 con 0x7ffa200fbe00 2024-03-24T10:53:51.965 INFO:teuthology.orchestra.run.smithi094.stdout:e30 2024-03-24T10:53:51.966 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T10:53:51.966 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:53:51.966 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-03-24T10:53:51.966 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:53:51.966 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-03-24T10:53:51.966 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-03-24T10:53:51.966 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 29 2024-03-24T10:53:51.966 INFO:teuthology.orchestra.run.smithi094.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-24T10:53:51.966 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-03-24T10:43:39.435738+0000 2024-03-24T10:53:51.966 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-03-24T10:53:49.399665+0000 2024-03-24T10:53:51.966 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-03-24T10:53:51.966 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-03-24T10:53:51.967 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 60 2024-03-24T10:53:51.967 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-03-24T10:53:51.967 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-03-24T10:53:51.967 INFO:teuthology.orchestra.run.smithi094.stdout:max_xattr_size 65536 2024-03-24T10:53:51.967 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-03-24T10:53:51.967 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-03-24T10:53:51.967 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 83 2024-03-24T10:53:51.967 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:53:51.967 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-03-24T10:53:51.967 INFO:teuthology.orchestra.run.smithi094.stdout:in 0 2024-03-24T10:53:51.967 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=24261} 2024-03-24T10:53:51.967 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-03-24T10:53:51.967 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-03-24T10:53:51.967 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-03-24T10:53:51.968 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-03-24T10:53:51.968 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-03-24T10:53:51.968 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-03-24T10:53:51.968 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-03-24T10:53:51.968 INFO:teuthology.orchestra.run.smithi094.stdout:bal_rank_mask -1 2024-03-24T10:53:51.968 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 1 2024-03-24T10:53:51.968 INFO:teuthology.orchestra.run.smithi094.stdout:qdb_cluster leader: 24261 members: 24261 2024-03-24T10:53:51.968 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi096.jtebhh{0:24261} state up:active seq 152 join_fscid=1 addr [v2:172.21.15.96:6826/1321087930,v1:172.21.15.96:6827/1321087930] compat {c=[1],r=[1],i=[7ff]}] 2024-03-24T10:53:51.968 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:53:51.968 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:53:51.968 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-03-24T10:53:51.968 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:53:51.968 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.efiqte{-1:34374} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.94:6826/2043632450,v1:172.21.15.94:6827/2043632450] compat {c=[1],r=[1],i=[1fff]}] 2024-03-24T10:53:51.968 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.llefgu{-1:34406} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.94:6828/2396893131,v1:172.21.15.94:6829/2396893131] compat {c=[1],r=[1],i=[1fff]}] 2024-03-24T10:53:51.968 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi096.egbmoq{-1:44321} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.96:6824/2694461318,v1:172.21.15.96:6825/2694461318] compat {c=[1],r=[1],i=[1fff]}] 2024-03-24T10:53:51.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.967+0000 7ffa28777700 1 -- 172.21.15.94:0/4157786329 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7ffa0c06d7e0 msgr2=0x7ffa0c06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:51.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.967+0000 7ffa28777700 1 --2- 172.21.15.94:0/4157786329 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7ffa0c06d7e0 0x7ffa0c06fc90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c007fd0 tx=0x7ffa1c009040 comp rx=0 tx=0).stop 2024-03-24T10:53:51.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.967+0000 7ffa28777700 1 -- 172.21.15.94:0/4157786329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa200fbe00 msgr2=0x7ffa20110750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:51.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.967+0000 7ffa28777700 1 --2- 172.21.15.94:0/4157786329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa200fbe00 0x7ffa20110750 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ffa10009150 tx=0x7ffa10009e30 comp rx=0 tx=0).stop 2024-03-24T10:53:51.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.968+0000 7ffa28777700 1 -- 172.21.15.94:0/4157786329 shutdown_connections 2024-03-24T10:53:51.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.968+0000 7ffa28777700 1 --2- 172.21.15.94:0/4157786329 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7ffa0c06d7e0 0x7ffa0c06fc90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:51.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.968+0000 7ffa28777700 1 --2- 172.21.15.94:0/4157786329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa200fbe00 0x7ffa20110750 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:51.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.968+0000 7ffa28777700 1 --2- 172.21.15.94:0/4157786329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa200fcf70 0x7ffa20112ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:51.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.968+0000 7ffa28777700 1 -- 172.21.15.94:0/4157786329 >> 172.21.15.94:0/4157786329 conn(0x7ffa200f74e0 msgr2=0x7ffa200f8bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:53:51.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.968+0000 7ffa28777700 1 -- 172.21.15.94:0/4157786329 shutdown_connections 2024-03-24T10:53:51.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:51.968+0000 7ffa28777700 1 -- 172.21.15.94:0/4157786329 wait complete. 2024-03-24T10:53:51.970 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 30 2024-03-24T10:53:52.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.070+0000 7f8073a51700 1 -- 172.21.15.94:0/2154827993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f806c0fbcd0 msgr2=0x7f806c0fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:52.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.070+0000 7f8073a51700 1 --2- 172.21.15.94:0/2154827993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f806c0fbcd0 0x7f806c0fc0e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8068004940 tx=0x7f8068009e30 comp rx=0 tx=0).stop 2024-03-24T10:53:52.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.070+0000 7f8073a51700 1 -- 172.21.15.94:0/2154827993 shutdown_connections 2024-03-24T10:53:52.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.070+0000 7f8073a51700 1 --2- 172.21.15.94:0/2154827993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f806c0fcf80 0x7f806c0fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:52.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.070+0000 7f8073a51700 1 --2- 172.21.15.94:0/2154827993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f806c0fbcd0 0x7f806c0fc0e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:52.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.070+0000 7f8073a51700 1 -- 172.21.15.94:0/2154827993 >> 172.21.15.94:0/2154827993 conn(0x7f806c0f7430 msgr2=0x7f806c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:53:52.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.070+0000 7f8073a51700 1 -- 172.21.15.94:0/2154827993 shutdown_connections 2024-03-24T10:53:52.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.070+0000 7f8073a51700 1 -- 172.21.15.94:0/2154827993 wait complete. 2024-03-24T10:53:52.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.071+0000 7f8073a51700 1 Processor -- start 2024-03-24T10:53:52.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.071+0000 7f8073a51700 1 -- start start 2024-03-24T10:53:52.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.071+0000 7f8073a51700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f806c0fbcd0 0x7f806c110630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:53:52.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.071+0000 7f8073a51700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f806c0fcf80 0x7f806c112b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:53:52.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.071+0000 7f8073a51700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806c0663b0 con 0x7f806c0fbcd0 2024-03-24T10:53:52.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.071+0000 7f8073a51700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806c0664f0 con 0x7f806c0fcf80 2024-03-24T10:53:52.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.071+0000 7f80717ed700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f806c0fbcd0 0x7f806c110630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:53:52.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.071+0000 7f80717ed700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f806c0fbcd0 0x7f806c110630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47454/0 (socket says 172.21.15.94:47454) 2024-03-24T10:53:52.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.071+0000 7f80717ed700 1 -- 172.21.15.94:0/1366668370 learned_addr learned my addr 172.21.15.94:0/1366668370 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:53:52.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.071+0000 7f8070fec700 1 --2- 172.21.15.94:0/1366668370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f806c0fcf80 0x7f806c112b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:53:52.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.071+0000 7f80717ed700 1 -- 172.21.15.94:0/1366668370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f806c0fcf80 msgr2=0x7f806c112b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:52.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.071+0000 7f80717ed700 1 --2- 172.21.15.94:0/1366668370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f806c0fcf80 0x7f806c112b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:52.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.071+0000 7f80717ed700 1 -- 172.21.15.94:0/1366668370 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8068005040 con 0x7f806c0fbcd0 2024-03-24T10:53:52.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.071+0000 7f8070fec700 1 --2- 172.21.15.94:0/1366668370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f806c0fcf80 0x7f806c112b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T10:53:52.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.071+0000 7f80717ed700 1 --2- 172.21.15.94:0/1366668370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f806c0fbcd0 0x7f806c110630 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8068013870 tx=0x7f8068009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:53:52.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.072+0000 7f8061ffb700 1 -- 172.21.15.94:0/1366668370 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8068021070 con 0x7f806c0fbcd0 2024-03-24T10:53:52.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.072+0000 7f8061ffb700 1 -- 172.21.15.94:0/1366668370 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8068009430 con 0x7f806c0fbcd0 2024-03-24T10:53:52.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.072+0000 7f8073a51700 1 -- 172.21.15.94:0/1366668370 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f806c113120 con 0x7f806c0fbcd0 2024-03-24T10:53:52.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.072+0000 7f8061ffb700 1 -- 172.21.15.94:0/1366668370 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f806800e070 con 0x7f806c0fbcd0 2024-03-24T10:53:52.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.072+0000 7f8073a51700 1 -- 172.21.15.94:0/1366668370 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f806c113660 con 0x7f806c0fbcd0 2024-03-24T10:53:52.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.073+0000 7f8073a51700 1 -- 172.21.15.94:0/1366668370 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f806c05f5f0 con 0x7f806c0fbcd0 2024-03-24T10:53:52.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.073+0000 7f8061ffb700 1 -- 172.21.15.94:0/1366668370 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f8068013af0 con 0x7f806c0fbcd0 2024-03-24T10:53:52.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.074+0000 7f8061ffb700 1 --2- 172.21.15.94:0/1366668370 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f805806d7e0 0x7f805806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:53:52.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.074+0000 7f8061ffb700 1 -- 172.21.15.94:0/1366668370 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f806808e040 con 0x7f806c0fbcd0 2024-03-24T10:53:52.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.074+0000 7f8070fec700 1 --2- 172.21.15.94:0/1366668370 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f805806d7e0 0x7f805806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:53:52.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.074+0000 7f8070fec700 1 --2- 172.21.15.94:0/1366668370 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f805806d7e0 0x7f805806fc90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f805c009b80 tx=0x7f805c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:53:52.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.078+0000 7f8061ffb700 1 -- 172.21.15.94:0/1366668370 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8068057900 con 0x7f806c0fbcd0 2024-03-24T10:53:52.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.291+0000 7f8073a51700 1 -- 172.21.15.94:0/1366668370 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f806c1012f0 con 0x7f805806d7e0 2024-03-24T10:53:52.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.293+0000 7f8061ffb700 1 -- 172.21.15.94:0/1366668370 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f806c1012f0 con 0x7f805806d7e0 2024-03-24T10:53:52.292 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-03-24T10:53:52.292 INFO:teuthology.orchestra.run.smithi094.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919", 2024-03-24T10:53:52.293 INFO:teuthology.orchestra.run.smithi094.stdout: "in_progress": true, 2024-03-24T10:53:52.293 INFO:teuthology.orchestra.run.smithi094.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T10:53:52.293 INFO:teuthology.orchestra.run.smithi094.stdout: "services_complete": [ 2024-03-24T10:53:52.293 INFO:teuthology.orchestra.run.smithi094.stdout: "mon", 2024-03-24T10:53:52.293 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-03-24T10:53:52.293 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr", 2024-03-24T10:53:52.293 INFO:teuthology.orchestra.run.smithi094.stdout: "osd" 2024-03-24T10:53:52.293 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-03-24T10:53:52.293 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": "14/21 daemons upgraded", 2024-03-24T10:53:52.293 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "Currently upgrading mds daemons", 2024-03-24T10:53:52.293 INFO:teuthology.orchestra.run.smithi094.stdout: "is_paused": false 2024-03-24T10:53:52.293 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-03-24T10:53:52.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.295+0000 7f8073a51700 1 -- 172.21.15.94:0/1366668370 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f805806d7e0 msgr2=0x7f805806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:52.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.295+0000 7f8073a51700 1 --2- 172.21.15.94:0/1366668370 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f805806d7e0 0x7f805806fc90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f805c009b80 tx=0x7f805c00b040 comp rx=0 tx=0).stop 2024-03-24T10:53:52.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.295+0000 7f8073a51700 1 -- 172.21.15.94:0/1366668370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f806c0fbcd0 msgr2=0x7f806c110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:52.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.295+0000 7f8073a51700 1 --2- 172.21.15.94:0/1366668370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f806c0fbcd0 0x7f806c110630 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8068013870 tx=0x7f8068009e30 comp rx=0 tx=0).stop 2024-03-24T10:53:52.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.295+0000 7f8073a51700 1 -- 172.21.15.94:0/1366668370 shutdown_connections 2024-03-24T10:53:52.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.295+0000 7f8073a51700 1 --2- 172.21.15.94:0/1366668370 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f805806d7e0 0x7f805806fc90 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:52.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.295+0000 7f8073a51700 1 --2- 172.21.15.94:0/1366668370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f806c0fbcd0 0x7f806c110630 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:52.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.295+0000 7f8073a51700 1 --2- 172.21.15.94:0/1366668370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f806c0fcf80 0x7f806c112b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:52.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.295+0000 7f8073a51700 1 -- 172.21.15.94:0/1366668370 >> 172.21.15.94:0/1366668370 conn(0x7f806c0f7430 msgr2=0x7f806c0f8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:53:52.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.295+0000 7f8073a51700 1 -- 172.21.15.94:0/1366668370 shutdown_connections 2024-03-24T10:53:52.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.295+0000 7f8073a51700 1 -- 172.21.15.94:0/1366668370 wait complete. 2024-03-24T10:53:52.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.396+0000 7f1f82bcc700 1 -- 172.21.15.94:0/2858121129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f7c0fe430 msgr2=0x7f1f7c0fe840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:52.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.396+0000 7f1f82bcc700 1 --2- 172.21.15.94:0/2858121129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f7c0fe430 0x7f1f7c0fe840 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1f64004940 tx=0x7f1f64009e30 comp rx=0 tx=0).stop 2024-03-24T10:53:52.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.396+0000 7f1f82bcc700 1 -- 172.21.15.94:0/2858121129 shutdown_connections 2024-03-24T10:53:52.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.396+0000 7f1f82bcc700 1 --2- 172.21.15.94:0/2858121129 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f7c0fed80 0x7f1f7c0faec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:52.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.396+0000 7f1f82bcc700 1 --2- 172.21.15.94:0/2858121129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f7c0fe430 0x7f1f7c0fe840 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:52.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.396+0000 7f1f82bcc700 1 -- 172.21.15.94:0/2858121129 >> 172.21.15.94:0/2858121129 conn(0x7f1f7c0f6bd0 msgr2=0x7f1f7c0f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:53:52.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.396+0000 7f1f82bcc700 1 -- 172.21.15.94:0/2858121129 shutdown_connections 2024-03-24T10:53:52.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.396+0000 7f1f82bcc700 1 -- 172.21.15.94:0/2858121129 wait complete. 2024-03-24T10:53:52.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.396+0000 7f1f82bcc700 1 Processor -- start 2024-03-24T10:53:52.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.396+0000 7f1f82bcc700 1 -- start start 2024-03-24T10:53:52.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.397+0000 7f1f82bcc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f7c0fe430 0x7f1f7c10d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:53:52.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.397+0000 7f1f82bcc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f7c0fed80 0x7f1f7c10de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:53:52.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.397+0000 7f1f82bcc700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c10e450 con 0x7f1f7c0fed80 2024-03-24T10:53:52.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.397+0000 7f1f82bcc700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c10e5c0 con 0x7f1f7c0fe430 2024-03-24T10:53:52.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.397+0000 7f1f7bfff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f7c0fed80 0x7f1f7c10de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:53:52.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.397+0000 7f1f7bfff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f7c0fed80 0x7f1f7c10de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47478/0 (socket says 172.21.15.94:47478) 2024-03-24T10:53:52.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.397+0000 7f1f7bfff700 1 -- 172.21.15.94:0/2183658428 learned_addr learned my addr 172.21.15.94:0/2183658428 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:53:52.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.397+0000 7f1f80968700 1 --2- 172.21.15.94:0/2183658428 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f7c0fe430 0x7f1f7c10d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:53:52.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.397+0000 7f1f7bfff700 1 -- 172.21.15.94:0/2183658428 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f7c0fe430 msgr2=0x7f1f7c10d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:52.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.397+0000 7f1f7bfff700 1 --2- 172.21.15.94:0/2183658428 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f7c0fe430 0x7f1f7c10d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:52.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.397+0000 7f1f7bfff700 1 -- 172.21.15.94:0/2183658428 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f64005040 con 0x7f1f7c0fed80 2024-03-24T10:53:52.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.397+0000 7f1f7bfff700 1 --2- 172.21.15.94:0/2183658428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f7c0fed80 0x7f1f7c10de50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c002a50 tx=0x7f1f6c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:53:52.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.397+0000 7f1f797fa700 1 -- 172.21.15.94:0/2183658428 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f6c007cc0 con 0x7f1f7c0fed80 2024-03-24T10:53:52.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.397+0000 7f1f797fa700 1 -- 172.21.15.94:0/2183658428 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f6c00f040 con 0x7f1f7c0fed80 2024-03-24T10:53:52.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.397+0000 7f1f797fa700 1 -- 172.21.15.94:0/2183658428 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f6c00da30 con 0x7f1f7c0fed80 2024-03-24T10:53:52.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.397+0000 7f1f82bcc700 1 -- 172.21.15.94:0/2183658428 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f7c113020 con 0x7f1f7c0fed80 2024-03-24T10:53:52.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.397+0000 7f1f82bcc700 1 -- 172.21.15.94:0/2183658428 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f7c1135b0 con 0x7f1f7c0fed80 2024-03-24T10:53:52.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.398+0000 7f1f82bcc700 1 -- 172.21.15.94:0/2183658428 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f7c04c710 con 0x7f1f7c0fed80 2024-03-24T10:53:52.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.399+0000 7f1f797fa700 1 -- 172.21.15.94:0/2183658428 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f1f6c00dbb0 con 0x7f1f7c0fed80 2024-03-24T10:53:52.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.399+0000 7f1f797fa700 1 --2- 172.21.15.94:0/2183658428 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1f6806d7e0 0x7f1f6806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:53:52.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.399+0000 7f1f797fa700 1 -- 172.21.15.94:0/2183658428 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f1f6c08cc20 con 0x7f1f7c0fed80 2024-03-24T10:53:52.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.399+0000 7f1f80968700 1 --2- 172.21.15.94:0/2183658428 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1f6806d7e0 0x7f1f6806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:53:52.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.400+0000 7f1f80968700 1 --2- 172.21.15.94:0/2183658428 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1f6806d7e0 0x7f1f6806fc90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1f6400b3b0 tx=0x7f1f64009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:53:52.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.404+0000 7f1f797fa700 1 -- 172.21.15.94:0/2183658428 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1f6c059f70 con 0x7f1f7c0fed80 2024-03-24T10:53:52.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.682+0000 7f1f82bcc700 1 -- 172.21.15.94:0/2183658428 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1f7c1058e0 con 0x7f1f7c0fed80 2024-03-24T10:53:52.682 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:52 smithi094 ceph-mon[93957]: from='client.44325 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:53:52.682 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:52 smithi094 ceph-mon[93957]: from='client.34414 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:53:52.682 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:52 smithi094 ceph-mon[93957]: from='client.34418 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:53:52.682 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:52 smithi094 ceph-mon[93957]: pgmap v195: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 85 B/s wr, 4 op/s 2024-03-24T10:53:52.682 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:52 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/2440460241' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:53:52.682 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:52 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/4157786329' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T10:53:52.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.682+0000 7f1f797fa700 1 -- 172.21.15.94:0/2183658428 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f1f6c059b00 con 0x7f1f7c0fed80 2024-03-24T10:53:52.683 INFO:teuthology.orchestra.run.smithi094.stdout:HEALTH_OK 2024-03-24T10:53:52.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.684+0000 7f1f82bcc700 1 -- 172.21.15.94:0/2183658428 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1f6806d7e0 msgr2=0x7f1f6806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:52.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.684+0000 7f1f82bcc700 1 --2- 172.21.15.94:0/2183658428 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1f6806d7e0 0x7f1f6806fc90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1f6400b3b0 tx=0x7f1f64009e30 comp rx=0 tx=0).stop 2024-03-24T10:53:52.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.684+0000 7f1f82bcc700 1 -- 172.21.15.94:0/2183658428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f7c0fed80 msgr2=0x7f1f7c10de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:53:52.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.684+0000 7f1f82bcc700 1 --2- 172.21.15.94:0/2183658428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f7c0fed80 0x7f1f7c10de50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c002a50 tx=0x7f1f6c0046b0 comp rx=0 tx=0).stop 2024-03-24T10:53:52.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.684+0000 7f1f82bcc700 1 -- 172.21.15.94:0/2183658428 shutdown_connections 2024-03-24T10:53:52.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.684+0000 7f1f82bcc700 1 --2- 172.21.15.94:0/2183658428 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1f6806d7e0 0x7f1f6806fc90 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:52.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.684+0000 7f1f82bcc700 1 --2- 172.21.15.94:0/2183658428 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f7c0fe430 0x7f1f7c10d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:52.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.684+0000 7f1f82bcc700 1 --2- 172.21.15.94:0/2183658428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f7c0fed80 0x7f1f7c10de50 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:53:52.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.684+0000 7f1f82bcc700 1 -- 172.21.15.94:0/2183658428 >> 172.21.15.94:0/2183658428 conn(0x7f1f7c0f6bd0 msgr2=0x7f1f7c0f82b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:53:52.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.684+0000 7f1f82bcc700 1 -- 172.21.15.94:0/2183658428 shutdown_connections 2024-03-24T10:53:52.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:53:52.684+0000 7f1f82bcc700 1 -- 172.21.15.94:0/2183658428 wait complete. 2024-03-24T10:53:52.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:52 smithi096 ceph-mon[84258]: from='client.44325 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:53:52.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:52 smithi096 ceph-mon[84258]: from='client.34414 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:53:52.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:52 smithi096 ceph-mon[84258]: from='client.34418 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:53:52.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:52 smithi096 ceph-mon[84258]: pgmap v195: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 85 B/s wr, 4 op/s 2024-03-24T10:53:52.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:52 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/2440460241' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:53:52.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:52 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/4157786329' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T10:53:53.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:53 smithi094 ceph-mon[93957]: from='client.34430 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:53:53.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:53 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/2183658428' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T10:53:53.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:53 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:53.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:53 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:53.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:53 smithi096 ceph-mon[84258]: from='client.34430 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:53:53.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:53 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/2183658428' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T10:53:53.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:53 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:53.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:53 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:54.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:54 smithi094 ceph-mon[93957]: pgmap v196: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-03-24T10:53:54.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:54 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:54.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:54 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:54.758 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:54 smithi096 ceph-mon[84258]: pgmap v196: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-03-24T10:53:54.758 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:54 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:54.758 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:54 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:56.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:56 smithi094 ceph-mon[93957]: pgmap v197: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 6.1 MiB/s rd, 4.3 KiB/s wr, 6 op/s 2024-03-24T10:53:56.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:56 smithi096 ceph-mon[84258]: pgmap v197: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 6.1 MiB/s rd, 4.3 KiB/s wr, 6 op/s 2024-03-24T10:53:58.739 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:58 smithi096 ceph-mon[84258]: pgmap v198: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 2.7 MiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-03-24T10:53:58.739 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:58 smithi096 ceph-mon[84258]: Detected new or changed devices on smithi096 2024-03-24T10:53:58.739 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:58.739 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:58.739 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:53:58.739 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:53:58.739 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:58.739 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:53:58.739 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:53:58.739 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:53:58.739 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:53:58.739 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:53:58.740 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi096.jtebhh"]}]: dispatch 2024-03-24T10:53:58.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:58 smithi094 ceph-mon[93957]: pgmap v198: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 2.7 MiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-03-24T10:53:58.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:58 smithi094 ceph-mon[93957]: Detected new or changed devices on smithi096 2024-03-24T10:53:58.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:58.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:58.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:53:58.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:53:58.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:58.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:53:58.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:53:58.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:53:58.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:53:58.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:53:58.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi096.jtebhh"]}]: dispatch 2024-03-24T10:53:59.623 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:59 smithi096 ceph-mon[84258]: Upgrade: It appears safe to stop mds.cephfs.smithi096.jtebhh 2024-03-24T10:53:59.624 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:59 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:59.624 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:59 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi096.jtebhh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T10:53:59.624 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:53:59 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:53:59.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:59 smithi094 ceph-mon[93957]: Upgrade: It appears safe to stop mds.cephfs.smithi096.jtebhh 2024-03-24T10:53:59.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:59 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:53:59.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:59 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi096.jtebhh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-24T10:53:59.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:53:59 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:54:00.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:00 smithi096 ceph-mon[84258]: Upgrade: Updating mds.cephfs.smithi096.jtebhh 2024-03-24T10:54:00.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:00 smithi096 ceph-mon[84258]: Deploying daemon mds.cephfs.smithi096.jtebhh on smithi096 2024-03-24T10:54:00.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:00 smithi096 ceph-mon[84258]: pgmap v199: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 2.1 MiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-03-24T10:54:00.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:00 smithi094 ceph-mon[93957]: Upgrade: Updating mds.cephfs.smithi096.jtebhh 2024-03-24T10:54:00.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:00 smithi094 ceph-mon[93957]: Deploying daemon mds.cephfs.smithi096.jtebhh on smithi096 2024-03-24T10:54:00.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:00 smithi094 ceph-mon[93957]: pgmap v199: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 2.1 MiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-03-24T10:54:01.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:01 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-mon-smithi094[93953]: 2024-03-24T10:54:01.168+0000 7f0fa10ce700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-24T10:54:01.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:01 smithi094 ceph-mon[93957]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-24T10:54:01.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:01 smithi094 ceph-mon[93957]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-24T10:54:01.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:01 smithi094 ceph-mon[93957]: osdmap e84: 6 total, 6 up, 6 in 2024-03-24T10:54:01.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:01 smithi094 ceph-mon[93957]: Standby daemon mds.cephfs.smithi094.efiqte assigned to filesystem cephfs as rank 0 2024-03-24T10:54:01.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:01 smithi094 ceph-mon[93957]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-24T10:54:01.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:01 smithi094 ceph-mon[93957]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-03-24T10:54:01.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:01 smithi094 ceph-mon[93957]: fsmap cephfs:1/1 {0=cephfs.smithi094.efiqte=up:replay} 2 up:standby 2024-03-24T10:54:01.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:01 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi096.jtebhh"}]: dispatch 2024-03-24T10:54:01.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:01 smithi096 ceph-mon[84258]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-24T10:54:01.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:01 smithi096 ceph-mon[84258]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-24T10:54:01.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:01 smithi096 ceph-mon[84258]: osdmap e84: 6 total, 6 up, 6 in 2024-03-24T10:54:01.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:01 smithi096 ceph-mon[84258]: Standby daemon mds.cephfs.smithi094.efiqte assigned to filesystem cephfs as rank 0 2024-03-24T10:54:01.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:01 smithi096 ceph-mon[84258]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-24T10:54:01.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:01 smithi096 ceph-mon[84258]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-03-24T10:54:01.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:01 smithi096 ceph-mon[84258]: fsmap cephfs:1/1 {0=cephfs.smithi094.efiqte=up:replay} 2 up:standby 2024-03-24T10:54:01.577 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:01 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi096.jtebhh"}]: dispatch 2024-03-24T10:54:01.577 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:54:01 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3[93576]: 2024-03-24T10:54:01.191+0000 7f1fd0afd700 -1 bdev(0x564294b52000 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 10 2024-03-24T10:54:01.577 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:54:01 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3[93576]: 2024-03-24T10:54:01.192+0000 7f1fd4304700 -1 bdev(0x564294b52000 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 17 2024-03-24T10:54:01.577 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:54:01 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3[93576]: 2024-03-24T10:54:01.193+0000 7f1fd6308700 -1 bdev(0x564294b52000 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 21 2024-03-24T10:54:02.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:02 smithi096 ceph-mon[84258]: pgmap v201: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 13 KiB/s rd, 5.1 KiB/s wr, 6 op/s 2024-03-24T10:54:02.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:02 smithi094 ceph-mon[93957]: pgmap v201: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 13 KiB/s rd, 5.1 KiB/s wr, 6 op/s 2024-03-24T10:54:04.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:04 smithi094 ceph-mon[93957]: pgmap v202: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 4.4 MiB/s rd, 0 B/s wr, 3 op/s 2024-03-24T10:54:04.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:04 smithi096 ceph-mon[84258]: pgmap v202: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 4.4 MiB/s rd, 0 B/s wr, 3 op/s 2024-03-24T10:54:06.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:06 smithi094 ceph-mon[93957]: pgmap v203: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 8.0 MiB/s rd, 0 B/s wr, 3 op/s 2024-03-24T10:54:06.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:06 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:06.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:06 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:54:06.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:06 smithi096 ceph-mon[84258]: pgmap v203: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 8.0 MiB/s rd, 0 B/s wr, 3 op/s 2024-03-24T10:54:06.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:06 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:06.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:06 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:54:08.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:08 smithi094 ceph-mon[93957]: pgmap v204: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 0 B/s wr, 4 op/s 2024-03-24T10:54:08.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:08 smithi096 ceph-mon[84258]: pgmap v204: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 0 B/s wr, 4 op/s 2024-03-24T10:54:10.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:10 smithi096 ceph-mon[84258]: pgmap v205: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 19 MiB/s rd, 6 op/s 2024-03-24T10:54:10.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:10 smithi096 ceph-mon[84258]: reconnect by client.14550 v1:192.168.0.1:0/449160647 after 0 2024-03-24T10:54:10.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:10 smithi096 ceph-mon[84258]: reconnect by client.14556 v1:172.21.15.96:0/2944380590 after 0.000999983 2024-03-24T10:54:10.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:10 smithi096 ceph-mon[84258]: mds.? [v2:172.21.15.94:6826/2043632450,v1:172.21.15.94:6827/2043632450] up:reconnect 2024-03-24T10:54:10.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:10 smithi096 ceph-mon[84258]: fsmap cephfs:1/1 {0=cephfs.smithi094.efiqte=up:reconnect} 2 up:standby 2024-03-24T10:54:10.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:10 smithi094 ceph-mon[93957]: pgmap v205: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 19 MiB/s rd, 6 op/s 2024-03-24T10:54:10.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:10 smithi094 ceph-mon[93957]: reconnect by client.14550 v1:192.168.0.1:0/449160647 after 0 2024-03-24T10:54:10.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:10 smithi094 ceph-mon[93957]: reconnect by client.14556 v1:172.21.15.96:0/2944380590 after 0.000999983 2024-03-24T10:54:10.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:10 smithi094 ceph-mon[93957]: mds.? [v2:172.21.15.94:6826/2043632450,v1:172.21.15.94:6827/2043632450] up:reconnect 2024-03-24T10:54:10.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:10 smithi094 ceph-mon[93957]: fsmap cephfs:1/1 {0=cephfs.smithi094.efiqte=up:reconnect} 2 up:standby 2024-03-24T10:54:11.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:11 smithi094 ceph-mon[93957]: mds.? [v2:172.21.15.94:6826/2043632450,v1:172.21.15.94:6827/2043632450] up:rejoin 2024-03-24T10:54:11.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:11 smithi094 ceph-mon[93957]: fsmap cephfs:1/1 {0=cephfs.smithi094.efiqte=up:rejoin} 2 up:standby 2024-03-24T10:54:11.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:11 smithi094 ceph-mon[93957]: daemon mds.cephfs.smithi094.efiqte is now active in filesystem cephfs as rank 0 2024-03-24T10:54:11.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:11 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:11.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:11 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:11.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:11 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:54:11.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:11 smithi096 ceph-mon[84258]: mds.? [v2:172.21.15.94:6826/2043632450,v1:172.21.15.94:6827/2043632450] up:rejoin 2024-03-24T10:54:11.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:11 smithi096 ceph-mon[84258]: fsmap cephfs:1/1 {0=cephfs.smithi094.efiqte=up:rejoin} 2 up:standby 2024-03-24T10:54:11.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:11 smithi096 ceph-mon[84258]: daemon mds.cephfs.smithi094.efiqte is now active in filesystem cephfs as rank 0 2024-03-24T10:54:11.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:11 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:11.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:11 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:11.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:11 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:54:12.736 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:12 smithi096 ceph-mon[84258]: pgmap v206: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 19 MiB/s rd, 6 op/s 2024-03-24T10:54:12.736 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:12 smithi096 ceph-mon[84258]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-24T10:54:12.736 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:12 smithi096 ceph-mon[84258]: Cluster is now healthy 2024-03-24T10:54:12.736 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:12 smithi096 ceph-mon[84258]: mds.? [v2:172.21.15.94:6826/2043632450,v1:172.21.15.94:6827/2043632450] up:active 2024-03-24T10:54:12.736 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:12 smithi096 ceph-mon[84258]: mds.? [v2:172.21.15.96:6826/2275611388,v1:172.21.15.96:6827/2275611388] up:boot 2024-03-24T10:54:12.736 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:12 smithi096 ceph-mon[84258]: fsmap cephfs:1 {0=cephfs.smithi094.efiqte=up:active} 3 up:standby 2024-03-24T10:54:12.737 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:12 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi096.jtebhh"}]: dispatch 2024-03-24T10:54:12.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:12 smithi094 ceph-mon[93957]: pgmap v206: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 19 MiB/s rd, 6 op/s 2024-03-24T10:54:12.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:12 smithi094 ceph-mon[93957]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-24T10:54:12.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:12 smithi094 ceph-mon[93957]: Cluster is now healthy 2024-03-24T10:54:12.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:12 smithi094 ceph-mon[93957]: mds.? [v2:172.21.15.94:6826/2043632450,v1:172.21.15.94:6827/2043632450] up:active 2024-03-24T10:54:12.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:12 smithi094 ceph-mon[93957]: mds.? [v2:172.21.15.96:6826/2275611388,v1:172.21.15.96:6827/2275611388] up:boot 2024-03-24T10:54:12.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:12 smithi094 ceph-mon[93957]: fsmap cephfs:1 {0=cephfs.smithi094.efiqte=up:active} 3 up:standby 2024-03-24T10:54:12.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:12 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi096.jtebhh"}]: dispatch 2024-03-24T10:54:14.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:14 smithi094 ceph-mon[93957]: pgmap v207: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 85 B/s wr, 7 op/s 2024-03-24T10:54:14.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:14 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:14.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:14 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:14.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:14 smithi096 ceph-mon[84258]: pgmap v207: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 85 B/s wr, 7 op/s 2024-03-24T10:54:14.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:14 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:14.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:14 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:15.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:15 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:15.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:15 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:15.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:15 smithi096 ceph-mon[84258]: pgmap v208: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 12 MiB/s rd, 170 B/s wr, 6 op/s 2024-03-24T10:54:15.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:15 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:15.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:15 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:15.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:15 smithi094 ceph-mon[93957]: pgmap v208: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 12 MiB/s rd, 170 B/s wr, 6 op/s 2024-03-24T10:54:18.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:18 smithi096 ceph-mon[84258]: pgmap v209: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 9.4 MiB/s rd, 170 B/s wr, 5 op/s 2024-03-24T10:54:18.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:18 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:18.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:18 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:18.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:18 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:54:18.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:18 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:54:18.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:18 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:18.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:18 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:54:18.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:18 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:18.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:18 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:18.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:18 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:18.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:18 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:18.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:18 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:18.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:18 smithi094 ceph-mon[93957]: pgmap v209: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 9.4 MiB/s rd, 170 B/s wr, 5 op/s 2024-03-24T10:54:18.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:18 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:18.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:18 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:18.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:18 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:54:18.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:18 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:54:18.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:18 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:18.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:18 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:54:18.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:18 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:18.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:18 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:18.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:18 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:18.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:18 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:18.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:18 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:19.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:19 smithi094 ceph-mon[93957]: Upgrade: Setting container_image for all mds 2024-03-24T10:54:19.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:19 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:19.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:19 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi094.efiqte"}]: dispatch 2024-03-24T10:54:19.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:19 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi094.efiqte"}]': finished 2024-03-24T10:54:19.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:19 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi094.llefgu"}]: dispatch 2024-03-24T10:54:19.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:19 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi094.llefgu"}]': finished 2024-03-24T10:54:19.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:19 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi096.egbmoq"}]: dispatch 2024-03-24T10:54:19.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:19 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi096.egbmoq"}]': finished 2024-03-24T10:54:19.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:19 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi096.jtebhh"}]: dispatch 2024-03-24T10:54:19.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:19 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi096.jtebhh"}]': finished 2024-03-24T10:54:19.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:19 smithi094 ceph-mon[93957]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-03-24T10:54:19.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:19 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-03-24T10:54:19.825 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:54:19 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3[93576]: 2024-03-24T10:54:19.410+0000 7f1fd3302700 -1 bdev(0x564294b52000 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 4 2024-03-24T10:54:19.825 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:54:19 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3[93576]: 2024-03-24T10:54:19.413+0000 7f1fd4304700 -1 bdev(0x564294b52000 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 10 2024-03-24T10:54:19.826 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:54:19 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3[93576]: 2024-03-24T10:54:19.414+0000 7f1fd2300700 -1 bdev(0x564294b52000 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 19 2024-03-24T10:54:19.826 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:54:19 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3[93576]: 2024-03-24T10:54:19.414+0000 7f1fd4b05700 -1 bdev(0x564294b52000 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 18 2024-03-24T10:54:19.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:19 smithi096 ceph-mon[84258]: Upgrade: Setting container_image for all mds 2024-03-24T10:54:19.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:19 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:19.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:19 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi094.efiqte"}]: dispatch 2024-03-24T10:54:19.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:19 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi094.efiqte"}]': finished 2024-03-24T10:54:19.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:19 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi094.llefgu"}]: dispatch 2024-03-24T10:54:19.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:19 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi094.llefgu"}]': finished 2024-03-24T10:54:19.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:19 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi096.egbmoq"}]: dispatch 2024-03-24T10:54:19.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:19 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi096.egbmoq"}]': finished 2024-03-24T10:54:19.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:19 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi096.jtebhh"}]: dispatch 2024-03-24T10:54:19.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:19 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi096.jtebhh"}]': finished 2024-03-24T10:54:19.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:19 smithi096 ceph-mon[84258]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-03-24T10:54:19.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:19 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-03-24T10:54:20.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:20 smithi094 ceph-mon[93957]: pgmap v210: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 7.1 MiB/s rd, 4.4 KiB/s wr, 6 op/s 2024-03-24T10:54:20.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:20 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-03-24T10:54:20.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:20 smithi094 ceph-mon[93957]: fsmap cephfs:1 {0=cephfs.smithi094.efiqte=up:active} 3 up:standby 2024-03-24T10:54:20.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:20 smithi094 ceph-mon[93957]: fsmap cephfs:1 {0=cephfs.smithi094.efiqte=up:active} 1 up:standby-replay 2 up:standby 2024-03-24T10:54:20.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:20 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:20.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:20 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:20.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:20 smithi094 ceph-mon[93957]: Upgrade: Setting container_image for all rgw 2024-03-24T10:54:20.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:20 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:20.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:20 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:20.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:20 smithi094 ceph-mon[93957]: Upgrade: Setting container_image for all rbd-mirror 2024-03-24T10:54:20.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:20 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:20.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:20 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:20.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:20 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:20.500 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:20 smithi094 ceph-mon[93957]: Upgrade: Setting container_image for all ceph-exporter 2024-03-24T10:54:20.500 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:20 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:20.500 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:20 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:20.500 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:20 smithi094 ceph-mon[93957]: Upgrade: Setting container_image for all iscsi 2024-03-24T10:54:20.500 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:20 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:20.500 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:20 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:20.500 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:20 smithi094 ceph-mon[93957]: Upgrade: Setting container_image for all nfs 2024-03-24T10:54:20.500 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:20 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:20.500 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:20 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:20.500 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:20 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:20.500 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:20 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:20.500 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:20 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:54:20.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:20 smithi096 ceph-mon[84258]: pgmap v210: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 7.1 MiB/s rd, 4.4 KiB/s wr, 6 op/s 2024-03-24T10:54:20.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:20 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-03-24T10:54:20.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:20 smithi096 ceph-mon[84258]: fsmap cephfs:1 {0=cephfs.smithi094.efiqte=up:active} 3 up:standby 2024-03-24T10:54:20.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:20 smithi096 ceph-mon[84258]: fsmap cephfs:1 {0=cephfs.smithi094.efiqte=up:active} 1 up:standby-replay 2 up:standby 2024-03-24T10:54:20.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:20 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:20.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:20 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:20.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:20 smithi096 ceph-mon[84258]: Upgrade: Setting container_image for all rgw 2024-03-24T10:54:20.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:20 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:20.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:20 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:20.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:20 smithi096 ceph-mon[84258]: Upgrade: Setting container_image for all rbd-mirror 2024-03-24T10:54:20.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:20 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:20.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:20 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:20.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:20 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:20.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:20 smithi096 ceph-mon[84258]: Upgrade: Setting container_image for all ceph-exporter 2024-03-24T10:54:20.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:20 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:20.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:20 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:20.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:20 smithi096 ceph-mon[84258]: Upgrade: Setting container_image for all iscsi 2024-03-24T10:54:20.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:20 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:20.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:20 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:20.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:20 smithi096 ceph-mon[84258]: Upgrade: Setting container_image for all nfs 2024-03-24T10:54:20.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:20 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:20.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:20 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:20.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:20 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:20.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:20 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:20.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:20 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:54:21.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:21 smithi094 ceph-mon[93957]: Upgrade: Setting container_image for all nvmeof 2024-03-24T10:54:21.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:21 smithi096 ceph-mon[84258]: Upgrade: Setting container_image for all nvmeof 2024-03-24T10:54:22.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:22 smithi094 ceph-mon[93957]: Upgrade: Updating node-exporter.smithi094 (1/2) 2024-03-24T10:54:22.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:22 smithi094 ceph-mon[93957]: Deploying daemon node-exporter.smithi094 on smithi094 2024-03-24T10:54:22.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:22 smithi094 ceph-mon[93957]: pgmap v211: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 352 KiB/s rd, 4.4 KiB/s wr, 5 op/s 2024-03-24T10:54:22.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:22.804+0000 7f9399e64700 1 -- 172.21.15.94:0/300163916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93940662e0 msgr2=0x7f93940ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:22.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:22.804+0000 7f9399e64700 1 --2- 172.21.15.94:0/300163916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93940662e0 0x7f93940ff9c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9384004940 tx=0x7f9384009e30 comp rx=0 tx=0).stop 2024-03-24T10:54:22.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:22.804+0000 7f9399e64700 1 -- 172.21.15.94:0/300163916 shutdown_connections 2024-03-24T10:54:22.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:22.804+0000 7f9399e64700 1 --2- 172.21.15.94:0/300163916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93940fff00 0x7f9394102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:22.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:22.804+0000 7f9399e64700 1 --2- 172.21.15.94:0/300163916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93940662e0 0x7f93940ff9c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:22.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:22.804+0000 7f9399e64700 1 -- 172.21.15.94:0/300163916 >> 172.21.15.94:0/300163916 conn(0x7f93940f7430 msgr2=0x7f93940f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:54:22.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:22.804+0000 7f9399e64700 1 -- 172.21.15.94:0/300163916 shutdown_connections 2024-03-24T10:54:22.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:22.804+0000 7f9399e64700 1 -- 172.21.15.94:0/300163916 wait complete. 2024-03-24T10:54:22.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:22.804+0000 7f9399e64700 1 Processor -- start 2024-03-24T10:54:22.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:22.805+0000 7f9399e64700 1 -- start start 2024-03-24T10:54:22.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:22.805+0000 7f9399e64700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93940662e0 0x7f93940fd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:54:22.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:22.805+0000 7f9399e64700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93940fff00 0x7f93940fb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:54:22.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:22.805+0000 7f9399e64700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93940fbe50 con 0x7f93940fff00 2024-03-24T10:54:22.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:22.805+0000 7f9399e64700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93940fbf90 con 0x7f93940662e0 2024-03-24T10:54:22.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:22.805+0000 7f9392ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93940fff00 0x7f93940fb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:54:22.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:22.805+0000 7f93937fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93940662e0 0x7f93940fd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:54:22.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:22.805+0000 7f9392ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93940fff00 0x7f93940fb8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45836/0 (socket says 172.21.15.94:45836) 2024-03-24T10:54:22.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:22.805+0000 7f9392ffd700 1 -- 172.21.15.94:0/1369307959 learned_addr learned my addr 172.21.15.94:0/1369307959 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:54:22.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:22.805+0000 7f93937fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93940662e0 0x7f93940fd3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.94:51142/0 (socket says 172.21.15.94:51142) 2024-03-24T10:54:22.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:22.806+0000 7f9392ffd700 1 -- 172.21.15.94:0/1369307959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93940662e0 msgr2=0x7f93940fd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:22.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:22.806+0000 7f9392ffd700 1 --2- 172.21.15.94:0/1369307959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93940662e0 0x7f93940fd3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:22.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:22.806+0000 7f9392ffd700 1 -- 172.21.15.94:0/1369307959 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9384005040 con 0x7f93940fff00 2024-03-24T10:54:22.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:22.806+0000 7f93937fe700 1 --2- 172.21.15.94:0/1369307959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93940662e0 0x7f93940fd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T10:54:22.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:22.806+0000 7f9392ffd700 1 --2- 172.21.15.94:0/1369307959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93940fff00 0x7f93940fb8c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f938800eb70 tx=0x7f938800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:54:22.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:22.806+0000 7f937bfff700 1 -- 172.21.15.94:0/1369307959 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9388007cc0 con 0x7f93940fff00 2024-03-24T10:54:22.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:22.806+0000 7f937bfff700 1 -- 172.21.15.94:0/1369307959 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f938800f040 con 0x7f93940fff00 2024-03-24T10:54:22.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:22.806+0000 7f9399e64700 1 -- 172.21.15.94:0/1369307959 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93940fc210 con 0x7f93940fff00 2024-03-24T10:54:22.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:22.806+0000 7f937bfff700 1 -- 172.21.15.94:0/1369307959 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f938800ca90 con 0x7f93940fff00 2024-03-24T10:54:22.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:22.806+0000 7f9399e64700 1 -- 172.21.15.94:0/1369307959 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9394115a50 con 0x7f93940fff00 2024-03-24T10:54:22.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:22.807+0000 7f9399e64700 1 -- 172.21.15.94:0/1369307959 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f939405f5f0 con 0x7f93940fff00 2024-03-24T10:54:22.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:22.808+0000 7f937bfff700 1 -- 172.21.15.94:0/1369307959 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f938800cbf0 con 0x7f93940fff00 2024-03-24T10:54:22.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:22.808+0000 7f937bfff700 1 --2- 172.21.15.94:0/1369307959 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f937c06d7f0 0x7f937c06fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:54:22.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:22.809+0000 7f937bfff700 1 -- 172.21.15.94:0/1369307959 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f938808cad0 con 0x7f93940fff00 2024-03-24T10:54:22.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:22.809+0000 7f93937fe700 1 --2- 172.21.15.94:0/1369307959 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f937c06d7f0 0x7f937c06fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:54:22.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:22.809+0000 7f93937fe700 1 --2- 172.21.15.94:0/1369307959 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f937c06d7f0 0x7f937c06fca0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9384013740 tx=0x7f9384009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:54:22.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:22.814+0000 7f937bfff700 1 -- 172.21.15.94:0/1369307959 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f93880562a0 con 0x7f93940fff00 2024-03-24T10:54:22.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:22 smithi096 ceph-mon[84258]: Upgrade: Updating node-exporter.smithi094 (1/2) 2024-03-24T10:54:22.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:22 smithi096 ceph-mon[84258]: Deploying daemon node-exporter.smithi094 on smithi094 2024-03-24T10:54:22.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:22 smithi096 ceph-mon[84258]: pgmap v211: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 352 KiB/s rd, 4.4 KiB/s wr, 5 op/s 2024-03-24T10:54:23.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.103+0000 7f9399e64700 1 -- 172.21.15.94:0/1369307959 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f93940f8f70 con 0x7f937c06d7f0 2024-03-24T10:54:23.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.104+0000 7f937bfff700 1 -- 172.21.15.94:0/1369307959 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7f93940f8f70 con 0x7f937c06d7f0 2024-03-24T10:54:23.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.108+0000 7f9399e64700 1 -- 172.21.15.94:0/1369307959 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f937c06d7f0 msgr2=0x7f937c06fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:23.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.108+0000 7f9399e64700 1 --2- 172.21.15.94:0/1369307959 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f937c06d7f0 0x7f937c06fca0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9384013740 tx=0x7f9384009e30 comp rx=0 tx=0).stop 2024-03-24T10:54:23.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.108+0000 7f9399e64700 1 -- 172.21.15.94:0/1369307959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93940fff00 msgr2=0x7f93940fb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:23.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.108+0000 7f9399e64700 1 --2- 172.21.15.94:0/1369307959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93940fff00 0x7f93940fb8c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f938800eb70 tx=0x7f938800b040 comp rx=0 tx=0).stop 2024-03-24T10:54:23.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.108+0000 7f9399e64700 1 -- 172.21.15.94:0/1369307959 shutdown_connections 2024-03-24T10:54:23.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.108+0000 7f9399e64700 1 --2- 172.21.15.94:0/1369307959 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f937c06d7f0 0x7f937c06fca0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:23.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.108+0000 7f9399e64700 1 --2- 172.21.15.94:0/1369307959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93940662e0 0x7f93940fd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:23.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.108+0000 7f9399e64700 1 --2- 172.21.15.94:0/1369307959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93940fff00 0x7f93940fb8c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:23.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.108+0000 7f9399e64700 1 -- 172.21.15.94:0/1369307959 >> 172.21.15.94:0/1369307959 conn(0x7f93940f7430 msgr2=0x7f93940f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:54:23.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.108+0000 7f9399e64700 1 -- 172.21.15.94:0/1369307959 shutdown_connections 2024-03-24T10:54:23.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.109+0000 7f9399e64700 1 -- 172.21.15.94:0/1369307959 wait complete. 2024-03-24T10:54:23.121 INFO:teuthology.orchestra.run.smithi094.stdout:true 2024-03-24T10:54:23.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.217+0000 7fe59e7b9700 1 -- 172.21.15.94:0/1354006916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe598066e50 msgr2=0x7fe5980ffa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:23.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.217+0000 7fe59e7b9700 1 --2- 172.21.15.94:0/1354006916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe598066e50 0x7fe5980ffa60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe588004940 tx=0x7fe588009e30 comp rx=0 tx=0).stop 2024-03-24T10:54:23.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.217+0000 7fe59e7b9700 1 -- 172.21.15.94:0/1354006916 shutdown_connections 2024-03-24T10:54:23.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.217+0000 7fe59e7b9700 1 --2- 172.21.15.94:0/1354006916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5980fffa0 0x7fe598102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:23.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.217+0000 7fe59e7b9700 1 --2- 172.21.15.94:0/1354006916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe598066e50 0x7fe5980ffa60 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:23.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.217+0000 7fe59e7b9700 1 -- 172.21.15.94:0/1354006916 >> 172.21.15.94:0/1354006916 conn(0x7fe5980f7430 msgr2=0x7fe5980f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:54:23.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.217+0000 7fe59e7b9700 1 -- 172.21.15.94:0/1354006916 shutdown_connections 2024-03-24T10:54:23.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.217+0000 7fe59e7b9700 1 -- 172.21.15.94:0/1354006916 wait complete. 2024-03-24T10:54:23.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.217+0000 7fe59e7b9700 1 Processor -- start 2024-03-24T10:54:23.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.218+0000 7fe59e7b9700 1 -- start start 2024-03-24T10:54:23.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.218+0000 7fe59e7b9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe598066e50 0x7fe59810d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:54:23.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.218+0000 7fe59e7b9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5980fffa0 0x7fe59810dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:54:23.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.218+0000 7fe59e7b9700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59810e4a0 con 0x7fe598066e50 2024-03-24T10:54:23.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.218+0000 7fe59e7b9700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59810e610 con 0x7fe5980fffa0 2024-03-24T10:54:23.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.218+0000 7fe597fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe598066e50 0x7fe59810d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:54:23.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.218+0000 7fe597fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe598066e50 0x7fe59810d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45858/0 (socket says 172.21.15.94:45858) 2024-03-24T10:54:23.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.218+0000 7fe597fff700 1 -- 172.21.15.94:0/2672615977 learned_addr learned my addr 172.21.15.94:0/2672615977 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:54:23.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.218+0000 7fe5977fe700 1 --2- 172.21.15.94:0/2672615977 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5980fffa0 0x7fe59810dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:54:23.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.218+0000 7fe597fff700 1 -- 172.21.15.94:0/2672615977 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5980fffa0 msgr2=0x7fe59810dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:23.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.218+0000 7fe597fff700 1 --2- 172.21.15.94:0/2672615977 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5980fffa0 0x7fe59810dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:23.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.218+0000 7fe597fff700 1 -- 172.21.15.94:0/2672615977 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe588005040 con 0x7fe598066e50 2024-03-24T10:54:23.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.218+0000 7fe5977fe700 1 --2- 172.21.15.94:0/2672615977 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5980fffa0 0x7fe59810dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T10:54:23.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.219+0000 7fe597fff700 1 --2- 172.21.15.94:0/2672615977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe598066e50 0x7fe59810d960 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe58800b3b0 tx=0x7fe588009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:54:23.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.219+0000 7fe594ff9700 1 -- 172.21.15.94:0/2672615977 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe58800e040 con 0x7fe598066e50 2024-03-24T10:54:23.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.219+0000 7fe594ff9700 1 -- 172.21.15.94:0/2672615977 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe58800bcc0 con 0x7fe598066e50 2024-03-24T10:54:23.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.219+0000 7fe59e7b9700 1 -- 172.21.15.94:0/2672615977 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe598113010 con 0x7fe598066e50 2024-03-24T10:54:23.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.219+0000 7fe594ff9700 1 -- 172.21.15.94:0/2672615977 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe58801c630 con 0x7fe598066e50 2024-03-24T10:54:23.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.219+0000 7fe59e7b9700 1 -- 172.21.15.94:0/2672615977 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5980fb8c0 con 0x7fe598066e50 2024-03-24T10:54:23.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.220+0000 7fe59e7b9700 1 -- 172.21.15.94:0/2672615977 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe59804c710 con 0x7fe598066e50 2024-03-24T10:54:23.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.221+0000 7fe594ff9700 1 -- 172.21.15.94:0/2672615977 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fe5880139f0 con 0x7fe598066e50 2024-03-24T10:54:23.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.222+0000 7fe594ff9700 1 --2- 172.21.15.94:0/2672615977 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fe580071bf0 0x7fe5800740a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:54:23.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.222+0000 7fe594ff9700 1 -- 172.21.15.94:0/2672615977 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fe58808dc90 con 0x7fe598066e50 2024-03-24T10:54:23.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.223+0000 7fe5977fe700 1 --2- 172.21.15.94:0/2672615977 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fe580071bf0 0x7fe5800740a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:54:23.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.225+0000 7fe5977fe700 1 --2- 172.21.15.94:0/2672615977 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fe580071bf0 0x7fe5800740a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe58c010520 tx=0x7fe58c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:54:23.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.231+0000 7fe594ff9700 1 -- 172.21.15.94:0/2672615977 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fe58805af10 con 0x7fe598066e50 2024-03-24T10:54:23.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.486+0000 7fe59e7b9700 1 -- 172.21.15.94:0/2672615977 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe5980f8f70 con 0x7fe580071bf0 2024-03-24T10:54:23.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.487+0000 7fe594ff9700 1 -- 172.21.15.94:0/2672615977 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7fe5980f8f70 con 0x7fe580071bf0 2024-03-24T10:54:23.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.489+0000 7fe59e7b9700 1 -- 172.21.15.94:0/2672615977 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fe580071bf0 msgr2=0x7fe5800740a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:23.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.489+0000 7fe59e7b9700 1 --2- 172.21.15.94:0/2672615977 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fe580071bf0 0x7fe5800740a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe58c010520 tx=0x7fe58c00b040 comp rx=0 tx=0).stop 2024-03-24T10:54:23.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.489+0000 7fe59e7b9700 1 -- 172.21.15.94:0/2672615977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe598066e50 msgr2=0x7fe59810d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:23.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.489+0000 7fe59e7b9700 1 --2- 172.21.15.94:0/2672615977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe598066e50 0x7fe59810d960 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe58800b3b0 tx=0x7fe588009e30 comp rx=0 tx=0).stop 2024-03-24T10:54:23.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.489+0000 7fe59e7b9700 1 -- 172.21.15.94:0/2672615977 shutdown_connections 2024-03-24T10:54:23.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.489+0000 7fe59e7b9700 1 --2- 172.21.15.94:0/2672615977 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fe580071bf0 0x7fe5800740a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:23.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.489+0000 7fe59e7b9700 1 --2- 172.21.15.94:0/2672615977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe598066e50 0x7fe59810d960 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:23.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.489+0000 7fe59e7b9700 1 --2- 172.21.15.94:0/2672615977 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5980fffa0 0x7fe59810dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:23.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.489+0000 7fe59e7b9700 1 -- 172.21.15.94:0/2672615977 >> 172.21.15.94:0/2672615977 conn(0x7fe5980f7430 msgr2=0x7fe5980f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:54:23.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.490+0000 7fe59e7b9700 1 -- 172.21.15.94:0/2672615977 shutdown_connections 2024-03-24T10:54:23.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.490+0000 7fe59e7b9700 1 -- 172.21.15.94:0/2672615977 wait complete. 2024-03-24T10:54:23.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.648+0000 7f9e88fa7700 1 -- 172.21.15.94:0/2606210055 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e840fbcd0 msgr2=0x7f9e840fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:23.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.648+0000 7f9e88fa7700 1 --2- 172.21.15.94:0/2606210055 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e840fbcd0 0x7f9e840fc0e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9e74004940 tx=0x7f9e74009e30 comp rx=0 tx=0).stop 2024-03-24T10:54:23.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.648+0000 7f9e88fa7700 1 -- 172.21.15.94:0/2606210055 shutdown_connections 2024-03-24T10:54:23.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.648+0000 7f9e88fa7700 1 --2- 172.21.15.94:0/2606210055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e840fcf80 0x7f9e840fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:23.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.648+0000 7f9e88fa7700 1 --2- 172.21.15.94:0/2606210055 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e840fbcd0 0x7f9e840fc0e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:23.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.648+0000 7f9e88fa7700 1 -- 172.21.15.94:0/2606210055 >> 172.21.15.94:0/2606210055 conn(0x7f9e840f7430 msgr2=0x7f9e840f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:54:23.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.648+0000 7f9e88fa7700 1 -- 172.21.15.94:0/2606210055 shutdown_connections 2024-03-24T10:54:23.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.648+0000 7f9e88fa7700 1 -- 172.21.15.94:0/2606210055 wait complete. 2024-03-24T10:54:23.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.648+0000 7f9e88fa7700 1 Processor -- start 2024-03-24T10:54:23.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.649+0000 7f9e88fa7700 1 -- start start 2024-03-24T10:54:23.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.649+0000 7f9e88fa7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e840fbcd0 0x7f9e84113f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:54:23.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.649+0000 7f9e88fa7700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e840fcf80 0x7f9e841144d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:54:23.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.649+0000 7f9e88fa7700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e84114ad0 con 0x7f9e840fcf80 2024-03-24T10:54:23.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.649+0000 7f9e88fa7700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e84114c40 con 0x7f9e840fbcd0 2024-03-24T10:54:23.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.649+0000 7f9e81d9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e840fcf80 0x7f9e841144d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:54:23.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.649+0000 7f9e81d9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e840fcf80 0x7f9e841144d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45880/0 (socket says 172.21.15.94:45880) 2024-03-24T10:54:23.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.649+0000 7f9e81d9b700 1 -- 172.21.15.94:0/4256592710 learned_addr learned my addr 172.21.15.94:0/4256592710 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:54:23.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.649+0000 7f9e8259c700 1 --2- 172.21.15.94:0/4256592710 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e840fbcd0 0x7f9e84113f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:54:23.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.650+0000 7f9e81d9b700 1 -- 172.21.15.94:0/4256592710 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e840fbcd0 msgr2=0x7f9e84113f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:23.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.650+0000 7f9e81d9b700 1 --2- 172.21.15.94:0/4256592710 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e840fbcd0 0x7f9e84113f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:23.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.650+0000 7f9e81d9b700 1 -- 172.21.15.94:0/4256592710 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e74005040 con 0x7f9e840fcf80 2024-03-24T10:54:23.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.650+0000 7f9e81d9b700 1 --2- 172.21.15.94:0/4256592710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e840fcf80 0x7f9e841144d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9e78002a50 tx=0x7f9e780046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:54:23.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.650+0000 7f9e72ffd700 1 -- 172.21.15.94:0/4256592710 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e78007cc0 con 0x7f9e840fcf80 2024-03-24T10:54:23.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.650+0000 7f9e88fa7700 1 -- 172.21.15.94:0/4256592710 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e841196a0 con 0x7f9e840fcf80 2024-03-24T10:54:23.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.650+0000 7f9e88fa7700 1 -- 172.21.15.94:0/4256592710 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e84119c60 con 0x7f9e840fcf80 2024-03-24T10:54:23.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.651+0000 7f9e72ffd700 1 -- 172.21.15.94:0/4256592710 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9e7800f040 con 0x7f9e840fcf80 2024-03-24T10:54:23.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.651+0000 7f9e72ffd700 1 -- 172.21.15.94:0/4256592710 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e7800da30 con 0x7f9e840fcf80 2024-03-24T10:54:23.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.652+0000 7f9e88fa7700 1 -- 172.21.15.94:0/4256592710 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e640052e0 con 0x7f9e840fcf80 2024-03-24T10:54:23.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.652+0000 7f9e72ffd700 1 -- 172.21.15.94:0/4256592710 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f9e7800dc70 con 0x7f9e840fcf80 2024-03-24T10:54:23.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.653+0000 7f9e72ffd700 1 --2- 172.21.15.94:0/4256592710 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f9e6c06d7e0 0x7f9e6c06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:54:23.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.653+0000 7f9e72ffd700 1 -- 172.21.15.94:0/4256592710 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f9e7808cc30 con 0x7f9e840fcf80 2024-03-24T10:54:23.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.653+0000 7f9e8259c700 1 --2- 172.21.15.94:0/4256592710 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f9e6c06d7e0 0x7f9e6c06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:54:23.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.654+0000 7f9e8259c700 1 --2- 172.21.15.94:0/4256592710 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f9e6c06d7e0 0x7f9e6c06fc90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9e7400b970 tx=0x7f9e74009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:54:23.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.658+0000 7f9e72ffd700 1 -- 172.21.15.94:0/4256592710 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9e78059eb0 con 0x7f9e840fcf80 2024-03-24T10:54:23.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.960+0000 7f9e88fa7700 1 -- 172.21.15.94:0/4256592710 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9e64000bf0 con 0x7f9e6c06d7e0 2024-03-24T10:54:23.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.978+0000 7f9e72ffd700 1 -- 172.21.15.94:0/4256592710 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f9e64000bf0 con 0x7f9e6c06d7e0 2024-03-24T10:54:23.979 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T10:54:23.979 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.smithi094 smithi094 *:9093,9094 running (7m) 54s ago 16m 24.4M - 0.25.0 c8568f914cd2 5c583af60554 2024-03-24T10:54:23.979 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi094 smithi094 running (4m) 54s ago 16m 6651k - 19.0.0-1855-g21b3bd67 553463a90fd5 2b0897df571d 2024-03-24T10:54:23.979 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi096 smithi096 running (4m) 10s ago 15m 6669k - 19.0.0-1855-g21b3bd67 553463a90fd5 1759a9ff2805 2024-03-24T10:54:23.979 INFO:teuthology.orchestra.run.smithi094.stdout:grafana.smithi094 smithi094 *:3000 running (7m) 54s ago 15m 84.8M - 9.4.7 954c08fa6188 d3d425bf1b7c 2024-03-24T10:54:23.979 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.efiqte smithi094 running (75s) 54s ago 10m 15.1M - 19.0.0-1855-g21b3bd67 553463a90fd5 732c47b17971 2024-03-24T10:54:23.979 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.llefgu smithi094 running (59s) 54s ago 10m 14.8M - 19.0.0-1855-g21b3bd67 553463a90fd5 6e06f8a6dde0 2024-03-24T10:54:23.980 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi096.egbmoq smithi096 running (34s) 10s ago 10m 14.0M - 19.0.0-1855-g21b3bd67 553463a90fd5 f35fd7f23742 2024-03-24T10:54:23.980 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi096.jtebhh smithi096 running (13s) 10s ago 10m 13.8M - 19.0.0-1855-g21b3bd67 553463a90fd5 8b5ab9f96e5a 2024-03-24T10:54:23.980 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi094.yaethu smithi094 *:8443,9283,8765 running (6m) 54s ago 17m 506M - 19.0.0-1855-g21b3bd67 553463a90fd5 d3adab76b18f 2024-03-24T10:54:23.980 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi096.gfonnu smithi096 *:8443,9283,8765 running (6m) 10s ago 15m 450M - 19.0.0-1855-g21b3bd67 553463a90fd5 22937a059eb8 2024-03-24T10:54:23.980 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi094 smithi094 running (5m) 54s ago 18m 52.8M 2048M 19.0.0-1855-g21b3bd67 553463a90fd5 c779d611e29a 2024-03-24T10:54:23.980 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi096 smithi096 running (5m) 10s ago 14m 37.7M 2048M 19.0.0-1855-g21b3bd67 553463a90fd5 ed9bd0bd57f6 2024-03-24T10:54:23.980 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi094 smithi094 *:9100 running (7m) 54s ago 16m 18.6M - 1.5.0 0da6a335fe13 ca67b53c64e2 2024-03-24T10:54:23.980 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi096 smithi096 *:9100 running (7m) 10s ago 15m 17.0M - 1.5.0 0da6a335fe13 65137a7a7549 2024-03-24T10:54:23.980 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (4m) 54s ago 14m 103M 1557M 19.0.0-1855-g21b3bd67 553463a90fd5 ec57ede0c9e4 2024-03-24T10:54:23.980 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi094 running (3m) 54s ago 13m 111M 1557M 19.0.0-1855-g21b3bd67 553463a90fd5 c5b8b2bbf8b5 2024-03-24T10:54:23.980 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi094 running (3m) 54s ago 13m 92.4M 1557M 19.0.0-1855-g21b3bd67 553463a90fd5 c961dcdc746f 2024-03-24T10:54:23.980 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi096 running (2m) 10s ago 12m 119M 2581M 19.0.0-1855-g21b3bd67 553463a90fd5 6323fdc6e6ca 2024-03-24T10:54:23.980 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi096 running (2m) 10s ago 12m 110M 2581M 19.0.0-1855-g21b3bd67 553463a90fd5 3e2536bcb263 2024-03-24T10:54:23.980 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi096 running (93s) 10s ago 12m 102M 2581M 19.0.0-1855-g21b3bd67 553463a90fd5 2f77f1fa20e2 2024-03-24T10:54:23.980 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.smithi094 smithi094 *:9095 running (6m) 54s ago 15m 60.7M - 2.43.0 a07b618ecd1d 6ce091a0c326 2024-03-24T10:54:23.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.981+0000 7f9e88fa7700 1 -- 172.21.15.94:0/4256592710 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f9e6c06d7e0 msgr2=0x7f9e6c06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:23.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.981+0000 7f9e88fa7700 1 --2- 172.21.15.94:0/4256592710 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f9e6c06d7e0 0x7f9e6c06fc90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9e7400b970 tx=0x7f9e74009e30 comp rx=0 tx=0).stop 2024-03-24T10:54:23.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.981+0000 7f9e88fa7700 1 -- 172.21.15.94:0/4256592710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e840fcf80 msgr2=0x7f9e841144d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:23.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.981+0000 7f9e88fa7700 1 --2- 172.21.15.94:0/4256592710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e840fcf80 0x7f9e841144d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9e78002a50 tx=0x7f9e780046b0 comp rx=0 tx=0).stop 2024-03-24T10:54:23.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.981+0000 7f9e88fa7700 1 -- 172.21.15.94:0/4256592710 shutdown_connections 2024-03-24T10:54:23.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.981+0000 7f9e88fa7700 1 --2- 172.21.15.94:0/4256592710 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f9e6c06d7e0 0x7f9e6c06fc90 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:23.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.982+0000 7f9e88fa7700 1 --2- 172.21.15.94:0/4256592710 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e840fbcd0 0x7f9e84113f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:23.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.982+0000 7f9e88fa7700 1 --2- 172.21.15.94:0/4256592710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e840fcf80 0x7f9e841144d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:23.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.982+0000 7f9e88fa7700 1 -- 172.21.15.94:0/4256592710 >> 172.21.15.94:0/4256592710 conn(0x7f9e840f7430 msgr2=0x7f9e840668c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:54:23.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.982+0000 7f9e88fa7700 1 -- 172.21.15.94:0/4256592710 shutdown_connections 2024-03-24T10:54:23.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:23.982+0000 7f9e88fa7700 1 -- 172.21.15.94:0/4256592710 wait complete. 2024-03-24T10:54:24.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.090+0000 7f0aa78e6700 1 -- 172.21.15.94:0/663139622 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0aa00fbcd0 msgr2=0x7f0aa00fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:24.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.090+0000 7f0aa78e6700 1 --2- 172.21.15.94:0/663139622 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0aa00fbcd0 0x7f0aa00fc0e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c004940 tx=0x7f0a9c009e30 comp rx=0 tx=0).stop 2024-03-24T10:54:24.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.090+0000 7f0aa78e6700 1 -- 172.21.15.94:0/663139622 shutdown_connections 2024-03-24T10:54:24.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.090+0000 7f0aa78e6700 1 --2- 172.21.15.94:0/663139622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0aa00fcf80 0x7f0aa00fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:24.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.090+0000 7f0aa78e6700 1 --2- 172.21.15.94:0/663139622 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0aa00fbcd0 0x7f0aa00fc0e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:24.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.090+0000 7f0aa78e6700 1 -- 172.21.15.94:0/663139622 >> 172.21.15.94:0/663139622 conn(0x7f0aa00f7430 msgr2=0x7f0aa00f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:54:24.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.090+0000 7f0aa78e6700 1 -- 172.21.15.94:0/663139622 shutdown_connections 2024-03-24T10:54:24.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.090+0000 7f0aa78e6700 1 -- 172.21.15.94:0/663139622 wait complete. 2024-03-24T10:54:24.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.091+0000 7f0aa78e6700 1 Processor -- start 2024-03-24T10:54:24.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.091+0000 7f0aa78e6700 1 -- start start 2024-03-24T10:54:24.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.091+0000 7f0aa78e6700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0aa00fbcd0 0x7f0aa01070c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:54:24.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.091+0000 7f0aa78e6700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0aa00fcf80 0x7f0aa0107600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:54:24.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.091+0000 7f0aa78e6700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa0107c50 con 0x7f0aa00fbcd0 2024-03-24T10:54:24.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.091+0000 7f0aa78e6700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa01084a0 con 0x7f0aa00fcf80 2024-03-24T10:54:24.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.092+0000 7f0aa5682700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0aa00fbcd0 0x7f0aa01070c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:54:24.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.092+0000 7f0aa4e81700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0aa00fcf80 0x7f0aa0107600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:54:24.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.092+0000 7f0aa4e81700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0aa00fcf80 0x7f0aa0107600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.94:51194/0 (socket says 172.21.15.94:51194) 2024-03-24T10:54:24.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.092+0000 7f0aa4e81700 1 -- 172.21.15.94:0/398464149 learned_addr learned my addr 172.21.15.94:0/398464149 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:54:24.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.092+0000 7f0aa4e81700 1 -- 172.21.15.94:0/398464149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0aa00fbcd0 msgr2=0x7f0aa01070c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:24.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.092+0000 7f0aa4e81700 1 --2- 172.21.15.94:0/398464149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0aa00fbcd0 0x7f0aa01070c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:24.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.092+0000 7f0aa4e81700 1 -- 172.21.15.94:0/398464149 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a9c005040 con 0x7f0aa00fcf80 2024-03-24T10:54:24.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.092+0000 7f0aa4e81700 1 --2- 172.21.15.94:0/398464149 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0aa00fcf80 0x7f0aa0107600 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0a90002a80 tx=0x7f0a900046b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:54:24.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.093+0000 7f0a95ffb700 1 -- 172.21.15.94:0/398464149 <== mon.1 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a90007cc0 con 0x7f0aa00fcf80 2024-03-24T10:54:24.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.093+0000 7f0aa78e6700 1 -- 172.21.15.94:0/398464149 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aa0108780 con 0x7f0aa00fcf80 2024-03-24T10:54:24.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.093+0000 7f0a95ffb700 1 -- 172.21.15.94:0/398464149 <== mon.1 v2:172.21.15.96:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0a9000f040 con 0x7f0aa00fcf80 2024-03-24T10:54:24.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.093+0000 7f0a95ffb700 1 -- 172.21.15.94:0/398464149 <== mon.1 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a900096f0 con 0x7f0aa00fcf80 2024-03-24T10:54:24.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.093+0000 7f0aa78e6700 1 -- 172.21.15.94:0/398464149 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aa0108d40 con 0x7f0aa00fcf80 2024-03-24T10:54:24.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.094+0000 7f0aa78e6700 1 -- 172.21.15.94:0/398464149 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0aa0100910 con 0x7f0aa00fcf80 2024-03-24T10:54:24.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.095+0000 7f0a95ffb700 1 -- 172.21.15.94:0/398464149 <== mon.1 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f0a9000d690 con 0x7f0aa00fcf80 2024-03-24T10:54:24.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.095+0000 7f0a95ffb700 1 --2- 172.21.15.94:0/398464149 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f0a8c06d7e0 0x7f0a8c06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:54:24.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.096+0000 7f0a95ffb700 1 -- 172.21.15.94:0/398464149 <== mon.1 v2:172.21.15.96:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f0a9008c7a0 con 0x7f0aa00fcf80 2024-03-24T10:54:24.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.096+0000 7f0aa5682700 1 --2- 172.21.15.94:0/398464149 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f0a8c06d7e0 0x7f0a8c06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:54:24.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.096+0000 7f0aa5682700 1 --2- 172.21.15.94:0/398464149 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f0a8c06d7e0 0x7f0a8c06fc90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c00b970 tx=0x7f0a9c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:54:24.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.100+0000 7f0a95ffb700 1 -- 172.21.15.94:0/398464149 <== mon.1 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f0a90055ef0 con 0x7f0aa00fcf80 2024-03-24T10:54:24.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.493+0000 7f0aa78e6700 1 -- 172.21.15.94:0/398464149 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0aa005f5f0 con 0x7f0aa00fcf80 2024-03-24T10:54:24.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.494+0000 7f0a95ffb700 1 -- 172.21.15.94:0/398464149 <== mon.1 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f0a900595b0 con 0x7f0aa00fcf80 2024-03-24T10:54:24.494 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-03-24T10:54:24.495 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": { 2024-03-24T10:54:24.495 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 2 2024-03-24T10:54:24.495 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:54:24.495 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": { 2024-03-24T10:54:24.495 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 2 2024-03-24T10:54:24.495 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:54:24.495 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": { 2024-03-24T10:54:24.495 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 6 2024-03-24T10:54:24.495 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:54:24.495 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-03-24T10:54:24.495 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 4 2024-03-24T10:54:24.495 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:54:24.495 INFO:teuthology.orchestra.run.smithi094.stdout: "overall": { 2024-03-24T10:54:24.495 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 14 2024-03-24T10:54:24.496 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-03-24T10:54:24.496 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-03-24T10:54:24.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.496+0000 7f0aa78e6700 1 -- 172.21.15.94:0/398464149 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f0a8c06d7e0 msgr2=0x7f0a8c06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:24.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.496+0000 7f0aa78e6700 1 --2- 172.21.15.94:0/398464149 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f0a8c06d7e0 0x7f0a8c06fc90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c00b970 tx=0x7f0a9c009e30 comp rx=0 tx=0).stop 2024-03-24T10:54:24.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.496+0000 7f0aa78e6700 1 -- 172.21.15.94:0/398464149 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0aa00fcf80 msgr2=0x7f0aa0107600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:24.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.496+0000 7f0aa78e6700 1 --2- 172.21.15.94:0/398464149 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0aa00fcf80 0x7f0aa0107600 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0a90002a80 tx=0x7f0a900046b0 comp rx=0 tx=0).stop 2024-03-24T10:54:24.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.496+0000 7f0aa78e6700 1 -- 172.21.15.94:0/398464149 shutdown_connections 2024-03-24T10:54:24.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.496+0000 7f0aa78e6700 1 --2- 172.21.15.94:0/398464149 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f0a8c06d7e0 0x7f0a8c06fc90 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:24.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.496+0000 7f0aa78e6700 1 --2- 172.21.15.94:0/398464149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0aa00fbcd0 0x7f0aa01070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:24.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.496+0000 7f0aa78e6700 1 --2- 172.21.15.94:0/398464149 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0aa00fcf80 0x7f0aa0107600 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:24.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.496+0000 7f0aa78e6700 1 -- 172.21.15.94:0/398464149 >> 172.21.15.94:0/398464149 conn(0x7f0aa00f7430 msgr2=0x7f0aa00f89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:54:24.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.496+0000 7f0aa78e6700 1 -- 172.21.15.94:0/398464149 shutdown_connections 2024-03-24T10:54:24.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.496+0000 7f0aa78e6700 1 -- 172.21.15.94:0/398464149 wait complete. 2024-03-24T10:54:24.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.603+0000 7f771960e700 1 -- 172.21.15.94:0/2154198330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77140fce80 msgr2=0x7f77140fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:24.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.603+0000 7f771960e700 1 --2- 172.21.15.94:0/2154198330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77140fce80 0x7f77140fd290 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7704004940 tx=0x7f7704009e30 comp rx=0 tx=0).stop 2024-03-24T10:54:24.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.603+0000 7f771960e700 1 -- 172.21.15.94:0/2154198330 shutdown_connections 2024-03-24T10:54:24.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.603+0000 7f771960e700 1 --2- 172.21.15.94:0/2154198330 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77140fdff0 0x7f77140fe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:24.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.603+0000 7f771960e700 1 --2- 172.21.15.94:0/2154198330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77140fce80 0x7f77140fd290 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:24.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.603+0000 7f771960e700 1 -- 172.21.15.94:0/2154198330 >> 172.21.15.94:0/2154198330 conn(0x7f77140f8540 msgr2=0x7f77140fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:54:24.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.604+0000 7f771960e700 1 -- 172.21.15.94:0/2154198330 shutdown_connections 2024-03-24T10:54:24.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.604+0000 7f771960e700 1 -- 172.21.15.94:0/2154198330 wait complete. 2024-03-24T10:54:24.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.604+0000 7f771960e700 1 Processor -- start 2024-03-24T10:54:24.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.604+0000 7f771960e700 1 -- start start 2024-03-24T10:54:24.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.604+0000 7f771960e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77140fce80 0x7f771410d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:54:24.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.604+0000 7f771960e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77140fdff0 0x7f771410df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:54:24.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.604+0000 7f771960e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f771410e500 con 0x7f77140fce80 2024-03-24T10:54:24.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.604+0000 7f771960e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f771410ed50 con 0x7f77140fdff0 2024-03-24T10:54:24.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.605+0000 7f7712ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77140fce80 0x7f771410d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:54:24.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.605+0000 7f7712ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77140fce80 0x7f771410d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45918/0 (socket says 172.21.15.94:45918) 2024-03-24T10:54:24.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.605+0000 7f7712ffd700 1 -- 172.21.15.94:0/3651671120 learned_addr learned my addr 172.21.15.94:0/3651671120 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:54:24.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.605+0000 7f77127fc700 1 --2- 172.21.15.94:0/3651671120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77140fdff0 0x7f771410df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:54:24.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.605+0000 7f77127fc700 1 -- 172.21.15.94:0/3651671120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77140fce80 msgr2=0x7f771410d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:24.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.605+0000 7f77127fc700 1 --2- 172.21.15.94:0/3651671120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77140fce80 0x7f771410d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:24.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.605+0000 7f77127fc700 1 -- 172.21.15.94:0/3651671120 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7704005040 con 0x7f77140fdff0 2024-03-24T10:54:24.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.605+0000 7f7712ffd700 1 --2- 172.21.15.94:0/3651671120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77140fce80 0x7f771410d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T10:54:24.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.605+0000 7f77127fc700 1 --2- 172.21.15.94:0/3651671120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77140fdff0 0x7f771410df00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7708009fd0 tx=0x7f770800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:54:24.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.605+0000 7f76fb7fe700 1 -- 172.21.15.94:0/3651671120 <== mon.1 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7708004e90 con 0x7f77140fdff0 2024-03-24T10:54:24.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.605+0000 7f76fb7fe700 1 -- 172.21.15.94:0/3651671120 <== mon.1 v2:172.21.15.96:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f770800f040 con 0x7f77140fdff0 2024-03-24T10:54:24.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.605+0000 7f771960e700 1 -- 172.21.15.94:0/3651671120 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f771410efd0 con 0x7f77140fdff0 2024-03-24T10:54:24.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.606+0000 7f76fb7fe700 1 -- 172.21.15.94:0/3651671120 <== mon.1 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f770800ca50 con 0x7f77140fdff0 2024-03-24T10:54:24.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.606+0000 7f771960e700 1 -- 172.21.15.94:0/3651671120 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f771410f510 con 0x7f77140fdff0 2024-03-24T10:54:24.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.607+0000 7f771960e700 1 -- 172.21.15.94:0/3651671120 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f771405f5f0 con 0x7f77140fdff0 2024-03-24T10:54:24.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.607+0000 7f76fb7fe700 1 -- 172.21.15.94:0/3651671120 <== mon.1 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f7708007cc0 con 0x7f77140fdff0 2024-03-24T10:54:24.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.608+0000 7f76fb7fe700 1 --2- 172.21.15.94:0/3651671120 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f76fc06d7f0 0x7f76fc06fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:54:24.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.608+0000 7f76fb7fe700 1 -- 172.21.15.94:0/3651671120 <== mon.1 v2:172.21.15.96:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f770808c8e0 con 0x7f77140fdff0 2024-03-24T10:54:24.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.608+0000 7f7712ffd700 1 --2- 172.21.15.94:0/3651671120 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f76fc06d7f0 0x7f76fc06fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:54:24.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.608+0000 7f7712ffd700 1 --2- 172.21.15.94:0/3651671120 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f76fc06d7f0 0x7f76fc06fca0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7704013740 tx=0x7f7704009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:54:24.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.612+0000 7f76fb7fe700 1 -- 172.21.15.94:0/3651671120 <== mon.1 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7708059b60 con 0x7f77140fdff0 2024-03-24T10:54:24.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:24 smithi094 ceph-mon[93957]: from='client.34438 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:54:24.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:24 smithi094 ceph-mon[93957]: pgmap v212: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 5.7 MiB/s rd, 4.4 KiB/s wr, 6 op/s 2024-03-24T10:54:24.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:24 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:24.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:24 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:24.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:24 smithi096 ceph-mon[84258]: from='client.34438 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:54:24.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:24 smithi096 ceph-mon[84258]: pgmap v212: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 5.7 MiB/s rd, 4.4 KiB/s wr, 6 op/s 2024-03-24T10:54:24.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:24 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:24.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:24 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:24.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.889+0000 7f771960e700 1 -- 172.21.15.94:0/3651671120 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f771410f780 con 0x7f77140fdff0 2024-03-24T10:54:24.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.890+0000 7f76fb7fe700 1 -- 172.21.15.94:0/3651671120 <== mon.1 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 37 v37) v1 ==== 76+0+1963 (secure 0 0 0) 0x7f770808ae60 con 0x7f77140fdff0 2024-03-24T10:54:24.891 INFO:teuthology.orchestra.run.smithi094.stdout:e37 2024-03-24T10:54:24.891 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T10:54:24.891 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:54:24.891 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-03-24T10:54:24.891 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:54:24.891 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-03-24T10:54:24.891 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-03-24T10:54:24.891 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 37 2024-03-24T10:54:24.891 INFO:teuthology.orchestra.run.smithi094.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-24T10:54:24.891 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-03-24T10:43:39.435738+0000 2024-03-24T10:54:24.892 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-03-24T10:54:19.399866+0000 2024-03-24T10:54:24.892 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-03-24T10:54:24.892 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-03-24T10:54:24.892 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 60 2024-03-24T10:54:24.892 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-03-24T10:54:24.892 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-03-24T10:54:24.892 INFO:teuthology.orchestra.run.smithi094.stdout:max_xattr_size 65536 2024-03-24T10:54:24.892 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-03-24T10:54:24.892 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-03-24T10:54:24.892 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 84 2024-03-24T10:54:24.892 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-24T10:54:24.892 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-03-24T10:54:24.892 INFO:teuthology.orchestra.run.smithi094.stdout:in 0 2024-03-24T10:54:24.892 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=34374} 2024-03-24T10:54:24.892 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-03-24T10:54:24.893 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-03-24T10:54:24.893 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-03-24T10:54:24.893 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-03-24T10:54:24.893 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-03-24T10:54:24.893 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-03-24T10:54:24.893 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-03-24T10:54:24.893 INFO:teuthology.orchestra.run.smithi094.stdout:bal_rank_mask -1 2024-03-24T10:54:24.893 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 1 2024-03-24T10:54:24.893 INFO:teuthology.orchestra.run.smithi094.stdout:qdb_cluster leader: 34374 members: 34374 2024-03-24T10:54:24.893 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.efiqte{0:34374} state up:active seq 19 join_fscid=1 addr [v2:172.21.15.94:6826/2043632450,v1:172.21.15.94:6827/2043632450] compat {c=[1],r=[1],i=[1fff]}] 2024-03-24T10:54:24.893 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.llefgu{0:34406} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.94:6828/2396893131,v1:172.21.15.94:6829/2396893131] compat {c=[1],r=[1],i=[1fff]}] 2024-03-24T10:54:24.893 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:54:24.893 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:54:24.893 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-03-24T10:54:24.893 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:54:24.893 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi096.egbmoq{-1:44321} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.96:6824/2694461318,v1:172.21.15.96:6825/2694461318] compat {c=[1],r=[1],i=[1fff]}] 2024-03-24T10:54:24.894 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi096.jtebhh{-1:44349} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.96:6826/2275611388,v1:172.21.15.96:6827/2275611388] compat {c=[1],r=[1],i=[1fff]}] 2024-03-24T10:54:24.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.893+0000 7f771960e700 1 -- 172.21.15.94:0/3651671120 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f76fc06d7f0 msgr2=0x7f76fc06fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:24.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.893+0000 7f771960e700 1 --2- 172.21.15.94:0/3651671120 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f76fc06d7f0 0x7f76fc06fca0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7704013740 tx=0x7f7704009e30 comp rx=0 tx=0).stop 2024-03-24T10:54:24.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.893+0000 7f771960e700 1 -- 172.21.15.94:0/3651671120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77140fdff0 msgr2=0x7f771410df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:24.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.893+0000 7f771960e700 1 --2- 172.21.15.94:0/3651671120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77140fdff0 0x7f771410df00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7708009fd0 tx=0x7f770800b040 comp rx=0 tx=0).stop 2024-03-24T10:54:24.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.893+0000 7f771960e700 1 -- 172.21.15.94:0/3651671120 shutdown_connections 2024-03-24T10:54:24.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.893+0000 7f771960e700 1 --2- 172.21.15.94:0/3651671120 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f76fc06d7f0 0x7f76fc06fca0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:24.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.893+0000 7f771960e700 1 --2- 172.21.15.94:0/3651671120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77140fce80 0x7f771410d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:24.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.893+0000 7f771960e700 1 --2- 172.21.15.94:0/3651671120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77140fdff0 0x7f771410df00 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:24.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.893+0000 7f771960e700 1 -- 172.21.15.94:0/3651671120 >> 172.21.15.94:0/3651671120 conn(0x7f77140f8540 msgr2=0x7f77140f9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:54:24.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.894+0000 7f771960e700 1 -- 172.21.15.94:0/3651671120 shutdown_connections 2024-03-24T10:54:24.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:24.894+0000 7f771960e700 1 -- 172.21.15.94:0/3651671120 wait complete. 2024-03-24T10:54:24.895 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 37 2024-03-24T10:54:25.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.003+0000 7fdf1a7ca700 1 -- 172.21.15.94:0/4158239931 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf140fcf80 msgr2=0x7fdf140fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:25.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.003+0000 7fdf1a7ca700 1 --2- 172.21.15.94:0/4158239931 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf140fcf80 0x7fdf140fd3f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdf08004940 tx=0x7fdf08009e30 comp rx=0 tx=0).stop 2024-03-24T10:54:25.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.003+0000 7fdf1a7ca700 1 -- 172.21.15.94:0/4158239931 shutdown_connections 2024-03-24T10:54:25.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.003+0000 7fdf1a7ca700 1 --2- 172.21.15.94:0/4158239931 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf140fcf80 0x7fdf140fd3f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:25.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.003+0000 7fdf1a7ca700 1 --2- 172.21.15.94:0/4158239931 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf140fbcd0 0x7fdf140fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:25.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.003+0000 7fdf1a7ca700 1 -- 172.21.15.94:0/4158239931 >> 172.21.15.94:0/4158239931 conn(0x7fdf140f7430 msgr2=0x7fdf140f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:54:25.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.003+0000 7fdf1a7ca700 1 -- 172.21.15.94:0/4158239931 shutdown_connections 2024-03-24T10:54:25.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.003+0000 7fdf1a7ca700 1 -- 172.21.15.94:0/4158239931 wait complete. 2024-03-24T10:54:25.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.004+0000 7fdf1a7ca700 1 Processor -- start 2024-03-24T10:54:25.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.004+0000 7fdf1a7ca700 1 -- start start 2024-03-24T10:54:25.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.004+0000 7fdf1a7ca700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf140fbcd0 0x7fdf1410c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:54:25.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.004+0000 7fdf1a7ca700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf140fcf80 0x7fdf1410cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:54:25.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.004+0000 7fdf1a7ca700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1410d3c0 con 0x7fdf140fbcd0 2024-03-24T10:54:25.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.004+0000 7fdf1a7ca700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1410dc10 con 0x7fdf140fcf80 2024-03-24T10:54:25.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.005+0000 7fdf13fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf140fbcd0 0x7fdf1410c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:54:25.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.005+0000 7fdf13fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf140fbcd0 0x7fdf1410c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45940/0 (socket says 172.21.15.94:45940) 2024-03-24T10:54:25.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.005+0000 7fdf13fff700 1 -- 172.21.15.94:0/3731945587 learned_addr learned my addr 172.21.15.94:0/3731945587 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:54:25.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.005+0000 7fdf137fe700 1 --2- 172.21.15.94:0/3731945587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf140fcf80 0x7fdf1410cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:54:25.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.005+0000 7fdf137fe700 1 -- 172.21.15.94:0/3731945587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf140fbcd0 msgr2=0x7fdf1410c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:25.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.005+0000 7fdf137fe700 1 --2- 172.21.15.94:0/3731945587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf140fbcd0 0x7fdf1410c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:25.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.005+0000 7fdf137fe700 1 -- 172.21.15.94:0/3731945587 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf08005040 con 0x7fdf140fcf80 2024-03-24T10:54:25.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.005+0000 7fdf13fff700 1 --2- 172.21.15.94:0/3731945587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf140fbcd0 0x7fdf1410c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T10:54:25.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.005+0000 7fdf137fe700 1 --2- 172.21.15.94:0/3731945587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf140fcf80 0x7fdf1410cdc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fdf08009fd0 tx=0x7fdf08009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:54:25.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.005+0000 7fdf10ff9700 1 -- 172.21.15.94:0/3731945587 <== mon.1 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf08021070 con 0x7fdf140fcf80 2024-03-24T10:54:25.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.005+0000 7fdf10ff9700 1 -- 172.21.15.94:0/3731945587 <== mon.1 v2:172.21.15.96:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdf0801c400 con 0x7fdf140fcf80 2024-03-24T10:54:25.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.006+0000 7fdf10ff9700 1 -- 172.21.15.94:0/3731945587 <== mon.1 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf0800e070 con 0x7fdf140fcf80 2024-03-24T10:54:25.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.006+0000 7fdf1a7ca700 1 -- 172.21.15.94:0/3731945587 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf1410de90 con 0x7fdf140fcf80 2024-03-24T10:54:25.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.006+0000 7fdf1a7ca700 1 -- 172.21.15.94:0/3731945587 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf1410e350 con 0x7fdf140fcf80 2024-03-24T10:54:25.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.007+0000 7fdf1a7ca700 1 -- 172.21.15.94:0/3731945587 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf1405f5f0 con 0x7fdf140fcf80 2024-03-24T10:54:25.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.008+0000 7fdf10ff9700 1 -- 172.21.15.94:0/3731945587 <== mon.1 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fdf08013af0 con 0x7fdf140fcf80 2024-03-24T10:54:25.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.008+0000 7fdf10ff9700 1 --2- 172.21.15.94:0/3731945587 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fdefc06d5e0 0x7fdefc06fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:54:25.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.008+0000 7fdf10ff9700 1 -- 172.21.15.94:0/3731945587 <== mon.1 v2:172.21.15.96:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fdf0808d960 con 0x7fdf140fcf80 2024-03-24T10:54:25.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.008+0000 7fdf13fff700 1 --2- 172.21.15.94:0/3731945587 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fdefc06d5e0 0x7fdefc06fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:54:25.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.009+0000 7fdf13fff700 1 --2- 172.21.15.94:0/3731945587 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fdefc06d5e0 0x7fdefc06fa90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdf040041d0 tx=0x7fdf0400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:54:25.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.013+0000 7fdf10ff9700 1 -- 172.21.15.94:0/3731945587 <== mon.1 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fdf0805add0 con 0x7fdf140fcf80 2024-03-24T10:54:25.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.264+0000 7fdf1a7ca700 1 -- 172.21.15.94:0/3731945587 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf1410e660 con 0x7fdefc06d5e0 2024-03-24T10:54:25.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.266+0000 7fdf10ff9700 1 -- 172.21.15.94:0/3731945587 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7fdf1410e660 con 0x7fdefc06d5e0 2024-03-24T10:54:25.267 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-03-24T10:54:25.267 INFO:teuthology.orchestra.run.smithi094.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919", 2024-03-24T10:54:25.267 INFO:teuthology.orchestra.run.smithi094.stdout: "in_progress": true, 2024-03-24T10:54:25.267 INFO:teuthology.orchestra.run.smithi094.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T10:54:25.267 INFO:teuthology.orchestra.run.smithi094.stdout: "services_complete": [ 2024-03-24T10:54:25.267 INFO:teuthology.orchestra.run.smithi094.stdout: "mds", 2024-03-24T10:54:25.267 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr", 2024-03-24T10:54:25.267 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-03-24T10:54:25.267 INFO:teuthology.orchestra.run.smithi094.stdout: "mon", 2024-03-24T10:54:25.267 INFO:teuthology.orchestra.run.smithi094.stdout: "osd" 2024-03-24T10:54:25.267 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-03-24T10:54:25.268 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": "16/21 daemons upgraded", 2024-03-24T10:54:25.268 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "Currently upgrading node-exporter daemons", 2024-03-24T10:54:25.268 INFO:teuthology.orchestra.run.smithi094.stdout: "is_paused": false 2024-03-24T10:54:25.268 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-03-24T10:54:25.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.268+0000 7fdf1a7ca700 1 -- 172.21.15.94:0/3731945587 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fdefc06d5e0 msgr2=0x7fdefc06fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:25.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.268+0000 7fdf1a7ca700 1 --2- 172.21.15.94:0/3731945587 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fdefc06d5e0 0x7fdefc06fa90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdf040041d0 tx=0x7fdf0400b040 comp rx=0 tx=0).stop 2024-03-24T10:54:25.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.268+0000 7fdf1a7ca700 1 -- 172.21.15.94:0/3731945587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf140fcf80 msgr2=0x7fdf1410cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:25.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.268+0000 7fdf1a7ca700 1 --2- 172.21.15.94:0/3731945587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf140fcf80 0x7fdf1410cdc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fdf08009fd0 tx=0x7fdf08009e30 comp rx=0 tx=0).stop 2024-03-24T10:54:25.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.269+0000 7fdf1a7ca700 1 -- 172.21.15.94:0/3731945587 shutdown_connections 2024-03-24T10:54:25.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.269+0000 7fdf1a7ca700 1 --2- 172.21.15.94:0/3731945587 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fdefc06d5e0 0x7fdefc06fa90 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:25.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.269+0000 7fdf1a7ca700 1 --2- 172.21.15.94:0/3731945587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf140fbcd0 0x7fdf1410c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:25.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.269+0000 7fdf1a7ca700 1 --2- 172.21.15.94:0/3731945587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf140fcf80 0x7fdf1410cdc0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:25.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.269+0000 7fdf1a7ca700 1 -- 172.21.15.94:0/3731945587 >> 172.21.15.94:0/3731945587 conn(0x7fdf140f7430 msgr2=0x7fdf140f8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:54:25.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.269+0000 7fdf1a7ca700 1 -- 172.21.15.94:0/3731945587 shutdown_connections 2024-03-24T10:54:25.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.269+0000 7fdf1a7ca700 1 -- 172.21.15.94:0/3731945587 wait complete. 2024-03-24T10:54:25.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.386+0000 7f9a37bfa700 1 -- 172.21.15.94:0/862990547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a300fff00 msgr2=0x7f9a30102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:25.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.386+0000 7f9a37bfa700 1 --2- 172.21.15.94:0/862990547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a300fff00 0x7f9a30102320 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9a20004940 tx=0x7f9a20009e30 comp rx=0 tx=0).stop 2024-03-24T10:54:25.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.386+0000 7f9a37bfa700 1 -- 172.21.15.94:0/862990547 shutdown_connections 2024-03-24T10:54:25.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.386+0000 7f9a37bfa700 1 --2- 172.21.15.94:0/862990547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a300fff00 0x7f9a30102320 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:25.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.386+0000 7f9a37bfa700 1 --2- 172.21.15.94:0/862990547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a300662e0 0x7f9a300ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:25.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.386+0000 7f9a37bfa700 1 -- 172.21.15.94:0/862990547 >> 172.21.15.94:0/862990547 conn(0x7f9a300f7430 msgr2=0x7f9a300f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:54:25.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.386+0000 7f9a37bfa700 1 -- 172.21.15.94:0/862990547 shutdown_connections 2024-03-24T10:54:25.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.386+0000 7f9a37bfa700 1 -- 172.21.15.94:0/862990547 wait complete. 2024-03-24T10:54:25.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.387+0000 7f9a37bfa700 1 Processor -- start 2024-03-24T10:54:25.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.387+0000 7f9a37bfa700 1 -- start start 2024-03-24T10:54:25.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.387+0000 7f9a37bfa700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a300662e0 0x7f9a3010da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:54:25.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.387+0000 7f9a37bfa700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a300fff00 0x7f9a3010df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:54:25.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.387+0000 7f9a37bfa700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3010e580 con 0x7f9a300662e0 2024-03-24T10:54:25.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.387+0000 7f9a37bfa700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3010ed80 con 0x7f9a300fff00 2024-03-24T10:54:25.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.387+0000 7f9a35996700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a300662e0 0x7f9a3010da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:54:25.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.387+0000 7f9a35996700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a300662e0 0x7f9a3010da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45958/0 (socket says 172.21.15.94:45958) 2024-03-24T10:54:25.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.387+0000 7f9a35996700 1 -- 172.21.15.94:0/1634556630 learned_addr learned my addr 172.21.15.94:0/1634556630 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:54:25.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.387+0000 7f9a35195700 1 --2- 172.21.15.94:0/1634556630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a300fff00 0x7f9a3010df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:54:25.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.387+0000 7f9a35996700 1 -- 172.21.15.94:0/1634556630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a300fff00 msgr2=0x7f9a3010df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:25.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.387+0000 7f9a35996700 1 --2- 172.21.15.94:0/1634556630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a300fff00 0x7f9a3010df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:25.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.387+0000 7f9a35996700 1 -- 172.21.15.94:0/1634556630 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a20005040 con 0x7f9a300662e0 2024-03-24T10:54:25.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.387+0000 7f9a35195700 1 --2- 172.21.15.94:0/1634556630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a300fff00 0x7f9a3010df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T10:54:25.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.388+0000 7f9a35996700 1 --2- 172.21.15.94:0/1634556630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a300662e0 0x7f9a3010da40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c00eb40 tx=0x7f9a2c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:54:25.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.388+0000 7f9a267fc700 1 -- 172.21.15.94:0/1634556630 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a2c007cc0 con 0x7f9a300662e0 2024-03-24T10:54:25.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.388+0000 7f9a37bfa700 1 -- 172.21.15.94:0/1634556630 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a3010f000 con 0x7f9a300662e0 2024-03-24T10:54:25.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.388+0000 7f9a267fc700 1 -- 172.21.15.94:0/1634556630 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9a2c00f040 con 0x7f9a300662e0 2024-03-24T10:54:25.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.388+0000 7f9a37bfa700 1 -- 172.21.15.94:0/1634556630 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a3010f540 con 0x7f9a300662e0 2024-03-24T10:54:25.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.388+0000 7f9a267fc700 1 -- 172.21.15.94:0/1634556630 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a2c0095e0 con 0x7f9a300662e0 2024-03-24T10:54:25.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.389+0000 7f9a37bfa700 1 -- 172.21.15.94:0/1634556630 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a140052e0 con 0x7f9a300662e0 2024-03-24T10:54:25.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.390+0000 7f9a267fc700 1 -- 172.21.15.94:0/1634556630 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f9a2c009740 con 0x7f9a300662e0 2024-03-24T10:54:25.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.390+0000 7f9a267fc700 1 --2- 172.21.15.94:0/1634556630 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f9a1c071bf0 0x7f9a1c0740a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:54:25.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.391+0000 7f9a267fc700 1 -- 172.21.15.94:0/1634556630 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f9a2c08c970 con 0x7f9a300662e0 2024-03-24T10:54:25.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.391+0000 7f9a35195700 1 --2- 172.21.15.94:0/1634556630 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f9a1c071bf0 0x7f9a1c0740a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:54:25.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.391+0000 7f9a35195700 1 --2- 172.21.15.94:0/1634556630 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f9a1c071bf0 0x7f9a1c0740a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9a20009b30 tx=0x7f9a20009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:54:25.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.399+0000 7f9a267fc700 1 -- 172.21.15.94:0/1634556630 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9a2c059d20 con 0x7f9a300662e0 2024-03-24T10:54:25.613 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:25 smithi094 ceph-mon[93957]: from='client.34442 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:54:25.613 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:25 smithi094 ceph-mon[93957]: from='client.34446 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:54:25.613 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:25 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/398464149' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:25.613 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:25 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/3651671120' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T10:54:25.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:25 smithi096 ceph-mon[84258]: from='client.34442 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:54:25.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:25 smithi096 ceph-mon[84258]: from='client.34446 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:54:25.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:25 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/398464149' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:25.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:25 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/3651671120' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T10:54:25.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.945+0000 7f9a37bfa700 1 -- 172.21.15.94:0/1634556630 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f9a14005590 con 0x7f9a300662e0 2024-03-24T10:54:25.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.946+0000 7f9a267fc700 1 -- 172.21.15.94:0/1634556630 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f9a2c0598b0 con 0x7f9a300662e0 2024-03-24T10:54:25.947 INFO:teuthology.orchestra.run.smithi094.stdout:HEALTH_OK 2024-03-24T10:54:25.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.948+0000 7f9a37bfa700 1 -- 172.21.15.94:0/1634556630 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f9a1c071bf0 msgr2=0x7f9a1c0740a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:25.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.948+0000 7f9a37bfa700 1 --2- 172.21.15.94:0/1634556630 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f9a1c071bf0 0x7f9a1c0740a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9a20009b30 tx=0x7f9a20009e30 comp rx=0 tx=0).stop 2024-03-24T10:54:25.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.948+0000 7f9a37bfa700 1 -- 172.21.15.94:0/1634556630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a300662e0 msgr2=0x7f9a3010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:25.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.948+0000 7f9a37bfa700 1 --2- 172.21.15.94:0/1634556630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a300662e0 0x7f9a3010da40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c00eb40 tx=0x7f9a2c00b040 comp rx=0 tx=0).stop 2024-03-24T10:54:25.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.948+0000 7f9a37bfa700 1 -- 172.21.15.94:0/1634556630 shutdown_connections 2024-03-24T10:54:25.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.948+0000 7f9a37bfa700 1 --2- 172.21.15.94:0/1634556630 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f9a1c071bf0 0x7f9a1c0740a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:25.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.948+0000 7f9a37bfa700 1 --2- 172.21.15.94:0/1634556630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a300662e0 0x7f9a3010da40 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:25.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.948+0000 7f9a37bfa700 1 --2- 172.21.15.94:0/1634556630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a300fff00 0x7f9a3010df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:25.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.948+0000 7f9a37bfa700 1 -- 172.21.15.94:0/1634556630 >> 172.21.15.94:0/1634556630 conn(0x7f9a300f7430 msgr2=0x7f9a300f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:54:25.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.948+0000 7f9a37bfa700 1 -- 172.21.15.94:0/1634556630 shutdown_connections 2024-03-24T10:54:25.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:25.948+0000 7f9a37bfa700 1 -- 172.21.15.94:0/1634556630 wait complete. 2024-03-24T10:54:26.680 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:26 smithi094 ceph-mon[93957]: from='client.44373 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:54:26.681 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:26 smithi094 ceph-mon[93957]: pgmap v213: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 9.0 MiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-03-24T10:54:26.681 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:26 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/1634556630' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T10:54:26.703 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:26 smithi096 ceph-mon[84258]: from='client.44373 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:54:26.703 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:26 smithi096 ceph-mon[84258]: pgmap v213: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 9.0 MiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-03-24T10:54:26.703 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:26 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/1634556630' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T10:54:27.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:27 smithi096 ceph-mon[84258]: Upgrade: Updating node-exporter.smithi096 (2/2) 2024-03-24T10:54:27.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:27 smithi096 ceph-mon[84258]: Deploying daemon node-exporter.smithi096 on smithi096 2024-03-24T10:54:27.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:27 smithi094 ceph-mon[93957]: Upgrade: Updating node-exporter.smithi096 (2/2) 2024-03-24T10:54:27.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:27 smithi094 ceph-mon[93957]: Deploying daemon node-exporter.smithi096 on smithi096 2024-03-24T10:54:28.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:28 smithi096 ceph-mon[84258]: pgmap v214: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-03-24T10:54:28.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:28 smithi094 ceph-mon[93957]: pgmap v214: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-03-24T10:54:29.647 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:29 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:29.648 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:29 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:29.648 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:29 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:54:29.648 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:29 smithi096 ceph-mon[84258]: pgmap v215: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-03-24T10:54:29.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:29 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:29.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:29 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:29.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:29 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:54:29.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:29 smithi094 ceph-mon[93957]: pgmap v215: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-03-24T10:54:32.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:31 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:32.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:31 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:32.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:31 smithi094 ceph-mon[93957]: pgmap v216: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 4 op/s 2024-03-24T10:54:32.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:31 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:32.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:31 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:32.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:31 smithi096 ceph-mon[84258]: pgmap v216: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 4 op/s 2024-03-24T10:54:33.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:32 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:33.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:32 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:33.280 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:32 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:33.280 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:32 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:34.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:33 smithi094 ceph-mon[93957]: pgmap v217: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-03-24T10:54:34.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:33 smithi096 ceph-mon[84258]: pgmap v217: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-03-24T10:54:35.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:35 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:54:35.643 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:35 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:54:36.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:36 smithi094 ceph-mon[93957]: pgmap v218: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 3 op/s 2024-03-24T10:54:36.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:36 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:36.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:36 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:36.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:36 smithi096 ceph-mon[84258]: pgmap v218: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 3 op/s 2024-03-24T10:54:36.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:36 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:36.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:36 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:38.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:37 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:38.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:37 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:38.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:37 smithi094 ceph-mon[93957]: pgmap v219: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 7.1 MiB/s rd, 2 op/s 2024-03-24T10:54:38.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:37 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:38.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:37 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:38.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:37 smithi096 ceph-mon[84258]: pgmap v219: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 7.1 MiB/s rd, 2 op/s 2024-03-24T10:54:39.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:38 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:39.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:38 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:39.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:38 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:39.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:38 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:40.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:39 smithi094 ceph-mon[93957]: pgmap v220: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 5.4 MiB/s rd, 2 op/s 2024-03-24T10:54:40.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:39 smithi096 ceph-mon[84258]: pgmap v220: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 5.4 MiB/s rd, 2 op/s 2024-03-24T10:54:42.603 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:42 smithi094 ceph-mon[93957]: pgmap v221: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-24T10:54:42.603 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:42 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:42.603 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:42 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:42.603 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:42 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:54:42.603 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:42 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:54:42.603 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:42 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:42.603 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:42 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:54:42.603 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:42 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:42.603 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:42 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:42.604 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:42 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:42.604 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:42 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:42.604 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:42 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:42.604 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:42 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:42.604 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:42 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:42.604 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:42 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:42.604 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:42 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:42.604 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:42 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:42.604 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:42 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:42.604 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:42 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:42.604 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:42 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:42.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:42 smithi096 ceph-mon[84258]: pgmap v221: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-24T10:54:42.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:42 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:42.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:42 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:42.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:42 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:54:42.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:42 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:54:42.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:42 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:42.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:42 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:54:42.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:42 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:42.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:42 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:42.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:42 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:42.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:42 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:42.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:42 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:42.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:42 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:42.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:42 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:42.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:42 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:42.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:42 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:42.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:42 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:42.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:42 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:42.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:42 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:42.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:42 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:44.565 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:44 smithi094 ceph-mon[93957]: pgmap v222: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-24T10:54:44.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:44 smithi096 ceph-mon[84258]: pgmap v222: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-24T10:54:45.650 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:45 smithi094 ceph-mon[93957]: Upgrade: Updating prometheus.smithi094 2024-03-24T10:54:45.650 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:45 smithi094 ceph-mon[93957]: Deploying daemon prometheus.smithi094 on smithi094 2024-03-24T10:54:45.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:45 smithi096 ceph-mon[84258]: Upgrade: Updating prometheus.smithi094 2024-03-24T10:54:45.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:45 smithi096 ceph-mon[84258]: Deploying daemon prometheus.smithi094 on smithi094 2024-03-24T10:54:46.654 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:46 smithi094 ceph-mon[93957]: pgmap v223: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T10:54:46.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:46 smithi096 ceph-mon[84258]: pgmap v223: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T10:54:48.464 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:48 smithi094 ceph-mon[93957]: pgmap v224: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:54:48.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:48 smithi096 ceph-mon[84258]: pgmap v224: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:54:49.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:49 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:49.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:49 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:49.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:49 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:54:49.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:49 smithi094 ceph-mon[93957]: pgmap v225: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:54:50.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:49 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:50.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:49 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:50.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:49 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:54:50.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:49 smithi096 ceph-mon[84258]: pgmap v225: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:54:50.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:50 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:54:51.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:50 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:54:52.116 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:51 smithi094 ceph-mon[93957]: pgmap v226: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:54:52.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:51 smithi096 ceph-mon[84258]: pgmap v226: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:54:54.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:54 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:54.575 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:54 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:54.576 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:54 smithi096 ceph-mon[84258]: pgmap v227: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-24T10:54:54.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:54 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:54.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:54 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:54.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:54 smithi094 ceph-mon[93957]: pgmap v227: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-24T10:54:55.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:55 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:55.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:55 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:55.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:55 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:55.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:55 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:56.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.058+0000 7f1fdf46b700 1 -- 172.21.15.94:0/2027464461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fd80fcf80 msgr2=0x7f1fd80fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:56.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.058+0000 7f1fdf46b700 1 --2- 172.21.15.94:0/2027464461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fd80fcf80 0x7f1fd80fd3f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8004940 tx=0x7f1fc8009e30 comp rx=0 tx=0).stop 2024-03-24T10:54:56.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.058+0000 7f1fdf46b700 1 -- 172.21.15.94:0/2027464461 shutdown_connections 2024-03-24T10:54:56.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.058+0000 7f1fdf46b700 1 --2- 172.21.15.94:0/2027464461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fd80fcf80 0x7f1fd80fd3f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:56.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.058+0000 7f1fdf46b700 1 --2- 172.21.15.94:0/2027464461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fd80fbcd0 0x7f1fd80fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:56.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.058+0000 7f1fdf46b700 1 -- 172.21.15.94:0/2027464461 >> 172.21.15.94:0/2027464461 conn(0x7f1fd80f7430 msgr2=0x7f1fd80f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:54:56.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.058+0000 7f1fdf46b700 1 -- 172.21.15.94:0/2027464461 shutdown_connections 2024-03-24T10:54:56.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.058+0000 7f1fdf46b700 1 -- 172.21.15.94:0/2027464461 wait complete. 2024-03-24T10:54:56.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.059+0000 7f1fdf46b700 1 Processor -- start 2024-03-24T10:54:56.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.059+0000 7f1fdf46b700 1 -- start start 2024-03-24T10:54:56.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.059+0000 7f1fdf46b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fd80fbcd0 0x7f1fd8110700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:54:56.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.059+0000 7f1fdf46b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fd80fcf80 0x7f1fd8112c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:54:56.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.059+0000 7f1fdf46b700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd8066370 con 0x7f1fd80fcf80 2024-03-24T10:54:56.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.059+0000 7f1fdf46b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd80664b0 con 0x7f1fd80fbcd0 2024-03-24T10:54:56.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.059+0000 7f1fdca06700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fd80fcf80 0x7f1fd8112c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:54:56.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.059+0000 7f1fdd207700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fd80fbcd0 0x7f1fd8110700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:54:56.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.059+0000 7f1fdca06700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fd80fcf80 0x7f1fd8112c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36148/0 (socket says 172.21.15.94:36148) 2024-03-24T10:54:56.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.059+0000 7f1fdca06700 1 -- 172.21.15.94:0/3826115212 learned_addr learned my addr 172.21.15.94:0/3826115212 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:54:56.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.060+0000 7f1fdca06700 1 -- 172.21.15.94:0/3826115212 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fd80fbcd0 msgr2=0x7f1fd8110700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:56.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.060+0000 7f1fdca06700 1 --2- 172.21.15.94:0/3826115212 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fd80fbcd0 0x7f1fd8110700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:56.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.060+0000 7f1fdca06700 1 -- 172.21.15.94:0/3826115212 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc8005040 con 0x7f1fd80fcf80 2024-03-24T10:54:56.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.060+0000 7f1fdd207700 1 --2- 172.21.15.94:0/3826115212 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fd80fbcd0 0x7f1fd8110700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T10:54:56.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.060+0000 7f1fdca06700 1 --2- 172.21.15.94:0/3826115212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fd80fcf80 0x7f1fd8112c50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1fc800b3b0 tx=0x7f1fc8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:54:56.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.060+0000 7f1fcdffb700 1 -- 172.21.15.94:0/3826115212 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1fc800e040 con 0x7f1fd80fcf80 2024-03-24T10:54:56.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.060+0000 7f1fdf46b700 1 -- 172.21.15.94:0/3826115212 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fd81131f0 con 0x7f1fd80fcf80 2024-03-24T10:54:56.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.060+0000 7f1fcdffb700 1 -- 172.21.15.94:0/3826115212 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1fc800bbf0 con 0x7f1fd80fcf80 2024-03-24T10:54:56.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.060+0000 7f1fcdffb700 1 -- 172.21.15.94:0/3826115212 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1fc801c630 con 0x7f1fd80fcf80 2024-03-24T10:54:56.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.060+0000 7f1fdf46b700 1 -- 172.21.15.94:0/3826115212 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fd8113780 con 0x7f1fd80fcf80 2024-03-24T10:54:56.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.061+0000 7f1fdf46b700 1 -- 172.21.15.94:0/3826115212 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fd805f5f0 con 0x7f1fd80fcf80 2024-03-24T10:54:56.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.062+0000 7f1fcdffb700 1 -- 172.21.15.94:0/3826115212 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f1fc800bd60 con 0x7f1fd80fcf80 2024-03-24T10:54:56.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.063+0000 7f1fcdffb700 1 --2- 172.21.15.94:0/3826115212 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1fc406d7e0 0x7f1fc406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:54:56.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.063+0000 7f1fcdffb700 1 -- 172.21.15.94:0/3826115212 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f1fc808db80 con 0x7f1fd80fcf80 2024-03-24T10:54:56.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.063+0000 7f1fdd207700 1 --2- 172.21.15.94:0/3826115212 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1fc406d7e0 0x7f1fc406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:54:56.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.063+0000 7f1fdd207700 1 --2- 172.21.15.94:0/3826115212 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1fc406d7e0 0x7f1fc406fc90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8065ff0 tx=0x7f1fd400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:54:56.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.068+0000 7f1fcdffb700 1 -- 172.21.15.94:0/3826115212 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1fc8057350 con 0x7f1fd80fcf80 2024-03-24T10:54:56.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.299+0000 7f1fdf46b700 1 -- 172.21.15.94:0/3826115212 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1fd8101360 con 0x7f1fc406d7e0 2024-03-24T10:54:56.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.301+0000 7f1fcdffb700 1 -- 172.21.15.94:0/3826115212 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+438 (secure 0 0 0) 0x7f1fd8101360 con 0x7f1fc406d7e0 2024-03-24T10:54:56.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.303+0000 7f1fdf46b700 1 -- 172.21.15.94:0/3826115212 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1fc406d7e0 msgr2=0x7f1fc406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:56.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.303+0000 7f1fdf46b700 1 --2- 172.21.15.94:0/3826115212 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1fc406d7e0 0x7f1fc406fc90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8065ff0 tx=0x7f1fd400b040 comp rx=0 tx=0).stop 2024-03-24T10:54:56.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.303+0000 7f1fdf46b700 1 -- 172.21.15.94:0/3826115212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fd80fcf80 msgr2=0x7f1fd8112c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:56.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.303+0000 7f1fdf46b700 1 --2- 172.21.15.94:0/3826115212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fd80fcf80 0x7f1fd8112c50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1fc800b3b0 tx=0x7f1fc8009e30 comp rx=0 tx=0).stop 2024-03-24T10:54:56.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.303+0000 7f1fdf46b700 1 -- 172.21.15.94:0/3826115212 shutdown_connections 2024-03-24T10:54:56.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.303+0000 7f1fdf46b700 1 --2- 172.21.15.94:0/3826115212 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1fc406d7e0 0x7f1fc406fc90 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:56.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.303+0000 7f1fdf46b700 1 --2- 172.21.15.94:0/3826115212 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fd80fbcd0 0x7f1fd8110700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:56.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.303+0000 7f1fdf46b700 1 --2- 172.21.15.94:0/3826115212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fd80fcf80 0x7f1fd8112c50 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:56.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.303+0000 7f1fdf46b700 1 -- 172.21.15.94:0/3826115212 >> 172.21.15.94:0/3826115212 conn(0x7f1fd80f7430 msgr2=0x7f1fd80f8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:54:56.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.303+0000 7f1fdf46b700 1 -- 172.21.15.94:0/3826115212 shutdown_connections 2024-03-24T10:54:56.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.303+0000 7f1fdf46b700 1 -- 172.21.15.94:0/3826115212 wait complete. 2024-03-24T10:54:56.314 INFO:teuthology.orchestra.run.smithi094.stdout:true 2024-03-24T10:54:56.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.427+0000 7f93dffbc700 1 -- 172.21.15.94:0/3953098064 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93d80fffd0 msgr2=0x7f93d81023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:56.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.427+0000 7f93dffbc700 1 --2- 172.21.15.94:0/3953098064 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93d80fffd0 0x7f93d81023b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f93c8004940 tx=0x7f93c8009e30 comp rx=0 tx=0).stop 2024-03-24T10:54:56.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.427+0000 7f93dffbc700 1 -- 172.21.15.94:0/3953098064 shutdown_connections 2024-03-24T10:54:56.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.427+0000 7f93dffbc700 1 --2- 172.21.15.94:0/3953098064 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93d80fffd0 0x7f93d81023b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:56.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.427+0000 7f93dffbc700 1 --2- 172.21.15.94:0/3953098064 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93d8066b30 0x7f93d80ffa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:56.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.427+0000 7f93dffbc700 1 -- 172.21.15.94:0/3953098064 >> 172.21.15.94:0/3953098064 conn(0x7f93d80f74e0 msgr2=0x7f93d80f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:54:56.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.427+0000 7f93dffbc700 1 -- 172.21.15.94:0/3953098064 shutdown_connections 2024-03-24T10:54:56.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.427+0000 7f93dffbc700 1 -- 172.21.15.94:0/3953098064 wait complete. 2024-03-24T10:54:56.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.428+0000 7f93dffbc700 1 Processor -- start 2024-03-24T10:54:56.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.428+0000 7f93dffbc700 1 -- start start 2024-03-24T10:54:56.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.428+0000 7f93dffbc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93d8066b30 0x7f93d810da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:54:56.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.428+0000 7f93dffbc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93d80fffd0 0x7f93d810dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:54:56.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.428+0000 7f93dffbc700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d810e5a0 con 0x7f93d8066b30 2024-03-24T10:54:56.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.428+0000 7f93dffbc700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d810eda0 con 0x7f93d80fffd0 2024-03-24T10:54:56.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.428+0000 7f93ddd58700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93d8066b30 0x7f93d810da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:54:56.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.428+0000 7f93ddd58700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93d8066b30 0x7f93d810da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36160/0 (socket says 172.21.15.94:36160) 2024-03-24T10:54:56.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.428+0000 7f93ddd58700 1 -- 172.21.15.94:0/12076454 learned_addr learned my addr 172.21.15.94:0/12076454 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:54:56.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.428+0000 7f93dd557700 1 --2- 172.21.15.94:0/12076454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93d80fffd0 0x7f93d810dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:54:56.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.428+0000 7f93ddd58700 1 -- 172.21.15.94:0/12076454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93d80fffd0 msgr2=0x7f93d810dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:56.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.428+0000 7f93ddd58700 1 --2- 172.21.15.94:0/12076454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93d80fffd0 0x7f93d810dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:56.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.428+0000 7f93ddd58700 1 -- 172.21.15.94:0/12076454 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c8005040 con 0x7f93d8066b30 2024-03-24T10:54:56.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.428+0000 7f93dd557700 1 --2- 172.21.15.94:0/12076454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93d80fffd0 0x7f93d810dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T10:54:56.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.428+0000 7f93ddd58700 1 --2- 172.21.15.94:0/12076454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93d8066b30 0x7f93d810da60 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f93d4002a80 tx=0x7f93d4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:54:56.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.429+0000 7f93ce7fc700 1 -- 172.21.15.94:0/12076454 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93d4007cc0 con 0x7f93d8066b30 2024-03-24T10:54:56.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.429+0000 7f93ce7fc700 1 -- 172.21.15.94:0/12076454 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f93d400f040 con 0x7f93d8066b30 2024-03-24T10:54:56.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.429+0000 7f93dffbc700 1 -- 172.21.15.94:0/12076454 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93d810f080 con 0x7f93d8066b30 2024-03-24T10:54:56.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.429+0000 7f93ce7fc700 1 -- 172.21.15.94:0/12076454 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93d400dad0 con 0x7f93d8066b30 2024-03-24T10:54:56.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.429+0000 7f93dffbc700 1 -- 172.21.15.94:0/12076454 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93d810f5c0 con 0x7f93d8066b30 2024-03-24T10:54:56.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.430+0000 7f93dffbc700 1 -- 172.21.15.94:0/12076454 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93d805f5f0 con 0x7f93d8066b30 2024-03-24T10:54:56.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.430+0000 7f93ce7fc700 1 -- 172.21.15.94:0/12076454 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f93d400dc30 con 0x7f93d8066b30 2024-03-24T10:54:56.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.431+0000 7f93ce7fc700 1 --2- 172.21.15.94:0/12076454 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f93c406d7e0 0x7f93c406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:54:56.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.431+0000 7f93ce7fc700 1 -- 172.21.15.94:0/12076454 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f93d408cd80 con 0x7f93d8066b30 2024-03-24T10:54:56.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.431+0000 7f93dd557700 1 --2- 172.21.15.94:0/12076454 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f93c406d7e0 0x7f93c406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:54:56.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.431+0000 7f93dd557700 1 --2- 172.21.15.94:0/12076454 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f93c406d7e0 0x7f93c406fc90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f93c800b3b0 tx=0x7f93c8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:54:56.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.435+0000 7f93ce7fc700 1 -- 172.21.15.94:0/12076454 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f93d405a000 con 0x7f93d8066b30 2024-03-24T10:54:56.551 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:56 smithi094 ceph-mon[93957]: pgmap v228: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:54:56.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.663+0000 7f93dffbc700 1 -- 172.21.15.94:0/12076454 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f93d810f950 con 0x7f93c406d7e0 2024-03-24T10:54:56.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.664+0000 7f93ce7fc700 1 -- 172.21.15.94:0/12076454 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+438 (secure 0 0 0) 0x7f93d810f950 con 0x7f93c406d7e0 2024-03-24T10:54:56.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.667+0000 7f93dffbc700 1 -- 172.21.15.94:0/12076454 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f93c406d7e0 msgr2=0x7f93c406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:56.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.667+0000 7f93dffbc700 1 --2- 172.21.15.94:0/12076454 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f93c406d7e0 0x7f93c406fc90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f93c800b3b0 tx=0x7f93c8009e30 comp rx=0 tx=0).stop 2024-03-24T10:54:56.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.667+0000 7f93dffbc700 1 -- 172.21.15.94:0/12076454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93d8066b30 msgr2=0x7f93d810da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:56.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.667+0000 7f93dffbc700 1 --2- 172.21.15.94:0/12076454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93d8066b30 0x7f93d810da60 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f93d4002a80 tx=0x7f93d4004600 comp rx=0 tx=0).stop 2024-03-24T10:54:56.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.667+0000 7f93dffbc700 1 -- 172.21.15.94:0/12076454 shutdown_connections 2024-03-24T10:54:56.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.667+0000 7f93dffbc700 1 --2- 172.21.15.94:0/12076454 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f93c406d7e0 0x7f93c406fc90 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:56.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.667+0000 7f93dffbc700 1 --2- 172.21.15.94:0/12076454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93d8066b30 0x7f93d810da60 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:56.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.667+0000 7f93dffbc700 1 --2- 172.21.15.94:0/12076454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93d80fffd0 0x7f93d810dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:56.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.667+0000 7f93dffbc700 1 -- 172.21.15.94:0/12076454 >> 172.21.15.94:0/12076454 conn(0x7f93d80f74e0 msgr2=0x7f93d80f78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:54:56.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.667+0000 7f93dffbc700 1 -- 172.21.15.94:0/12076454 shutdown_connections 2024-03-24T10:54:56.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.667+0000 7f93dffbc700 1 -- 172.21.15.94:0/12076454 wait complete. 2024-03-24T10:54:56.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.771+0000 7f1e21999700 1 -- 172.21.15.94:0/695836897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e1c0fbb90 msgr2=0x7f1e1c1023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:56.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.771+0000 7f1e21999700 1 --2- 172.21.15.94:0/695836897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e1c0fbb90 0x7f1e1c1023c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1e10004940 tx=0x7f1e10009e30 comp rx=0 tx=0).stop 2024-03-24T10:54:56.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.771+0000 7f1e21999700 1 -- 172.21.15.94:0/695836897 shutdown_connections 2024-03-24T10:54:56.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.771+0000 7f1e21999700 1 --2- 172.21.15.94:0/695836897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e1c0fbb90 0x7f1e1c1023c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:56.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.771+0000 7f1e21999700 1 --2- 172.21.15.94:0/695836897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e1c0fb240 0x7f1e1c0fb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:56.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.771+0000 7f1e21999700 1 -- 172.21.15.94:0/695836897 >> 172.21.15.94:0/695836897 conn(0x7f1e1c0f6be0 msgr2=0x7f1e1c0f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:54:56.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.772+0000 7f1e21999700 1 -- 172.21.15.94:0/695836897 shutdown_connections 2024-03-24T10:54:56.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.772+0000 7f1e21999700 1 -- 172.21.15.94:0/695836897 wait complete. 2024-03-24T10:54:56.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.772+0000 7f1e21999700 1 Processor -- start 2024-03-24T10:54:56.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.772+0000 7f1e21999700 1 -- start start 2024-03-24T10:54:56.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.773+0000 7f1e21999700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e1c0fb240 0x7f1e1c10d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:54:56.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.773+0000 7f1e21999700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e1c0fbb90 0x7f1e1c10de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:54:56.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.773+0000 7f1e21999700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c10e480 con 0x7f1e1c0fbb90 2024-03-24T10:54:56.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.773+0000 7f1e21999700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c10e5f0 con 0x7f1e1c0fb240 2024-03-24T10:54:56.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.773+0000 7f1e1a7fc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e1c0fbb90 0x7f1e1c10de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:54:56.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.773+0000 7f1e1a7fc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e1c0fbb90 0x7f1e1c10de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36168/0 (socket says 172.21.15.94:36168) 2024-03-24T10:54:56.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.773+0000 7f1e1a7fc700 1 -- 172.21.15.94:0/2045326246 learned_addr learned my addr 172.21.15.94:0/2045326246 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:54:56.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.773+0000 7f1e1affd700 1 --2- 172.21.15.94:0/2045326246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e1c0fb240 0x7f1e1c10d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:54:56.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.773+0000 7f1e1a7fc700 1 -- 172.21.15.94:0/2045326246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e1c0fb240 msgr2=0x7f1e1c10d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:56.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.773+0000 7f1e1a7fc700 1 --2- 172.21.15.94:0/2045326246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e1c0fb240 0x7f1e1c10d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:56.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.773+0000 7f1e1a7fc700 1 -- 172.21.15.94:0/2045326246 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e10005040 con 0x7f1e1c0fbb90 2024-03-24T10:54:56.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.773+0000 7f1e1a7fc700 1 --2- 172.21.15.94:0/2045326246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e1c0fbb90 0x7f1e1c10de80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1e1000b3b0 tx=0x7f1e10009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:54:56.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.773+0000 7f1e037fe700 1 -- 172.21.15.94:0/2045326246 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e1000e040 con 0x7f1e1c0fbb90 2024-03-24T10:54:56.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.773+0000 7f1e037fe700 1 -- 172.21.15.94:0/2045326246 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1e1000bbf0 con 0x7f1e1c0fbb90 2024-03-24T10:54:56.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.773+0000 7f1e21999700 1 -- 172.21.15.94:0/2045326246 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e1c112ff0 con 0x7f1e1c0fbb90 2024-03-24T10:54:56.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.773+0000 7f1e037fe700 1 -- 172.21.15.94:0/2045326246 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e1001c630 con 0x7f1e1c0fbb90 2024-03-24T10:54:56.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.773+0000 7f1e21999700 1 -- 172.21.15.94:0/2045326246 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e1c113580 con 0x7f1e1c0fbb90 2024-03-24T10:54:56.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.774+0000 7f1e21999700 1 -- 172.21.15.94:0/2045326246 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e1c04c710 con 0x7f1e1c0fbb90 2024-03-24T10:54:56.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.775+0000 7f1e037fe700 1 -- 172.21.15.94:0/2045326246 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f1e1000bd60 con 0x7f1e1c0fbb90 2024-03-24T10:54:56.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.776+0000 7f1e037fe700 1 --2- 172.21.15.94:0/2045326246 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1e0406d7e0 0x7f1e0406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:54:56.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.776+0000 7f1e037fe700 1 -- 172.21.15.94:0/2045326246 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f1e1008db80 con 0x7f1e1c0fbb90 2024-03-24T10:54:56.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.776+0000 7f1e1affd700 1 --2- 172.21.15.94:0/2045326246 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1e0406d7e0 0x7f1e0406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:54:56.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.776+0000 7f1e1affd700 1 --2- 172.21.15.94:0/2045326246 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1e0406d7e0 0x7f1e0406fc90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c004f40 tx=0x7f1e0c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:54:56.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.780+0000 7f1e037fe700 1 -- 172.21.15.94:0/2045326246 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1e1005ae80 con 0x7f1e1c0fbb90 2024-03-24T10:54:56.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:56 smithi096 ceph-mon[84258]: pgmap v228: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:54:57.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:56.999+0000 7f1e21999700 1 -- 172.21.15.94:0/2045326246 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e1c101ba0 con 0x7f1e0406d7e0 2024-03-24T10:54:57.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.010+0000 7f1e037fe700 1 -- 172.21.15.94:0/2045326246 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f1e1c101ba0 con 0x7f1e0406d7e0 2024-03-24T10:54:57.011 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T10:54:57.011 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.smithi094 smithi094 *:9093,9094 running (8m) 3s ago 17m 24.4M - 0.25.0 c8568f914cd2 5c583af60554 2024-03-24T10:54:57.011 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi094 smithi094 running (5m) 3s ago 17m 6651k - 19.0.0-1855-g21b3bd67 553463a90fd5 2b0897df571d 2024-03-24T10:54:57.011 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi096 smithi096 running (5m) 26s ago 15m 6669k - 19.0.0-1855-g21b3bd67 553463a90fd5 1759a9ff2805 2024-03-24T10:54:57.011 INFO:teuthology.orchestra.run.smithi094.stdout:grafana.smithi094 smithi094 *:3000 running (7m) 3s ago 16m 84.1M - 9.4.7 954c08fa6188 d3d425bf1b7c 2024-03-24T10:54:57.011 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.efiqte smithi094 running (108s) 3s ago 11m 85.9M - 19.0.0-1855-g21b3bd67 553463a90fd5 732c47b17971 2024-03-24T10:54:57.011 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.llefgu smithi094 running (92s) 3s ago 11m 87.2M - 19.0.0-1855-g21b3bd67 553463a90fd5 6e06f8a6dde0 2024-03-24T10:54:57.011 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi096.egbmoq smithi096 running (67s) 26s ago 11m 14.1M - 19.0.0-1855-g21b3bd67 553463a90fd5 f35fd7f23742 2024-03-24T10:54:57.011 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi096.jtebhh smithi096 running (46s) 26s ago 11m 13.9M - 19.0.0-1855-g21b3bd67 553463a90fd5 8b5ab9f96e5a 2024-03-24T10:54:57.011 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi094.yaethu smithi094 *:8443,9283,8765 running (7m) 3s ago 18m 510M - 19.0.0-1855-g21b3bd67 553463a90fd5 d3adab76b18f 2024-03-24T10:54:57.011 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi096.gfonnu smithi096 *:8443,9283,8765 running (6m) 26s ago 15m 450M - 19.0.0-1855-g21b3bd67 553463a90fd5 22937a059eb8 2024-03-24T10:54:57.011 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi094 smithi094 running (6m) 3s ago 18m 54.6M 2048M 19.0.0-1855-g21b3bd67 553463a90fd5 c779d611e29a 2024-03-24T10:54:57.011 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi096 smithi096 running (6m) 26s ago 15m 38.5M 2048M 19.0.0-1855-g21b3bd67 553463a90fd5 ed9bd0bd57f6 2024-03-24T10:54:57.011 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi094 smithi094 *:9100 running (33s) 3s ago 17m 8652k - 1.5.0 0da6a335fe13 003df2346f4f 2024-03-24T10:54:57.012 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi096 smithi096 *:9100 running (28s) 26s ago 15m 5481k - 1.5.0 0da6a335fe13 67d65fecf9df 2024-03-24T10:54:57.012 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (5m) 3s ago 14m 110M 1557M 19.0.0-1855-g21b3bd67 553463a90fd5 ec57ede0c9e4 2024-03-24T10:54:57.012 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi094 running (4m) 3s ago 14m 124M 1557M 19.0.0-1855-g21b3bd67 553463a90fd5 c5b8b2bbf8b5 2024-03-24T10:54:57.012 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi094 running (3m) 3s ago 13m 102M 1557M 19.0.0-1855-g21b3bd67 553463a90fd5 c961dcdc746f 2024-03-24T10:54:57.012 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi096 running (3m) 26s ago 13m 121M 2581M 19.0.0-1855-g21b3bd67 553463a90fd5 6323fdc6e6ca 2024-03-24T10:54:57.012 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi096 running (2m) 26s ago 13m 111M 2581M 19.0.0-1855-g21b3bd67 553463a90fd5 3e2536bcb263 2024-03-24T10:54:57.012 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi096 running (2m) 26s ago 12m 103M 2581M 19.0.0-1855-g21b3bd67 553463a90fd5 2f77f1fa20e2 2024-03-24T10:54:57.012 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.smithi094 smithi094 *:9095 running (8s) 3s ago 16m 41.4M - 2.43.0 a07b618ecd1d 326ddd336bc5 2024-03-24T10:54:57.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.012+0000 7f1e21999700 1 -- 172.21.15.94:0/2045326246 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1e0406d7e0 msgr2=0x7f1e0406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:57.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.012+0000 7f1e21999700 1 --2- 172.21.15.94:0/2045326246 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1e0406d7e0 0x7f1e0406fc90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c004f40 tx=0x7f1e0c00b040 comp rx=0 tx=0).stop 2024-03-24T10:54:57.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.012+0000 7f1e21999700 1 -- 172.21.15.94:0/2045326246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e1c0fbb90 msgr2=0x7f1e1c10de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:57.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.012+0000 7f1e21999700 1 --2- 172.21.15.94:0/2045326246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e1c0fbb90 0x7f1e1c10de80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1e1000b3b0 tx=0x7f1e10009e30 comp rx=0 tx=0).stop 2024-03-24T10:54:57.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.012+0000 7f1e21999700 1 -- 172.21.15.94:0/2045326246 shutdown_connections 2024-03-24T10:54:57.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.012+0000 7f1e21999700 1 --2- 172.21.15.94:0/2045326246 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1e0406d7e0 0x7f1e0406fc90 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:57.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.012+0000 7f1e21999700 1 --2- 172.21.15.94:0/2045326246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e1c0fb240 0x7f1e1c10d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:57.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.013+0000 7f1e21999700 1 --2- 172.21.15.94:0/2045326246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e1c0fbb90 0x7f1e1c10de80 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:57.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.013+0000 7f1e21999700 1 -- 172.21.15.94:0/2045326246 >> 172.21.15.94:0/2045326246 conn(0x7f1e1c0f6be0 msgr2=0x7f1e1c100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:54:57.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.013+0000 7f1e21999700 1 -- 172.21.15.94:0/2045326246 shutdown_connections 2024-03-24T10:54:57.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.013+0000 7f1e21999700 1 -- 172.21.15.94:0/2045326246 wait complete. 2024-03-24T10:54:57.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.120+0000 7f8557ec4700 1 -- 172.21.15.94:0/3049320468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85500fe040 msgr2=0x7f85500fe4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:57.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.120+0000 7f8557ec4700 1 --2- 172.21.15.94:0/3049320468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85500fe040 0x7f85500fe4b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8540004940 tx=0x7f8540009e30 comp rx=0 tx=0).stop 2024-03-24T10:54:57.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.120+0000 7f8557ec4700 1 -- 172.21.15.94:0/3049320468 shutdown_connections 2024-03-24T10:54:57.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.120+0000 7f8557ec4700 1 --2- 172.21.15.94:0/3049320468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85500fe040 0x7f85500fe4b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:57.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.120+0000 7f8557ec4700 1 --2- 172.21.15.94:0/3049320468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85500fce40 0x7f85500fd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:57.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.120+0000 7f8557ec4700 1 -- 172.21.15.94:0/3049320468 >> 172.21.15.94:0/3049320468 conn(0x7f85500f8550 msgr2=0x7f85500fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:54:57.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.120+0000 7f8557ec4700 1 -- 172.21.15.94:0/3049320468 shutdown_connections 2024-03-24T10:54:57.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.120+0000 7f8557ec4700 1 -- 172.21.15.94:0/3049320468 wait complete. 2024-03-24T10:54:57.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.121+0000 7f8557ec4700 1 Processor -- start 2024-03-24T10:54:57.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.121+0000 7f8557ec4700 1 -- start start 2024-03-24T10:54:57.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.121+0000 7f8557ec4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85500fce40 0x7f855010da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:54:57.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.121+0000 7f8557ec4700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85500fe040 0x7f855010dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:54:57.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.121+0000 7f8557ec4700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855010e5a0 con 0x7f85500fce40 2024-03-24T10:54:57.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.121+0000 7f8557ec4700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855010edf0 con 0x7f85500fe040 2024-03-24T10:54:57.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.121+0000 7f8555c60700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85500fce40 0x7f855010da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:54:57.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.121+0000 7f8555c60700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85500fce40 0x7f855010da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36190/0 (socket says 172.21.15.94:36190) 2024-03-24T10:54:57.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.121+0000 7f8555c60700 1 -- 172.21.15.94:0/2502715187 learned_addr learned my addr 172.21.15.94:0/2502715187 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:54:57.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.121+0000 7f855545f700 1 --2- 172.21.15.94:0/2502715187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85500fe040 0x7f855010dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:54:57.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.122+0000 7f8555c60700 1 -- 172.21.15.94:0/2502715187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85500fe040 msgr2=0x7f855010dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:57.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.122+0000 7f8555c60700 1 --2- 172.21.15.94:0/2502715187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85500fe040 0x7f855010dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:57.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.122+0000 7f8555c60700 1 -- 172.21.15.94:0/2502715187 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8540005040 con 0x7f85500fce40 2024-03-24T10:54:57.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.122+0000 7f8555c60700 1 --2- 172.21.15.94:0/2502715187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85500fce40 0x7f855010da60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f854c00c8e0 tx=0x7f854c0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:54:57.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.122+0000 7f85467fc700 1 -- 172.21.15.94:0/2502715187 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f854c00a760 con 0x7f85500fce40 2024-03-24T10:54:57.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.122+0000 7f8557ec4700 1 -- 172.21.15.94:0/2502715187 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f855010f0d0 con 0x7f85500fce40 2024-03-24T10:54:57.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.122+0000 7f85467fc700 1 -- 172.21.15.94:0/2502715187 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f854c00ad90 con 0x7f85500fce40 2024-03-24T10:54:57.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.122+0000 7f85467fc700 1 -- 172.21.15.94:0/2502715187 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f854c0113a0 con 0x7f85500fce40 2024-03-24T10:54:57.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.122+0000 7f8557ec4700 1 -- 172.21.15.94:0/2502715187 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f855010f610 con 0x7f85500fce40 2024-03-24T10:54:57.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.124+0000 7f8557ec4700 1 -- 172.21.15.94:0/2502715187 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f855005f5f0 con 0x7f85500fce40 2024-03-24T10:54:57.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.124+0000 7f85467fc700 1 -- 172.21.15.94:0/2502715187 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f854c012400 con 0x7f85500fce40 2024-03-24T10:54:57.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.124+0000 7f85467fc700 1 --2- 172.21.15.94:0/2502715187 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f853c06d7e0 0x7f853c06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:54:57.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.125+0000 7f85467fc700 1 -- 172.21.15.94:0/2502715187 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f854c08cf80 con 0x7f85500fce40 2024-03-24T10:54:57.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.125+0000 7f855545f700 1 --2- 172.21.15.94:0/2502715187 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f853c06d7e0 0x7f853c06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:54:57.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.125+0000 7f855545f700 1 --2- 172.21.15.94:0/2502715187 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f853c06d7e0 0x7f853c06fc90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f854000b3b0 tx=0x7f8540009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:54:57.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.131+0000 7f85467fc700 1 -- 172.21.15.94:0/2502715187 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f854c0c1280 con 0x7f85500fce40 2024-03-24T10:54:57.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.448+0000 7f8557ec4700 1 -- 172.21.15.94:0/2502715187 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f855010f8e0 con 0x7f85500fce40 2024-03-24T10:54:57.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.450+0000 7f85467fc700 1 -- 172.21.15.94:0/2502715187 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f854c05a280 con 0x7f85500fce40 2024-03-24T10:54:57.451 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-03-24T10:54:57.451 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": { 2024-03-24T10:54:57.451 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 2 2024-03-24T10:54:57.451 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:54:57.451 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": { 2024-03-24T10:54:57.452 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 2 2024-03-24T10:54:57.452 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:54:57.452 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": { 2024-03-24T10:54:57.452 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 6 2024-03-24T10:54:57.452 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:54:57.452 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-03-24T10:54:57.452 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 4 2024-03-24T10:54:57.452 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:54:57.452 INFO:teuthology.orchestra.run.smithi094.stdout: "overall": { 2024-03-24T10:54:57.452 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 14 2024-03-24T10:54:57.452 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-03-24T10:54:57.452 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-03-24T10:54:57.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.453+0000 7f8557ec4700 1 -- 172.21.15.94:0/2502715187 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f853c06d7e0 msgr2=0x7f853c06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:57.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.453+0000 7f8557ec4700 1 --2- 172.21.15.94:0/2502715187 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f853c06d7e0 0x7f853c06fc90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f854000b3b0 tx=0x7f8540009e30 comp rx=0 tx=0).stop 2024-03-24T10:54:57.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.453+0000 7f8557ec4700 1 -- 172.21.15.94:0/2502715187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85500fce40 msgr2=0x7f855010da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:57.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.453+0000 7f8557ec4700 1 --2- 172.21.15.94:0/2502715187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85500fce40 0x7f855010da60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f854c00c8e0 tx=0x7f854c0044c0 comp rx=0 tx=0).stop 2024-03-24T10:54:57.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.453+0000 7f8557ec4700 1 -- 172.21.15.94:0/2502715187 shutdown_connections 2024-03-24T10:54:57.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.453+0000 7f8557ec4700 1 --2- 172.21.15.94:0/2502715187 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f853c06d7e0 0x7f853c06fc90 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:57.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.453+0000 7f8557ec4700 1 --2- 172.21.15.94:0/2502715187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85500fce40 0x7f855010da60 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:57.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.453+0000 7f8557ec4700 1 --2- 172.21.15.94:0/2502715187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85500fe040 0x7f855010dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:57.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.453+0000 7f8557ec4700 1 -- 172.21.15.94:0/2502715187 >> 172.21.15.94:0/2502715187 conn(0x7f85500f8550 msgr2=0x7f85500f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:54:57.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.453+0000 7f8557ec4700 1 -- 172.21.15.94:0/2502715187 shutdown_connections 2024-03-24T10:54:57.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.453+0000 7f8557ec4700 1 -- 172.21.15.94:0/2502715187 wait complete. 2024-03-24T10:54:57.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.557+0000 7f380475e700 1 -- 172.21.15.94:0/256879577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37fc0fcf80 msgr2=0x7f37fc0fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:57.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.557+0000 7f380475e700 1 --2- 172.21.15.94:0/256879577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37fc0fcf80 0x7f37fc0fd3f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f37ec004940 tx=0x7f37ec009e30 comp rx=0 tx=0).stop 2024-03-24T10:54:57.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.557+0000 7f380475e700 1 -- 172.21.15.94:0/256879577 shutdown_connections 2024-03-24T10:54:57.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.557+0000 7f380475e700 1 --2- 172.21.15.94:0/256879577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37fc0fcf80 0x7f37fc0fd3f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:57.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.557+0000 7f380475e700 1 --2- 172.21.15.94:0/256879577 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f37fc0fbcd0 0x7f37fc0fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:57.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.557+0000 7f380475e700 1 -- 172.21.15.94:0/256879577 >> 172.21.15.94:0/256879577 conn(0x7f37fc0f7430 msgr2=0x7f37fc0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:54:57.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.557+0000 7f380475e700 1 -- 172.21.15.94:0/256879577 shutdown_connections 2024-03-24T10:54:57.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.557+0000 7f380475e700 1 -- 172.21.15.94:0/256879577 wait complete. 2024-03-24T10:54:57.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.558+0000 7f380475e700 1 Processor -- start 2024-03-24T10:54:57.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.558+0000 7f380475e700 1 -- start start 2024-03-24T10:54:57.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.558+0000 7f380475e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f37fc0fbcd0 0x7f37fc110720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:54:57.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.558+0000 7f380475e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37fc0fcf80 0x7f37fc112c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:54:57.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.558+0000 7f380475e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37fc066350 con 0x7f37fc0fcf80 2024-03-24T10:54:57.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.558+0000 7f380475e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37fc066490 con 0x7f37fc0fbcd0 2024-03-24T10:54:57.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.558+0000 7f3801cf9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37fc0fcf80 0x7f37fc112c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:54:57.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.558+0000 7f3801cf9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37fc0fcf80 0x7f37fc112c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36206/0 (socket says 172.21.15.94:36206) 2024-03-24T10:54:57.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.558+0000 7f3801cf9700 1 -- 172.21.15.94:0/4221811842 learned_addr learned my addr 172.21.15.94:0/4221811842 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:54:57.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.558+0000 7f38024fa700 1 --2- 172.21.15.94:0/4221811842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f37fc0fbcd0 0x7f37fc110720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:54:57.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.559+0000 7f3801cf9700 1 -- 172.21.15.94:0/4221811842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f37fc0fbcd0 msgr2=0x7f37fc110720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:57.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.559+0000 7f3801cf9700 1 --2- 172.21.15.94:0/4221811842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f37fc0fbcd0 0x7f37fc110720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:57.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.559+0000 7f3801cf9700 1 -- 172.21.15.94:0/4221811842 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ec005040 con 0x7f37fc0fcf80 2024-03-24T10:54:57.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.559+0000 7f3801cf9700 1 --2- 172.21.15.94:0/4221811842 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37fc0fcf80 0x7f37fc112c70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f37ec00b3b0 tx=0x7f37ec009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:54:57.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.559+0000 7f37f2ffd700 1 -- 172.21.15.94:0/4221811842 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f37ec00e040 con 0x7f37fc0fcf80 2024-03-24T10:54:57.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.559+0000 7f380475e700 1 -- 172.21.15.94:0/4221811842 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37fc1131b0 con 0x7f37fc0fcf80 2024-03-24T10:54:57.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.559+0000 7f37f2ffd700 1 -- 172.21.15.94:0/4221811842 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f37ec00bcc0 con 0x7f37fc0fcf80 2024-03-24T10:54:57.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.560+0000 7f37f2ffd700 1 -- 172.21.15.94:0/4221811842 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f37ec01c630 con 0x7f37fc0fcf80 2024-03-24T10:54:57.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.560+0000 7f380475e700 1 -- 172.21.15.94:0/4221811842 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37fc113770 con 0x7f37fc0fcf80 2024-03-24T10:54:57.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.560+0000 7f380475e700 1 -- 172.21.15.94:0/4221811842 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37fc05f5f0 con 0x7f37fc0fcf80 2024-03-24T10:54:57.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.560+0000 7f37f2ffd700 1 -- 172.21.15.94:0/4221811842 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f37ec013980 con 0x7f37fc0fcf80 2024-03-24T10:54:57.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.561+0000 7f37f2ffd700 1 --2- 172.21.15.94:0/4221811842 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f37e806d7e0 0x7f37e806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:54:57.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.561+0000 7f37f2ffd700 1 -- 172.21.15.94:0/4221811842 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f37ec08dc30 con 0x7f37fc0fcf80 2024-03-24T10:54:57.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.561+0000 7f38024fa700 1 --2- 172.21.15.94:0/4221811842 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f37e806d7e0 0x7f37e806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:54:57.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.561+0000 7f38024fa700 1 --2- 172.21.15.94:0/4221811842 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f37e806d7e0 0x7f37e806fc90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f37f8007fd0 tx=0x7f37f8009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:54:57.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.566+0000 7f37f2ffd700 1 -- 172.21.15.94:0/4221811842 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f37ec094080 con 0x7f37fc0fcf80 2024-03-24T10:54:57.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:57 smithi094 ceph-mon[93957]: from='client.34466 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:54:57.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:57 smithi096 ceph-mon[84258]: from='client.34466 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:54:57.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.829+0000 7f380475e700 1 -- 172.21.15.94:0/4221811842 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f37fc04bed0 con 0x7f37fc0fcf80 2024-03-24T10:54:57.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.829+0000 7f37f2ffd700 1 -- 172.21.15.94:0/4221811842 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 37 v37) v1 ==== 76+0+1963 (secure 0 0 0) 0x7f37ec08d070 con 0x7f37fc0fcf80 2024-03-24T10:54:57.830 INFO:teuthology.orchestra.run.smithi094.stdout:e37 2024-03-24T10:54:57.830 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T10:54:57.830 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:54:57.830 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-03-24T10:54:57.830 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:54:57.830 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-03-24T10:54:57.830 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-03-24T10:54:57.830 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 37 2024-03-24T10:54:57.831 INFO:teuthology.orchestra.run.smithi094.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-24T10:54:57.831 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-03-24T10:43:39.435738+0000 2024-03-24T10:54:57.831 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-03-24T10:54:19.399866+0000 2024-03-24T10:54:57.831 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-03-24T10:54:57.831 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-03-24T10:54:57.831 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 60 2024-03-24T10:54:57.831 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-03-24T10:54:57.831 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-03-24T10:54:57.831 INFO:teuthology.orchestra.run.smithi094.stdout:max_xattr_size 65536 2024-03-24T10:54:57.831 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-03-24T10:54:57.831 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-03-24T10:54:57.831 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 84 2024-03-24T10:54:57.831 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-24T10:54:57.831 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-03-24T10:54:57.831 INFO:teuthology.orchestra.run.smithi094.stdout:in 0 2024-03-24T10:54:57.831 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=34374} 2024-03-24T10:54:57.832 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-03-24T10:54:57.832 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-03-24T10:54:57.832 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-03-24T10:54:57.832 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-03-24T10:54:57.832 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-03-24T10:54:57.832 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-03-24T10:54:57.832 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-03-24T10:54:57.832 INFO:teuthology.orchestra.run.smithi094.stdout:bal_rank_mask -1 2024-03-24T10:54:57.832 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 1 2024-03-24T10:54:57.832 INFO:teuthology.orchestra.run.smithi094.stdout:qdb_cluster leader: 34374 members: 34374 2024-03-24T10:54:57.832 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.efiqte{0:34374} state up:active seq 19 join_fscid=1 addr [v2:172.21.15.94:6826/2043632450,v1:172.21.15.94:6827/2043632450] compat {c=[1],r=[1],i=[1fff]}] 2024-03-24T10:54:57.832 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.llefgu{0:34406} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.94:6828/2396893131,v1:172.21.15.94:6829/2396893131] compat {c=[1],r=[1],i=[1fff]}] 2024-03-24T10:54:57.832 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:54:57.833 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:54:57.833 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-03-24T10:54:57.833 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:54:57.833 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi096.egbmoq{-1:44321} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.96:6824/2694461318,v1:172.21.15.96:6825/2694461318] compat {c=[1],r=[1],i=[1fff]}] 2024-03-24T10:54:57.833 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi096.jtebhh{-1:44349} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.96:6826/2275611388,v1:172.21.15.96:6827/2275611388] compat {c=[1],r=[1],i=[1fff]}] 2024-03-24T10:54:57.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.832+0000 7f380475e700 1 -- 172.21.15.94:0/4221811842 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f37e806d7e0 msgr2=0x7f37e806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:57.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.832+0000 7f380475e700 1 --2- 172.21.15.94:0/4221811842 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f37e806d7e0 0x7f37e806fc90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f37f8007fd0 tx=0x7f37f8009040 comp rx=0 tx=0).stop 2024-03-24T10:54:57.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.832+0000 7f380475e700 1 -- 172.21.15.94:0/4221811842 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37fc0fcf80 msgr2=0x7f37fc112c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:57.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.832+0000 7f380475e700 1 --2- 172.21.15.94:0/4221811842 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37fc0fcf80 0x7f37fc112c70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f37ec00b3b0 tx=0x7f37ec009e30 comp rx=0 tx=0).stop 2024-03-24T10:54:57.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.832+0000 7f380475e700 1 -- 172.21.15.94:0/4221811842 shutdown_connections 2024-03-24T10:54:57.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.832+0000 7f380475e700 1 --2- 172.21.15.94:0/4221811842 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f37e806d7e0 0x7f37e806fc90 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:57.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.832+0000 7f380475e700 1 --2- 172.21.15.94:0/4221811842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f37fc0fbcd0 0x7f37fc110720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:57.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.832+0000 7f380475e700 1 --2- 172.21.15.94:0/4221811842 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37fc0fcf80 0x7f37fc112c70 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:57.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.832+0000 7f380475e700 1 -- 172.21.15.94:0/4221811842 >> 172.21.15.94:0/4221811842 conn(0x7f37fc0f7430 msgr2=0x7f37fc0f8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:54:57.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.832+0000 7f380475e700 1 -- 172.21.15.94:0/4221811842 shutdown_connections 2024-03-24T10:54:57.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.832+0000 7f380475e700 1 -- 172.21.15.94:0/4221811842 wait complete. 2024-03-24T10:54:57.835 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 37 2024-03-24T10:54:57.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.964+0000 7fb5c8a0c700 1 -- 172.21.15.94:0/2780175791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5c40fff00 msgr2=0x7fb5c4102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:57.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.964+0000 7fb5c8a0c700 1 --2- 172.21.15.94:0/2780175791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5c40fff00 0x7fb5c4102320 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8004940 tx=0x7fb5b8009e30 comp rx=0 tx=0).stop 2024-03-24T10:54:57.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.964+0000 7fb5c8a0c700 1 -- 172.21.15.94:0/2780175791 shutdown_connections 2024-03-24T10:54:57.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.964+0000 7fb5c8a0c700 1 --2- 172.21.15.94:0/2780175791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5c40fff00 0x7fb5c4102320 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:57.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.964+0000 7fb5c8a0c700 1 --2- 172.21.15.94:0/2780175791 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5c40662e0 0x7fb5c40ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:57.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.964+0000 7fb5c8a0c700 1 -- 172.21.15.94:0/2780175791 >> 172.21.15.94:0/2780175791 conn(0x7fb5c40f7430 msgr2=0x7fb5c40f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:54:57.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.965+0000 7fb5c8a0c700 1 -- 172.21.15.94:0/2780175791 shutdown_connections 2024-03-24T10:54:57.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.965+0000 7fb5c8a0c700 1 -- 172.21.15.94:0/2780175791 wait complete. 2024-03-24T10:54:57.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.965+0000 7fb5c8a0c700 1 Processor -- start 2024-03-24T10:54:57.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.965+0000 7fb5c8a0c700 1 -- start start 2024-03-24T10:54:57.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.965+0000 7fb5c8a0c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5c40662e0 0x7fb5c40fd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:54:57.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.965+0000 7fb5c8a0c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5c40fff00 0x7fb5c40fb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:54:57.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.965+0000 7fb5c8a0c700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c40fbe70 con 0x7fb5c40fff00 2024-03-24T10:54:57.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.965+0000 7fb5c8a0c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c40fbfe0 con 0x7fb5c40662e0 2024-03-24T10:54:57.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.965+0000 7fb5c1d9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5c40fff00 0x7fb5c40fb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:54:57.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.965+0000 7fb5c1d9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5c40fff00 0x7fb5c40fb8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36222/0 (socket says 172.21.15.94:36222) 2024-03-24T10:54:57.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.965+0000 7fb5c1d9b700 1 -- 172.21.15.94:0/2780916325 learned_addr learned my addr 172.21.15.94:0/2780916325 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:54:57.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.966+0000 7fb5c259c700 1 --2- 172.21.15.94:0/2780916325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5c40662e0 0x7fb5c40fd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:54:57.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.966+0000 7fb5c1d9b700 1 -- 172.21.15.94:0/2780916325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5c40662e0 msgr2=0x7fb5c40fd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:57.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.966+0000 7fb5c1d9b700 1 --2- 172.21.15.94:0/2780916325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5c40662e0 0x7fb5c40fd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:57.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.966+0000 7fb5c1d9b700 1 -- 172.21.15.94:0/2780916325 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b8005040 con 0x7fb5c40fff00 2024-03-24T10:54:57.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.966+0000 7fb5c1d9b700 1 --2- 172.21.15.94:0/2780916325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5c40fff00 0x7fb5c40fb8c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb5b800b3b0 tx=0x7fb5b8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:54:57.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.966+0000 7fb5b2ffd700 1 -- 172.21.15.94:0/2780916325 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5b800e040 con 0x7fb5c40fff00 2024-03-24T10:54:57.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.966+0000 7fb5c8a0c700 1 -- 172.21.15.94:0/2780916325 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5c40fc260 con 0x7fb5c40fff00 2024-03-24T10:54:57.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.966+0000 7fb5b2ffd700 1 -- 172.21.15.94:0/2780916325 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb5b800bbf0 con 0x7fb5c40fff00 2024-03-24T10:54:57.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.966+0000 7fb5b2ffd700 1 -- 172.21.15.94:0/2780916325 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5b801c630 con 0x7fb5c40fff00 2024-03-24T10:54:57.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.966+0000 7fb5c8a0c700 1 -- 172.21.15.94:0/2780916325 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5c4115a90 con 0x7fb5c40fff00 2024-03-24T10:54:57.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.967+0000 7fb5c8a0c700 1 -- 172.21.15.94:0/2780916325 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5c405f5f0 con 0x7fb5c40fff00 2024-03-24T10:54:57.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.968+0000 7fb5b2ffd700 1 -- 172.21.15.94:0/2780916325 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fb5b800bd60 con 0x7fb5c40fff00 2024-03-24T10:54:57.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.969+0000 7fb5b2ffd700 1 --2- 172.21.15.94:0/2780916325 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fb5ac06d7e0 0x7fb5ac06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:54:57.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.969+0000 7fb5b2ffd700 1 -- 172.21.15.94:0/2780916325 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fb5b808da80 con 0x7fb5c40fff00 2024-03-24T10:54:57.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.969+0000 7fb5c259c700 1 --2- 172.21.15.94:0/2780916325 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fb5ac06d7e0 0x7fb5ac06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:54:57.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.969+0000 7fb5c259c700 1 --2- 172.21.15.94:0/2780916325 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fb5ac06d7e0 0x7fb5ac06fc90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb5c40fd180 tx=0x7fb5b400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:54:57.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:57.974+0000 7fb5b2ffd700 1 -- 172.21.15.94:0/2780916325 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb5b8057310 con 0x7fb5c40fff00 2024-03-24T10:54:58.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.208+0000 7fb5c8a0c700 1 -- 172.21.15.94:0/2780916325 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb5c40f8f70 con 0x7fb5ac06d7e0 2024-03-24T10:54:58.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.210+0000 7fb5b2ffd700 1 -- 172.21.15.94:0/2780916325 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+438 (secure 0 0 0) 0x7fb5c40f8f70 con 0x7fb5ac06d7e0 2024-03-24T10:54:58.211 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-03-24T10:54:58.211 INFO:teuthology.orchestra.run.smithi094.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919", 2024-03-24T10:54:58.211 INFO:teuthology.orchestra.run.smithi094.stdout: "in_progress": true, 2024-03-24T10:54:58.211 INFO:teuthology.orchestra.run.smithi094.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T10:54:58.212 INFO:teuthology.orchestra.run.smithi094.stdout: "services_complete": [ 2024-03-24T10:54:58.212 INFO:teuthology.orchestra.run.smithi094.stdout: "mds", 2024-03-24T10:54:58.212 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr", 2024-03-24T10:54:58.212 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-03-24T10:54:58.212 INFO:teuthology.orchestra.run.smithi094.stdout: "mon", 2024-03-24T10:54:58.212 INFO:teuthology.orchestra.run.smithi094.stdout: "osd" 2024-03-24T10:54:58.212 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-03-24T10:54:58.212 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": "16/21 daemons upgraded", 2024-03-24T10:54:58.212 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "Currently upgrading prometheus daemons", 2024-03-24T10:54:58.212 INFO:teuthology.orchestra.run.smithi094.stdout: "is_paused": false 2024-03-24T10:54:58.212 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-03-24T10:54:58.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.213+0000 7fb5c8a0c700 1 -- 172.21.15.94:0/2780916325 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fb5ac06d7e0 msgr2=0x7fb5ac06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:58.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.213+0000 7fb5c8a0c700 1 --2- 172.21.15.94:0/2780916325 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fb5ac06d7e0 0x7fb5ac06fc90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb5c40fd180 tx=0x7fb5b400b040 comp rx=0 tx=0).stop 2024-03-24T10:54:58.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.213+0000 7fb5c8a0c700 1 -- 172.21.15.94:0/2780916325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5c40fff00 msgr2=0x7fb5c40fb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:58.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.213+0000 7fb5c8a0c700 1 --2- 172.21.15.94:0/2780916325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5c40fff00 0x7fb5c40fb8c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb5b800b3b0 tx=0x7fb5b8009e30 comp rx=0 tx=0).stop 2024-03-24T10:54:58.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.213+0000 7fb5c8a0c700 1 -- 172.21.15.94:0/2780916325 shutdown_connections 2024-03-24T10:54:58.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.213+0000 7fb5c8a0c700 1 --2- 172.21.15.94:0/2780916325 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fb5ac06d7e0 0x7fb5ac06fc90 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:58.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.213+0000 7fb5c8a0c700 1 --2- 172.21.15.94:0/2780916325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5c40662e0 0x7fb5c40fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:58.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.213+0000 7fb5c8a0c700 1 --2- 172.21.15.94:0/2780916325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5c40fff00 0x7fb5c40fb8c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:58.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.213+0000 7fb5c8a0c700 1 -- 172.21.15.94:0/2780916325 >> 172.21.15.94:0/2780916325 conn(0x7fb5c40f7430 msgr2=0x7fb5c40f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:54:58.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.213+0000 7fb5c8a0c700 1 -- 172.21.15.94:0/2780916325 shutdown_connections 2024-03-24T10:54:58.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.213+0000 7fb5c8a0c700 1 -- 172.21.15.94:0/2780916325 wait complete. 2024-03-24T10:54:58.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.335+0000 7ff54bc20700 1 -- 172.21.15.94:0/2383024252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5440fcf80 msgr2=0x7ff5440fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:58.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.335+0000 7ff54bc20700 1 --2- 172.21.15.94:0/2383024252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5440fcf80 0x7ff5440fd3f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff534004940 tx=0x7ff534009e30 comp rx=0 tx=0).stop 2024-03-24T10:54:58.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.335+0000 7ff54bc20700 1 -- 172.21.15.94:0/2383024252 shutdown_connections 2024-03-24T10:54:58.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.335+0000 7ff54bc20700 1 --2- 172.21.15.94:0/2383024252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5440fcf80 0x7ff5440fd3f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:58.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.335+0000 7ff54bc20700 1 --2- 172.21.15.94:0/2383024252 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5440fbcd0 0x7ff5440fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:58.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.335+0000 7ff54bc20700 1 -- 172.21.15.94:0/2383024252 >> 172.21.15.94:0/2383024252 conn(0x7ff5440f7430 msgr2=0x7ff5440f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:54:58.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.335+0000 7ff54bc20700 1 -- 172.21.15.94:0/2383024252 shutdown_connections 2024-03-24T10:54:58.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.335+0000 7ff54bc20700 1 -- 172.21.15.94:0/2383024252 wait complete. 2024-03-24T10:54:58.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.336+0000 7ff54bc20700 1 Processor -- start 2024-03-24T10:54:58.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.336+0000 7ff54bc20700 1 -- start start 2024-03-24T10:54:58.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.336+0000 7ff54bc20700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5440fbcd0 0x7ff5440ffda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:54:58.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.336+0000 7ff54bc20700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5440fcf80 0x7ff5440fe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:54:58.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.336+0000 7ff54bc20700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5440fe850 con 0x7ff5440fbcd0 2024-03-24T10:54:58.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.336+0000 7ff54bc20700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5440fe990 con 0x7ff5440fcf80 2024-03-24T10:54:58.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.336+0000 7ff5499bc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5440fbcd0 0x7ff5440ffda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:54:58.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.336+0000 7ff5499bc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5440fbcd0 0x7ff5440ffda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36242/0 (socket says 172.21.15.94:36242) 2024-03-24T10:54:58.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.336+0000 7ff5499bc700 1 -- 172.21.15.94:0/948760001 learned_addr learned my addr 172.21.15.94:0/948760001 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:54:58.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.336+0000 7ff5491bb700 1 --2- 172.21.15.94:0/948760001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5440fcf80 0x7ff5440fe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:54:58.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.337+0000 7ff5499bc700 1 -- 172.21.15.94:0/948760001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5440fcf80 msgr2=0x7ff5440fe2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:58.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.337+0000 7ff5499bc700 1 --2- 172.21.15.94:0/948760001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5440fcf80 0x7ff5440fe2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:58.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.337+0000 7ff5499bc700 1 -- 172.21.15.94:0/948760001 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff534005040 con 0x7ff5440fbcd0 2024-03-24T10:54:58.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.337+0000 7ff5499bc700 1 --2- 172.21.15.94:0/948760001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5440fbcd0 0x7ff5440ffda0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff54000eb40 tx=0x7ff54000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:54:58.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.337+0000 7ff53a7fc700 1 -- 172.21.15.94:0/948760001 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff540007cc0 con 0x7ff5440fbcd0 2024-03-24T10:54:58.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.337+0000 7ff54bc20700 1 -- 172.21.15.94:0/948760001 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5440fec10 con 0x7ff5440fbcd0 2024-03-24T10:54:58.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.337+0000 7ff53a7fc700 1 -- 172.21.15.94:0/948760001 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff54000f040 con 0x7ff5440fbcd0 2024-03-24T10:54:58.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.337+0000 7ff53a7fc700 1 -- 172.21.15.94:0/948760001 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff54000ca10 con 0x7ff5440fbcd0 2024-03-24T10:54:58.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.337+0000 7ff54bc20700 1 -- 172.21.15.94:0/948760001 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5440ff120 con 0x7ff5440fbcd0 2024-03-24T10:54:58.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.338+0000 7ff54bc20700 1 -- 172.21.15.94:0/948760001 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff544101b10 con 0x7ff5440fbcd0 2024-03-24T10:54:58.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.339+0000 7ff53a7fc700 1 -- 172.21.15.94:0/948760001 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7ff54000cc80 con 0x7ff5440fbcd0 2024-03-24T10:54:58.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.339+0000 7ff53a7fc700 1 --2- 172.21.15.94:0/948760001 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7ff53006d7e0 0x7ff53006fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:54:58.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.339+0000 7ff53a7fc700 1 -- 172.21.15.94:0/948760001 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7ff54008cba0 con 0x7ff5440fbcd0 2024-03-24T10:54:58.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.339+0000 7ff5491bb700 1 --2- 172.21.15.94:0/948760001 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7ff53006d7e0 0x7ff53006fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:54:58.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.340+0000 7ff5491bb700 1 --2- 172.21.15.94:0/948760001 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7ff53006d7e0 0x7ff53006fc90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff534009b30 tx=0x7ff534009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:54:58.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.348+0000 7ff53a7fc700 1 -- 172.21.15.94:0/948760001 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff540056410 con 0x7ff5440fbcd0 2024-03-24T10:54:58.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:58 smithi094 ceph-mon[93957]: from='client.34470 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:54:58.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:58 smithi094 ceph-mon[93957]: from='client.34474 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:54:58.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:58 smithi094 ceph-mon[93957]: pgmap v229: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:54:58.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:58 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/2502715187' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:58.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:58 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/4221811842' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T10:54:58.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:58.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:58.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:54:58.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:54:58.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:58.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T10:54:58.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:54:58.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:58.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:58.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:58.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:58.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:58.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:58.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:58.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:58.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:58.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:58.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:58.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:58.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:58.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:58 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:58.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.646+0000 7ff54bc20700 1 -- 172.21.15.94:0/948760001 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff54405f5f0 con 0x7ff5440fbcd0 2024-03-24T10:54:58.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.646+0000 7ff53a7fc700 1 -- 172.21.15.94:0/948760001 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7ff540059ad0 con 0x7ff5440fbcd0 2024-03-24T10:54:58.647 INFO:teuthology.orchestra.run.smithi094.stdout:HEALTH_OK 2024-03-24T10:54:58.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.649+0000 7ff54bc20700 1 -- 172.21.15.94:0/948760001 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7ff53006d7e0 msgr2=0x7ff53006fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:58.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.649+0000 7ff54bc20700 1 --2- 172.21.15.94:0/948760001 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7ff53006d7e0 0x7ff53006fc90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff534009b30 tx=0x7ff534009e30 comp rx=0 tx=0).stop 2024-03-24T10:54:58.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.649+0000 7ff54bc20700 1 -- 172.21.15.94:0/948760001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5440fbcd0 msgr2=0x7ff5440ffda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:54:58.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.649+0000 7ff54bc20700 1 --2- 172.21.15.94:0/948760001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5440fbcd0 0x7ff5440ffda0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff54000eb40 tx=0x7ff54000b040 comp rx=0 tx=0).stop 2024-03-24T10:54:58.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.649+0000 7ff54bc20700 1 -- 172.21.15.94:0/948760001 shutdown_connections 2024-03-24T10:54:58.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.649+0000 7ff54bc20700 1 --2- 172.21.15.94:0/948760001 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7ff53006d7e0 0x7ff53006fc90 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:58.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.649+0000 7ff54bc20700 1 --2- 172.21.15.94:0/948760001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5440fbcd0 0x7ff5440ffda0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:58.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.649+0000 7ff54bc20700 1 --2- 172.21.15.94:0/948760001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5440fcf80 0x7ff5440fe2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:54:58.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.649+0000 7ff54bc20700 1 -- 172.21.15.94:0/948760001 >> 172.21.15.94:0/948760001 conn(0x7ff5440f7430 msgr2=0x7ff544065ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:54:58.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.649+0000 7ff54bc20700 1 -- 172.21.15.94:0/948760001 shutdown_connections 2024-03-24T10:54:58.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:54:58.649+0000 7ff54bc20700 1 -- 172.21.15.94:0/948760001 wait complete. 2024-03-24T10:54:58.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:58 smithi096 ceph-mon[84258]: from='client.34470 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:54:58.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:58 smithi096 ceph-mon[84258]: from='client.34474 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:54:58.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:58 smithi096 ceph-mon[84258]: pgmap v229: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:54:58.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:58 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/2502715187' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:58.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:58 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/4221811842' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T10:54:58.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:58.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:58.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:54:58.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:54:58.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:54:58.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T10:54:58.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:54:58.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:58.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:58.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:58.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:58.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:58.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:58.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:58.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:58.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:58.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:58.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:58.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:58.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:58.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:58 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:54:59.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:59 smithi094 ceph-mon[93957]: from='client.34486 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:54:59.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:59 smithi094 ceph-mon[93957]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T10:54:59.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:54:59 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/948760001' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T10:54:59.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:59 smithi096 ceph-mon[84258]: from='client.34486 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:54:59.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:59 smithi096 ceph-mon[84258]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T10:54:59.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:54:59 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/948760001' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T10:55:00.726 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:00 smithi096 ceph-mon[84258]: pgmap v230: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:55:00.727 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:00 smithi096 ceph-mon[84258]: Upgrade: Updating alertmanager.smithi094 2024-03-24T10:55:00.727 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:00 smithi096 ceph-mon[84258]: Deploying daemon alertmanager.smithi094 on smithi094 2024-03-24T10:55:00.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:00 smithi094 ceph-mon[93957]: pgmap v230: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:55:00.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:00 smithi094 ceph-mon[93957]: Upgrade: Updating alertmanager.smithi094 2024-03-24T10:55:00.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:00 smithi094 ceph-mon[93957]: Deploying daemon alertmanager.smithi094 on smithi094 2024-03-24T10:55:01.632 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:01 smithi094 ceph-mon[93957]: pgmap v231: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:55:01.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:01 smithi096 ceph-mon[84258]: pgmap v231: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:55:04.650 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:04 smithi094 ceph-mon[93957]: pgmap v232: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-24T10:55:04.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:04 smithi096 ceph-mon[84258]: pgmap v232: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-24T10:55:05.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:05 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:55:05.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:05 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:55:06.649 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:06 smithi094 ceph-mon[93957]: pgmap v233: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:55:06.649 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:06 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:55:06.649 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:06 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:55:06.649 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:06 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:55:06.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:06 smithi096 ceph-mon[84258]: pgmap v233: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:55:06.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:06 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:55:06.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:06 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:55:06.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:06 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:55:08.712 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:08 smithi094 ceph-mon[93957]: pgmap v234: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:55:08.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:08 smithi096 ceph-mon[84258]: pgmap v234: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:55:10.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:10 smithi094 ceph-mon[93957]: pgmap v235: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:55:10.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:10 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:55:10.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:10 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:55:10.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:10 smithi096 ceph-mon[84258]: pgmap v235: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:55:10.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:10 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:55:10.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:10 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:55:12.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:12 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:55:12.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:12 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:55:12.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:12 smithi094 ceph-mon[93957]: pgmap v236: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:55:12.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:12 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:55:12.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:12 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:55:12.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:12 smithi096 ceph-mon[84258]: pgmap v236: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:55:14.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:14 smithi094 ceph-mon[93957]: pgmap v237: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-24T10:55:14.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:14 smithi096 ceph-mon[84258]: pgmap v237: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-24T10:55:15.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:15 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:55:15.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:15 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:55:15.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:15 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:55:15.750 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:15 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:55:15.750 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:15 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:55:15.750 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:15 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-24T10:55:15.750 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:15 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:55:15.750 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:15 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:15.750 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:15 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:15.750 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:15 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:15.750 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:15 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:15.750 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:15 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:15.750 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:15 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:15.750 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:15 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:15.750 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:15 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:15.750 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:15 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:15.751 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:15 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:15.751 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:15 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:15.751 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:15 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:15.751 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:15 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:15.751 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:15 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:15.751 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:15 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:15.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:15 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:55:15.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:15 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:55:15.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:15 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:55:15.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:15 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:55:15.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:15 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:55:15.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:15 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-24T10:55:15.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:15 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:55:15.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:15 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:15.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:15 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:15.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:15 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:15.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:15 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:15.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:15 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:15.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:15 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:15.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:15 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:15.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:15 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:15.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:15 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:15.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:15 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:15.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:15 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:15.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:15 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:15.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:15 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:15.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:15 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:15.827 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:15 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:16.687 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:16 smithi094 ceph-mon[93957]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-24T10:55:16.687 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:16 smithi094 ceph-mon[93957]: pgmap v238: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:55:16.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:16 smithi096 ceph-mon[84258]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-24T10:55:16.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:16 smithi096 ceph-mon[84258]: pgmap v238: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:55:17.609 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:17 smithi094 ceph-mon[93957]: Upgrade: Updating grafana.smithi094 2024-03-24T10:55:17.609 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:17 smithi094 ceph-mon[93957]: Deploying daemon grafana.smithi094 on smithi094 2024-03-24T10:55:17.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:17 smithi096 ceph-mon[84258]: Upgrade: Updating grafana.smithi094 2024-03-24T10:55:17.826 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:17 smithi096 ceph-mon[84258]: Deploying daemon grafana.smithi094 on smithi094 2024-03-24T10:55:18.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:18 smithi094 ceph-mon[93957]: pgmap v239: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:55:18.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:18 smithi096 ceph-mon[84258]: pgmap v239: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:55:20.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:20 smithi094 ceph-mon[93957]: pgmap v240: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:55:20.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:20 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:55:20.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:20 smithi096 ceph-mon[84258]: pgmap v240: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:55:20.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:20 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:55:21.992 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:21 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:55:21.992 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:21 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:55:21.992 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:21 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:55:21.992 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:21 smithi094 ceph-mon[93957]: pgmap v241: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:55:22.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:21 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:55:22.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:21 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:55:22.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:21 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:55:22.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:21 smithi096 ceph-mon[84258]: pgmap v241: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:55:24.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:24 smithi094 ceph-mon[93957]: pgmap v242: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-24T10:55:24.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:24 smithi096 ceph-mon[84258]: pgmap v242: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-24T10:55:26.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:26 smithi094 ceph-mon[93957]: pgmap v243: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:55:26.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:26 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:55:26.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:26 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:55:26.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:26 smithi096 ceph-mon[84258]: pgmap v243: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:55:26.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:26 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:55:26.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:26 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:55:27.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:27 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:55:27.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:27 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:55:27.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:27 smithi094 ceph-mon[93957]: pgmap v244: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:55:27.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:27 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:55:27.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:27 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:55:27.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:27 smithi096 ceph-mon[84258]: pgmap v244: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:55:28.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:28.788+0000 7f85f9fef700 1 -- 172.21.15.94:0/301608689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85f40fed90 msgr2=0x7f85f40faed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:55:28.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:28.788+0000 7f85f9fef700 1 --2- 172.21.15.94:0/301608689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85f40fed90 0x7f85f40faed0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f85e4004940 tx=0x7f85e4009e30 comp rx=0 tx=0).stop 2024-03-24T10:55:28.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:28.788+0000 7f85f9fef700 1 -- 172.21.15.94:0/301608689 shutdown_connections 2024-03-24T10:55:28.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:28.788+0000 7f85f9fef700 1 --2- 172.21.15.94:0/301608689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85f40fed90 0x7f85f40faed0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:55:28.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:28.788+0000 7f85f9fef700 1 --2- 172.21.15.94:0/301608689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85f40fe440 0x7f85f40fe850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:55:28.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:28.788+0000 7f85f9fef700 1 -- 172.21.15.94:0/301608689 >> 172.21.15.94:0/301608689 conn(0x7f85f40f6be0 msgr2=0x7f85f40f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:55:28.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:28.788+0000 7f85f9fef700 1 -- 172.21.15.94:0/301608689 shutdown_connections 2024-03-24T10:55:28.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:28.788+0000 7f85f9fef700 1 -- 172.21.15.94:0/301608689 wait complete. 2024-03-24T10:55:28.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:28.789+0000 7f85f9fef700 1 Processor -- start 2024-03-24T10:55:28.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:28.789+0000 7f85f9fef700 1 -- start start 2024-03-24T10:55:28.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:28.789+0000 7f85f9fef700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85f40fe440 0x7f85f410d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:55:28.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:28.789+0000 7f85f9fef700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85f40fed90 0x7f85f410de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:55:28.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:28.789+0000 7f85f9fef700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f410e450 con 0x7f85f40fe440 2024-03-24T10:55:28.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:28.789+0000 7f85f9fef700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f410e5c0 con 0x7f85f40fed90 2024-03-24T10:55:28.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:28.789+0000 7f85f37fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85f40fe440 0x7f85f410d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:55:28.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:28.789+0000 7f85f37fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85f40fe440 0x7f85f410d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39200/0 (socket says 172.21.15.94:39200) 2024-03-24T10:55:28.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:28.789+0000 7f85f37fe700 1 -- 172.21.15.94:0/2303419639 learned_addr learned my addr 172.21.15.94:0/2303419639 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:55:28.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:28.790+0000 7f85f2ffd700 1 --2- 172.21.15.94:0/2303419639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85f40fed90 0x7f85f410de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:55:28.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:28.790+0000 7f85f37fe700 1 -- 172.21.15.94:0/2303419639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85f40fed90 msgr2=0x7f85f410de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:55:28.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:28.790+0000 7f85f37fe700 1 --2- 172.21.15.94:0/2303419639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85f40fed90 0x7f85f410de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:55:28.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:28.790+0000 7f85f37fe700 1 -- 172.21.15.94:0/2303419639 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e4005040 con 0x7f85f40fe440 2024-03-24T10:55:28.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:28.790+0000 7f85f2ffd700 1 --2- 172.21.15.94:0/2303419639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85f40fed90 0x7f85f410de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T10:55:28.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:28.790+0000 7f85f37fe700 1 --2- 172.21.15.94:0/2303419639 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85f40fe440 0x7f85f410d910 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f85e800c830 tx=0x7f85e8004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:55:28.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:28.790+0000 7f85dbfff700 1 -- 172.21.15.94:0/2303419639 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85e800a760 con 0x7f85f40fe440 2024-03-24T10:55:28.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:28.790+0000 7f85dbfff700 1 -- 172.21.15.94:0/2303419639 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f85e800ad90 con 0x7f85f40fe440 2024-03-24T10:55:28.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:28.790+0000 7f85dbfff700 1 -- 172.21.15.94:0/2303419639 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85e80113a0 con 0x7f85f40fe440 2024-03-24T10:55:28.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:28.790+0000 7f85f9fef700 1 -- 172.21.15.94:0/2303419639 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85f4112fc0 con 0x7f85f40fe440 2024-03-24T10:55:28.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:28.790+0000 7f85f9fef700 1 -- 172.21.15.94:0/2303419639 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85f41134f0 con 0x7f85f40fe440 2024-03-24T10:55:28.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:28.791+0000 7f85f9fef700 1 -- 172.21.15.94:0/2303419639 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85f404c710 con 0x7f85f40fe440 2024-03-24T10:55:28.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:28.792+0000 7f85dbfff700 1 -- 172.21.15.94:0/2303419639 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f85e800a8c0 con 0x7f85f40fe440 2024-03-24T10:55:28.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:28.792+0000 7f85dbfff700 1 --2- 172.21.15.94:0/2303419639 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f85dc06d7e0 0x7f85dc06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:55:28.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:28.792+0000 7f85dbfff700 1 -- 172.21.15.94:0/2303419639 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f85e808ce90 con 0x7f85f40fe440 2024-03-24T10:55:28.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:28.792+0000 7f85f2ffd700 1 --2- 172.21.15.94:0/2303419639 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f85dc06d7e0 0x7f85dc06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:55:28.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:28.793+0000 7f85f2ffd700 1 --2- 172.21.15.94:0/2303419639 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f85dc06d7e0 0x7f85dc06fc90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f85e400b7a0 tx=0x7f85e4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:55:28.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:28.797+0000 7f85dbfff700 1 -- 172.21.15.94:0/2303419639 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f85e805a250 con 0x7f85f40fe440 2024-03-24T10:55:29.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.038+0000 7f85f9fef700 1 -- 172.21.15.94:0/2303419639 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f85f41137c0 con 0x7f85dc06d7e0 2024-03-24T10:55:29.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.039+0000 7f85dbfff700 1 -- 172.21.15.94:0/2303419639 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+435 (secure 0 0 0) 0x7f85f41137c0 con 0x7f85dc06d7e0 2024-03-24T10:55:29.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.042+0000 7f85f9fef700 1 -- 172.21.15.94:0/2303419639 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f85dc06d7e0 msgr2=0x7f85dc06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:55:29.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.042+0000 7f85f9fef700 1 --2- 172.21.15.94:0/2303419639 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f85dc06d7e0 0x7f85dc06fc90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f85e400b7a0 tx=0x7f85e4009e30 comp rx=0 tx=0).stop 2024-03-24T10:55:29.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.042+0000 7f85f9fef700 1 -- 172.21.15.94:0/2303419639 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85f40fe440 msgr2=0x7f85f410d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:55:29.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.042+0000 7f85f9fef700 1 --2- 172.21.15.94:0/2303419639 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85f40fe440 0x7f85f410d910 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f85e800c830 tx=0x7f85e8004470 comp rx=0 tx=0).stop 2024-03-24T10:55:29.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.042+0000 7f85f9fef700 1 -- 172.21.15.94:0/2303419639 shutdown_connections 2024-03-24T10:55:29.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.042+0000 7f85f9fef700 1 --2- 172.21.15.94:0/2303419639 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f85dc06d7e0 0x7f85dc06fc90 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:55:29.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.042+0000 7f85f9fef700 1 --2- 172.21.15.94:0/2303419639 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85f40fe440 0x7f85f410d910 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:55:29.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.042+0000 7f85f9fef700 1 --2- 172.21.15.94:0/2303419639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85f40fed90 0x7f85f410de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:55:29.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.042+0000 7f85f9fef700 1 -- 172.21.15.94:0/2303419639 >> 172.21.15.94:0/2303419639 conn(0x7f85f40f6be0 msgr2=0x7f85f40f8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:55:29.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.042+0000 7f85f9fef700 1 -- 172.21.15.94:0/2303419639 shutdown_connections 2024-03-24T10:55:29.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.042+0000 7f85f9fef700 1 -- 172.21.15.94:0/2303419639 wait complete. 2024-03-24T10:55:29.055 INFO:teuthology.orchestra.run.smithi094.stdout:true 2024-03-24T10:55:29.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.170+0000 7f5d41b50700 1 -- 172.21.15.94:0/630950340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d3c0fbcd0 msgr2=0x7f5d3c0fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:55:29.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.170+0000 7f5d41b50700 1 --2- 172.21.15.94:0/630950340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d3c0fbcd0 0x7f5d3c0fc0e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c004940 tx=0x7f5d2c009e30 comp rx=0 tx=0).stop 2024-03-24T10:55:29.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.170+0000 7f5d41b50700 1 -- 172.21.15.94:0/630950340 shutdown_connections 2024-03-24T10:55:29.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.170+0000 7f5d41b50700 1 --2- 172.21.15.94:0/630950340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d3c0fcf80 0x7f5d3c0fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:55:29.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.170+0000 7f5d41b50700 1 --2- 172.21.15.94:0/630950340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d3c0fbcd0 0x7f5d3c0fc0e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:55:29.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.170+0000 7f5d41b50700 1 -- 172.21.15.94:0/630950340 >> 172.21.15.94:0/630950340 conn(0x7f5d3c0f7430 msgr2=0x7f5d3c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:55:29.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.170+0000 7f5d41b50700 1 -- 172.21.15.94:0/630950340 shutdown_connections 2024-03-24T10:55:29.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.171+0000 7f5d41b50700 1 -- 172.21.15.94:0/630950340 wait complete. 2024-03-24T10:55:29.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.171+0000 7f5d41b50700 1 Processor -- start 2024-03-24T10:55:29.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.171+0000 7f5d41b50700 1 -- start start 2024-03-24T10:55:29.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.172+0000 7f5d41b50700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d3c0fbcd0 0x7f5d3c110700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:55:29.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.172+0000 7f5d41b50700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d3c0fcf80 0x7f5d3c112c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:55:29.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.172+0000 7f5d41b50700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c066370 con 0x7f5d3c0fcf80 2024-03-24T10:55:29.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.172+0000 7f5d41b50700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c0664b0 con 0x7f5d3c0fbcd0 2024-03-24T10:55:29.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.172+0000 7f5d3affd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d3c0fcf80 0x7f5d3c112c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:55:29.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.172+0000 7f5d3affd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d3c0fcf80 0x7f5d3c112c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39214/0 (socket says 172.21.15.94:39214) 2024-03-24T10:55:29.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.172+0000 7f5d3affd700 1 -- 172.21.15.94:0/931052529 learned_addr learned my addr 172.21.15.94:0/931052529 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:55:29.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.172+0000 7f5d3b7fe700 1 --2- 172.21.15.94:0/931052529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d3c0fbcd0 0x7f5d3c110700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:55:29.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.172+0000 7f5d3affd700 1 -- 172.21.15.94:0/931052529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d3c0fbcd0 msgr2=0x7f5d3c110700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:55:29.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.172+0000 7f5d3affd700 1 --2- 172.21.15.94:0/931052529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d3c0fbcd0 0x7f5d3c110700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:55:29.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.172+0000 7f5d3affd700 1 -- 172.21.15.94:0/931052529 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d2c005040 con 0x7f5d3c0fcf80 2024-03-24T10:55:29.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.172+0000 7f5d3b7fe700 1 --2- 172.21.15.94:0/931052529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d3c0fbcd0 0x7f5d3c110700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T10:55:29.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.172+0000 7f5d3affd700 1 --2- 172.21.15.94:0/931052529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d3c0fcf80 0x7f5d3c112c50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5d30002a50 tx=0x7f5d300046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:55:29.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.172+0000 7f5d23fff700 1 -- 172.21.15.94:0/931052529 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d30007cc0 con 0x7f5d3c0fcf80 2024-03-24T10:55:29.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.173+0000 7f5d23fff700 1 -- 172.21.15.94:0/931052529 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5d3000f040 con 0x7f5d3c0fcf80 2024-03-24T10:55:29.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.173+0000 7f5d23fff700 1 -- 172.21.15.94:0/931052529 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d30009640 con 0x7f5d3c0fcf80 2024-03-24T10:55:29.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.173+0000 7f5d41b50700 1 -- 172.21.15.94:0/931052529 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d3c113250 con 0x7f5d3c0fcf80 2024-03-24T10:55:29.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.173+0000 7f5d41b50700 1 -- 172.21.15.94:0/931052529 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d3c1137e0 con 0x7f5d3c0fcf80 2024-03-24T10:55:29.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.174+0000 7f5d41b50700 1 -- 172.21.15.94:0/931052529 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d3c05f5f0 con 0x7f5d3c0fcf80 2024-03-24T10:55:29.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.174+0000 7f5d23fff700 1 -- 172.21.15.94:0/931052529 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f5d300097c0 con 0x7f5d3c0fcf80 2024-03-24T10:55:29.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.175+0000 7f5d23fff700 1 --2- 172.21.15.94:0/931052529 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f5d2406d7e0 0x7f5d2406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:55:29.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.175+0000 7f5d23fff700 1 -- 172.21.15.94:0/931052529 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f5d3008cc80 con 0x7f5d3c0fcf80 2024-03-24T10:55:29.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.175+0000 7f5d3b7fe700 1 --2- 172.21.15.94:0/931052529 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f5d2406d7e0 0x7f5d2406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:55:29.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.175+0000 7f5d3b7fe700 1 --2- 172.21.15.94:0/931052529 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f5d2406d7e0 0x7f5d2406fc90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c009fd0 tx=0x7f5d2c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:55:29.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.181+0000 7f5d23fff700 1 -- 172.21.15.94:0/931052529 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5d30056450 con 0x7f5d3c0fcf80 2024-03-24T10:55:29.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.457+0000 7f5d41b50700 1 -- 172.21.15.94:0/931052529 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5d3c101290 con 0x7f5d2406d7e0 2024-03-24T10:55:29.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.458+0000 7f5d23fff700 1 -- 172.21.15.94:0/931052529 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+435 (secure 0 0 0) 0x7f5d3c101290 con 0x7f5d2406d7e0 2024-03-24T10:55:29.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.460+0000 7f5d41b50700 1 -- 172.21.15.94:0/931052529 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f5d2406d7e0 msgr2=0x7f5d2406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:55:29.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.460+0000 7f5d41b50700 1 --2- 172.21.15.94:0/931052529 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f5d2406d7e0 0x7f5d2406fc90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c009fd0 tx=0x7f5d2c009e30 comp rx=0 tx=0).stop 2024-03-24T10:55:29.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.460+0000 7f5d41b50700 1 -- 172.21.15.94:0/931052529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d3c0fcf80 msgr2=0x7f5d3c112c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:55:29.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.460+0000 7f5d41b50700 1 --2- 172.21.15.94:0/931052529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d3c0fcf80 0x7f5d3c112c50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5d30002a50 tx=0x7f5d300046b0 comp rx=0 tx=0).stop 2024-03-24T10:55:29.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.460+0000 7f5d41b50700 1 -- 172.21.15.94:0/931052529 shutdown_connections 2024-03-24T10:55:29.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.460+0000 7f5d41b50700 1 --2- 172.21.15.94:0/931052529 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f5d2406d7e0 0x7f5d2406fc90 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:55:29.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.460+0000 7f5d41b50700 1 --2- 172.21.15.94:0/931052529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d3c0fbcd0 0x7f5d3c110700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:55:29.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.460+0000 7f5d41b50700 1 --2- 172.21.15.94:0/931052529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d3c0fcf80 0x7f5d3c112c50 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:55:29.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.460+0000 7f5d41b50700 1 -- 172.21.15.94:0/931052529 >> 172.21.15.94:0/931052529 conn(0x7f5d3c0f7430 msgr2=0x7f5d3c0f8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:55:29.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.460+0000 7f5d41b50700 1 -- 172.21.15.94:0/931052529 shutdown_connections 2024-03-24T10:55:29.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.460+0000 7f5d41b50700 1 -- 172.21.15.94:0/931052529 wait complete. 2024-03-24T10:55:29.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.564+0000 7fc0db61b700 1 -- 172.21.15.94:0/459350734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0d40fb1c0 msgr2=0x7fc0d40fb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:55:29.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.564+0000 7fc0db61b700 1 --2- 172.21.15.94:0/459350734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0d40fb1c0 0x7fc0d40fb5d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0004940 tx=0x7fc0d0009e30 comp rx=0 tx=0).stop 2024-03-24T10:55:29.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.565+0000 7fc0db61b700 1 -- 172.21.15.94:0/459350734 shutdown_connections 2024-03-24T10:55:29.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.565+0000 7fc0db61b700 1 --2- 172.21.15.94:0/459350734 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0d40fbb10 0x7fc0d40fe370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:55:29.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.565+0000 7fc0db61b700 1 --2- 172.21.15.94:0/459350734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0d40fb1c0 0x7fc0d40fb5d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:55:29.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.565+0000 7fc0db61b700 1 -- 172.21.15.94:0/459350734 >> 172.21.15.94:0/459350734 conn(0x7fc0d40f6ba0 msgr2=0x7fc0d40f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:55:29.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.565+0000 7fc0db61b700 1 -- 172.21.15.94:0/459350734 shutdown_connections 2024-03-24T10:55:29.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.565+0000 7fc0db61b700 1 -- 172.21.15.94:0/459350734 wait complete. 2024-03-24T10:55:29.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.565+0000 7fc0db61b700 1 Processor -- start 2024-03-24T10:55:29.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.565+0000 7fc0db61b700 1 -- start start 2024-03-24T10:55:29.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.565+0000 7fc0db61b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0d40fb1c0 0x7fc0d40fdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:55:29.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.565+0000 7fc0db61b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0d40fbb10 0x7fc0d40fc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:55:29.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.565+0000 7fc0db61b700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d40fc7e0 con 0x7fc0d40fbb10 2024-03-24T10:55:29.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.565+0000 7fc0db61b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d40fc950 con 0x7fc0d40fb1c0 2024-03-24T10:55:29.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.566+0000 7fc0d8bb6700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0d40fbb10 0x7fc0d40fc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:55:29.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.566+0000 7fc0d8bb6700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0d40fbb10 0x7fc0d40fc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39228/0 (socket says 172.21.15.94:39228) 2024-03-24T10:55:29.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.566+0000 7fc0d8bb6700 1 -- 172.21.15.94:0/3835664563 learned_addr learned my addr 172.21.15.94:0/3835664563 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:55:29.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.566+0000 7fc0d93b7700 1 --2- 172.21.15.94:0/3835664563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0d40fb1c0 0x7fc0d40fdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:55:29.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.566+0000 7fc0d8bb6700 1 -- 172.21.15.94:0/3835664563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0d40fb1c0 msgr2=0x7fc0d40fdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:55:29.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.566+0000 7fc0d8bb6700 1 --2- 172.21.15.94:0/3835664563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0d40fb1c0 0x7fc0d40fdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:55:29.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.566+0000 7fc0d8bb6700 1 -- 172.21.15.94:0/3835664563 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d0005040 con 0x7fc0d40fbb10 2024-03-24T10:55:29.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.566+0000 7fc0d8bb6700 1 --2- 172.21.15.94:0/3835664563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0d40fbb10 0x7fc0d40fc1a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc0c400b790 tx=0x7fc0c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:55:29.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.566+0000 7fc0c9ffb700 1 -- 172.21.15.94:0/3835664563 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0c400f770 con 0x7fc0d40fbb10 2024-03-24T10:55:29.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.566+0000 7fc0db61b700 1 -- 172.21.15.94:0/3835664563 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0d40fcc30 con 0x7fc0d40fbb10 2024-03-24T10:55:29.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.566+0000 7fc0c9ffb700 1 -- 172.21.15.94:0/3835664563 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc0c400fda0 con 0x7fc0d40fbb10 2024-03-24T10:55:29.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.566+0000 7fc0c9ffb700 1 -- 172.21.15.94:0/3835664563 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0c400d310 con 0x7fc0d40fbb10 2024-03-24T10:55:29.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.567+0000 7fc0db61b700 1 -- 172.21.15.94:0/3835664563 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0d4115d10 con 0x7fc0d40fbb10 2024-03-24T10:55:29.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.567+0000 7fc0db61b700 1 -- 172.21.15.94:0/3835664563 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0d405f5f0 con 0x7fc0d40fbb10 2024-03-24T10:55:29.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.569+0000 7fc0c9ffb700 1 -- 172.21.15.94:0/3835664563 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fc0c4010420 con 0x7fc0d40fbb10 2024-03-24T10:55:29.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.569+0000 7fc0c9ffb700 1 --2- 172.21.15.94:0/3835664563 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fc0c006d7e0 0x7fc0c006fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:55:29.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.569+0000 7fc0c9ffb700 1 -- 172.21.15.94:0/3835664563 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fc0c408cff0 con 0x7fc0d40fbb10 2024-03-24T10:55:29.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.569+0000 7fc0d93b7700 1 --2- 172.21.15.94:0/3835664563 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fc0c006d7e0 0x7fc0c006fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:55:29.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.570+0000 7fc0d93b7700 1 --2- 172.21.15.94:0/3835664563 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fc0c006d7e0 0x7fc0c006fc90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc0d000b970 tx=0x7fc0d0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:55:29.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.578+0000 7fc0c9ffb700 1 -- 172.21.15.94:0/3835664563 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc0c40567c0 con 0x7fc0d40fbb10 2024-03-24T10:55:29.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.798+0000 7fc0db61b700 1 -- 172.21.15.94:0/3835664563 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc0d4102660 con 0x7fc0c006d7e0 2024-03-24T10:55:29.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.818+0000 7fc0c9ffb700 1 -- 172.21.15.94:0/3835664563 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fc0d4102660 con 0x7fc0c006d7e0 2024-03-24T10:55:29.817 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T10:55:29.818 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.smithi094 smithi094 *:9093,9094 running (24s) 4s ago 17m 15.0M - 0.25.0 c8568f914cd2 aca214c66fba 2024-03-24T10:55:29.818 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi094 smithi094 running (5m) 4s ago 17m 6651k - 19.0.0-1855-g21b3bd67 553463a90fd5 2b0897df571d 2024-03-24T10:55:29.818 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi096 smithi096 running (5m) 58s ago 16m 6669k - 19.0.0-1855-g21b3bd67 553463a90fd5 1759a9ff2805 2024-03-24T10:55:29.818 INFO:teuthology.orchestra.run.smithi094.stdout:grafana.smithi094 smithi094 *:3000 running (9s) 4s ago 16m 54.3M - 9.4.12 a0ea5e3c46d9 d506da6aef36 2024-03-24T10:55:29.818 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.efiqte smithi094 running (2m) 4s ago 11m 85.9M - 19.0.0-1855-g21b3bd67 553463a90fd5 732c47b17971 2024-03-24T10:55:29.818 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.llefgu smithi094 running (2m) 4s ago 11m 87.2M - 19.0.0-1855-g21b3bd67 553463a90fd5 6e06f8a6dde0 2024-03-24T10:55:29.818 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi096.egbmoq smithi096 running (100s) 58s ago 11m 14.1M - 19.0.0-1855-g21b3bd67 553463a90fd5 f35fd7f23742 2024-03-24T10:55:29.818 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi096.jtebhh smithi096 running (79s) 58s ago 11m 13.9M - 19.0.0-1855-g21b3bd67 553463a90fd5 8b5ab9f96e5a 2024-03-24T10:55:29.818 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi094.yaethu smithi094 *:8443,9283,8765 running (7m) 4s ago 19m 511M - 19.0.0-1855-g21b3bd67 553463a90fd5 d3adab76b18f 2024-03-24T10:55:29.818 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi096.gfonnu smithi096 *:8443,9283,8765 running (7m) 58s ago 16m 450M - 19.0.0-1855-g21b3bd67 553463a90fd5 22937a059eb8 2024-03-24T10:55:29.818 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi094 smithi094 running (7m) 4s ago 19m 56.9M 2048M 19.0.0-1855-g21b3bd67 553463a90fd5 c779d611e29a 2024-03-24T10:55:29.818 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi096 smithi096 running (6m) 58s ago 16m 38.5M 2048M 19.0.0-1855-g21b3bd67 553463a90fd5 ed9bd0bd57f6 2024-03-24T10:55:29.818 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi094 smithi094 *:9100 running (66s) 4s ago 17m 7860k - 1.5.0 0da6a335fe13 003df2346f4f 2024-03-24T10:55:29.818 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi096 smithi096 *:9100 running (61s) 58s ago 16m 5481k - 1.5.0 0da6a335fe13 67d65fecf9df 2024-03-24T10:55:29.818 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (5m) 4s ago 15m 111M 1557M 19.0.0-1855-g21b3bd67 553463a90fd5 ec57ede0c9e4 2024-03-24T10:55:29.819 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi094 running (4m) 4s ago 14m 124M 1557M 19.0.0-1855-g21b3bd67 553463a90fd5 c5b8b2bbf8b5 2024-03-24T10:55:29.819 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi094 running (4m) 4s ago 14m 102M 1557M 19.0.0-1855-g21b3bd67 553463a90fd5 c961dcdc746f 2024-03-24T10:55:29.819 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi096 running (3m) 58s ago 14m 121M 2581M 19.0.0-1855-g21b3bd67 553463a90fd5 6323fdc6e6ca 2024-03-24T10:55:29.819 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi096 running (3m) 58s ago 13m 111M 2581M 19.0.0-1855-g21b3bd67 553463a90fd5 3e2536bcb263 2024-03-24T10:55:29.819 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi096 running (2m) 58s ago 13m 103M 2581M 19.0.0-1855-g21b3bd67 553463a90fd5 2f77f1fa20e2 2024-03-24T10:55:29.819 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.smithi094 smithi094 *:9095 running (41s) 4s ago 16m 44.8M - 2.43.0 a07b618ecd1d 326ddd336bc5 2024-03-24T10:55:29.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.820+0000 7fc0db61b700 1 -- 172.21.15.94:0/3835664563 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fc0c006d7e0 msgr2=0x7fc0c006fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:55:29.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.820+0000 7fc0db61b700 1 --2- 172.21.15.94:0/3835664563 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fc0c006d7e0 0x7fc0c006fc90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc0d000b970 tx=0x7fc0d0009e30 comp rx=0 tx=0).stop 2024-03-24T10:55:29.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.820+0000 7fc0db61b700 1 -- 172.21.15.94:0/3835664563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0d40fbb10 msgr2=0x7fc0d40fc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:55:29.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.820+0000 7fc0db61b700 1 --2- 172.21.15.94:0/3835664563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0d40fbb10 0x7fc0d40fc1a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc0c400b790 tx=0x7fc0c4009250 comp rx=0 tx=0).stop 2024-03-24T10:55:29.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.820+0000 7fc0db61b700 1 -- 172.21.15.94:0/3835664563 shutdown_connections 2024-03-24T10:55:29.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.820+0000 7fc0db61b700 1 --2- 172.21.15.94:0/3835664563 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fc0c006d7e0 0x7fc0c006fc90 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:55:29.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.820+0000 7fc0db61b700 1 --2- 172.21.15.94:0/3835664563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0d40fb1c0 0x7fc0d40fdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:55:29.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.820+0000 7fc0db61b700 1 --2- 172.21.15.94:0/3835664563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0d40fbb10 0x7fc0d40fc1a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:55:29.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.820+0000 7fc0db61b700 1 -- 172.21.15.94:0/3835664563 >> 172.21.15.94:0/3835664563 conn(0x7fc0d40f6ba0 msgr2=0x7fc0d40f8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:55:29.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.820+0000 7fc0db61b700 1 -- 172.21.15.94:0/3835664563 shutdown_connections 2024-03-24T10:55:29.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.821+0000 7fc0db61b700 1 -- 172.21.15.94:0/3835664563 wait complete. 2024-03-24T10:55:29.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.927+0000 7f8c26676700 1 -- 172.21.15.94:0/3463849768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c200662e0 msgr2=0x7f8c200ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:55:29.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.927+0000 7f8c26676700 1 --2- 172.21.15.94:0/3463849768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c200662e0 0x7f8c200ff9c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8c10004940 tx=0x7f8c10009e30 comp rx=0 tx=0).stop 2024-03-24T10:55:29.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.927+0000 7f8c26676700 1 -- 172.21.15.94:0/3463849768 shutdown_connections 2024-03-24T10:55:29.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.927+0000 7f8c26676700 1 --2- 172.21.15.94:0/3463849768 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c200fff00 0x7f8c20102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:55:29.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.927+0000 7f8c26676700 1 --2- 172.21.15.94:0/3463849768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c200662e0 0x7f8c200ff9c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:55:29.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.927+0000 7f8c26676700 1 -- 172.21.15.94:0/3463849768 >> 172.21.15.94:0/3463849768 conn(0x7f8c200f7430 msgr2=0x7f8c200f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:55:29.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.927+0000 7f8c26676700 1 -- 172.21.15.94:0/3463849768 shutdown_connections 2024-03-24T10:55:29.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.927+0000 7f8c26676700 1 -- 172.21.15.94:0/3463849768 wait complete. 2024-03-24T10:55:29.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.928+0000 7f8c26676700 1 Processor -- start 2024-03-24T10:55:29.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.928+0000 7f8c26676700 1 -- start start 2024-03-24T10:55:29.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.928+0000 7f8c26676700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c200662e0 0x7f8c2010dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:55:29.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.928+0000 7f8c26676700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c200fff00 0x7f8c2010e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:55:29.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.928+0000 7f8c26676700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2010e6a0 con 0x7f8c200fff00 2024-03-24T10:55:29.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.928+0000 7f8c26676700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2010edf0 con 0x7f8c200662e0 2024-03-24T10:55:29.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.928+0000 7f8c1f7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c200fff00 0x7f8c2010e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:55:29.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.928+0000 7f8c1f7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c200fff00 0x7f8c2010e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38296/0 (socket says 172.21.15.94:38296) 2024-03-24T10:55:29.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.928+0000 7f8c1f7fe700 1 -- 172.21.15.94:0/1493776406 learned_addr learned my addr 172.21.15.94:0/1493776406 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:55:29.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.929+0000 7f8c1ffff700 1 --2- 172.21.15.94:0/1493776406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c200662e0 0x7f8c2010dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:55:29.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.929+0000 7f8c1f7fe700 1 -- 172.21.15.94:0/1493776406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c200662e0 msgr2=0x7f8c2010dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:55:29.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.929+0000 7f8c1f7fe700 1 --2- 172.21.15.94:0/1493776406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c200662e0 0x7f8c2010dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:55:29.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.929+0000 7f8c1f7fe700 1 -- 172.21.15.94:0/1493776406 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c10005040 con 0x7f8c200fff00 2024-03-24T10:55:29.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.929+0000 7f8c1f7fe700 1 --2- 172.21.15.94:0/1493776406 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c200fff00 0x7f8c2010e010 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8c1400b790 tx=0x7f8c14009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:55:29.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.929+0000 7f8c1cff9700 1 -- 172.21.15.94:0/1493776406 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c1400f6f0 con 0x7f8c200fff00 2024-03-24T10:55:29.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.929+0000 7f8c26676700 1 -- 172.21.15.94:0/1493776406 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c2010f0d0 con 0x7f8c200fff00 2024-03-24T10:55:29.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.929+0000 7f8c1cff9700 1 -- 172.21.15.94:0/1493776406 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8c1400fd20 con 0x7f8c200fff00 2024-03-24T10:55:29.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.929+0000 7f8c1cff9700 1 -- 172.21.15.94:0/1493776406 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c140103a0 con 0x7f8c200fff00 2024-03-24T10:55:29.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.929+0000 7f8c26676700 1 -- 172.21.15.94:0/1493776406 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c2010f690 con 0x7f8c200fff00 2024-03-24T10:55:29.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.930+0000 7f8c26676700 1 -- 172.21.15.94:0/1493776406 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c2005f5f0 con 0x7f8c200fff00 2024-03-24T10:55:29.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.931+0000 7f8c1cff9700 1 -- 172.21.15.94:0/1493776406 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f8c14011400 con 0x7f8c200fff00 2024-03-24T10:55:29.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.931+0000 7f8c1cff9700 1 --2- 172.21.15.94:0/1493776406 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f8c0806d7e0 0x7f8c0806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:55:29.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.931+0000 7f8c1cff9700 1 -- 172.21.15.94:0/1493776406 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f8c1408cdb0 con 0x7f8c200fff00 2024-03-24T10:55:29.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.931+0000 7f8c1ffff700 1 --2- 172.21.15.94:0/1493776406 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f8c0806d7e0 0x7f8c0806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:55:29.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.932+0000 7f8c1ffff700 1 --2- 172.21.15.94:0/1493776406 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f8c0806d7e0 0x7f8c0806fc90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8c1000b970 tx=0x7f8c10009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:55:29.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:29.940+0000 7f8c1cff9700 1 -- 172.21.15.94:0/1493776406 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8c1405a0b0 con 0x7f8c200fff00 2024-03-24T10:55:30.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.340+0000 7f8c26676700 1 -- 172.21.15.94:0/1493776406 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8c2004bed0 con 0x7f8c200fff00 2024-03-24T10:55:30.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.341+0000 7f8c1cff9700 1 -- 172.21.15.94:0/1493776406 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f8c14059c40 con 0x7f8c200fff00 2024-03-24T10:55:30.340 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-03-24T10:55:30.341 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": { 2024-03-24T10:55:30.341 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 2 2024-03-24T10:55:30.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:55:30.341 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": { 2024-03-24T10:55:30.341 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 2 2024-03-24T10:55:30.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:55:30.341 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": { 2024-03-24T10:55:30.341 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 6 2024-03-24T10:55:30.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:55:30.341 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-03-24T10:55:30.341 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 4 2024-03-24T10:55:30.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:55:30.341 INFO:teuthology.orchestra.run.smithi094.stdout: "overall": { 2024-03-24T10:55:30.341 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 14 2024-03-24T10:55:30.342 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-03-24T10:55:30.342 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-03-24T10:55:30.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.343+0000 7f8c26676700 1 -- 172.21.15.94:0/1493776406 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f8c0806d7e0 msgr2=0x7f8c0806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:55:30.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.343+0000 7f8c26676700 1 --2- 172.21.15.94:0/1493776406 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f8c0806d7e0 0x7f8c0806fc90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8c1000b970 tx=0x7f8c10009e30 comp rx=0 tx=0).stop 2024-03-24T10:55:30.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.343+0000 7f8c26676700 1 -- 172.21.15.94:0/1493776406 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c200fff00 msgr2=0x7f8c2010e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:55:30.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.343+0000 7f8c26676700 1 --2- 172.21.15.94:0/1493776406 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c200fff00 0x7f8c2010e010 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8c1400b790 tx=0x7f8c14009250 comp rx=0 tx=0).stop 2024-03-24T10:55:30.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.343+0000 7f8c26676700 1 -- 172.21.15.94:0/1493776406 shutdown_connections 2024-03-24T10:55:30.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.343+0000 7f8c26676700 1 --2- 172.21.15.94:0/1493776406 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f8c0806d7e0 0x7f8c0806fc90 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:55:30.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.343+0000 7f8c26676700 1 --2- 172.21.15.94:0/1493776406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c200662e0 0x7f8c2010dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:55:30.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.343+0000 7f8c26676700 1 --2- 172.21.15.94:0/1493776406 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c200fff00 0x7f8c2010e010 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:55:30.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.343+0000 7f8c26676700 1 -- 172.21.15.94:0/1493776406 >> 172.21.15.94:0/1493776406 conn(0x7f8c200f7430 msgr2=0x7f8c200f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:55:30.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.344+0000 7f8c26676700 1 -- 172.21.15.94:0/1493776406 shutdown_connections 2024-03-24T10:55:30.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.344+0000 7f8c26676700 1 -- 172.21.15.94:0/1493776406 wait complete. 2024-03-24T10:55:30.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.474+0000 7fc9f9d2b700 1 -- 172.21.15.94:0/268898237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9f41023f0 msgr2=0x7fc9f41047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:55:30.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.474+0000 7fc9f9d2b700 1 --2- 172.21.15.94:0/268898237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9f41023f0 0x7fc9f41047d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8004940 tx=0x7fc9e8009e30 comp rx=0 tx=0).stop 2024-03-24T10:55:30.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.474+0000 7fc9f9d2b700 1 -- 172.21.15.94:0/268898237 shutdown_connections 2024-03-24T10:55:30.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.474+0000 7fc9f9d2b700 1 --2- 172.21.15.94:0/268898237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9f41023f0 0x7fc9f41047d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:55:30.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.474+0000 7fc9f9d2b700 1 --2- 172.21.15.94:0/268898237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9f4066b10 0x7fc9f4101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:55:30.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.474+0000 7fc9f9d2b700 1 -- 172.21.15.94:0/268898237 >> 172.21.15.94:0/268898237 conn(0x7fc9f40f74e0 msgr2=0x7fc9f40f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:55:30.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.475+0000 7fc9f9d2b700 1 -- 172.21.15.94:0/268898237 shutdown_connections 2024-03-24T10:55:30.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.475+0000 7fc9f9d2b700 1 -- 172.21.15.94:0/268898237 wait complete. 2024-03-24T10:55:30.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.475+0000 7fc9f9d2b700 1 Processor -- start 2024-03-24T10:55:30.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.475+0000 7fc9f9d2b700 1 -- start start 2024-03-24T10:55:30.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.476+0000 7fc9f9d2b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9f4066b10 0x7fc9f410fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:55:30.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.476+0000 7fc9f9d2b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9f41023f0 0x7fc9f4110100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:55:30.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.476+0000 7fc9f9d2b700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f4110700 con 0x7fc9f4066b10 2024-03-24T10:55:30.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.476+0000 7fc9f9d2b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f4110f50 con 0x7fc9f41023f0 2024-03-24T10:55:30.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.476+0000 7fc9f37fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9f4066b10 0x7fc9f410fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:55:30.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.476+0000 7fc9f37fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9f4066b10 0x7fc9f410fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38310/0 (socket says 172.21.15.94:38310) 2024-03-24T10:55:30.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.476+0000 7fc9f37fe700 1 -- 172.21.15.94:0/3113868871 learned_addr learned my addr 172.21.15.94:0/3113868871 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:55:30.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.476+0000 7fc9f2ffd700 1 --2- 172.21.15.94:0/3113868871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9f41023f0 0x7fc9f4110100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:55:30.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.476+0000 7fc9f37fe700 1 -- 172.21.15.94:0/3113868871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9f41023f0 msgr2=0x7fc9f4110100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:55:30.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.476+0000 7fc9f37fe700 1 --2- 172.21.15.94:0/3113868871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9f41023f0 0x7fc9f4110100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:55:30.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.476+0000 7fc9f37fe700 1 -- 172.21.15.94:0/3113868871 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e8005040 con 0x7fc9f4066b10 2024-03-24T10:55:30.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.476+0000 7fc9f2ffd700 1 --2- 172.21.15.94:0/3113868871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9f41023f0 0x7fc9f4110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T10:55:30.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.477+0000 7fc9f37fe700 1 --2- 172.21.15.94:0/3113868871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9f4066b10 0x7fc9f410fbc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc9e400eb70 tx=0x7fc9e400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:55:30.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.477+0000 7fc9dbfff700 1 -- 172.21.15.94:0/3113868871 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9e4007cc0 con 0x7fc9f4066b10 2024-03-24T10:55:30.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.477+0000 7fc9f9d2b700 1 -- 172.21.15.94:0/3113868871 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9f41111d0 con 0x7fc9f4066b10 2024-03-24T10:55:30.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.477+0000 7fc9f9d2b700 1 -- 172.21.15.94:0/3113868871 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9f4111710 con 0x7fc9f4066b10 2024-03-24T10:55:30.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.477+0000 7fc9dbfff700 1 -- 172.21.15.94:0/3113868871 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc9e400f040 con 0x7fc9f4066b10 2024-03-24T10:55:30.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.478+0000 7fc9dbfff700 1 -- 172.21.15.94:0/3113868871 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9e400cb20 con 0x7fc9f4066b10 2024-03-24T10:55:30.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.478+0000 7fc9f9d2b700 1 -- 172.21.15.94:0/3113868871 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9e00052e0 con 0x7fc9f4066b10 2024-03-24T10:55:30.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.479+0000 7fc9dbfff700 1 -- 172.21.15.94:0/3113868871 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fc9e40097c0 con 0x7fc9f4066b10 2024-03-24T10:55:30.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.480+0000 7fc9dbfff700 1 --2- 172.21.15.94:0/3113868871 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fc9dc06d7e0 0x7fc9dc06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:55:30.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.480+0000 7fc9dbfff700 1 -- 172.21.15.94:0/3113868871 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fc9e408c940 con 0x7fc9f4066b10 2024-03-24T10:55:30.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.480+0000 7fc9f2ffd700 1 --2- 172.21.15.94:0/3113868871 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fc9dc06d7e0 0x7fc9dc06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:55:30.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.480+0000 7fc9f2ffd700 1 --2- 172.21.15.94:0/3113868871 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fc9dc06d7e0 0x7fc9dc06fc90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8009b30 tx=0x7fc9e8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:55:30.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.488+0000 7fc9dbfff700 1 -- 172.21.15.94:0/3113868871 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc9e4050280 con 0x7fc9f4066b10 2024-03-24T10:55:30.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:30 smithi094 ceph-mon[93957]: from='client.34494 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:55:30.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:30 smithi094 ceph-mon[93957]: pgmap v245: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:55:30.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:30 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:55:30.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:30 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:55:30.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:30 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:55:30.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:30 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:55:30.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:30 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:55:30.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:30 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-24T10:55:30.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:30 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:55:30.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:30 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:30.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:30 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:30.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:30 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:30.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:30 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/1493776406' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:30.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:30 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:30.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:30 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:30.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:30 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:30.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:30 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:30.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:30 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:30.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:30 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:30.750 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:30 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:30.750 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:30 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:30.750 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:30 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:30.750 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:30 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:30.750 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:30 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:30.750 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:30 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:30.750 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:30 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:30.750 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:30 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:30.750 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:30 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:30.750 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:30 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:55:30.750 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:30 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-24T10:55:30.750 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:30 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-24T10:55:30.750 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:30 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-24T10:55:30.750 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:30 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-24T10:55:30.751 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:30 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-24T10:55:30.751 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:30 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-03-24T10:55:30.751 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:30 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-24T10:55:30.751 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:30 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-03-24T10:55:30.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.766+0000 7fc9f9d2b700 1 -- 172.21.15.94:0/3113868871 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc9e0005860 con 0x7fc9f4066b10 2024-03-24T10:55:30.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.767+0000 7fc9dbfff700 1 -- 172.21.15.94:0/3113868871 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 37 v37) v1 ==== 76+0+1963 (secure 0 0 0) 0x7fc9e4059cf0 con 0x7fc9f4066b10 2024-03-24T10:55:30.767 INFO:teuthology.orchestra.run.smithi094.stdout:e37 2024-03-24T10:55:30.767 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T10:55:30.767 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:55:30.767 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-03-24T10:55:30.767 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:55:30.767 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-03-24T10:55:30.767 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-03-24T10:55:30.767 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 37 2024-03-24T10:55:30.767 INFO:teuthology.orchestra.run.smithi094.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-24T10:55:30.767 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-03-24T10:43:39.435738+0000 2024-03-24T10:55:30.767 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-03-24T10:54:19.399866+0000 2024-03-24T10:55:30.767 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-03-24T10:55:30.767 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-03-24T10:55:30.767 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 60 2024-03-24T10:55:30.768 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-03-24T10:55:30.768 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-03-24T10:55:30.768 INFO:teuthology.orchestra.run.smithi094.stdout:max_xattr_size 65536 2024-03-24T10:55:30.768 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-03-24T10:55:30.768 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-03-24T10:55:30.768 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 84 2024-03-24T10:55:30.768 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-24T10:55:30.768 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-03-24T10:55:30.768 INFO:teuthology.orchestra.run.smithi094.stdout:in 0 2024-03-24T10:55:30.768 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=34374} 2024-03-24T10:55:30.768 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-03-24T10:55:30.768 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-03-24T10:55:30.768 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-03-24T10:55:30.768 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-03-24T10:55:30.769 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-03-24T10:55:30.769 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-03-24T10:55:30.769 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-03-24T10:55:30.769 INFO:teuthology.orchestra.run.smithi094.stdout:bal_rank_mask -1 2024-03-24T10:55:30.769 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 1 2024-03-24T10:55:30.769 INFO:teuthology.orchestra.run.smithi094.stdout:qdb_cluster leader: 34374 members: 34374 2024-03-24T10:55:30.769 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.efiqte{0:34374} state up:active seq 19 join_fscid=1 addr [v2:172.21.15.94:6826/2043632450,v1:172.21.15.94:6827/2043632450] compat {c=[1],r=[1],i=[1fff]}] 2024-03-24T10:55:30.769 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.llefgu{0:34406} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.94:6828/2396893131,v1:172.21.15.94:6829/2396893131] compat {c=[1],r=[1],i=[1fff]}] 2024-03-24T10:55:30.769 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:55:30.769 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:55:30.769 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-03-24T10:55:30.769 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:55:30.769 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi096.egbmoq{-1:44321} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.96:6824/2694461318,v1:172.21.15.96:6825/2694461318] compat {c=[1],r=[1],i=[1fff]}] 2024-03-24T10:55:30.769 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi096.jtebhh{-1:44349} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.96:6826/2275611388,v1:172.21.15.96:6827/2275611388] compat {c=[1],r=[1],i=[1fff]}] 2024-03-24T10:55:30.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.769+0000 7fc9f9d2b700 1 -- 172.21.15.94:0/3113868871 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fc9dc06d7e0 msgr2=0x7fc9dc06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:55:30.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.769+0000 7fc9f9d2b700 1 --2- 172.21.15.94:0/3113868871 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fc9dc06d7e0 0x7fc9dc06fc90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8009b30 tx=0x7fc9e8009e30 comp rx=0 tx=0).stop 2024-03-24T10:55:30.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.769+0000 7fc9f9d2b700 1 -- 172.21.15.94:0/3113868871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9f4066b10 msgr2=0x7fc9f410fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:55:30.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.769+0000 7fc9f9d2b700 1 --2- 172.21.15.94:0/3113868871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9f4066b10 0x7fc9f410fbc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc9e400eb70 tx=0x7fc9e400b040 comp rx=0 tx=0).stop 2024-03-24T10:55:30.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.769+0000 7fc9f9d2b700 1 -- 172.21.15.94:0/3113868871 shutdown_connections 2024-03-24T10:55:30.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.769+0000 7fc9f9d2b700 1 --2- 172.21.15.94:0/3113868871 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fc9dc06d7e0 0x7fc9dc06fc90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:55:30.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.769+0000 7fc9f9d2b700 1 --2- 172.21.15.94:0/3113868871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9f4066b10 0x7fc9f410fbc0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:55:30.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.769+0000 7fc9f9d2b700 1 --2- 172.21.15.94:0/3113868871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9f41023f0 0x7fc9f4110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:55:30.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.769+0000 7fc9f9d2b700 1 -- 172.21.15.94:0/3113868871 >> 172.21.15.94:0/3113868871 conn(0x7fc9f40f74e0 msgr2=0x7fc9f40f78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:55:30.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.769+0000 7fc9f9d2b700 1 -- 172.21.15.94:0/3113868871 shutdown_connections 2024-03-24T10:55:30.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.769+0000 7fc9f9d2b700 1 -- 172.21.15.94:0/3113868871 wait complete. 2024-03-24T10:55:30.771 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 37 2024-03-24T10:55:30.791 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:30 smithi096 ceph-mon[84258]: from='client.34494 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:55:30.791 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:30 smithi096 ceph-mon[84258]: pgmap v245: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:55:30.791 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:30 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:55:30.791 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:30 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:55:30.791 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:30 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:55:30.791 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:30 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:55:30.791 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:30 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:55:30.791 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:30 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-24T10:55:30.791 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:30 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:55:30.791 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:30 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:30.791 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:30 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:30.791 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:30 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:30.792 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:30 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/1493776406' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:30.792 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:30 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:30.792 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:30 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:30.792 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:30 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:30.792 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:30 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:30.792 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:30 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:30.792 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:30 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:30.792 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:30 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:30.792 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:30 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:30.792 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:30 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:30.792 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:30 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:30.792 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:30 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:30.792 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:30 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:30.792 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:30 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:30.792 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:30 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:30.792 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:30 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:55:30.792 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:30 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:55:30.793 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:30 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-24T10:55:30.793 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:30 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-24T10:55:30.793 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:30 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-24T10:55:30.793 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:30 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-24T10:55:30.793 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:30 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-24T10:55:30.793 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:30 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-03-24T10:55:30.793 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:30 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-24T10:55:30.793 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:30 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-03-24T10:55:30.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.874+0000 7ff7f9ea5700 1 -- 172.21.15.94:0/3546572978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7f40fcf80 msgr2=0x7ff7f40fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:55:30.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.874+0000 7ff7f9ea5700 1 --2- 172.21.15.94:0/3546572978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7f40fcf80 0x7ff7f40fd3f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8004940 tx=0x7ff7e8009e30 comp rx=0 tx=0).stop 2024-03-24T10:55:30.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.874+0000 7ff7f9ea5700 1 -- 172.21.15.94:0/3546572978 shutdown_connections 2024-03-24T10:55:30.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.874+0000 7ff7f9ea5700 1 --2- 172.21.15.94:0/3546572978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7f40fcf80 0x7ff7f40fd3f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:55:30.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.874+0000 7ff7f9ea5700 1 --2- 172.21.15.94:0/3546572978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7f40fbcd0 0x7ff7f40fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:55:30.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.874+0000 7ff7f9ea5700 1 -- 172.21.15.94:0/3546572978 >> 172.21.15.94:0/3546572978 conn(0x7ff7f40f7430 msgr2=0x7ff7f40f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:55:30.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.874+0000 7ff7f9ea5700 1 -- 172.21.15.94:0/3546572978 shutdown_connections 2024-03-24T10:55:30.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.874+0000 7ff7f9ea5700 1 -- 172.21.15.94:0/3546572978 wait complete. 2024-03-24T10:55:30.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.875+0000 7ff7f9ea5700 1 Processor -- start 2024-03-24T10:55:30.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.875+0000 7ff7f9ea5700 1 -- start start 2024-03-24T10:55:30.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.875+0000 7ff7f9ea5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7f40fbcd0 0x7ff7f4110760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:55:30.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.875+0000 7ff7f9ea5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7f40fcf80 0x7ff7f4112cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:55:30.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.875+0000 7ff7f9ea5700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f4066350 con 0x7ff7f40fcf80 2024-03-24T10:55:30.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.875+0000 7ff7f9ea5700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f4066490 con 0x7ff7f40fbcd0 2024-03-24T10:55:30.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.875+0000 7ff7f2ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7f40fcf80 0x7ff7f4112cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:55:30.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.875+0000 7ff7f2ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7f40fcf80 0x7ff7f4112cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38336/0 (socket says 172.21.15.94:38336) 2024-03-24T10:55:30.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.875+0000 7ff7f2ffd700 1 -- 172.21.15.94:0/1374826202 learned_addr learned my addr 172.21.15.94:0/1374826202 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:55:30.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.875+0000 7ff7f37fe700 1 --2- 172.21.15.94:0/1374826202 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7f40fbcd0 0x7ff7f4110760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:55:30.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.876+0000 7ff7f2ffd700 1 -- 172.21.15.94:0/1374826202 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7f40fbcd0 msgr2=0x7ff7f4110760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:55:30.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.876+0000 7ff7f2ffd700 1 --2- 172.21.15.94:0/1374826202 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7f40fbcd0 0x7ff7f4110760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:55:30.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.876+0000 7ff7f2ffd700 1 -- 172.21.15.94:0/1374826202 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e8005040 con 0x7ff7f40fcf80 2024-03-24T10:55:30.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.876+0000 7ff7f2ffd700 1 --2- 172.21.15.94:0/1374826202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7f40fcf80 0x7ff7f4112cb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff7e800b3b0 tx=0x7ff7e8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:55:30.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.876+0000 7ff7dbfff700 1 -- 172.21.15.94:0/1374826202 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7e800e040 con 0x7ff7f40fcf80 2024-03-24T10:55:30.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.876+0000 7ff7dbfff700 1 -- 172.21.15.94:0/1374826202 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff7e800bbf0 con 0x7ff7f40fcf80 2024-03-24T10:55:30.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.876+0000 7ff7f9ea5700 1 -- 172.21.15.94:0/1374826202 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7f41131f0 con 0x7ff7f40fcf80 2024-03-24T10:55:30.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.876+0000 7ff7f9ea5700 1 -- 172.21.15.94:0/1374826202 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7f41137b0 con 0x7ff7f40fcf80 2024-03-24T10:55:30.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.876+0000 7ff7dbfff700 1 -- 172.21.15.94:0/1374826202 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7e801c630 con 0x7ff7f40fcf80 2024-03-24T10:55:30.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.877+0000 7ff7f9ea5700 1 -- 172.21.15.94:0/1374826202 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7e00052e0 con 0x7ff7f40fcf80 2024-03-24T10:55:30.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.878+0000 7ff7dbfff700 1 -- 172.21.15.94:0/1374826202 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7ff7e800bd60 con 0x7ff7f40fcf80 2024-03-24T10:55:30.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.878+0000 7ff7dbfff700 1 --2- 172.21.15.94:0/1374826202 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7ff7dc06d8b0 0x7ff7dc06fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:55:30.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.878+0000 7ff7dbfff700 1 -- 172.21.15.94:0/1374826202 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7ff7e808da70 con 0x7ff7f40fcf80 2024-03-24T10:55:30.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.878+0000 7ff7f37fe700 1 --2- 172.21.15.94:0/1374826202 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7ff7dc06d8b0 0x7ff7dc06fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:55:30.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.879+0000 7ff7f37fe700 1 --2- 172.21.15.94:0/1374826202 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7ff7dc06d8b0 0x7ff7dc06fd60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4065fd0 tx=0x7ff7e400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:55:30.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:30.884+0000 7ff7dbfff700 1 -- 172.21.15.94:0/1374826202 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff7e8057240 con 0x7ff7f40fcf80 2024-03-24T10:55:31.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.104+0000 7ff7f9ea5700 1 -- 172.21.15.94:0/1374826202 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff7e0000bf0 con 0x7ff7dc06d8b0 2024-03-24T10:55:31.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.105+0000 7ff7dbfff700 1 -- 172.21.15.94:0/1374826202 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7ff7e0000bf0 con 0x7ff7dc06d8b0 2024-03-24T10:55:31.105 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-03-24T10:55:31.105 INFO:teuthology.orchestra.run.smithi094.stdout: "target_image": null, 2024-03-24T10:55:31.105 INFO:teuthology.orchestra.run.smithi094.stdout: "in_progress": false, 2024-03-24T10:55:31.105 INFO:teuthology.orchestra.run.smithi094.stdout: "which": "", 2024-03-24T10:55:31.105 INFO:teuthology.orchestra.run.smithi094.stdout: "services_complete": [], 2024-03-24T10:55:31.105 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": null, 2024-03-24T10:55:31.105 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "", 2024-03-24T10:55:31.105 INFO:teuthology.orchestra.run.smithi094.stdout: "is_paused": false 2024-03-24T10:55:31.105 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-03-24T10:55:31.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.107+0000 7ff7f9ea5700 1 -- 172.21.15.94:0/1374826202 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7ff7dc06d8b0 msgr2=0x7ff7dc06fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:55:31.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.107+0000 7ff7f9ea5700 1 --2- 172.21.15.94:0/1374826202 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7ff7dc06d8b0 0x7ff7dc06fd60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4065fd0 tx=0x7ff7e400b040 comp rx=0 tx=0).stop 2024-03-24T10:55:31.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.107+0000 7ff7f9ea5700 1 -- 172.21.15.94:0/1374826202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7f40fcf80 msgr2=0x7ff7f4112cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:55:31.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.107+0000 7ff7f9ea5700 1 --2- 172.21.15.94:0/1374826202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7f40fcf80 0x7ff7f4112cb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff7e800b3b0 tx=0x7ff7e8009e30 comp rx=0 tx=0).stop 2024-03-24T10:55:31.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.107+0000 7ff7f9ea5700 1 -- 172.21.15.94:0/1374826202 shutdown_connections 2024-03-24T10:55:31.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.107+0000 7ff7f9ea5700 1 --2- 172.21.15.94:0/1374826202 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7ff7dc06d8b0 0x7ff7dc06fd60 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:55:31.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.107+0000 7ff7f9ea5700 1 --2- 172.21.15.94:0/1374826202 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7f40fbcd0 0x7ff7f4110760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:55:31.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.107+0000 7ff7f9ea5700 1 --2- 172.21.15.94:0/1374826202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7f40fcf80 0x7ff7f4112cb0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:55:31.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.107+0000 7ff7f9ea5700 1 -- 172.21.15.94:0/1374826202 >> 172.21.15.94:0/1374826202 conn(0x7ff7f40f7430 msgr2=0x7ff7f40f8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:55:31.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.107+0000 7ff7f9ea5700 1 -- 172.21.15.94:0/1374826202 shutdown_connections 2024-03-24T10:55:31.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.107+0000 7ff7f9ea5700 1 -- 172.21.15.94:0/1374826202 wait complete. 2024-03-24T10:55:31.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.210+0000 7f1f43c6e700 1 -- 172.21.15.94:0/24740573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f3c0fffc0 msgr2=0x7f1f3c1023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:55:31.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.210+0000 7f1f43c6e700 1 --2- 172.21.15.94:0/24740573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f3c0fffc0 0x7f1f3c1023a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c004940 tx=0x7f1f2c009e30 comp rx=0 tx=0).stop 2024-03-24T10:55:31.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.210+0000 7f1f43c6e700 1 -- 172.21.15.94:0/24740573 shutdown_connections 2024-03-24T10:55:31.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.210+0000 7f1f43c6e700 1 --2- 172.21.15.94:0/24740573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f3c0fffc0 0x7f1f3c1023a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:55:31.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.210+0000 7f1f43c6e700 1 --2- 172.21.15.94:0/24740573 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f3c066220 0x7f1f3c0ffa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:55:31.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.210+0000 7f1f43c6e700 1 -- 172.21.15.94:0/24740573 >> 172.21.15.94:0/24740573 conn(0x7f1f3c0f7430 msgr2=0x7f1f3c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:55:31.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.210+0000 7f1f43c6e700 1 -- 172.21.15.94:0/24740573 shutdown_connections 2024-03-24T10:55:31.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.210+0000 7f1f43c6e700 1 -- 172.21.15.94:0/24740573 wait complete. 2024-03-24T10:55:31.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.210+0000 7f1f43c6e700 1 Processor -- start 2024-03-24T10:55:31.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.211+0000 7f1f43c6e700 1 -- start start 2024-03-24T10:55:31.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.211+0000 7f1f43c6e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f3c066220 0x7f1f3c0fd460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:55:31.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.211+0000 7f1f43c6e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f3c0fffc0 0x7f1f3c0fb980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:55:31.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.211+0000 7f1f43c6e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3c0fbf10 con 0x7f1f3c066220 2024-03-24T10:55:31.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.211+0000 7f1f43c6e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3c0fc050 con 0x7f1f3c0fffc0 2024-03-24T10:55:31.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.211+0000 7f1f41a0a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f3c066220 0x7f1f3c0fd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:55:31.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.211+0000 7f1f41209700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f3c0fffc0 0x7f1f3c0fb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:55:31.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.211+0000 7f1f41a0a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f3c066220 0x7f1f3c0fd460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38350/0 (socket says 172.21.15.94:38350) 2024-03-24T10:55:31.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.211+0000 7f1f41209700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f3c0fffc0 0x7f1f3c0fb980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.94:45056/0 (socket says 172.21.15.94:45056) 2024-03-24T10:55:31.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.211+0000 7f1f41a0a700 1 -- 172.21.15.94:0/116995408 learned_addr learned my addr 172.21.15.94:0/116995408 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:55:31.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.211+0000 7f1f41209700 1 -- 172.21.15.94:0/116995408 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f3c066220 msgr2=0x7f1f3c0fd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:55:31.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.211+0000 7f1f41209700 1 --2- 172.21.15.94:0/116995408 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f3c066220 0x7f1f3c0fd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:55:31.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.211+0000 7f1f41209700 1 -- 172.21.15.94:0/116995408 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f2c005040 con 0x7f1f3c0fffc0 2024-03-24T10:55:31.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.211+0000 7f1f41a0a700 1 --2- 172.21.15.94:0/116995408 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f3c066220 0x7f1f3c0fd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T10:55:31.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.211+0000 7f1f41209700 1 --2- 172.21.15.94:0/116995408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f3c0fffc0 0x7f1f3c0fb980 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c009fd0 tx=0x7f1f2c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:55:31.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.212+0000 7f1f327fc700 1 -- 172.21.15.94:0/116995408 <== mon.1 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f2c021070 con 0x7f1f3c0fffc0 2024-03-24T10:55:31.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.212+0000 7f1f327fc700 1 -- 172.21.15.94:0/116995408 <== mon.1 v2:172.21.15.96:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f2c01c400 con 0x7f1f3c0fffc0 2024-03-24T10:55:31.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.212+0000 7f1f43c6e700 1 -- 172.21.15.94:0/116995408 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f3c0fc2d0 con 0x7f1f3c0fffc0 2024-03-24T10:55:31.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.212+0000 7f1f327fc700 1 -- 172.21.15.94:0/116995408 <== mon.1 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f2c00e070 con 0x7f1f3c0fffc0 2024-03-24T10:55:31.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.212+0000 7f1f43c6e700 1 -- 172.21.15.94:0/116995408 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f3c115a70 con 0x7f1f3c0fffc0 2024-03-24T10:55:31.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.212+0000 7f1f43c6e700 1 -- 172.21.15.94:0/116995408 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f3c05f5f0 con 0x7f1f3c0fffc0 2024-03-24T10:55:31.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.213+0000 7f1f327fc700 1 -- 172.21.15.94:0/116995408 <== mon.1 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f1f2c01ca20 con 0x7f1f3c0fffc0 2024-03-24T10:55:31.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.213+0000 7f1f327fc700 1 --2- 172.21.15.94:0/116995408 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1f2806d7f0 0x7f1f2806fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:55:31.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.214+0000 7f1f327fc700 1 -- 172.21.15.94:0/116995408 <== mon.1 v2:172.21.15.96:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f1f2c08e120 con 0x7f1f3c0fffc0 2024-03-24T10:55:31.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.214+0000 7f1f41a0a700 1 --2- 172.21.15.94:0/116995408 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1f2806d7f0 0x7f1f2806fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:55:31.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.214+0000 7f1f41a0a700 1 --2- 172.21.15.94:0/116995408 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1f2806d7f0 0x7f1f2806fca0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c0fd1f0 tx=0x7f1f3400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:55:31.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.218+0000 7f1f327fc700 1 -- 172.21.15.94:0/116995408 <== mon.1 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1f2c057930 con 0x7f1f3c0fffc0 2024-03-24T10:55:31.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.501+0000 7f1f43c6e700 1 -- 172.21.15.94:0/116995408 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1f3c04bed0 con 0x7f1f3c0fffc0 2024-03-24T10:55:31.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.502+0000 7f1f327fc700 1 -- 172.21.15.94:0/116995408 <== mon.1 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f1f2c05aff0 con 0x7f1f3c0fffc0 2024-03-24T10:55:31.502 INFO:teuthology.orchestra.run.smithi094.stdout:HEALTH_OK 2024-03-24T10:55:31.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.504+0000 7f1f43c6e700 1 -- 172.21.15.94:0/116995408 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1f2806d7f0 msgr2=0x7f1f2806fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:55:31.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.504+0000 7f1f43c6e700 1 --2- 172.21.15.94:0/116995408 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1f2806d7f0 0x7f1f2806fca0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c0fd1f0 tx=0x7f1f3400b040 comp rx=0 tx=0).stop 2024-03-24T10:55:31.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.504+0000 7f1f43c6e700 1 -- 172.21.15.94:0/116995408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f3c0fffc0 msgr2=0x7f1f3c0fb980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:55:31.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.504+0000 7f1f43c6e700 1 --2- 172.21.15.94:0/116995408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f3c0fffc0 0x7f1f3c0fb980 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c009fd0 tx=0x7f1f2c009e30 comp rx=0 tx=0).stop 2024-03-24T10:55:31.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.504+0000 7f1f43c6e700 1 -- 172.21.15.94:0/116995408 shutdown_connections 2024-03-24T10:55:31.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.504+0000 7f1f43c6e700 1 --2- 172.21.15.94:0/116995408 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1f2806d7f0 0x7f1f2806fca0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:55:31.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.504+0000 7f1f43c6e700 1 --2- 172.21.15.94:0/116995408 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f3c066220 0x7f1f3c0fd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:55:31.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.504+0000 7f1f43c6e700 1 --2- 172.21.15.94:0/116995408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f3c0fffc0 0x7f1f3c0fb980 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:55:31.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.504+0000 7f1f43c6e700 1 -- 172.21.15.94:0/116995408 >> 172.21.15.94:0/116995408 conn(0x7f1f3c0f7430 msgr2=0x7f1f3c0f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:55:31.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.504+0000 7f1f43c6e700 1 -- 172.21.15.94:0/116995408 shutdown_connections 2024-03-24T10:55:31.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:55:31.504+0000 7f1f43c6e700 1 -- 172.21.15.94:0/116995408 wait complete. 2024-03-24T10:55:31.714 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:31 smithi096 ceph-mon[84258]: from='client.34498 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:55:31.714 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:31 smithi096 ceph-mon[84258]: from='client.34502 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:55:31.714 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:31 smithi096 ceph-mon[84258]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-24T10:55:31.714 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:31 smithi096 ceph-mon[84258]: Upgrade: Finalizing container_image settings 2024-03-24T10:55:31.714 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:31 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-24T10:55:31.714 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:31 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-03-24T10:55:31.714 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:31 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-24T10:55:31.715 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:31 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-24T10:55:31.715 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:31 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-24T10:55:31.715 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:31 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-24T10:55:31.715 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:31 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-24T10:55:31.715 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:31 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-24T10:55:31.715 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:31 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-03-24T10:55:31.715 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:31 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-24T10:55:31.715 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:31 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-24T10:55:31.715 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:31 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-24T10:55:31.715 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:31 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-24T10:55:31.715 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:31 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-24T10:55:31.715 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:31 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-24T10:55:31.715 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:31 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-24T10:55:31.715 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:31 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-24T10:55:31.715 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:31 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-24T10:55:31.716 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:31 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-24T10:55:31.716 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:31 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-24T10:55:31.716 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:31 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-24T10:55:31.716 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:31 smithi096 ceph-mon[84258]: Upgrade: Complete! 2024-03-24T10:55:31.716 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:31 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-24T10:55:31.716 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:31 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-24T10:55:31.716 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:31 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:55:31.716 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:31 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:55:31.716 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:31 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:55:31.716 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:31 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:55:31.716 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:31 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:55:31.716 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:31 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:55:31.716 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:31 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:55:31.716 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:31 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:55:31.716 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:31 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/3113868871' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T10:55:31.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:31 smithi094 ceph-mon[93957]: from='client.34498 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:55:31.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:31 smithi094 ceph-mon[93957]: from='client.34502 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:55:31.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:31 smithi094 ceph-mon[93957]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-24T10:55:31.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:31 smithi094 ceph-mon[93957]: Upgrade: Finalizing container_image settings 2024-03-24T10:55:31.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:31 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-24T10:55:31.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:31 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-03-24T10:55:31.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:31 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-24T10:55:31.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:31 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-24T10:55:31.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:31 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-24T10:55:31.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:31 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-24T10:55:31.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:31 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-24T10:55:31.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:31 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-24T10:55:31.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:31 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-03-24T10:55:31.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:31 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-24T10:55:31.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:31 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-24T10:55:31.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:31 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-24T10:55:31.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:31 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-24T10:55:31.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:31 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-24T10:55:31.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:31 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-24T10:55:31.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:31 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-24T10:55:31.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:31 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-24T10:55:31.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:31 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-24T10:55:31.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:31 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-24T10:55:31.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:31 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-24T10:55:31.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:31 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-24T10:55:31.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:31 smithi094 ceph-mon[93957]: Upgrade: Complete! 2024-03-24T10:55:31.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:31 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-24T10:55:31.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:31 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-24T10:55:31.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:31 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:55:31.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:31 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:55:31.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:31 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:55:31.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:31 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:55:31.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:31 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:55:31.750 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:31 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:55:31.750 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:31 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:55:31.750 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:31 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:55:31.750 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:31 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/3113868871' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T10:55:32.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:32 smithi094 ceph-mon[93957]: from='client.34514 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:55:32.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:32 smithi094 ceph-mon[93957]: pgmap v246: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:55:32.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:32 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/116995408' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T10:55:32.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:32 smithi096 ceph-mon[84258]: from='client.34514 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:55:32.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:32 smithi096 ceph-mon[84258]: pgmap v246: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:55:32.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:32 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/116995408' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T10:55:34.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:34 smithi094 ceph-mon[93957]: pgmap v247: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-24T10:55:34.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:34 smithi096 ceph-mon[84258]: pgmap v247: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-24T10:55:35.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:35 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:55:35.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:35 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:55:35.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:35 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:55:35.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:35 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:55:36.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:36 smithi094 ceph-mon[93957]: pgmap v248: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:55:36.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:36 smithi096 ceph-mon[84258]: pgmap v248: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:55:38.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:38 smithi094 ceph-mon[93957]: pgmap v249: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:55:38.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:38 smithi096 ceph-mon[84258]: pgmap v249: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:55:40.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:40 smithi094 ceph-mon[93957]: pgmap v250: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:55:40.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:40 smithi096 ceph-mon[84258]: pgmap v250: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:55:42.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:42 smithi094 ceph-mon[93957]: pgmap v251: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:55:42.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:42 smithi096 ceph-mon[84258]: pgmap v251: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:55:44.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:44 smithi094 ceph-mon[93957]: pgmap v252: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-24T10:55:44.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:44 smithi096 ceph-mon[84258]: pgmap v252: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-24T10:55:46.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:46 smithi094 ceph-mon[93957]: pgmap v253: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:55:46.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:46 smithi096 ceph-mon[84258]: pgmap v253: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:55:48.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:48 smithi094 ceph-mon[93957]: pgmap v254: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:55:48.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:48 smithi096 ceph-mon[84258]: pgmap v254: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:55:50.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:50 smithi094 ceph-mon[93957]: pgmap v255: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:55:50.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:50 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:55:50.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:50 smithi096 ceph-mon[84258]: pgmap v255: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:55:50.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:50 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:55:52.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:52 smithi094 ceph-mon[93957]: pgmap v256: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:55:52.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:52 smithi096 ceph-mon[84258]: pgmap v256: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:55:54.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:54 smithi094 ceph-mon[93957]: pgmap v257: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-24T10:55:54.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:54 smithi096 ceph-mon[84258]: pgmap v257: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-24T10:55:56.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:56 smithi094 ceph-mon[93957]: pgmap v258: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:55:56.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:56 smithi096 ceph-mon[84258]: pgmap v258: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:55:58.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:55:58 smithi094 ceph-mon[93957]: pgmap v259: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:55:58.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:55:58 smithi096 ceph-mon[84258]: pgmap v259: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:56:00.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:56:00 smithi094 ceph-mon[93957]: pgmap v260: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:56:00.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:56:00 smithi096 ceph-mon[84258]: pgmap v260: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:56:01.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:01.607+0000 7f75ddcb7700 1 -- 172.21.15.94:0/3919254313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75d80fdd40 msgr2=0x7f75d8104690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:56:01.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:01.607+0000 7f75ddcb7700 1 --2- 172.21.15.94:0/3919254313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75d80fdd40 0x7f75d8104690 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f75cc004940 tx=0x7f75cc009e30 comp rx=0 tx=0).stop 2024-03-24T10:56:01.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:01.607+0000 7f75ddcb7700 1 -- 172.21.15.94:0/3919254313 shutdown_connections 2024-03-24T10:56:01.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:01.607+0000 7f75ddcb7700 1 --2- 172.21.15.94:0/3919254313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75d80fdd40 0x7f75d8104690 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:56:01.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:01.607+0000 7f75ddcb7700 1 --2- 172.21.15.94:0/3919254313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75d80fd3f0 0x7f75d80fd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:56:01.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:01.607+0000 7f75ddcb7700 1 -- 172.21.15.94:0/3919254313 >> 172.21.15.94:0/3919254313 conn(0x7f75d80f6ba0 msgr2=0x7f75d80f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:56:01.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:01.607+0000 7f75ddcb7700 1 -- 172.21.15.94:0/3919254313 shutdown_connections 2024-03-24T10:56:01.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:01.607+0000 7f75ddcb7700 1 -- 172.21.15.94:0/3919254313 wait complete. 2024-03-24T10:56:01.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:01.608+0000 7f75ddcb7700 1 Processor -- start 2024-03-24T10:56:01.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:01.608+0000 7f75ddcb7700 1 -- start start 2024-03-24T10:56:01.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:01.608+0000 7f75ddcb7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75d80fd3f0 0x7f75d810fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:56:01.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:01.608+0000 7f75ddcb7700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75d80fdd40 0x7f75d8110180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:56:01.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:01.608+0000 7f75ddcb7700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d8110810 con 0x7f75d80fdd40 2024-03-24T10:56:01.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:01.608+0000 7f75ddcb7700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d8110fb0 con 0x7f75d80fd3f0 2024-03-24T10:56:01.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:01.608+0000 7f75d6ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75d80fdd40 0x7f75d8110180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:56:01.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:01.608+0000 7f75d6ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75d80fdd40 0x7f75d8110180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55010/0 (socket says 172.21.15.94:55010) 2024-03-24T10:56:01.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:01.608+0000 7f75d6ffd700 1 -- 172.21.15.94:0/3970597300 learned_addr learned my addr 172.21.15.94:0/3970597300 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:56:01.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:01.608+0000 7f75d77fe700 1 --2- 172.21.15.94:0/3970597300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75d80fd3f0 0x7f75d810fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:56:01.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:01.609+0000 7f75d6ffd700 1 -- 172.21.15.94:0/3970597300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75d80fd3f0 msgr2=0x7f75d810fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:56:01.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:01.609+0000 7f75d6ffd700 1 --2- 172.21.15.94:0/3970597300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75d80fd3f0 0x7f75d810fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:56:01.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:01.609+0000 7f75d6ffd700 1 -- 172.21.15.94:0/3970597300 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75cc005040 con 0x7f75d80fdd40 2024-03-24T10:56:01.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:01.609+0000 7f75d6ffd700 1 --2- 172.21.15.94:0/3970597300 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75d80fdd40 0x7f75d8110180 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f75cc009150 tx=0x7f75cc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:56:01.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:01.609+0000 7f75bffff700 1 -- 172.21.15.94:0/3970597300 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75cc00e040 con 0x7f75d80fdd40 2024-03-24T10:56:01.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:01.609+0000 7f75ddcb7700 1 -- 172.21.15.94:0/3970597300 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75d8111230 con 0x7f75d80fdd40 2024-03-24T10:56:01.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:01.609+0000 7f75bffff700 1 -- 172.21.15.94:0/3970597300 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f75cc004290 con 0x7f75d80fdd40 2024-03-24T10:56:01.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:01.609+0000 7f75bffff700 1 -- 172.21.15.94:0/3970597300 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75cc01c630 con 0x7f75d80fdd40 2024-03-24T10:56:01.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:01.609+0000 7f75ddcb7700 1 -- 172.21.15.94:0/3970597300 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75d81117f0 con 0x7f75d80fdd40 2024-03-24T10:56:01.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:01.610+0000 7f75ddcb7700 1 -- 172.21.15.94:0/3970597300 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75d805f5f0 con 0x7f75d80fdd40 2024-03-24T10:56:01.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:01.610+0000 7f75bffff700 1 -- 172.21.15.94:0/3970597300 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f75cc013420 con 0x7f75d80fdd40 2024-03-24T10:56:01.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:01.611+0000 7f75bffff700 1 --2- 172.21.15.94:0/3970597300 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f75c006d7e0 0x7f75c006fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:56:01.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:01.611+0000 7f75bffff700 1 -- 172.21.15.94:0/3970597300 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f75cc08db40 con 0x7f75d80fdd40 2024-03-24T10:56:01.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:01.611+0000 7f75d77fe700 1 --2- 172.21.15.94:0/3970597300 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f75c006d7e0 0x7f75c006fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:56:01.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:01.611+0000 7f75d77fe700 1 --2- 172.21.15.94:0/3970597300 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f75c006d7e0 0x7f75c006fc90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f75c8009bf0 tx=0x7f75c80091a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:56:01.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:01.616+0000 7f75bffff700 1 -- 172.21.15.94:0/3970597300 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f75cc05ae40 con 0x7f75d80fdd40 2024-03-24T10:56:01.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:01.835+0000 7f75ddcb7700 1 -- 172.21.15.94:0/3970597300 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f75d8111a80 con 0x7f75c006d7e0 2024-03-24T10:56:01.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:01.837+0000 7f75bffff700 1 -- 172.21.15.94:0/3970597300 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f75d8111a80 con 0x7f75c006d7e0 2024-03-24T10:56:01.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:01.839+0000 7f75ddcb7700 1 -- 172.21.15.94:0/3970597300 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f75c006d7e0 msgr2=0x7f75c006fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:56:01.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:01.839+0000 7f75ddcb7700 1 --2- 172.21.15.94:0/3970597300 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f75c006d7e0 0x7f75c006fc90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f75c8009bf0 tx=0x7f75c80091a0 comp rx=0 tx=0).stop 2024-03-24T10:56:01.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:01.839+0000 7f75ddcb7700 1 -- 172.21.15.94:0/3970597300 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75d80fdd40 msgr2=0x7f75d8110180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:56:01.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:01.839+0000 7f75ddcb7700 1 --2- 172.21.15.94:0/3970597300 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75d80fdd40 0x7f75d8110180 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f75cc009150 tx=0x7f75cc009e30 comp rx=0 tx=0).stop 2024-03-24T10:56:01.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:01.839+0000 7f75ddcb7700 1 -- 172.21.15.94:0/3970597300 shutdown_connections 2024-03-24T10:56:01.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:01.839+0000 7f75ddcb7700 1 --2- 172.21.15.94:0/3970597300 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f75c006d7e0 0x7f75c006fc90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:56:01.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:01.839+0000 7f75ddcb7700 1 --2- 172.21.15.94:0/3970597300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75d80fd3f0 0x7f75d810fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:56:01.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:01.839+0000 7f75ddcb7700 1 --2- 172.21.15.94:0/3970597300 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75d80fdd40 0x7f75d8110180 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:56:01.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:01.839+0000 7f75ddcb7700 1 -- 172.21.15.94:0/3970597300 >> 172.21.15.94:0/3970597300 conn(0x7f75d80f6ba0 msgr2=0x7f75d80f6fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:56:01.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:01.839+0000 7f75ddcb7700 1 -- 172.21.15.94:0/3970597300 shutdown_connections 2024-03-24T10:56:01.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:01.839+0000 7f75ddcb7700 1 -- 172.21.15.94:0/3970597300 wait complete. 2024-03-24T10:56:02.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:56:02 smithi094 ceph-mon[93957]: pgmap v261: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:56:02.562 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -e sha1=21b3bd672163092b4e542f772797fcba02d3d4de -- bash -c 'ceph orch ps' 2024-03-24T10:56:02.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:56:02 smithi096 ceph-mon[84258]: pgmap v261: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:56:02.870 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:56:03.670 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:56:03 smithi094 ceph-mon[93957]: from='client.34522 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:56:03.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:56:03 smithi096 ceph-mon[84258]: from='client.34522 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:56:04.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:04.458+0000 7f2a478e1700 1 -- 172.21.15.94:0/1052606989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a400fe820 msgr2=0x7f2a40106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:56:04.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:04.458+0000 7f2a478e1700 1 --2- 172.21.15.94:0/1052606989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a400fe820 0x7f2a40106e60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2a30004940 tx=0x7f2a30009e30 comp rx=0 tx=0).stop 2024-03-24T10:56:04.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:04.458+0000 7f2a478e1700 1 -- 172.21.15.94:0/1052606989 shutdown_connections 2024-03-24T10:56:04.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:04.458+0000 7f2a478e1700 1 --2- 172.21.15.94:0/1052606989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a400fe820 0x7f2a40106e60 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:56:04.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:04.458+0000 7f2a478e1700 1 --2- 172.21.15.94:0/1052606989 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a400fde80 0x7f2a400fe250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:56:04.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:04.458+0000 7f2a478e1700 1 -- 172.21.15.94:0/1052606989 >> 172.21.15.94:0/1052606989 conn(0x7f2a400f78d0 msgr2=0x7f2a400f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:56:04.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:04.458+0000 7f2a478e1700 1 -- 172.21.15.94:0/1052606989 shutdown_connections 2024-03-24T10:56:04.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:04.458+0000 7f2a478e1700 1 -- 172.21.15.94:0/1052606989 wait complete. 2024-03-24T10:56:04.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:04.458+0000 7f2a478e1700 1 Processor -- start 2024-03-24T10:56:04.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:04.458+0000 7f2a478e1700 1 -- start start 2024-03-24T10:56:04.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:04.459+0000 7f2a478e1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a400fde80 0x7f2a40113b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:56:04.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:04.459+0000 7f2a478e1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a400fe820 0x7f2a40114080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:56:04.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:04.459+0000 7f2a478e1700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a401146d0 con 0x7f2a400fde80 2024-03-24T10:56:04.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:04.459+0000 7f2a478e1700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4010dac0 con 0x7f2a400fe820 2024-03-24T10:56:04.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:04.459+0000 7f2a4567d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a400fde80 0x7f2a40113b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:56:04.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:04.459+0000 7f2a4567d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a400fde80 0x7f2a40113b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55022/0 (socket says 172.21.15.94:55022) 2024-03-24T10:56:04.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:04.459+0000 7f2a4567d700 1 -- 172.21.15.94:0/1147737383 learned_addr learned my addr 172.21.15.94:0/1147737383 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:56:04.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:04.459+0000 7f2a44e7c700 1 --2- 172.21.15.94:0/1147737383 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a400fe820 0x7f2a40114080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:56:04.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:04.459+0000 7f2a4567d700 1 -- 172.21.15.94:0/1147737383 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a400fe820 msgr2=0x7f2a40114080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:56:04.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:04.459+0000 7f2a4567d700 1 --2- 172.21.15.94:0/1147737383 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a400fe820 0x7f2a40114080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:56:04.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:04.459+0000 7f2a4567d700 1 -- 172.21.15.94:0/1147737383 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a30005040 con 0x7f2a400fde80 2024-03-24T10:56:04.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:04.459+0000 7f2a4567d700 1 --2- 172.21.15.94:0/1147737383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a400fde80 0x7f2a40113b40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c002a50 tx=0x7f2a3c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:56:04.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:04.459+0000 7f2a35ffb700 1 -- 172.21.15.94:0/1147737383 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a3c007cc0 con 0x7f2a400fde80 2024-03-24T10:56:04.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:04.459+0000 7f2a35ffb700 1 -- 172.21.15.94:0/1147737383 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a3c00f040 con 0x7f2a400fde80 2024-03-24T10:56:04.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:04.459+0000 7f2a478e1700 1 -- 172.21.15.94:0/1147737383 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a4010dda0 con 0x7f2a400fde80 2024-03-24T10:56:04.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:04.460+0000 7f2a35ffb700 1 -- 172.21.15.94:0/1147737383 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a3c009640 con 0x7f2a400fde80 2024-03-24T10:56:04.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:04.460+0000 7f2a478e1700 1 -- 172.21.15.94:0/1147737383 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a4010e2e0 con 0x7f2a400fde80 2024-03-24T10:56:04.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:04.461+0000 7f2a478e1700 1 -- 172.21.15.94:0/1147737383 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a40066710 con 0x7f2a400fde80 2024-03-24T10:56:04.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:04.461+0000 7f2a35ffb700 1 -- 172.21.15.94:0/1147737383 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f2a3c0097a0 con 0x7f2a400fde80 2024-03-24T10:56:04.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:04.461+0000 7f2a35ffb700 1 --2- 172.21.15.94:0/1147737383 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f2a2c06d7e0 0x7f2a2c06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:56:04.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:04.461+0000 7f2a35ffb700 1 -- 172.21.15.94:0/1147737383 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f2a3c08cd80 con 0x7f2a400fde80 2024-03-24T10:56:04.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:04.461+0000 7f2a44e7c700 1 --2- 172.21.15.94:0/1147737383 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f2a2c06d7e0 0x7f2a2c06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:56:04.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:04.462+0000 7f2a44e7c700 1 --2- 172.21.15.94:0/1147737383 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f2a2c06d7e0 0x7f2a2c06fc90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2a4010ec80 tx=0x7f2a30009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:56:04.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:04.466+0000 7f2a35ffb700 1 -- 172.21.15.94:0/1147737383 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f2a3c05a000 con 0x7f2a400fde80 2024-03-24T10:56:04.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:56:04 smithi094 ceph-mon[93957]: pgmap v262: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-24T10:56:04.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:04.676+0000 7f2a478e1700 1 -- 172.21.15.94:0/1147737383 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a40109e40 con 0x7f2a2c06d7e0 2024-03-24T10:56:04.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:04.686+0000 7f2a35ffb700 1 -- 172.21.15.94:0/1147737383 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f2a40109e40 con 0x7f2a2c06d7e0 2024-03-24T10:56:04.687 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T10:56:04.687 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.smithi094 smithi094 *:9093,9094 running (59s) 39s ago 18m 15.0M - 0.25.0 c8568f914cd2 aca214c66fba 2024-03-24T10:56:04.687 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi094 smithi094 running (6m) 39s ago 18m 6651k - 19.0.0-1855-g21b3bd67 553463a90fd5 2b0897df571d 2024-03-24T10:56:04.687 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi096 smithi096 running (6m) 93s ago 16m 6669k - 19.0.0-1855-g21b3bd67 553463a90fd5 1759a9ff2805 2024-03-24T10:56:04.687 INFO:teuthology.orchestra.run.smithi094.stdout:grafana.smithi094 smithi094 *:3000 running (44s) 39s ago 17m 54.3M - 9.4.12 a0ea5e3c46d9 d506da6aef36 2024-03-24T10:56:04.687 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.efiqte smithi094 running (2m) 39s ago 12m 85.9M - 19.0.0-1855-g21b3bd67 553463a90fd5 732c47b17971 2024-03-24T10:56:04.687 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.llefgu smithi094 running (2m) 39s ago 12m 87.2M - 19.0.0-1855-g21b3bd67 553463a90fd5 6e06f8a6dde0 2024-03-24T10:56:04.687 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi096.egbmoq smithi096 running (2m) 93s ago 12m 14.1M - 19.0.0-1855-g21b3bd67 553463a90fd5 f35fd7f23742 2024-03-24T10:56:04.687 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi096.jtebhh smithi096 running (113s) 93s ago 12m 13.9M - 19.0.0-1855-g21b3bd67 553463a90fd5 8b5ab9f96e5a 2024-03-24T10:56:04.687 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi094.yaethu smithi094 *:8443,9283,8765 running (8m) 39s ago 19m 511M - 19.0.0-1855-g21b3bd67 553463a90fd5 d3adab76b18f 2024-03-24T10:56:04.688 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi096.gfonnu smithi096 *:8443,9283,8765 running (7m) 93s ago 16m 450M - 19.0.0-1855-g21b3bd67 553463a90fd5 22937a059eb8 2024-03-24T10:56:04.688 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi094 smithi094 running (7m) 39s ago 19m 56.9M 2048M 19.0.0-1855-g21b3bd67 553463a90fd5 c779d611e29a 2024-03-24T10:56:04.688 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi096 smithi096 running (7m) 93s ago 16m 38.5M 2048M 19.0.0-1855-g21b3bd67 553463a90fd5 ed9bd0bd57f6 2024-03-24T10:56:04.688 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi094 smithi094 *:9100 running (100s) 39s ago 18m 7860k - 1.5.0 0da6a335fe13 003df2346f4f 2024-03-24T10:56:04.688 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi096 smithi096 *:9100 running (96s) 93s ago 16m 5481k - 1.5.0 0da6a335fe13 67d65fecf9df 2024-03-24T10:56:04.688 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (6m) 39s ago 15m 111M 1557M 19.0.0-1855-g21b3bd67 553463a90fd5 ec57ede0c9e4 2024-03-24T10:56:04.688 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi094 running (5m) 39s ago 15m 124M 1557M 19.0.0-1855-g21b3bd67 553463a90fd5 c5b8b2bbf8b5 2024-03-24T10:56:04.688 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi094 running (4m) 39s ago 15m 102M 1557M 19.0.0-1855-g21b3bd67 553463a90fd5 c961dcdc746f 2024-03-24T10:56:04.688 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi096 running (4m) 93s ago 14m 121M 2581M 19.0.0-1855-g21b3bd67 553463a90fd5 6323fdc6e6ca 2024-03-24T10:56:04.688 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi096 running (3m) 93s ago 14m 111M 2581M 19.0.0-1855-g21b3bd67 553463a90fd5 3e2536bcb263 2024-03-24T10:56:04.688 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi096 running (3m) 93s ago 13m 103M 2581M 19.0.0-1855-g21b3bd67 553463a90fd5 2f77f1fa20e2 2024-03-24T10:56:04.688 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.smithi094 smithi094 *:9095 running (76s) 39s ago 17m 44.8M - 2.43.0 a07b618ecd1d 326ddd336bc5 2024-03-24T10:56:04.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:04.688+0000 7f2a478e1700 1 -- 172.21.15.94:0/1147737383 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f2a2c06d7e0 msgr2=0x7f2a2c06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:56:04.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:04.688+0000 7f2a478e1700 1 --2- 172.21.15.94:0/1147737383 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f2a2c06d7e0 0x7f2a2c06fc90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2a4010ec80 tx=0x7f2a30009e30 comp rx=0 tx=0).stop 2024-03-24T10:56:04.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:04.688+0000 7f2a478e1700 1 -- 172.21.15.94:0/1147737383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a400fde80 msgr2=0x7f2a40113b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:56:04.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:04.688+0000 7f2a478e1700 1 --2- 172.21.15.94:0/1147737383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a400fde80 0x7f2a40113b40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c002a50 tx=0x7f2a3c0046b0 comp rx=0 tx=0).stop 2024-03-24T10:56:04.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:04.688+0000 7f2a478e1700 1 -- 172.21.15.94:0/1147737383 shutdown_connections 2024-03-24T10:56:04.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:04.688+0000 7f2a478e1700 1 --2- 172.21.15.94:0/1147737383 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f2a2c06d7e0 0x7f2a2c06fc90 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:56:04.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:04.688+0000 7f2a478e1700 1 --2- 172.21.15.94:0/1147737383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a400fde80 0x7f2a40113b40 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:56:04.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:04.688+0000 7f2a478e1700 1 --2- 172.21.15.94:0/1147737383 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a400fe820 0x7f2a40114080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:56:04.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:04.688+0000 7f2a478e1700 1 -- 172.21.15.94:0/1147737383 >> 172.21.15.94:0/1147737383 conn(0x7f2a400f78d0 msgr2=0x7f2a400f8ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:56:04.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:04.688+0000 7f2a478e1700 1 -- 172.21.15.94:0/1147737383 shutdown_connections 2024-03-24T10:56:04.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:04.688+0000 7f2a478e1700 1 -- 172.21.15.94:0/1147737383 wait complete. 2024-03-24T10:56:04.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:56:04 smithi096 ceph-mon[84258]: pgmap v262: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-24T10:56:05.443 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -e sha1=21b3bd672163092b4e542f772797fcba02d3d4de -- bash -c 'ceph orch upgrade status' 2024-03-24T10:56:05.723 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:56:05 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:56:05.740 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:56:05.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:56:05 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:56:06.723 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:56:06 smithi094 ceph-mon[93957]: from='client.34526 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:56:06.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:56:06 smithi096 ceph-mon[84258]: from='client.34526 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:56:07.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:07.577+0000 7f063f37b700 1 -- 172.21.15.94:0/3418619985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0638101d10 msgr2=0x7f06381020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:56:07.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:07.577+0000 7f063f37b700 1 --2- 172.21.15.94:0/3418619985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0638101d10 0x7f06381020e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0630009ce0 tx=0x7f0630004d10 comp rx=0 tx=0).stop 2024-03-24T10:56:07.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:07.577+0000 7f063f37b700 1 -- 172.21.15.94:0/3418619985 shutdown_connections 2024-03-24T10:56:07.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:07.577+0000 7f063f37b700 1 --2- 172.21.15.94:0/3418619985 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06381026b0 0x7f06380fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:56:07.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:07.577+0000 7f063f37b700 1 --2- 172.21.15.94:0/3418619985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0638101d10 0x7f06381020e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:56:07.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:07.577+0000 7f063f37b700 1 -- 172.21.15.94:0/3418619985 >> 172.21.15.94:0/3418619985 conn(0x7f06380f7880 msgr2=0x7f06380f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:56:07.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:07.577+0000 7f063f37b700 1 -- 172.21.15.94:0/3418619985 shutdown_connections 2024-03-24T10:56:07.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:07.577+0000 7f063f37b700 1 -- 172.21.15.94:0/3418619985 wait complete. 2024-03-24T10:56:07.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:07.578+0000 7f063f37b700 1 Processor -- start 2024-03-24T10:56:07.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:07.578+0000 7f063f37b700 1 -- start start 2024-03-24T10:56:07.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:07.578+0000 7f063f37b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0638101d10 0x7f0638112ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:56:07.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:07.578+0000 7f063f37b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06381026b0 0x7f0638113420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:56:07.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:07.578+0000 7f063f37b700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0638113b00 con 0x7f0638101d10 2024-03-24T10:56:07.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:07.578+0000 7f063f37b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063810ce40 con 0x7f06381026b0 2024-03-24T10:56:07.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:07.578+0000 7f063d117700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0638101d10 0x7f0638112ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:56:07.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:07.578+0000 7f063d117700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0638101d10 0x7f0638112ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55048/0 (socket says 172.21.15.94:55048) 2024-03-24T10:56:07.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:07.578+0000 7f063d117700 1 -- 172.21.15.94:0/442131175 learned_addr learned my addr 172.21.15.94:0/442131175 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:56:07.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:07.579+0000 7f063c916700 1 --2- 172.21.15.94:0/442131175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06381026b0 0x7f0638113420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:56:07.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:07.579+0000 7f063d117700 1 -- 172.21.15.94:0/442131175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06381026b0 msgr2=0x7f0638113420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:56:07.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:07.579+0000 7f063d117700 1 --2- 172.21.15.94:0/442131175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06381026b0 0x7f0638113420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:56:07.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:07.579+0000 7f063d117700 1 -- 172.21.15.94:0/442131175 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0630009940 con 0x7f0638101d10 2024-03-24T10:56:07.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:07.579+0000 7f063d117700 1 --2- 172.21.15.94:0/442131175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0638101d10 0x7f0638112ee0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f063000be90 tx=0x7f0630004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:56:07.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:07.579+0000 7f0635ffb700 1 -- 172.21.15.94:0/442131175 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f063000e040 con 0x7f0638101d10 2024-03-24T10:56:07.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:07.579+0000 7f063f37b700 1 -- 172.21.15.94:0/442131175 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f063810d060 con 0x7f0638101d10 2024-03-24T10:56:07.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:07.579+0000 7f0635ffb700 1 -- 172.21.15.94:0/442131175 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0630008ea0 con 0x7f0638101d10 2024-03-24T10:56:07.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:07.579+0000 7f0635ffb700 1 -- 172.21.15.94:0/442131175 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0630019860 con 0x7f0638101d10 2024-03-24T10:56:07.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:07.579+0000 7f063f37b700 1 -- 172.21.15.94:0/442131175 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f063810d5c0 con 0x7f0638101d10 2024-03-24T10:56:07.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:07.580+0000 7f063f37b700 1 -- 172.21.15.94:0/442131175 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06380fd460 con 0x7f0638101d10 2024-03-24T10:56:07.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:07.581+0000 7f0635ffb700 1 -- 172.21.15.94:0/442131175 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f0630023c30 con 0x7f0638101d10 2024-03-24T10:56:07.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:07.581+0000 7f0635ffb700 1 --2- 172.21.15.94:0/442131175 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f062006d7e0 0x7f062006fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:56:07.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:07.581+0000 7f0635ffb700 1 -- 172.21.15.94:0/442131175 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f0630090b00 con 0x7f0638101d10 2024-03-24T10:56:07.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:07.581+0000 7f063c916700 1 --2- 172.21.15.94:0/442131175 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f062006d7e0 0x7f062006fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:56:07.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:07.582+0000 7f063c916700 1 --2- 172.21.15.94:0/442131175 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f062006d7e0 0x7f062006fc90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f062c006fd0 tx=0x7f062c008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:56:07.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:07.586+0000 7f0635ffb700 1 -- 172.21.15.94:0/442131175 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f063005dd80 con 0x7f0638101d10 2024-03-24T10:56:07.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:56:07 smithi094 ceph-mon[93957]: pgmap v263: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:56:07.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:07.804+0000 7f063f37b700 1 -- 172.21.15.94:0/442131175 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f063804bed0 con 0x7f062006d7e0 2024-03-24T10:56:07.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:07.805+0000 7f0635ffb700 1 -- 172.21.15.94:0/442131175 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f063804bed0 con 0x7f062006d7e0 2024-03-24T10:56:07.806 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-03-24T10:56:07.806 INFO:teuthology.orchestra.run.smithi094.stdout: "target_image": null, 2024-03-24T10:56:07.806 INFO:teuthology.orchestra.run.smithi094.stdout: "in_progress": false, 2024-03-24T10:56:07.806 INFO:teuthology.orchestra.run.smithi094.stdout: "which": "", 2024-03-24T10:56:07.806 INFO:teuthology.orchestra.run.smithi094.stdout: "services_complete": [], 2024-03-24T10:56:07.806 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": null, 2024-03-24T10:56:07.806 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "", 2024-03-24T10:56:07.806 INFO:teuthology.orchestra.run.smithi094.stdout: "is_paused": false 2024-03-24T10:56:07.806 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-03-24T10:56:07.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:07.807+0000 7f063f37b700 1 -- 172.21.15.94:0/442131175 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f062006d7e0 msgr2=0x7f062006fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:56:07.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:07.807+0000 7f063f37b700 1 --2- 172.21.15.94:0/442131175 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f062006d7e0 0x7f062006fc90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f062c006fd0 tx=0x7f062c008040 comp rx=0 tx=0).stop 2024-03-24T10:56:07.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:07.807+0000 7f063f37b700 1 -- 172.21.15.94:0/442131175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0638101d10 msgr2=0x7f0638112ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:56:07.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:07.807+0000 7f063f37b700 1 --2- 172.21.15.94:0/442131175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0638101d10 0x7f0638112ee0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f063000be90 tx=0x7f0630004d10 comp rx=0 tx=0).stop 2024-03-24T10:56:07.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:07.807+0000 7f063f37b700 1 -- 172.21.15.94:0/442131175 shutdown_connections 2024-03-24T10:56:07.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:07.807+0000 7f063f37b700 1 --2- 172.21.15.94:0/442131175 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f062006d7e0 0x7f062006fc90 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:56:07.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:07.807+0000 7f063f37b700 1 --2- 172.21.15.94:0/442131175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0638101d10 0x7f0638112ee0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:56:07.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:07.807+0000 7f063f37b700 1 --2- 172.21.15.94:0/442131175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06381026b0 0x7f0638113420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:56:07.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:07.807+0000 7f063f37b700 1 -- 172.21.15.94:0/442131175 >> 172.21.15.94:0/442131175 conn(0x7f06380f7880 msgr2=0x7f06380f8830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:56:07.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:07.807+0000 7f063f37b700 1 -- 172.21.15.94:0/442131175 shutdown_connections 2024-03-24T10:56:07.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:07.807+0000 7f063f37b700 1 -- 172.21.15.94:0/442131175 wait complete. 2024-03-24T10:56:07.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:56:07 smithi096 ceph-mon[84258]: pgmap v263: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:56:08.406 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -e sha1=21b3bd672163092b4e542f772797fcba02d3d4de -- bash -c 'ceph health detail' 2024-03-24T10:56:08.671 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:56:09.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:56:09 smithi094 ceph-mon[93957]: pgmap v264: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 833 B/s rd, 1 op/s 2024-03-24T10:56:09.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:56:09 smithi094 ceph-mon[93957]: from='client.34530 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:56:09.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:56:09 smithi096 ceph-mon[84258]: pgmap v264: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 833 B/s rd, 1 op/s 2024-03-24T10:56:09.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:56:09 smithi096 ceph-mon[84258]: from='client.34530 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:56:10.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:10.283+0000 7f8f73649700 1 -- 172.21.15.94:0/1073050379 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f6c0fde10 msgr2=0x7f8f6c0fe1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:56:10.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:10.283+0000 7f8f73649700 1 --2- 172.21.15.94:0/1073050379 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f6c0fde10 0x7f8f6c0fe1e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8f68004940 tx=0x7f8f68009e30 comp rx=0 tx=0).stop 2024-03-24T10:56:10.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:10.283+0000 7f8f73649700 1 -- 172.21.15.94:0/1073050379 shutdown_connections 2024-03-24T10:56:10.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:10.283+0000 7f8f73649700 1 --2- 172.21.15.94:0/1073050379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f6c0fe7b0 0x7f8f6c106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:56:10.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:10.283+0000 7f8f73649700 1 --2- 172.21.15.94:0/1073050379 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f6c0fde10 0x7f8f6c0fe1e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:56:10.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:10.283+0000 7f8f73649700 1 -- 172.21.15.94:0/1073050379 >> 172.21.15.94:0/1073050379 conn(0x7f8f6c0f7880 msgr2=0x7f8f6c0f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:56:10.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:10.283+0000 7f8f73649700 1 -- 172.21.15.94:0/1073050379 shutdown_connections 2024-03-24T10:56:10.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:10.283+0000 7f8f73649700 1 -- 172.21.15.94:0/1073050379 wait complete. 2024-03-24T10:56:10.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:10.284+0000 7f8f73649700 1 Processor -- start 2024-03-24T10:56:10.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:10.284+0000 7f8f73649700 1 -- start start 2024-03-24T10:56:10.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:10.284+0000 7f8f73649700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f6c0fde10 0x7f8f6c113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:56:10.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:10.284+0000 7f8f73649700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f6c0fe7b0 0x7f8f6c114400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:56:10.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:10.284+0000 7f8f73649700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c114a50 con 0x7f8f6c0fde10 2024-03-24T10:56:10.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:10.284+0000 7f8f73649700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c10de20 con 0x7f8f6c0fe7b0 2024-03-24T10:56:10.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:10.284+0000 7f8f713e5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f6c0fde10 0x7f8f6c113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:56:10.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:10.284+0000 7f8f713e5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f6c0fde10 0x7f8f6c113ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33884/0 (socket says 172.21.15.94:33884) 2024-03-24T10:56:10.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:10.284+0000 7f8f713e5700 1 -- 172.21.15.94:0/1813736511 learned_addr learned my addr 172.21.15.94:0/1813736511 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:56:10.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:10.284+0000 7f8f70be4700 1 --2- 172.21.15.94:0/1813736511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f6c0fe7b0 0x7f8f6c114400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:56:10.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:10.284+0000 7f8f713e5700 1 -- 172.21.15.94:0/1813736511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f6c0fe7b0 msgr2=0x7f8f6c114400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:56:10.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:10.284+0000 7f8f713e5700 1 --2- 172.21.15.94:0/1813736511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f6c0fe7b0 0x7f8f6c114400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:56:10.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:10.284+0000 7f8f713e5700 1 -- 172.21.15.94:0/1813736511 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f68005040 con 0x7f8f6c0fde10 2024-03-24T10:56:10.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:10.284+0000 7f8f70be4700 1 --2- 172.21.15.94:0/1813736511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f6c0fe7b0 0x7f8f6c114400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T10:56:10.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:10.284+0000 7f8f713e5700 1 --2- 172.21.15.94:0/1813736511 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f6c0fde10 0x7f8f6c113ec0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8f6800b3b0 tx=0x7f8f68009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:56:10.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:10.285+0000 7f8f61ffb700 1 -- 172.21.15.94:0/1813736511 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f6800e040 con 0x7f8f6c0fde10 2024-03-24T10:56:10.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:10.285+0000 7f8f73649700 1 -- 172.21.15.94:0/1813736511 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f6c10e0a0 con 0x7f8f6c0fde10 2024-03-24T10:56:10.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:10.285+0000 7f8f61ffb700 1 -- 172.21.15.94:0/1813736511 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8f68013420 con 0x7f8f6c0fde10 2024-03-24T10:56:10.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:10.285+0000 7f8f61ffb700 1 -- 172.21.15.94:0/1813736511 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f6801c630 con 0x7f8f6c0fde10 2024-03-24T10:56:10.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:10.285+0000 7f8f73649700 1 -- 172.21.15.94:0/1813736511 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f6c10e5e0 con 0x7f8f6c0fde10 2024-03-24T10:56:10.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:10.286+0000 7f8f73649700 1 -- 172.21.15.94:0/1813736511 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f6c066770 con 0x7f8f6c0fde10 2024-03-24T10:56:10.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:10.287+0000 7f8f61ffb700 1 -- 172.21.15.94:0/1813736511 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f8f6800baf0 con 0x7f8f6c0fde10 2024-03-24T10:56:10.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:10.287+0000 7f8f61ffb700 1 --2- 172.21.15.94:0/1813736511 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f8f5806d7f0 0x7f8f5806fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:56:10.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:10.287+0000 7f8f61ffb700 1 -- 172.21.15.94:0/1813736511 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f8f6808e2e0 con 0x7f8f6c0fde10 2024-03-24T10:56:10.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:10.287+0000 7f8f70be4700 1 --2- 172.21.15.94:0/1813736511 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f8f5806d7f0 0x7f8f5806fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:56:10.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:10.288+0000 7f8f70be4700 1 --2- 172.21.15.94:0/1813736511 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f8f5806d7f0 0x7f8f5806fca0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c007fd0 tx=0x7f8f5c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:56:10.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:10.292+0000 7f8f61ffb700 1 -- 172.21.15.94:0/1813736511 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8f6805b5e0 con 0x7f8f6c0fde10 2024-03-24T10:56:10.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:10.639+0000 7f8f73649700 1 -- 172.21.15.94:0/1813736511 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f8f6c05f5f0 con 0x7f8f6c0fde10 2024-03-24T10:56:10.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:10.639+0000 7f8f61ffb700 1 -- 172.21.15.94:0/1813736511 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f8f6805b170 con 0x7f8f6c0fde10 2024-03-24T10:56:10.640 INFO:teuthology.orchestra.run.smithi094.stdout:HEALTH_OK 2024-03-24T10:56:10.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:10.641+0000 7f8f73649700 1 -- 172.21.15.94:0/1813736511 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f8f5806d7f0 msgr2=0x7f8f5806fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:56:10.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:10.641+0000 7f8f73649700 1 --2- 172.21.15.94:0/1813736511 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f8f5806d7f0 0x7f8f5806fca0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c007fd0 tx=0x7f8f5c009040 comp rx=0 tx=0).stop 2024-03-24T10:56:10.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:10.641+0000 7f8f73649700 1 -- 172.21.15.94:0/1813736511 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f6c0fde10 msgr2=0x7f8f6c113ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:56:10.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:10.641+0000 7f8f73649700 1 --2- 172.21.15.94:0/1813736511 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f6c0fde10 0x7f8f6c113ec0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8f6800b3b0 tx=0x7f8f68009e30 comp rx=0 tx=0).stop 2024-03-24T10:56:10.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:10.641+0000 7f8f73649700 1 -- 172.21.15.94:0/1813736511 shutdown_connections 2024-03-24T10:56:10.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:10.641+0000 7f8f73649700 1 --2- 172.21.15.94:0/1813736511 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f8f5806d7f0 0x7f8f5806fca0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:56:10.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:10.641+0000 7f8f73649700 1 --2- 172.21.15.94:0/1813736511 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f6c0fde10 0x7f8f6c113ec0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:56:10.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:10.641+0000 7f8f73649700 1 --2- 172.21.15.94:0/1813736511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f6c0fe7b0 0x7f8f6c114400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:56:10.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:10.641+0000 7f8f73649700 1 -- 172.21.15.94:0/1813736511 >> 172.21.15.94:0/1813736511 conn(0x7f8f6c0f7880 msgr2=0x7f8f6c0f8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:56:10.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:10.641+0000 7f8f73649700 1 -- 172.21.15.94:0/1813736511 shutdown_connections 2024-03-24T10:56:10.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:10.641+0000 7f8f73649700 1 -- 172.21.15.94:0/1813736511 wait complete. 2024-03-24T10:56:11.238 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -e sha1=21b3bd672163092b4e542f772797fcba02d3d4de -- bash -c 'ceph versions' 2024-03-24T10:56:11.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:56:11 smithi094 ceph-mon[93957]: pgmap v265: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:56:11.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:56:11 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/1813736511' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T10:56:11.505 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:56:11.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:56:11 smithi096 ceph-mon[84258]: pgmap v265: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:56:11.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:56:11 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/1813736511' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T10:56:13.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:13.307+0000 7fa9d24a6700 1 -- 172.21.15.94:0/1083565154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9cc101d70 msgr2=0x7fa9cc102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:56:13.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:13.307+0000 7fa9d24a6700 1 --2- 172.21.15.94:0/1083565154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9cc101d70 0x7fa9cc102140 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc004940 tx=0x7fa9bc009e30 comp rx=0 tx=0).stop 2024-03-24T10:56:13.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:13.307+0000 7fa9d24a6700 1 -- 172.21.15.94:0/1083565154 shutdown_connections 2024-03-24T10:56:13.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:13.307+0000 7fa9d24a6700 1 --2- 172.21.15.94:0/1083565154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9cc102710 0x7fa9cc0fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:56:13.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:13.307+0000 7fa9d24a6700 1 --2- 172.21.15.94:0/1083565154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9cc101d70 0x7fa9cc102140 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:56:13.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:13.307+0000 7fa9d24a6700 1 -- 172.21.15.94:0/1083565154 >> 172.21.15.94:0/1083565154 conn(0x7fa9cc072bc0 msgr2=0x7fa9cc072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:56:13.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:13.307+0000 7fa9d24a6700 1 -- 172.21.15.94:0/1083565154 shutdown_connections 2024-03-24T10:56:13.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:13.307+0000 7fa9d24a6700 1 -- 172.21.15.94:0/1083565154 wait complete. 2024-03-24T10:56:13.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:13.307+0000 7fa9d24a6700 1 Processor -- start 2024-03-24T10:56:13.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:13.308+0000 7fa9d24a6700 1 -- start start 2024-03-24T10:56:13.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:13.308+0000 7fa9d24a6700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9cc101d70 0x7fa9cc112a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:56:13.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:13.308+0000 7fa9d24a6700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9cc102710 0x7fa9cc112fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:56:13.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:13.308+0000 7fa9d24a6700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc1135f0 con 0x7fa9cc102710 2024-03-24T10:56:13.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:13.308+0000 7fa9d24a6700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc10c9e0 con 0x7fa9cc101d70 2024-03-24T10:56:13.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:13.308+0000 7fa9cb7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9cc102710 0x7fa9cc112fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:56:13.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:13.308+0000 7fa9cb7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9cc102710 0x7fa9cc112fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33906/0 (socket says 172.21.15.94:33906) 2024-03-24T10:56:13.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:13.308+0000 7fa9cb7fe700 1 -- 172.21.15.94:0/39177720 learned_addr learned my addr 172.21.15.94:0/39177720 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:56:13.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:13.308+0000 7fa9cbfff700 1 --2- 172.21.15.94:0/39177720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9cc101d70 0x7fa9cc112a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:56:13.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:13.308+0000 7fa9cb7fe700 1 -- 172.21.15.94:0/39177720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9cc101d70 msgr2=0x7fa9cc112a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:56:13.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:13.308+0000 7fa9cb7fe700 1 --2- 172.21.15.94:0/39177720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9cc101d70 0x7fa9cc112a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:56:13.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:13.308+0000 7fa9cb7fe700 1 -- 172.21.15.94:0/39177720 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9bc005040 con 0x7fa9cc102710 2024-03-24T10:56:13.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:13.308+0000 7fa9cb7fe700 1 --2- 172.21.15.94:0/39177720 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9cc102710 0x7fa9cc112fa0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0002a80 tx=0x7fa9c0004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:56:13.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:13.309+0000 7fa9c8ff9700 1 -- 172.21.15.94:0/39177720 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9c0004e80 con 0x7fa9cc102710 2024-03-24T10:56:13.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:13.309+0000 7fa9c8ff9700 1 -- 172.21.15.94:0/39177720 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa9c000f040 con 0x7fa9cc102710 2024-03-24T10:56:13.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:13.309+0000 7fa9d24a6700 1 -- 172.21.15.94:0/39177720 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9cc10ccc0 con 0x7fa9cc102710 2024-03-24T10:56:13.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:13.309+0000 7fa9c8ff9700 1 -- 172.21.15.94:0/39177720 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9c000daa0 con 0x7fa9cc102710 2024-03-24T10:56:13.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:13.309+0000 7fa9d24a6700 1 -- 172.21.15.94:0/39177720 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9cc10d280 con 0x7fa9cc102710 2024-03-24T10:56:13.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:13.310+0000 7fa9d24a6700 1 -- 172.21.15.94:0/39177720 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9cc0ff080 con 0x7fa9cc102710 2024-03-24T10:56:13.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:13.310+0000 7fa9c8ff9700 1 -- 172.21.15.94:0/39177720 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fa9c0007cc0 con 0x7fa9cc102710 2024-03-24T10:56:13.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:13.311+0000 7fa9c8ff9700 1 --2- 172.21.15.94:0/39177720 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fa9b406d7e0 0x7fa9b406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:56:13.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:13.311+0000 7fa9c8ff9700 1 -- 172.21.15.94:0/39177720 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fa9c008c560 con 0x7fa9cc102710 2024-03-24T10:56:13.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:13.311+0000 7fa9cbfff700 1 --2- 172.21.15.94:0/39177720 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fa9b406d7e0 0x7fa9b406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:56:13.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:13.311+0000 7fa9cbfff700 1 --2- 172.21.15.94:0/39177720 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fa9b406d7e0 0x7fa9b406fc90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc004f90 tx=0x7fa9bc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:56:13.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:13.316+0000 7fa9c8ff9700 1 -- 172.21.15.94:0/39177720 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa9c0059860 con 0x7fa9cc102710 2024-03-24T10:56:13.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:13.607+0000 7fa9d24a6700 1 -- 172.21.15.94:0/39177720 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa9cc0fd4c0 con 0x7fa9cc102710 2024-03-24T10:56:13.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:13.607+0000 7fa9c8ff9700 1 -- 172.21.15.94:0/39177720 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fa9c00593f0 con 0x7fa9cc102710 2024-03-24T10:56:13.608 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-03-24T10:56:13.608 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": { 2024-03-24T10:56:13.608 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 2 2024-03-24T10:56:13.608 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:56:13.608 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": { 2024-03-24T10:56:13.608 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 2 2024-03-24T10:56:13.609 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:56:13.609 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": { 2024-03-24T10:56:13.609 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 6 2024-03-24T10:56:13.609 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:56:13.609 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-03-24T10:56:13.609 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 4 2024-03-24T10:56:13.609 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:56:13.609 INFO:teuthology.orchestra.run.smithi094.stdout: "overall": { 2024-03-24T10:56:13.609 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 14 2024-03-24T10:56:13.609 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-03-24T10:56:13.609 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-03-24T10:56:13.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:13.609+0000 7fa9d24a6700 1 -- 172.21.15.94:0/39177720 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fa9b406d7e0 msgr2=0x7fa9b406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:56:13.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:13.609+0000 7fa9d24a6700 1 --2- 172.21.15.94:0/39177720 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fa9b406d7e0 0x7fa9b406fc90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc004f90 tx=0x7fa9bc009e30 comp rx=0 tx=0).stop 2024-03-24T10:56:13.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:13.609+0000 7fa9d24a6700 1 -- 172.21.15.94:0/39177720 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9cc102710 msgr2=0x7fa9cc112fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:56:13.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:13.609+0000 7fa9d24a6700 1 --2- 172.21.15.94:0/39177720 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9cc102710 0x7fa9cc112fa0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0002a80 tx=0x7fa9c0004630 comp rx=0 tx=0).stop 2024-03-24T10:56:13.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:13.609+0000 7fa9d24a6700 1 -- 172.21.15.94:0/39177720 shutdown_connections 2024-03-24T10:56:13.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:13.609+0000 7fa9d24a6700 1 --2- 172.21.15.94:0/39177720 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fa9b406d7e0 0x7fa9b406fc90 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:56:13.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:13.609+0000 7fa9d24a6700 1 --2- 172.21.15.94:0/39177720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9cc101d70 0x7fa9cc112a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:56:13.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:13.609+0000 7fa9d24a6700 1 --2- 172.21.15.94:0/39177720 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9cc102710 0x7fa9cc112fa0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:56:13.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:13.609+0000 7fa9d24a6700 1 -- 172.21.15.94:0/39177720 >> 172.21.15.94:0/39177720 conn(0x7fa9cc072bc0 msgr2=0x7fa9cc0fafe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:56:13.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:13.609+0000 7fa9d24a6700 1 -- 172.21.15.94:0/39177720 shutdown_connections 2024-03-24T10:56:13.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:56:13.609+0000 7fa9d24a6700 1 -- 172.21.15.94:0/39177720 wait complete. 2024-03-24T10:56:13.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:56:13 smithi094 ceph-mon[93957]: pgmap v266: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 833 B/s rd, 1 op/s 2024-03-24T10:56:13.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:56:13 smithi096 ceph-mon[84258]: pgmap v266: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 833 B/s rd, 1 op/s 2024-03-24T10:56:14.178 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -e sha1=21b3bd672163092b4e542f772797fcba02d3d4de -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-03-24T10:56:14.481 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:56:14.714 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:56:14 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/39177720' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:56:14.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:56:14 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/39177720' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:56:15.696 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:56:15 smithi094 ceph-mon[93957]: pgmap v267: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:56:15.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:56:15 smithi096 ceph-mon[84258]: pgmap v267: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:56:16.216 INFO:teuthology.orchestra.run.smithi094.stdout:wait for servicemap items w/ changing names to refresh 2024-03-24T10:56:16.788 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:56:16 smithi094 ceph-mon[93957]: pgmap v268: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T10:56:16.812 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -e sha1=21b3bd672163092b4e542f772797fcba02d3d4de -- bash -c 'sleep 60' 2024-03-24T10:56:16.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:56:16 smithi096 ceph-mon[84258]: pgmap v268: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T10:56:17.088 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:56:19.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:56:18 smithi094 ceph-mon[93957]: pgmap v269: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:56:19.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:56:18 smithi096 ceph-mon[84258]: pgmap v269: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:56:21.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:56:20 smithi094 ceph-mon[93957]: pgmap v270: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:56:21.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:56:20 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:56:21.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:56:20 smithi096 ceph-mon[84258]: pgmap v270: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:56:21.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:56:20 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:56:23.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:56:22 smithi094 ceph-mon[93957]: pgmap v271: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:56:23.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:56:22 smithi096 ceph-mon[84258]: pgmap v271: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:56:25.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:56:24 smithi094 ceph-mon[93957]: pgmap v272: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:56:25.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:56:24 smithi096 ceph-mon[84258]: pgmap v272: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:56:27.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:56:26 smithi094 ceph-mon[93957]: pgmap v273: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:56:27.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:56:26 smithi096 ceph-mon[84258]: pgmap v273: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:56:29.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:56:28 smithi094 ceph-mon[93957]: pgmap v274: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:56:29.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:56:28 smithi096 ceph-mon[84258]: pgmap v274: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:56:31.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:56:30 smithi094 ceph-mon[93957]: pgmap v275: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:56:31.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:56:30 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:56:31.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:56:30 smithi096 ceph-mon[84258]: pgmap v275: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:56:31.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:56:30 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:56:32.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:56:31 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:56:32.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:56:31 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:56:32.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:56:31 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:56:32.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:56:31 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:56:32.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:56:31 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:56:32.325 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:56:31 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:56:33.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:56:33 smithi094 ceph-mon[93957]: pgmap v276: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:56:33.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:56:33 smithi096 ceph-mon[84258]: pgmap v276: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:56:35.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:56:35 smithi094 ceph-mon[93957]: pgmap v277: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:56:35.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:56:35 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:56:35.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:56:35 smithi096 ceph-mon[84258]: pgmap v277: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:56:35.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:56:35 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:56:37.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:56:37 smithi094 ceph-mon[93957]: pgmap v278: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:56:37.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:56:37 smithi096 ceph-mon[84258]: pgmap v278: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:56:39.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:56:39 smithi094 ceph-mon[93957]: pgmap v279: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:56:39.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:56:39 smithi096 ceph-mon[84258]: pgmap v279: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:56:41.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:56:41 smithi094 ceph-mon[93957]: pgmap v280: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:56:41.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:56:41 smithi096 ceph-mon[84258]: pgmap v280: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:56:43.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:56:43 smithi094 ceph-mon[93957]: pgmap v281: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:56:43.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:56:43 smithi096 ceph-mon[84258]: pgmap v281: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:56:45.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:56:45 smithi094 ceph-mon[93957]: pgmap v282: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:56:45.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:56:45 smithi096 ceph-mon[84258]: pgmap v282: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:56:47.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:56:47 smithi094 ceph-mon[93957]: pgmap v283: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:56:47.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:56:47 smithi096 ceph-mon[84258]: pgmap v283: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:56:49.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:56:49 smithi094 ceph-mon[93957]: pgmap v284: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:56:49.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:56:49 smithi096 ceph-mon[84258]: pgmap v284: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:56:50.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:56:50 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:56:50.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:56:50 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:56:51.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:56:51 smithi094 ceph-mon[93957]: pgmap v285: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:56:51.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:56:51 smithi096 ceph-mon[84258]: pgmap v285: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:56:53.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:56:53 smithi094 ceph-mon[93957]: pgmap v286: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:56:53.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:56:53 smithi096 ceph-mon[84258]: pgmap v286: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:56:55.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:56:55 smithi094 ceph-mon[93957]: pgmap v287: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:56:55.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:56:55 smithi096 ceph-mon[84258]: pgmap v287: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:56:57.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:56:57 smithi094 ceph-mon[93957]: pgmap v288: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:56:57.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:56:57 smithi096 ceph-mon[84258]: pgmap v288: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:56:59.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:56:59 smithi094 ceph-mon[93957]: pgmap v289: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:56:59.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:56:59 smithi096 ceph-mon[84258]: pgmap v289: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:57:01.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:01 smithi094 ceph-mon[93957]: pgmap v290: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:57:01.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:01 smithi096 ceph-mon[84258]: pgmap v290: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:57:03.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:03 smithi094 ceph-mon[93957]: pgmap v291: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:57:03.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:03 smithi096 ceph-mon[84258]: pgmap v291: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:57:05.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:05 smithi094 ceph-mon[93957]: pgmap v292: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:57:05.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:05 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:57:05.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:05 smithi096 ceph-mon[84258]: pgmap v292: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:57:05.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:05 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:57:07.509 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:07 smithi094 ceph-mon[93957]: pgmap v293: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:57:07.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:07 smithi096 ceph-mon[84258]: pgmap v293: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:57:09.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:09 smithi094 ceph-mon[93957]: pgmap v294: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:57:09.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:09 smithi096 ceph-mon[84258]: pgmap v294: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:57:11.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:11 smithi094 ceph-mon[93957]: pgmap v295: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:57:11.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:11 smithi096 ceph-mon[84258]: pgmap v295: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:57:13.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:13 smithi094 ceph-mon[93957]: pgmap v296: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:57:13.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:13 smithi096 ceph-mon[84258]: pgmap v296: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:57:15.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:15 smithi094 ceph-mon[93957]: pgmap v297: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:57:15.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:15 smithi096 ceph-mon[84258]: pgmap v297: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:57:17.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:17 smithi094 ceph-mon[93957]: pgmap v298: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:57:17.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:17 smithi096 ceph-mon[84258]: pgmap v298: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:57:19.318 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -e sha1=21b3bd672163092b4e542f772797fcba02d3d4de -- bash -c 'ceph orch ps' 2024-03-24T10:57:19.585 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:57:19.707 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:19 smithi094 ceph-mon[93957]: pgmap v299: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:57:19.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:19 smithi096 ceph-mon[84258]: pgmap v299: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:57:20.726 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:20 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:57:20.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:20 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:57:21.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:21.215+0000 7f7f009fc700 1 -- 172.21.15.94:0/2303540631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7efc102710 msgr2=0x7f7efc0fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:21.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:21.215+0000 7f7f009fc700 1 --2- 172.21.15.94:0/2303540631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7efc102710 0x7f7efc0fbd80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0004940 tx=0x7f7ef0009e30 comp rx=0 tx=0).stop 2024-03-24T10:57:21.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:21.215+0000 7f7f009fc700 1 -- 172.21.15.94:0/2303540631 shutdown_connections 2024-03-24T10:57:21.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:21.215+0000 7f7f009fc700 1 --2- 172.21.15.94:0/2303540631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7efc102710 0x7f7efc0fbd80 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:21.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:21.215+0000 7f7f009fc700 1 --2- 172.21.15.94:0/2303540631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7efc101d70 0x7f7efc102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:21.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:21.215+0000 7f7f009fc700 1 -- 172.21.15.94:0/2303540631 >> 172.21.15.94:0/2303540631 conn(0x7f7efc072bc0 msgr2=0x7f7efc072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:21.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:21.215+0000 7f7f009fc700 1 -- 172.21.15.94:0/2303540631 shutdown_connections 2024-03-24T10:57:21.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:21.215+0000 7f7f009fc700 1 -- 172.21.15.94:0/2303540631 wait complete. 2024-03-24T10:57:21.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:21.216+0000 7f7f009fc700 1 Processor -- start 2024-03-24T10:57:21.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:21.216+0000 7f7f009fc700 1 -- start start 2024-03-24T10:57:21.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:21.216+0000 7f7f009fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7efc101d70 0x7f7efc113b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:21.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:21.216+0000 7f7f009fc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7efc102710 0x7f7efc114070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:21.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:21.216+0000 7f7f009fc700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc114750 con 0x7f7efc102710 2024-03-24T10:57:21.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:21.216+0000 7f7f009fc700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc10daf0 con 0x7f7efc101d70 2024-03-24T10:57:21.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:21.216+0000 7f7ef9d9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7efc102710 0x7f7efc114070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:21.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:21.216+0000 7f7ef9d9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7efc102710 0x7f7efc114070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41932/0 (socket says 172.21.15.94:41932) 2024-03-24T10:57:21.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:21.216+0000 7f7ef9d9b700 1 -- 172.21.15.94:0/2453442542 learned_addr learned my addr 172.21.15.94:0/2453442542 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:57:21.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:21.216+0000 7f7efa59c700 1 --2- 172.21.15.94:0/2453442542 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7efc101d70 0x7f7efc113b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:21.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:21.216+0000 7f7ef9d9b700 1 -- 172.21.15.94:0/2453442542 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7efc101d70 msgr2=0x7f7efc113b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:21.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:21.216+0000 7f7ef9d9b700 1 --2- 172.21.15.94:0/2453442542 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7efc101d70 0x7f7efc113b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:21.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:21.216+0000 7f7ef9d9b700 1 -- 172.21.15.94:0/2453442542 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef0005040 con 0x7f7efc102710 2024-03-24T10:57:21.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:21.216+0000 7f7efa59c700 1 --2- 172.21.15.94:0/2453442542 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7efc101d70 0x7f7efc113b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T10:57:21.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:21.216+0000 7f7ef9d9b700 1 --2- 172.21.15.94:0/2453442542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7efc102710 0x7f7efc114070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0009100 tx=0x7f7ef0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:21.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:21.217+0000 7f7eeaffd700 1 -- 172.21.15.94:0/2453442542 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ef000e040 con 0x7f7efc102710 2024-03-24T10:57:21.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:21.217+0000 7f7eeaffd700 1 -- 172.21.15.94:0/2453442542 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7ef0013420 con 0x7f7efc102710 2024-03-24T10:57:21.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:21.217+0000 7f7f009fc700 1 -- 172.21.15.94:0/2453442542 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7efc10dd70 con 0x7f7efc102710 2024-03-24T10:57:21.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:21.217+0000 7f7eeaffd700 1 -- 172.21.15.94:0/2453442542 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ef001c630 con 0x7f7efc102710 2024-03-24T10:57:21.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:21.217+0000 7f7f009fc700 1 -- 172.21.15.94:0/2453442542 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7efc10e330 con 0x7f7efc102710 2024-03-24T10:57:21.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:21.218+0000 7f7f009fc700 1 -- 172.21.15.94:0/2453442542 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7efc0fd4c0 con 0x7f7efc102710 2024-03-24T10:57:21.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:21.218+0000 7f7eeaffd700 1 -- 172.21.15.94:0/2453442542 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f7ef0013590 con 0x7f7efc102710 2024-03-24T10:57:21.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:21.219+0000 7f7eeaffd700 1 --2- 172.21.15.94:0/2453442542 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f7ee406d7f0 0x7f7ee406fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:21.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:21.219+0000 7f7eeaffd700 1 -- 172.21.15.94:0/2453442542 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f7ef008e1a0 con 0x7f7efc102710 2024-03-24T10:57:21.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:21.219+0000 7f7efa59c700 1 --2- 172.21.15.94:0/2453442542 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f7ee406d7f0 0x7f7ee406fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:21.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:21.220+0000 7f7efa59c700 1 --2- 172.21.15.94:0/2453442542 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f7ee406d7f0 0x7f7ee406fca0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7eec006fd0 tx=0x7f7eec008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:21.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:21.224+0000 7f7eeaffd700 1 -- 172.21.15.94:0/2453442542 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7ef005b4a0 con 0x7f7efc102710 2024-03-24T10:57:21.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:21.436+0000 7f7f009fc700 1 -- 172.21.15.94:0/2453442542 --> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7efc108f30 con 0x7f7ee406d7f0 2024-03-24T10:57:21.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:21.449+0000 7f7eeaffd700 1 -- 172.21.15.94:0/2453442542 <== mgr.34104 v2:172.21.15.94:6800/109020447 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f7efc108f30 con 0x7f7ee406d7f0 2024-03-24T10:57:21.449 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T10:57:21.449 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.smithi094 smithi094 *:9093,9094 running (2m) 116s ago 19m 15.0M - 0.25.0 c8568f914cd2 aca214c66fba 2024-03-24T10:57:21.450 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi094 smithi094 running (7m) 116s ago 19m 6651k - 19.0.0-1855-g21b3bd67 553463a90fd5 2b0897df571d 2024-03-24T10:57:21.450 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi096 smithi096 running (7m) 2m ago 18m 6669k - 19.0.0-1855-g21b3bd67 553463a90fd5 1759a9ff2805 2024-03-24T10:57:21.450 INFO:teuthology.orchestra.run.smithi094.stdout:grafana.smithi094 smithi094 *:3000 running (2m) 116s ago 18m 54.3M - 9.4.12 a0ea5e3c46d9 d506da6aef36 2024-03-24T10:57:21.450 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.efiqte smithi094 running (4m) 116s ago 13m 85.9M - 19.0.0-1855-g21b3bd67 553463a90fd5 732c47b17971 2024-03-24T10:57:21.450 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.llefgu smithi094 running (3m) 116s ago 13m 87.2M - 19.0.0-1855-g21b3bd67 553463a90fd5 6e06f8a6dde0 2024-03-24T10:57:21.450 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi096.egbmoq smithi096 running (3m) 2m ago 13m 14.1M - 19.0.0-1855-g21b3bd67 553463a90fd5 f35fd7f23742 2024-03-24T10:57:21.450 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi096.jtebhh smithi096 running (3m) 2m ago 13m 13.9M - 19.0.0-1855-g21b3bd67 553463a90fd5 8b5ab9f96e5a 2024-03-24T10:57:21.450 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi094.yaethu smithi094 *:8443,9283,8765 running (9m) 116s ago 20m 511M - 19.0.0-1855-g21b3bd67 553463a90fd5 d3adab76b18f 2024-03-24T10:57:21.450 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi096.gfonnu smithi096 *:8443,9283,8765 running (9m) 2m ago 18m 450M - 19.0.0-1855-g21b3bd67 553463a90fd5 22937a059eb8 2024-03-24T10:57:21.450 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi094 smithi094 running (8m) 116s ago 21m 56.9M 2048M 19.0.0-1855-g21b3bd67 553463a90fd5 c779d611e29a 2024-03-24T10:57:21.450 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi096 smithi096 running (8m) 2m ago 17m 38.5M 2048M 19.0.0-1855-g21b3bd67 553463a90fd5 ed9bd0bd57f6 2024-03-24T10:57:21.450 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi094 smithi094 *:9100 running (2m) 116s ago 19m 7860k - 1.5.0 0da6a335fe13 003df2346f4f 2024-03-24T10:57:21.450 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi096 smithi096 *:9100 running (2m) 2m ago 18m 5481k - 1.5.0 0da6a335fe13 67d65fecf9df 2024-03-24T10:57:21.450 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (7m) 116s ago 17m 111M 1557M 19.0.0-1855-g21b3bd67 553463a90fd5 ec57ede0c9e4 2024-03-24T10:57:21.450 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi094 running (6m) 116s ago 16m 124M 1557M 19.0.0-1855-g21b3bd67 553463a90fd5 c5b8b2bbf8b5 2024-03-24T10:57:21.451 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi094 running (6m) 116s ago 16m 102M 1557M 19.0.0-1855-g21b3bd67 553463a90fd5 c961dcdc746f 2024-03-24T10:57:21.451 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi096 running (5m) 2m ago 15m 121M 2581M 19.0.0-1855-g21b3bd67 553463a90fd5 6323fdc6e6ca 2024-03-24T10:57:21.451 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi096 running (5m) 2m ago 15m 111M 2581M 19.0.0-1855-g21b3bd67 553463a90fd5 3e2536bcb263 2024-03-24T10:57:21.451 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi096 running (4m) 2m ago 15m 103M 2581M 19.0.0-1855-g21b3bd67 553463a90fd5 2f77f1fa20e2 2024-03-24T10:57:21.451 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.smithi094 smithi094 *:9095 running (2m) 116s ago 18m 44.8M - 2.43.0 a07b618ecd1d 326ddd336bc5 2024-03-24T10:57:21.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:21.451+0000 7f7f009fc700 1 -- 172.21.15.94:0/2453442542 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f7ee406d7f0 msgr2=0x7f7ee406fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:21.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:21.451+0000 7f7f009fc700 1 --2- 172.21.15.94:0/2453442542 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f7ee406d7f0 0x7f7ee406fca0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7eec006fd0 tx=0x7f7eec008040 comp rx=0 tx=0).stop 2024-03-24T10:57:21.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:21.451+0000 7f7f009fc700 1 -- 172.21.15.94:0/2453442542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7efc102710 msgr2=0x7f7efc114070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:21.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:21.451+0000 7f7f009fc700 1 --2- 172.21.15.94:0/2453442542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7efc102710 0x7f7efc114070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0009100 tx=0x7f7ef0009e30 comp rx=0 tx=0).stop 2024-03-24T10:57:21.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:21.451+0000 7f7f009fc700 1 -- 172.21.15.94:0/2453442542 shutdown_connections 2024-03-24T10:57:21.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:21.451+0000 7f7f009fc700 1 --2- 172.21.15.94:0/2453442542 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f7ee406d7f0 0x7f7ee406fca0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:21.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:21.451+0000 7f7f009fc700 1 --2- 172.21.15.94:0/2453442542 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7efc101d70 0x7f7efc113b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:21.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:21.451+0000 7f7f009fc700 1 --2- 172.21.15.94:0/2453442542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7efc102710 0x7f7efc114070 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:21.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:21.451+0000 7f7f009fc700 1 -- 172.21.15.94:0/2453442542 >> 172.21.15.94:0/2453442542 conn(0x7f7efc072bc0 msgr2=0x7f7efc065940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:21.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:21.451+0000 7f7f009fc700 1 -- 172.21.15.94:0/2453442542 shutdown_connections 2024-03-24T10:57:21.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:21.451+0000 7f7f009fc700 1 -- 172.21.15.94:0/2453442542 wait complete. 2024-03-24T10:57:21.749 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:21 smithi094 ceph-mon[93957]: pgmap v300: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:57:21.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:21 smithi096 ceph-mon[84258]: pgmap v300: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:57:22.101 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -e sha1=21b3bd672163092b4e542f772797fcba02d3d4de -- bash -c 'ceph versions' 2024-03-24T10:57:22.387 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:57:22.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:22 smithi094 ceph-mon[93957]: from='client.34542 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:57:22.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:22 smithi096 ceph-mon[84258]: from='client.34542 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T10:57:23.747 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:23 smithi094 ceph-mon[93957]: pgmap v301: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:57:23.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:23 smithi096 ceph-mon[84258]: pgmap v301: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:57:24.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:24.029+0000 7f70550fd700 1 -- 172.21.15.94:0/4206733655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7050102710 msgr2=0x7f70500fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:24.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:24.029+0000 7f70550fd700 1 --2- 172.21.15.94:0/4206733655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7050102710 0x7f70500fbd80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7040004940 tx=0x7f7040009e30 comp rx=0 tx=0).stop 2024-03-24T10:57:24.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:24.029+0000 7f70550fd700 1 -- 172.21.15.94:0/4206733655 shutdown_connections 2024-03-24T10:57:24.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:24.029+0000 7f70550fd700 1 --2- 172.21.15.94:0/4206733655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7050102710 0x7f70500fbd80 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:24.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:24.029+0000 7f70550fd700 1 --2- 172.21.15.94:0/4206733655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7050101d70 0x7f7050102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:24.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:24.029+0000 7f70550fd700 1 -- 172.21.15.94:0/4206733655 >> 172.21.15.94:0/4206733655 conn(0x7f7050072bc0 msgr2=0x7f7050072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:24.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:24.029+0000 7f70550fd700 1 -- 172.21.15.94:0/4206733655 shutdown_connections 2024-03-24T10:57:24.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:24.029+0000 7f70550fd700 1 -- 172.21.15.94:0/4206733655 wait complete. 2024-03-24T10:57:24.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:24.030+0000 7f70550fd700 1 Processor -- start 2024-03-24T10:57:24.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:24.030+0000 7f70550fd700 1 -- start start 2024-03-24T10:57:24.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:24.030+0000 7f70550fd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7050101d70 0x7f7050112aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:24.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:24.030+0000 7f70550fd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7050102710 0x7f7050112fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:24.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:24.030+0000 7f70550fd700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7050113630 con 0x7f7050101d70 2024-03-24T10:57:24.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:24.030+0000 7f70550fd700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705010ca20 con 0x7f7050102710 2024-03-24T10:57:24.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:24.030+0000 7f704ed9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7050101d70 0x7f7050112aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:24.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:24.030+0000 7f704ed9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7050101d70 0x7f7050112aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41950/0 (socket says 172.21.15.94:41950) 2024-03-24T10:57:24.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:24.030+0000 7f704ed9d700 1 -- 172.21.15.94:0/3238724866 learned_addr learned my addr 172.21.15.94:0/3238724866 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:57:24.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:24.030+0000 7f704e59c700 1 --2- 172.21.15.94:0/3238724866 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7050102710 0x7f7050112fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:24.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:24.031+0000 7f704ed9d700 1 -- 172.21.15.94:0/3238724866 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7050102710 msgr2=0x7f7050112fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:24.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:24.031+0000 7f704ed9d700 1 --2- 172.21.15.94:0/3238724866 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7050102710 0x7f7050112fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:24.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:24.031+0000 7f704ed9d700 1 -- 172.21.15.94:0/3238724866 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7040005040 con 0x7f7050101d70 2024-03-24T10:57:24.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:24.031+0000 7f704ed9d700 1 --2- 172.21.15.94:0/3238724866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7050101d70 0x7f7050112aa0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f704400c8c0 tx=0x7f70440044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:24.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:24.031+0000 7f70377fe700 1 -- 172.21.15.94:0/3238724866 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f704400a760 con 0x7f7050101d70 2024-03-24T10:57:24.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:24.031+0000 7f70377fe700 1 -- 172.21.15.94:0/3238724866 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f704400ad90 con 0x7f7050101d70 2024-03-24T10:57:24.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:24.031+0000 7f70377fe700 1 -- 172.21.15.94:0/3238724866 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70440113a0 con 0x7f7050101d70 2024-03-24T10:57:24.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:24.031+0000 7f70550fd700 1 -- 172.21.15.94:0/3238724866 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f705010cd00 con 0x7f7050101d70 2024-03-24T10:57:24.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:24.031+0000 7f70550fd700 1 -- 172.21.15.94:0/3238724866 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f705010d290 con 0x7f7050101d70 2024-03-24T10:57:24.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:24.032+0000 7f70550fd700 1 -- 172.21.15.94:0/3238724866 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7050104240 con 0x7f7050101d70 2024-03-24T10:57:24.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:24.033+0000 7f70377fe700 1 -- 172.21.15.94:0/3238724866 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f7044012400 con 0x7f7050101d70 2024-03-24T10:57:24.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:24.033+0000 7f70377fe700 1 --2- 172.21.15.94:0/3238724866 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f703806d7e0 0x7f703806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:24.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:24.033+0000 7f70377fe700 1 -- 172.21.15.94:0/3238724866 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f704408cf80 con 0x7f7050101d70 2024-03-24T10:57:24.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:24.033+0000 7f704e59c700 1 --2- 172.21.15.94:0/3238724866 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f703806d7e0 0x7f703806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:24.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:24.034+0000 7f704e59c700 1 --2- 172.21.15.94:0/3238724866 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f703806d7e0 0x7f703806fc90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f705010dd00 tx=0x7f7040009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:24.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:24.038+0000 7f70377fe700 1 -- 172.21.15.94:0/3238724866 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f70440c1280 con 0x7f7050101d70 2024-03-24T10:57:24.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:24.327+0000 7f70550fd700 1 -- 172.21.15.94:0/3238724866 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f705010e0f0 con 0x7f7050101d70 2024-03-24T10:57:24.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:24.327+0000 7f70377fe700 1 -- 172.21.15.94:0/3238724866 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f704405a280 con 0x7f7050101d70 2024-03-24T10:57:24.327 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-03-24T10:57:24.327 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": { 2024-03-24T10:57:24.327 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 2 2024-03-24T10:57:24.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:57:24.327 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": { 2024-03-24T10:57:24.328 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 2 2024-03-24T10:57:24.328 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:57:24.328 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": { 2024-03-24T10:57:24.328 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 6 2024-03-24T10:57:24.328 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:57:24.328 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-03-24T10:57:24.328 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 4 2024-03-24T10:57:24.328 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-03-24T10:57:24.328 INFO:teuthology.orchestra.run.smithi094.stdout: "overall": { 2024-03-24T10:57:24.328 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)": 14 2024-03-24T10:57:24.328 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-03-24T10:57:24.328 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-03-24T10:57:24.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:24.329+0000 7f70550fd700 1 -- 172.21.15.94:0/3238724866 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f703806d7e0 msgr2=0x7f703806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:24.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:24.329+0000 7f70550fd700 1 --2- 172.21.15.94:0/3238724866 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f703806d7e0 0x7f703806fc90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f705010dd00 tx=0x7f7040009e30 comp rx=0 tx=0).stop 2024-03-24T10:57:24.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:24.329+0000 7f70550fd700 1 -- 172.21.15.94:0/3238724866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7050101d70 msgr2=0x7f7050112aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:24.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:24.329+0000 7f70550fd700 1 --2- 172.21.15.94:0/3238724866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7050101d70 0x7f7050112aa0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f704400c8c0 tx=0x7f70440044c0 comp rx=0 tx=0).stop 2024-03-24T10:57:24.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:24.329+0000 7f70550fd700 1 -- 172.21.15.94:0/3238724866 shutdown_connections 2024-03-24T10:57:24.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:24.329+0000 7f70550fd700 1 --2- 172.21.15.94:0/3238724866 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f703806d7e0 0x7f703806fc90 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:24.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:24.329+0000 7f70550fd700 1 --2- 172.21.15.94:0/3238724866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7050101d70 0x7f7050112aa0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:24.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:24.329+0000 7f70550fd700 1 --2- 172.21.15.94:0/3238724866 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7050102710 0x7f7050112fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:24.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:24.329+0000 7f70550fd700 1 -- 172.21.15.94:0/3238724866 >> 172.21.15.94:0/3238724866 conn(0x7f7050072bc0 msgr2=0x7f70500fb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:24.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:24.329+0000 7f70550fd700 1 -- 172.21.15.94:0/3238724866 shutdown_connections 2024-03-24T10:57:24.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:24.329+0000 7f70550fd700 1 -- 172.21.15.94:0/3238724866 wait complete. 2024-03-24T10:57:24.670 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:24 smithi094 ceph-mon[93957]: pgmap v302: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:57:24.670 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:24 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/3238724866' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:57:24.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:24 smithi096 ceph-mon[84258]: pgmap v302: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:57:24.825 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:24 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/3238724866' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:57:24.933 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -e sha1=21b3bd672163092b4e542f772797fcba02d3d4de -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-03-24T10:57:25.195 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:57:26.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:26.873+0000 7f82fae45700 1 -- 172.21.15.94:0/552656606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82f40fe830 msgr2=0x7f82f4106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:26.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:26.873+0000 7f82fae45700 1 --2- 172.21.15.94:0/552656606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82f40fe830 0x7f82f4106f00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f82e0004940 tx=0x7f82e0009e30 comp rx=0 tx=0).stop 2024-03-24T10:57:26.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:26.873+0000 7f82fae45700 1 -- 172.21.15.94:0/552656606 shutdown_connections 2024-03-24T10:57:26.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:26.873+0000 7f82fae45700 1 --2- 172.21.15.94:0/552656606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82f40fe830 0x7f82f4106f00 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:26.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:26.873+0000 7f82fae45700 1 --2- 172.21.15.94:0/552656606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82f40fde90 0x7f82f40fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:26.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:26.873+0000 7f82fae45700 1 -- 172.21.15.94:0/552656606 >> 172.21.15.94:0/552656606 conn(0x7f82f4072bc0 msgr2=0x7f82f4072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:26.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:26.873+0000 7f82fae45700 1 -- 172.21.15.94:0/552656606 shutdown_connections 2024-03-24T10:57:26.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:26.873+0000 7f82fae45700 1 -- 172.21.15.94:0/552656606 wait complete. 2024-03-24T10:57:26.874 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:26 smithi094 ceph-mon[93957]: pgmap v303: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:57:26.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:26.874+0000 7f82fae45700 1 Processor -- start 2024-03-24T10:57:26.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:26.874+0000 7f82fae45700 1 -- start start 2024-03-24T10:57:26.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:26.874+0000 7f82fae45700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82f40fde90 0x7f82f4113c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:26.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:26.874+0000 7f82fae45700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82f40fe830 0x7f82f4114180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:26.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:26.874+0000 7f82fae45700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f41147d0 con 0x7f82f40fe830 2024-03-24T10:57:26.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:26.874+0000 7f82fae45700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f410dba0 con 0x7f82f40fde90 2024-03-24T10:57:26.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:26.875+0000 7f82f3fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82f40fe830 0x7f82f4114180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:26.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:26.875+0000 7f82f3fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82f40fe830 0x7f82f4114180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41978/0 (socket says 172.21.15.94:41978) 2024-03-24T10:57:26.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:26.875+0000 7f82f3fff700 1 -- 172.21.15.94:0/180900187 learned_addr learned my addr 172.21.15.94:0/180900187 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:57:26.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:26.875+0000 7f82f8be1700 1 --2- 172.21.15.94:0/180900187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82f40fde90 0x7f82f4113c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:26.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:26.875+0000 7f82f3fff700 1 -- 172.21.15.94:0/180900187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82f40fde90 msgr2=0x7f82f4113c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:26.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:26.875+0000 7f82f3fff700 1 --2- 172.21.15.94:0/180900187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82f40fde90 0x7f82f4113c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:26.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:26.875+0000 7f82f3fff700 1 -- 172.21.15.94:0/180900187 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e0005040 con 0x7f82f40fe830 2024-03-24T10:57:26.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:26.875+0000 7f82f3fff700 1 --2- 172.21.15.94:0/180900187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82f40fe830 0x7f82f4114180 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f82e000b3b0 tx=0x7f82e0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:26.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:26.875+0000 7f82f17fa700 1 -- 172.21.15.94:0/180900187 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82e000e040 con 0x7f82f40fe830 2024-03-24T10:57:26.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:26.875+0000 7f82fae45700 1 -- 172.21.15.94:0/180900187 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82f410de20 con 0x7f82f40fe830 2024-03-24T10:57:26.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:26.875+0000 7f82fae45700 1 -- 172.21.15.94:0/180900187 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82f410e360 con 0x7f82f40fe830 2024-03-24T10:57:26.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:26.875+0000 7f82f17fa700 1 -- 172.21.15.94:0/180900187 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f82e0004db0 con 0x7f82f40fe830 2024-03-24T10:57:26.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:26.875+0000 7f82f17fa700 1 -- 172.21.15.94:0/180900187 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82e0013db0 con 0x7f82f40fe830 2024-03-24T10:57:26.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:26.877+0000 7f82fae45700 1 -- 172.21.15.94:0/180900187 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82d80052b0 con 0x7f82f40fe830 2024-03-24T10:57:26.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:26.877+0000 7f82f17fa700 1 -- 172.21.15.94:0/180900187 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f82e0013420 con 0x7f82f40fe830 2024-03-24T10:57:26.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:26.877+0000 7f82f17fa700 1 --2- 172.21.15.94:0/180900187 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f82e406d7e0 0x7f82e406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:26.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:26.878+0000 7f82f17fa700 1 -- 172.21.15.94:0/180900187 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f82e008e110 con 0x7f82f40fe830 2024-03-24T10:57:26.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:26.878+0000 7f82f8be1700 1 --2- 172.21.15.94:0/180900187 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f82e406d7e0 0x7f82e406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:26.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:26.878+0000 7f82f8be1700 1 --2- 172.21.15.94:0/180900187 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f82e406d7e0 0x7f82e406fc90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f82e8004f60 tx=0x7f82e800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:26.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:26.882+0000 7f82f17fa700 1 -- 172.21.15.94:0/180900187 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f82e005b410 con 0x7f82f40fe830 2024-03-24T10:57:27.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:26 smithi096 ceph-mon[84258]: pgmap v303: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:57:27.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:27.320+0000 7f82fae45700 1 -- 172.21.15.94:0/180900187 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f82d8005830 con 0x7f82f40fe830 2024-03-24T10:57:27.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:27.321+0000 7f82f17fa700 1 -- 172.21.15.94:0/180900187 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f82e005afa0 con 0x7f82f40fe830 2024-03-24T10:57:27.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:27.323+0000 7f82fae45700 1 -- 172.21.15.94:0/180900187 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f82e406d7e0 msgr2=0x7f82e406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:27.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:27.323+0000 7f82fae45700 1 --2- 172.21.15.94:0/180900187 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f82e406d7e0 0x7f82e406fc90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f82e8004f60 tx=0x7f82e800b040 comp rx=0 tx=0).stop 2024-03-24T10:57:27.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:27.323+0000 7f82fae45700 1 -- 172.21.15.94:0/180900187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82f40fe830 msgr2=0x7f82f4114180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:27.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:27.323+0000 7f82fae45700 1 --2- 172.21.15.94:0/180900187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82f40fe830 0x7f82f4114180 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f82e000b3b0 tx=0x7f82e0009e30 comp rx=0 tx=0).stop 2024-03-24T10:57:27.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:27.323+0000 7f82fae45700 1 -- 172.21.15.94:0/180900187 shutdown_connections 2024-03-24T10:57:27.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:27.323+0000 7f82fae45700 1 --2- 172.21.15.94:0/180900187 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f82e406d7e0 0x7f82e406fc90 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:27.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:27.323+0000 7f82fae45700 1 --2- 172.21.15.94:0/180900187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82f40fde90 0x7f82f4113c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:27.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:27.323+0000 7f82fae45700 1 --2- 172.21.15.94:0/180900187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82f40fe830 0x7f82f4114180 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:27.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:27.323+0000 7f82fae45700 1 -- 172.21.15.94:0/180900187 >> 172.21.15.94:0/180900187 conn(0x7f82f4072bc0 msgr2=0x7f82f4103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:27.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:27.323+0000 7f82fae45700 1 -- 172.21.15.94:0/180900187 shutdown_connections 2024-03-24T10:57:27.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:27.323+0000 7f82fae45700 1 -- 172.21.15.94:0/180900187 wait complete. 2024-03-24T10:57:27.332 INFO:teuthology.orchestra.run.smithi094.stdout:true 2024-03-24T10:57:27.916 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:27 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/180900187' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:57:27.940 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -e sha1=21b3bd672163092b4e542f772797fcba02d3d4de -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-03-24T10:57:28.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:27 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/180900187' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:57:28.204 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:57:28.998 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:28 smithi094 ceph-mon[93957]: pgmap v304: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:57:29.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:28 smithi096 ceph-mon[84258]: pgmap v304: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:57:29.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:29.860+0000 7fa7b5b36700 1 -- 172.21.15.94:0/4117250487 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7b0102710 msgr2=0x7fa7b00fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:29.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:29.860+0000 7fa7b5b36700 1 --2- 172.21.15.94:0/4117250487 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7b0102710 0x7fa7b00fbd80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa798004940 tx=0x7fa798009e30 comp rx=0 tx=0).stop 2024-03-24T10:57:29.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:29.860+0000 7fa7b5b36700 1 -- 172.21.15.94:0/4117250487 shutdown_connections 2024-03-24T10:57:29.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:29.860+0000 7fa7b5b36700 1 --2- 172.21.15.94:0/4117250487 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7b0102710 0x7fa7b00fbd80 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:29.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:29.860+0000 7fa7b5b36700 1 --2- 172.21.15.94:0/4117250487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7b0101d70 0x7fa7b0102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:29.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:29.860+0000 7fa7b5b36700 1 -- 172.21.15.94:0/4117250487 >> 172.21.15.94:0/4117250487 conn(0x7fa7b0072bc0 msgr2=0x7fa7b0072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:29.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:29.860+0000 7fa7b5b36700 1 -- 172.21.15.94:0/4117250487 shutdown_connections 2024-03-24T10:57:29.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:29.860+0000 7fa7b5b36700 1 -- 172.21.15.94:0/4117250487 wait complete. 2024-03-24T10:57:29.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:29.861+0000 7fa7b5b36700 1 Processor -- start 2024-03-24T10:57:29.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:29.861+0000 7fa7b5b36700 1 -- start start 2024-03-24T10:57:29.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:29.861+0000 7fa7b5b36700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7b0101d70 0x7fa7b0113aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:29.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:29.861+0000 7fa7b5b36700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7b0102710 0x7fa7b0113fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:29.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:29.861+0000 7fa7b5b36700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b0114630 con 0x7fa7b0101d70 2024-03-24T10:57:29.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:29.861+0000 7fa7b5b36700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b010da60 con 0x7fa7b0102710 2024-03-24T10:57:29.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:29.861+0000 7fa7af7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7b0101d70 0x7fa7b0113aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:29.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:29.861+0000 7fa7af7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7b0101d70 0x7fa7b0113aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51498/0 (socket says 172.21.15.94:51498) 2024-03-24T10:57:29.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:29.861+0000 7fa7af7fe700 1 -- 172.21.15.94:0/1594398991 learned_addr learned my addr 172.21.15.94:0/1594398991 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:57:29.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:29.861+0000 7fa7aeffd700 1 --2- 172.21.15.94:0/1594398991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7b0102710 0x7fa7b0113fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:29.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:29.861+0000 7fa7af7fe700 1 -- 172.21.15.94:0/1594398991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7b0102710 msgr2=0x7fa7b0113fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:29.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:29.861+0000 7fa7af7fe700 1 --2- 172.21.15.94:0/1594398991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7b0102710 0x7fa7b0113fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:29.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:29.861+0000 7fa7af7fe700 1 -- 172.21.15.94:0/1594398991 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa798005040 con 0x7fa7b0101d70 2024-03-24T10:57:29.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:29.862+0000 7fa7af7fe700 1 --2- 172.21.15.94:0/1594398991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7b0101d70 0x7fa7b0113aa0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa7a000eb40 tx=0x7fa7a000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:29.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:29.862+0000 7fa7a7fff700 1 -- 172.21.15.94:0/1594398991 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7a0007cc0 con 0x7fa7b0101d70 2024-03-24T10:57:29.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:29.862+0000 7fa7a7fff700 1 -- 172.21.15.94:0/1594398991 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa7a000f040 con 0x7fa7b0101d70 2024-03-24T10:57:29.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:29.862+0000 7fa7a7fff700 1 -- 172.21.15.94:0/1594398991 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7a000ca10 con 0x7fa7b0101d70 2024-03-24T10:57:29.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:29.862+0000 7fa7b5b36700 1 -- 172.21.15.94:0/1594398991 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7b010dce0 con 0x7fa7b0101d70 2024-03-24T10:57:29.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:29.862+0000 7fa7b5b36700 1 -- 172.21.15.94:0/1594398991 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7b010e1f0 con 0x7fa7b0101d70 2024-03-24T10:57:29.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:29.863+0000 7fa7b5b36700 1 -- 172.21.15.94:0/1594398991 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7b00fd4c0 con 0x7fa7b0101d70 2024-03-24T10:57:29.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:29.863+0000 7fa7a7fff700 1 -- 172.21.15.94:0/1594398991 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fa7a000cb70 con 0x7fa7b0101d70 2024-03-24T10:57:29.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:29.864+0000 7fa7a7fff700 1 --2- 172.21.15.94:0/1594398991 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fa79c06d7e0 0x7fa79c06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:29.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:29.864+0000 7fa7aeffd700 1 --2- 172.21.15.94:0/1594398991 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fa79c06d7e0 0x7fa79c06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:29.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:29.864+0000 7fa7a7fff700 1 -- 172.21.15.94:0/1594398991 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fa7a008cab0 con 0x7fa7b0101d70 2024-03-24T10:57:29.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:29.864+0000 7fa7aeffd700 1 --2- 172.21.15.94:0/1594398991 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fa79c06d7e0 0x7fa79c06fc90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa798004890 tx=0x7fa798009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:29.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:29.869+0000 7fa7a7fff700 1 -- 172.21.15.94:0/1594398991 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa7a0059e60 con 0x7fa7b0101d70 2024-03-24T10:57:30.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:30.156+0000 7fa7b5b36700 1 -- 172.21.15.94:0/1594398991 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa7b004bed0 con 0x7fa7b0101d70 2024-03-24T10:57:30.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:30.157+0000 7fa7a7fff700 1 -- 172.21.15.94:0/1594398991 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fa7a00599f0 con 0x7fa7b0101d70 2024-03-24T10:57:30.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:30.159+0000 7fa7b5b36700 1 -- 172.21.15.94:0/1594398991 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fa79c06d7e0 msgr2=0x7fa79c06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:30.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:30.159+0000 7fa7b5b36700 1 --2- 172.21.15.94:0/1594398991 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fa79c06d7e0 0x7fa79c06fc90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa798004890 tx=0x7fa798009e30 comp rx=0 tx=0).stop 2024-03-24T10:57:30.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:30.159+0000 7fa7b5b36700 1 -- 172.21.15.94:0/1594398991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7b0101d70 msgr2=0x7fa7b0113aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:30.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:30.159+0000 7fa7b5b36700 1 --2- 172.21.15.94:0/1594398991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7b0101d70 0x7fa7b0113aa0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa7a000eb40 tx=0x7fa7a000b040 comp rx=0 tx=0).stop 2024-03-24T10:57:30.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:30.159+0000 7fa7b5b36700 1 -- 172.21.15.94:0/1594398991 shutdown_connections 2024-03-24T10:57:30.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:30.159+0000 7fa7b5b36700 1 --2- 172.21.15.94:0/1594398991 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fa79c06d7e0 0x7fa79c06fc90 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:30.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:30.159+0000 7fa7b5b36700 1 --2- 172.21.15.94:0/1594398991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7b0101d70 0x7fa7b0113aa0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:30.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:30.159+0000 7fa7b5b36700 1 --2- 172.21.15.94:0/1594398991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7b0102710 0x7fa7b0113fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:30.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:30.159+0000 7fa7b5b36700 1 -- 172.21.15.94:0/1594398991 >> 172.21.15.94:0/1594398991 conn(0x7fa7b0072bc0 msgr2=0x7fa7b0065940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:30.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:30.159+0000 7fa7b5b36700 1 -- 172.21.15.94:0/1594398991 shutdown_connections 2024-03-24T10:57:30.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:30.159+0000 7fa7b5b36700 1 -- 172.21.15.94:0/1594398991 wait complete. 2024-03-24T10:57:30.169 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)" 2024-03-24T10:57:30.766 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:30 smithi094 ceph-mon[93957]: pgmap v305: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:57:30.766 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:30 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/1594398991' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:57:30.790 DEBUG:teuthology.parallel:result is None 2024-03-24T10:57:30.791 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-24T10:57:30.807 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi094.front.sepia.ceph.com 2024-03-24T10:57:30.808 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-24T10:57:31.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:30 smithi096 ceph-mon[84258]: pgmap v305: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:57:31.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:30 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/1594398991' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T10:57:31.076 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/mon.smithi094/config 2024-03-24T10:57:31.976 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:31 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:57:31.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:31 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T10:57:32.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:32.710+0000 7f95f9bb3700 1 -- 172.21.15.94:0/3404956253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95f4101d70 msgr2=0x7f95f4102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:32.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:32.710+0000 7f95f9bb3700 1 --2- 172.21.15.94:0/3404956253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95f4101d70 0x7f95f4102140 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f95e4004940 tx=0x7f95e4009e30 comp rx=0 tx=0).stop 2024-03-24T10:57:32.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:32.710+0000 7f95f9bb3700 1 -- 172.21.15.94:0/3404956253 shutdown_connections 2024-03-24T10:57:32.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:32.710+0000 7f95f9bb3700 1 --2- 172.21.15.94:0/3404956253 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95f4102710 0x7f95f40fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:32.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:32.710+0000 7f95f9bb3700 1 --2- 172.21.15.94:0/3404956253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95f4101d70 0x7f95f4102140 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:32.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:32.710+0000 7f95f9bb3700 1 -- 172.21.15.94:0/3404956253 >> 172.21.15.94:0/3404956253 conn(0x7f95f4072bc0 msgr2=0x7f95f4072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:32.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:32.710+0000 7f95f9bb3700 1 -- 172.21.15.94:0/3404956253 shutdown_connections 2024-03-24T10:57:32.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:32.710+0000 7f95f9bb3700 1 -- 172.21.15.94:0/3404956253 wait complete. 2024-03-24T10:57:32.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:32.711+0000 7f95f9bb3700 1 Processor -- start 2024-03-24T10:57:32.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:32.711+0000 7f95f9bb3700 1 -- start start 2024-03-24T10:57:32.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:32.711+0000 7f95f9bb3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95f4101d70 0x7f95f4112a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:32.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:32.711+0000 7f95f9bb3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95f4102710 0x7f95f4112f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:32.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:32.711+0000 7f95f9bb3700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f41135b0 con 0x7f95f4102710 2024-03-24T10:57:32.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:32.711+0000 7f95f9bb3700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f410c9a0 con 0x7f95f4101d70 2024-03-24T10:57:32.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:32.711+0000 7f95f2ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95f4102710 0x7f95f4112f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:32.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:32.711+0000 7f95f2ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95f4102710 0x7f95f4112f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51522/0 (socket says 172.21.15.94:51522) 2024-03-24T10:57:32.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:32.711+0000 7f95f2ffd700 1 -- 172.21.15.94:0/684662505 learned_addr learned my addr 172.21.15.94:0/684662505 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:57:32.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:32.711+0000 7f95f37fe700 1 --2- 172.21.15.94:0/684662505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95f4101d70 0x7f95f4112a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:32.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:32.712+0000 7f95f2ffd700 1 -- 172.21.15.94:0/684662505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95f4101d70 msgr2=0x7f95f4112a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:32.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:32.712+0000 7f95f2ffd700 1 --2- 172.21.15.94:0/684662505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95f4101d70 0x7f95f4112a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:32.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:32.712+0000 7f95f2ffd700 1 -- 172.21.15.94:0/684662505 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e4005040 con 0x7f95f4102710 2024-03-24T10:57:32.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:32.712+0000 7f95f2ffd700 1 --2- 172.21.15.94:0/684662505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95f4102710 0x7f95f4112f60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f95e800eb40 tx=0x7f95e800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:32.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:32.712+0000 7f95dbfff700 1 -- 172.21.15.94:0/684662505 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95e8007cc0 con 0x7f95f4102710 2024-03-24T10:57:32.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:32.712+0000 7f95f9bb3700 1 -- 172.21.15.94:0/684662505 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95f410cc20 con 0x7f95f4102710 2024-03-24T10:57:32.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:32.712+0000 7f95dbfff700 1 -- 172.21.15.94:0/684662505 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f95e800f040 con 0x7f95f4102710 2024-03-24T10:57:32.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:32.712+0000 7f95dbfff700 1 -- 172.21.15.94:0/684662505 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95e800c9d0 con 0x7f95f4102710 2024-03-24T10:57:32.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:32.712+0000 7f95f9bb3700 1 -- 172.21.15.94:0/684662505 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95f410d160 con 0x7f95f4102710 2024-03-24T10:57:32.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:32.713+0000 7f95f9bb3700 1 -- 172.21.15.94:0/684662505 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95f40ff080 con 0x7f95f4102710 2024-03-24T10:57:32.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:32.714+0000 7f95dbfff700 1 -- 172.21.15.94:0/684662505 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f95e800cc40 con 0x7f95f4102710 2024-03-24T10:57:32.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:32.714+0000 7f95dbfff700 1 --2- 172.21.15.94:0/684662505 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f95dc06d7f0 0x7f95dc06fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:32.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:32.714+0000 7f95dbfff700 1 -- 172.21.15.94:0/684662505 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f95e808ca50 con 0x7f95f4102710 2024-03-24T10:57:32.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:32.714+0000 7f95f37fe700 1 --2- 172.21.15.94:0/684662505 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f95dc06d7f0 0x7f95dc06fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:32.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:32.715+0000 7f95f37fe700 1 --2- 172.21.15.94:0/684662505 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f95dc06d7f0 0x7f95dc06fca0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f95e4007f50 tx=0x7f95e4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:32.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:32.719+0000 7f95dbfff700 1 -- 172.21.15.94:0/684662505 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f95e8059ec0 con 0x7f95f4102710 2024-03-24T10:57:32.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:32.962+0000 7f95f9bb3700 1 -- 172.21.15.94:0/684662505 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f95f40fd4c0 con 0x7f95f4102710 2024-03-24T10:57:32.962 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:32 smithi094 ceph-mon[93957]: pgmap v306: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:57:32.962 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:32 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:57:32.962 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:32 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:57:32.962 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:32 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:57:32.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:32.962+0000 7f95dbfff700 1 -- 172.21.15.94:0/684662505 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 37 v37) v1 ==== 76+0+1963 (secure 0 0 0) 0x7f95e8059a50 con 0x7f95f4102710 2024-03-24T10:57:32.963 INFO:teuthology.orchestra.run.smithi094.stdout:e37 2024-03-24T10:57:32.963 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T10:57:32.963 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T10:57:32.963 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-03-24T10:57:32.963 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:57:32.963 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-03-24T10:57:32.963 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-03-24T10:57:32.963 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 37 2024-03-24T10:57:32.963 INFO:teuthology.orchestra.run.smithi094.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-24T10:57:32.963 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-03-24T10:43:39.435738+0000 2024-03-24T10:57:32.963 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-03-24T10:54:19.399866+0000 2024-03-24T10:57:32.963 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-03-24T10:57:32.963 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-03-24T10:57:32.963 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 60 2024-03-24T10:57:32.964 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-03-24T10:57:32.964 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-03-24T10:57:32.964 INFO:teuthology.orchestra.run.smithi094.stdout:max_xattr_size 65536 2024-03-24T10:57:32.964 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-03-24T10:57:32.964 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-03-24T10:57:32.964 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 84 2024-03-24T10:57:32.964 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-24T10:57:32.964 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-03-24T10:57:32.964 INFO:teuthology.orchestra.run.smithi094.stdout:in 0 2024-03-24T10:57:32.964 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=34374} 2024-03-24T10:57:32.964 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-03-24T10:57:32.964 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-03-24T10:57:32.964 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-03-24T10:57:32.964 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-03-24T10:57:32.964 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-03-24T10:57:32.965 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-03-24T10:57:32.965 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-03-24T10:57:32.965 INFO:teuthology.orchestra.run.smithi094.stdout:bal_rank_mask -1 2024-03-24T10:57:32.965 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 1 2024-03-24T10:57:32.965 INFO:teuthology.orchestra.run.smithi094.stdout:qdb_cluster leader: 34374 members: 34374 2024-03-24T10:57:32.965 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.efiqte{0:34374} state up:active seq 19 join_fscid=1 addr [v2:172.21.15.94:6826/2043632450,v1:172.21.15.94:6827/2043632450] compat {c=[1],r=[1],i=[1fff]}] 2024-03-24T10:57:32.965 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.llefgu{0:34406} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.94:6828/2396893131,v1:172.21.15.94:6829/2396893131] compat {c=[1],r=[1],i=[1fff]}] 2024-03-24T10:57:32.965 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:57:32.965 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:57:32.965 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-03-24T10:57:32.965 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:57:32.965 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi096.egbmoq{-1:44321} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.96:6824/2694461318,v1:172.21.15.96:6825/2694461318] compat {c=[1],r=[1],i=[1fff]}] 2024-03-24T10:57:32.965 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi096.jtebhh{-1:44349} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.96:6826/2275611388,v1:172.21.15.96:6827/2275611388] compat {c=[1],r=[1],i=[1fff]}] 2024-03-24T10:57:32.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:32.964+0000 7f95f9bb3700 1 -- 172.21.15.94:0/684662505 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f95dc06d7f0 msgr2=0x7f95dc06fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:32.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:32.964+0000 7f95f9bb3700 1 --2- 172.21.15.94:0/684662505 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f95dc06d7f0 0x7f95dc06fca0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f95e4007f50 tx=0x7f95e4009e30 comp rx=0 tx=0).stop 2024-03-24T10:57:32.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:32.964+0000 7f95f9bb3700 1 -- 172.21.15.94:0/684662505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95f4102710 msgr2=0x7f95f4112f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:32.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:32.964+0000 7f95f9bb3700 1 --2- 172.21.15.94:0/684662505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95f4102710 0x7f95f4112f60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f95e800eb40 tx=0x7f95e800b040 comp rx=0 tx=0).stop 2024-03-24T10:57:32.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:32.965+0000 7f95f9bb3700 1 -- 172.21.15.94:0/684662505 shutdown_connections 2024-03-24T10:57:32.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:32.965+0000 7f95f9bb3700 1 --2- 172.21.15.94:0/684662505 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f95dc06d7f0 0x7f95dc06fca0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:32.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:32.965+0000 7f95f9bb3700 1 --2- 172.21.15.94:0/684662505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95f4101d70 0x7f95f4112a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:32.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:32.965+0000 7f95f9bb3700 1 --2- 172.21.15.94:0/684662505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95f4102710 0x7f95f4112f60 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:32.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:32.965+0000 7f95f9bb3700 1 -- 172.21.15.94:0/684662505 >> 172.21.15.94:0/684662505 conn(0x7f95f4072bc0 msgr2=0x7f95f40faec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:32.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:32.965+0000 7f95f9bb3700 1 -- 172.21.15.94:0/684662505 shutdown_connections 2024-03-24T10:57:32.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:32.965+0000 7f95f9bb3700 1 -- 172.21.15.94:0/684662505 wait complete. 2024-03-24T10:57:32.967 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 37 2024-03-24T10:57:33.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:32 smithi096 ceph-mon[84258]: pgmap v306: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:57:33.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:32 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T10:57:33.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:32 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T10:57:33.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:32 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' 2024-03-24T10:57:33.581 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-03-24T10:57:33.591 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-24T10:57:33.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:33.692+0000 7f726766e640 1 -- 172.21.15.94:0/1899871557 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7260102240 msgr2=0x7f7260102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:33.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:33.692+0000 7f726766e640 1 --2- 172.21.15.94:0/1899871557 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7260102240 0x7f7260102610 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f725c0048d0 tx=0x7f725c009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:33.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:33.692+0000 7f726766e640 1 -- 172.21.15.94:0/1899871557 shutdown_connections 2024-03-24T10:57:33.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:33.692+0000 7f726766e640 1 --2- 172.21.15.94:0/1899871557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72600643a0 0x7f72600647d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:33.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:33.692+0000 7f726766e640 1 --2- 172.21.15.94:0/1899871557 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7260102240 0x7f7260102610 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:33.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:33.692+0000 7f726766e640 1 -- 172.21.15.94:0/1899871557 >> 172.21.15.94:0/1899871557 conn(0x7f7260071130 msgr2=0x7f7260071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:33.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:33.692+0000 7f726766e640 1 -- 172.21.15.94:0/1899871557 shutdown_connections 2024-03-24T10:57:33.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:33.692+0000 7f726766e640 1 -- 172.21.15.94:0/1899871557 wait complete. 2024-03-24T10:57:33.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:33.693+0000 7f726766e640 1 Processor -- start 2024-03-24T10:57:33.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:33.693+0000 7f726766e640 1 -- start start 2024-03-24T10:57:33.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:33.693+0000 7f726766e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72600643a0 0x7f7260112c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:33.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:33.693+0000 7f726766e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7260102240 0x7f72601131d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:33.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:33.693+0000 7f726766e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72601137d0 con 0x7f72600643a0 2024-03-24T10:57:33.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:33.693+0000 7f726766e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726010ce20 con 0x7f7260102240 2024-03-24T10:57:33.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:33.693+0000 7f726540a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72600643a0 0x7f7260112c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:33.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:33.693+0000 7f726540a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72600643a0 0x7f7260112c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51532/0 (socket says 172.21.15.94:51532) 2024-03-24T10:57:33.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:33.693+0000 7f726540a640 1 -- 172.21.15.94:0/2530031714 learned_addr learned my addr 172.21.15.94:0/2530031714 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:57:33.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:33.693+0000 7f7264c09640 1 --2- 172.21.15.94:0/2530031714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7260102240 0x7f72601131d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:33.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:33.694+0000 7f726540a640 1 -- 172.21.15.94:0/2530031714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7260102240 msgr2=0x7f72601131d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:33.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:33.694+0000 7f726540a640 1 --2- 172.21.15.94:0/2530031714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7260102240 0x7f72601131d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:33.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:33.694+0000 7f726540a640 1 -- 172.21.15.94:0/2530031714 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f725c005040 con 0x7f72600643a0 2024-03-24T10:57:33.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:33.694+0000 7f7264c09640 1 --2- 172.21.15.94:0/2530031714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7260102240 0x7f72601131d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T10:57:33.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:33.694+0000 7f726540a640 1 --2- 172.21.15.94:0/2530031714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72600643a0 0x7f7260112c90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f725c004930 tx=0x7f725c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:33.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:33.694+0000 7f7255ffb640 1 -- 172.21.15.94:0/2530031714 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f725c0094c0 con 0x7f72600643a0 2024-03-24T10:57:33.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:33.694+0000 7f7255ffb640 1 -- 172.21.15.94:0/2530031714 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f725c032e50 con 0x7f72600643a0 2024-03-24T10:57:33.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:33.694+0000 7f726766e640 1 -- 172.21.15.94:0/2530031714 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f726010d0a0 con 0x7f72600643a0 2024-03-24T10:57:33.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:33.694+0000 7f7255ffb640 1 -- 172.21.15.94:0/2530031714 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f725c0433b0 con 0x7f72600643a0 2024-03-24T10:57:33.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:33.694+0000 7f726766e640 1 -- 172.21.15.94:0/2530031714 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f726010d560 con 0x7f72600643a0 2024-03-24T10:57:33.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:33.695+0000 7f726766e640 1 -- 172.21.15.94:0/2530031714 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72600fd940 con 0x7f72600643a0 2024-03-24T10:57:33.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:33.696+0000 7f7255ffb640 1 -- 172.21.15.94:0/2530031714 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f725c02f070 con 0x7f72600643a0 2024-03-24T10:57:33.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:33.696+0000 7f7255ffb640 1 --2- 172.21.15.94:0/2530031714 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f724c06d750 0x7f724c06fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:33.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:33.696+0000 7f7255ffb640 1 -- 172.21.15.94:0/2530031714 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f725c0af000 con 0x7f72600643a0 2024-03-24T10:57:33.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:33.696+0000 7f7264c09640 1 --2- 172.21.15.94:0/2530031714 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f724c06d750 0x7f724c06fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:33.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:33.697+0000 7f7264c09640 1 --2- 172.21.15.94:0/2530031714 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f724c06d750 0x7f724c06fc00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f726010e550 tx=0x7f7250009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:33.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:33.701+0000 7f7255ffb640 1 -- 172.21.15.94:0/2530031714 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f725c078750 con 0x7f72600643a0 2024-03-24T10:57:33.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:33.883+0000 7f726766e640 1 -- 172.21.15.94:0/2530031714 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f726004bcc0 con 0x7f72600643a0 2024-03-24T10:57:33.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:33.885+0000 7f7255ffb640 1 -- 172.21.15.94:0/2530031714 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16865 (secure 0 0 0) 0x7f725c07be10 con 0x7f72600643a0 2024-03-24T10:57:33.885 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:57:33.885 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":84,"fsid":"386110d4-e9ca-11ee-95cd-87774f69a715","created":"2024-03-24T10:36:12.011908+0000","modified":"2024-03-24T10:54:01.169537+0000","last_up_change":"2024-03-24T10:52:57.465293+0000","last_in_change":"2024-03-24T10:42:04.898193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-24T10:41:09.161003+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-24T10:43:37.651734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-24T10:43:38.660001+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58fd53f9-8ad5-4296-8804-96a98647bba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":78,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1160876432},{"type":"v1","addr":"172.21.15.94:6803","nonce":1160876432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1160876432},{"type":"v1","addr":"172.21.15.94:6805","nonce":1160876432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1160876432},{"type":"v1","addr":"172.21.15.94:6809","nonce":1160876432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1160876432},{"type":"v1","addr":"172.21.15.94:6807","nonce":1160876432}]},"public_addr":"172.21.15.94:6803/1160876432","cluster_addr":"172.21.15.94:6805/1160876432","heartbeat_back_addr":"172.21.15.94:6809/1160876432","heartbeat_front_addr":"172.21.15.94:6807/1160876432","state":["exists","up"]},{"osd":1,"uuid":"3087b482-aede-4e88-848c-3e7438eaccef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":58,"up_thru":78,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3303603595},{"type":"v1","addr":"172.21.15.94:6811","nonce":3303603595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3303603595},{"type":"v1","addr":"172.21.15.94:6813","nonce":3303603595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3303603595},{"type":"v1","addr":"172.21.15.94:6817","nonce":3303603595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3303603595},{"type":"v1","addr":"172.21.15.94:6815","nonce":3303603595}]},"public_addr":"172.21.15.94:6811/3303603595","cluster_addr":"172.21.15.94:6813/3303603595","heartbeat_back_addr":"172.21.15.94:6817/3303603595","heartbeat_front_addr":"172.21.15.94:6815/3303603595","state":["exists","up"]},{"osd":2,"uuid":"48c00627-22a6-480c-9aef-38bc326fc4dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":59,"up_from":63,"up_thru":78,"down_at":60,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3153715097},{"type":"v1","addr":"172.21.15.94:6819","nonce":3153715097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3153715097},{"type":"v1","addr":"172.21.15.94:6821","nonce":3153715097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3153715097},{"type":"v1","addr":"172.21.15.94:6825","nonce":3153715097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3153715097},{"type":"v1","addr":"172.21.15.94:6823","nonce":3153715097}]},"public_addr":"172.21.15.94:6819/3153715097","cluster_addr":"172.21.15.94:6821/3153715097","heartbeat_back_addr":"172.21.15.94:6825/3153715097","heartbeat_front_addr":"172.21.15.94:6823/3153715097","state":["exists","up"]},{"osd":3,"uuid":"910859cd-d028-45f8-8e28-af4e46be460b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":64,"up_from":68,"up_thru":78,"down_at":65,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2693214290},{"type":"v1","addr":"172.21.15.96:6801","nonce":2693214290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2693214290},{"type":"v1","addr":"172.21.15.96:6803","nonce":2693214290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2693214290},{"type":"v1","addr":"172.21.15.96:6807","nonce":2693214290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2693214290},{"type":"v1","addr":"172.21.15.96:6805","nonce":2693214290}]},"public_addr":"172.21.15.96:6801/2693214290","cluster_addr":"172.21.15.96:6803/2693214290","heartbeat_back_addr":"172.21.15.96:6807/2693214290","heartbeat_front_addr":"172.21.15.96:6805/2693214290","state":["exists","up"]},{"osd":4,"uuid":"9d37e21d-38dc-4e5b-b5ef-3828d7b25096","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":69,"up_from":73,"up_thru":78,"down_at":70,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3911504706},{"type":"v1","addr":"172.21.15.96:6809","nonce":3911504706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3911504706},{"type":"v1","addr":"172.21.15.96:6811","nonce":3911504706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3911504706},{"type":"v1","addr":"172.21.15.96:6815","nonce":3911504706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3911504706},{"type":"v1","addr":"172.21.15.96:6813","nonce":3911504706}]},"public_addr":"172.21.15.96:6809/3911504706","cluster_addr":"172.21.15.96:6811/3911504706","heartbeat_back_addr":"172.21.15.96:6815/3911504706","heartbeat_front_addr":"172.21.15.96:6813/3911504706","state":["exists","up"]},{"osd":5,"uuid":"838cf934-b936-4257-8510-3d5bbf4ef034","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":74,"up_from":78,"up_thru":78,"down_at":75,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":853588375},{"type":"v1","addr":"172.21.15.96:6817","nonce":853588375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":853588375},{"type":"v1","addr":"172.21.15.96:6819","nonce":853588375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":853588375},{"type":"v1","addr":"172.21.15.96:6823","nonce":853588375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":853588375},{"type":"v1","addr":"172.21.15.96:6821","nonce":853588375}]},"public_addr":"172.21.15.96:6817/853588375","cluster_addr":"172.21.15.96:6819/853588375","heartbeat_back_addr":"172.21.15.96:6823/853588375","heartbeat_front_addr":"172.21.15.96:6821/853588375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T10:40:12.038732+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T10:40:38.407784+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T10:41:06.693615+0000","dead_epoch":59},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T10:41:30.382956+0000","dead_epoch":64},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T10:41:56.733747+0000","dead_epoch":69},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T10:42:22.324376+0000","dead_epoch":74}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:6825/1539752208":"2024-03-25T10:53:38.029430+0000","172.21.15.94:0/403370730":"2024-03-25T10:48:35.235438+0000","172.21.15.94:0/3351913962":"2024-03-25T10:48:35.235438+0000","172.21.15.94:0/659933905":"2024-03-25T10:48:35.235438+0000","172.21.15.96:6827/1321087930":"2024-03-25T10:54:01.169508+0000","172.21.15.96:6828/1632816545":"2024-03-25T10:47:50.745695+0000","172.21.15.96:6829/1632816545":"2024-03-25T10:47:50.745695+0000","172.21.15.96:6826/1321087930":"2024-03-25T10:54:01.169508+0000","172.21.15.94:0/1027963628":"2024-03-25T10:46:29.516417+0000","172.21.15.94:6801/1013449112":"2024-03-25T10:36:48.432382+0000","172.21.15.94:0/2700857461":"2024-03-25T10:39:00.830398+0000","172.21.15.94:6800/1996608546":"2024-03-25T10:37:18.183215+0000","172.21.15.94:6826/2001471799":"2024-03-25T10:43:55.084207+0000","172.21.15.94:6801/1996608546":"2024-03-25T10:37:18.183215+0000","172.21.15.94:0/2837872991":"2024-03-25T10:36:48.432382+0000","172.21.15.94:0/2130298637":"2024-03-25T10:39:00.830398+0000","172.21.15.96:0/109124621":"2024-03-25T10:47:50.745695+0000","172.21.15.96:0/3508170171":"2024-03-25T10:47:50.745695+0000","172.21.15.94:6800/2406899470":"2024-03-25T10:39:00.830398+0000","172.21.15.94:0/206764114":"2024-03-25T10:46:29.516417+0000","172.21.15.94:6800/1013449112":"2024-03-25T10:36:48.432382+0000","172.21.15.94:6801/2184357820":"2024-03-25T10:46:29.516417+0000","172.21.15.96:0/2983574460":"2024-03-25T10:47:50.745695+0000","172.21.15.94:6800/4143126939":"2024-03-25T10:48:35.235438+0000","172.21.15.96:0/3848335112":"2024-03-25T10:47:50.745695+0000","172.21.15.94:6801/2406899470":"2024-03-25T10:39:00.830398+0000","172.21.15.94:0/629634217":"2024-03-25T10:37:18.183215+0000","172.21.15.94:0/3628417758":"2024-03-25T10:37:18.183215+0000","172.21.15.96:0/3826472619":"2024-03-25T10:47:50.745695+0000","172.21.15.94:0/2173796904":"2024-03-25T10:48:35.235438+0000","172.21.15.94:0/4215128639":"2024-03-25T10:48:35.235438+0000","172.21.15.94:0/2155575468":"2024-03-25T10:39:00.830398+0000","172.21.15.94:0/265930160":"2024-03-25T10:46:29.516417+0000","172.21.15.94:0/978791860":"2024-03-25T10:46:29.516417+0000","172.21.15.94:0/2641430900":"2024-03-25T10:36:48.432382+0000","172.21.15.94:6827/2001471799":"2024-03-25T10:43:55.084207+0000","172.21.15.94:6800/2184357820":"2024-03-25T10:46:29.516417+0000","172.21.15.94:0/4239612108":"2024-03-25T10:36:48.432382+0000","172.21.15.94:0/2661420513":"2024-03-25T10:37:18.183215+0000","172.21.15.94:0/4063210620":"2024-03-25T10:48:35.235438+0000","172.21.15.96:0/3832405488":"2024-03-25T10:47:50.745695+0000","172.21.15.96:6824/1539752208":"2024-03-25T10:53:38.029430+0000","172.21.15.94:6801/4143126939":"2024-03-25T10:48:35.235438+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-24T10:57:33.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:33.887+0000 7f726766e640 1 -- 172.21.15.94:0/2530031714 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f724c06d750 msgr2=0x7f724c06fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:33.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:33.887+0000 7f726766e640 1 --2- 172.21.15.94:0/2530031714 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f724c06d750 0x7f724c06fc00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f726010e550 tx=0x7f7250009040 comp rx=0 tx=0).stop 2024-03-24T10:57:33.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:33.887+0000 7f726766e640 1 -- 172.21.15.94:0/2530031714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72600643a0 msgr2=0x7f7260112c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:33.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:33.887+0000 7f726766e640 1 --2- 172.21.15.94:0/2530031714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72600643a0 0x7f7260112c90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f725c004930 tx=0x7f725c009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:33.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:33.887+0000 7f726766e640 1 -- 172.21.15.94:0/2530031714 shutdown_connections 2024-03-24T10:57:33.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:33.887+0000 7f726766e640 1 --2- 172.21.15.94:0/2530031714 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f724c06d750 0x7f724c06fc00 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:33.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:33.887+0000 7f726766e640 1 --2- 172.21.15.94:0/2530031714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7260102240 0x7f72601131d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:33.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:33.887+0000 7f726766e640 1 --2- 172.21.15.94:0/2530031714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72600643a0 0x7f7260112c90 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:33.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:33.887+0000 7f726766e640 1 -- 172.21.15.94:0/2530031714 >> 172.21.15.94:0/2530031714 conn(0x7f7260071130 msgr2=0x7f72600f9510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:33.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:33.887+0000 7f726766e640 1 -- 172.21.15.94:0/2530031714 shutdown_connections 2024-03-24T10:57:33.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:33.887+0000 7f726766e640 1 -- 172.21.15.94:0/2530031714 wait complete. 2024-03-24T10:57:33.896 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:33 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/684662505' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T10:57:33.898 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-24T10:41:09.161003+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-24T10:43:37.651734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-24T10:43:38.660001+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-24T10:57:33.898 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-24T10:57:34.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.000+0000 7f4b2e8ae640 1 -- 172.21.15.94:0/31024252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b280643a0 msgr2=0x7f4b280647d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:34.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.000+0000 7f4b2e8ae640 1 --2- 172.21.15.94:0/31024252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b280643a0 0x7f4b280647d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c0048d0 tx=0x7f4b1c009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:34.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.000+0000 7f4b2e8ae640 1 -- 172.21.15.94:0/31024252 shutdown_connections 2024-03-24T10:57:34.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.000+0000 7f4b2e8ae640 1 --2- 172.21.15.94:0/31024252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b280643a0 0x7f4b280647d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:34.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.000+0000 7f4b2e8ae640 1 --2- 172.21.15.94:0/31024252 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b28102240 0x7f4b28102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:34.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.000+0000 7f4b2e8ae640 1 -- 172.21.15.94:0/31024252 >> 172.21.15.94:0/31024252 conn(0x7f4b280710b0 msgr2=0x7f4b280714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:34.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.000+0000 7f4b2e8ae640 1 -- 172.21.15.94:0/31024252 shutdown_connections 2024-03-24T10:57:34.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.000+0000 7f4b2e8ae640 1 -- 172.21.15.94:0/31024252 wait complete. 2024-03-24T10:57:34.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.001+0000 7f4b2e8ae640 1 Processor -- start 2024-03-24T10:57:34.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.001+0000 7f4b2e8ae640 1 -- start start 2024-03-24T10:57:34.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.001+0000 7f4b2e8ae640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b280643a0 0x7f4b280fb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:34.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.001+0000 7f4b2e8ae640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b28102240 0x7f4b280fb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:34.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.001+0000 7f4b2e8ae640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b280fd0c0 con 0x7f4b28102240 2024-03-24T10:57:34.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.001+0000 7f4b2e8ae640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b280fbc60 con 0x7f4b280643a0 2024-03-24T10:57:34.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.001+0000 7f4b277fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b28102240 0x7f4b280fb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:34.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.001+0000 7f4b277fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b28102240 0x7f4b280fb720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51552/0 (socket says 172.21.15.94:51552) 2024-03-24T10:57:34.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.001+0000 7f4b277fe640 1 -- 172.21.15.94:0/2422086695 learned_addr learned my addr 172.21.15.94:0/2422086695 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:57:34.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.001+0000 7f4b27fff640 1 --2- 172.21.15.94:0/2422086695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b280643a0 0x7f4b280fb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:34.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.002+0000 7f4b277fe640 1 -- 172.21.15.94:0/2422086695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b280643a0 msgr2=0x7f4b280fb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:34.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.002+0000 7f4b277fe640 1 --2- 172.21.15.94:0/2422086695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b280643a0 0x7f4b280fb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:34.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.002+0000 7f4b277fe640 1 -- 172.21.15.94:0/2422086695 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b1c005040 con 0x7f4b28102240 2024-03-24T10:57:34.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.002+0000 7f4b27fff640 1 --2- 172.21.15.94:0/2422086695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b280643a0 0x7f4b280fb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T10:57:34.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.002+0000 7f4b277fe640 1 --2- 172.21.15.94:0/2422086695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b28102240 0x7f4b280fb720 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c034040 tx=0x7f4b1c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:34.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.002+0000 7f4b24ff9640 1 -- 172.21.15.94:0/2422086695 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b1c0092b0 con 0x7f4b28102240 2024-03-24T10:57:34.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.002+0000 7f4b24ff9640 1 -- 172.21.15.94:0/2422086695 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b1c03b400 con 0x7f4b28102240 2024-03-24T10:57:34.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.002+0000 7f4b2e8ae640 1 -- 172.21.15.94:0/2422086695 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b280fbee0 con 0x7f4b28102240 2024-03-24T10:57:34.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.002+0000 7f4b24ff9640 1 -- 172.21.15.94:0/2422086695 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b1c0435d0 con 0x7f4b28102240 2024-03-24T10:57:34.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.002+0000 7f4b2e8ae640 1 -- 172.21.15.94:0/2422086695 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b281193a0 con 0x7f4b28102240 2024-03-24T10:57:34.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.003+0000 7f4b2e8ae640 1 -- 172.21.15.94:0/2422086695 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b280fd940 con 0x7f4b28102240 2024-03-24T10:57:34.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.003+0000 7f4b24ff9640 1 -- 172.21.15.94:0/2422086695 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f4b1c02f070 con 0x7f4b28102240 2024-03-24T10:57:34.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.004+0000 7f4b24ff9640 1 --2- 172.21.15.94:0/2422086695 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f4b1006d750 0x7f4b1006fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:34.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.004+0000 7f4b24ff9640 1 -- 172.21.15.94:0/2422086695 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f4b1c0aed50 con 0x7f4b28102240 2024-03-24T10:57:34.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.004+0000 7f4b27fff640 1 --2- 172.21.15.94:0/2422086695 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f4b1006d750 0x7f4b1006fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:34.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.004+0000 7f4b27fff640 1 --2- 172.21.15.94:0/2422086695 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f4b1006d750 0x7f4b1006fc00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4b180049d0 tx=0x7f4b1800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:34.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.012+0000 7f4b24ff9640 1 -- 172.21.15.94:0/2422086695 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4b1c0784a0 con 0x7f4b28102240 2024-03-24T10:57:34.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:33 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/684662505' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-24T10:57:34.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.205+0000 7f4b2e8ae640 1 -- 172.21.15.94:0/2422086695 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4b2804bcc0 con 0x7f4b28102240 2024-03-24T10:57:34.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.205+0000 7f4b24ff9640 1 -- 172.21.15.94:0/2422086695 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4b2804bcc0 con 0x7f4b28102240 2024-03-24T10:57:34.205 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-03-24T10:57:34.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.208+0000 7f4b2e8ae640 1 -- 172.21.15.94:0/2422086695 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f4b1006d750 msgr2=0x7f4b1006fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:34.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.208+0000 7f4b2e8ae640 1 --2- 172.21.15.94:0/2422086695 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f4b1006d750 0x7f4b1006fc00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4b180049d0 tx=0x7f4b1800b040 comp rx=0 tx=0).stop 2024-03-24T10:57:34.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.208+0000 7f4b2e8ae640 1 -- 172.21.15.94:0/2422086695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b28102240 msgr2=0x7f4b280fb720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:34.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.208+0000 7f4b2e8ae640 1 --2- 172.21.15.94:0/2422086695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b28102240 0x7f4b280fb720 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c034040 tx=0x7f4b1c009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:34.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.208+0000 7f4b2e8ae640 1 -- 172.21.15.94:0/2422086695 shutdown_connections 2024-03-24T10:57:34.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.208+0000 7f4b2e8ae640 1 --2- 172.21.15.94:0/2422086695 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f4b1006d750 0x7f4b1006fc00 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:34.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.208+0000 7f4b2e8ae640 1 --2- 172.21.15.94:0/2422086695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b28102240 0x7f4b280fb720 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:34.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.208+0000 7f4b2e8ae640 1 --2- 172.21.15.94:0/2422086695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b280643a0 0x7f4b280fb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:34.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.208+0000 7f4b2e8ae640 1 -- 172.21.15.94:0/2422086695 >> 172.21.15.94:0/2422086695 conn(0x7f4b280710b0 msgr2=0x7f4b280f94a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:34.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.208+0000 7f4b2e8ae640 1 -- 172.21.15.94:0/2422086695 shutdown_connections 2024-03-24T10:57:34.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.208+0000 7f4b2e8ae640 1 -- 172.21.15.94:0/2422086695 wait complete. 2024-03-24T10:57:34.219 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-24T10:57:34.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.325+0000 7f90e9c6e640 1 -- 172.21.15.94:0/4184804705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90e40fb110 msgr2=0x7f90e40fb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:34.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.325+0000 7f90e9c6e640 1 --2- 172.21.15.94:0/4184804705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90e40fb110 0x7f90e40fb560 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f90d80048d0 tx=0x7f90d8009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:34.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.325+0000 7f90e9c6e640 1 -- 172.21.15.94:0/4184804705 shutdown_connections 2024-03-24T10:57:34.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.325+0000 7f90e9c6e640 1 --2- 172.21.15.94:0/4184804705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90e40fb110 0x7f90e40fb560 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:34.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.325+0000 7f90e9c6e640 1 --2- 172.21.15.94:0/4184804705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90e4101140 0x7f90e4101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:34.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.325+0000 7f90e9c6e640 1 -- 172.21.15.94:0/4184804705 >> 172.21.15.94:0/4184804705 conn(0x7f90e40710b0 msgr2=0x7f90e40714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:34.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.325+0000 7f90e9c6e640 1 -- 172.21.15.94:0/4184804705 shutdown_connections 2024-03-24T10:57:34.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.325+0000 7f90e9c6e640 1 -- 172.21.15.94:0/4184804705 wait complete. 2024-03-24T10:57:34.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.325+0000 7f90e9c6e640 1 Processor -- start 2024-03-24T10:57:34.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.325+0000 7f90e9c6e640 1 -- start start 2024-03-24T10:57:34.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.326+0000 7f90e9c6e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90e40fb110 0x7f90e4111be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:34.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.326+0000 7f90e9c6e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90e4101140 0x7f90e4112120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:34.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.326+0000 7f90e9c6e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e4112720 con 0x7f90e40fb110 2024-03-24T10:57:34.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.326+0000 7f90e9c6e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e410bd70 con 0x7f90e4101140 2024-03-24T10:57:34.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.326+0000 7f90e37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90e40fb110 0x7f90e4111be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:34.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.326+0000 7f90e37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90e40fb110 0x7f90e4111be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51574/0 (socket says 172.21.15.94:51574) 2024-03-24T10:57:34.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.326+0000 7f90e37fe640 1 -- 172.21.15.94:0/2910121282 learned_addr learned my addr 172.21.15.94:0/2910121282 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:57:34.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.326+0000 7f90e2ffd640 1 --2- 172.21.15.94:0/2910121282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90e4101140 0x7f90e4112120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:34.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.326+0000 7f90e37fe640 1 -- 172.21.15.94:0/2910121282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90e4101140 msgr2=0x7f90e4112120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:34.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.326+0000 7f90e37fe640 1 --2- 172.21.15.94:0/2910121282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90e4101140 0x7f90e4112120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:34.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.326+0000 7f90e37fe640 1 -- 172.21.15.94:0/2910121282 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d8005040 con 0x7f90e40fb110 2024-03-24T10:57:34.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.326+0000 7f90e37fe640 1 --2- 172.21.15.94:0/2910121282 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90e40fb110 0x7f90e4111be0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f90d4002a80 tx=0x7f90d4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:34.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.326+0000 7f90cbfff640 1 -- 172.21.15.94:0/2910121282 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90d4007c30 con 0x7f90e40fb110 2024-03-24T10:57:34.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.326+0000 7f90cbfff640 1 -- 172.21.15.94:0/2910121282 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f90d400f040 con 0x7f90e40fb110 2024-03-24T10:57:34.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.326+0000 7f90e9c6e640 1 -- 172.21.15.94:0/2910121282 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90e410c050 con 0x7f90e40fb110 2024-03-24T10:57:34.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.326+0000 7f90cbfff640 1 -- 172.21.15.94:0/2910121282 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90d40096b0 con 0x7f90e40fb110 2024-03-24T10:57:34.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.326+0000 7f90e9c6e640 1 -- 172.21.15.94:0/2910121282 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90e410c540 con 0x7f90e40fb110 2024-03-24T10:57:34.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.327+0000 7f90e9c6e640 1 -- 172.21.15.94:0/2910121282 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90e40fc800 con 0x7f90e40fb110 2024-03-24T10:57:34.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.328+0000 7f90cbfff640 1 -- 172.21.15.94:0/2910121282 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f90d400d6b0 con 0x7f90e40fb110 2024-03-24T10:57:34.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.328+0000 7f90cbfff640 1 --2- 172.21.15.94:0/2910121282 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f90cc06d750 0x7f90cc06fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:34.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.328+0000 7f90cbfff640 1 -- 172.21.15.94:0/2910121282 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f90d408c940 con 0x7f90e40fb110 2024-03-24T10:57:34.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.328+0000 7f90e2ffd640 1 --2- 172.21.15.94:0/2910121282 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f90cc06d750 0x7f90cc06fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:34.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.329+0000 7f90e2ffd640 1 --2- 172.21.15.94:0/2910121282 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f90cc06d750 0x7f90cc06fc00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f90d8000c00 tx=0x7f90d8009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:34.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.333+0000 7f90cbfff640 1 -- 172.21.15.94:0/2910121282 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f90d4056140 con 0x7f90e40fb110 2024-03-24T10:57:34.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.514+0000 7f90e9c6e640 1 -- 172.21.15.94:0/2910121282 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f90e4108200 con 0x7f90e40fb110 2024-03-24T10:57:34.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.514+0000 7f90cbfff640 1 -- 172.21.15.94:0/2910121282 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7f90d405ee60 con 0x7f90e40fb110 2024-03-24T10:57:34.514 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-03-24T10:57:34.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.517+0000 7f90e9c6e640 1 -- 172.21.15.94:0/2910121282 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f90cc06d750 msgr2=0x7f90cc06fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:34.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.517+0000 7f90e9c6e640 1 --2- 172.21.15.94:0/2910121282 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f90cc06d750 0x7f90cc06fc00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f90d8000c00 tx=0x7f90d8009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:34.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.517+0000 7f90e9c6e640 1 -- 172.21.15.94:0/2910121282 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90e40fb110 msgr2=0x7f90e4111be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:34.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.517+0000 7f90e9c6e640 1 --2- 172.21.15.94:0/2910121282 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90e40fb110 0x7f90e4111be0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f90d4002a80 tx=0x7f90d4004600 comp rx=0 tx=0).stop 2024-03-24T10:57:34.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.517+0000 7f90e9c6e640 1 -- 172.21.15.94:0/2910121282 shutdown_connections 2024-03-24T10:57:34.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.517+0000 7f90e9c6e640 1 --2- 172.21.15.94:0/2910121282 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f90cc06d750 0x7f90cc06fc00 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:34.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.517+0000 7f90e9c6e640 1 --2- 172.21.15.94:0/2910121282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90e4101140 0x7f90e4112120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:34.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.517+0000 7f90e9c6e640 1 --2- 172.21.15.94:0/2910121282 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90e40fb110 0x7f90e4111be0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:34.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.517+0000 7f90e9c6e640 1 -- 172.21.15.94:0/2910121282 >> 172.21.15.94:0/2910121282 conn(0x7f90e40710b0 msgr2=0x7f90e40f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:34.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.518+0000 7f90e9c6e640 1 -- 172.21.15.94:0/2910121282 shutdown_connections 2024-03-24T10:57:34.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.518+0000 7f90e9c6e640 1 -- 172.21.15.94:0/2910121282 wait complete. 2024-03-24T10:57:34.528 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-24T10:57:34.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.631+0000 7f3d1942e640 1 -- 172.21.15.94:0/2678774772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d141023d0 msgr2=0x7f3d141027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:34.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.631+0000 7f3d1942e640 1 --2- 172.21.15.94:0/2678774772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d141023d0 0x7f3d141027a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3d040048d0 tx=0x7f3d04009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:34.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.632+0000 7f3d1942e640 1 -- 172.21.15.94:0/2678774772 shutdown_connections 2024-03-24T10:57:34.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.632+0000 7f3d1942e640 1 --2- 172.21.15.94:0/2678774772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d14102d70 0x7f3d140fc400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:34.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.632+0000 7f3d1942e640 1 --2- 172.21.15.94:0/2678774772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d141023d0 0x7f3d141027a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:34.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.632+0000 7f3d1942e640 1 -- 172.21.15.94:0/2678774772 >> 172.21.15.94:0/2678774772 conn(0x7f3d140f7fe0 msgr2=0x7f3d140fa3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:34.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.632+0000 7f3d1942e640 1 -- 172.21.15.94:0/2678774772 shutdown_connections 2024-03-24T10:57:34.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.632+0000 7f3d1942e640 1 -- 172.21.15.94:0/2678774772 wait complete. 2024-03-24T10:57:34.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.632+0000 7f3d1942e640 1 Processor -- start 2024-03-24T10:57:34.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.632+0000 7f3d1942e640 1 -- start start 2024-03-24T10:57:34.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.632+0000 7f3d1942e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d141023d0 0x7f3d14112f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:34.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.632+0000 7f3d1942e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d14102d70 0x7f3d14113490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:34.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.632+0000 7f3d1942e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d14113a90 con 0x7f3d141023d0 2024-03-24T10:57:34.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.632+0000 7f3d1942e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1410d0e0 con 0x7f3d14102d70 2024-03-24T10:57:34.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.633+0000 7f3d12ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d141023d0 0x7f3d14112f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:34.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.633+0000 7f3d12ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d141023d0 0x7f3d14112f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51600/0 (socket says 172.21.15.94:51600) 2024-03-24T10:57:34.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.633+0000 7f3d12ffd640 1 -- 172.21.15.94:0/4158269037 learned_addr learned my addr 172.21.15.94:0/4158269037 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:57:34.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.633+0000 7f3d12ffd640 1 -- 172.21.15.94:0/4158269037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d14102d70 msgr2=0x7f3d14113490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:34.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.633+0000 7f3d12ffd640 1 --2- 172.21.15.94:0/4158269037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d14102d70 0x7f3d14113490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:34.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.633+0000 7f3d12ffd640 1 -- 172.21.15.94:0/4158269037 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d04005040 con 0x7f3d141023d0 2024-03-24T10:57:34.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.633+0000 7f3d12ffd640 1 --2- 172.21.15.94:0/4158269037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d141023d0 0x7f3d14112f50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3d04004930 tx=0x7f3d04009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:34.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.633+0000 7f3d0b7fe640 1 -- 172.21.15.94:0/4158269037 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d040094c0 con 0x7f3d141023d0 2024-03-24T10:57:34.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.633+0000 7f3d1942e640 1 -- 172.21.15.94:0/4158269037 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d1410d360 con 0x7f3d141023d0 2024-03-24T10:57:34.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.633+0000 7f3d0b7fe640 1 -- 172.21.15.94:0/4158269037 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3d04032e50 con 0x7f3d141023d0 2024-03-24T10:57:34.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.633+0000 7f3d0b7fe640 1 -- 172.21.15.94:0/4158269037 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d040433b0 con 0x7f3d141023d0 2024-03-24T10:57:34.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.633+0000 7f3d1942e640 1 -- 172.21.15.94:0/4158269037 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d1410d820 con 0x7f3d141023d0 2024-03-24T10:57:34.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.634+0000 7f3d1942e640 1 -- 172.21.15.94:0/4158269037 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d140fdb40 con 0x7f3d141023d0 2024-03-24T10:57:34.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.635+0000 7f3d0b7fe640 1 -- 172.21.15.94:0/4158269037 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f3d0402f070 con 0x7f3d141023d0 2024-03-24T10:57:34.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.635+0000 7f3d0b7fe640 1 --2- 172.21.15.94:0/4158269037 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f3d0006d750 0x7f3d0006fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:34.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.635+0000 7f3d0b7fe640 1 -- 172.21.15.94:0/4158269037 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f3d040aee20 con 0x7f3d141023d0 2024-03-24T10:57:34.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.636+0000 7f3d0bfff640 1 --2- 172.21.15.94:0/4158269037 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f3d0006d750 0x7f3d0006fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:34.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.636+0000 7f3d0bfff640 1 --2- 172.21.15.94:0/4158269037 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f3d0006d750 0x7f3d0006fc00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3d1410e810 tx=0x7f3cfc009c50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:34.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.640+0000 7f3d0b7fe640 1 -- 172.21.15.94:0/4158269037 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3d04078570 con 0x7f3d141023d0 2024-03-24T10:57:34.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.937+0000 7f3d1942e640 1 -- 172.21.15.94:0/4158269037 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f3d1410e5c0 con 0x7f3d141023d0 2024-03-24T10:57:34.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.938+0000 7f3d0b7fe640 1 -- 172.21.15.94:0/4158269037 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7f3d0407bc30 con 0x7f3d141023d0 2024-03-24T10:57:34.938 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-03-24T10:57:34.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.940+0000 7f3d1942e640 1 -- 172.21.15.94:0/4158269037 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f3d0006d750 msgr2=0x7f3d0006fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:34.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.940+0000 7f3d1942e640 1 --2- 172.21.15.94:0/4158269037 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f3d0006d750 0x7f3d0006fc00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3d1410e810 tx=0x7f3cfc009c50 comp rx=0 tx=0).stop 2024-03-24T10:57:34.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.940+0000 7f3d1942e640 1 -- 172.21.15.94:0/4158269037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d141023d0 msgr2=0x7f3d14112f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:34.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.940+0000 7f3d1942e640 1 --2- 172.21.15.94:0/4158269037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d141023d0 0x7f3d14112f50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3d04004930 tx=0x7f3d04009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:34.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.940+0000 7f3d1942e640 1 -- 172.21.15.94:0/4158269037 shutdown_connections 2024-03-24T10:57:34.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.940+0000 7f3d1942e640 1 --2- 172.21.15.94:0/4158269037 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f3d0006d750 0x7f3d0006fc00 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:34.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.940+0000 7f3d1942e640 1 --2- 172.21.15.94:0/4158269037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d14102d70 0x7f3d14113490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:34.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.940+0000 7f3d1942e640 1 --2- 172.21.15.94:0/4158269037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d141023d0 0x7f3d14112f50 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:34.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.940+0000 7f3d1942e640 1 -- 172.21.15.94:0/4158269037 >> 172.21.15.94:0/4158269037 conn(0x7f3d140f7fe0 msgr2=0x7f3d140f8f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:34.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.941+0000 7f3d1942e640 1 -- 172.21.15.94:0/4158269037 shutdown_connections 2024-03-24T10:57:34.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:34.941+0000 7f3d1942e640 1 -- 172.21.15.94:0/4158269037 wait complete. 2024-03-24T10:57:34.950 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:34 smithi094 ceph-mon[93957]: pgmap v307: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:57:34.950 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:34 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/2530031714' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T10:57:34.950 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:34 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/2422086695' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T10:57:34.951 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:34 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/2910121282' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-24T10:57:34.953 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-24T10:57:35.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.056+0000 7f2a496ae640 1 -- 172.21.15.94:0/1405690429 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a44101140 msgr2=0x7f2a44101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:35.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.056+0000 7f2a496ae640 1 --2- 172.21.15.94:0/1405690429 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a44101140 0x7f2a44101510 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2a340048d0 tx=0x7f2a34009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:35.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.056+0000 7f2a496ae640 1 -- 172.21.15.94:0/1405690429 shutdown_connections 2024-03-24T10:57:35.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.056+0000 7f2a496ae640 1 --2- 172.21.15.94:0/1405690429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a440fb110 0x7f2a440fb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:35.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.056+0000 7f2a496ae640 1 --2- 172.21.15.94:0/1405690429 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a44101140 0x7f2a44101510 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:35.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.056+0000 7f2a496ae640 1 -- 172.21.15.94:0/1405690429 >> 172.21.15.94:0/1405690429 conn(0x7f2a44071130 msgr2=0x7f2a44071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:35.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.056+0000 7f2a496ae640 1 -- 172.21.15.94:0/1405690429 shutdown_connections 2024-03-24T10:57:35.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.056+0000 7f2a496ae640 1 -- 172.21.15.94:0/1405690429 wait complete. 2024-03-24T10:57:35.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.057+0000 7f2a496ae640 1 Processor -- start 2024-03-24T10:57:35.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.057+0000 7f2a496ae640 1 -- start start 2024-03-24T10:57:35.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.057+0000 7f2a496ae640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a440fb110 0x7f2a44112ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:35.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.057+0000 7f2a496ae640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a44101140 0x7f2a441131e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:35.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.057+0000 7f2a496ae640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a441137e0 con 0x7f2a440fb110 2024-03-24T10:57:35.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.057+0000 7f2a496ae640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a44113950 con 0x7f2a44101140 2024-03-24T10:57:35.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.057+0000 7f2a42ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a440fb110 0x7f2a44112ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:35.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.057+0000 7f2a42ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a440fb110 0x7f2a44112ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51626/0 (socket says 172.21.15.94:51626) 2024-03-24T10:57:35.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.057+0000 7f2a42ffd640 1 -- 172.21.15.94:0/120937006 learned_addr learned my addr 172.21.15.94:0/120937006 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:57:35.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.057+0000 7f2a427fc640 1 --2- 172.21.15.94:0/120937006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a44101140 0x7f2a441131e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:35.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.057+0000 7f2a42ffd640 1 -- 172.21.15.94:0/120937006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a44101140 msgr2=0x7f2a441131e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:35.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.057+0000 7f2a42ffd640 1 --2- 172.21.15.94:0/120937006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a44101140 0x7f2a441131e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:35.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.057+0000 7f2a42ffd640 1 -- 172.21.15.94:0/120937006 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a34005040 con 0x7f2a440fb110 2024-03-24T10:57:35.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.057+0000 7f2a427fc640 1 --2- 172.21.15.94:0/120937006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a44101140 0x7f2a441131e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T10:57:35.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.057+0000 7f2a42ffd640 1 --2- 172.21.15.94:0/120937006 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a440fb110 0x7f2a44112ca0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2a34030640 tx=0x7f2a34009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:35.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.058+0000 7f2a2b7fe640 1 -- 172.21.15.94:0/120937006 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a34030820 con 0x7f2a440fb110 2024-03-24T10:57:35.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.058+0000 7f2a496ae640 1 -- 172.21.15.94:0/120937006 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a4410cf80 con 0x7f2a440fb110 2024-03-24T10:57:35.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.058+0000 7f2a2b7fe640 1 -- 172.21.15.94:0/120937006 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a340092e0 con 0x7f2a440fb110 2024-03-24T10:57:35.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.058+0000 7f2a2b7fe640 1 -- 172.21.15.94:0/120937006 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a34043400 con 0x7f2a440fb110 2024-03-24T10:57:35.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.058+0000 7f2a496ae640 1 -- 172.21.15.94:0/120937006 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a4410d4a0 con 0x7f2a440fb110 2024-03-24T10:57:35.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.059+0000 7f2a496ae640 1 -- 172.21.15.94:0/120937006 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a440fc800 con 0x7f2a440fb110 2024-03-24T10:57:35.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.059+0000 7f2a2b7fe640 1 -- 172.21.15.94:0/120937006 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f2a3403b420 con 0x7f2a440fb110 2024-03-24T10:57:35.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.059+0000 7f2a2b7fe640 1 --2- 172.21.15.94:0/120937006 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f2a2c06d750 0x7f2a2c06fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:35.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.059+0000 7f2a2b7fe640 1 -- 172.21.15.94:0/120937006 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f2a340afe00 con 0x7f2a440fb110 2024-03-24T10:57:35.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.059+0000 7f2a427fc640 1 --2- 172.21.15.94:0/120937006 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f2a2c06d750 0x7f2a2c06fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:35.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.060+0000 7f2a427fc640 1 --2- 172.21.15.94:0/120937006 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f2a2c06d750 0x7f2a2c06fc00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2a4410e5a0 tx=0x7f2a3800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:35.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.064+0000 7f2a2b7fe640 1 -- 172.21.15.94:0/120937006 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f2a34079690 con 0x7f2a440fb110 2024-03-24T10:57:35.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:34 smithi096 ceph-mon[84258]: pgmap v307: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:57:35.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:34 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/2530031714' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T10:57:35.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:34 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/2422086695' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T10:57:35.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:34 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/2910121282' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-24T10:57:35.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.281+0000 7f2a496ae640 1 -- 172.21.15.94:0/120937006 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a4404bcc0 con 0x7f2a440fb110 2024-03-24T10:57:35.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.282+0000 7f2a2b7fe640 1 -- 172.21.15.94:0/120937006 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+5215 (secure 0 0 0) 0x7f2a3407cd50 con 0x7f2a440fb110 2024-03-24T10:57:35.282 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:57:35.282 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44321,"name":"cephfs.smithi096.egbmoq","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6825/2694461318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2694461318},{"type":"v1","addr":"172.21.15.96:6825","nonce":2694461318}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":44349,"name":"cephfs.smithi096.jtebhh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6827/2275611388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":2275611388},{"type":"v1","addr":"172.21.15.96:6827","nonce":2275611388}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-24T10:43:39.435738+0000","modified":"2024-03-24T10:54:19.399866+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34374},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34374":{"gid":34374,"name":"cephfs.smithi094.efiqte","rank":0,"incarnation":32,"state":"up:active","state_seq":19,"addr":"172.21.15.94:6827/2043632450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2043632450},{"type":"v1","addr":"172.21.15.94:6827","nonce":2043632450}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"cephfs.smithi094.llefgu","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6829/2396893131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2396893131},{"type":"v1","addr":"172.21.15.94:6829","nonce":2396893131}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34374,"qdb_cluster":[34374]},"id":1}]} 2024-03-24T10:57:35.282 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 37 2024-03-24T10:57:35.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.285+0000 7f2a496ae640 1 -- 172.21.15.94:0/120937006 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f2a2c06d750 msgr2=0x7f2a2c06fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:35.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.285+0000 7f2a496ae640 1 --2- 172.21.15.94:0/120937006 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f2a2c06d750 0x7f2a2c06fc00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2a4410e5a0 tx=0x7f2a3800b040 comp rx=0 tx=0).stop 2024-03-24T10:57:35.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.285+0000 7f2a496ae640 1 -- 172.21.15.94:0/120937006 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a440fb110 msgr2=0x7f2a44112ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:35.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.285+0000 7f2a496ae640 1 --2- 172.21.15.94:0/120937006 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a440fb110 0x7f2a44112ca0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2a34030640 tx=0x7f2a34009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:35.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.285+0000 7f2a496ae640 1 -- 172.21.15.94:0/120937006 shutdown_connections 2024-03-24T10:57:35.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.285+0000 7f2a496ae640 1 --2- 172.21.15.94:0/120937006 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f2a2c06d750 0x7f2a2c06fc00 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:35.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.285+0000 7f2a496ae640 1 --2- 172.21.15.94:0/120937006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a44101140 0x7f2a441131e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:35.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.285+0000 7f2a496ae640 1 --2- 172.21.15.94:0/120937006 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a440fb110 0x7f2a44112ca0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:35.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.285+0000 7f2a496ae640 1 -- 172.21.15.94:0/120937006 >> 172.21.15.94:0/120937006 conn(0x7f2a44071130 msgr2=0x7f2a44064370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:35.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.285+0000 7f2a496ae640 1 -- 172.21.15.94:0/120937006 shutdown_connections 2024-03-24T10:57:35.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.285+0000 7f2a496ae640 1 -- 172.21.15.94:0/120937006 wait complete. 2024-03-24T10:57:35.295 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 18, 'max_mds': 1, 'flags': 50} 2024-03-24T10:57:35.295 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 19 2024-03-24T10:57:35.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.397+0000 7f1e6195d640 1 -- 172.21.15.94:0/1106050540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e5c0643a0 msgr2=0x7f1e5c0647d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:35.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.397+0000 7f1e6195d640 1 --2- 172.21.15.94:0/1106050540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e5c0643a0 0x7f1e5c0647d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1e440048d0 tx=0x7f1e44009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:35.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.397+0000 7f1e6195d640 1 -- 172.21.15.94:0/1106050540 shutdown_connections 2024-03-24T10:57:35.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.397+0000 7f1e6195d640 1 --2- 172.21.15.94:0/1106050540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e5c0643a0 0x7f1e5c0647d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:35.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.397+0000 7f1e6195d640 1 --2- 172.21.15.94:0/1106050540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e5c102240 0x7f1e5c102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:35.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.397+0000 7f1e6195d640 1 -- 172.21.15.94:0/1106050540 >> 172.21.15.94:0/1106050540 conn(0x7f1e5c0710b0 msgr2=0x7f1e5c0714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:35.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.397+0000 7f1e6195d640 1 -- 172.21.15.94:0/1106050540 shutdown_connections 2024-03-24T10:57:35.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.397+0000 7f1e6195d640 1 -- 172.21.15.94:0/1106050540 wait complete. 2024-03-24T10:57:35.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.398+0000 7f1e6195d640 1 Processor -- start 2024-03-24T10:57:35.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.398+0000 7f1e6195d640 1 -- start start 2024-03-24T10:57:35.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.398+0000 7f1e6195d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e5c0643a0 0x7f1e5c112cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:35.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.398+0000 7f1e6195d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e5c102240 0x7f1e5c113200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:35.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.398+0000 7f1e6195d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c113800 con 0x7f1e5c102240 2024-03-24T10:57:35.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.398+0000 7f1e6195d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c10ce50 con 0x7f1e5c0643a0 2024-03-24T10:57:35.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.398+0000 7f1e53fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e5c102240 0x7f1e5c113200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:35.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.398+0000 7f1e53fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e5c102240 0x7f1e5c113200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51650/0 (socket says 172.21.15.94:51650) 2024-03-24T10:57:35.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.398+0000 7f1e53fff640 1 -- 172.21.15.94:0/229906493 learned_addr learned my addr 172.21.15.94:0/229906493 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:57:35.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.399+0000 7f1e53fff640 1 -- 172.21.15.94:0/229906493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e5c0643a0 msgr2=0x7f1e5c112cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:35.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.399+0000 7f1e53fff640 1 --2- 172.21.15.94:0/229906493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e5c0643a0 0x7f1e5c112cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:35.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.399+0000 7f1e53fff640 1 -- 172.21.15.94:0/229906493 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e44005040 con 0x7f1e5c102240 2024-03-24T10:57:35.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.399+0000 7f1e53fff640 1 --2- 172.21.15.94:0/229906493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e5c102240 0x7f1e5c113200 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1e44004930 tx=0x7f1e44009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:35.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.399+0000 7f1e537fe640 1 -- 172.21.15.94:0/229906493 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e440094c0 con 0x7f1e5c102240 2024-03-24T10:57:35.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.399+0000 7f1e537fe640 1 -- 172.21.15.94:0/229906493 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1e44032e50 con 0x7f1e5c102240 2024-03-24T10:57:35.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.399+0000 7f1e537fe640 1 -- 172.21.15.94:0/229906493 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e440433b0 con 0x7f1e5c102240 2024-03-24T10:57:35.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.399+0000 7f1e6195d640 1 -- 172.21.15.94:0/229906493 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e5c10d0d0 con 0x7f1e5c102240 2024-03-24T10:57:35.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.399+0000 7f1e6195d640 1 -- 172.21.15.94:0/229906493 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e5c10d590 con 0x7f1e5c102240 2024-03-24T10:57:35.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.400+0000 7f1e6195d640 1 -- 172.21.15.94:0/229906493 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e5c0fd940 con 0x7f1e5c102240 2024-03-24T10:57:35.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.401+0000 7f1e537fe640 1 -- 172.21.15.94:0/229906493 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f1e4402f070 con 0x7f1e5c102240 2024-03-24T10:57:35.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.401+0000 7f1e537fe640 1 --2- 172.21.15.94:0/229906493 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1e4806d750 0x7f1e4806fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:35.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.401+0000 7f1e537fe640 1 -- 172.21.15.94:0/229906493 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f1e440af000 con 0x7f1e5c102240 2024-03-24T10:57:35.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.401+0000 7f1e5affd640 1 --2- 172.21.15.94:0/229906493 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1e4806d750 0x7f1e4806fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:35.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.402+0000 7f1e5affd640 1 --2- 172.21.15.94:0/229906493 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1e4806d750 0x7f1e4806fc00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c004af0 tx=0x7f1e4c007c30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:35.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.408+0000 7f1e537fe640 1 -- 172.21.15.94:0/229906493 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1e44078750 con 0x7f1e5c102240 2024-03-24T10:57:35.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.625+0000 7f1e6195d640 1 -- 172.21.15.94:0/229906493 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) v1 -- 0x7f1e5c109300 con 0x7f1e5c102240 2024-03-24T10:57:35.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.626+0000 7f1e537fe640 1 -- 172.21.15.94:0/229906493 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v37) v1 ==== 107+0+4088 (secure 0 0 0) 0x7f1e4407be10 con 0x7f1e5c102240 2024-03-24T10:57:35.626 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:57:35.626 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14512,"name":"cephfs.smithi094.efiqte","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6827/1636528860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1636528860},{"type":"v1","addr":"172.21.15.94:6827","nonce":1636528860}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":24261,"name":"cephfs.smithi096.jtebhh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.96:6827/1321087930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1321087930},{"type":"v1","addr":"172.21.15.96:6827","nonce":1321087930}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-24T10:43:39.435738+0000","modified":"2024-03-24T10:52:59.495272+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24251":{"gid":24251,"name":"cephfs.smithi096.egbmoq","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.96:6825/1539752208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1539752208},{"type":"v1","addr":"172.21.15.96:6825","nonce":1539752208}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24251,"qdb_cluster":[24251]},"id":1}]} 2024-03-24T10:57:35.626 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 19 2024-03-24T10:57:35.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.628+0000 7f1e6195d640 1 -- 172.21.15.94:0/229906493 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1e4806d750 msgr2=0x7f1e4806fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:35.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.628+0000 7f1e6195d640 1 --2- 172.21.15.94:0/229906493 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1e4806d750 0x7f1e4806fc00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c004af0 tx=0x7f1e4c007c30 comp rx=0 tx=0).stop 2024-03-24T10:57:35.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.628+0000 7f1e6195d640 1 -- 172.21.15.94:0/229906493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e5c102240 msgr2=0x7f1e5c113200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:35.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.628+0000 7f1e6195d640 1 --2- 172.21.15.94:0/229906493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e5c102240 0x7f1e5c113200 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1e44004930 tx=0x7f1e44009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:35.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.628+0000 7f1e6195d640 1 -- 172.21.15.94:0/229906493 shutdown_connections 2024-03-24T10:57:35.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.628+0000 7f1e6195d640 1 --2- 172.21.15.94:0/229906493 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1e4806d750 0x7f1e4806fc00 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:35.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.628+0000 7f1e6195d640 1 --2- 172.21.15.94:0/229906493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e5c102240 0x7f1e5c113200 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:35.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.628+0000 7f1e6195d640 1 --2- 172.21.15.94:0/229906493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e5c0643a0 0x7f1e5c112cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:35.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.628+0000 7f1e6195d640 1 -- 172.21.15.94:0/229906493 >> 172.21.15.94:0/229906493 conn(0x7f1e5c0710b0 msgr2=0x7f1e5c0f94b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:35.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.628+0000 7f1e6195d640 1 -- 172.21.15.94:0/229906493 shutdown_connections 2024-03-24T10:57:35.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.628+0000 7f1e6195d640 1 -- 172.21.15.94:0/229906493 wait complete. 2024-03-24T10:57:35.639 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 19 2024-03-24T10:57:35.639 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 20 2024-03-24T10:57:35.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.738+0000 7f70f1407640 1 -- 172.21.15.94:0/1202126150 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70ec0fb110 msgr2=0x7f70ec0fb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:35.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.738+0000 7f70f1407640 1 --2- 172.21.15.94:0/1202126150 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70ec0fb110 0x7f70ec0fb560 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f70d40048d0 tx=0x7f70d4009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:35.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.738+0000 7f70f1407640 1 -- 172.21.15.94:0/1202126150 shutdown_connections 2024-03-24T10:57:35.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.738+0000 7f70f1407640 1 --2- 172.21.15.94:0/1202126150 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70ec0fb110 0x7f70ec0fb560 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:35.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.738+0000 7f70f1407640 1 --2- 172.21.15.94:0/1202126150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f70ec101140 0x7f70ec101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:35.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.738+0000 7f70f1407640 1 -- 172.21.15.94:0/1202126150 >> 172.21.15.94:0/1202126150 conn(0x7f70ec0710b0 msgr2=0x7f70ec0714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:35.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.738+0000 7f70f1407640 1 -- 172.21.15.94:0/1202126150 shutdown_connections 2024-03-24T10:57:35.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.738+0000 7f70f1407640 1 -- 172.21.15.94:0/1202126150 wait complete. 2024-03-24T10:57:35.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.739+0000 7f70f1407640 1 Processor -- start 2024-03-24T10:57:35.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.739+0000 7f70f1407640 1 -- start start 2024-03-24T10:57:35.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.739+0000 7f70f1407640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70ec0fb110 0x7f70ec0fcd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:35.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.739+0000 7f70f1407640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f70ec101140 0x7f70ec0fd270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:35.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.739+0000 7f70f1407640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ec0fec10 con 0x7f70ec0fb110 2024-03-24T10:57:35.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.739+0000 7f70f1407640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ec0fd7b0 con 0x7f70ec101140 2024-03-24T10:57:35.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.739+0000 7f70eaffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70ec0fb110 0x7f70ec0fcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:35.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.739+0000 7f70eaffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70ec0fb110 0x7f70ec0fcd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51676/0 (socket says 172.21.15.94:51676) 2024-03-24T10:57:35.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.739+0000 7f70eaffd640 1 -- 172.21.15.94:0/346865450 learned_addr learned my addr 172.21.15.94:0/346865450 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:57:35.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.739+0000 7f70e3fff640 1 --2- 172.21.15.94:0/346865450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f70ec101140 0x7f70ec0fd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:35.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.739+0000 7f70eaffd640 1 -- 172.21.15.94:0/346865450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f70ec101140 msgr2=0x7f70ec0fd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:35.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.739+0000 7f70eaffd640 1 --2- 172.21.15.94:0/346865450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f70ec101140 0x7f70ec0fd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:35.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.739+0000 7f70eaffd640 1 -- 172.21.15.94:0/346865450 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d4005040 con 0x7f70ec0fb110 2024-03-24T10:57:35.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.739+0000 7f70eaffd640 1 --2- 172.21.15.94:0/346865450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70ec0fb110 0x7f70ec0fcd30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f70dc002a50 tx=0x7f70dc004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:35.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.739+0000 7f70e37fe640 1 -- 172.21.15.94:0/346865450 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70dc007c30 con 0x7f70ec0fb110 2024-03-24T10:57:35.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.740+0000 7f70f1407640 1 -- 172.21.15.94:0/346865450 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70ec0fda90 con 0x7f70ec0fb110 2024-03-24T10:57:35.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.740+0000 7f70e37fe640 1 -- 172.21.15.94:0/346865450 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f70dc00f040 con 0x7f70ec0fb110 2024-03-24T10:57:35.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.740+0000 7f70e37fe640 1 -- 172.21.15.94:0/346865450 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70dc00dbe0 con 0x7f70ec0fb110 2024-03-24T10:57:35.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.740+0000 7f70f1407640 1 -- 172.21.15.94:0/346865450 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70ec119400 con 0x7f70ec0fb110 2024-03-24T10:57:35.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.740+0000 7f70f1407640 1 -- 172.21.15.94:0/346865450 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70ec0fc800 con 0x7f70ec0fb110 2024-03-24T10:57:35.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.741+0000 7f70e37fe640 1 -- 172.21.15.94:0/346865450 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f70dc00dd40 con 0x7f70ec0fb110 2024-03-24T10:57:35.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.741+0000 7f70e37fe640 1 --2- 172.21.15.94:0/346865450 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f70d806d760 0x7f70d806fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:35.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.741+0000 7f70e37fe640 1 -- 172.21.15.94:0/346865450 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f70dc08cfc0 con 0x7f70ec0fb110 2024-03-24T10:57:35.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.741+0000 7f70e3fff640 1 --2- 172.21.15.94:0/346865450 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f70d806d760 0x7f70d806fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:35.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.742+0000 7f70e3fff640 1 --2- 172.21.15.94:0/346865450 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f70d806d760 0x7f70d806fc10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f70d40043d0 tx=0x7f70d4009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:35.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.746+0000 7f70e37fe640 1 -- 172.21.15.94:0/346865450 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f70dc056710 con 0x7f70ec0fb110 2024-03-24T10:57:35.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.962+0000 7f70f1407640 1 -- 172.21.15.94:0/346865450 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7f70ec1195f0 con 0x7f70ec0fb110 2024-03-24T10:57:35.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.963+0000 7f70e37fe640 1 -- 172.21.15.94:0/346865450 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v37) v1 ==== 107+0+4867 (secure 0 0 0) 0x7f70dc059dd0 con 0x7f70ec0fb110 2024-03-24T10:57:35.963 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:57:35.963 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14512,"name":"cephfs.smithi094.efiqte","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6827/1636528860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1636528860},{"type":"v1","addr":"172.21.15.94:6827","nonce":1636528860}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":24261,"name":"cephfs.smithi096.jtebhh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.96:6827/1321087930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1321087930},{"type":"v1","addr":"172.21.15.96:6827","nonce":1321087930}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":34370,"name":"cephfs.smithi094.llefgu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6829/3011883830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3011883830},{"type":"v1","addr":"172.21.15.94:6829","nonce":3011883830}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-24T10:43:39.435738+0000","modified":"2024-03-24T10:52:59.495272+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24251":{"gid":24251,"name":"cephfs.smithi096.egbmoq","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.96:6825/1539752208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1539752208},{"type":"v1","addr":"172.21.15.96:6825","nonce":1539752208}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24251,"qdb_cluster":[24251]},"id":1}]} 2024-03-24T10:57:35.963 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 20 2024-03-24T10:57:35.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.965+0000 7f70f1407640 1 -- 172.21.15.94:0/346865450 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f70d806d760 msgr2=0x7f70d806fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:35.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.965+0000 7f70f1407640 1 --2- 172.21.15.94:0/346865450 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f70d806d760 0x7f70d806fc10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f70d40043d0 tx=0x7f70d4009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:35.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.965+0000 7f70f1407640 1 -- 172.21.15.94:0/346865450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70ec0fb110 msgr2=0x7f70ec0fcd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:35.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.965+0000 7f70f1407640 1 --2- 172.21.15.94:0/346865450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70ec0fb110 0x7f70ec0fcd30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f70dc002a50 tx=0x7f70dc004600 comp rx=0 tx=0).stop 2024-03-24T10:57:35.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.965+0000 7f70f1407640 1 -- 172.21.15.94:0/346865450 shutdown_connections 2024-03-24T10:57:35.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.965+0000 7f70f1407640 1 --2- 172.21.15.94:0/346865450 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f70d806d760 0x7f70d806fc10 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:35.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.965+0000 7f70f1407640 1 --2- 172.21.15.94:0/346865450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f70ec101140 0x7f70ec0fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:35.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.965+0000 7f70f1407640 1 --2- 172.21.15.94:0/346865450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70ec0fb110 0x7f70ec0fcd30 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:35.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.965+0000 7f70f1407640 1 -- 172.21.15.94:0/346865450 >> 172.21.15.94:0/346865450 conn(0x7f70ec0710b0 msgr2=0x7f70ec0f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:35.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.965+0000 7f70f1407640 1 -- 172.21.15.94:0/346865450 shutdown_connections 2024-03-24T10:57:35.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:35.966+0000 7f70f1407640 1 -- 172.21.15.94:0/346865450 wait complete. 2024-03-24T10:57:35.975 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:35 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/4158269037' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-24T10:57:35.975 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:35 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/120937006' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T10:57:35.975 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:35 smithi094 ceph-mon[93957]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:57:35.976 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:35 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/229906493' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-03-24T10:57:35.976 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 20 2024-03-24T10:57:35.977 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 21 2024-03-24T10:57:36.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:35 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/4158269037' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-24T10:57:36.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:35 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/120937006' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-24T10:57:36.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:35 smithi096 ceph-mon[84258]: from='mgr.34104 172.21.15.94:0/2181579193' entity='mgr.smithi094.yaethu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T10:57:36.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:35 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/229906493' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-03-24T10:57:36.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.079+0000 7f855aa6e640 1 -- 172.21.15.94:0/2507515332 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8554102b80 msgr2=0x7f85540fc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:36.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.079+0000 7f855aa6e640 1 --2- 172.21.15.94:0/2507515332 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8554102b80 0x7f85540fc210 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f85440048d0 tx=0x7f8544009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:36.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.079+0000 7f855aa6e640 1 -- 172.21.15.94:0/2507515332 shutdown_connections 2024-03-24T10:57:36.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.079+0000 7f855aa6e640 1 --2- 172.21.15.94:0/2507515332 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8554102b80 0x7f85540fc210 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:36.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.079+0000 7f855aa6e640 1 --2- 172.21.15.94:0/2507515332 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85541021e0 0x7f85541025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:36.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.079+0000 7f855aa6e640 1 -- 172.21.15.94:0/2507515332 >> 172.21.15.94:0/2507515332 conn(0x7f85540f7dd0 msgr2=0x7f85540fa1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:36.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.079+0000 7f855aa6e640 1 -- 172.21.15.94:0/2507515332 shutdown_connections 2024-03-24T10:57:36.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.079+0000 7f855aa6e640 1 -- 172.21.15.94:0/2507515332 wait complete. 2024-03-24T10:57:36.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.080+0000 7f855aa6e640 1 Processor -- start 2024-03-24T10:57:36.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.080+0000 7f855aa6e640 1 -- start start 2024-03-24T10:57:36.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.080+0000 7f855aa6e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85541021e0 0x7f8554112d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:36.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.080+0000 7f855aa6e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8554102b80 0x7f8554113240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:36.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.080+0000 7f855aa6e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8554113840 con 0x7f85541021e0 2024-03-24T10:57:36.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.080+0000 7f855aa6e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855410ce90 con 0x7f8554102b80 2024-03-24T10:57:36.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.080+0000 7f855880a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85541021e0 0x7f8554112d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:36.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.080+0000 7f855880a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85541021e0 0x7f8554112d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51702/0 (socket says 172.21.15.94:51702) 2024-03-24T10:57:36.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.080+0000 7f855880a640 1 -- 172.21.15.94:0/119418255 learned_addr learned my addr 172.21.15.94:0/119418255 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:57:36.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.080+0000 7f854bfff640 1 --2- 172.21.15.94:0/119418255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8554102b80 0x7f8554113240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:36.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.080+0000 7f855880a640 1 -- 172.21.15.94:0/119418255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8554102b80 msgr2=0x7f8554113240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:36.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.080+0000 7f855880a640 1 --2- 172.21.15.94:0/119418255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8554102b80 0x7f8554113240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:36.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.080+0000 7f855880a640 1 -- 172.21.15.94:0/119418255 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8544005040 con 0x7f85541021e0 2024-03-24T10:57:36.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.081+0000 7f855880a640 1 --2- 172.21.15.94:0/119418255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85541021e0 0x7f8554112d00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f855000eb70 tx=0x7f855000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:36.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.081+0000 7f85497fa640 1 -- 172.21.15.94:0/119418255 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8550007c30 con 0x7f85541021e0 2024-03-24T10:57:36.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.081+0000 7f85497fa640 1 -- 172.21.15.94:0/119418255 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f855000f040 con 0x7f85541021e0 2024-03-24T10:57:36.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.081+0000 7f855aa6e640 1 -- 172.21.15.94:0/119418255 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f855410d110 con 0x7f85541021e0 2024-03-24T10:57:36.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.081+0000 7f85497fa640 1 -- 172.21.15.94:0/119418255 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85500095b0 con 0x7f85541021e0 2024-03-24T10:57:36.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.081+0000 7f855aa6e640 1 -- 172.21.15.94:0/119418255 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f855410d600 con 0x7f85541021e0 2024-03-24T10:57:36.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.082+0000 7f855aa6e640 1 -- 172.21.15.94:0/119418255 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85540fd950 con 0x7f85541021e0 2024-03-24T10:57:36.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.082+0000 7f85497fa640 1 -- 172.21.15.94:0/119418255 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f8550009730 con 0x7f85541021e0 2024-03-24T10:57:36.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.083+0000 7f85497fa640 1 --2- 172.21.15.94:0/119418255 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f853c06d750 0x7f853c06fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:36.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.083+0000 7f85497fa640 1 -- 172.21.15.94:0/119418255 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f855008cd50 con 0x7f85541021e0 2024-03-24T10:57:36.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.083+0000 7f854bfff640 1 --2- 172.21.15.94:0/119418255 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f853c06d750 0x7f853c06fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:36.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.083+0000 7f854bfff640 1 --2- 172.21.15.94:0/119418255 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f853c06d750 0x7f853c06fc00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f85540fa800 tx=0x7f8544009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:36.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.087+0000 7f85497fa640 1 -- 172.21.15.94:0/119418255 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f85500564a0 con 0x7f85541021e0 2024-03-24T10:57:36.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.323+0000 7f855aa6e640 1 -- 172.21.15.94:0/119418255 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7f855410e400 con 0x7f85541021e0 2024-03-24T10:57:36.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.324+0000 7f85497fa640 1 -- 172.21.15.94:0/119418255 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v37) v1 ==== 107+0+4088 (secure 0 0 0) 0x7f8550059b60 con 0x7f85541021e0 2024-03-24T10:57:36.323 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:57:36.323 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24261,"name":"cephfs.smithi096.jtebhh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.96:6827/1321087930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1321087930},{"type":"v1","addr":"172.21.15.96:6827","nonce":1321087930}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":34370,"name":"cephfs.smithi094.llefgu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6829/3011883830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3011883830},{"type":"v1","addr":"172.21.15.94:6829","nonce":3011883830}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-24T10:43:39.435738+0000","modified":"2024-03-24T10:52:59.495272+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24251":{"gid":24251,"name":"cephfs.smithi096.egbmoq","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.96:6825/1539752208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1539752208},{"type":"v1","addr":"172.21.15.96:6825","nonce":1539752208}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24251,"qdb_cluster":[24251]},"id":1}]} 2024-03-24T10:57:36.324 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 21 2024-03-24T10:57:36.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.325+0000 7f855aa6e640 1 -- 172.21.15.94:0/119418255 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f853c06d750 msgr2=0x7f853c06fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:36.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.325+0000 7f855aa6e640 1 --2- 172.21.15.94:0/119418255 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f853c06d750 0x7f853c06fc00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f85540fa800 tx=0x7f8544009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:36.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.325+0000 7f855aa6e640 1 -- 172.21.15.94:0/119418255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85541021e0 msgr2=0x7f8554112d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:36.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.325+0000 7f855aa6e640 1 --2- 172.21.15.94:0/119418255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85541021e0 0x7f8554112d00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f855000eb70 tx=0x7f855000b040 comp rx=0 tx=0).stop 2024-03-24T10:57:36.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.326+0000 7f855aa6e640 1 -- 172.21.15.94:0/119418255 shutdown_connections 2024-03-24T10:57:36.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.326+0000 7f855aa6e640 1 --2- 172.21.15.94:0/119418255 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f853c06d750 0x7f853c06fc00 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:36.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.326+0000 7f855aa6e640 1 --2- 172.21.15.94:0/119418255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8554102b80 0x7f8554113240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:36.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.326+0000 7f855aa6e640 1 --2- 172.21.15.94:0/119418255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85541021e0 0x7f8554112d00 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:36.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.326+0000 7f855aa6e640 1 -- 172.21.15.94:0/119418255 >> 172.21.15.94:0/119418255 conn(0x7f85540f7dd0 msgr2=0x7f85540f8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:36.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.326+0000 7f855aa6e640 1 -- 172.21.15.94:0/119418255 shutdown_connections 2024-03-24T10:57:36.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.326+0000 7f855aa6e640 1 -- 172.21.15.94:0/119418255 wait complete. 2024-03-24T10:57:36.337 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 21 2024-03-24T10:57:36.337 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 22 2024-03-24T10:57:36.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.436+0000 7f4f97d9d640 1 -- 172.21.15.94:0/1839173555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f901021f0 msgr2=0x7f4f901025c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:36.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.436+0000 7f4f97d9d640 1 --2- 172.21.15.94:0/1839173555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f901021f0 0x7f4f901025c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c0048d0 tx=0x7f4f8c009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:36.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.437+0000 7f4f97d9d640 1 -- 172.21.15.94:0/1839173555 shutdown_connections 2024-03-24T10:57:36.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.437+0000 7f4f97d9d640 1 --2- 172.21.15.94:0/1839173555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f90102b90 0x7f4f900fc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:36.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.437+0000 7f4f97d9d640 1 --2- 172.21.15.94:0/1839173555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f901021f0 0x7f4f901025c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:36.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.437+0000 7f4f97d9d640 1 -- 172.21.15.94:0/1839173555 >> 172.21.15.94:0/1839173555 conn(0x7f4f900f7dc0 msgr2=0x7f4f900fa1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:36.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.437+0000 7f4f97d9d640 1 -- 172.21.15.94:0/1839173555 shutdown_connections 2024-03-24T10:57:36.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.437+0000 7f4f97d9d640 1 -- 172.21.15.94:0/1839173555 wait complete. 2024-03-24T10:57:36.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.437+0000 7f4f97d9d640 1 Processor -- start 2024-03-24T10:57:36.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.437+0000 7f4f97d9d640 1 -- start start 2024-03-24T10:57:36.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.437+0000 7f4f97d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f901021f0 0x7f4f9006f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:36.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.437+0000 7f4f97d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f90102b90 0x7f4f9006fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:36.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.437+0000 7f4f97d9d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f900716b0 con 0x7f4f901021f0 2024-03-24T10:57:36.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.437+0000 7f4f97d9d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f90070250 con 0x7f4f90102b90 2024-03-24T10:57:36.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.437+0000 7f4f95b39640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f901021f0 0x7f4f9006f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:36.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.437+0000 7f4f95b39640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f901021f0 0x7f4f9006f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51712/0 (socket says 172.21.15.94:51712) 2024-03-24T10:57:36.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.437+0000 7f4f95b39640 1 -- 172.21.15.94:0/3631815528 learned_addr learned my addr 172.21.15.94:0/3631815528 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:57:36.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.438+0000 7f4f95b39640 1 -- 172.21.15.94:0/3631815528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f90102b90 msgr2=0x7f4f9006fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:36.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.438+0000 7f4f95338640 1 --2- 172.21.15.94:0/3631815528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f90102b90 0x7f4f9006fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:36.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.438+0000 7f4f95b39640 1 --2- 172.21.15.94:0/3631815528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f90102b90 0x7f4f9006fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:36.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.438+0000 7f4f95b39640 1 -- 172.21.15.94:0/3631815528 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f8c005040 con 0x7f4f901021f0 2024-03-24T10:57:36.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.438+0000 7f4f95b39640 1 --2- 172.21.15.94:0/3631815528 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f901021f0 0x7f4f9006f7d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c004930 tx=0x7f4f8c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:36.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.438+0000 7f4f867fc640 1 -- 172.21.15.94:0/3631815528 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f8c0094c0 con 0x7f4f901021f0 2024-03-24T10:57:36.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.438+0000 7f4f867fc640 1 -- 172.21.15.94:0/3631815528 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4f8c032e50 con 0x7f4f901021f0 2024-03-24T10:57:36.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.438+0000 7f4f97d9d640 1 -- 172.21.15.94:0/3631815528 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f900704d0 con 0x7f4f901021f0 2024-03-24T10:57:36.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.438+0000 7f4f867fc640 1 -- 172.21.15.94:0/3631815528 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f8c0433b0 con 0x7f4f901021f0 2024-03-24T10:57:36.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.438+0000 7f4f97d9d640 1 -- 172.21.15.94:0/3631815528 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f901193b0 con 0x7f4f901021f0 2024-03-24T10:57:36.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.439+0000 7f4f97d9d640 1 -- 172.21.15.94:0/3631815528 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f900fd960 con 0x7f4f901021f0 2024-03-24T10:57:36.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.439+0000 7f4f867fc640 1 -- 172.21.15.94:0/3631815528 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f4f8c02f070 con 0x7f4f901021f0 2024-03-24T10:57:36.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.440+0000 7f4f867fc640 1 --2- 172.21.15.94:0/3631815528 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f4f7c06d750 0x7f4f7c06fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:36.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.440+0000 7f4f867fc640 1 -- 172.21.15.94:0/3631815528 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f4f8c0aee70 con 0x7f4f901021f0 2024-03-24T10:57:36.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.440+0000 7f4f95338640 1 --2- 172.21.15.94:0/3631815528 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f4f7c06d750 0x7f4f7c06fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:36.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.440+0000 7f4f95338640 1 --2- 172.21.15.94:0/3631815528 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f4f7c06d750 0x7f4f7c06fc00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4f90070f00 tx=0x7f4f8000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:36.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.445+0000 7f4f867fc640 1 -- 172.21.15.94:0/3631815528 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4f8c0785c0 con 0x7f4f901021f0 2024-03-24T10:57:36.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.685+0000 7f4f97d9d640 1 -- 172.21.15.94:0/3631815528 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7f4f90119540 con 0x7f4f901021f0 2024-03-24T10:57:36.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.686+0000 7f4f867fc640 1 -- 172.21.15.94:0/3631815528 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v37) v1 ==== 107+0+4935 (secure 0 0 0) 0x7f4f8c07bc80 con 0x7f4f901021f0 2024-03-24T10:57:36.686 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:57:36.686 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24261,"name":"cephfs.smithi096.jtebhh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.96:6827/1321087930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1321087930},{"type":"v1","addr":"172.21.15.96:6827","nonce":1321087930}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":34370,"name":"cephfs.smithi094.llefgu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6829/3011883830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3011883830},{"type":"v1","addr":"172.21.15.94:6829","nonce":3011883830}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":34374,"name":"cephfs.smithi094.efiqte","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6827/2043632450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2043632450},{"type":"v1","addr":"172.21.15.94:6827","nonce":2043632450}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-24T10:43:39.435738+0000","modified":"2024-03-24T10:52:59.495272+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24251":{"gid":24251,"name":"cephfs.smithi096.egbmoq","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.96:6825/1539752208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1539752208},{"type":"v1","addr":"172.21.15.96:6825","nonce":1539752208}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24251,"qdb_cluster":[24251]},"id":1}]} 2024-03-24T10:57:36.686 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 22 2024-03-24T10:57:36.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.688+0000 7f4f97d9d640 1 -- 172.21.15.94:0/3631815528 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f4f7c06d750 msgr2=0x7f4f7c06fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:36.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.688+0000 7f4f97d9d640 1 --2- 172.21.15.94:0/3631815528 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f4f7c06d750 0x7f4f7c06fc00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4f90070f00 tx=0x7f4f8000b040 comp rx=0 tx=0).stop 2024-03-24T10:57:36.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.688+0000 7f4f97d9d640 1 -- 172.21.15.94:0/3631815528 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f901021f0 msgr2=0x7f4f9006f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:36.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.688+0000 7f4f97d9d640 1 --2- 172.21.15.94:0/3631815528 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f901021f0 0x7f4f9006f7d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c004930 tx=0x7f4f8c009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:36.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.688+0000 7f4f97d9d640 1 -- 172.21.15.94:0/3631815528 shutdown_connections 2024-03-24T10:57:36.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.688+0000 7f4f97d9d640 1 --2- 172.21.15.94:0/3631815528 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f4f7c06d750 0x7f4f7c06fc00 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:36.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.688+0000 7f4f97d9d640 1 --2- 172.21.15.94:0/3631815528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f90102b90 0x7f4f9006fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:36.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.688+0000 7f4f97d9d640 1 --2- 172.21.15.94:0/3631815528 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f901021f0 0x7f4f9006f7d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:36.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.688+0000 7f4f97d9d640 1 -- 172.21.15.94:0/3631815528 >> 172.21.15.94:0/3631815528 conn(0x7f4f900f7dc0 msgr2=0x7f4f900f8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:36.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.688+0000 7f4f97d9d640 1 -- 172.21.15.94:0/3631815528 shutdown_connections 2024-03-24T10:57:36.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.688+0000 7f4f97d9d640 1 -- 172.21.15.94:0/3631815528 wait complete. 2024-03-24T10:57:36.699 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 22 2024-03-24T10:57:36.699 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 23 2024-03-24T10:57:36.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.798+0000 7fb64cb9d640 1 -- 172.21.15.94:0/3798752626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb648102240 msgr2=0x7fb648102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:36.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.798+0000 7fb64cb9d640 1 --2- 172.21.15.94:0/3798752626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb648102240 0x7fb648102610 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb6380048d0 tx=0x7fb638009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:36.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.798+0000 7fb64cb9d640 1 -- 172.21.15.94:0/3798752626 shutdown_connections 2024-03-24T10:57:36.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.798+0000 7fb64cb9d640 1 --2- 172.21.15.94:0/3798752626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6480643a0 0x7fb6480647d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:36.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.798+0000 7fb64cb9d640 1 --2- 172.21.15.94:0/3798752626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb648102240 0x7fb648102610 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:36.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.799+0000 7fb64cb9d640 1 -- 172.21.15.94:0/3798752626 >> 172.21.15.94:0/3798752626 conn(0x7fb6480710b0 msgr2=0x7fb6480714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:36.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.799+0000 7fb64cb9d640 1 -- 172.21.15.94:0/3798752626 shutdown_connections 2024-03-24T10:57:36.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.799+0000 7fb64cb9d640 1 -- 172.21.15.94:0/3798752626 wait complete. 2024-03-24T10:57:36.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.799+0000 7fb64cb9d640 1 Processor -- start 2024-03-24T10:57:36.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.799+0000 7fb64cb9d640 1 -- start start 2024-03-24T10:57:36.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.799+0000 7fb64cb9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6480643a0 0x7fb648112d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:36.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.799+0000 7fb64cb9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb648102240 0x7fb648113250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:36.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.799+0000 7fb64cb9d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb648113850 con 0x7fb648102240 2024-03-24T10:57:36.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.799+0000 7fb64cb9d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb64810cea0 con 0x7fb6480643a0 2024-03-24T10:57:36.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.799+0000 7fb645d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb648102240 0x7fb648113250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:36.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.799+0000 7fb645d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb648102240 0x7fb648113250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51730/0 (socket says 172.21.15.94:51730) 2024-03-24T10:57:36.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.799+0000 7fb645d9b640 1 -- 172.21.15.94:0/3428028183 learned_addr learned my addr 172.21.15.94:0/3428028183 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:57:36.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.799+0000 7fb64659c640 1 --2- 172.21.15.94:0/3428028183 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6480643a0 0x7fb648112d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:36.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.800+0000 7fb645d9b640 1 -- 172.21.15.94:0/3428028183 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6480643a0 msgr2=0x7fb648112d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:36.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.800+0000 7fb645d9b640 1 --2- 172.21.15.94:0/3428028183 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6480643a0 0x7fb648112d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:36.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.800+0000 7fb645d9b640 1 -- 172.21.15.94:0/3428028183 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb638005040 con 0x7fb648102240 2024-03-24T10:57:36.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.800+0000 7fb645d9b640 1 --2- 172.21.15.94:0/3428028183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb648102240 0x7fb648113250 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb63c002a50 tx=0x7fb63c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:36.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.800+0000 7fb636ffd640 1 -- 172.21.15.94:0/3428028183 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb63c007c30 con 0x7fb648102240 2024-03-24T10:57:36.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.800+0000 7fb636ffd640 1 -- 172.21.15.94:0/3428028183 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb63c00f040 con 0x7fb648102240 2024-03-24T10:57:36.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.800+0000 7fb64cb9d640 1 -- 172.21.15.94:0/3428028183 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb64810d180 con 0x7fb648102240 2024-03-24T10:57:36.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.800+0000 7fb636ffd640 1 -- 172.21.15.94:0/3428028183 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb63c00dbe0 con 0x7fb648102240 2024-03-24T10:57:36.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.800+0000 7fb64cb9d640 1 -- 172.21.15.94:0/3428028183 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb64810d6a0 con 0x7fb648102240 2024-03-24T10:57:36.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.801+0000 7fb64cb9d640 1 -- 172.21.15.94:0/3428028183 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6480fd940 con 0x7fb648102240 2024-03-24T10:57:36.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.801+0000 7fb636ffd640 1 -- 172.21.15.94:0/3428028183 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fb63c00dd40 con 0x7fb648102240 2024-03-24T10:57:36.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.802+0000 7fb636ffd640 1 --2- 172.21.15.94:0/3428028183 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fb63006d750 0x7fb63006fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:36.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.802+0000 7fb636ffd640 1 -- 172.21.15.94:0/3428028183 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fb63c08cee0 con 0x7fb648102240 2024-03-24T10:57:36.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.802+0000 7fb64659c640 1 --2- 172.21.15.94:0/3428028183 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fb63006d750 0x7fb63006fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:36.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.802+0000 7fb64659c640 1 --2- 172.21.15.94:0/3428028183 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fb63006d750 0x7fb63006fc00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb638003080 tx=0x7fb638009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:36.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:36.807+0000 7fb636ffd640 1 -- 172.21.15.94:0/3428028183 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb63c056630 con 0x7fb648102240 2024-03-24T10:57:36.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:36 smithi094 ceph-mon[93957]: pgmap v308: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:57:36.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:36 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/346865450' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-03-24T10:57:36.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:36 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/119418255' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-03-24T10:57:36.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:36 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/3631815528' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-03-24T10:57:37.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.023+0000 7fb64cb9d640 1 -- 172.21.15.94:0/3428028183 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7fb64804bcc0 con 0x7fb648102240 2024-03-24T10:57:37.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.024+0000 7fb636ffd640 1 -- 172.21.15.94:0/3428028183 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v37) v1 ==== 107+0+4156 (secure 0 0 0) 0x7fb63c059cf0 con 0x7fb648102240 2024-03-24T10:57:37.024 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:57:37.024 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24261,"name":"cephfs.smithi096.jtebhh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.96:6827/1321087930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1321087930},{"type":"v1","addr":"172.21.15.96:6827","nonce":1321087930}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":34374,"name":"cephfs.smithi094.efiqte","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6827/2043632450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2043632450},{"type":"v1","addr":"172.21.15.94:6827","nonce":2043632450}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-24T10:43:39.435738+0000","modified":"2024-03-24T10:52:59.495272+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24251":{"gid":24251,"name":"cephfs.smithi096.egbmoq","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.96:6825/1539752208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1539752208},{"type":"v1","addr":"172.21.15.96:6825","nonce":1539752208}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24251,"qdb_cluster":[24251]},"id":1}]} 2024-03-24T10:57:37.024 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 23 2024-03-24T10:57:37.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.026+0000 7fb64cb9d640 1 -- 172.21.15.94:0/3428028183 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fb63006d750 msgr2=0x7fb63006fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:37.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.026+0000 7fb64cb9d640 1 --2- 172.21.15.94:0/3428028183 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fb63006d750 0x7fb63006fc00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb638003080 tx=0x7fb638009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:37.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.026+0000 7fb64cb9d640 1 -- 172.21.15.94:0/3428028183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb648102240 msgr2=0x7fb648113250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:37.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.026+0000 7fb64cb9d640 1 --2- 172.21.15.94:0/3428028183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb648102240 0x7fb648113250 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb63c002a50 tx=0x7fb63c004600 comp rx=0 tx=0).stop 2024-03-24T10:57:37.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.026+0000 7fb64cb9d640 1 -- 172.21.15.94:0/3428028183 shutdown_connections 2024-03-24T10:57:37.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.026+0000 7fb64cb9d640 1 --2- 172.21.15.94:0/3428028183 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fb63006d750 0x7fb63006fc00 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:37.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.026+0000 7fb64cb9d640 1 --2- 172.21.15.94:0/3428028183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb648102240 0x7fb648113250 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:37.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.026+0000 7fb64cb9d640 1 --2- 172.21.15.94:0/3428028183 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6480643a0 0x7fb648112d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:37.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.026+0000 7fb64cb9d640 1 -- 172.21.15.94:0/3428028183 >> 172.21.15.94:0/3428028183 conn(0x7fb6480710b0 msgr2=0x7fb6480f9520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:37.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.026+0000 7fb64cb9d640 1 -- 172.21.15.94:0/3428028183 shutdown_connections 2024-03-24T10:57:37.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.026+0000 7fb64cb9d640 1 -- 172.21.15.94:0/3428028183 wait complete. 2024-03-24T10:57:37.037 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 23 2024-03-24T10:57:37.037 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 24 2024-03-24T10:57:37.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:36 smithi096 ceph-mon[84258]: pgmap v308: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:57:37.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:36 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/346865450' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-03-24T10:57:37.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:36 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/119418255' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-03-24T10:57:37.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:36 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/3631815528' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-03-24T10:57:37.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.136+0000 7f9c16b1d640 1 -- 172.21.15.94:0/1747096474 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c100fdd30 msgr2=0x7f9c10106490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:37.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.136+0000 7f9c16b1d640 1 --2- 172.21.15.94:0/1747096474 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c100fdd30 0x7f9c10106490 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc0048d0 tx=0x7f9bfc009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:37.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.136+0000 7f9c16b1d640 1 -- 172.21.15.94:0/1747096474 shutdown_connections 2024-03-24T10:57:37.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.136+0000 7f9c16b1d640 1 --2- 172.21.15.94:0/1747096474 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c100fdd30 0x7f9c10106490 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:37.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.136+0000 7f9c16b1d640 1 --2- 172.21.15.94:0/1747096474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c100fd420 0x7f9c100fd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:37.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.136+0000 7f9c16b1d640 1 -- 172.21.15.94:0/1747096474 >> 172.21.15.94:0/1747096474 conn(0x7f9c100f6ea0 msgr2=0x7f9c100f92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:37.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.136+0000 7f9c16b1d640 1 -- 172.21.15.94:0/1747096474 shutdown_connections 2024-03-24T10:57:37.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.136+0000 7f9c16b1d640 1 -- 172.21.15.94:0/1747096474 wait complete. 2024-03-24T10:57:37.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.137+0000 7f9c16b1d640 1 Processor -- start 2024-03-24T10:57:37.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.137+0000 7f9c16b1d640 1 -- start start 2024-03-24T10:57:37.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.137+0000 7f9c16b1d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c100fd420 0x7f9c10112f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:37.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.137+0000 7f9c16b1d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c100fdd30 0x7f9c101134c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:37.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.137+0000 7f9c16b1d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c10113ac0 con 0x7f9c100fdd30 2024-03-24T10:57:37.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.137+0000 7f9c16b1d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1010d110 con 0x7f9c100fd420 2024-03-24T10:57:37.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.137+0000 7f9c0ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c100fdd30 0x7f9c101134c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:37.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.137+0000 7f9c0ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c100fdd30 0x7f9c101134c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51744/0 (socket says 172.21.15.94:51744) 2024-03-24T10:57:37.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.137+0000 7f9c0ffff640 1 -- 172.21.15.94:0/2834049859 learned_addr learned my addr 172.21.15.94:0/2834049859 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:57:37.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.137+0000 7f9c148b9640 1 --2- 172.21.15.94:0/2834049859 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c100fd420 0x7f9c10112f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:37.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.137+0000 7f9c0ffff640 1 -- 172.21.15.94:0/2834049859 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c100fd420 msgr2=0x7f9c10112f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:37.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.137+0000 7f9c0ffff640 1 --2- 172.21.15.94:0/2834049859 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c100fd420 0x7f9c10112f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:37.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.137+0000 7f9c0ffff640 1 -- 172.21.15.94:0/2834049859 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bfc005040 con 0x7f9c100fdd30 2024-03-24T10:57:37.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.138+0000 7f9c0ffff640 1 --2- 172.21.15.94:0/2834049859 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c100fdd30 0x7f9c101134c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc004930 tx=0x7f9bfc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:37.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.138+0000 7f9c0d7fa640 1 -- 172.21.15.94:0/2834049859 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bfc0092b0 con 0x7f9c100fdd30 2024-03-24T10:57:37.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.138+0000 7f9c0d7fa640 1 -- 172.21.15.94:0/2834049859 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9bfc03b400 con 0x7f9c100fdd30 2024-03-24T10:57:37.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.138+0000 7f9c16b1d640 1 -- 172.21.15.94:0/2834049859 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c1010d390 con 0x7f9c100fdd30 2024-03-24T10:57:37.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.138+0000 7f9c0d7fa640 1 -- 172.21.15.94:0/2834049859 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bfc0435b0 con 0x7f9c100fdd30 2024-03-24T10:57:37.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.138+0000 7f9c16b1d640 1 -- 172.21.15.94:0/2834049859 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c1010d850 con 0x7f9c100fdd30 2024-03-24T10:57:37.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.139+0000 7f9c16b1d640 1 -- 172.21.15.94:0/2834049859 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c10065d10 con 0x7f9c100fdd30 2024-03-24T10:57:37.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.139+0000 7f9c0d7fa640 1 -- 172.21.15.94:0/2834049859 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f9bfc02f070 con 0x7f9c100fdd30 2024-03-24T10:57:37.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.140+0000 7f9c0d7fa640 1 --2- 172.21.15.94:0/2834049859 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f9c0006d750 0x7f9c0006fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:37.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.140+0000 7f9c148b9640 1 --2- 172.21.15.94:0/2834049859 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f9c0006d750 0x7f9c0006fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:37.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.140+0000 7f9c0d7fa640 1 -- 172.21.15.94:0/2834049859 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f9bfc0aed10 con 0x7f9c100fdd30 2024-03-24T10:57:37.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.140+0000 7f9c148b9640 1 --2- 172.21.15.94:0/2834049859 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f9c0006d750 0x7f9c0006fc00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9c04007fd0 tx=0x7f9c04009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:37.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.144+0000 7f9c0d7fa640 1 -- 172.21.15.94:0/2834049859 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9bfc078590 con 0x7f9c100fdd30 2024-03-24T10:57:37.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.385+0000 7f9c16b1d640 1 -- 172.21.15.94:0/2834049859 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7f9c1004bcc0 con 0x7f9c100fdd30 2024-03-24T10:57:37.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.386+0000 7f9c0d7fa640 1 -- 172.21.15.94:0/2834049859 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v37) v1 ==== 107+0+5003 (secure 0 0 0) 0x7f9bfc076e60 con 0x7f9c100fdd30 2024-03-24T10:57:37.386 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:57:37.386 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24261,"name":"cephfs.smithi096.jtebhh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.96:6827/1321087930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1321087930},{"type":"v1","addr":"172.21.15.96:6827","nonce":1321087930}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":34374,"name":"cephfs.smithi094.efiqte","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6827/2043632450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2043632450},{"type":"v1","addr":"172.21.15.94:6827","nonce":2043632450}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":34406,"name":"cephfs.smithi094.llefgu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6829/2396893131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2396893131},{"type":"v1","addr":"172.21.15.94:6829","nonce":2396893131}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-24T10:43:39.435738+0000","modified":"2024-03-24T10:52:59.495272+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24251":{"gid":24251,"name":"cephfs.smithi096.egbmoq","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.96:6825/1539752208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1539752208},{"type":"v1","addr":"172.21.15.96:6825","nonce":1539752208}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24251,"qdb_cluster":[24251]},"id":1}]} 2024-03-24T10:57:37.386 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 24 2024-03-24T10:57:37.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.388+0000 7f9c16b1d640 1 -- 172.21.15.94:0/2834049859 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f9c0006d750 msgr2=0x7f9c0006fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:37.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.388+0000 7f9c16b1d640 1 --2- 172.21.15.94:0/2834049859 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f9c0006d750 0x7f9c0006fc00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9c04007fd0 tx=0x7f9c04009040 comp rx=0 tx=0).stop 2024-03-24T10:57:37.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.388+0000 7f9c16b1d640 1 -- 172.21.15.94:0/2834049859 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c100fdd30 msgr2=0x7f9c101134c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:37.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.388+0000 7f9c16b1d640 1 --2- 172.21.15.94:0/2834049859 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c100fdd30 0x7f9c101134c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc004930 tx=0x7f9bfc009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:37.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.388+0000 7f9c16b1d640 1 -- 172.21.15.94:0/2834049859 shutdown_connections 2024-03-24T10:57:37.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.388+0000 7f9c16b1d640 1 --2- 172.21.15.94:0/2834049859 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f9c0006d750 0x7f9c0006fc00 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:37.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.388+0000 7f9c16b1d640 1 --2- 172.21.15.94:0/2834049859 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c100fdd30 0x7f9c101134c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:37.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.388+0000 7f9c16b1d640 1 --2- 172.21.15.94:0/2834049859 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c100fd420 0x7f9c10112f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:37.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.388+0000 7f9c16b1d640 1 -- 172.21.15.94:0/2834049859 >> 172.21.15.94:0/2834049859 conn(0x7f9c100f6ea0 msgr2=0x7f9c100f8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:37.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.388+0000 7f9c16b1d640 1 -- 172.21.15.94:0/2834049859 shutdown_connections 2024-03-24T10:57:37.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.388+0000 7f9c16b1d640 1 -- 172.21.15.94:0/2834049859 wait complete. 2024-03-24T10:57:37.399 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 24 2024-03-24T10:57:37.399 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 25 2024-03-24T10:57:37.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.501+0000 7f1e53aae640 1 -- 172.21.15.94:0/2564459785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e4c102b90 msgr2=0x7f1e4c0fc220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:37.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.501+0000 7f1e53aae640 1 --2- 172.21.15.94:0/2564459785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e4c102b90 0x7f1e4c0fc220 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c0048d0 tx=0x7f1e3c009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:37.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.501+0000 7f1e53aae640 1 -- 172.21.15.94:0/2564459785 shutdown_connections 2024-03-24T10:57:37.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.501+0000 7f1e53aae640 1 --2- 172.21.15.94:0/2564459785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e4c102b90 0x7f1e4c0fc220 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:37.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.501+0000 7f1e53aae640 1 --2- 172.21.15.94:0/2564459785 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e4c1021f0 0x7f1e4c1025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:37.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.501+0000 7f1e53aae640 1 -- 172.21.15.94:0/2564459785 >> 172.21.15.94:0/2564459785 conn(0x7f1e4c0f7dc0 msgr2=0x7f1e4c0fa1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:37.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.501+0000 7f1e53aae640 1 -- 172.21.15.94:0/2564459785 shutdown_connections 2024-03-24T10:57:37.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.501+0000 7f1e53aae640 1 -- 172.21.15.94:0/2564459785 wait complete. 2024-03-24T10:57:37.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.502+0000 7f1e53aae640 1 Processor -- start 2024-03-24T10:57:37.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.502+0000 7f1e53aae640 1 -- start start 2024-03-24T10:57:37.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.502+0000 7f1e53aae640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e4c1021f0 0x7f1e4c112cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:37.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.502+0000 7f1e53aae640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e4c102b90 0x7f1e4c113230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:37.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.502+0000 7f1e53aae640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4c113830 con 0x7f1e4c102b90 2024-03-24T10:57:37.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.502+0000 7f1e53aae640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4c10ce80 con 0x7f1e4c1021f0 2024-03-24T10:57:37.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.502+0000 7f1e51049640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e4c102b90 0x7f1e4c113230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:37.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.502+0000 7f1e51049640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e4c102b90 0x7f1e4c113230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51766/0 (socket says 172.21.15.94:51766) 2024-03-24T10:57:37.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.502+0000 7f1e51049640 1 -- 172.21.15.94:0/3842862066 learned_addr learned my addr 172.21.15.94:0/3842862066 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:57:37.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.502+0000 7f1e5184a640 1 --2- 172.21.15.94:0/3842862066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e4c1021f0 0x7f1e4c112cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:37.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.502+0000 7f1e51049640 1 -- 172.21.15.94:0/3842862066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e4c1021f0 msgr2=0x7f1e4c112cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:37.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.502+0000 7f1e51049640 1 --2- 172.21.15.94:0/3842862066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e4c1021f0 0x7f1e4c112cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:37.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.502+0000 7f1e51049640 1 -- 172.21.15.94:0/3842862066 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e3c005040 con 0x7f1e4c102b90 2024-03-24T10:57:37.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.502+0000 7f1e5184a640 1 --2- 172.21.15.94:0/3842862066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e4c1021f0 0x7f1e4c112cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T10:57:37.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.502+0000 7f1e51049640 1 --2- 172.21.15.94:0/3842862066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e4c102b90 0x7f1e4c113230 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c004930 tx=0x7f1e3c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:37.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.503+0000 7f1e427fc640 1 -- 172.21.15.94:0/3842862066 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e3c0094c0 con 0x7f1e4c102b90 2024-03-24T10:57:37.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.503+0000 7f1e427fc640 1 -- 172.21.15.94:0/3842862066 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1e3c032e50 con 0x7f1e4c102b90 2024-03-24T10:57:37.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.503+0000 7f1e53aae640 1 -- 172.21.15.94:0/3842862066 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e4c10d100 con 0x7f1e4c102b90 2024-03-24T10:57:37.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.503+0000 7f1e427fc640 1 -- 172.21.15.94:0/3842862066 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e3c0433b0 con 0x7f1e4c102b90 2024-03-24T10:57:37.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.503+0000 7f1e53aae640 1 -- 172.21.15.94:0/3842862066 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e4c10d5c0 con 0x7f1e4c102b90 2024-03-24T10:57:37.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.504+0000 7f1e53aae640 1 -- 172.21.15.94:0/3842862066 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e4c0fd960 con 0x7f1e4c102b90 2024-03-24T10:57:37.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.504+0000 7f1e427fc640 1 -- 172.21.15.94:0/3842862066 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f1e3c02f070 con 0x7f1e4c102b90 2024-03-24T10:57:37.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.504+0000 7f1e427fc640 1 --2- 172.21.15.94:0/3842862066 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1e3806d750 0x7f1e3806fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:37.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.504+0000 7f1e427fc640 1 -- 172.21.15.94:0/3842862066 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f1e3c0aef50 con 0x7f1e4c102b90 2024-03-24T10:57:37.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.504+0000 7f1e5184a640 1 --2- 172.21.15.94:0/3842862066 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1e3806d750 0x7f1e3806fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:37.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.505+0000 7f1e5184a640 1 --2- 172.21.15.94:0/3842862066 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1e3806d750 0x7f1e3806fc00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1e48004c60 tx=0x7f1e48009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:37.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.509+0000 7f1e427fc640 1 -- 172.21.15.94:0/3842862066 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1e3c0786a0 con 0x7f1e4c102b90 2024-03-24T10:57:37.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.750+0000 7f1e53aae640 1 -- 172.21.15.94:0/3842862066 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7f1e4c109330 con 0x7f1e4c102b90 2024-03-24T10:57:37.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.751+0000 7f1e427fc640 1 -- 172.21.15.94:0/3842862066 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v37) v1 ==== 107+0+4205 (secure 0 0 0) 0x7f1e3c07bd60 con 0x7f1e4c102b90 2024-03-24T10:57:37.751 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:57:37.751 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24261,"name":"cephfs.smithi096.jtebhh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.96:6827/1321087930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1321087930},{"type":"v1","addr":"172.21.15.96:6827","nonce":1321087930}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":34374,"name":"cephfs.smithi094.efiqte","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6827/2043632450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2043632450},{"type":"v1","addr":"172.21.15.94:6827","nonce":2043632450}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":34406,"name":"cephfs.smithi094.llefgu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6829/2396893131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2396893131},{"type":"v1","addr":"172.21.15.94:6829","nonce":2396893131}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-24T10:43:39.435738+0000","modified":"2024-03-24T10:53:38.029679+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-24T10:57:37.751 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 25 2024-03-24T10:57:37.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.754+0000 7f1e53aae640 1 -- 172.21.15.94:0/3842862066 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1e3806d750 msgr2=0x7f1e3806fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:37.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.754+0000 7f1e53aae640 1 --2- 172.21.15.94:0/3842862066 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1e3806d750 0x7f1e3806fc00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1e48004c60 tx=0x7f1e48009040 comp rx=0 tx=0).stop 2024-03-24T10:57:37.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.754+0000 7f1e53aae640 1 -- 172.21.15.94:0/3842862066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e4c102b90 msgr2=0x7f1e4c113230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:37.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.754+0000 7f1e53aae640 1 --2- 172.21.15.94:0/3842862066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e4c102b90 0x7f1e4c113230 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c004930 tx=0x7f1e3c009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:37.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.754+0000 7f1e53aae640 1 -- 172.21.15.94:0/3842862066 shutdown_connections 2024-03-24T10:57:37.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.754+0000 7f1e53aae640 1 --2- 172.21.15.94:0/3842862066 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1e3806d750 0x7f1e3806fc00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:37.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.754+0000 7f1e53aae640 1 --2- 172.21.15.94:0/3842862066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e4c102b90 0x7f1e4c113230 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:37.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.754+0000 7f1e53aae640 1 --2- 172.21.15.94:0/3842862066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e4c1021f0 0x7f1e4c112cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:37.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.754+0000 7f1e53aae640 1 -- 172.21.15.94:0/3842862066 >> 172.21.15.94:0/3842862066 conn(0x7f1e4c0f7dc0 msgr2=0x7f1e4c0f8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:37.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.754+0000 7f1e53aae640 1 -- 172.21.15.94:0/3842862066 shutdown_connections 2024-03-24T10:57:37.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.754+0000 7f1e53aae640 1 -- 172.21.15.94:0/3842862066 wait complete. 2024-03-24T10:57:37.765 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 25 2024-03-24T10:57:37.765 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 26 2024-03-24T10:57:37.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.885+0000 7efd2f8ae640 1 -- 172.21.15.94:0/1557575170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd280643a0 msgr2=0x7efd280647f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:37.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.885+0000 7efd2f8ae640 1 --2- 172.21.15.94:0/1557575170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd280643a0 0x7efd280647f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7efd180048d0 tx=0x7efd18009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:37.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.885+0000 7efd2f8ae640 1 -- 172.21.15.94:0/1557575170 shutdown_connections 2024-03-24T10:57:37.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.885+0000 7efd2f8ae640 1 --2- 172.21.15.94:0/1557575170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd280643a0 0x7efd280647f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:37.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.885+0000 7efd2f8ae640 1 --2- 172.21.15.94:0/1557575170 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd28102210 0x7efd281025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:37.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.885+0000 7efd2f8ae640 1 -- 172.21.15.94:0/1557575170 >> 172.21.15.94:0/1557575170 conn(0x7efd280710b0 msgr2=0x7efd280714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:37.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.885+0000 7efd2f8ae640 1 -- 172.21.15.94:0/1557575170 shutdown_connections 2024-03-24T10:57:37.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.885+0000 7efd2f8ae640 1 -- 172.21.15.94:0/1557575170 wait complete. 2024-03-24T10:57:37.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.885+0000 7efd2f8ae640 1 Processor -- start 2024-03-24T10:57:37.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.886+0000 7efd2f8ae640 1 -- start start 2024-03-24T10:57:37.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.886+0000 7efd2f8ae640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd280643a0 0x7efd28112c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:37.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.886+0000 7efd2f8ae640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd28102210 0x7efd281131d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:37.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.886+0000 7efd2f8ae640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd281137d0 con 0x7efd28102210 2024-03-24T10:57:37.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.886+0000 7efd2f8ae640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd2810ce20 con 0x7efd280643a0 2024-03-24T10:57:37.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.886+0000 7efd2ce49640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd28102210 0x7efd281131d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:37.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.886+0000 7efd2ce49640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd28102210 0x7efd281131d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51782/0 (socket says 172.21.15.94:51782) 2024-03-24T10:57:37.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.886+0000 7efd2ce49640 1 -- 172.21.15.94:0/3515863926 learned_addr learned my addr 172.21.15.94:0/3515863926 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:57:37.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.886+0000 7efd2d64a640 1 --2- 172.21.15.94:0/3515863926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd280643a0 0x7efd28112c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:37.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.886+0000 7efd2ce49640 1 -- 172.21.15.94:0/3515863926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd280643a0 msgr2=0x7efd28112c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:37.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.886+0000 7efd2ce49640 1 --2- 172.21.15.94:0/3515863926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd280643a0 0x7efd28112c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:37.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.886+0000 7efd2ce49640 1 -- 172.21.15.94:0/3515863926 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd18005040 con 0x7efd28102210 2024-03-24T10:57:37.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.886+0000 7efd2ce49640 1 --2- 172.21.15.94:0/3515863926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd28102210 0x7efd281131d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7efd18004930 tx=0x7efd18009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:37.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.887+0000 7efd1dffb640 1 -- 172.21.15.94:0/3515863926 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd180094c0 con 0x7efd28102210 2024-03-24T10:57:37.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.887+0000 7efd2f8ae640 1 -- 172.21.15.94:0/3515863926 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd2810d0a0 con 0x7efd28102210 2024-03-24T10:57:37.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.887+0000 7efd1dffb640 1 -- 172.21.15.94:0/3515863926 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efd18032e50 con 0x7efd28102210 2024-03-24T10:57:37.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.887+0000 7efd1dffb640 1 -- 172.21.15.94:0/3515863926 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd180433b0 con 0x7efd28102210 2024-03-24T10:57:37.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.887+0000 7efd2f8ae640 1 -- 172.21.15.94:0/3515863926 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd2810d560 con 0x7efd28102210 2024-03-24T10:57:37.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.888+0000 7efd2f8ae640 1 -- 172.21.15.94:0/3515863926 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd280fd910 con 0x7efd28102210 2024-03-24T10:57:37.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.889+0000 7efd1dffb640 1 -- 172.21.15.94:0/3515863926 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7efd1802f070 con 0x7efd28102210 2024-03-24T10:57:37.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.890+0000 7efd1dffb640 1 --2- 172.21.15.94:0/3515863926 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7efd1406d750 0x7efd1406fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:37.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.890+0000 7efd1dffb640 1 -- 172.21.15.94:0/3515863926 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7efd180af2b0 con 0x7efd28102210 2024-03-24T10:57:37.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.890+0000 7efd2d64a640 1 --2- 172.21.15.94:0/3515863926 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7efd1406d750 0x7efd1406fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:37.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.891+0000 7efd2d64a640 1 --2- 172.21.15.94:0/3515863926 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7efd1406d750 0x7efd1406fc00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efd24009fd0 tx=0x7efd24009e50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:37.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:37.894+0000 7efd1dffb640 1 -- 172.21.15.94:0/3515863926 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7efd18078a80 con 0x7efd28102210 2024-03-24T10:57:37.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:37 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/3428028183' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-03-24T10:57:37.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:37 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/2834049859' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-03-24T10:57:38.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:37 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/3428028183' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-03-24T10:57:38.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:37 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/2834049859' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-03-24T10:57:38.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.113+0000 7efd2f8ae640 1 -- 172.21.15.94:0/3515863926 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7efd28109340 con 0x7efd28102210 2024-03-24T10:57:38.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.113+0000 7efd1dffb640 1 -- 172.21.15.94:0/3515863926 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v37) v1 ==== 107+0+4216 (secure 0 0 0) 0x7efd1807c140 con 0x7efd28102210 2024-03-24T10:57:38.113 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:57:38.113 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34374,"name":"cephfs.smithi094.efiqte","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6827/2043632450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2043632450},{"type":"v1","addr":"172.21.15.94:6827","nonce":2043632450}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":34406,"name":"cephfs.smithi094.llefgu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6829/2396893131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2396893131},{"type":"v1","addr":"172.21.15.94:6829","nonce":2396893131}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-24T10:43:39.435738+0000","modified":"2024-03-24T10:53:38.034256+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24261":{"gid":24261,"name":"cephfs.smithi096.jtebhh","rank":0,"incarnation":26,"state":"up:replay","state_seq":2,"addr":"172.21.15.96:6827/1321087930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1321087930},{"type":"v1","addr":"172.21.15.96:6827","nonce":1321087930}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-24T10:57:38.113 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 26 2024-03-24T10:57:38.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.115+0000 7efd2f8ae640 1 -- 172.21.15.94:0/3515863926 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7efd1406d750 msgr2=0x7efd1406fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:38.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.115+0000 7efd2f8ae640 1 --2- 172.21.15.94:0/3515863926 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7efd1406d750 0x7efd1406fc00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efd24009fd0 tx=0x7efd24009e50 comp rx=0 tx=0).stop 2024-03-24T10:57:38.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.115+0000 7efd2f8ae640 1 -- 172.21.15.94:0/3515863926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd28102210 msgr2=0x7efd281131d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:38.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.115+0000 7efd2f8ae640 1 --2- 172.21.15.94:0/3515863926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd28102210 0x7efd281131d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7efd18004930 tx=0x7efd18009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:38.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.115+0000 7efd2f8ae640 1 -- 172.21.15.94:0/3515863926 shutdown_connections 2024-03-24T10:57:38.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.115+0000 7efd2f8ae640 1 --2- 172.21.15.94:0/3515863926 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7efd1406d750 0x7efd1406fc00 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:38.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.115+0000 7efd2f8ae640 1 --2- 172.21.15.94:0/3515863926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd28102210 0x7efd281131d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:38.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.115+0000 7efd2f8ae640 1 --2- 172.21.15.94:0/3515863926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd280643a0 0x7efd28112c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:38.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.115+0000 7efd2f8ae640 1 -- 172.21.15.94:0/3515863926 >> 172.21.15.94:0/3515863926 conn(0x7efd280710b0 msgr2=0x7efd280f9560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:38.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.115+0000 7efd2f8ae640 1 -- 172.21.15.94:0/3515863926 shutdown_connections 2024-03-24T10:57:38.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.115+0000 7efd2f8ae640 1 -- 172.21.15.94:0/3515863926 wait complete. 2024-03-24T10:57:38.128 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 26 2024-03-24T10:57:38.128 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 27 2024-03-24T10:57:38.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.244+0000 7fa49102e640 1 -- 172.21.15.94:0/1021517344 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa48c0fcf20 msgr2=0x7fa48c064370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:38.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.244+0000 7fa49102e640 1 --2- 172.21.15.94:0/1021517344 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa48c0fcf20 0x7fa48c064370 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa47c004900 tx=0x7fa47c009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:38.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.244+0000 7fa49102e640 1 -- 172.21.15.94:0/1021517344 shutdown_connections 2024-03-24T10:57:38.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.244+0000 7fa49102e640 1 --2- 172.21.15.94:0/1021517344 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa48c064940 0x7fa48c064d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:38.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.244+0000 7fa49102e640 1 --2- 172.21.15.94:0/1021517344 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa48c0fcf20 0x7fa48c064370 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:38.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.244+0000 7fa49102e640 1 -- 172.21.15.94:0/1021517344 >> 172.21.15.94:0/1021517344 conn(0x7fa48c071120 msgr2=0x7fa48c071520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:38.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.244+0000 7fa49102e640 1 -- 172.21.15.94:0/1021517344 shutdown_connections 2024-03-24T10:57:38.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.244+0000 7fa49102e640 1 -- 172.21.15.94:0/1021517344 wait complete. 2024-03-24T10:57:38.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.245+0000 7fa49102e640 1 Processor -- start 2024-03-24T10:57:38.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.245+0000 7fa49102e640 1 -- start start 2024-03-24T10:57:38.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.245+0000 7fa49102e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa48c064940 0x7fa48c114f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:38.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.245+0000 7fa49102e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa48c0fcf20 0x7fa48c115460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:38.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.245+0000 7fa49102e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c115a60 con 0x7fa48c064940 2024-03-24T10:57:38.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.245+0000 7fa49102e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c10f0b0 con 0x7fa48c0fcf20 2024-03-24T10:57:38.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.245+0000 7fa48ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa48c064940 0x7fa48c114f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:38.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.245+0000 7fa48ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa48c064940 0x7fa48c114f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51810/0 (socket says 172.21.15.94:51810) 2024-03-24T10:57:38.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.245+0000 7fa48ad9d640 1 -- 172.21.15.94:0/1874227547 learned_addr learned my addr 172.21.15.94:0/1874227547 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:57:38.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.245+0000 7fa48a59c640 1 --2- 172.21.15.94:0/1874227547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa48c0fcf20 0x7fa48c115460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:38.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.245+0000 7fa48ad9d640 1 -- 172.21.15.94:0/1874227547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa48c0fcf20 msgr2=0x7fa48c115460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:38.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.245+0000 7fa48ad9d640 1 --2- 172.21.15.94:0/1874227547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa48c0fcf20 0x7fa48c115460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:38.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.245+0000 7fa48ad9d640 1 -- 172.21.15.94:0/1874227547 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa47c005040 con 0x7fa48c064940 2024-03-24T10:57:38.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.246+0000 7fa48ad9d640 1 --2- 172.21.15.94:0/1874227547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa48c064940 0x7fa48c114f20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa47c000c00 tx=0x7fa47c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:38.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.246+0000 7fa4737fe640 1 -- 172.21.15.94:0/1874227547 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa47c0094c0 con 0x7fa48c064940 2024-03-24T10:57:38.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.246+0000 7fa4737fe640 1 -- 172.21.15.94:0/1874227547 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa47c03d920 con 0x7fa48c064940 2024-03-24T10:57:38.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.246+0000 7fa4737fe640 1 -- 172.21.15.94:0/1874227547 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa47c03b8c0 con 0x7fa48c064940 2024-03-24T10:57:38.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.246+0000 7fa49102e640 1 -- 172.21.15.94:0/1874227547 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa48c10f330 con 0x7fa48c064940 2024-03-24T10:57:38.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.246+0000 7fa49102e640 1 -- 172.21.15.94:0/1874227547 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa48c10f850 con 0x7fa48c064940 2024-03-24T10:57:38.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.247+0000 7fa49102e640 1 -- 172.21.15.94:0/1874227547 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa48c0fb690 con 0x7fa48c064940 2024-03-24T10:57:38.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.247+0000 7fa4737fe640 1 -- 172.21.15.94:0/1874227547 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fa47c03da90 con 0x7fa48c064940 2024-03-24T10:57:38.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.248+0000 7fa4737fe640 1 --2- 172.21.15.94:0/1874227547 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fa47406d750 0x7fa47406fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:38.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.248+0000 7fa4737fe640 1 -- 172.21.15.94:0/1874227547 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fa47c0af770 con 0x7fa48c064940 2024-03-24T10:57:38.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.248+0000 7fa48a59c640 1 --2- 172.21.15.94:0/1874227547 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fa47406d750 0x7fa47406fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:38.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.248+0000 7fa48a59c640 1 --2- 172.21.15.94:0/1874227547 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fa47406d750 0x7fa47406fc00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa48c1107e0 tx=0x7fa48000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:38.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.252+0000 7fa4737fe640 1 -- 172.21.15.94:0/1874227547 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa47c078ec0 con 0x7fa48c064940 2024-03-24T10:57:38.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.487+0000 7fa49102e640 1 -- 172.21.15.94:0/1874227547 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7fa48c110590 con 0x7fa48c064940 2024-03-24T10:57:38.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.487+0000 7fa4737fe640 1 -- 172.21.15.94:0/1874227547 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v37) v1 ==== 107+0+4221 (secure 0 0 0) 0x7fa47c001050 con 0x7fa48c064940 2024-03-24T10:57:38.487 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:57:38.487 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34374,"name":"cephfs.smithi094.efiqte","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6827/2043632450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2043632450},{"type":"v1","addr":"172.21.15.94:6827","nonce":2043632450}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":34406,"name":"cephfs.smithi094.llefgu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6829/2396893131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2396893131},{"type":"v1","addr":"172.21.15.94:6829","nonce":2396893131}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-24T10:43:39.435738+0000","modified":"2024-03-24T10:53:46.455552+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24261":{"gid":24261,"name":"cephfs.smithi096.jtebhh","rank":0,"incarnation":26,"state":"up:reconnect","state_seq":150,"addr":"172.21.15.96:6827/1321087930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1321087930},{"type":"v1","addr":"172.21.15.96:6827","nonce":1321087930}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-24T10:57:38.487 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 27 2024-03-24T10:57:38.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.489+0000 7fa49102e640 1 -- 172.21.15.94:0/1874227547 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fa47406d750 msgr2=0x7fa47406fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:38.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.489+0000 7fa49102e640 1 --2- 172.21.15.94:0/1874227547 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fa47406d750 0x7fa47406fc00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa48c1107e0 tx=0x7fa48000b040 comp rx=0 tx=0).stop 2024-03-24T10:57:38.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.489+0000 7fa49102e640 1 -- 172.21.15.94:0/1874227547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa48c064940 msgr2=0x7fa48c114f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:38.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.489+0000 7fa49102e640 1 --2- 172.21.15.94:0/1874227547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa48c064940 0x7fa48c114f20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa47c000c00 tx=0x7fa47c009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:38.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.489+0000 7fa49102e640 1 -- 172.21.15.94:0/1874227547 shutdown_connections 2024-03-24T10:57:38.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.489+0000 7fa49102e640 1 --2- 172.21.15.94:0/1874227547 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fa47406d750 0x7fa47406fc00 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:38.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.489+0000 7fa49102e640 1 --2- 172.21.15.94:0/1874227547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa48c0fcf20 0x7fa48c115460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:38.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.489+0000 7fa49102e640 1 --2- 172.21.15.94:0/1874227547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa48c064940 0x7fa48c114f20 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:38.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.489+0000 7fa49102e640 1 -- 172.21.15.94:0/1874227547 >> 172.21.15.94:0/1874227547 conn(0x7fa48c071120 msgr2=0x7fa48c0f90f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:38.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.490+0000 7fa49102e640 1 -- 172.21.15.94:0/1874227547 shutdown_connections 2024-03-24T10:57:38.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.490+0000 7fa49102e640 1 -- 172.21.15.94:0/1874227547 wait complete. 2024-03-24T10:57:38.501 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 27 2024-03-24T10:57:38.501 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 28 2024-03-24T10:57:38.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.601+0000 7f381fcae640 1 -- 172.21.15.94:0/287685610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38180fb110 msgr2=0x7f38180fb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:38.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.601+0000 7f381fcae640 1 --2- 172.21.15.94:0/287685610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38180fb110 0x7f38180fb560 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f38080048d0 tx=0x7f3808009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:38.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.601+0000 7f381fcae640 1 -- 172.21.15.94:0/287685610 shutdown_connections 2024-03-24T10:57:38.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.601+0000 7f381fcae640 1 --2- 172.21.15.94:0/287685610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38180fb110 0x7f38180fb560 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:38.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.601+0000 7f381fcae640 1 --2- 172.21.15.94:0/287685610 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3818101140 0x7f3818101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:38.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.601+0000 7f381fcae640 1 -- 172.21.15.94:0/287685610 >> 172.21.15.94:0/287685610 conn(0x7f38180710b0 msgr2=0x7f38180714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:38.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.601+0000 7f381fcae640 1 -- 172.21.15.94:0/287685610 shutdown_connections 2024-03-24T10:57:38.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.601+0000 7f381fcae640 1 -- 172.21.15.94:0/287685610 wait complete. 2024-03-24T10:57:38.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.602+0000 7f381fcae640 1 Processor -- start 2024-03-24T10:57:38.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.602+0000 7f381fcae640 1 -- start start 2024-03-24T10:57:38.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.602+0000 7f381fcae640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38180fb110 0x7f381810de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:38.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.602+0000 7f381fcae640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3818101140 0x7f381810e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:38.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.602+0000 7f381fcae640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3818065330 con 0x7f38180fb110 2024-03-24T10:57:38.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.602+0000 7f381fcae640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381810e940 con 0x7f3818101140 2024-03-24T10:57:38.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.602+0000 7f381da4a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38180fb110 0x7f381810de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:38.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.602+0000 7f381da4a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38180fb110 0x7f381810de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51834/0 (socket says 172.21.15.94:51834) 2024-03-24T10:57:38.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.602+0000 7f381da4a640 1 -- 172.21.15.94:0/503535041 learned_addr learned my addr 172.21.15.94:0/503535041 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:57:38.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.602+0000 7f381d249640 1 --2- 172.21.15.94:0/503535041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3818101140 0x7f381810e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:38.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.602+0000 7f381da4a640 1 -- 172.21.15.94:0/503535041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3818101140 msgr2=0x7f381810e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:38.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.602+0000 7f381da4a640 1 --2- 172.21.15.94:0/503535041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3818101140 0x7f381810e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:38.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.602+0000 7f381da4a640 1 -- 172.21.15.94:0/503535041 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3808005040 con 0x7f38180fb110 2024-03-24T10:57:38.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.603+0000 7f381da4a640 1 --2- 172.21.15.94:0/503535041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38180fb110 0x7f381810de90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3814002a50 tx=0x7f3814004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:38.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.603+0000 7f380e7fc640 1 -- 172.21.15.94:0/503535041 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3814007c30 con 0x7f38180fb110 2024-03-24T10:57:38.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.603+0000 7f380e7fc640 1 -- 172.21.15.94:0/503535041 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f381400f040 con 0x7f38180fb110 2024-03-24T10:57:38.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.603+0000 7f381fcae640 1 -- 172.21.15.94:0/503535041 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f381810ec20 con 0x7f38180fb110 2024-03-24T10:57:38.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.603+0000 7f380e7fc640 1 -- 172.21.15.94:0/503535041 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f381400dbe0 con 0x7f38180fb110 2024-03-24T10:57:38.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.603+0000 7f381fcae640 1 -- 172.21.15.94:0/503535041 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38181171b0 con 0x7f38180fb110 2024-03-24T10:57:38.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.604+0000 7f381fcae640 1 -- 172.21.15.94:0/503535041 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38180fc800 con 0x7f38180fb110 2024-03-24T10:57:38.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.604+0000 7f380e7fc640 1 -- 172.21.15.94:0/503535041 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f381400dd40 con 0x7f38180fb110 2024-03-24T10:57:38.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.605+0000 7f380e7fc640 1 --2- 172.21.15.94:0/503535041 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f380406d750 0x7f380406fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:38.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.605+0000 7f380e7fc640 1 -- 172.21.15.94:0/503535041 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f381408ce90 con 0x7f38180fb110 2024-03-24T10:57:38.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.605+0000 7f381d249640 1 --2- 172.21.15.94:0/503535041 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f380406d750 0x7f380406fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:38.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.605+0000 7f381d249640 1 --2- 172.21.15.94:0/503535041 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f380406d750 0x7f380406fc00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3808009fa0 tx=0x7f3808009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:38.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.610+0000 7f380e7fc640 1 -- 172.21.15.94:0/503535041 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3814056660 con 0x7f38180fb110 2024-03-24T10:57:38.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.865+0000 7f381fcae640 1 -- 172.21.15.94:0/503535041 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7f381804bcc0 con 0x7f38180fb110 2024-03-24T10:57:38.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.866+0000 7f380e7fc640 1 -- 172.21.15.94:0/503535041 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v37) v1 ==== 107+0+4218 (secure 0 0 0) 0x7f3814007d90 con 0x7f38180fb110 2024-03-24T10:57:38.866 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:57:38.866 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34374,"name":"cephfs.smithi094.efiqte","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6827/2043632450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2043632450},{"type":"v1","addr":"172.21.15.94:6827","nonce":2043632450}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":34406,"name":"cephfs.smithi094.llefgu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6829/2396893131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2396893131},{"type":"v1","addr":"172.21.15.94:6829","nonce":2396893131}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-24T10:43:39.435738+0000","modified":"2024-03-24T10:53:47.397232+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24261":{"gid":24261,"name":"cephfs.smithi096.jtebhh","rank":0,"incarnation":26,"state":"up:rejoin","state_seq":151,"addr":"172.21.15.96:6827/1321087930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1321087930},{"type":"v1","addr":"172.21.15.96:6827","nonce":1321087930}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-24T10:57:38.866 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 28 2024-03-24T10:57:38.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.868+0000 7f381fcae640 1 -- 172.21.15.94:0/503535041 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f380406d750 msgr2=0x7f380406fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:38.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.868+0000 7f381fcae640 1 --2- 172.21.15.94:0/503535041 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f380406d750 0x7f380406fc00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3808009fa0 tx=0x7f3808009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:38.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.868+0000 7f381fcae640 1 -- 172.21.15.94:0/503535041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38180fb110 msgr2=0x7f381810de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:38.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.868+0000 7f381fcae640 1 --2- 172.21.15.94:0/503535041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38180fb110 0x7f381810de90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3814002a50 tx=0x7f3814004600 comp rx=0 tx=0).stop 2024-03-24T10:57:38.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.868+0000 7f381fcae640 1 -- 172.21.15.94:0/503535041 shutdown_connections 2024-03-24T10:57:38.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.868+0000 7f381fcae640 1 --2- 172.21.15.94:0/503535041 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f380406d750 0x7f380406fc00 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:38.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.868+0000 7f381fcae640 1 --2- 172.21.15.94:0/503535041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3818101140 0x7f381810e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:38.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.868+0000 7f381fcae640 1 --2- 172.21.15.94:0/503535041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38180fb110 0x7f381810de90 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:38.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.868+0000 7f381fcae640 1 -- 172.21.15.94:0/503535041 >> 172.21.15.94:0/503535041 conn(0x7f38180710b0 msgr2=0x7f38180f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:38.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.868+0000 7f381fcae640 1 -- 172.21.15.94:0/503535041 shutdown_connections 2024-03-24T10:57:38.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.868+0000 7f381fcae640 1 -- 172.21.15.94:0/503535041 wait complete. 2024-03-24T10:57:38.877 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:38 smithi094 ceph-mon[93957]: pgmap v309: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:57:38.877 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:38 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/3842862066' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-03-24T10:57:38.877 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:38 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/3515863926' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-03-24T10:57:38.878 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:38 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/1874227547' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-03-24T10:57:38.879 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 28 2024-03-24T10:57:38.879 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 29 2024-03-24T10:57:38.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.978+0000 7f63b0b1d640 1 -- 172.21.15.94:0/951013711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63ac102240 msgr2=0x7f63ac102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:38.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.978+0000 7f63b0b1d640 1 --2- 172.21.15.94:0/951013711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63ac102240 0x7f63ac102610 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f639c0048d0 tx=0x7f639c009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:38.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.978+0000 7f63b0b1d640 1 -- 172.21.15.94:0/951013711 shutdown_connections 2024-03-24T10:57:38.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.978+0000 7f63b0b1d640 1 --2- 172.21.15.94:0/951013711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63ac0643a0 0x7f63ac0647d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:38.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.978+0000 7f63b0b1d640 1 --2- 172.21.15.94:0/951013711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63ac102240 0x7f63ac102610 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:38.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.978+0000 7f63b0b1d640 1 -- 172.21.15.94:0/951013711 >> 172.21.15.94:0/951013711 conn(0x7f63ac0710b0 msgr2=0x7f63ac0714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:38.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.978+0000 7f63b0b1d640 1 -- 172.21.15.94:0/951013711 shutdown_connections 2024-03-24T10:57:38.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.978+0000 7f63b0b1d640 1 -- 172.21.15.94:0/951013711 wait complete. 2024-03-24T10:57:38.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.979+0000 7f63b0b1d640 1 Processor -- start 2024-03-24T10:57:38.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.979+0000 7f63b0b1d640 1 -- start start 2024-03-24T10:57:38.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.979+0000 7f63b0b1d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63ac0643a0 0x7f63ac112cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:38.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.979+0000 7f63b0b1d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63ac102240 0x7f63ac113210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:38.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.979+0000 7f63b0b1d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac113810 con 0x7f63ac102240 2024-03-24T10:57:38.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.979+0000 7f63b0b1d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac10ce60 con 0x7f63ac0643a0 2024-03-24T10:57:38.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.979+0000 7f63a9d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63ac102240 0x7f63ac113210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:38.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.979+0000 7f63a9d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63ac102240 0x7f63ac113210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51846/0 (socket says 172.21.15.94:51846) 2024-03-24T10:57:38.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.979+0000 7f63a9d9b640 1 -- 172.21.15.94:0/382186340 learned_addr learned my addr 172.21.15.94:0/382186340 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:57:38.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.980+0000 7f63aa59c640 1 --2- 172.21.15.94:0/382186340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63ac0643a0 0x7f63ac112cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:38.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.980+0000 7f63a9d9b640 1 -- 172.21.15.94:0/382186340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63ac0643a0 msgr2=0x7f63ac112cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:38.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.980+0000 7f63a9d9b640 1 --2- 172.21.15.94:0/382186340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63ac0643a0 0x7f63ac112cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:38.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.980+0000 7f63a9d9b640 1 -- 172.21.15.94:0/382186340 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f639c005040 con 0x7f63ac102240 2024-03-24T10:57:38.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.980+0000 7f63aa59c640 1 --2- 172.21.15.94:0/382186340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63ac0643a0 0x7f63ac112cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T10:57:38.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.980+0000 7f63a9d9b640 1 --2- 172.21.15.94:0/382186340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63ac102240 0x7f63ac113210 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f63a0002a50 tx=0x7f63a0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:38.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.980+0000 7f639affd640 1 -- 172.21.15.94:0/382186340 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63a0007c30 con 0x7f63ac102240 2024-03-24T10:57:38.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.980+0000 7f639affd640 1 -- 172.21.15.94:0/382186340 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f63a000f040 con 0x7f63ac102240 2024-03-24T10:57:38.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.980+0000 7f63b0b1d640 1 -- 172.21.15.94:0/382186340 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63ac10d140 con 0x7f63ac102240 2024-03-24T10:57:38.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.980+0000 7f639affd640 1 -- 172.21.15.94:0/382186340 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63a000db50 con 0x7f63ac102240 2024-03-24T10:57:38.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.980+0000 7f63b0b1d640 1 -- 172.21.15.94:0/382186340 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63ac10d660 con 0x7f63ac102240 2024-03-24T10:57:38.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.981+0000 7f63b0b1d640 1 -- 172.21.15.94:0/382186340 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63ac0fd940 con 0x7f63ac102240 2024-03-24T10:57:38.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.981+0000 7f639affd640 1 -- 172.21.15.94:0/382186340 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f63a001c400 con 0x7f63ac102240 2024-03-24T10:57:38.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.982+0000 7f639affd640 1 --2- 172.21.15.94:0/382186340 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f639406d760 0x7f639406fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:38.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.982+0000 7f63aa59c640 1 --2- 172.21.15.94:0/382186340 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f639406d760 0x7f639406fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:38.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.982+0000 7f639affd640 1 -- 172.21.15.94:0/382186340 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f63a008c8a0 con 0x7f63ac102240 2024-03-24T10:57:38.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.982+0000 7f63aa59c640 1 --2- 172.21.15.94:0/382186340 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f639406d760 0x7f639406fc10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f639c000c00 tx=0x7f639c009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:38.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:38.986+0000 7f639affd640 1 -- 172.21.15.94:0/382186340 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f63a0055ff0 con 0x7f63ac102240 2024-03-24T10:57:39.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:38 smithi096 ceph-mon[84258]: pgmap v309: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:57:39.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:38 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/3842862066' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-03-24T10:57:39.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:38 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/3515863926' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-03-24T10:57:39.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:38 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/1874227547' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-03-24T10:57:39.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.205+0000 7f63b0b1d640 1 -- 172.21.15.94:0/382186340 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7f63ac109350 con 0x7f63ac102240 2024-03-24T10:57:39.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.205+0000 7f639affd640 1 -- 172.21.15.94:0/382186340 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v37) v1 ==== 107+0+4227 (secure 0 0 0) 0x7f63a00596b0 con 0x7f63ac102240 2024-03-24T10:57:39.205 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:57:39.205 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34374,"name":"cephfs.smithi094.efiqte","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6827/2043632450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2043632450},{"type":"v1","addr":"172.21.15.94:6827","nonce":2043632450}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":34406,"name":"cephfs.smithi094.llefgu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6829/2396893131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2396893131},{"type":"v1","addr":"172.21.15.94:6829","nonce":2396893131}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-24T10:43:39.435738+0000","modified":"2024-03-24T10:53:49.399665+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24261":{"gid":24261,"name":"cephfs.smithi096.jtebhh","rank":0,"incarnation":26,"state":"up:active","state_seq":152,"addr":"172.21.15.96:6827/1321087930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1321087930},{"type":"v1","addr":"172.21.15.96:6827","nonce":1321087930}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24261,"qdb_cluster":[24261]},"id":1}]} 2024-03-24T10:57:39.206 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 29 2024-03-24T10:57:39.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.207+0000 7f63b0b1d640 1 -- 172.21.15.94:0/382186340 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f639406d760 msgr2=0x7f639406fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:39.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.207+0000 7f63b0b1d640 1 --2- 172.21.15.94:0/382186340 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f639406d760 0x7f639406fc10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f639c000c00 tx=0x7f639c009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:39.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.207+0000 7f63b0b1d640 1 -- 172.21.15.94:0/382186340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63ac102240 msgr2=0x7f63ac113210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:39.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.207+0000 7f63b0b1d640 1 --2- 172.21.15.94:0/382186340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63ac102240 0x7f63ac113210 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f63a0002a50 tx=0x7f63a0004600 comp rx=0 tx=0).stop 2024-03-24T10:57:39.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.207+0000 7f63b0b1d640 1 -- 172.21.15.94:0/382186340 shutdown_connections 2024-03-24T10:57:39.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.207+0000 7f63b0b1d640 1 --2- 172.21.15.94:0/382186340 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f639406d760 0x7f639406fc10 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:39.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.207+0000 7f63b0b1d640 1 --2- 172.21.15.94:0/382186340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63ac102240 0x7f63ac113210 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:39.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.207+0000 7f63b0b1d640 1 --2- 172.21.15.94:0/382186340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63ac0643a0 0x7f63ac112cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:39.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.207+0000 7f63b0b1d640 1 -- 172.21.15.94:0/382186340 >> 172.21.15.94:0/382186340 conn(0x7f63ac0710b0 msgr2=0x7f63ac0f9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:39.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.208+0000 7f63b0b1d640 1 -- 172.21.15.94:0/382186340 shutdown_connections 2024-03-24T10:57:39.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.208+0000 7f63b0b1d640 1 -- 172.21.15.94:0/382186340 wait complete. 2024-03-24T10:57:39.218 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 29 2024-03-24T10:57:39.218 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 30 2024-03-24T10:57:39.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.320+0000 7fb21df5d640 1 -- 172.21.15.94:0/1997888536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb218102240 msgr2=0x7fb218102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:39.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.320+0000 7fb21df5d640 1 --2- 172.21.15.94:0/1997888536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb218102240 0x7fb218102610 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb2080048d0 tx=0x7fb208009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:39.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.320+0000 7fb21df5d640 1 -- 172.21.15.94:0/1997888536 shutdown_connections 2024-03-24T10:57:39.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.320+0000 7fb21df5d640 1 --2- 172.21.15.94:0/1997888536 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2180643a0 0x7fb2180647d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:39.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.320+0000 7fb21df5d640 1 --2- 172.21.15.94:0/1997888536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb218102240 0x7fb218102610 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:39.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.320+0000 7fb21df5d640 1 -- 172.21.15.94:0/1997888536 >> 172.21.15.94:0/1997888536 conn(0x7fb2180710b0 msgr2=0x7fb2180714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:39.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.320+0000 7fb21df5d640 1 -- 172.21.15.94:0/1997888536 shutdown_connections 2024-03-24T10:57:39.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.320+0000 7fb21df5d640 1 -- 172.21.15.94:0/1997888536 wait complete. 2024-03-24T10:57:39.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.321+0000 7fb21df5d640 1 Processor -- start 2024-03-24T10:57:39.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.321+0000 7fb21df5d640 1 -- start start 2024-03-24T10:57:39.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.321+0000 7fb21df5d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2180643a0 0x7fb218112d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:39.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.321+0000 7fb21df5d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb218102240 0x7fb218113250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:39.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.321+0000 7fb21df5d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb218113850 con 0x7fb2180643a0 2024-03-24T10:57:39.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.321+0000 7fb21df5d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb21810cea0 con 0x7fb218102240 2024-03-24T10:57:39.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.321+0000 7fb2177fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2180643a0 0x7fb218112d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:39.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.321+0000 7fb2177fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2180643a0 0x7fb218112d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51862/0 (socket says 172.21.15.94:51862) 2024-03-24T10:57:39.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.321+0000 7fb2177fe640 1 -- 172.21.15.94:0/3271523562 learned_addr learned my addr 172.21.15.94:0/3271523562 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:57:39.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.321+0000 7fb216ffd640 1 --2- 172.21.15.94:0/3271523562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb218102240 0x7fb218113250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:39.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.321+0000 7fb2177fe640 1 -- 172.21.15.94:0/3271523562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb218102240 msgr2=0x7fb218113250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:39.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.321+0000 7fb2177fe640 1 --2- 172.21.15.94:0/3271523562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb218102240 0x7fb218113250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:39.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.321+0000 7fb2177fe640 1 -- 172.21.15.94:0/3271523562 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb208005040 con 0x7fb2180643a0 2024-03-24T10:57:39.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.321+0000 7fb2177fe640 1 --2- 172.21.15.94:0/3271523562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2180643a0 0x7fb218112d10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb2080047f0 tx=0x7fb208009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:39.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.322+0000 7fb1fffff640 1 -- 172.21.15.94:0/3271523562 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2080094c0 con 0x7fb2180643a0 2024-03-24T10:57:39.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.322+0000 7fb1fffff640 1 -- 172.21.15.94:0/3271523562 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb208032e50 con 0x7fb2180643a0 2024-03-24T10:57:39.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.322+0000 7fb21df5d640 1 -- 172.21.15.94:0/3271523562 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb21810d120 con 0x7fb2180643a0 2024-03-24T10:57:39.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.322+0000 7fb1fffff640 1 -- 172.21.15.94:0/3271523562 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2080435f0 con 0x7fb2180643a0 2024-03-24T10:57:39.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.322+0000 7fb21df5d640 1 -- 172.21.15.94:0/3271523562 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb21810d5e0 con 0x7fb2180643a0 2024-03-24T10:57:39.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.323+0000 7fb21df5d640 1 -- 172.21.15.94:0/3271523562 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2180fd940 con 0x7fb2180643a0 2024-03-24T10:57:39.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.323+0000 7fb1fffff640 1 -- 172.21.15.94:0/3271523562 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fb20802f070 con 0x7fb2180643a0 2024-03-24T10:57:39.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.324+0000 7fb1fffff640 1 --2- 172.21.15.94:0/3271523562 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fb20006d760 0x7fb20006fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:39.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.324+0000 7fb216ffd640 1 --2- 172.21.15.94:0/3271523562 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fb20006d760 0x7fb20006fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:39.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.324+0000 7fb1fffff640 1 -- 172.21.15.94:0/3271523562 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fb2080aed10 con 0x7fb2180643a0 2024-03-24T10:57:39.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.324+0000 7fb216ffd640 1 --2- 172.21.15.94:0/3271523562 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fb20006d760 0x7fb20006fc10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb21810e5d0 tx=0x7fb20c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:39.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.328+0000 7fb1fffff640 1 -- 172.21.15.94:0/3271523562 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb208078510 con 0x7fb2180643a0 2024-03-24T10:57:39.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.547+0000 7fb21df5d640 1 -- 172.21.15.94:0/3271523562 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7fb21810e380 con 0x7fb2180643a0 2024-03-24T10:57:39.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.548+0000 7fb1fffff640 1 -- 172.21.15.94:0/3271523562 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v37) v1 ==== 107+0+5074 (secure 0 0 0) 0x7fb20807bbd0 con 0x7fb2180643a0 2024-03-24T10:57:39.548 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:57:39.548 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34374,"name":"cephfs.smithi094.efiqte","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6827/2043632450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2043632450},{"type":"v1","addr":"172.21.15.94:6827","nonce":2043632450}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":34406,"name":"cephfs.smithi094.llefgu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6829/2396893131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2396893131},{"type":"v1","addr":"172.21.15.94:6829","nonce":2396893131}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44321,"name":"cephfs.smithi096.egbmoq","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6825/2694461318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2694461318},{"type":"v1","addr":"172.21.15.96:6825","nonce":2694461318}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-24T10:43:39.435738+0000","modified":"2024-03-24T10:53:49.399665+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24261":{"gid":24261,"name":"cephfs.smithi096.jtebhh","rank":0,"incarnation":26,"state":"up:active","state_seq":152,"addr":"172.21.15.96:6827/1321087930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1321087930},{"type":"v1","addr":"172.21.15.96:6827","nonce":1321087930}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24261,"qdb_cluster":[24261]},"id":1}]} 2024-03-24T10:57:39.548 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 30 2024-03-24T10:57:39.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.550+0000 7fb21df5d640 1 -- 172.21.15.94:0/3271523562 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fb20006d760 msgr2=0x7fb20006fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:39.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.550+0000 7fb21df5d640 1 --2- 172.21.15.94:0/3271523562 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fb20006d760 0x7fb20006fc10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb21810e5d0 tx=0x7fb20c009040 comp rx=0 tx=0).stop 2024-03-24T10:57:39.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.550+0000 7fb21df5d640 1 -- 172.21.15.94:0/3271523562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2180643a0 msgr2=0x7fb218112d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:39.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.550+0000 7fb21df5d640 1 --2- 172.21.15.94:0/3271523562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2180643a0 0x7fb218112d10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb2080047f0 tx=0x7fb208009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:39.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.550+0000 7fb21df5d640 1 -- 172.21.15.94:0/3271523562 shutdown_connections 2024-03-24T10:57:39.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.550+0000 7fb21df5d640 1 --2- 172.21.15.94:0/3271523562 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fb20006d760 0x7fb20006fc10 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:39.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.550+0000 7fb21df5d640 1 --2- 172.21.15.94:0/3271523562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb218102240 0x7fb218113250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:39.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.550+0000 7fb21df5d640 1 --2- 172.21.15.94:0/3271523562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2180643a0 0x7fb218112d10 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:39.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.550+0000 7fb21df5d640 1 -- 172.21.15.94:0/3271523562 >> 172.21.15.94:0/3271523562 conn(0x7fb2180710b0 msgr2=0x7fb2180f9520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:39.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.550+0000 7fb21df5d640 1 -- 172.21.15.94:0/3271523562 shutdown_connections 2024-03-24T10:57:39.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.551+0000 7fb21df5d640 1 -- 172.21.15.94:0/3271523562 wait complete. 2024-03-24T10:57:39.561 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 30 2024-03-24T10:57:39.561 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 31 2024-03-24T10:57:39.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.669+0000 7f366a55d640 1 -- 172.21.15.94:0/683165709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36640643a0 msgr2=0x7f36640647d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:39.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.669+0000 7f366a55d640 1 --2- 172.21.15.94:0/683165709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36640643a0 0x7f36640647d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f36580048d0 tx=0x7f3658009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:39.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.669+0000 7f366a55d640 1 -- 172.21.15.94:0/683165709 shutdown_connections 2024-03-24T10:57:39.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.669+0000 7f366a55d640 1 --2- 172.21.15.94:0/683165709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36640643a0 0x7f36640647d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:39.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.669+0000 7f366a55d640 1 --2- 172.21.15.94:0/683165709 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3664102240 0x7f3664102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:39.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.669+0000 7f366a55d640 1 -- 172.21.15.94:0/683165709 >> 172.21.15.94:0/683165709 conn(0x7f36640710b0 msgr2=0x7f36640714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:39.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.669+0000 7f366a55d640 1 -- 172.21.15.94:0/683165709 shutdown_connections 2024-03-24T10:57:39.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.669+0000 7f366a55d640 1 -- 172.21.15.94:0/683165709 wait complete. 2024-03-24T10:57:39.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.670+0000 7f366a55d640 1 Processor -- start 2024-03-24T10:57:39.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.670+0000 7f366a55d640 1 -- start start 2024-03-24T10:57:39.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.670+0000 7f366a55d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36640643a0 0x7f36640fb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:39.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.670+0000 7f366a55d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3664102240 0x7f36640fb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:39.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.670+0000 7f366a55d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36640fd0c0 con 0x7f3664102240 2024-03-24T10:57:39.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.670+0000 7f366a55d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36640fbc60 con 0x7f36640643a0 2024-03-24T10:57:39.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.670+0000 7f36637fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3664102240 0x7f36640fb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:39.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.670+0000 7f36637fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3664102240 0x7f36640fb720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40370/0 (socket says 172.21.15.94:40370) 2024-03-24T10:57:39.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.670+0000 7f36637fe640 1 -- 172.21.15.94:0/1897337668 learned_addr learned my addr 172.21.15.94:0/1897337668 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:57:39.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.671+0000 7f3663fff640 1 --2- 172.21.15.94:0/1897337668 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36640643a0 0x7f36640fb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:39.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.671+0000 7f36637fe640 1 -- 172.21.15.94:0/1897337668 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36640643a0 msgr2=0x7f36640fb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:39.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.671+0000 7f36637fe640 1 --2- 172.21.15.94:0/1897337668 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36640643a0 0x7f36640fb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:39.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.671+0000 7f36637fe640 1 -- 172.21.15.94:0/1897337668 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3654005040 con 0x7f3664102240 2024-03-24T10:57:39.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.671+0000 7f36637fe640 1 --2- 172.21.15.94:0/1897337668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3664102240 0x7f36640fb720 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3658034040 tx=0x7f3658009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:39.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.671+0000 7f3660ff9640 1 -- 172.21.15.94:0/1897337668 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36580092b0 con 0x7f3664102240 2024-03-24T10:57:39.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.671+0000 7f366a55d640 1 -- 172.21.15.94:0/1897337668 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3658005040 con 0x7f3664102240 2024-03-24T10:57:39.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.671+0000 7f3660ff9640 1 -- 172.21.15.94:0/1897337668 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f365803b400 con 0x7f3664102240 2024-03-24T10:57:39.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.671+0000 7f3660ff9640 1 -- 172.21.15.94:0/1897337668 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36580435d0 con 0x7f3664102240 2024-03-24T10:57:39.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.671+0000 7f366a55d640 1 -- 172.21.15.94:0/1897337668 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3664119390 con 0x7f3664102240 2024-03-24T10:57:39.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.672+0000 7f366a55d640 1 -- 172.21.15.94:0/1897337668 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36640fd940 con 0x7f3664102240 2024-03-24T10:57:39.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.673+0000 7f3660ff9640 1 -- 172.21.15.94:0/1897337668 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f365802f070 con 0x7f3664102240 2024-03-24T10:57:39.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.673+0000 7f3660ff9640 1 --2- 172.21.15.94:0/1897337668 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f364c06d750 0x7f364c06fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:39.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.673+0000 7f3660ff9640 1 -- 172.21.15.94:0/1897337668 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f36580aed50 con 0x7f3664102240 2024-03-24T10:57:39.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.673+0000 7f3663fff640 1 --2- 172.21.15.94:0/1897337668 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f364c06d750 0x7f364c06fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:39.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.674+0000 7f3663fff640 1 --2- 172.21.15.94:0/1897337668 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f364c06d750 0x7f364c06fc00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3654007fd0 tx=0x7f3654009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:39.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.678+0000 7f3660ff9640 1 -- 172.21.15.94:0/1897337668 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f36580784a0 con 0x7f3664102240 2024-03-24T10:57:39.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.906+0000 7f366a55d640 1 -- 172.21.15.94:0/1897337668 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7f3664109300 con 0x7f3664102240 2024-03-24T10:57:39.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.906+0000 7f3660ff9640 1 -- 172.21.15.94:0/1897337668 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v37) v1 ==== 107+0+4273 (secure 0 0 0) 0x7f365807bb60 con 0x7f3664102240 2024-03-24T10:57:39.906 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:57:39.906 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34374,"name":"cephfs.smithi094.efiqte","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6827/2043632450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2043632450},{"type":"v1","addr":"172.21.15.94:6827","nonce":2043632450}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":34406,"name":"cephfs.smithi094.llefgu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6829/2396893131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2396893131},{"type":"v1","addr":"172.21.15.94:6829","nonce":2396893131}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44321,"name":"cephfs.smithi096.egbmoq","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6825/2694461318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2694461318},{"type":"v1","addr":"172.21.15.96:6825","nonce":2694461318}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-24T10:43:39.435738+0000","modified":"2024-03-24T10:54:01.169767+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-24T10:57:39.907 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 31 2024-03-24T10:57:39.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.908+0000 7f366a55d640 1 -- 172.21.15.94:0/1897337668 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f364c06d750 msgr2=0x7f364c06fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:39.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.908+0000 7f366a55d640 1 --2- 172.21.15.94:0/1897337668 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f364c06d750 0x7f364c06fc00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3654007fd0 tx=0x7f3654009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:39.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.908+0000 7f366a55d640 1 -- 172.21.15.94:0/1897337668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3664102240 msgr2=0x7f36640fb720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:39.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.908+0000 7f366a55d640 1 --2- 172.21.15.94:0/1897337668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3664102240 0x7f36640fb720 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3658034040 tx=0x7f3658009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:39.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.908+0000 7f366a55d640 1 -- 172.21.15.94:0/1897337668 shutdown_connections 2024-03-24T10:57:39.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.908+0000 7f366a55d640 1 --2- 172.21.15.94:0/1897337668 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f364c06d750 0x7f364c06fc00 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:39.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.909+0000 7f366a55d640 1 --2- 172.21.15.94:0/1897337668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3664102240 0x7f36640fb720 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:39.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.909+0000 7f366a55d640 1 --2- 172.21.15.94:0/1897337668 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36640643a0 0x7f36640fb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:39.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.909+0000 7f366a55d640 1 -- 172.21.15.94:0/1897337668 >> 172.21.15.94:0/1897337668 conn(0x7f36640710b0 msgr2=0x7f36640f94b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:39.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.909+0000 7f366a55d640 1 -- 172.21.15.94:0/1897337668 shutdown_connections 2024-03-24T10:57:39.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:39.909+0000 7f366a55d640 1 -- 172.21.15.94:0/1897337668 wait complete. 2024-03-24T10:57:39.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:39 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/503535041' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-03-24T10:57:39.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:39 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/382186340' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-03-24T10:57:39.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:39 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/3271523562' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-03-24T10:57:39.919 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 31 2024-03-24T10:57:39.920 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 32 2024-03-24T10:57:40.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.019+0000 7f90f522e640 1 -- 172.21.15.94:0/2295486699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90f0101140 msgr2=0x7f90f0101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:40.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.019+0000 7f90f522e640 1 --2- 172.21.15.94:0/2295486699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90f0101140 0x7f90f0101510 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f90e00048d0 tx=0x7f90e0009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:40.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.019+0000 7f90f522e640 1 -- 172.21.15.94:0/2295486699 shutdown_connections 2024-03-24T10:57:40.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.019+0000 7f90f522e640 1 --2- 172.21.15.94:0/2295486699 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90f00fb110 0x7f90f00fb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:40.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.019+0000 7f90f522e640 1 --2- 172.21.15.94:0/2295486699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90f0101140 0x7f90f0101510 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:40.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.019+0000 7f90f522e640 1 -- 172.21.15.94:0/2295486699 >> 172.21.15.94:0/2295486699 conn(0x7f90f00710b0 msgr2=0x7f90f00714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:40.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.019+0000 7f90f522e640 1 -- 172.21.15.94:0/2295486699 shutdown_connections 2024-03-24T10:57:40.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.019+0000 7f90f522e640 1 -- 172.21.15.94:0/2295486699 wait complete. 2024-03-24T10:57:40.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.020+0000 7f90f522e640 1 Processor -- start 2024-03-24T10:57:40.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.020+0000 7f90f522e640 1 -- start start 2024-03-24T10:57:40.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.020+0000 7f90f522e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90f00fb110 0x7f90f0111bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:40.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.020+0000 7f90f522e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90f0101140 0x7f90f01120f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:40.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.020+0000 7f90f522e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f01126f0 con 0x7f90f0101140 2024-03-24T10:57:40.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.020+0000 7f90f522e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f010bd40 con 0x7f90f00fb110 2024-03-24T10:57:40.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.020+0000 7f90ee59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90f0101140 0x7f90f01120f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:40.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.020+0000 7f90ee59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90f0101140 0x7f90f01120f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40394/0 (socket says 172.21.15.94:40394) 2024-03-24T10:57:40.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.020+0000 7f90ee59c640 1 -- 172.21.15.94:0/3227820404 learned_addr learned my addr 172.21.15.94:0/3227820404 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:57:40.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.021+0000 7f90eed9d640 1 --2- 172.21.15.94:0/3227820404 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90f00fb110 0x7f90f0111bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:40.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.021+0000 7f90ee59c640 1 -- 172.21.15.94:0/3227820404 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90f00fb110 msgr2=0x7f90f0111bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:40.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.021+0000 7f90ee59c640 1 --2- 172.21.15.94:0/3227820404 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90f00fb110 0x7f90f0111bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:40.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.021+0000 7f90ee59c640 1 -- 172.21.15.94:0/3227820404 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e0005040 con 0x7f90f0101140 2024-03-24T10:57:40.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.021+0000 7f90ee59c640 1 --2- 172.21.15.94:0/3227820404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90f0101140 0x7f90f01120f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f90e4002a50 tx=0x7f90e4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:40.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.021+0000 7f90d77fe640 1 -- 172.21.15.94:0/3227820404 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90e4007c30 con 0x7f90f0101140 2024-03-24T10:57:40.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.021+0000 7f90d77fe640 1 -- 172.21.15.94:0/3227820404 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f90e400f040 con 0x7f90f0101140 2024-03-24T10:57:40.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.021+0000 7f90f522e640 1 -- 172.21.15.94:0/3227820404 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90f010c020 con 0x7f90f0101140 2024-03-24T10:57:40.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.021+0000 7f90d77fe640 1 -- 172.21.15.94:0/3227820404 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90e400db50 con 0x7f90f0101140 2024-03-24T10:57:40.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.021+0000 7f90f522e640 1 -- 172.21.15.94:0/3227820404 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90f010c540 con 0x7f90f0101140 2024-03-24T10:57:40.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.022+0000 7f90f522e640 1 -- 172.21.15.94:0/3227820404 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90f00fc800 con 0x7f90f0101140 2024-03-24T10:57:40.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.023+0000 7f90d77fe640 1 -- 172.21.15.94:0/3227820404 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f90e401c400 con 0x7f90f0101140 2024-03-24T10:57:40.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.023+0000 7f90d77fe640 1 --2- 172.21.15.94:0/3227820404 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f90d806d750 0x7f90d806fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:40.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.023+0000 7f90d77fe640 1 -- 172.21.15.94:0/3227820404 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f90e408c8a0 con 0x7f90f0101140 2024-03-24T10:57:40.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.023+0000 7f90eed9d640 1 --2- 172.21.15.94:0/3227820404 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f90d806d750 0x7f90d806fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:40.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.024+0000 7f90eed9d640 1 --2- 172.21.15.94:0/3227820404 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f90d806d750 0x7f90d806fc00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f90e0009990 tx=0x7f90e0009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:40.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.028+0000 7f90d77fe640 1 -- 172.21.15.94:0/3227820404 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f90e4055ff0 con 0x7f90f0101140 2024-03-24T10:57:40.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:39 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/503535041' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-03-24T10:57:40.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:39 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/382186340' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-03-24T10:57:40.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:39 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/3271523562' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-03-24T10:57:40.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.248+0000 7f90f522e640 1 -- 172.21.15.94:0/3227820404 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7f90f0108220 con 0x7f90f0101140 2024-03-24T10:57:40.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.248+0000 7f90d77fe640 1 -- 172.21.15.94:0/3227820404 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v37) v1 ==== 107+0+4352 (secure 0 0 0) 0x7f90e40596b0 con 0x7f90f0101140 2024-03-24T10:57:40.248 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:57:40.248 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34406,"name":"cephfs.smithi094.llefgu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6829/2396893131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2396893131},{"type":"v1","addr":"172.21.15.94:6829","nonce":2396893131}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44321,"name":"cephfs.smithi096.egbmoq","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6825/2694461318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2694461318},{"type":"v1","addr":"172.21.15.96:6825","nonce":2694461318}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-24T10:43:39.435738+0000","modified":"2024-03-24T10:54:01.174870+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34374},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34374":{"gid":34374,"name":"cephfs.smithi094.efiqte","rank":0,"incarnation":32,"state":"up:replay","state_seq":1,"addr":"172.21.15.94:6827/2043632450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2043632450},{"type":"v1","addr":"172.21.15.94:6827","nonce":2043632450}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-24T10:57:40.249 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 32 2024-03-24T10:57:40.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.250+0000 7f90f522e640 1 -- 172.21.15.94:0/3227820404 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f90d806d750 msgr2=0x7f90d806fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:40.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.250+0000 7f90f522e640 1 --2- 172.21.15.94:0/3227820404 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f90d806d750 0x7f90d806fc00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f90e0009990 tx=0x7f90e0009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:40.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.250+0000 7f90f522e640 1 -- 172.21.15.94:0/3227820404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90f0101140 msgr2=0x7f90f01120f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:40.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.250+0000 7f90f522e640 1 --2- 172.21.15.94:0/3227820404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90f0101140 0x7f90f01120f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f90e4002a50 tx=0x7f90e4004600 comp rx=0 tx=0).stop 2024-03-24T10:57:40.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.250+0000 7f90f522e640 1 -- 172.21.15.94:0/3227820404 shutdown_connections 2024-03-24T10:57:40.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.250+0000 7f90f522e640 1 --2- 172.21.15.94:0/3227820404 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f90d806d750 0x7f90d806fc00 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:40.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.250+0000 7f90f522e640 1 --2- 172.21.15.94:0/3227820404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90f0101140 0x7f90f01120f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:40.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.250+0000 7f90f522e640 1 --2- 172.21.15.94:0/3227820404 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90f00fb110 0x7f90f0111bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:40.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.250+0000 7f90f522e640 1 -- 172.21.15.94:0/3227820404 >> 172.21.15.94:0/3227820404 conn(0x7f90f00710b0 msgr2=0x7f90f00f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:40.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.250+0000 7f90f522e640 1 -- 172.21.15.94:0/3227820404 shutdown_connections 2024-03-24T10:57:40.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.250+0000 7f90f522e640 1 -- 172.21.15.94:0/3227820404 wait complete. 2024-03-24T10:57:40.262 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 32 2024-03-24T10:57:40.262 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 33 2024-03-24T10:57:40.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.389+0000 7efc5871d640 1 -- 172.21.15.94:0/3297899800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc500fb110 msgr2=0x7efc500fb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:40.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.389+0000 7efc5871d640 1 --2- 172.21.15.94:0/3297899800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc500fb110 0x7efc500fb560 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7efc400048d0 tx=0x7efc40009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:40.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.389+0000 7efc5871d640 1 -- 172.21.15.94:0/3297899800 shutdown_connections 2024-03-24T10:57:40.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.389+0000 7efc5871d640 1 --2- 172.21.15.94:0/3297899800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc500fb110 0x7efc500fb560 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:40.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.389+0000 7efc5871d640 1 --2- 172.21.15.94:0/3297899800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc50101140 0x7efc50101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:40.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.389+0000 7efc5871d640 1 -- 172.21.15.94:0/3297899800 >> 172.21.15.94:0/3297899800 conn(0x7efc500710b0 msgr2=0x7efc500714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:40.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.389+0000 7efc5871d640 1 -- 172.21.15.94:0/3297899800 shutdown_connections 2024-03-24T10:57:40.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.389+0000 7efc5871d640 1 -- 172.21.15.94:0/3297899800 wait complete. 2024-03-24T10:57:40.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.389+0000 7efc5871d640 1 Processor -- start 2024-03-24T10:57:40.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.389+0000 7efc5871d640 1 -- start start 2024-03-24T10:57:40.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.389+0000 7efc5871d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc500fb110 0x7efc5010deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:40.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.389+0000 7efc5871d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc50101140 0x7efc5010e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:40.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.389+0000 7efc5871d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc50065330 con 0x7efc500fb110 2024-03-24T10:57:40.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.389+0000 7efc5871d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5010e960 con 0x7efc50101140 2024-03-24T10:57:40.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.389+0000 7efc564b9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc500fb110 0x7efc5010deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:40.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.390+0000 7efc564b9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc500fb110 0x7efc5010deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40406/0 (socket says 172.21.15.94:40406) 2024-03-24T10:57:40.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.390+0000 7efc564b9640 1 -- 172.21.15.94:0/4249386386 learned_addr learned my addr 172.21.15.94:0/4249386386 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:57:40.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.390+0000 7efc55cb8640 1 --2- 172.21.15.94:0/4249386386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc50101140 0x7efc5010e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:40.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.390+0000 7efc564b9640 1 -- 172.21.15.94:0/4249386386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc50101140 msgr2=0x7efc5010e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:40.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.390+0000 7efc564b9640 1 --2- 172.21.15.94:0/4249386386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc50101140 0x7efc5010e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:40.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.390+0000 7efc564b9640 1 -- 172.21.15.94:0/4249386386 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc40005040 con 0x7efc500fb110 2024-03-24T10:57:40.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.390+0000 7efc564b9640 1 --2- 172.21.15.94:0/4249386386 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc500fb110 0x7efc5010deb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7efc4c00eb70 tx=0x7efc4c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:40.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.390+0000 7efc46ffd640 1 -- 172.21.15.94:0/4249386386 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc4c007c30 con 0x7efc500fb110 2024-03-24T10:57:40.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.390+0000 7efc46ffd640 1 -- 172.21.15.94:0/4249386386 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efc4c00f040 con 0x7efc500fb110 2024-03-24T10:57:40.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.390+0000 7efc5871d640 1 -- 172.21.15.94:0/4249386386 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc5010ebe0 con 0x7efc500fb110 2024-03-24T10:57:40.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.390+0000 7efc46ffd640 1 -- 172.21.15.94:0/4249386386 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc4c0095b0 con 0x7efc500fb110 2024-03-24T10:57:40.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.390+0000 7efc5871d640 1 -- 172.21.15.94:0/4249386386 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc50117170 con 0x7efc500fb110 2024-03-24T10:57:40.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.391+0000 7efc5871d640 1 -- 172.21.15.94:0/4249386386 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc500fc800 con 0x7efc500fb110 2024-03-24T10:57:40.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.392+0000 7efc46ffd640 1 -- 172.21.15.94:0/4249386386 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7efc4c009730 con 0x7efc500fb110 2024-03-24T10:57:40.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.392+0000 7efc46ffd640 1 --2- 172.21.15.94:0/4249386386 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7efc3c06d750 0x7efc3c06fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:40.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.392+0000 7efc55cb8640 1 --2- 172.21.15.94:0/4249386386 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7efc3c06d750 0x7efc3c06fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:40.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.392+0000 7efc46ffd640 1 -- 172.21.15.94:0/4249386386 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7efc4c08cd20 con 0x7efc500fb110 2024-03-24T10:57:40.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.393+0000 7efc55cb8640 1 --2- 172.21.15.94:0/4249386386 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7efc3c06d750 0x7efc3c06fc00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efc40009fa0 tx=0x7efc40009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:40.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.397+0000 7efc46ffd640 1 -- 172.21.15.94:0/4249386386 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7efc4c056470 con 0x7efc500fb110 2024-03-24T10:57:40.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.616+0000 7efc5871d640 1 -- 172.21.15.94:0/4249386386 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7efc5004bcc0 con 0x7efc500fb110 2024-03-24T10:57:40.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.616+0000 7efc46ffd640 1 -- 172.21.15.94:0/4249386386 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v37) v1 ==== 107+0+4356 (secure 0 0 0) 0x7efc4c059b30 con 0x7efc500fb110 2024-03-24T10:57:40.616 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:57:40.616 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34406,"name":"cephfs.smithi094.llefgu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6829/2396893131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2396893131},{"type":"v1","addr":"172.21.15.94:6829","nonce":2396893131}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44321,"name":"cephfs.smithi096.egbmoq","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6825/2694461318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2694461318},{"type":"v1","addr":"172.21.15.96:6825","nonce":2694461318}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-24T10:43:39.435738+0000","modified":"2024-03-24T10:54:09.327532+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34374},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34374":{"gid":34374,"name":"cephfs.smithi094.efiqte","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":17,"addr":"172.21.15.94:6827/2043632450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2043632450},{"type":"v1","addr":"172.21.15.94:6827","nonce":2043632450}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-24T10:57:40.616 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 33 2024-03-24T10:57:40.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.618+0000 7efc5871d640 1 -- 172.21.15.94:0/4249386386 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7efc3c06d750 msgr2=0x7efc3c06fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:40.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.618+0000 7efc5871d640 1 --2- 172.21.15.94:0/4249386386 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7efc3c06d750 0x7efc3c06fc00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efc40009fa0 tx=0x7efc40009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:40.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.618+0000 7efc5871d640 1 -- 172.21.15.94:0/4249386386 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc500fb110 msgr2=0x7efc5010deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:40.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.618+0000 7efc5871d640 1 --2- 172.21.15.94:0/4249386386 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc500fb110 0x7efc5010deb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7efc4c00eb70 tx=0x7efc4c00b040 comp rx=0 tx=0).stop 2024-03-24T10:57:40.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.618+0000 7efc5871d640 1 -- 172.21.15.94:0/4249386386 shutdown_connections 2024-03-24T10:57:40.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.618+0000 7efc5871d640 1 --2- 172.21.15.94:0/4249386386 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7efc3c06d750 0x7efc3c06fc00 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:40.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.618+0000 7efc5871d640 1 --2- 172.21.15.94:0/4249386386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc50101140 0x7efc5010e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:40.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.618+0000 7efc5871d640 1 --2- 172.21.15.94:0/4249386386 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc500fb110 0x7efc5010deb0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:40.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.618+0000 7efc5871d640 1 -- 172.21.15.94:0/4249386386 >> 172.21.15.94:0/4249386386 conn(0x7efc500710b0 msgr2=0x7efc500f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:40.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.618+0000 7efc5871d640 1 -- 172.21.15.94:0/4249386386 shutdown_connections 2024-03-24T10:57:40.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.618+0000 7efc5871d640 1 -- 172.21.15.94:0/4249386386 wait complete. 2024-03-24T10:57:40.630 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 33 2024-03-24T10:57:40.630 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 34 2024-03-24T10:57:40.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.732+0000 7f6e0fe6e640 1 -- 172.21.15.94:0/3880077624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e08102b50 msgr2=0x7f6e080fc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:40.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.732+0000 7f6e0fe6e640 1 --2- 172.21.15.94:0/3880077624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e08102b50 0x7f6e080fc1e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6df80048d0 tx=0x7f6df8009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:40.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.732+0000 7f6e0fe6e640 1 -- 172.21.15.94:0/3880077624 shutdown_connections 2024-03-24T10:57:40.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.732+0000 7f6e0fe6e640 1 --2- 172.21.15.94:0/3880077624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e08102b50 0x7f6e080fc1e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:40.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.732+0000 7f6e0fe6e640 1 --2- 172.21.15.94:0/3880077624 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e081021b0 0x7f6e08102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:40.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.732+0000 7f6e0fe6e640 1 -- 172.21.15.94:0/3880077624 >> 172.21.15.94:0/3880077624 conn(0x7f6e080f7dc0 msgr2=0x7f6e080fa1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:40.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.732+0000 7f6e0fe6e640 1 -- 172.21.15.94:0/3880077624 shutdown_connections 2024-03-24T10:57:40.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.732+0000 7f6e0fe6e640 1 -- 172.21.15.94:0/3880077624 wait complete. 2024-03-24T10:57:40.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.732+0000 7f6e0fe6e640 1 Processor -- start 2024-03-24T10:57:40.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.732+0000 7f6e0fe6e640 1 -- start start 2024-03-24T10:57:40.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.733+0000 7f6e0fe6e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e081021b0 0x7f6e08112cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:40.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.733+0000 7f6e0fe6e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e08102b50 0x7f6e08113210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:40.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.733+0000 7f6e0fe6e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e08113810 con 0x7f6e081021b0 2024-03-24T10:57:40.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.733+0000 7f6e0fe6e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0810ce60 con 0x7f6e08102b50 2024-03-24T10:57:40.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.733+0000 7f6e0dc0a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e081021b0 0x7f6e08112cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:40.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.733+0000 7f6e0dc0a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e081021b0 0x7f6e08112cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40420/0 (socket says 172.21.15.94:40420) 2024-03-24T10:57:40.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.733+0000 7f6e0dc0a640 1 -- 172.21.15.94:0/1434960971 learned_addr learned my addr 172.21.15.94:0/1434960971 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:57:40.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.733+0000 7f6e0d409640 1 --2- 172.21.15.94:0/1434960971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e08102b50 0x7f6e08113210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:40.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.733+0000 7f6e0dc0a640 1 -- 172.21.15.94:0/1434960971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e08102b50 msgr2=0x7f6e08113210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:40.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.733+0000 7f6e0dc0a640 1 --2- 172.21.15.94:0/1434960971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e08102b50 0x7f6e08113210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:40.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.733+0000 7f6e0dc0a640 1 -- 172.21.15.94:0/1434960971 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df8005040 con 0x7f6e081021b0 2024-03-24T10:57:40.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.733+0000 7f6e0dc0a640 1 --2- 172.21.15.94:0/1434960971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e081021b0 0x7f6e08112cd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6e04002a50 tx=0x7f6e04004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:40.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.733+0000 7f6dfe7fc640 1 -- 172.21.15.94:0/1434960971 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e04007c30 con 0x7f6e081021b0 2024-03-24T10:57:40.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.733+0000 7f6dfe7fc640 1 -- 172.21.15.94:0/1434960971 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6e0400f040 con 0x7f6e081021b0 2024-03-24T10:57:40.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.733+0000 7f6e0fe6e640 1 -- 172.21.15.94:0/1434960971 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e0810d140 con 0x7f6e081021b0 2024-03-24T10:57:40.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.733+0000 7f6dfe7fc640 1 -- 172.21.15.94:0/1434960971 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e0400db60 con 0x7f6e081021b0 2024-03-24T10:57:40.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.733+0000 7f6e0fe6e640 1 -- 172.21.15.94:0/1434960971 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e0810d630 con 0x7f6e081021b0 2024-03-24T10:57:40.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.734+0000 7f6e0fe6e640 1 -- 172.21.15.94:0/1434960971 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e080fd920 con 0x7f6e081021b0 2024-03-24T10:57:40.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.735+0000 7f6dfe7fc640 1 -- 172.21.15.94:0/1434960971 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f6e0400dcc0 con 0x7f6e081021b0 2024-03-24T10:57:40.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.735+0000 7f6dfe7fc640 1 --2- 172.21.15.94:0/1434960971 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f6df406d760 0x7f6df406fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:40.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.735+0000 7f6e0d409640 1 --2- 172.21.15.94:0/1434960971 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f6df406d760 0x7f6df406fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:40.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.735+0000 7f6dfe7fc640 1 -- 172.21.15.94:0/1434960971 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f6e0408cdc0 con 0x7f6e081021b0 2024-03-24T10:57:40.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.736+0000 7f6e0d409640 1 --2- 172.21.15.94:0/1434960971 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f6df406d760 0x7f6df406fc10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6df8000c00 tx=0x7f6df8009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:40.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.740+0000 7f6dfe7fc640 1 -- 172.21.15.94:0/1434960971 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6e04056600 con 0x7f6e081021b0 2024-03-24T10:57:40.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.958+0000 7f6e0fe6e640 1 -- 172.21.15.94:0/1434960971 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) v1 -- 0x7f6e0804bcc0 con 0x7f6e081021b0 2024-03-24T10:57:40.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.959+0000 7f6dfe7fc640 1 -- 172.21.15.94:0/1434960971 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v37) v1 ==== 107+0+4353 (secure 0 0 0) 0x7f6e04059cc0 con 0x7f6e081021b0 2024-03-24T10:57:40.959 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:57:40.959 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34406,"name":"cephfs.smithi094.llefgu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6829/2396893131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2396893131},{"type":"v1","addr":"172.21.15.94:6829","nonce":2396893131}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44321,"name":"cephfs.smithi096.egbmoq","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6825/2694461318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2694461318},{"type":"v1","addr":"172.21.15.96:6825","nonce":2694461318}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-24T10:43:39.435738+0000","modified":"2024-03-24T10:54:09.338509+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34374},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34374":{"gid":34374,"name":"cephfs.smithi094.efiqte","rank":0,"incarnation":32,"state":"up:rejoin","state_seq":18,"addr":"172.21.15.94:6827/2043632450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2043632450},{"type":"v1","addr":"172.21.15.94:6827","nonce":2043632450}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-24T10:57:40.959 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 34 2024-03-24T10:57:40.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.961+0000 7f6e0fe6e640 1 -- 172.21.15.94:0/1434960971 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f6df406d760 msgr2=0x7f6df406fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:40.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.961+0000 7f6e0fe6e640 1 --2- 172.21.15.94:0/1434960971 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f6df406d760 0x7f6df406fc10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6df8000c00 tx=0x7f6df8009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:40.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.961+0000 7f6e0fe6e640 1 -- 172.21.15.94:0/1434960971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e081021b0 msgr2=0x7f6e08112cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:40.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.961+0000 7f6e0fe6e640 1 --2- 172.21.15.94:0/1434960971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e081021b0 0x7f6e08112cd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6e04002a50 tx=0x7f6e04004600 comp rx=0 tx=0).stop 2024-03-24T10:57:40.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.961+0000 7f6e0fe6e640 1 -- 172.21.15.94:0/1434960971 shutdown_connections 2024-03-24T10:57:40.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.961+0000 7f6e0fe6e640 1 --2- 172.21.15.94:0/1434960971 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f6df406d760 0x7f6df406fc10 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:40.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.961+0000 7f6e0fe6e640 1 --2- 172.21.15.94:0/1434960971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e08102b50 0x7f6e08113210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:40.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.961+0000 7f6e0fe6e640 1 --2- 172.21.15.94:0/1434960971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e081021b0 0x7f6e08112cd0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:40.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.961+0000 7f6e0fe6e640 1 -- 172.21.15.94:0/1434960971 >> 172.21.15.94:0/1434960971 conn(0x7f6e080f7dc0 msgr2=0x7f6e080f8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:40.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.961+0000 7f6e0fe6e640 1 -- 172.21.15.94:0/1434960971 shutdown_connections 2024-03-24T10:57:40.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:40.961+0000 7f6e0fe6e640 1 -- 172.21.15.94:0/1434960971 wait complete. 2024-03-24T10:57:40.970 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:40 smithi094 ceph-mon[93957]: pgmap v310: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:57:40.970 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:40 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/1897337668' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-03-24T10:57:40.970 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:40 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/3227820404' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-03-24T10:57:40.971 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:40 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/4249386386' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-03-24T10:57:40.972 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 34 2024-03-24T10:57:40.972 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 35 2024-03-24T10:57:41.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.072+0000 7fb2b2007640 1 -- 172.21.15.94:0/3327117686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2ac0fb110 msgr2=0x7fb2ac0fb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:41.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.072+0000 7fb2b2007640 1 --2- 172.21.15.94:0/3327117686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2ac0fb110 0x7fb2ac0fb560 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb2a00048d0 tx=0x7fb2a0009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:41.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.072+0000 7fb2b2007640 1 -- 172.21.15.94:0/3327117686 shutdown_connections 2024-03-24T10:57:41.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.072+0000 7fb2b2007640 1 --2- 172.21.15.94:0/3327117686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2ac0fb110 0x7fb2ac0fb560 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:41.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.072+0000 7fb2b2007640 1 --2- 172.21.15.94:0/3327117686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2ac101140 0x7fb2ac101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:41.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.072+0000 7fb2b2007640 1 -- 172.21.15.94:0/3327117686 >> 172.21.15.94:0/3327117686 conn(0x7fb2ac0710b0 msgr2=0x7fb2ac0714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:41.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.072+0000 7fb2b2007640 1 -- 172.21.15.94:0/3327117686 shutdown_connections 2024-03-24T10:57:41.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.072+0000 7fb2b2007640 1 -- 172.21.15.94:0/3327117686 wait complete. 2024-03-24T10:57:41.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.072+0000 7fb2b2007640 1 Processor -- start 2024-03-24T10:57:41.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.073+0000 7fb2b2007640 1 -- start start 2024-03-24T10:57:41.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.073+0000 7fb2b2007640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2ac0fb110 0x7fb2ac10deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:41.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.073+0000 7fb2b2007640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2ac101140 0x7fb2ac10e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:41.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.073+0000 7fb2b2007640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ac065330 con 0x7fb2ac0fb110 2024-03-24T10:57:41.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.073+0000 7fb2b2007640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ac10e960 con 0x7fb2ac101140 2024-03-24T10:57:41.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.073+0000 7fb2ab7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2ac0fb110 0x7fb2ac10deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:41.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.073+0000 7fb2ab7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2ac0fb110 0x7fb2ac10deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40438/0 (socket says 172.21.15.94:40438) 2024-03-24T10:57:41.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.073+0000 7fb2ab7fe640 1 -- 172.21.15.94:0/2241027018 learned_addr learned my addr 172.21.15.94:0/2241027018 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:57:41.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.073+0000 7fb2ab7fe640 1 -- 172.21.15.94:0/2241027018 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2ac101140 msgr2=0x7fb2ac10e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:41.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.073+0000 7fb2ab7fe640 1 --2- 172.21.15.94:0/2241027018 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2ac101140 0x7fb2ac10e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:41.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.073+0000 7fb2ab7fe640 1 -- 172.21.15.94:0/2241027018 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a0005040 con 0x7fb2ac0fb110 2024-03-24T10:57:41.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.073+0000 7fb2ab7fe640 1 --2- 172.21.15.94:0/2241027018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2ac0fb110 0x7fb2ac10deb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb29c002a50 tx=0x7fb29c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:41.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.073+0000 7fb293fff640 1 -- 172.21.15.94:0/2241027018 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb29c007c30 con 0x7fb2ac0fb110 2024-03-24T10:57:41.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.074+0000 7fb293fff640 1 -- 172.21.15.94:0/2241027018 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb29c00f040 con 0x7fb2ac0fb110 2024-03-24T10:57:41.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.074+0000 7fb293fff640 1 -- 172.21.15.94:0/2241027018 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb29c00db50 con 0x7fb2ac0fb110 2024-03-24T10:57:41.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.074+0000 7fb2b2007640 1 -- 172.21.15.94:0/2241027018 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2ac10ec40 con 0x7fb2ac0fb110 2024-03-24T10:57:41.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.074+0000 7fb2b2007640 1 -- 172.21.15.94:0/2241027018 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2ac1171d0 con 0x7fb2ac0fb110 2024-03-24T10:57:41.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.074+0000 7fb2b2007640 1 -- 172.21.15.94:0/2241027018 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2ac0fc800 con 0x7fb2ac0fb110 2024-03-24T10:57:41.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.075+0000 7fb293fff640 1 -- 172.21.15.94:0/2241027018 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fb29c01c400 con 0x7fb2ac0fb110 2024-03-24T10:57:41.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:40 smithi096 ceph-mon[84258]: pgmap v310: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:57:41.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:40 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/1897337668' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-03-24T10:57:41.076 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:40 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/3227820404' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-03-24T10:57:41.077 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:40 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/4249386386' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-03-24T10:57:41.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.075+0000 7fb293fff640 1 --2- 172.21.15.94:0/2241027018 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fb29406d750 0x7fb29406fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:41.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.076+0000 7fb2aaffd640 1 --2- 172.21.15.94:0/2241027018 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fb29406d750 0x7fb29406fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:41.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.076+0000 7fb293fff640 1 -- 172.21.15.94:0/2241027018 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fb29c08c870 con 0x7fb2ac0fb110 2024-03-24T10:57:41.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.076+0000 7fb2aaffd640 1 --2- 172.21.15.94:0/2241027018 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fb29406d750 0x7fb29406fc00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0009990 tx=0x7fb2a0009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:41.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.080+0000 7fb293fff640 1 -- 172.21.15.94:0/2241027018 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb29c055fc0 con 0x7fb2ac0fb110 2024-03-24T10:57:41.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.298+0000 7fb2b2007640 1 -- 172.21.15.94:0/2241027018 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 35, "format": "json"} v 0) v1 -- 0x7fb2ac1173f0 con 0x7fb2ac0fb110 2024-03-24T10:57:41.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.299+0000 7fb293fff640 1 -- 172.21.15.94:0/2241027018 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 35, "format": "json"}]=0 dumped fsmap epoch 35 v37) v1 ==== 107+0+5209 (secure 0 0 0) 0x7fb29c059680 con 0x7fb2ac0fb110 2024-03-24T10:57:41.299 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:57:41.299 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34406,"name":"cephfs.smithi094.llefgu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6829/2396893131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2396893131},{"type":"v1","addr":"172.21.15.94:6829","nonce":2396893131}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44321,"name":"cephfs.smithi096.egbmoq","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6825/2694461318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2694461318},{"type":"v1","addr":"172.21.15.96:6825","nonce":2694461318}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":44349,"name":"cephfs.smithi096.jtebhh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6827/2275611388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":2275611388},{"type":"v1","addr":"172.21.15.96:6827","nonce":2275611388}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-24T10:43:39.435738+0000","modified":"2024-03-24T10:54:11.341853+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34374},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34374":{"gid":34374,"name":"cephfs.smithi094.efiqte","rank":0,"incarnation":32,"state":"up:active","state_seq":19,"addr":"172.21.15.94:6827/2043632450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2043632450},{"type":"v1","addr":"172.21.15.94:6827","nonce":2043632450}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34374,"qdb_cluster":[34374]},"id":1}]} 2024-03-24T10:57:41.299 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 35 2024-03-24T10:57:41.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.301+0000 7fb2b2007640 1 -- 172.21.15.94:0/2241027018 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fb29406d750 msgr2=0x7fb29406fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:41.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.301+0000 7fb2b2007640 1 --2- 172.21.15.94:0/2241027018 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fb29406d750 0x7fb29406fc00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0009990 tx=0x7fb2a0009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:41.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.301+0000 7fb2b2007640 1 -- 172.21.15.94:0/2241027018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2ac0fb110 msgr2=0x7fb2ac10deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:41.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.301+0000 7fb2b2007640 1 --2- 172.21.15.94:0/2241027018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2ac0fb110 0x7fb2ac10deb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb29c002a50 tx=0x7fb29c004600 comp rx=0 tx=0).stop 2024-03-24T10:57:41.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.301+0000 7fb2b2007640 1 -- 172.21.15.94:0/2241027018 shutdown_connections 2024-03-24T10:57:41.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.301+0000 7fb2b2007640 1 --2- 172.21.15.94:0/2241027018 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fb29406d750 0x7fb29406fc00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:41.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.301+0000 7fb2b2007640 1 --2- 172.21.15.94:0/2241027018 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2ac101140 0x7fb2ac10e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:41.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.301+0000 7fb2b2007640 1 --2- 172.21.15.94:0/2241027018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2ac0fb110 0x7fb2ac10deb0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:41.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.301+0000 7fb2b2007640 1 -- 172.21.15.94:0/2241027018 >> 172.21.15.94:0/2241027018 conn(0x7fb2ac0710b0 msgr2=0x7fb2ac0f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:41.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.301+0000 7fb2b2007640 1 -- 172.21.15.94:0/2241027018 shutdown_connections 2024-03-24T10:57:41.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.301+0000 7fb2b2007640 1 -- 172.21.15.94:0/2241027018 wait complete. 2024-03-24T10:57:41.311 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 35 2024-03-24T10:57:41.312 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 36 2024-03-24T10:57:41.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.412+0000 7f62e7d1d640 1 -- 172.21.15.94:0/985816265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62e00fb110 msgr2=0x7f62e00fb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:41.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.412+0000 7f62e7d1d640 1 --2- 172.21.15.94:0/985816265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62e00fb110 0x7f62e00fb560 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f62d00048d0 tx=0x7f62d0009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:41.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.412+0000 7f62e7d1d640 1 -- 172.21.15.94:0/985816265 shutdown_connections 2024-03-24T10:57:41.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.412+0000 7f62e7d1d640 1 --2- 172.21.15.94:0/985816265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62e00fb110 0x7f62e00fb560 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:41.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.412+0000 7f62e7d1d640 1 --2- 172.21.15.94:0/985816265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62e0101140 0x7f62e0101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:41.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.412+0000 7f62e7d1d640 1 -- 172.21.15.94:0/985816265 >> 172.21.15.94:0/985816265 conn(0x7f62e00710b0 msgr2=0x7f62e00714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:41.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.412+0000 7f62e7d1d640 1 -- 172.21.15.94:0/985816265 shutdown_connections 2024-03-24T10:57:41.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.412+0000 7f62e7d1d640 1 -- 172.21.15.94:0/985816265 wait complete. 2024-03-24T10:57:41.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.413+0000 7f62e7d1d640 1 Processor -- start 2024-03-24T10:57:41.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.413+0000 7f62e7d1d640 1 -- start start 2024-03-24T10:57:41.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.413+0000 7f62e7d1d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62e00fb110 0x7f62e0112ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:41.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.413+0000 7f62e7d1d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62e0101140 0x7f62e0113220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:41.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.413+0000 7f62e7d1d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e0113820 con 0x7f62e00fb110 2024-03-24T10:57:41.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.413+0000 7f62e7d1d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e010ce70 con 0x7f62e0101140 2024-03-24T10:57:41.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.413+0000 7f62e5ab9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62e00fb110 0x7f62e0112ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:41.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.413+0000 7f62e5ab9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62e00fb110 0x7f62e0112ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40450/0 (socket says 172.21.15.94:40450) 2024-03-24T10:57:41.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.413+0000 7f62e5ab9640 1 -- 172.21.15.94:0/4261415093 learned_addr learned my addr 172.21.15.94:0/4261415093 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:57:41.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.413+0000 7f62e5ab9640 1 -- 172.21.15.94:0/4261415093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62e0101140 msgr2=0x7f62e0113220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:41.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.413+0000 7f62e5ab9640 1 --2- 172.21.15.94:0/4261415093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62e0101140 0x7f62e0113220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:41.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.413+0000 7f62e5ab9640 1 -- 172.21.15.94:0/4261415093 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d0005040 con 0x7f62e00fb110 2024-03-24T10:57:41.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.413+0000 7f62e5ab9640 1 --2- 172.21.15.94:0/4261415093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62e00fb110 0x7f62e0112ce0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f62dc002a50 tx=0x7f62dc004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:41.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.413+0000 7f62d67fc640 1 -- 172.21.15.94:0/4261415093 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62dc007c30 con 0x7f62e00fb110 2024-03-24T10:57:41.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.414+0000 7f62d67fc640 1 -- 172.21.15.94:0/4261415093 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f62dc00f040 con 0x7f62e00fb110 2024-03-24T10:57:41.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.414+0000 7f62e7d1d640 1 -- 172.21.15.94:0/4261415093 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62e010d150 con 0x7f62e00fb110 2024-03-24T10:57:41.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.414+0000 7f62d67fc640 1 -- 172.21.15.94:0/4261415093 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62dc00dbe0 con 0x7f62e00fb110 2024-03-24T10:57:41.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.414+0000 7f62e7d1d640 1 -- 172.21.15.94:0/4261415093 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62e010d670 con 0x7f62e00fb110 2024-03-24T10:57:41.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.415+0000 7f62e7d1d640 1 -- 172.21.15.94:0/4261415093 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62e00fc800 con 0x7f62e00fb110 2024-03-24T10:57:41.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.415+0000 7f62d67fc640 1 -- 172.21.15.94:0/4261415093 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f62dc00dd40 con 0x7f62e00fb110 2024-03-24T10:57:41.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.415+0000 7f62d67fc640 1 --2- 172.21.15.94:0/4261415093 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f62cc06d750 0x7f62cc06fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:41.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.415+0000 7f62d67fc640 1 -- 172.21.15.94:0/4261415093 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f62dc08cfc0 con 0x7f62e00fb110 2024-03-24T10:57:41.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.416+0000 7f62e52b8640 1 --2- 172.21.15.94:0/4261415093 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f62cc06d750 0x7f62cc06fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:41.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.416+0000 7f62e52b8640 1 --2- 172.21.15.94:0/4261415093 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f62cc06d750 0x7f62cc06fc00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f62d0009990 tx=0x7f62d0009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:41.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.420+0000 7f62d67fc640 1 -- 172.21.15.94:0/4261415093 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f62dc056710 con 0x7f62e00fb110 2024-03-24T10:57:41.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.636+0000 7f62e7d1d640 1 -- 172.21.15.94:0/4261415093 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 36, "format": "json"} v 0) v1 -- 0x7f62e004bcc0 con 0x7f62e00fb110 2024-03-24T10:57:41.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.637+0000 7f62d67fc640 1 -- 172.21.15.94:0/4261415093 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 36, "format": "json"}]=0 dumped fsmap epoch 36 v37) v1 ==== 107+0+5208 (secure 0 0 0) 0x7f62e004bcc0 con 0x7f62e00fb110 2024-03-24T10:57:41.637 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:57:41.637 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34406,"name":"cephfs.smithi094.llefgu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6829/2396893131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2396893131},{"type":"v1","addr":"172.21.15.94:6829","nonce":2396893131}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44321,"name":"cephfs.smithi096.egbmoq","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6825/2694461318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2694461318},{"type":"v1","addr":"172.21.15.96:6825","nonce":2694461318}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":44349,"name":"cephfs.smithi096.jtebhh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6827/2275611388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":2275611388},{"type":"v1","addr":"172.21.15.96:6827","nonce":2275611388}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-24T10:43:39.435738+0000","modified":"2024-03-24T10:54:18.397541+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34374},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34374":{"gid":34374,"name":"cephfs.smithi094.efiqte","rank":0,"incarnation":32,"state":"up:active","state_seq":19,"addr":"172.21.15.94:6827/2043632450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2043632450},{"type":"v1","addr":"172.21.15.94:6827","nonce":2043632450}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34374,"qdb_cluster":[34374]},"id":1}]} 2024-03-24T10:57:41.637 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 36 2024-03-24T10:57:41.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.639+0000 7f62e7d1d640 1 -- 172.21.15.94:0/4261415093 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f62cc06d750 msgr2=0x7f62cc06fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:41.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.639+0000 7f62e7d1d640 1 --2- 172.21.15.94:0/4261415093 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f62cc06d750 0x7f62cc06fc00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f62d0009990 tx=0x7f62d0009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:41.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.639+0000 7f62e7d1d640 1 -- 172.21.15.94:0/4261415093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62e00fb110 msgr2=0x7f62e0112ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:41.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.639+0000 7f62e7d1d640 1 --2- 172.21.15.94:0/4261415093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62e00fb110 0x7f62e0112ce0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f62dc002a50 tx=0x7f62dc004600 comp rx=0 tx=0).stop 2024-03-24T10:57:41.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.639+0000 7f62e7d1d640 1 -- 172.21.15.94:0/4261415093 shutdown_connections 2024-03-24T10:57:41.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.639+0000 7f62e7d1d640 1 --2- 172.21.15.94:0/4261415093 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f62cc06d750 0x7f62cc06fc00 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:41.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.639+0000 7f62e7d1d640 1 --2- 172.21.15.94:0/4261415093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62e0101140 0x7f62e0113220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:41.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.639+0000 7f62e7d1d640 1 --2- 172.21.15.94:0/4261415093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62e00fb110 0x7f62e0112ce0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:41.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.639+0000 7f62e7d1d640 1 -- 172.21.15.94:0/4261415093 >> 172.21.15.94:0/4261415093 conn(0x7f62e00710b0 msgr2=0x7f62e00f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:41.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.639+0000 7f62e7d1d640 1 -- 172.21.15.94:0/4261415093 shutdown_connections 2024-03-24T10:57:41.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.639+0000 7f62e7d1d640 1 -- 172.21.15.94:0/4261415093 wait complete. 2024-03-24T10:57:41.652 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-24T10:57:41.662 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-24T10:57:41.663 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-03-24T10:57:41.663 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-24T10:57:41.679 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-03-24T10:57:41.679 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-24T10:57:41.736 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-24T10:57:41.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.875+0000 7f71e826e640 1 -- 172.21.15.94:0/1195833549 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71e0102210 msgr2=0x7f71e01025e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:41.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.875+0000 7f71e826e640 1 --2- 172.21.15.94:0/1195833549 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71e0102210 0x7f71e01025e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f71dc0048d0 tx=0x7f71dc009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:41.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.875+0000 7f71e826e640 1 -- 172.21.15.94:0/1195833549 shutdown_connections 2024-03-24T10:57:41.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.875+0000 7f71e826e640 1 --2- 172.21.15.94:0/1195833549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71e00643a0 0x7f71e00647f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:41.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.875+0000 7f71e826e640 1 --2- 172.21.15.94:0/1195833549 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71e0102210 0x7f71e01025e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:41.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.875+0000 7f71e826e640 1 -- 172.21.15.94:0/1195833549 >> 172.21.15.94:0/1195833549 conn(0x7f71e0071130 msgr2=0x7f71e0071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:41.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.875+0000 7f71e826e640 1 -- 172.21.15.94:0/1195833549 shutdown_connections 2024-03-24T10:57:41.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.875+0000 7f71e826e640 1 -- 172.21.15.94:0/1195833549 wait complete. 2024-03-24T10:57:41.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.876+0000 7f71e826e640 1 Processor -- start 2024-03-24T10:57:41.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.876+0000 7f71e826e640 1 -- start start 2024-03-24T10:57:41.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.876+0000 7f71e826e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71e00643a0 0x7f71e00fb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:41.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.876+0000 7f71e826e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71e0102210 0x7f71e00fb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:41.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.876+0000 7f71e826e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e00fd0c0 con 0x7f71e00643a0 2024-03-24T10:57:41.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.876+0000 7f71e826e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e00fbc60 con 0x7f71e0102210 2024-03-24T10:57:41.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.876+0000 7f71e600a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71e00643a0 0x7f71e00fb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:41.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.876+0000 7f71e600a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71e00643a0 0x7f71e00fb1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40478/0 (socket says 172.21.15.94:40478) 2024-03-24T10:57:41.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.876+0000 7f71e600a640 1 -- 172.21.15.94:0/2662848427 learned_addr learned my addr 172.21.15.94:0/2662848427 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:57:41.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.876+0000 7f71e5809640 1 --2- 172.21.15.94:0/2662848427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71e0102210 0x7f71e00fb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:41.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.876+0000 7f71e600a640 1 -- 172.21.15.94:0/2662848427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71e0102210 msgr2=0x7f71e00fb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:41.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.876+0000 7f71e600a640 1 --2- 172.21.15.94:0/2662848427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71e0102210 0x7f71e00fb720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:41.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.876+0000 7f71e600a640 1 -- 172.21.15.94:0/2662848427 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71dc005040 con 0x7f71e00643a0 2024-03-24T10:57:41.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.876+0000 7f71e5809640 1 --2- 172.21.15.94:0/2662848427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71e0102210 0x7f71e00fb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T10:57:41.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.876+0000 7f71e600a640 1 --2- 172.21.15.94:0/2662848427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71e00643a0 0x7f71e00fb1e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f71dc004930 tx=0x7f71dc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:41.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.877+0000 7f71d6ffd640 1 -- 172.21.15.94:0/2662848427 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71dc0094c0 con 0x7f71e00643a0 2024-03-24T10:57:41.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.877+0000 7f71d6ffd640 1 -- 172.21.15.94:0/2662848427 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f71dc032e50 con 0x7f71e00643a0 2024-03-24T10:57:41.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.877+0000 7f71d6ffd640 1 -- 172.21.15.94:0/2662848427 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71dc0433b0 con 0x7f71e00643a0 2024-03-24T10:57:41.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.877+0000 7f71e826e640 1 -- 172.21.15.94:0/2662848427 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71e00fbee0 con 0x7f71e00643a0 2024-03-24T10:57:41.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.877+0000 7f71e826e640 1 -- 172.21.15.94:0/2662848427 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71e0119380 con 0x7f71e00643a0 2024-03-24T10:57:41.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.878+0000 7f71e826e640 1 -- 172.21.15.94:0/2662848427 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71e00fd910 con 0x7f71e00643a0 2024-03-24T10:57:41.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.878+0000 7f71d6ffd640 1 -- 172.21.15.94:0/2662848427 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f71dc02f070 con 0x7f71e00643a0 2024-03-24T10:57:41.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.879+0000 7f71d6ffd640 1 --2- 172.21.15.94:0/2662848427 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f71cc06d750 0x7f71cc06fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:41.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.879+0000 7f71d6ffd640 1 -- 172.21.15.94:0/2662848427 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f71dc0aee00 con 0x7f71e00643a0 2024-03-24T10:57:41.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.879+0000 7f71e5809640 1 --2- 172.21.15.94:0/2662848427 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f71cc06d750 0x7f71cc06fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:41.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.879+0000 7f71e5809640 1 --2- 172.21.15.94:0/2662848427 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f71cc06d750 0x7f71cc06fc00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f71e00fc910 tx=0x7f71d0009ed0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:41.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:41.883+0000 7f71d6ffd640 1 -- 172.21.15.94:0/2662848427 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f71dc078550 con 0x7f71e00643a0 2024-03-24T10:57:41.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:41 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/1434960971' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-03-24T10:57:41.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:41 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/2241027018' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-03-24T10:57:41.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:41 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/4261415093' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-03-24T10:57:42.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.064+0000 7f71e826e640 1 -- 172.21.15.94:0/2662848427 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71e004bcc0 con 0x7f71e00643a0 2024-03-24T10:57:42.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.065+0000 7f71d6ffd640 1 -- 172.21.15.94:0/2662848427 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16865 (secure 0 0 0) 0x7f71dc07bc10 con 0x7f71e00643a0 2024-03-24T10:57:42.066 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:57:42.067 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":84,"fsid":"386110d4-e9ca-11ee-95cd-87774f69a715","created":"2024-03-24T10:36:12.011908+0000","modified":"2024-03-24T10:54:01.169537+0000","last_up_change":"2024-03-24T10:52:57.465293+0000","last_in_change":"2024-03-24T10:42:04.898193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-24T10:41:09.161003+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-24T10:43:37.651734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-24T10:43:38.660001+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58fd53f9-8ad5-4296-8804-96a98647bba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":78,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1160876432},{"type":"v1","addr":"172.21.15.94:6803","nonce":1160876432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1160876432},{"type":"v1","addr":"172.21.15.94:6805","nonce":1160876432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1160876432},{"type":"v1","addr":"172.21.15.94:6809","nonce":1160876432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1160876432},{"type":"v1","addr":"172.21.15.94:6807","nonce":1160876432}]},"public_addr":"172.21.15.94:6803/1160876432","cluster_addr":"172.21.15.94:6805/1160876432","heartbeat_back_addr":"172.21.15.94:6809/1160876432","heartbeat_front_addr":"172.21.15.94:6807/1160876432","state":["exists","up"]},{"osd":1,"uuid":"3087b482-aede-4e88-848c-3e7438eaccef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":58,"up_thru":78,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3303603595},{"type":"v1","addr":"172.21.15.94:6811","nonce":3303603595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3303603595},{"type":"v1","addr":"172.21.15.94:6813","nonce":3303603595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3303603595},{"type":"v1","addr":"172.21.15.94:6817","nonce":3303603595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3303603595},{"type":"v1","addr":"172.21.15.94:6815","nonce":3303603595}]},"public_addr":"172.21.15.94:6811/3303603595","cluster_addr":"172.21.15.94:6813/3303603595","heartbeat_back_addr":"172.21.15.94:6817/3303603595","heartbeat_front_addr":"172.21.15.94:6815/3303603595","state":["exists","up"]},{"osd":2,"uuid":"48c00627-22a6-480c-9aef-38bc326fc4dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":59,"up_from":63,"up_thru":78,"down_at":60,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3153715097},{"type":"v1","addr":"172.21.15.94:6819","nonce":3153715097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3153715097},{"type":"v1","addr":"172.21.15.94:6821","nonce":3153715097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3153715097},{"type":"v1","addr":"172.21.15.94:6825","nonce":3153715097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3153715097},{"type":"v1","addr":"172.21.15.94:6823","nonce":3153715097}]},"public_addr":"172.21.15.94:6819/3153715097","cluster_addr":"172.21.15.94:6821/3153715097","heartbeat_back_addr":"172.21.15.94:6825/3153715097","heartbeat_front_addr":"172.21.15.94:6823/3153715097","state":["exists","up"]},{"osd":3,"uuid":"910859cd-d028-45f8-8e28-af4e46be460b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":64,"up_from":68,"up_thru":78,"down_at":65,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2693214290},{"type":"v1","addr":"172.21.15.96:6801","nonce":2693214290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2693214290},{"type":"v1","addr":"172.21.15.96:6803","nonce":2693214290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2693214290},{"type":"v1","addr":"172.21.15.96:6807","nonce":2693214290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2693214290},{"type":"v1","addr":"172.21.15.96:6805","nonce":2693214290}]},"public_addr":"172.21.15.96:6801/2693214290","cluster_addr":"172.21.15.96:6803/2693214290","heartbeat_back_addr":"172.21.15.96:6807/2693214290","heartbeat_front_addr":"172.21.15.96:6805/2693214290","state":["exists","up"]},{"osd":4,"uuid":"9d37e21d-38dc-4e5b-b5ef-3828d7b25096","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":69,"up_from":73,"up_thru":78,"down_at":70,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3911504706},{"type":"v1","addr":"172.21.15.96:6809","nonce":3911504706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3911504706},{"type":"v1","addr":"172.21.15.96:6811","nonce":3911504706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3911504706},{"type":"v1","addr":"172.21.15.96:6815","nonce":3911504706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3911504706},{"type":"v1","addr":"172.21.15.96:6813","nonce":3911504706}]},"public_addr":"172.21.15.96:6809/3911504706","cluster_addr":"172.21.15.96:6811/3911504706","heartbeat_back_addr":"172.21.15.96:6815/3911504706","heartbeat_front_addr":"172.21.15.96:6813/3911504706","state":["exists","up"]},{"osd":5,"uuid":"838cf934-b936-4257-8510-3d5bbf4ef034","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":74,"up_from":78,"up_thru":78,"down_at":75,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":853588375},{"type":"v1","addr":"172.21.15.96:6817","nonce":853588375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":853588375},{"type":"v1","addr":"172.21.15.96:6819","nonce":853588375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":853588375},{"type":"v1","addr":"172.21.15.96:6823","nonce":853588375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":853588375},{"type":"v1","addr":"172.21.15.96:6821","nonce":853588375}]},"public_addr":"172.21.15.96:6817/853588375","cluster_addr":"172.21.15.96:6819/853588375","heartbeat_back_addr":"172.21.15.96:6823/853588375","heartbeat_front_addr":"172.21.15.96:6821/853588375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T10:40:12.038732+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T10:40:38.407784+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T10:41:06.693615+0000","dead_epoch":59},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T10:41:30.382956+0000","dead_epoch":64},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T10:41:56.733747+0000","dead_epoch":69},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T10:42:22.324376+0000","dead_epoch":74}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:6825/1539752208":"2024-03-25T10:53:38.029430+0000","172.21.15.94:0/403370730":"2024-03-25T10:48:35.235438+0000","172.21.15.94:0/3351913962":"2024-03-25T10:48:35.235438+0000","172.21.15.94:0/659933905":"2024-03-25T10:48:35.235438+0000","172.21.15.96:6827/1321087930":"2024-03-25T10:54:01.169508+0000","172.21.15.96:6828/1632816545":"2024-03-25T10:47:50.745695+0000","172.21.15.96:6829/1632816545":"2024-03-25T10:47:50.745695+0000","172.21.15.96:6826/1321087930":"2024-03-25T10:54:01.169508+0000","172.21.15.94:0/1027963628":"2024-03-25T10:46:29.516417+0000","172.21.15.94:6801/1013449112":"2024-03-25T10:36:48.432382+0000","172.21.15.94:0/2700857461":"2024-03-25T10:39:00.830398+0000","172.21.15.94:6800/1996608546":"2024-03-25T10:37:18.183215+0000","172.21.15.94:6826/2001471799":"2024-03-25T10:43:55.084207+0000","172.21.15.94:6801/1996608546":"2024-03-25T10:37:18.183215+0000","172.21.15.94:0/2837872991":"2024-03-25T10:36:48.432382+0000","172.21.15.94:0/2130298637":"2024-03-25T10:39:00.830398+0000","172.21.15.96:0/109124621":"2024-03-25T10:47:50.745695+0000","172.21.15.96:0/3508170171":"2024-03-25T10:47:50.745695+0000","172.21.15.94:6800/2406899470":"2024-03-25T10:39:00.830398+0000","172.21.15.94:0/206764114":"2024-03-25T10:46:29.516417+0000","172.21.15.94:6800/1013449112":"2024-03-25T10:36:48.432382+0000","172.21.15.94:6801/2184357820":"2024-03-25T10:46:29.516417+0000","172.21.15.96:0/2983574460":"2024-03-25T10:47:50.745695+0000","172.21.15.94:6800/4143126939":"2024-03-25T10:48:35.235438+0000","172.21.15.96:0/3848335112":"2024-03-25T10:47:50.745695+0000","172.21.15.94:6801/2406899470":"2024-03-25T10:39:00.830398+0000","172.21.15.94:0/629634217":"2024-03-25T10:37:18.183215+0000","172.21.15.94:0/3628417758":"2024-03-25T10:37:18.183215+0000","172.21.15.96:0/3826472619":"2024-03-25T10:47:50.745695+0000","172.21.15.94:0/2173796904":"2024-03-25T10:48:35.235438+0000","172.21.15.94:0/4215128639":"2024-03-25T10:48:35.235438+0000","172.21.15.94:0/2155575468":"2024-03-25T10:39:00.830398+0000","172.21.15.94:0/265930160":"2024-03-25T10:46:29.516417+0000","172.21.15.94:0/978791860":"2024-03-25T10:46:29.516417+0000","172.21.15.94:0/2641430900":"2024-03-25T10:36:48.432382+0000","172.21.15.94:6827/2001471799":"2024-03-25T10:43:55.084207+0000","172.21.15.94:6800/2184357820":"2024-03-25T10:46:29.516417+0000","172.21.15.94:0/4239612108":"2024-03-25T10:36:48.432382+0000","172.21.15.94:0/2661420513":"2024-03-25T10:37:18.183215+0000","172.21.15.94:0/4063210620":"2024-03-25T10:48:35.235438+0000","172.21.15.96:0/3832405488":"2024-03-25T10:47:50.745695+0000","172.21.15.96:6824/1539752208":"2024-03-25T10:53:38.029430+0000","172.21.15.94:6801/4143126939":"2024-03-25T10:48:35.235438+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-24T10:57:42.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.067+0000 7f71e826e640 1 -- 172.21.15.94:0/2662848427 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f71cc06d750 msgr2=0x7f71cc06fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:42.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.067+0000 7f71e826e640 1 --2- 172.21.15.94:0/2662848427 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f71cc06d750 0x7f71cc06fc00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f71e00fc910 tx=0x7f71d0009ed0 comp rx=0 tx=0).stop 2024-03-24T10:57:42.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.067+0000 7f71e826e640 1 -- 172.21.15.94:0/2662848427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71e00643a0 msgr2=0x7f71e00fb1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:42.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.067+0000 7f71e826e640 1 --2- 172.21.15.94:0/2662848427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71e00643a0 0x7f71e00fb1e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f71dc004930 tx=0x7f71dc009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:42.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.067+0000 7f71e826e640 1 -- 172.21.15.94:0/2662848427 shutdown_connections 2024-03-24T10:57:42.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.067+0000 7f71e826e640 1 --2- 172.21.15.94:0/2662848427 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f71cc06d750 0x7f71cc06fc00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:42.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.067+0000 7f71e826e640 1 --2- 172.21.15.94:0/2662848427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71e0102210 0x7f71e00fb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:42.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.067+0000 7f71e826e640 1 --2- 172.21.15.94:0/2662848427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71e00643a0 0x7f71e00fb1e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:42.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.067+0000 7f71e826e640 1 -- 172.21.15.94:0/2662848427 >> 172.21.15.94:0/2662848427 conn(0x7f71e0071130 msgr2=0x7f71e00f9510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:42.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.067+0000 7f71e826e640 1 -- 172.21.15.94:0/2662848427 shutdown_connections 2024-03-24T10:57:42.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.067+0000 7f71e826e640 1 -- 172.21.15.94:0/2662848427 wait complete. 2024-03-24T10:57:42.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:41 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/1434960971' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-03-24T10:57:42.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:41 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/2241027018' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-03-24T10:57:42.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:41 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/4261415093' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-03-24T10:57:42.077 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-24T10:41:09.161003+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-24T10:43:37.651734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-24T10:43:38.660001+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-24T10:57:42.078 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-24T10:57:42.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.181+0000 7f5962218640 1 -- 172.21.15.94:0/3063049004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f595c0fe490 msgr2=0x7f595c0fe900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:42.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.181+0000 7f5962218640 1 --2- 172.21.15.94:0/3063049004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f595c0fe490 0x7f595c0fe900 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f59440048d0 tx=0x7f5944009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:42.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.181+0000 7f5962218640 1 -- 172.21.15.94:0/3063049004 shutdown_connections 2024-03-24T10:57:42.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.181+0000 7f5962218640 1 --2- 172.21.15.94:0/3063049004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f595c0fe490 0x7f595c0fe900 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:42.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.181+0000 7f5962218640 1 --2- 172.21.15.94:0/3063049004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f595c1044d0 0x7f595c1048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:42.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.181+0000 7f5962218640 1 -- 172.21.15.94:0/3063049004 >> 172.21.15.94:0/3063049004 conn(0x7f595c0fa050 msgr2=0x7f595c0fc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:42.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.181+0000 7f5962218640 1 -- 172.21.15.94:0/3063049004 shutdown_connections 2024-03-24T10:57:42.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.181+0000 7f5962218640 1 -- 172.21.15.94:0/3063049004 wait complete. 2024-03-24T10:57:42.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.181+0000 7f5962218640 1 Processor -- start 2024-03-24T10:57:42.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.182+0000 7f5962218640 1 -- start start 2024-03-24T10:57:42.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.182+0000 7f5962218640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f595c0fe490 0x7f595c114ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:42.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.182+0000 7f5962218640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f595c1044d0 0x7f595c1153e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:42.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.182+0000 7f5962218640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595c1159e0 con 0x7f595c1044d0 2024-03-24T10:57:42.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.182+0000 7f5962218640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595c10f030 con 0x7f595c0fe490 2024-03-24T10:57:42.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.182+0000 7f595affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f595c1044d0 0x7f595c1153e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:42.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.182+0000 7f595affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f595c1044d0 0x7f595c1153e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40504/0 (socket says 172.21.15.94:40504) 2024-03-24T10:57:42.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.182+0000 7f595affd640 1 -- 172.21.15.94:0/2995526389 learned_addr learned my addr 172.21.15.94:0/2995526389 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:57:42.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.182+0000 7f595b7fe640 1 --2- 172.21.15.94:0/2995526389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f595c0fe490 0x7f595c114ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:42.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.182+0000 7f595affd640 1 -- 172.21.15.94:0/2995526389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f595c0fe490 msgr2=0x7f595c114ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:42.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.182+0000 7f595affd640 1 --2- 172.21.15.94:0/2995526389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f595c0fe490 0x7f595c114ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:42.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.182+0000 7f595affd640 1 -- 172.21.15.94:0/2995526389 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5944005040 con 0x7f595c1044d0 2024-03-24T10:57:42.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.182+0000 7f595affd640 1 --2- 172.21.15.94:0/2995526389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f595c1044d0 0x7f595c1153e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5944004930 tx=0x7f5944009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:42.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.183+0000 7f5953fff640 1 -- 172.21.15.94:0/2995526389 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59440094c0 con 0x7f595c1044d0 2024-03-24T10:57:42.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.183+0000 7f5953fff640 1 -- 172.21.15.94:0/2995526389 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5944032e50 con 0x7f595c1044d0 2024-03-24T10:57:42.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.183+0000 7f5953fff640 1 -- 172.21.15.94:0/2995526389 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59440433b0 con 0x7f595c1044d0 2024-03-24T10:57:42.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.183+0000 7f5962218640 1 -- 172.21.15.94:0/2995526389 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f595c10f2b0 con 0x7f595c1044d0 2024-03-24T10:57:42.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.183+0000 7f5962218640 1 -- 172.21.15.94:0/2995526389 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f595c10f770 con 0x7f595c1044d0 2024-03-24T10:57:42.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.184+0000 7f5962218640 1 -- 172.21.15.94:0/2995526389 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f595c0ffbd0 con 0x7f595c1044d0 2024-03-24T10:57:42.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.184+0000 7f5953fff640 1 -- 172.21.15.94:0/2995526389 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f594402f070 con 0x7f595c1044d0 2024-03-24T10:57:42.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.184+0000 7f5953fff640 1 --2- 172.21.15.94:0/2995526389 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f594806d750 0x7f594806fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:42.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.184+0000 7f5953fff640 1 -- 172.21.15.94:0/2995526389 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f59440aee20 con 0x7f595c1044d0 2024-03-24T10:57:42.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.184+0000 7f595b7fe640 1 --2- 172.21.15.94:0/2995526389 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f594806d750 0x7f594806fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:42.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.185+0000 7f595b7fe640 1 --2- 172.21.15.94:0/2995526389 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f594806d750 0x7f594806fc00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f594c009470 tx=0x7f594c009ed0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:42.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.189+0000 7f5953fff640 1 -- 172.21.15.94:0/2995526389 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5944078570 con 0x7f595c1044d0 2024-03-24T10:57:42.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.368+0000 7f5962218640 1 -- 172.21.15.94:0/2995526389 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f595c10b5a0 con 0x7f595c1044d0 2024-03-24T10:57:42.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.369+0000 7f5953fff640 1 -- 172.21.15.94:0/2995526389 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f594407bc30 con 0x7f595c1044d0 2024-03-24T10:57:42.369 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-03-24T10:57:42.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.371+0000 7f5962218640 1 -- 172.21.15.94:0/2995526389 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f594806d750 msgr2=0x7f594806fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:42.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.371+0000 7f5962218640 1 --2- 172.21.15.94:0/2995526389 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f594806d750 0x7f594806fc00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f594c009470 tx=0x7f594c009ed0 comp rx=0 tx=0).stop 2024-03-24T10:57:42.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.371+0000 7f5962218640 1 -- 172.21.15.94:0/2995526389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f595c1044d0 msgr2=0x7f595c1153e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:42.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.371+0000 7f5962218640 1 --2- 172.21.15.94:0/2995526389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f595c1044d0 0x7f595c1153e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5944004930 tx=0x7f5944009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:42.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.371+0000 7f5962218640 1 -- 172.21.15.94:0/2995526389 shutdown_connections 2024-03-24T10:57:42.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.371+0000 7f5962218640 1 --2- 172.21.15.94:0/2995526389 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f594806d750 0x7f594806fc00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:42.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.371+0000 7f5962218640 1 --2- 172.21.15.94:0/2995526389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f595c1044d0 0x7f595c1153e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:42.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.371+0000 7f5962218640 1 --2- 172.21.15.94:0/2995526389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f595c0fe490 0x7f595c114ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:42.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.371+0000 7f5962218640 1 -- 172.21.15.94:0/2995526389 >> 172.21.15.94:0/2995526389 conn(0x7f595c0fa050 msgr2=0x7f595c0fac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:42.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.371+0000 7f5962218640 1 -- 172.21.15.94:0/2995526389 shutdown_connections 2024-03-24T10:57:42.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.371+0000 7f5962218640 1 -- 172.21.15.94:0/2995526389 wait complete. 2024-03-24T10:57:42.382 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-24T10:57:42.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.494+0000 7f4cc046e640 1 -- 172.21.15.94:0/3733105539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cb8101140 msgr2=0x7f4cb8101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:42.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.494+0000 7f4cc046e640 1 --2- 172.21.15.94:0/3733105539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cb8101140 0x7f4cb8101510 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4ca80048d0 tx=0x7f4ca8009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:42.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.494+0000 7f4cc046e640 1 -- 172.21.15.94:0/3733105539 shutdown_connections 2024-03-24T10:57:42.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.494+0000 7f4cc046e640 1 --2- 172.21.15.94:0/3733105539 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4cb80fb110 0x7f4cb80fb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:42.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.494+0000 7f4cc046e640 1 --2- 172.21.15.94:0/3733105539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cb8101140 0x7f4cb8101510 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:42.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.494+0000 7f4cc046e640 1 -- 172.21.15.94:0/3733105539 >> 172.21.15.94:0/3733105539 conn(0x7f4cb80710b0 msgr2=0x7f4cb80714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:42.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.495+0000 7f4cc046e640 1 -- 172.21.15.94:0/3733105539 shutdown_connections 2024-03-24T10:57:42.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.495+0000 7f4cc046e640 1 -- 172.21.15.94:0/3733105539 wait complete. 2024-03-24T10:57:42.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.495+0000 7f4cc046e640 1 Processor -- start 2024-03-24T10:57:42.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.495+0000 7f4cc046e640 1 -- start start 2024-03-24T10:57:42.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.495+0000 7f4cc046e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cb80fb110 0x7f4cb8112d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:42.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.495+0000 7f4cc046e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4cb8101140 0x7f4cb8113240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:42.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.495+0000 7f4cc046e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb8113840 con 0x7f4cb80fb110 2024-03-24T10:57:42.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.495+0000 7f4cc046e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb810ce90 con 0x7f4cb8101140 2024-03-24T10:57:42.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.495+0000 7f4cbe20a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cb80fb110 0x7f4cb8112d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:42.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.495+0000 7f4cbe20a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cb80fb110 0x7f4cb8112d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40522/0 (socket says 172.21.15.94:40522) 2024-03-24T10:57:42.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.495+0000 7f4cbe20a640 1 -- 172.21.15.94:0/3305655097 learned_addr learned my addr 172.21.15.94:0/3305655097 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:57:42.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.495+0000 7f4cbda09640 1 --2- 172.21.15.94:0/3305655097 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4cb8101140 0x7f4cb8113240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:42.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.496+0000 7f4cbe20a640 1 -- 172.21.15.94:0/3305655097 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4cb8101140 msgr2=0x7f4cb8113240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:42.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.496+0000 7f4cbe20a640 1 --2- 172.21.15.94:0/3305655097 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4cb8101140 0x7f4cb8113240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:42.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.496+0000 7f4cbe20a640 1 -- 172.21.15.94:0/3305655097 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ca8005040 con 0x7f4cb80fb110 2024-03-24T10:57:42.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.496+0000 7f4cbe20a640 1 --2- 172.21.15.94:0/3305655097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cb80fb110 0x7f4cb8112d00 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4ca80041d0 tx=0x7f4ca8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:42.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.496+0000 7f4caeffd640 1 -- 172.21.15.94:0/3305655097 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ca8030870 con 0x7f4cb80fb110 2024-03-24T10:57:42.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.496+0000 7f4caeffd640 1 -- 172.21.15.94:0/3305655097 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ca80092e0 con 0x7f4cb80fb110 2024-03-24T10:57:42.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.496+0000 7f4cc046e640 1 -- 172.21.15.94:0/3305655097 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cb810d110 con 0x7f4cb80fb110 2024-03-24T10:57:42.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.496+0000 7f4caeffd640 1 -- 172.21.15.94:0/3305655097 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ca8043400 con 0x7f4cb80fb110 2024-03-24T10:57:42.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.496+0000 7f4cc046e640 1 -- 172.21.15.94:0/3305655097 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cb810d630 con 0x7f4cb80fb110 2024-03-24T10:57:42.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.497+0000 7f4cc046e640 1 -- 172.21.15.94:0/3305655097 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cb80fc800 con 0x7f4cb80fb110 2024-03-24T10:57:42.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.497+0000 7f4caeffd640 1 -- 172.21.15.94:0/3305655097 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f4ca803b420 con 0x7f4cb80fb110 2024-03-24T10:57:42.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.498+0000 7f4caeffd640 1 --2- 172.21.15.94:0/3305655097 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f4ca406d750 0x7f4ca406fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:42.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.498+0000 7f4caeffd640 1 -- 172.21.15.94:0/3305655097 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f4ca80afe50 con 0x7f4cb80fb110 2024-03-24T10:57:42.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.498+0000 7f4cbda09640 1 --2- 172.21.15.94:0/3305655097 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f4ca406d750 0x7f4ca406fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:42.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.498+0000 7f4cbda09640 1 --2- 172.21.15.94:0/3305655097 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f4ca406d750 0x7f4ca406fc00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4007fd0 tx=0x7f4cb4009e90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:42.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.503+0000 7f4caeffd640 1 -- 172.21.15.94:0/3305655097 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4ca8079620 con 0x7f4cb80fb110 2024-03-24T10:57:42.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.685+0000 7f4cc046e640 1 -- 172.21.15.94:0/3305655097 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f4cb8108220 con 0x7f4cb80fb110 2024-03-24T10:57:42.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.686+0000 7f4caeffd640 1 -- 172.21.15.94:0/3305655097 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7f4ca807cce0 con 0x7f4cb80fb110 2024-03-24T10:57:42.686 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-03-24T10:57:42.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.687+0000 7f4cc046e640 1 -- 172.21.15.94:0/3305655097 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f4ca406d750 msgr2=0x7f4ca406fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:42.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.687+0000 7f4cc046e640 1 --2- 172.21.15.94:0/3305655097 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f4ca406d750 0x7f4ca406fc00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4007fd0 tx=0x7f4cb4009e90 comp rx=0 tx=0).stop 2024-03-24T10:57:42.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.687+0000 7f4cc046e640 1 -- 172.21.15.94:0/3305655097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cb80fb110 msgr2=0x7f4cb8112d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:42.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.687+0000 7f4cc046e640 1 --2- 172.21.15.94:0/3305655097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cb80fb110 0x7f4cb8112d00 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4ca80041d0 tx=0x7f4ca8009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:42.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.687+0000 7f4cc046e640 1 -- 172.21.15.94:0/3305655097 shutdown_connections 2024-03-24T10:57:42.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.687+0000 7f4cc046e640 1 --2- 172.21.15.94:0/3305655097 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f4ca406d750 0x7f4ca406fc00 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:42.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.687+0000 7f4cc046e640 1 --2- 172.21.15.94:0/3305655097 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4cb8101140 0x7f4cb8113240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:42.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.687+0000 7f4cc046e640 1 --2- 172.21.15.94:0/3305655097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cb80fb110 0x7f4cb8112d00 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:42.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.688+0000 7f4cc046e640 1 -- 172.21.15.94:0/3305655097 >> 172.21.15.94:0/3305655097 conn(0x7f4cb80710b0 msgr2=0x7f4cb80f94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:42.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.688+0000 7f4cc046e640 1 -- 172.21.15.94:0/3305655097 shutdown_connections 2024-03-24T10:57:42.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.688+0000 7f4cc046e640 1 -- 172.21.15.94:0/3305655097 wait complete. 2024-03-24T10:57:42.698 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-24T10:57:42.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.800+0000 7f582ad54640 1 -- 172.21.15.94:0/2811409950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58240fb110 msgr2=0x7f58240fb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:42.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.800+0000 7f582ad54640 1 --2- 172.21.15.94:0/2811409950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58240fb110 0x7f58240fb560 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f58100048d0 tx=0x7f5810009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:42.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.800+0000 7f582ad54640 1 -- 172.21.15.94:0/2811409950 shutdown_connections 2024-03-24T10:57:42.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.800+0000 7f582ad54640 1 --2- 172.21.15.94:0/2811409950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58240fb110 0x7f58240fb560 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:42.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.800+0000 7f582ad54640 1 --2- 172.21.15.94:0/2811409950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5824101140 0x7f5824101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:42.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.800+0000 7f582ad54640 1 -- 172.21.15.94:0/2811409950 >> 172.21.15.94:0/2811409950 conn(0x7f58240710b0 msgr2=0x7f58240714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:42.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.800+0000 7f582ad54640 1 -- 172.21.15.94:0/2811409950 shutdown_connections 2024-03-24T10:57:42.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.800+0000 7f582ad54640 1 -- 172.21.15.94:0/2811409950 wait complete. 2024-03-24T10:57:42.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.800+0000 7f582ad54640 1 Processor -- start 2024-03-24T10:57:42.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.801+0000 7f582ad54640 1 -- start start 2024-03-24T10:57:42.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.801+0000 7f582ad54640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58240fb110 0x7f5824112d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:42.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.801+0000 7f582ad54640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5824101140 0x7f58241132d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:42.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.801+0000 7f582ad54640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5824113960 con 0x7f58240fb110 2024-03-24T10:57:42.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.801+0000 7f582ad54640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582410cf20 con 0x7f5824101140 2024-03-24T10:57:42.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.801+0000 7f5828af0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58240fb110 0x7f5824112d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:42.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.801+0000 7f5828af0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58240fb110 0x7f5824112d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40548/0 (socket says 172.21.15.94:40548) 2024-03-24T10:57:42.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.801+0000 7f5828af0640 1 -- 172.21.15.94:0/1151092123 learned_addr learned my addr 172.21.15.94:0/1151092123 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:57:42.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.801+0000 7f5823fff640 1 --2- 172.21.15.94:0/1151092123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5824101140 0x7f58241132d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:42.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.801+0000 7f5828af0640 1 -- 172.21.15.94:0/1151092123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5824101140 msgr2=0x7f58241132d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:42.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.801+0000 7f5828af0640 1 --2- 172.21.15.94:0/1151092123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5824101140 0x7f58241132d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:42.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.801+0000 7f5828af0640 1 -- 172.21.15.94:0/1151092123 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5810005040 con 0x7f58240fb110 2024-03-24T10:57:42.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.801+0000 7f5823fff640 1 --2- 172.21.15.94:0/1151092123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5824101140 0x7f58241132d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T10:57:42.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.802+0000 7f5828af0640 1 --2- 172.21.15.94:0/1151092123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58240fb110 0x7f5824112d90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f581800b790 tx=0x7f58180091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:42.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.802+0000 7f58217fa640 1 -- 172.21.15.94:0/1151092123 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f581800c700 con 0x7f58240fb110 2024-03-24T10:57:42.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.802+0000 7f582ad54640 1 -- 172.21.15.94:0/1151092123 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f582410d200 con 0x7f58240fb110 2024-03-24T10:57:42.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.802+0000 7f58217fa640 1 -- 172.21.15.94:0/1151092123 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f581800cd30 con 0x7f58240fb110 2024-03-24T10:57:42.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.802+0000 7f58217fa640 1 -- 172.21.15.94:0/1151092123 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5818013370 con 0x7f58240fb110 2024-03-24T10:57:42.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.802+0000 7f582ad54640 1 -- 172.21.15.94:0/1151092123 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f582410d720 con 0x7f58240fb110 2024-03-24T10:57:42.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.803+0000 7f582ad54640 1 -- 172.21.15.94:0/1151092123 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58240fc800 con 0x7f58240fb110 2024-03-24T10:57:42.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.803+0000 7f58217fa640 1 -- 172.21.15.94:0/1151092123 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f5818012070 con 0x7f58240fb110 2024-03-24T10:57:42.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.804+0000 7f58217fa640 1 --2- 172.21.15.94:0/1151092123 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f581406d760 0x7f581406fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:42.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.804+0000 7f58217fa640 1 -- 172.21.15.94:0/1151092123 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f581808cc10 con 0x7f58240fb110 2024-03-24T10:57:42.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.804+0000 7f5823fff640 1 --2- 172.21.15.94:0/1151092123 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f581406d760 0x7f581406fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:42.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.804+0000 7f5823fff640 1 --2- 172.21.15.94:0/1151092123 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f581406d760 0x7f581406fc10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5810009fa0 tx=0x7f5810009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:42.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:42.809+0000 7f58217fa640 1 -- 172.21.15.94:0/1151092123 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5818056360 con 0x7f58240fb110 2024-03-24T10:57:42.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:42 smithi094 ceph-mon[93957]: pgmap v311: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:57:42.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:42 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/2662848427' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T10:57:42.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:42 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/2995526389' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T10:57:42.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:42 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/3305655097' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-24T10:57:43.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:42 smithi096 ceph-mon[84258]: pgmap v311: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-24T10:57:43.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:42 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/2662848427' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T10:57:43.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:42 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/2995526389' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T10:57:43.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:42 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/3305655097' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-24T10:57:43.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.104+0000 7f582ad54640 1 -- 172.21.15.94:0/1151092123 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f582404bcc0 con 0x7f58240fb110 2024-03-24T10:57:43.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.105+0000 7f58217fa640 1 -- 172.21.15.94:0/1151092123 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7f5818059a20 con 0x7f58240fb110 2024-03-24T10:57:43.105 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-03-24T10:57:43.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.107+0000 7f582ad54640 1 -- 172.21.15.94:0/1151092123 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f581406d760 msgr2=0x7f581406fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:43.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.107+0000 7f582ad54640 1 --2- 172.21.15.94:0/1151092123 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f581406d760 0x7f581406fc10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5810009fa0 tx=0x7f5810009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:43.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.107+0000 7f582ad54640 1 -- 172.21.15.94:0/1151092123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58240fb110 msgr2=0x7f5824112d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:43.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.107+0000 7f582ad54640 1 --2- 172.21.15.94:0/1151092123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58240fb110 0x7f5824112d90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f581800b790 tx=0x7f58180091d0 comp rx=0 tx=0).stop 2024-03-24T10:57:43.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.107+0000 7f582ad54640 1 -- 172.21.15.94:0/1151092123 shutdown_connections 2024-03-24T10:57:43.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.107+0000 7f582ad54640 1 --2- 172.21.15.94:0/1151092123 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f581406d760 0x7f581406fc10 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:43.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.107+0000 7f582ad54640 1 --2- 172.21.15.94:0/1151092123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5824101140 0x7f58241132d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:43.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.107+0000 7f582ad54640 1 --2- 172.21.15.94:0/1151092123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58240fb110 0x7f5824112d90 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:43.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.107+0000 7f582ad54640 1 -- 172.21.15.94:0/1151092123 >> 172.21.15.94:0/1151092123 conn(0x7f58240710b0 msgr2=0x7f58240f9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:43.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.107+0000 7f582ad54640 1 -- 172.21.15.94:0/1151092123 shutdown_connections 2024-03-24T10:57:43.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.107+0000 7f582ad54640 1 -- 172.21.15.94:0/1151092123 wait complete. 2024-03-24T10:57:43.117 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-24T10:57:43.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.225+0000 7f11ad35d640 1 -- 172.21.15.94:0/2610482239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11a8102210 msgr2=0x7f11a81025e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:43.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.225+0000 7f11ad35d640 1 --2- 172.21.15.94:0/2610482239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11a8102210 0x7f11a81025e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f11980048d0 tx=0x7f1198009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:43.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.225+0000 7f11ad35d640 1 -- 172.21.15.94:0/2610482239 shutdown_connections 2024-03-24T10:57:43.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.225+0000 7f11ad35d640 1 --2- 172.21.15.94:0/2610482239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11a80643a0 0x7f11a80647f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:43.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.225+0000 7f11ad35d640 1 --2- 172.21.15.94:0/2610482239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11a8102210 0x7f11a81025e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:43.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.225+0000 7f11ad35d640 1 -- 172.21.15.94:0/2610482239 >> 172.21.15.94:0/2610482239 conn(0x7f11a80710b0 msgr2=0x7f11a80714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:43.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.225+0000 7f11ad35d640 1 -- 172.21.15.94:0/2610482239 shutdown_connections 2024-03-24T10:57:43.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.226+0000 7f11ad35d640 1 -- 172.21.15.94:0/2610482239 wait complete. 2024-03-24T10:57:43.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.226+0000 7f11ad35d640 1 Processor -- start 2024-03-24T10:57:43.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.226+0000 7f11ad35d640 1 -- start start 2024-03-24T10:57:43.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.226+0000 7f11ad35d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11a80643a0 0x7f11a8112ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:43.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.226+0000 7f11ad35d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11a8102210 0x7f11a8113220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:43.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.226+0000 7f11ad35d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a8113820 con 0x7f11a80643a0 2024-03-24T10:57:43.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.226+0000 7f11ad35d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a810ce70 con 0x7f11a8102210 2024-03-24T10:57:43.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.227+0000 7f11a6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11a80643a0 0x7f11a8112ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:43.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.227+0000 7f11a6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11a80643a0 0x7f11a8112ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40560/0 (socket says 172.21.15.94:40560) 2024-03-24T10:57:43.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.227+0000 7f11a6ffd640 1 -- 172.21.15.94:0/2136251201 learned_addr learned my addr 172.21.15.94:0/2136251201 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:57:43.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.227+0000 7f11a6ffd640 1 -- 172.21.15.94:0/2136251201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11a8102210 msgr2=0x7f11a8113220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:43.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.227+0000 7f11a6ffd640 1 --2- 172.21.15.94:0/2136251201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11a8102210 0x7f11a8113220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:43.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.227+0000 7f11a6ffd640 1 -- 172.21.15.94:0/2136251201 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1198005040 con 0x7f11a80643a0 2024-03-24T10:57:43.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.227+0000 7f11a6ffd640 1 --2- 172.21.15.94:0/2136251201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11a80643a0 0x7f11a8112ce0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1198004930 tx=0x7f1198009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:43.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.227+0000 7f119f7fe640 1 -- 172.21.15.94:0/2136251201 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11980094c0 con 0x7f11a80643a0 2024-03-24T10:57:43.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.227+0000 7f11ad35d640 1 -- 172.21.15.94:0/2136251201 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11a810d0f0 con 0x7f11a80643a0 2024-03-24T10:57:43.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.227+0000 7f119f7fe640 1 -- 172.21.15.94:0/2136251201 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1198032e50 con 0x7f11a80643a0 2024-03-24T10:57:43.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.227+0000 7f119f7fe640 1 -- 172.21.15.94:0/2136251201 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11980433b0 con 0x7f11a80643a0 2024-03-24T10:57:43.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.227+0000 7f11ad35d640 1 -- 172.21.15.94:0/2136251201 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11a810d5b0 con 0x7f11a80643a0 2024-03-24T10:57:43.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.228+0000 7f11ad35d640 1 -- 172.21.15.94:0/2136251201 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11a80fd910 con 0x7f11a80643a0 2024-03-24T10:57:43.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.229+0000 7f119f7fe640 1 -- 172.21.15.94:0/2136251201 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f119802f070 con 0x7f11a80643a0 2024-03-24T10:57:43.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.229+0000 7f119f7fe640 1 --2- 172.21.15.94:0/2136251201 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f119406d750 0x7f119406fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:43.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.229+0000 7f119f7fe640 1 -- 172.21.15.94:0/2136251201 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f11980aee20 con 0x7f11a80643a0 2024-03-24T10:57:43.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.229+0000 7f119ffff640 1 --2- 172.21.15.94:0/2136251201 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f119406d750 0x7f119406fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:43.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.229+0000 7f119ffff640 1 --2- 172.21.15.94:0/2136251201 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f119406d750 0x7f119406fc00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1190009af0 tx=0x7f1190009cd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:43.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.234+0000 7f119f7fe640 1 -- 172.21.15.94:0/2136251201 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1198078570 con 0x7f11a80643a0 2024-03-24T10:57:43.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.425+0000 7f11ad35d640 1 -- 172.21.15.94:0/2136251201 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f11a810e350 con 0x7f11a80643a0 2024-03-24T10:57:43.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.426+0000 7f119f7fe640 1 -- 172.21.15.94:0/2136251201 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 43 entries v84) v1 ==== 81+0+2541 (secure 0 0 0) 0x7f119807bc30 con 0x7f11a80643a0 2024-03-24T10:57:43.426 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.96:6825/1539752208 2024-03-25T10:53:38.029430+0000 2024-03-24T10:57:43.426 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/403370730 2024-03-25T10:48:35.235438+0000 2024-03-24T10:57:43.426 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/3351913962 2024-03-25T10:48:35.235438+0000 2024-03-24T10:57:43.426 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/659933905 2024-03-25T10:48:35.235438+0000 2024-03-24T10:57:43.426 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.96:6827/1321087930 2024-03-25T10:54:01.169508+0000 2024-03-24T10:57:43.427 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.96:6828/1632816545 2024-03-25T10:47:50.745695+0000 2024-03-24T10:57:43.427 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.96:6829/1632816545 2024-03-25T10:47:50.745695+0000 2024-03-24T10:57:43.427 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.96:6826/1321087930 2024-03-25T10:54:01.169508+0000 2024-03-24T10:57:43.427 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/1027963628 2024-03-25T10:46:29.516417+0000 2024-03-24T10:57:43.427 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6801/1013449112 2024-03-25T10:36:48.432382+0000 2024-03-24T10:57:43.427 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/2700857461 2024-03-25T10:39:00.830398+0000 2024-03-24T10:57:43.427 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6800/1996608546 2024-03-25T10:37:18.183215+0000 2024-03-24T10:57:43.427 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6826/2001471799 2024-03-25T10:43:55.084207+0000 2024-03-24T10:57:43.427 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6801/1996608546 2024-03-25T10:37:18.183215+0000 2024-03-24T10:57:43.427 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/2837872991 2024-03-25T10:36:48.432382+0000 2024-03-24T10:57:43.427 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/2130298637 2024-03-25T10:39:00.830398+0000 2024-03-24T10:57:43.427 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.96:0/109124621 2024-03-25T10:47:50.745695+0000 2024-03-24T10:57:43.428 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.96:0/3508170171 2024-03-25T10:47:50.745695+0000 2024-03-24T10:57:43.428 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6800/2406899470 2024-03-25T10:39:00.830398+0000 2024-03-24T10:57:43.428 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/206764114 2024-03-25T10:46:29.516417+0000 2024-03-24T10:57:43.428 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6800/1013449112 2024-03-25T10:36:48.432382+0000 2024-03-24T10:57:43.428 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6801/2184357820 2024-03-25T10:46:29.516417+0000 2024-03-24T10:57:43.428 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.96:0/2983574460 2024-03-25T10:47:50.745695+0000 2024-03-24T10:57:43.428 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6800/4143126939 2024-03-25T10:48:35.235438+0000 2024-03-24T10:57:43.428 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.96:0/3848335112 2024-03-25T10:47:50.745695+0000 2024-03-24T10:57:43.428 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6801/2406899470 2024-03-25T10:39:00.830398+0000 2024-03-24T10:57:43.428 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/629634217 2024-03-25T10:37:18.183215+0000 2024-03-24T10:57:43.428 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/3628417758 2024-03-25T10:37:18.183215+0000 2024-03-24T10:57:43.428 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.96:0/3826472619 2024-03-25T10:47:50.745695+0000 2024-03-24T10:57:43.428 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/2173796904 2024-03-25T10:48:35.235438+0000 2024-03-24T10:57:43.428 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/4215128639 2024-03-25T10:48:35.235438+0000 2024-03-24T10:57:43.429 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/2155575468 2024-03-25T10:39:00.830398+0000 2024-03-24T10:57:43.429 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/265930160 2024-03-25T10:46:29.516417+0000 2024-03-24T10:57:43.429 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/978791860 2024-03-25T10:46:29.516417+0000 2024-03-24T10:57:43.429 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/2641430900 2024-03-25T10:36:48.432382+0000 2024-03-24T10:57:43.429 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6827/2001471799 2024-03-25T10:43:55.084207+0000 2024-03-24T10:57:43.429 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6800/2184357820 2024-03-25T10:46:29.516417+0000 2024-03-24T10:57:43.429 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/4239612108 2024-03-25T10:36:48.432382+0000 2024-03-24T10:57:43.429 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/2661420513 2024-03-25T10:37:18.183215+0000 2024-03-24T10:57:43.429 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/4063210620 2024-03-25T10:48:35.235438+0000 2024-03-24T10:57:43.429 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.96:0/3832405488 2024-03-25T10:47:50.745695+0000 2024-03-24T10:57:43.429 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.96:6824/1539752208 2024-03-25T10:53:38.029430+0000 2024-03-24T10:57:43.429 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6801/4143126939 2024-03-25T10:48:35.235438+0000 2024-03-24T10:57:43.430 INFO:teuthology.orchestra.run.smithi094.stderr:listed 43 entries 2024-03-24T10:57:43.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.428+0000 7f11ad35d640 1 -- 172.21.15.94:0/2136251201 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f119406d750 msgr2=0x7f119406fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:43.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.428+0000 7f11ad35d640 1 --2- 172.21.15.94:0/2136251201 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f119406d750 0x7f119406fc00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1190009af0 tx=0x7f1190009cd0 comp rx=0 tx=0).stop 2024-03-24T10:57:43.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.428+0000 7f11ad35d640 1 -- 172.21.15.94:0/2136251201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11a80643a0 msgr2=0x7f11a8112ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:43.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.428+0000 7f11ad35d640 1 --2- 172.21.15.94:0/2136251201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11a80643a0 0x7f11a8112ce0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1198004930 tx=0x7f1198009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:43.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.428+0000 7f11ad35d640 1 -- 172.21.15.94:0/2136251201 shutdown_connections 2024-03-24T10:57:43.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.428+0000 7f11ad35d640 1 --2- 172.21.15.94:0/2136251201 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f119406d750 0x7f119406fc00 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:43.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.428+0000 7f11ad35d640 1 --2- 172.21.15.94:0/2136251201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11a8102210 0x7f11a8113220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:43.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.428+0000 7f11ad35d640 1 --2- 172.21.15.94:0/2136251201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11a80643a0 0x7f11a8112ce0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:43.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.428+0000 7f11ad35d640 1 -- 172.21.15.94:0/2136251201 >> 172.21.15.94:0/2136251201 conn(0x7f11a80710b0 msgr2=0x7f11a80f9480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:43.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.428+0000 7f11ad35d640 1 -- 172.21.15.94:0/2136251201 shutdown_connections 2024-03-24T10:57:43.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.428+0000 7f11ad35d640 1 -- 172.21.15.94:0/2136251201 wait complete. 2024-03-24T10:57:43.439 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-24T10:57:43.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T10:57:43.439 DEBUG:teuthology.orchestra.run.smithi094:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-24T10:57:43.470 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi094.front.sepia.ceph.com 2024-03-24T10:57:43.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T10:57:43.471 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-24T10:57:43.542 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-24T10:57:43.542 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-24T10:57:43.562 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-24T10:57:43.562 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-24T10:57:43.621 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-24T10:57:43.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.720+0000 7f1d3255d640 1 -- 172.21.15.94:0/2429350266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d2c1021b0 msgr2=0x7f1d2c102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:43.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.720+0000 7f1d3255d640 1 --2- 172.21.15.94:0/2429350266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d2c1021b0 0x7f1d2c102580 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c0048d0 tx=0x7f1d1c009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:43.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.720+0000 7f1d3255d640 1 -- 172.21.15.94:0/2429350266 shutdown_connections 2024-03-24T10:57:43.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.720+0000 7f1d3255d640 1 --2- 172.21.15.94:0/2429350266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d2c102b50 0x7f1d2c0fc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:43.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.720+0000 7f1d3255d640 1 --2- 172.21.15.94:0/2429350266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d2c1021b0 0x7f1d2c102580 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:43.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.720+0000 7f1d3255d640 1 -- 172.21.15.94:0/2429350266 >> 172.21.15.94:0/2429350266 conn(0x7f1d2c0f7da0 msgr2=0x7f1d2c0fa1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:43.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.720+0000 7f1d3255d640 1 -- 172.21.15.94:0/2429350266 shutdown_connections 2024-03-24T10:57:43.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.720+0000 7f1d3255d640 1 -- 172.21.15.94:0/2429350266 wait complete. 2024-03-24T10:57:43.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.720+0000 7f1d3255d640 1 Processor -- start 2024-03-24T10:57:43.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.720+0000 7f1d3255d640 1 -- start start 2024-03-24T10:57:43.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.721+0000 7f1d3255d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d2c1021b0 0x7f1d2c06f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:43.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.721+0000 7f1d3255d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d2c102b50 0x7f1d2c06fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:43.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.721+0000 7f1d3255d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2c0716b0 con 0x7f1d2c102b50 2024-03-24T10:57:43.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.721+0000 7f1d3255d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2c070250 con 0x7f1d2c1021b0 2024-03-24T10:57:43.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.721+0000 7f1d2b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d2c102b50 0x7f1d2c06fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:43.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.721+0000 7f1d2b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d2c102b50 0x7f1d2c06fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40576/0 (socket says 172.21.15.94:40576) 2024-03-24T10:57:43.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.721+0000 7f1d2b7fe640 1 -- 172.21.15.94:0/330414270 learned_addr learned my addr 172.21.15.94:0/330414270 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:57:43.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.721+0000 7f1d2bfff640 1 --2- 172.21.15.94:0/330414270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d2c1021b0 0x7f1d2c06f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:43.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.721+0000 7f1d2b7fe640 1 -- 172.21.15.94:0/330414270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d2c1021b0 msgr2=0x7f1d2c06f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:43.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.721+0000 7f1d2b7fe640 1 --2- 172.21.15.94:0/330414270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d2c1021b0 0x7f1d2c06f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:43.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.721+0000 7f1d2b7fe640 1 -- 172.21.15.94:0/330414270 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d1c005040 con 0x7f1d2c102b50 2024-03-24T10:57:43.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.721+0000 7f1d2b7fe640 1 --2- 172.21.15.94:0/330414270 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d2c102b50 0x7f1d2c06fd10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1d20002a50 tx=0x7f1d20004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:43.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.721+0000 7f1d28ff9640 1 -- 172.21.15.94:0/330414270 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d20007c30 con 0x7f1d2c102b50 2024-03-24T10:57:43.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.721+0000 7f1d28ff9640 1 -- 172.21.15.94:0/330414270 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1d2000f040 con 0x7f1d2c102b50 2024-03-24T10:57:43.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.721+0000 7f1d3255d640 1 -- 172.21.15.94:0/330414270 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d2c070530 con 0x7f1d2c102b50 2024-03-24T10:57:43.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.721+0000 7f1d28ff9640 1 -- 172.21.15.94:0/330414270 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d2000db50 con 0x7f1d2c102b50 2024-03-24T10:57:43.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.721+0000 7f1d3255d640 1 -- 172.21.15.94:0/330414270 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d2c119420 con 0x7f1d2c102b50 2024-03-24T10:57:43.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.722+0000 7f1d3255d640 1 -- 172.21.15.94:0/330414270 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d2c0fd920 con 0x7f1d2c102b50 2024-03-24T10:57:43.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.723+0000 7f1d28ff9640 1 -- 172.21.15.94:0/330414270 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f1d2001c400 con 0x7f1d2c102b50 2024-03-24T10:57:43.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.723+0000 7f1d28ff9640 1 --2- 172.21.15.94:0/330414270 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1d1406d750 0x7f1d1406fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:43.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.723+0000 7f1d28ff9640 1 -- 172.21.15.94:0/330414270 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f1d2008c7f0 con 0x7f1d2c102b50 2024-03-24T10:57:43.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.723+0000 7f1d2bfff640 1 --2- 172.21.15.94:0/330414270 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1d1406d750 0x7f1d1406fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:43.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.724+0000 7f1d2bfff640 1 --2- 172.21.15.94:0/330414270 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1d1406d750 0x7f1d1406fc00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c003080 tx=0x7f1d1c009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:43.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.728+0000 7f1d28ff9640 1 -- 172.21.15.94:0/330414270 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1d20055ff0 con 0x7f1d2c102b50 2024-03-24T10:57:43.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.910+0000 7f1d3255d640 1 -- 172.21.15.94:0/330414270 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d2c109370 con 0x7f1d2c102b50 2024-03-24T10:57:43.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.911+0000 7f1d28ff9640 1 -- 172.21.15.94:0/330414270 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16865 (secure 0 0 0) 0x7f1d200596b0 con 0x7f1d2c102b50 2024-03-24T10:57:43.912 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-03-24T10:57:43.912 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":84,"fsid":"386110d4-e9ca-11ee-95cd-87774f69a715","created":"2024-03-24T10:36:12.011908+0000","modified":"2024-03-24T10:54:01.169537+0000","last_up_change":"2024-03-24T10:52:57.465293+0000","last_in_change":"2024-03-24T10:42:04.898193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-24T10:41:09.161003+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-24T10:43:37.651734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-24T10:43:38.660001+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58fd53f9-8ad5-4296-8804-96a98647bba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":78,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1160876432},{"type":"v1","addr":"172.21.15.94:6803","nonce":1160876432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1160876432},{"type":"v1","addr":"172.21.15.94:6805","nonce":1160876432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1160876432},{"type":"v1","addr":"172.21.15.94:6809","nonce":1160876432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1160876432},{"type":"v1","addr":"172.21.15.94:6807","nonce":1160876432}]},"public_addr":"172.21.15.94:6803/1160876432","cluster_addr":"172.21.15.94:6805/1160876432","heartbeat_back_addr":"172.21.15.94:6809/1160876432","heartbeat_front_addr":"172.21.15.94:6807/1160876432","state":["exists","up"]},{"osd":1,"uuid":"3087b482-aede-4e88-848c-3e7438eaccef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":58,"up_thru":78,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3303603595},{"type":"v1","addr":"172.21.15.94:6811","nonce":3303603595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3303603595},{"type":"v1","addr":"172.21.15.94:6813","nonce":3303603595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3303603595},{"type":"v1","addr":"172.21.15.94:6817","nonce":3303603595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3303603595},{"type":"v1","addr":"172.21.15.94:6815","nonce":3303603595}]},"public_addr":"172.21.15.94:6811/3303603595","cluster_addr":"172.21.15.94:6813/3303603595","heartbeat_back_addr":"172.21.15.94:6817/3303603595","heartbeat_front_addr":"172.21.15.94:6815/3303603595","state":["exists","up"]},{"osd":2,"uuid":"48c00627-22a6-480c-9aef-38bc326fc4dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":59,"up_from":63,"up_thru":78,"down_at":60,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3153715097},{"type":"v1","addr":"172.21.15.94:6819","nonce":3153715097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3153715097},{"type":"v1","addr":"172.21.15.94:6821","nonce":3153715097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3153715097},{"type":"v1","addr":"172.21.15.94:6825","nonce":3153715097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3153715097},{"type":"v1","addr":"172.21.15.94:6823","nonce":3153715097}]},"public_addr":"172.21.15.94:6819/3153715097","cluster_addr":"172.21.15.94:6821/3153715097","heartbeat_back_addr":"172.21.15.94:6825/3153715097","heartbeat_front_addr":"172.21.15.94:6823/3153715097","state":["exists","up"]},{"osd":3,"uuid":"910859cd-d028-45f8-8e28-af4e46be460b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":64,"up_from":68,"up_thru":78,"down_at":65,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2693214290},{"type":"v1","addr":"172.21.15.96:6801","nonce":2693214290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2693214290},{"type":"v1","addr":"172.21.15.96:6803","nonce":2693214290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2693214290},{"type":"v1","addr":"172.21.15.96:6807","nonce":2693214290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2693214290},{"type":"v1","addr":"172.21.15.96:6805","nonce":2693214290}]},"public_addr":"172.21.15.96:6801/2693214290","cluster_addr":"172.21.15.96:6803/2693214290","heartbeat_back_addr":"172.21.15.96:6807/2693214290","heartbeat_front_addr":"172.21.15.96:6805/2693214290","state":["exists","up"]},{"osd":4,"uuid":"9d37e21d-38dc-4e5b-b5ef-3828d7b25096","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":69,"up_from":73,"up_thru":78,"down_at":70,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3911504706},{"type":"v1","addr":"172.21.15.96:6809","nonce":3911504706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3911504706},{"type":"v1","addr":"172.21.15.96:6811","nonce":3911504706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3911504706},{"type":"v1","addr":"172.21.15.96:6815","nonce":3911504706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3911504706},{"type":"v1","addr":"172.21.15.96:6813","nonce":3911504706}]},"public_addr":"172.21.15.96:6809/3911504706","cluster_addr":"172.21.15.96:6811/3911504706","heartbeat_back_addr":"172.21.15.96:6815/3911504706","heartbeat_front_addr":"172.21.15.96:6813/3911504706","state":["exists","up"]},{"osd":5,"uuid":"838cf934-b936-4257-8510-3d5bbf4ef034","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":74,"up_from":78,"up_thru":78,"down_at":75,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":853588375},{"type":"v1","addr":"172.21.15.96:6817","nonce":853588375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":853588375},{"type":"v1","addr":"172.21.15.96:6819","nonce":853588375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":853588375},{"type":"v1","addr":"172.21.15.96:6823","nonce":853588375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":853588375},{"type":"v1","addr":"172.21.15.96:6821","nonce":853588375}]},"public_addr":"172.21.15.96:6817/853588375","cluster_addr":"172.21.15.96:6819/853588375","heartbeat_back_addr":"172.21.15.96:6823/853588375","heartbeat_front_addr":"172.21.15.96:6821/853588375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T10:40:12.038732+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T10:40:38.407784+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T10:41:06.693615+0000","dead_epoch":59},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T10:41:30.382956+0000","dead_epoch":64},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T10:41:56.733747+0000","dead_epoch":69},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T10:42:22.324376+0000","dead_epoch":74}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:6825/1539752208":"2024-03-25T10:53:38.029430+0000","172.21.15.94:0/403370730":"2024-03-25T10:48:35.235438+0000","172.21.15.94:0/3351913962":"2024-03-25T10:48:35.235438+0000","172.21.15.94:0/659933905":"2024-03-25T10:48:35.235438+0000","172.21.15.96:6827/1321087930":"2024-03-25T10:54:01.169508+0000","172.21.15.96:6828/1632816545":"2024-03-25T10:47:50.745695+0000","172.21.15.96:6829/1632816545":"2024-03-25T10:47:50.745695+0000","172.21.15.96:6826/1321087930":"2024-03-25T10:54:01.169508+0000","172.21.15.94:0/1027963628":"2024-03-25T10:46:29.516417+0000","172.21.15.94:6801/1013449112":"2024-03-25T10:36:48.432382+0000","172.21.15.94:0/2700857461":"2024-03-25T10:39:00.830398+0000","172.21.15.94:6800/1996608546":"2024-03-25T10:37:18.183215+0000","172.21.15.94:6826/2001471799":"2024-03-25T10:43:55.084207+0000","172.21.15.94:6801/1996608546":"2024-03-25T10:37:18.183215+0000","172.21.15.94:0/2837872991":"2024-03-25T10:36:48.432382+0000","172.21.15.94:0/2130298637":"2024-03-25T10:39:00.830398+0000","172.21.15.96:0/109124621":"2024-03-25T10:47:50.745695+0000","172.21.15.96:0/3508170171":"2024-03-25T10:47:50.745695+0000","172.21.15.94:6800/2406899470":"2024-03-25T10:39:00.830398+0000","172.21.15.94:0/206764114":"2024-03-25T10:46:29.516417+0000","172.21.15.94:6800/1013449112":"2024-03-25T10:36:48.432382+0000","172.21.15.94:6801/2184357820":"2024-03-25T10:46:29.516417+0000","172.21.15.96:0/2983574460":"2024-03-25T10:47:50.745695+0000","172.21.15.94:6800/4143126939":"2024-03-25T10:48:35.235438+0000","172.21.15.96:0/3848335112":"2024-03-25T10:47:50.745695+0000","172.21.15.94:6801/2406899470":"2024-03-25T10:39:00.830398+0000","172.21.15.94:0/629634217":"2024-03-25T10:37:18.183215+0000","172.21.15.94:0/3628417758":"2024-03-25T10:37:18.183215+0000","172.21.15.96:0/3826472619":"2024-03-25T10:47:50.745695+0000","172.21.15.94:0/2173796904":"2024-03-25T10:48:35.235438+0000","172.21.15.94:0/4215128639":"2024-03-25T10:48:35.235438+0000","172.21.15.94:0/2155575468":"2024-03-25T10:39:00.830398+0000","172.21.15.94:0/265930160":"2024-03-25T10:46:29.516417+0000","172.21.15.94:0/978791860":"2024-03-25T10:46:29.516417+0000","172.21.15.94:0/2641430900":"2024-03-25T10:36:48.432382+0000","172.21.15.94:6827/2001471799":"2024-03-25T10:43:55.084207+0000","172.21.15.94:6800/2184357820":"2024-03-25T10:46:29.516417+0000","172.21.15.94:0/4239612108":"2024-03-25T10:36:48.432382+0000","172.21.15.94:0/2661420513":"2024-03-25T10:37:18.183215+0000","172.21.15.94:0/4063210620":"2024-03-25T10:48:35.235438+0000","172.21.15.96:0/3832405488":"2024-03-25T10:47:50.745695+0000","172.21.15.96:6824/1539752208":"2024-03-25T10:53:38.029430+0000","172.21.15.94:6801/4143126939":"2024-03-25T10:48:35.235438+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-24T10:57:43.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.913+0000 7f1d3255d640 1 -- 172.21.15.94:0/330414270 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1d1406d750 msgr2=0x7f1d1406fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:43.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.913+0000 7f1d3255d640 1 --2- 172.21.15.94:0/330414270 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1d1406d750 0x7f1d1406fc00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c003080 tx=0x7f1d1c009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:43.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.914+0000 7f1d3255d640 1 -- 172.21.15.94:0/330414270 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d2c102b50 msgr2=0x7f1d2c06fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:43.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.914+0000 7f1d3255d640 1 --2- 172.21.15.94:0/330414270 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d2c102b50 0x7f1d2c06fd10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1d20002a50 tx=0x7f1d20004600 comp rx=0 tx=0).stop 2024-03-24T10:57:43.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.914+0000 7f1d3255d640 1 -- 172.21.15.94:0/330414270 shutdown_connections 2024-03-24T10:57:43.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.914+0000 7f1d3255d640 1 --2- 172.21.15.94:0/330414270 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f1d1406d750 0x7f1d1406fc00 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:43.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.914+0000 7f1d3255d640 1 --2- 172.21.15.94:0/330414270 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d2c102b50 0x7f1d2c06fd10 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:43.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.914+0000 7f1d3255d640 1 --2- 172.21.15.94:0/330414270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d2c1021b0 0x7f1d2c06f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:43.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.914+0000 7f1d3255d640 1 -- 172.21.15.94:0/330414270 >> 172.21.15.94:0/330414270 conn(0x7f1d2c0f7da0 msgr2=0x7f1d2c0f8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:43.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.914+0000 7f1d3255d640 1 -- 172.21.15.94:0/330414270 shutdown_connections 2024-03-24T10:57:43.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:43.914+0000 7f1d3255d640 1 -- 172.21.15.94:0/330414270 wait complete. 2024-03-24T10:57:43.925 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-24T10:41:09.161003+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-24T10:43:37.651734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-24T10:43:38.660001+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-24T10:57:43.925 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-24T10:57:43.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:43 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/1151092123' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-24T10:57:43.997 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:43 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/2136251201' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-24T10:57:44.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.027+0000 7f45e9d5d640 1 -- 172.21.15.94:0/2185091333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45e40fd420 msgr2=0x7f45e40fd7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:44.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.027+0000 7f45e9d5d640 1 --2- 172.21.15.94:0/2185091333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45e40fd420 0x7f45e40fd7f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f45d40048d0 tx=0x7f45d4009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:44.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.027+0000 7f45e9d5d640 1 -- 172.21.15.94:0/2185091333 shutdown_connections 2024-03-24T10:57:44.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.027+0000 7f45e9d5d640 1 --2- 172.21.15.94:0/2185091333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45e40fdd30 0x7f45e4106490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:44.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.027+0000 7f45e9d5d640 1 --2- 172.21.15.94:0/2185091333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45e40fd420 0x7f45e40fd7f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:44.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.027+0000 7f45e9d5d640 1 -- 172.21.15.94:0/2185091333 >> 172.21.15.94:0/2185091333 conn(0x7f45e40f6ea0 msgr2=0x7f45e40f92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:44.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.027+0000 7f45e9d5d640 1 -- 172.21.15.94:0/2185091333 shutdown_connections 2024-03-24T10:57:44.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.027+0000 7f45e9d5d640 1 -- 172.21.15.94:0/2185091333 wait complete. 2024-03-24T10:57:44.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.028+0000 7f45e9d5d640 1 Processor -- start 2024-03-24T10:57:44.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.028+0000 7f45e9d5d640 1 -- start start 2024-03-24T10:57:44.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.028+0000 7f45e9d5d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45e40fd420 0x7f45e40fcd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:44.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.028+0000 7f45e9d5d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45e40fdd30 0x7f45e40fb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:44.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.028+0000 7f45e9d5d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e40fb820 con 0x7f45e40fd420 2024-03-24T10:57:44.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.028+0000 7f45e9d5d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e40fb990 con 0x7f45e40fdd30 2024-03-24T10:57:44.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.028+0000 7f45e37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45e40fd420 0x7f45e40fcd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:44.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.028+0000 7f45e37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45e40fd420 0x7f45e40fcd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40586/0 (socket says 172.21.15.94:40586) 2024-03-24T10:57:44.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.028+0000 7f45e37fe640 1 -- 172.21.15.94:0/2450249690 learned_addr learned my addr 172.21.15.94:0/2450249690 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:57:44.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.028+0000 7f45e2ffd640 1 --2- 172.21.15.94:0/2450249690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45e40fdd30 0x7f45e40fb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:44.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.028+0000 7f45e37fe640 1 -- 172.21.15.94:0/2450249690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45e40fdd30 msgr2=0x7f45e40fb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:44.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.028+0000 7f45e37fe640 1 --2- 172.21.15.94:0/2450249690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45e40fdd30 0x7f45e40fb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:44.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.028+0000 7f45e37fe640 1 -- 172.21.15.94:0/2450249690 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d4005040 con 0x7f45e40fd420 2024-03-24T10:57:44.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.029+0000 7f45e37fe640 1 --2- 172.21.15.94:0/2450249690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45e40fd420 0x7f45e40fcd70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f45d4004930 tx=0x7f45d4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:44.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.029+0000 7f45cbfff640 1 -- 172.21.15.94:0/2450249690 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45d40092b0 con 0x7f45e40fd420 2024-03-24T10:57:44.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.029+0000 7f45cbfff640 1 -- 172.21.15.94:0/2450249690 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f45d403b400 con 0x7f45e40fd420 2024-03-24T10:57:44.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.029+0000 7f45e9d5d640 1 -- 172.21.15.94:0/2450249690 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45e40fbbb0 con 0x7f45e40fd420 2024-03-24T10:57:44.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.029+0000 7f45cbfff640 1 -- 172.21.15.94:0/2450249690 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45d40435b0 con 0x7f45e40fd420 2024-03-24T10:57:44.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.029+0000 7f45e9d5d640 1 -- 172.21.15.94:0/2450249690 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45e40fc070 con 0x7f45e40fd420 2024-03-24T10:57:44.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.030+0000 7f45e9d5d640 1 -- 172.21.15.94:0/2450249690 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45e4065d10 con 0x7f45e40fd420 2024-03-24T10:57:44.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.030+0000 7f45cbfff640 1 -- 172.21.15.94:0/2450249690 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f45d402f070 con 0x7f45e40fd420 2024-03-24T10:57:44.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.031+0000 7f45cbfff640 1 --2- 172.21.15.94:0/2450249690 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f45cc06d750 0x7f45cc06fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:44.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.031+0000 7f45cbfff640 1 -- 172.21.15.94:0/2450249690 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f45d40aecf0 con 0x7f45e40fd420 2024-03-24T10:57:44.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.031+0000 7f45e2ffd640 1 --2- 172.21.15.94:0/2450249690 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f45cc06d750 0x7f45cc06fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:44.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.031+0000 7f45e2ffd640 1 --2- 172.21.15.94:0/2450249690 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f45cc06d750 0x7f45cc06fc00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f45d8007fd0 tx=0x7f45d8009e90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:44.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.035+0000 7f45cbfff640 1 -- 172.21.15.94:0/2450249690 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f45d40784c0 con 0x7f45e40fd420 2024-03-24T10:57:44.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:43 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/1151092123' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-24T10:57:44.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:43 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/2136251201' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-24T10:57:44.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.214+0000 7f45e9d5d640 1 -- 172.21.15.94:0/2450249690 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f45e40fc7c0 con 0x7f45e40fd420 2024-03-24T10:57:44.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.215+0000 7f45cbfff640 1 -- 172.21.15.94:0/2450249690 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f45d407bb80 con 0x7f45e40fd420 2024-03-24T10:57:44.214 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-03-24T10:57:44.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.216+0000 7f45e9d5d640 1 -- 172.21.15.94:0/2450249690 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f45cc06d750 msgr2=0x7f45cc06fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:44.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.216+0000 7f45e9d5d640 1 --2- 172.21.15.94:0/2450249690 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f45cc06d750 0x7f45cc06fc00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f45d8007fd0 tx=0x7f45d8009e90 comp rx=0 tx=0).stop 2024-03-24T10:57:44.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.216+0000 7f45e9d5d640 1 -- 172.21.15.94:0/2450249690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45e40fd420 msgr2=0x7f45e40fcd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:44.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.216+0000 7f45e9d5d640 1 --2- 172.21.15.94:0/2450249690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45e40fd420 0x7f45e40fcd70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f45d4004930 tx=0x7f45d4009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:44.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.216+0000 7f45e9d5d640 1 -- 172.21.15.94:0/2450249690 shutdown_connections 2024-03-24T10:57:44.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.216+0000 7f45e9d5d640 1 --2- 172.21.15.94:0/2450249690 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f45cc06d750 0x7f45cc06fc00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:44.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.216+0000 7f45e9d5d640 1 --2- 172.21.15.94:0/2450249690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45e40fdd30 0x7f45e40fb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:44.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.216+0000 7f45e9d5d640 1 --2- 172.21.15.94:0/2450249690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45e40fd420 0x7f45e40fcd70 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:44.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.216+0000 7f45e9d5d640 1 -- 172.21.15.94:0/2450249690 >> 172.21.15.94:0/2450249690 conn(0x7f45e40f6ea0 msgr2=0x7f45e40f8330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:44.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.217+0000 7f45e9d5d640 1 -- 172.21.15.94:0/2450249690 shutdown_connections 2024-03-24T10:57:44.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.217+0000 7f45e9d5d640 1 -- 172.21.15.94:0/2450249690 wait complete. 2024-03-24T10:57:44.227 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-24T10:57:44.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.329+0000 7f07f3aae640 1 -- 172.21.15.94:0/1770153960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07ec0fa8c0 msgr2=0x7f07ec0fad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:44.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.329+0000 7f07f3aae640 1 --2- 172.21.15.94:0/1770153960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07ec0fa8c0 0x7f07ec0fad10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f07e80048d0 tx=0x7f07e8009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:44.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.329+0000 7f07f3aae640 1 -- 172.21.15.94:0/1770153960 shutdown_connections 2024-03-24T10:57:44.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.329+0000 7f07f3aae640 1 --2- 172.21.15.94:0/1770153960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07ec0fa8c0 0x7f07ec0fad10 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:44.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.329+0000 7f07f3aae640 1 --2- 172.21.15.94:0/1770153960 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07ec1008c0 0x7f07ec100c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:44.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.329+0000 7f07f3aae640 1 -- 172.21.15.94:0/1770153960 >> 172.21.15.94:0/1770153960 conn(0x7f07ec070880 msgr2=0x7f07ec070c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:44.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.329+0000 7f07f3aae640 1 -- 172.21.15.94:0/1770153960 shutdown_connections 2024-03-24T10:57:44.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.329+0000 7f07f3aae640 1 -- 172.21.15.94:0/1770153960 wait complete. 2024-03-24T10:57:44.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.330+0000 7f07f3aae640 1 Processor -- start 2024-03-24T10:57:44.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.330+0000 7f07f3aae640 1 -- start start 2024-03-24T10:57:44.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.330+0000 7f07f3aae640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07ec0fa8c0 0x7f07ec111c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:44.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.330+0000 7f07f3aae640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07ec1008c0 0x7f07ec112160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:44.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.330+0000 7f07f3aae640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07ec1127f0 con 0x7f07ec0fa8c0 2024-03-24T10:57:44.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.330+0000 7f07f3aae640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07ec10bdf0 con 0x7f07ec1008c0 2024-03-24T10:57:44.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.330+0000 7f07f184a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07ec0fa8c0 0x7f07ec111c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:44.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.330+0000 7f07f184a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07ec0fa8c0 0x7f07ec111c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40616/0 (socket says 172.21.15.94:40616) 2024-03-24T10:57:44.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.330+0000 7f07f184a640 1 -- 172.21.15.94:0/2128965271 learned_addr learned my addr 172.21.15.94:0/2128965271 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:57:44.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.330+0000 7f07f184a640 1 -- 172.21.15.94:0/2128965271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07ec1008c0 msgr2=0x7f07ec112160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:44.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.330+0000 7f07f184a640 1 --2- 172.21.15.94:0/2128965271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07ec1008c0 0x7f07ec112160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:44.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.330+0000 7f07f184a640 1 -- 172.21.15.94:0/2128965271 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e8005040 con 0x7f07ec0fa8c0 2024-03-24T10:57:44.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.330+0000 7f07f184a640 1 --2- 172.21.15.94:0/2128965271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07ec0fa8c0 0x7f07ec111c20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f07dc00b790 tx=0x7f07dc0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:44.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.331+0000 7f07e27fc640 1 -- 172.21.15.94:0/2128965271 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f07dc00c700 con 0x7f07ec0fa8c0 2024-03-24T10:57:44.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.331+0000 7f07e27fc640 1 -- 172.21.15.94:0/2128965271 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f07dc00cd30 con 0x7f07ec0fa8c0 2024-03-24T10:57:44.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.331+0000 7f07e27fc640 1 -- 172.21.15.94:0/2128965271 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f07dc013370 con 0x7f07ec0fa8c0 2024-03-24T10:57:44.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.331+0000 7f07f3aae640 1 -- 172.21.15.94:0/2128965271 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07ec10c0d0 con 0x7f07ec0fa8c0 2024-03-24T10:57:44.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.331+0000 7f07f3aae640 1 -- 172.21.15.94:0/2128965271 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07ec10c5f0 con 0x7f07ec0fa8c0 2024-03-24T10:57:44.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.332+0000 7f07f3aae640 1 -- 172.21.15.94:0/2128965271 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07ec0fbfa0 con 0x7f07ec0fa8c0 2024-03-24T10:57:44.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.332+0000 7f07e27fc640 1 -- 172.21.15.94:0/2128965271 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f07dc012070 con 0x7f07ec0fa8c0 2024-03-24T10:57:44.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.332+0000 7f07e27fc640 1 --2- 172.21.15.94:0/2128965271 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f07d806d750 0x7f07d806fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:44.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.333+0000 7f07f1049640 1 --2- 172.21.15.94:0/2128965271 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f07d806d750 0x7f07d806fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:44.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.333+0000 7f07e27fc640 1 -- 172.21.15.94:0/2128965271 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f07dc08cc10 con 0x7f07ec0fa8c0 2024-03-24T10:57:44.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.333+0000 7f07f1049640 1 --2- 172.21.15.94:0/2128965271 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f07d806d750 0x7f07d806fc00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f07e80043d0 tx=0x7f07e8009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:44.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.341+0000 7f07e27fc640 1 -- 172.21.15.94:0/2128965271 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f07dc056360 con 0x7f07ec0fa8c0 2024-03-24T10:57:44.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.550+0000 7f07f3aae640 1 -- 172.21.15.94:0/2128965271 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f07ec107a80 con 0x7f07ec0fa8c0 2024-03-24T10:57:44.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.550+0000 7f07e27fc640 1 -- 172.21.15.94:0/2128965271 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7f07dc059a20 con 0x7f07ec0fa8c0 2024-03-24T10:57:44.550 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-03-24T10:57:44.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.552+0000 7f07f3aae640 1 -- 172.21.15.94:0/2128965271 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f07d806d750 msgr2=0x7f07d806fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:44.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.552+0000 7f07f3aae640 1 --2- 172.21.15.94:0/2128965271 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f07d806d750 0x7f07d806fc00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f07e80043d0 tx=0x7f07e8009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:44.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.552+0000 7f07f3aae640 1 -- 172.21.15.94:0/2128965271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07ec0fa8c0 msgr2=0x7f07ec111c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:44.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.552+0000 7f07f3aae640 1 --2- 172.21.15.94:0/2128965271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07ec0fa8c0 0x7f07ec111c20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f07dc00b790 tx=0x7f07dc0091d0 comp rx=0 tx=0).stop 2024-03-24T10:57:44.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.552+0000 7f07f3aae640 1 -- 172.21.15.94:0/2128965271 shutdown_connections 2024-03-24T10:57:44.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.552+0000 7f07f3aae640 1 --2- 172.21.15.94:0/2128965271 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7f07d806d750 0x7f07d806fc00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:44.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.552+0000 7f07f3aae640 1 --2- 172.21.15.94:0/2128965271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07ec1008c0 0x7f07ec112160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:44.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.552+0000 7f07f3aae640 1 --2- 172.21.15.94:0/2128965271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07ec0fa8c0 0x7f07ec111c20 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:44.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.552+0000 7f07f3aae640 1 -- 172.21.15.94:0/2128965271 >> 172.21.15.94:0/2128965271 conn(0x7f07ec070880 msgr2=0x7f07ec1038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:44.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.552+0000 7f07f3aae640 1 -- 172.21.15.94:0/2128965271 shutdown_connections 2024-03-24T10:57:44.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.552+0000 7f07f3aae640 1 -- 172.21.15.94:0/2128965271 wait complete. 2024-03-24T10:57:44.562 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-24T10:57:44.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.663+0000 7fd797625640 1 -- 172.21.15.94:0/464450044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd790064cb0 msgr2=0x7fd790108780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:44.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.663+0000 7fd797625640 1 --2- 172.21.15.94:0/464450044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd790064cb0 0x7fd790108780 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd7800048d0 tx=0x7fd780009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:44.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.663+0000 7fd797625640 1 -- 172.21.15.94:0/464450044 shutdown_connections 2024-03-24T10:57:44.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.663+0000 7fd797625640 1 --2- 172.21.15.94:0/464450044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd790064cb0 0x7fd790108780 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:44.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.663+0000 7fd797625640 1 --2- 172.21.15.94:0/464450044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7900643a0 0x7fd790064770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:44.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.663+0000 7fd797625640 1 -- 172.21.15.94:0/464450044 >> 172.21.15.94:0/464450044 conn(0x7fd7900710b0 msgr2=0x7fd7900714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:44.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.664+0000 7fd797625640 1 -- 172.21.15.94:0/464450044 shutdown_connections 2024-03-24T10:57:44.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.664+0000 7fd797625640 1 -- 172.21.15.94:0/464450044 wait complete. 2024-03-24T10:57:44.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.664+0000 7fd797625640 1 Processor -- start 2024-03-24T10:57:44.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.664+0000 7fd797625640 1 -- start start 2024-03-24T10:57:44.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.664+0000 7fd797625640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7900643a0 0x7fd790114ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:44.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.664+0000 7fd797625640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd790064cb0 0x7fd790115420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:44.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.664+0000 7fd797625640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd790115a20 con 0x7fd790064cb0 2024-03-24T10:57:44.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.664+0000 7fd797625640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd79010f070 con 0x7fd7900643a0 2024-03-24T10:57:44.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.664+0000 7fd794bc0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd790064cb0 0x7fd790115420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:44.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.664+0000 7fd794bc0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd790064cb0 0x7fd790115420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40630/0 (socket says 172.21.15.94:40630) 2024-03-24T10:57:44.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.664+0000 7fd794bc0640 1 -- 172.21.15.94:0/698089398 learned_addr learned my addr 172.21.15.94:0/698089398 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:57:44.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.665+0000 7fd7953c1640 1 --2- 172.21.15.94:0/698089398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7900643a0 0x7fd790114ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:44.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.665+0000 7fd794bc0640 1 -- 172.21.15.94:0/698089398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7900643a0 msgr2=0x7fd790114ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:44.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.665+0000 7fd794bc0640 1 --2- 172.21.15.94:0/698089398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7900643a0 0x7fd790114ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:44.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.665+0000 7fd794bc0640 1 -- 172.21.15.94:0/698089398 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd780005040 con 0x7fd790064cb0 2024-03-24T10:57:44.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.665+0000 7fd7953c1640 1 --2- 172.21.15.94:0/698089398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7900643a0 0x7fd790114ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T10:57:44.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.665+0000 7fd794bc0640 1 --2- 172.21.15.94:0/698089398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd790064cb0 0x7fd790115420 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd780004930 tx=0x7fd780009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:44.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.665+0000 7fd785ffb640 1 -- 172.21.15.94:0/698089398 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7800094c0 con 0x7fd790064cb0 2024-03-24T10:57:44.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.665+0000 7fd797625640 1 -- 172.21.15.94:0/698089398 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd79010f2f0 con 0x7fd790064cb0 2024-03-24T10:57:44.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.665+0000 7fd785ffb640 1 -- 172.21.15.94:0/698089398 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd780032e50 con 0x7fd790064cb0 2024-03-24T10:57:44.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.665+0000 7fd785ffb640 1 -- 172.21.15.94:0/698089398 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7800433b0 con 0x7fd790064cb0 2024-03-24T10:57:44.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.665+0000 7fd797625640 1 -- 172.21.15.94:0/698089398 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd79010f7b0 con 0x7fd790064cb0 2024-03-24T10:57:44.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.666+0000 7fd797625640 1 -- 172.21.15.94:0/698089398 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7900fb730 con 0x7fd790064cb0 2024-03-24T10:57:44.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.667+0000 7fd785ffb640 1 -- 172.21.15.94:0/698089398 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fd78002f070 con 0x7fd790064cb0 2024-03-24T10:57:44.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.667+0000 7fd785ffb640 1 --2- 172.21.15.94:0/698089398 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fd77c06d750 0x7fd77c06fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:44.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.667+0000 7fd785ffb640 1 -- 172.21.15.94:0/698089398 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fd7800af000 con 0x7fd790064cb0 2024-03-24T10:57:44.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.667+0000 7fd7953c1640 1 --2- 172.21.15.94:0/698089398 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fd77c06d750 0x7fd77c06fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:44.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.668+0000 7fd7953c1640 1 --2- 172.21.15.94:0/698089398 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fd77c06d750 0x7fd77c06fc00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd78c004c80 tx=0x7fd78c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:44.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.672+0000 7fd785ffb640 1 -- 172.21.15.94:0/698089398 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd780078750 con 0x7fd790064cb0 2024-03-24T10:57:44.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.879+0000 7fd797625640 1 -- 172.21.15.94:0/698089398 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fd79010b4f0 con 0x7fd790064cb0 2024-03-24T10:57:44.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.879+0000 7fd785ffb640 1 -- 172.21.15.94:0/698089398 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7fd78007be10 con 0x7fd790064cb0 2024-03-24T10:57:44.879 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-03-24T10:57:44.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.881+0000 7fd797625640 1 -- 172.21.15.94:0/698089398 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fd77c06d750 msgr2=0x7fd77c06fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:44.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.881+0000 7fd797625640 1 --2- 172.21.15.94:0/698089398 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fd77c06d750 0x7fd77c06fc00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd78c004c80 tx=0x7fd78c009040 comp rx=0 tx=0).stop 2024-03-24T10:57:44.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.881+0000 7fd797625640 1 -- 172.21.15.94:0/698089398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd790064cb0 msgr2=0x7fd790115420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:44.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.881+0000 7fd797625640 1 --2- 172.21.15.94:0/698089398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd790064cb0 0x7fd790115420 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd780004930 tx=0x7fd780009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:44.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.882+0000 7fd797625640 1 -- 172.21.15.94:0/698089398 shutdown_connections 2024-03-24T10:57:44.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.882+0000 7fd797625640 1 --2- 172.21.15.94:0/698089398 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fd77c06d750 0x7fd77c06fc00 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:44.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.882+0000 7fd797625640 1 --2- 172.21.15.94:0/698089398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd790064cb0 0x7fd790115420 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:44.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.882+0000 7fd797625640 1 --2- 172.21.15.94:0/698089398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7900643a0 0x7fd790114ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:44.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.882+0000 7fd797625640 1 -- 172.21.15.94:0/698089398 >> 172.21.15.94:0/698089398 conn(0x7fd7900710b0 msgr2=0x7fd7900f9100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:44.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.882+0000 7fd797625640 1 -- 172.21.15.94:0/698089398 shutdown_connections 2024-03-24T10:57:44.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.882+0000 7fd797625640 1 -- 172.21.15.94:0/698089398 wait complete. 2024-03-24T10:57:44.891 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:44 smithi094 ceph-mon[93957]: pgmap v312: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:57:44.891 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:44 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/330414270' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T10:57:44.891 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:44 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/2450249690' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T10:57:44.891 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:44 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/2128965271' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-24T10:57:44.892 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-24T10:57:44.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.992+0000 7fd627418640 1 -- 172.21.15.94:0/4197491587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6200643a0 msgr2=0x7fd6200647d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:44.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.992+0000 7fd627418640 1 --2- 172.21.15.94:0/4197491587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6200643a0 0x7fd6200647d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd6100048d0 tx=0x7fd610009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:44.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.992+0000 7fd627418640 1 -- 172.21.15.94:0/4197491587 shutdown_connections 2024-03-24T10:57:44.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.992+0000 7fd627418640 1 --2- 172.21.15.94:0/4197491587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6200643a0 0x7fd6200647d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:44.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.992+0000 7fd627418640 1 --2- 172.21.15.94:0/4197491587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd620102240 0x7fd620102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:44.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.992+0000 7fd627418640 1 -- 172.21.15.94:0/4197491587 >> 172.21.15.94:0/4197491587 conn(0x7fd6200710b0 msgr2=0x7fd6200714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:44.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.993+0000 7fd627418640 1 -- 172.21.15.94:0/4197491587 shutdown_connections 2024-03-24T10:57:44.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.993+0000 7fd627418640 1 -- 172.21.15.94:0/4197491587 wait complete. 2024-03-24T10:57:44.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.993+0000 7fd627418640 1 Processor -- start 2024-03-24T10:57:44.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.993+0000 7fd627418640 1 -- start start 2024-03-24T10:57:44.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.993+0000 7fd627418640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6200643a0 0x7fd620112cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:44.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.993+0000 7fd627418640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd620102240 0x7fd620113210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:44.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.993+0000 7fd627418640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd620113810 con 0x7fd620102240 2024-03-24T10:57:44.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.993+0000 7fd627418640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62010ce60 con 0x7fd6200643a0 2024-03-24T10:57:44.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.994+0000 7fd6249b3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd620102240 0x7fd620113210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:44.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.994+0000 7fd6249b3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd620102240 0x7fd620113210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40644/0 (socket says 172.21.15.94:40644) 2024-03-24T10:57:44.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.994+0000 7fd6249b3640 1 -- 172.21.15.94:0/3113900947 learned_addr learned my addr 172.21.15.94:0/3113900947 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-03-24T10:57:44.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.994+0000 7fd6251b4640 1 --2- 172.21.15.94:0/3113900947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6200643a0 0x7fd620112cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:44.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.994+0000 7fd6249b3640 1 -- 172.21.15.94:0/3113900947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6200643a0 msgr2=0x7fd620112cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:44.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.994+0000 7fd6249b3640 1 --2- 172.21.15.94:0/3113900947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6200643a0 0x7fd620112cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:44.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.994+0000 7fd6249b3640 1 -- 172.21.15.94:0/3113900947 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd610005040 con 0x7fd620102240 2024-03-24T10:57:44.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.994+0000 7fd6251b4640 1 --2- 172.21.15.94:0/3113900947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6200643a0 0x7fd620112cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T10:57:44.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.994+0000 7fd6249b3640 1 --2- 172.21.15.94:0/3113900947 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd620102240 0x7fd620113210 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd610004930 tx=0x7fd610009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:44.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.994+0000 7fd615ffb640 1 -- 172.21.15.94:0/3113900947 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6100094c0 con 0x7fd620102240 2024-03-24T10:57:44.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.994+0000 7fd615ffb640 1 -- 172.21.15.94:0/3113900947 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd610032e50 con 0x7fd620102240 2024-03-24T10:57:44.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.994+0000 7fd627418640 1 -- 172.21.15.94:0/3113900947 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd62010d0e0 con 0x7fd620102240 2024-03-24T10:57:44.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.994+0000 7fd615ffb640 1 -- 172.21.15.94:0/3113900947 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6100433b0 con 0x7fd620102240 2024-03-24T10:57:44.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.994+0000 7fd627418640 1 -- 172.21.15.94:0/3113900947 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd62010d5a0 con 0x7fd620102240 2024-03-24T10:57:44.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.996+0000 7fd627418640 1 -- 172.21.15.94:0/3113900947 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6200fd940 con 0x7fd620102240 2024-03-24T10:57:44.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.996+0000 7fd615ffb640 1 -- 172.21.15.94:0/3113900947 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fd61002f070 con 0x7fd620102240 2024-03-24T10:57:44.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.996+0000 7fd615ffb640 1 --2- 172.21.15.94:0/3113900947 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fd60c06d750 0x7fd60c06fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T10:57:44.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.996+0000 7fd615ffb640 1 -- 172.21.15.94:0/3113900947 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fd6100aeea0 con 0x7fd620102240 2024-03-24T10:57:44.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.997+0000 7fd6251b4640 1 --2- 172.21.15.94:0/3113900947 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fd60c06d750 0x7fd60c06fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T10:57:44.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:44.997+0000 7fd6251b4640 1 --2- 172.21.15.94:0/3113900947 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fd60c06d750 0x7fd60c06fc00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd61c007ee0 tx=0x7fd61c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T10:57:45.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:45.001+0000 7fd615ffb640 1 -- 172.21.15.94:0/3113900947 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd6100785f0 con 0x7fd620102240 2024-03-24T10:57:45.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:44 smithi096 ceph-mon[84258]: pgmap v312: 65 pgs: 65 active+clean; 196 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-24T10:57:45.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:44 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/330414270' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T10:57:45.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:44 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/2450249690' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T10:57:45.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:44 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/2128965271' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-24T10:57:45.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:45.188+0000 7fd627418640 1 -- 172.21.15.94:0/3113900947 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd620109320 con 0x7fd620102240 2024-03-24T10:57:45.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:45.189+0000 7fd615ffb640 1 -- 172.21.15.94:0/3113900947 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 43 entries v84) v1 ==== 81+0+2541 (secure 0 0 0) 0x7fd61007bcb0 con 0x7fd620102240 2024-03-24T10:57:45.189 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.96:6825/1539752208 2024-03-25T10:53:38.029430+0000 2024-03-24T10:57:45.189 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/403370730 2024-03-25T10:48:35.235438+0000 2024-03-24T10:57:45.189 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/3351913962 2024-03-25T10:48:35.235438+0000 2024-03-24T10:57:45.189 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/659933905 2024-03-25T10:48:35.235438+0000 2024-03-24T10:57:45.189 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.96:6827/1321087930 2024-03-25T10:54:01.169508+0000 2024-03-24T10:57:45.189 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.96:6828/1632816545 2024-03-25T10:47:50.745695+0000 2024-03-24T10:57:45.189 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.96:6829/1632816545 2024-03-25T10:47:50.745695+0000 2024-03-24T10:57:45.189 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.96:6826/1321087930 2024-03-25T10:54:01.169508+0000 2024-03-24T10:57:45.190 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/1027963628 2024-03-25T10:46:29.516417+0000 2024-03-24T10:57:45.190 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6801/1013449112 2024-03-25T10:36:48.432382+0000 2024-03-24T10:57:45.190 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/2700857461 2024-03-25T10:39:00.830398+0000 2024-03-24T10:57:45.190 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6800/1996608546 2024-03-25T10:37:18.183215+0000 2024-03-24T10:57:45.190 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6826/2001471799 2024-03-25T10:43:55.084207+0000 2024-03-24T10:57:45.190 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6801/1996608546 2024-03-25T10:37:18.183215+0000 2024-03-24T10:57:45.190 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/2837872991 2024-03-25T10:36:48.432382+0000 2024-03-24T10:57:45.190 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/2130298637 2024-03-25T10:39:00.830398+0000 2024-03-24T10:57:45.190 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.96:0/109124621 2024-03-25T10:47:50.745695+0000 2024-03-24T10:57:45.190 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.96:0/3508170171 2024-03-25T10:47:50.745695+0000 2024-03-24T10:57:45.190 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6800/2406899470 2024-03-25T10:39:00.830398+0000 2024-03-24T10:57:45.190 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/206764114 2024-03-25T10:46:29.516417+0000 2024-03-24T10:57:45.190 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6800/1013449112 2024-03-25T10:36:48.432382+0000 2024-03-24T10:57:45.190 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6801/2184357820 2024-03-25T10:46:29.516417+0000 2024-03-24T10:57:45.191 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.96:0/2983574460 2024-03-25T10:47:50.745695+0000 2024-03-24T10:57:45.191 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6800/4143126939 2024-03-25T10:48:35.235438+0000 2024-03-24T10:57:45.191 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.96:0/3848335112 2024-03-25T10:47:50.745695+0000 2024-03-24T10:57:45.191 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6801/2406899470 2024-03-25T10:39:00.830398+0000 2024-03-24T10:57:45.191 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/629634217 2024-03-25T10:37:18.183215+0000 2024-03-24T10:57:45.191 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/3628417758 2024-03-25T10:37:18.183215+0000 2024-03-24T10:57:45.191 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.96:0/3826472619 2024-03-25T10:47:50.745695+0000 2024-03-24T10:57:45.191 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/2173796904 2024-03-25T10:48:35.235438+0000 2024-03-24T10:57:45.191 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/4215128639 2024-03-25T10:48:35.235438+0000 2024-03-24T10:57:45.191 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/2155575468 2024-03-25T10:39:00.830398+0000 2024-03-24T10:57:45.191 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/265930160 2024-03-25T10:46:29.516417+0000 2024-03-24T10:57:45.191 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/978791860 2024-03-25T10:46:29.516417+0000 2024-03-24T10:57:45.191 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/2641430900 2024-03-25T10:36:48.432382+0000 2024-03-24T10:57:45.191 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6827/2001471799 2024-03-25T10:43:55.084207+0000 2024-03-24T10:57:45.191 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6800/2184357820 2024-03-25T10:46:29.516417+0000 2024-03-24T10:57:45.191 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/4239612108 2024-03-25T10:36:48.432382+0000 2024-03-24T10:57:45.192 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/2661420513 2024-03-25T10:37:18.183215+0000 2024-03-24T10:57:45.192 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/4063210620 2024-03-25T10:48:35.235438+0000 2024-03-24T10:57:45.192 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.96:0/3832405488 2024-03-25T10:47:50.745695+0000 2024-03-24T10:57:45.192 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.96:6824/1539752208 2024-03-25T10:53:38.029430+0000 2024-03-24T10:57:45.192 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6801/4143126939 2024-03-25T10:48:35.235438+0000 2024-03-24T10:57:45.192 INFO:teuthology.orchestra.run.smithi094.stderr:listed 43 entries 2024-03-24T10:57:45.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:45.191+0000 7fd627418640 1 -- 172.21.15.94:0/3113900947 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fd60c06d750 msgr2=0x7fd60c06fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:45.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:45.191+0000 7fd627418640 1 --2- 172.21.15.94:0/3113900947 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fd60c06d750 0x7fd60c06fc00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd61c007ee0 tx=0x7fd61c00b040 comp rx=0 tx=0).stop 2024-03-24T10:57:45.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:45.191+0000 7fd627418640 1 -- 172.21.15.94:0/3113900947 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd620102240 msgr2=0x7fd620113210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T10:57:45.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:45.191+0000 7fd627418640 1 --2- 172.21.15.94:0/3113900947 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd620102240 0x7fd620113210 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd610004930 tx=0x7fd610009da0 comp rx=0 tx=0).stop 2024-03-24T10:57:45.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:45.191+0000 7fd627418640 1 -- 172.21.15.94:0/3113900947 shutdown_connections 2024-03-24T10:57:45.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:45.191+0000 7fd627418640 1 --2- 172.21.15.94:0/3113900947 >> [v2:172.21.15.94:6800/109020447,v1:172.21.15.94:6801/109020447] conn(0x7fd60c06d750 0x7fd60c06fc00 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:45.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:45.191+0000 7fd627418640 1 --2- 172.21.15.94:0/3113900947 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd620102240 0x7fd620113210 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:45.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:45.191+0000 7fd627418640 1 --2- 172.21.15.94:0/3113900947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6200643a0 0x7fd620112cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T10:57:45.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:45.191+0000 7fd627418640 1 -- 172.21.15.94:0/3113900947 >> 172.21.15.94:0/3113900947 conn(0x7fd6200710b0 msgr2=0x7fd6200f94a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T10:57:45.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:45.191+0000 7fd627418640 1 -- 172.21.15.94:0/3113900947 shutdown_connections 2024-03-24T10:57:45.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-03-24T10:57:45.191+0000 7fd627418640 1 -- 172.21.15.94:0/3113900947 wait complete. 2024-03-24T10:57:45.205 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-03-24T10:57:45.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T10:57:45.205 DEBUG:teuthology.orchestra.run.smithi096:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-03-24T10:57:45.241 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi096.front.sepia.ceph.com 2024-03-24T10:57:45.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T10:57:45.241 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-24T10:57:45.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T10:57:45.429 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-03-24T10:57:45.444 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-24T10:57:45.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T10:57:45.445 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-24T10:57:45.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T10:57:45.515 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link delete ceph-brx 2024-03-24T10:57:45.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T10:57:45.612 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list 2024-03-24T10:57:45.631 INFO:teuthology.orchestra.run.smithi096.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-24T10:57:45.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T10:57:45.631 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-24T10:57:45.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-24T10:57:45.704 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link delete ceph-brx 2024-03-24T10:57:45.803 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-24T10:57:45.815 INFO:tasks.cephadm:Teardown begin 2024-03-24T10:57:45.816 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-24T10:57:45.841 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:45 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/698089398' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-24T10:57:45.841 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:45 smithi094 ceph-mon[93957]: from='client.? 172.21.15.94:0/3113900947' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-24T10:57:45.845 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-24T10:57:45.873 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-24T10:57:45.874 DEBUG:teuthology.orchestra.run.smithi094:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-24T10:57:45.899 DEBUG:teuthology.orchestra.run.smithi096:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-24T10:57:45.929 INFO:tasks.cephadm:Stopping all daemons... 2024-03-24T10:57:45.929 INFO:tasks.cephadm.mon.smithi094:Stopping mon.smithi094... 2024-03-24T10:57:45.930 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-386110d4-e9ca-11ee-95cd-87774f69a715@mon.smithi094 2024-03-24T10:57:46.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:45 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/698089398' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-24T10:57:46.075 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:45 smithi096 ceph-mon[84258]: from='client.? 172.21.15.94:0/3113900947' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-24T10:57:46.231 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:45 smithi094 systemd[1]: Stopping Ceph mon.smithi094 for 386110d4-e9ca-11ee-95cd-87774f69a715... 2024-03-24T10:57:46.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:46 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-mon-smithi094[93953]: 2024-03-24T10:57:46.231+0000 7f0fa68d9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi094 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T10:57:46.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:46 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-mon-smithi094[93953]: 2024-03-24T10:57:46.231+0000 7f0fa68d9700 -1 mon.smithi094@0(leader) e3 *** Got Signal Terminated *** 2024-03-24T10:57:46.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:46 smithi094 podman[149688]: 2024-03-24 10:57:46.262169914 +0000 UTC m=+0.130071542 container died c779d611e29a976824345c01fe0209d2f25bbb268cb0ec946f706d92946b80f6 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-mon-smithi094, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319) 2024-03-24T10:57:46.864 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:46 smithi094 podman[149688]: 2024-03-24 10:57:46.624105677 +0000 UTC m=+0.492007313 container cleanup c779d611e29a976824345c01fe0209d2f25bbb268cb0ec946f706d92946b80f6 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-mon-smithi094, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T10:57:46.864 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Mar 24 10:57:46 smithi094 bash[149688]: ceph-386110d4-e9ca-11ee-95cd-87774f69a715-mon-smithi094 2024-03-24T10:57:47.074 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-386110d4-e9ca-11ee-95cd-87774f69a715@mon.smithi094.service' 2024-03-24T10:57:47.122 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T10:57:47.122 INFO:tasks.cephadm.mon.smithi094:Stopped mon.smithi094 2024-03-24T10:57:47.122 INFO:tasks.cephadm.mon.smithi096:Stopping mon.smithi096... 2024-03-24T10:57:47.122 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-386110d4-e9ca-11ee-95cd-87774f69a715@mon.smithi096 2024-03-24T10:57:47.381 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:47 smithi096 systemd[1]: Stopping Ceph mon.smithi096 for 386110d4-e9ca-11ee-95cd-87774f69a715... 2024-03-24T10:57:47.668 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:47 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-mon-smithi096[84249]: 2024-03-24T10:57:47.474+0000 7f66f5008700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi096 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T10:57:47.668 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:47 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-mon-smithi096[84249]: 2024-03-24T10:57:47.474+0000 7f66f5008700 -1 mon.smithi096@1(peon) e3 *** Got Signal Terminated *** 2024-03-24T10:57:47.925 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:47 smithi096 podman[120046]: 2024-03-24 10:57:47.668292011 +0000 UTC m=+0.329855734 container died ed9bd0bd57f6827b4d30ceb525471127356c7d2b1879c6048709be361c5a8f51 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-mon-smithi096, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T10:57:48.309 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:48 smithi096 podman[120046]: 2024-03-24 10:57:48.100056693 +0000 UTC m=+0.761620433 container cleanup ed9bd0bd57f6827b4d30ceb525471127356c7d2b1879c6048709be361c5a8f51 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-mon-smithi096, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2) 2024-03-24T10:57:48.310 INFO:journalctl@ceph.mon.smithi096.smithi096.stdout:Mar 24 10:57:48 smithi096 bash[120046]: ceph-386110d4-e9ca-11ee-95cd-87774f69a715-mon-smithi096 2024-03-24T10:57:48.521 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-386110d4-e9ca-11ee-95cd-87774f69a715@mon.smithi096.service' 2024-03-24T10:57:48.568 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T10:57:48.568 INFO:tasks.cephadm.mon.smithi096:Stopped mon.smithi096 2024-03-24T10:57:48.568 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-24T10:57:48.569 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-386110d4-e9ca-11ee-95cd-87774f69a715@osd.0 2024-03-24T10:57:48.879 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:57:48 smithi094 systemd[1]: Stopping Ceph osd.0 for 386110d4-e9ca-11ee-95cd-87774f69a715... 2024-03-24T10:57:49.247 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:57:48 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-0[102665]: 2024-03-24T10:57:48.879+0000 7ff3cc6b0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T10:57:49.247 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:57:48 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-0[102665]: 2024-03-24T10:57:48.879+0000 7ff3cc6b0700 -1 osd.0 84 *** Got signal Terminated *** 2024-03-24T10:57:49.248 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:57:48 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-0[102665]: 2024-03-24T10:57:48.879+0000 7ff3cc6b0700 -1 osd.0 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T10:57:54.614 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:57:54 smithi094 podman[149910]: 2024-03-24 10:57:54.322276493 +0000 UTC m=+5.560918623 container died ec57ede0c9e45d834b70169db96f6d70e295e7936c8a7337eb0f7bbd6b3f6b4e (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-24T10:57:54.939 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:57:54 smithi094 podman[149910]: 2024-03-24 10:57:54.70503223 +0000 UTC m=+5.943674368 container cleanup ec57ede0c9e45d834b70169db96f6d70e295e7936c8a7337eb0f7bbd6b3f6b4e (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319) 2024-03-24T10:57:54.939 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:57:54 smithi094 bash[149910]: ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-0 2024-03-24T10:57:55.247 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:57:54 smithi094 podman[149928]: 2024-03-24 10:57:54.939861488 +0000 UTC m=+0.616858849 container remove ec57ede0c9e45d834b70169db96f6d70e295e7936c8a7337eb0f7bbd6b3f6b4e (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T10:57:55.602 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:57:55 smithi094 podman[150018]: 2024-03-24 10:57:55.274368513 +0000 UTC m=+0.112632647 container create 3e9f791bcba90033c0e7bd7c153c6829296c4e030f428906d7113ee8226bcf1f (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=) 2024-03-24T10:57:55.602 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:57:55 smithi094 podman[150018]: 2024-03-24 10:57:55.180886217 +0000 UTC m=+0.019150368 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-24T10:57:55.602 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:57:55 smithi094 podman[150018]: 2024-03-24 10:57:55.511515522 +0000 UTC m=+0.349779683 container init 3e9f791bcba90033c0e7bd7c153c6829296c4e030f428906d7113ee8226bcf1f (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-0-deactivate, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_BRANCH=HEAD) 2024-03-24T10:57:55.602 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:57:55 smithi094 podman[150018]: 2024-03-24 10:57:55.515128724 +0000 UTC m=+0.353392868 container start 3e9f791bcba90033c0e7bd7c153c6829296c4e030f428906d7113ee8226bcf1f (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, io.buildah.version=1.33.5) 2024-03-24T10:57:55.602 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:57:55 smithi094 podman[150018]: 2024-03-24 10:57:55.602498226 +0000 UTC m=+0.440762369 container attach 3e9f791bcba90033c0e7bd7c153c6829296c4e030f428906d7113ee8226bcf1f (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-0-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-24T10:57:55.997 INFO:journalctl@ceph.osd.0.smithi094.stdout:Mar 24 10:57:55 smithi094 podman[150018]: 2024-03-24 10:57:55.810740929 +0000 UTC m=+0.649005064 container died 3e9f791bcba90033c0e7bd7c153c6829296c4e030f428906d7113ee8226bcf1f (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.license=GPLv2) 2024-03-24T10:57:56.373 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-386110d4-e9ca-11ee-95cd-87774f69a715@osd.0.service' 2024-03-24T10:57:56.415 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T10:57:56.415 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-24T10:57:56.415 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-24T10:57:56.415 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-386110d4-e9ca-11ee-95cd-87774f69a715@osd.1 2024-03-24T10:57:56.652 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:57:56 smithi094 systemd[1]: Stopping Ceph osd.1 for 386110d4-e9ca-11ee-95cd-87774f69a715... 2024-03-24T10:57:56.997 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:57:56 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-1[108085]: 2024-03-24T10:57:56.741+0000 7f4070cf0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T10:57:56.997 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:57:56 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-1[108085]: 2024-03-24T10:57:56.741+0000 7f4070cf0700 -1 osd.1 84 *** Got signal Terminated *** 2024-03-24T10:57:56.997 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:57:56 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-1[108085]: 2024-03-24T10:57:56.741+0000 7f4070cf0700 -1 osd.1 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T10:58:02.497 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:58:02 smithi094 podman[150123]: 2024-03-24 10:58:02.212593345 +0000 UTC m=+5.567738186 container died c5b8b2bbf8b54b6935ced95e2a31dc82592fb302a31c78dc0ed63026cc367a80 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-24T10:58:02.997 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:58:02 smithi094 podman[150123]: 2024-03-24 10:58:02.627378457 +0000 UTC m=+5.982523288 container cleanup c5b8b2bbf8b54b6935ced95e2a31dc82592fb302a31c78dc0ed63026cc367a80 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-1, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-24T10:58:02.997 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:58:02 smithi094 bash[150123]: ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-1 2024-03-24T10:58:02.998 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:58:02 smithi094 podman[150153]: 2024-03-24 10:58:02.83746427 +0000 UTC m=+0.618405370 container remove c5b8b2bbf8b54b6935ced95e2a31dc82592fb302a31c78dc0ed63026cc367a80 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-1, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True) 2024-03-24T10:58:03.497 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:58:03 smithi094 podman[150243]: 2024-03-24 10:58:03.151137306 +0000 UTC m=+0.020494539 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-24T10:58:03.497 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:58:03 smithi094 podman[150243]: 2024-03-24 10:58:03.27192258 +0000 UTC m=+0.141279793 container create ee9051bd103b24500fbc24854abdc50f5095491c8048b12b1aa06c3211ec0347 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, CEPH_POINT_RELEASE=) 2024-03-24T10:58:03.498 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:58:03 smithi094 podman[150243]: 2024-03-24 10:58:03.4411936 +0000 UTC m=+0.310550811 container init ee9051bd103b24500fbc24854abdc50f5095491c8048b12b1aa06c3211ec0347 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-24T10:58:03.498 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:58:03 smithi094 podman[150243]: 2024-03-24 10:58:03.444217406 +0000 UTC m=+0.313574616 container start ee9051bd103b24500fbc24854abdc50f5095491c8048b12b1aa06c3211ec0347 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T10:58:03.998 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:58:03 smithi094 podman[150243]: 2024-03-24 10:58:03.550150375 +0000 UTC m=+0.419507588 container attach ee9051bd103b24500fbc24854abdc50f5095491c8048b12b1aa06c3211ec0347 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-24T10:58:03.998 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:58:03 smithi094 podman[150243]: 2024-03-24 10:58:03.74433453 +0000 UTC m=+0.613691732 container died ee9051bd103b24500fbc24854abdc50f5095491c8048b12b1aa06c3211ec0347 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T10:58:04.278 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-386110d4-e9ca-11ee-95cd-87774f69a715@osd.1.service' 2024-03-24T10:58:04.303 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:58:04 smithi094 podman[150269]: 2024-03-24 10:58:04.253741538 +0000 UTC m=+0.503151974 container remove ee9051bd103b24500fbc24854abdc50f5095491c8048b12b1aa06c3211ec0347 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T10:58:04.303 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:58:04 smithi094 systemd[1]: ceph-386110d4-e9ca-11ee-95cd-87774f69a715@osd.1.service: Deactivated successfully. 2024-03-24T10:58:04.303 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:58:04 smithi094 systemd[1]: Stopped Ceph osd.1 for 386110d4-e9ca-11ee-95cd-87774f69a715. 2024-03-24T10:58:04.303 INFO:journalctl@ceph.osd.1.smithi094.stdout:Mar 24 10:58:04 smithi094 systemd[1]: ceph-386110d4-e9ca-11ee-95cd-87774f69a715@osd.1.service: Consumed 8.833s CPU time. 2024-03-24T10:58:04.319 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T10:58:04.320 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-24T10:58:04.320 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-24T10:58:04.320 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-386110d4-e9ca-11ee-95cd-87774f69a715@osd.2 2024-03-24T10:58:04.640 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:58:04 smithi094 systemd[1]: Stopping Ceph osd.2 for 386110d4-e9ca-11ee-95cd-87774f69a715... 2024-03-24T10:58:04.997 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:58:04 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-2[113746]: 2024-03-24T10:58:04.639+0000 7f07c9b92700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T10:58:04.997 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:58:04 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-2[113746]: 2024-03-24T10:58:04.639+0000 7f07c9b92700 -1 osd.2 84 *** Got signal Terminated *** 2024-03-24T10:58:04.997 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:58:04 smithi094 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-2[113746]: 2024-03-24T10:58:04.639+0000 7f07c9b92700 -1 osd.2 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T10:58:10.497 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:58:10 smithi094 podman[150348]: 2024-03-24 10:58:10.228783969 +0000 UTC m=+5.684202443 container died c961dcdc746feeda53bc276575966b3a3d633503f742b4a744b289741956888a (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T10:58:10.997 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:58:10 smithi094 podman[150348]: 2024-03-24 10:58:10.65818255 +0000 UTC m=+6.113601030 container cleanup c961dcdc746feeda53bc276575966b3a3d633503f742b4a744b289741956888a (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-24T10:58:10.997 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:58:10 smithi094 bash[150348]: ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-2 2024-03-24T10:58:10.998 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:58:10 smithi094 podman[150372]: 2024-03-24 10:58:10.859744992 +0000 UTC m=+0.629951697 container remove c961dcdc746feeda53bc276575966b3a3d633503f742b4a744b289741956888a (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6) 2024-03-24T10:58:11.526 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:58:11 smithi094 podman[150461]: 2024-03-24 10:58:11.178702314 +0000 UTC m=+0.017261294 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-24T10:58:11.526 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:58:11 smithi094 podman[150461]: 2024-03-24 10:58:11.294107081 +0000 UTC m=+0.132666047 container create 516b536e8f10e8bec5f164825c4ad7c0ac8d6a5ce8683fcf565dc1ae8b39c178 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS) 2024-03-24T10:58:11.526 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:58:11 smithi094 podman[150461]: 2024-03-24 10:58:11.522564994 +0000 UTC m=+0.361123964 container init 516b536e8f10e8bec5f164825c4ad7c0ac8d6a5ce8683fcf565dc1ae8b39c178 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-2-deactivate, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-24T10:58:11.526 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:58:11 smithi094 podman[150461]: 2024-03-24 10:58:11.526017182 +0000 UTC m=+0.364576156 container start 516b536e8f10e8bec5f164825c4ad7c0ac8d6a5ce8683fcf565dc1ae8b39c178 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-24T10:58:11.810 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:58:11 smithi094 podman[150461]: 2024-03-24 10:58:11.622034893 +0000 UTC m=+0.460593877 container attach 516b536e8f10e8bec5f164825c4ad7c0ac8d6a5ce8683fcf565dc1ae8b39c178 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-2-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.build-date=20240319) 2024-03-24T10:58:12.099 INFO:journalctl@ceph.osd.2.smithi094.stdout:Mar 24 10:58:11 smithi094 podman[150461]: 2024-03-24 10:58:11.811659142 +0000 UTC m=+0.650218116 container died 516b536e8f10e8bec5f164825c4ad7c0ac8d6a5ce8683fcf565dc1ae8b39c178 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-24T10:58:12.383 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-386110d4-e9ca-11ee-95cd-87774f69a715@osd.2.service' 2024-03-24T10:58:12.424 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T10:58:12.424 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-24T10:58:12.424 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-24T10:58:12.424 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-386110d4-e9ca-11ee-95cd-87774f69a715@osd.3 2024-03-24T10:58:12.718 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:58:12 smithi096 systemd[1]: Stopping Ceph osd.3 for 386110d4-e9ca-11ee-95cd-87774f69a715... 2024-03-24T10:58:12.980 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:58:12 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3[93576]: 2024-03-24T10:58:12.717+0000 7f1fefd40700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T10:58:12.980 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:58:12 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3[93576]: 2024-03-24T10:58:12.717+0000 7f1fefd40700 -1 osd.3 84 *** Got signal Terminated *** 2024-03-24T10:58:12.980 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:58:12 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3[93576]: 2024-03-24T10:58:12.717+0000 7f1fefd40700 -1 osd.3 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T10:58:17.325 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:58:16 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4[98609]: 2024-03-24T10:58:16.984+0000 7f0ad7dc0700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-03-24T10:57:52.491370+0000 front 2024-03-24T10:57:52.491315+0000 (oldest deadline 2024-03-24T10:58:16.591266+0000) 2024-03-24T10:58:17.326 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:58:17 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5[103518]: 2024-03-24T10:58:17.139+0000 7f108c87c700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-03-24T10:57:52.947341+0000 front 2024-03-24T10:57:52.947436+0000 (oldest deadline 2024-03-24T10:58:17.047385+0000) 2024-03-24T10:58:18.325 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:58:18 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5[103518]: 2024-03-24T10:58:18.095+0000 7f108c87c700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-03-24T10:57:52.947341+0000 front 2024-03-24T10:57:52.947436+0000 (oldest deadline 2024-03-24T10:58:17.047385+0000) 2024-03-24T10:58:18.326 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:58:18 smithi096 podman[120178]: 2024-03-24 10:58:18.071869847 +0000 UTC m=+5.454840470 container died 6323fdc6e6ca1dd1cce9a0c443c9b490aaf4f251f346523b9bfb0b96cb0dfd6f (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3, org.label-schema.build-date=20240319, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T10:58:18.326 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:58:17 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4[98609]: 2024-03-24T10:58:17.972+0000 7f0ad7dc0700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-03-24T10:57:52.491370+0000 front 2024-03-24T10:57:52.491315+0000 (oldest deadline 2024-03-24T10:58:16.591266+0000) 2024-03-24T10:58:18.766 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:58:18 smithi096 podman[120178]: 2024-03-24 10:58:18.500544364 +0000 UTC m=+5.883514994 container cleanup 6323fdc6e6ca1dd1cce9a0c443c9b490aaf4f251f346523b9bfb0b96cb0dfd6f (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T10:58:18.766 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:58:18 smithi096 bash[120178]: ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3 2024-03-24T10:58:19.068 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:58:18 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4[98609]: 2024-03-24T10:58:18.953+0000 7f0ad7dc0700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-03-24T10:57:52.491370+0000 front 2024-03-24T10:57:52.491315+0000 (oldest deadline 2024-03-24T10:58:16.591266+0000) 2024-03-24T10:58:19.068 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:58:18 smithi096 podman[120208]: 2024-03-24 10:58:18.766749911 +0000 UTC m=+0.695215593 container remove 6323fdc6e6ca1dd1cce9a0c443c9b490aaf4f251f346523b9bfb0b96cb0dfd6f (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T10:58:19.322 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:58:19 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5[103518]: 2024-03-24T10:58:19.067+0000 7f108c87c700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-03-24T10:57:52.947341+0000 front 2024-03-24T10:57:52.947436+0000 (oldest deadline 2024-03-24T10:58:17.047385+0000) 2024-03-24T10:58:19.322 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:58:19 smithi096 podman[120303]: 2024-03-24 10:58:19.066485203 +0000 UTC m=+0.018329051 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-24T10:58:19.323 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:58:19 smithi096 podman[120303]: 2024-03-24 10:58:19.18648722 +0000 UTC m=+0.138331070 container create 0f3613c7e0d1f6b0aadb03ae99b6d2c5a76e39e1f9eb6cf41380c81a7091a0ad (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6) 2024-03-24T10:58:19.575 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:58:19 smithi096 podman[120303]: 2024-03-24 10:58:19.413911532 +0000 UTC m=+0.365755390 container init 0f3613c7e0d1f6b0aadb03ae99b6d2c5a76e39e1f9eb6cf41380c81a7091a0ad (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, io.buildah.version=1.33.5) 2024-03-24T10:58:19.575 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:58:19 smithi096 podman[120303]: 2024-03-24 10:58:19.417046592 +0000 UTC m=+0.368890441 container start 0f3613c7e0d1f6b0aadb03ae99b6d2c5a76e39e1f9eb6cf41380c81a7091a0ad (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-24T10:58:19.575 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:58:19 smithi096 podman[120303]: 2024-03-24 10:58:19.497144688 +0000 UTC m=+0.448988546 container attach 0f3613c7e0d1f6b0aadb03ae99b6d2c5a76e39e1f9eb6cf41380c81a7091a0ad (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, io.buildah.version=1.33.5) 2024-03-24T10:58:19.967 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 24 10:58:19 smithi096 podman[120303]: 2024-03-24 10:58:19.704540357 +0000 UTC m=+0.656384214 container died 0f3613c7e0d1f6b0aadb03ae99b6d2c5a76e39e1f9eb6cf41380c81a7091a0ad (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True) 2024-03-24T10:58:20.300 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:58:19 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4[98609]: 2024-03-24T10:58:19.966+0000 7f0ad7dc0700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-03-24T10:57:52.491370+0000 front 2024-03-24T10:57:52.491315+0000 (oldest deadline 2024-03-24T10:58:16.591266+0000) 2024-03-24T10:58:20.300 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:58:20 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5[103518]: 2024-03-24T10:58:20.097+0000 7f108c87c700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-03-24T10:57:52.947341+0000 front 2024-03-24T10:57:52.947436+0000 (oldest deadline 2024-03-24T10:58:17.047385+0000) 2024-03-24T10:58:20.324 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-386110d4-e9ca-11ee-95cd-87774f69a715@osd.3.service' 2024-03-24T10:58:20.366 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T10:58:20.366 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-24T10:58:20.366 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-24T10:58:20.366 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-386110d4-e9ca-11ee-95cd-87774f69a715@osd.4 2024-03-24T10:58:20.575 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:58:20 smithi096 systemd[1]: Stopping Ceph osd.4 for 386110d4-e9ca-11ee-95cd-87774f69a715... 2024-03-24T10:58:20.978 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:58:20 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4[98609]: 2024-03-24T10:58:20.677+0000 7f0adefd3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T10:58:20.978 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:58:20 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4[98609]: 2024-03-24T10:58:20.677+0000 7f0adefd3700 -1 osd.4 84 *** Got signal Terminated *** 2024-03-24T10:58:20.978 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:58:20 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4[98609]: 2024-03-24T10:58:20.677+0000 7f0adefd3700 -1 osd.4 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T10:58:21.325 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:58:20 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4[98609]: 2024-03-24T10:58:20.977+0000 7f0ad7dc0700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-03-24T10:57:52.491370+0000 front 2024-03-24T10:57:52.491315+0000 (oldest deadline 2024-03-24T10:58:16.591266+0000) 2024-03-24T10:58:21.325 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:58:21 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5[103518]: 2024-03-24T10:58:21.075+0000 7f108c87c700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-03-24T10:57:52.947341+0000 front 2024-03-24T10:57:52.947436+0000 (oldest deadline 2024-03-24T10:58:17.047385+0000) 2024-03-24T10:58:22.325 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:58:21 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4[98609]: 2024-03-24T10:58:21.947+0000 7f0ad7dc0700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-03-24T10:57:52.491370+0000 front 2024-03-24T10:57:52.491315+0000 (oldest deadline 2024-03-24T10:58:16.591266+0000) 2024-03-24T10:58:22.325 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:58:22 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5[103518]: 2024-03-24T10:58:22.062+0000 7f108c87c700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-03-24T10:57:52.947341+0000 front 2024-03-24T10:57:52.947436+0000 (oldest deadline 2024-03-24T10:58:17.047385+0000) 2024-03-24T10:58:23.325 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:58:22 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4[98609]: 2024-03-24T10:58:22.973+0000 7f0ad7dc0700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-03-24T10:57:52.491370+0000 front 2024-03-24T10:57:52.491315+0000 (oldest deadline 2024-03-24T10:58:16.591266+0000) 2024-03-24T10:58:23.325 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:58:23 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5[103518]: 2024-03-24T10:58:23.017+0000 7f108c87c700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-03-24T10:57:52.947341+0000 front 2024-03-24T10:57:52.947436+0000 (oldest deadline 2024-03-24T10:58:17.047385+0000) 2024-03-24T10:58:23.325 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:58:23 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5[103518]: 2024-03-24T10:58:23.017+0000 7f108c87c700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.94:6814 osd.1 since back 2024-03-24T10:57:57.548130+0000 front 2024-03-24T10:57:57.548212+0000 (oldest deadline 2024-03-24T10:58:22.248108+0000) 2024-03-24T10:58:24.325 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:58:23 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4[98609]: 2024-03-24T10:58:23.993+0000 7f0ad7dc0700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-03-24T10:57:52.491370+0000 front 2024-03-24T10:57:52.491315+0000 (oldest deadline 2024-03-24T10:58:16.591266+0000) 2024-03-24T10:58:24.325 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:58:24 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5[103518]: 2024-03-24T10:58:24.056+0000 7f108c87c700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-03-24T10:57:52.947341+0000 front 2024-03-24T10:57:52.947436+0000 (oldest deadline 2024-03-24T10:58:17.047385+0000) 2024-03-24T10:58:24.325 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:58:24 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5[103518]: 2024-03-24T10:58:24.056+0000 7f108c87c700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.94:6814 osd.1 since back 2024-03-24T10:57:57.548130+0000 front 2024-03-24T10:57:57.548212+0000 (oldest deadline 2024-03-24T10:58:22.248108+0000) 2024-03-24T10:58:25.325 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:58:24 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4[98609]: 2024-03-24T10:58:24.996+0000 7f0ad7dc0700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-03-24T10:57:52.491370+0000 front 2024-03-24T10:57:52.491315+0000 (oldest deadline 2024-03-24T10:58:16.591266+0000) 2024-03-24T10:58:25.325 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:58:24 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4[98609]: 2024-03-24T10:58:24.996+0000 7f0ad7dc0700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.94:6814 osd.1 since back 2024-03-24T10:58:00.592473+0000 front 2024-03-24T10:58:00.592559+0000 (oldest deadline 2024-03-24T10:58:24.692447+0000) 2024-03-24T10:58:25.325 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:58:25 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5[103518]: 2024-03-24T10:58:25.097+0000 7f108c87c700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-03-24T10:57:52.947341+0000 front 2024-03-24T10:57:52.947436+0000 (oldest deadline 2024-03-24T10:58:17.047385+0000) 2024-03-24T10:58:25.326 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:58:25 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5[103518]: 2024-03-24T10:58:25.097+0000 7f108c87c700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.94:6814 osd.1 since back 2024-03-24T10:57:57.548130+0000 front 2024-03-24T10:57:57.548212+0000 (oldest deadline 2024-03-24T10:58:22.248108+0000) 2024-03-24T10:58:26.329 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:58:26 smithi096 podman[120410]: 2024-03-24 10:58:26.079265749 +0000 UTC m=+5.492599471 container died 3e2536bcb263e8614ec90c43a6ef4b6036d4baaf782ca8ebcf2d1bdd13cf00aa (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4, ceph=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-24T10:58:26.330 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:58:26 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5[103518]: 2024-03-24T10:58:26.096+0000 7f108c87c700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-03-24T10:57:52.947341+0000 front 2024-03-24T10:57:52.947436+0000 (oldest deadline 2024-03-24T10:58:17.047385+0000) 2024-03-24T10:58:26.330 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:58:26 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5[103518]: 2024-03-24T10:58:26.096+0000 7f108c87c700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.94:6814 osd.1 since back 2024-03-24T10:57:57.548130+0000 front 2024-03-24T10:57:57.548212+0000 (oldest deadline 2024-03-24T10:58:22.248108+0000) 2024-03-24T10:58:26.611 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:58:26 smithi096 podman[120410]: 2024-03-24 10:58:26.411076837 +0000 UTC m=+5.824410562 container cleanup 3e2536bcb263e8614ec90c43a6ef4b6036d4baaf782ca8ebcf2d1bdd13cf00aa (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-24T10:58:26.612 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:58:26 smithi096 bash[120410]: ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4 2024-03-24T10:58:27.020 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:58:26 smithi096 podman[120427]: 2024-03-24 10:58:26.612313681 +0000 UTC m=+0.529947372 container remove 3e2536bcb263e8614ec90c43a6ef4b6036d4baaf782ca8ebcf2d1bdd13cf00aa (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T10:58:27.325 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:58:27 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5[103518]: 2024-03-24T10:58:27.103+0000 7f108c87c700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-03-24T10:57:52.947341+0000 front 2024-03-24T10:57:52.947436+0000 (oldest deadline 2024-03-24T10:58:17.047385+0000) 2024-03-24T10:58:27.325 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:58:27 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5[103518]: 2024-03-24T10:58:27.103+0000 7f108c87c700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.94:6814 osd.1 since back 2024-03-24T10:57:57.548130+0000 front 2024-03-24T10:57:57.548212+0000 (oldest deadline 2024-03-24T10:58:22.248108+0000) 2024-03-24T10:58:27.325 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:58:27 smithi096 podman[120519]: 2024-03-24 10:58:26.923320264 +0000 UTC m=+0.018076807 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-24T10:58:27.325 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:58:27 smithi096 podman[120519]: 2024-03-24 10:58:27.029696288 +0000 UTC m=+0.124452814 container create 98c17375baf2dfd420c933b6d66c6d9001cabf453ac5fcc6dbcc0a15ceb4cea1 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4-deactivate, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-24T10:58:27.326 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:58:27 smithi096 podman[120519]: 2024-03-24 10:58:27.248775278 +0000 UTC m=+0.343531814 container init 98c17375baf2dfd420c933b6d66c6d9001cabf453ac5fcc6dbcc0a15ceb4cea1 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4-deactivate, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-24T10:58:27.326 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:58:27 smithi096 podman[120519]: 2024-03-24 10:58:27.251728225 +0000 UTC m=+0.346484759 container start 98c17375baf2dfd420c933b6d66c6d9001cabf453ac5fcc6dbcc0a15ceb4cea1 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-24T10:58:27.825 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:58:27 smithi096 podman[120519]: 2024-03-24 10:58:27.348602387 +0000 UTC m=+0.443358926 container attach 98c17375baf2dfd420c933b6d66c6d9001cabf453ac5fcc6dbcc0a15ceb4cea1 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD) 2024-03-24T10:58:27.825 INFO:journalctl@ceph.osd.4.smithi096.stdout:Mar 24 10:58:27 smithi096 podman[120519]: 2024-03-24 10:58:27.534525093 +0000 UTC m=+0.629281620 container died 98c17375baf2dfd420c933b6d66c6d9001cabf453ac5fcc6dbcc0a15ceb4cea1 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, CEPH_POINT_RELEASE=) 2024-03-24T10:58:28.110 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-386110d4-e9ca-11ee-95cd-87774f69a715@osd.4.service' 2024-03-24T10:58:28.154 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T10:58:28.154 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-24T10:58:28.154 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-24T10:58:28.155 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-386110d4-e9ca-11ee-95cd-87774f69a715@osd.5 2024-03-24T10:58:28.480 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:58:28 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5[103518]: 2024-03-24T10:58:28.151+0000 7f108c87c700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-03-24T10:57:52.947341+0000 front 2024-03-24T10:57:52.947436+0000 (oldest deadline 2024-03-24T10:58:17.047385+0000) 2024-03-24T10:58:28.480 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:58:28 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5[103518]: 2024-03-24T10:58:28.151+0000 7f108c87c700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.94:6814 osd.1 since back 2024-03-24T10:57:57.548130+0000 front 2024-03-24T10:57:57.548212+0000 (oldest deadline 2024-03-24T10:58:22.248108+0000) 2024-03-24T10:58:28.480 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:58:28 smithi096 systemd[1]: Stopping Ceph osd.5 for 386110d4-e9ca-11ee-95cd-87774f69a715... 2024-03-24T10:58:28.480 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:58:28 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5[103518]: 2024-03-24T10:58:28.479+0000 7f108fa87700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T10:58:28.481 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:58:28 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5[103518]: 2024-03-24T10:58:28.479+0000 7f108fa87700 -1 osd.5 84 *** Got signal Terminated *** 2024-03-24T10:58:28.481 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:58:28 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5[103518]: 2024-03-24T10:58:28.479+0000 7f108fa87700 -1 osd.5 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T10:58:29.575 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:58:29 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5[103518]: 2024-03-24T10:58:29.142+0000 7f108c87c700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-03-24T10:57:52.947341+0000 front 2024-03-24T10:57:52.947436+0000 (oldest deadline 2024-03-24T10:58:17.047385+0000) 2024-03-24T10:58:29.575 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:58:29 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5[103518]: 2024-03-24T10:58:29.142+0000 7f108c87c700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.94:6814 osd.1 since back 2024-03-24T10:57:57.548130+0000 front 2024-03-24T10:57:57.548212+0000 (oldest deadline 2024-03-24T10:58:22.248108+0000) 2024-03-24T10:58:30.575 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:58:30 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5[103518]: 2024-03-24T10:58:30.147+0000 7f108c87c700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-03-24T10:57:52.947341+0000 front 2024-03-24T10:57:52.947436+0000 (oldest deadline 2024-03-24T10:58:17.047385+0000) 2024-03-24T10:58:30.575 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:58:30 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5[103518]: 2024-03-24T10:58:30.147+0000 7f108c87c700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.94:6814 osd.1 since back 2024-03-24T10:57:57.548130+0000 front 2024-03-24T10:57:57.548212+0000 (oldest deadline 2024-03-24T10:58:22.248108+0000) 2024-03-24T10:58:31.575 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:58:31 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5[103518]: 2024-03-24T10:58:31.197+0000 7f108c87c700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-03-24T10:57:52.947341+0000 front 2024-03-24T10:57:52.947436+0000 (oldest deadline 2024-03-24T10:58:17.047385+0000) 2024-03-24T10:58:31.575 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:58:31 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5[103518]: 2024-03-24T10:58:31.197+0000 7f108c87c700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.94:6814 osd.1 since back 2024-03-24T10:57:57.548130+0000 front 2024-03-24T10:57:57.548212+0000 (oldest deadline 2024-03-24T10:58:22.248108+0000) 2024-03-24T10:58:32.575 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:58:32 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5[103518]: 2024-03-24T10:58:32.175+0000 7f108c87c700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-03-24T10:57:52.947341+0000 front 2024-03-24T10:57:52.947436+0000 (oldest deadline 2024-03-24T10:58:17.047385+0000) 2024-03-24T10:58:32.575 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:58:32 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5[103518]: 2024-03-24T10:58:32.175+0000 7f108c87c700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.94:6814 osd.1 since back 2024-03-24T10:57:57.548130+0000 front 2024-03-24T10:57:57.548212+0000 (oldest deadline 2024-03-24T10:58:22.248108+0000) 2024-03-24T10:58:32.575 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:58:32 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5[103518]: 2024-03-24T10:58:32.175+0000 7f108c87c700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.94:6822 osd.2 since back 2024-03-24T10:58:06.249268+0000 front 2024-03-24T10:58:06.249303+0000 (oldest deadline 2024-03-24T10:58:31.549278+0000) 2024-03-24T10:58:33.575 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:58:33 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5[103518]: 2024-03-24T10:58:33.224+0000 7f108c87c700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-03-24T10:57:52.947341+0000 front 2024-03-24T10:57:52.947436+0000 (oldest deadline 2024-03-24T10:58:17.047385+0000) 2024-03-24T10:58:33.575 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:58:33 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5[103518]: 2024-03-24T10:58:33.224+0000 7f108c87c700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.94:6814 osd.1 since back 2024-03-24T10:57:57.548130+0000 front 2024-03-24T10:57:57.548212+0000 (oldest deadline 2024-03-24T10:58:22.248108+0000) 2024-03-24T10:58:33.575 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:58:33 smithi096 ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5[103518]: 2024-03-24T10:58:33.224+0000 7f108c87c700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.94:6822 osd.2 since back 2024-03-24T10:58:06.249268+0000 front 2024-03-24T10:58:06.249303+0000 (oldest deadline 2024-03-24T10:58:31.549278+0000) 2024-03-24T10:58:34.121 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:58:33 smithi096 podman[120622]: 2024-03-24 10:58:33.855319837 +0000 UTC m=+5.479303537 container died 2f77f1fa20e22543f6889f1475e5d16a1dd370623b224be8f94d243655f60b7d (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6) 2024-03-24T10:58:34.471 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:58:34 smithi096 podman[120622]: 2024-03-24 10:58:34.220553663 +0000 UTC m=+5.844537363 container cleanup 2f77f1fa20e22543f6889f1475e5d16a1dd370623b224be8f94d243655f60b7d (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T10:58:34.471 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:58:34 smithi096 bash[120622]: ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5 2024-03-24T10:58:34.780 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:58:34 smithi096 podman[120637]: 2024-03-24 10:58:34.471601902 +0000 UTC m=+0.617009865 container remove 2f77f1fa20e22543f6889f1475e5d16a1dd370623b224be8f94d243655f60b7d (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5, ceph=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-24T10:58:35.048 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:58:34 smithi096 podman[120728]: 2024-03-24 10:58:34.780552976 +0000 UTC m=+0.107092371 container create 904c57c3175b491d438de70704a469460bb5396e564ef0a48fc449ee7e38ddd2 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-24T10:58:35.049 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:58:34 smithi096 podman[120728]: 2024-03-24 10:58:34.692194608 +0000 UTC m=+0.018734012 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-24T10:58:35.049 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:58:34 smithi096 podman[120728]: 2024-03-24 10:58:34.966316815 +0000 UTC m=+0.292856212 container init 904c57c3175b491d438de70704a469460bb5396e564ef0a48fc449ee7e38ddd2 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-24T10:58:35.049 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:58:34 smithi096 podman[120728]: 2024-03-24 10:58:34.969534557 +0000 UTC m=+0.296073956 container start 904c57c3175b491d438de70704a469460bb5396e564ef0a48fc449ee7e38ddd2 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5-deactivate, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T10:58:35.049 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:58:35 smithi096 podman[120728]: 2024-03-24 10:58:35.049337214 +0000 UTC m=+0.375876610 container attach 904c57c3175b491d438de70704a469460bb5396e564ef0a48fc449ee7e38ddd2 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T10:58:35.325 INFO:journalctl@ceph.osd.5.smithi096.stdout:Mar 24 10:58:35 smithi096 podman[120728]: 2024-03-24 10:58:35.260722776 +0000 UTC m=+0.587262182 container died 904c57c3175b491d438de70704a469460bb5396e564ef0a48fc449ee7e38ddd2 (image=quay.ceph.io/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-386110d4-e9ca-11ee-95cd-87774f69a715-osd-5-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T10:58:35.971 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-386110d4-e9ca-11ee-95cd-87774f69a715@osd.5.service' 2024-03-24T10:58:36.015 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T10:58:36.015 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-24T10:58:36.016 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-24T10:58:57.945 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-24T10:59:12.480 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-24T10:59:12.516 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-24T10:59:12.546 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-24T10:59:12.547 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-03-24_04:56:01-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7619070/remote/smithi094/crash 2024-03-24T10:59:12.548 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/crash -- . 2024-03-24T10:59:12.585 INFO:teuthology.orchestra.run.smithi094.stderr:tar: /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-24T10:59:12.585 INFO:teuthology.orchestra.run.smithi094.stderr:tar: Error is not recoverable: exiting now 2024-03-24T10:59:12.587 DEBUG:teuthology.misc:Transferring archived files from smithi096:/var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-03-24_04:56:01-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7619070/remote/smithi096/crash 2024-03-24T10:59:12.587 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/crash -- . 2024-03-24T10:59:12.614 INFO:teuthology.orchestra.run.smithi096.stderr:tar: /var/lib/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-24T10:59:12.614 INFO:teuthology.orchestra.run.smithi096.stderr:tar: Error is not recoverable: exiting now 2024-03-24T10:59:12.615 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-24T10:59:12.615 DEBUG:teuthology.orchestra.run.smithi094:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-03-24T10:59:12.658 INFO:teuthology.orchestra.run.smithi094.stderr:grep: /var/log/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/ceph.log: No such file or directory 2024-03-24T10:59:12.664 INFO:tasks.cephadm:Compressing logs... 2024-03-24T10:59:12.664 DEBUG:teuthology.orchestra.run.smithi094:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-24T10:59:12.707 DEBUG:teuthology.orchestra.run.smithi096:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-24T10:59:12.733 INFO:teuthology.orchestra.run.smithi094.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi094.efiqte.log 2024-03-24T10:59:12.734 INFO:teuthology.orchestra.run.smithi094.stderr:: No such file or directory 2024-03-24T10:59:12.734 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/ceph-volume.log 2024-03-24T10:59:12.734 INFO:teuthology.orchestra.run.smithi094.stderr:gzip: /var/log/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi094.efiqte.log.gz already exists; not overwritten 2024-03-24T10:59:12.734 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi094.llefgu.log 2024-03-24T10:59:12.734 INFO:teuthology.orchestra.run.smithi094.stderr:gzip: /var/log/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/ceph-volume.log.gz already exists; not overwritten 2024-03-24T10:59:12.734 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-24T10:59:12.734 INFO:teuthology.orchestra.run.smithi094.stderr:gzip: /var/log/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi094.llefgu.log.gz already exists; not overwritten 2024-03-24T10:59:12.734 INFO:teuthology.orchestra.run.smithi094.stderr:gzip: /var/log/ceph/cephadm.log.gz already exists; not overwritten 2024-03-24T10:59:12.736 DEBUG:teuthology.orchestra.run:got remote process result: 123 2024-03-24T10:59:12.736 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-03-24T10:59:12.736 INFO:teuthology.orchestra.run.smithi094.stderr:real 0m0.015s 2024-03-24T10:59:12.736 INFO:teuthology.orchestra.run.smithi094.stderr:user 0m0.012s 2024-03-24T10:59:12.736 INFO:teuthology.orchestra.run.smithi094.stderr:sys 0m0.017s 2024-03-24T10:59:12.737 INFO:tasks.cephadm:Removing cluster... 2024-03-24T10:59:12.737 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 --force 2024-03-24T10:59:12.737 INFO:teuthology.orchestra.run.smithi096.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/ceph.audit.log 2024-03-24T10:59:12.738 INFO:teuthology.orchestra.run.smithi096.stderr:: No such file or directory 2024-03-24T10:59:12.738 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/ceph-mgr.smithi096.gfonnu.log 2024-03-24T10:59:12.738 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/ceph-osd.4.log 2024-03-24T10:59:12.738 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/ceph.audit.log: /var/log/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/ceph-mgr.smithi096.gfonnu.log: gzip -5 --verbose -- /var/log/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/ceph-osd.5.log 2024-03-24T10:59:12.738 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/ceph-osd.3.log 2024-03-24T10:59:12.741 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi096.egbmoq.log 2024-03-24T10:59:12.741 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/ceph-osd.5.log: /var/log/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/ceph-mon.smithi096.log 2024-03-24T10:59:12.741 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi096.egbmoq.log: gzip -5 --verbose -- /var/log/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi096.jtebhh.log 2024-03-24T10:59:12.741 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/ceph-mon.smithi096.log: gzip -5 --verbose -- /var/log/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/ceph.cephadm.log 2024-03-24T10:59:12.741 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi096.jtebhh.log: 91.3% -- replaced with /var/log/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/ceph.audit.log.gz 2024-03-24T10:59:12.741 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/ceph.log 2024-03-24T10:59:12.741 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/ceph.cephadm.log: 85.2% -- replaced with /var/log/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/ceph.cephadm.log.gz 2024-03-24T10:59:12.742 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/ceph-volume.log 2024-03-24T10:59:12.744 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/ceph.log: 87.5% -- replaced with /var/log/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/ceph.log.gz 2024-03-24T10:59:12.744 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-24T10:59:12.766 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/ceph-volume.log: /var/log/ceph/cephadm.log: 92.8% 91.1% -- replaced with /var/log/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/ceph-volume.log.gz 2024-03-24T10:59:12.767 INFO:teuthology.orchestra.run.smithi096.stderr: -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-24T10:59:12.788 INFO:teuthology.orchestra.run.smithi096.stderr: 88.7% -- replaced with /var/log/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/ceph-mgr.smithi096.gfonnu.log.gz 2024-03-24T10:59:13.084 INFO:teuthology.orchestra.run.smithi096.stderr: 92.5% -- replaced with /var/log/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/ceph-mon.smithi096.log.gz 2024-03-24T10:59:13.439 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 386110d4-e9ca-11ee-95cd-87774f69a715 --force 2024-03-24T10:59:14.164 INFO:tasks.cephadm:Removing cephadm ... 2024-03-24T10:59:14.164 DEBUG:teuthology.orchestra.run.smithi094:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-24T10:59:14.180 DEBUG:teuthology.orchestra.run.smithi096:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-24T10:59:14.199 INFO:tasks.cephadm:Teardown complete 2024-03-24T10:59:14.199 ERROR:teuthology.run_tasks:Manager failed: cephadm Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_batrick_ceph_d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b/qa/tasks/cephadm.py", line 1810, in task log.info('Teardown begin') File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_batrick_ceph_d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b/qa/tasks/cephadm.py", line 1686, in initialize_config yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_batrick_ceph_d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b/qa/tasks/cephadm.py", line 397, in ceph_log run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi094 with status 123: "time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose --" 2024-03-24T10:59:14.201 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-24T10:59:14.211 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_batrick_ceph_d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b/qa/tasks/cephadm.py", line 1810, in task log.info('Teardown begin') File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_batrick_ceph_d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b/qa/tasks/cephadm.py", line 1686, in initialize_config yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_batrick_ceph_d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b/qa/tasks/cephadm.py", line 397, in ceph_log run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi094 with status 123: "time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose --" 2024-03-24T10:59:14.212 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-24T10:59:14.212 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-24T10:59:14.223 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-24T10:59:14.324 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-24T10:59:14.335 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-24T10:59:14.335 DEBUG:teuthology.orchestra.run.smithi094:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-24T10:59:14.337 DEBUG:teuthology.orchestra.run.smithi096:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-24T10:59:14.349 INFO:teuthology.orchestra.run.smithi094.stderr:bash: line 1: ntpq: command not found 2024-03-24T10:59:14.353 INFO:teuthology.orchestra.run.smithi094.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-24T10:59:14.353 INFO:teuthology.orchestra.run.smithi094.stdout:=============================================================================== 2024-03-24T10:59:14.353 INFO:teuthology.orchestra.run.smithi094.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 1 +575us[ +575us] +/- 94ms 2024-03-24T10:59:14.353 INFO:teuthology.orchestra.run.smithi094.stdout:^* hv02.front.sepia.ceph.com 3 7 377 2 -252us[ -283us] +/- 56ms 2024-03-24T10:59:14.353 INFO:teuthology.orchestra.run.smithi094.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 3 -118us[ -149us] +/- 64ms 2024-03-24T10:59:14.353 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T10:59:14.355 INFO:teuthology.orchestra.run.smithi096.stderr:bash: line 1: ntpq: command not found 2024-03-24T10:59:14.360 INFO:teuthology.orchestra.run.smithi096.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-24T10:59:14.360 INFO:teuthology.orchestra.run.smithi096.stdout:=============================================================================== 2024-03-24T10:59:14.360 INFO:teuthology.orchestra.run.smithi096.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 64 +579us[ +586us] +/- 93ms 2024-03-24T10:59:14.360 INFO:teuthology.orchestra.run.smithi096.stdout:^* hv02.front.sepia.ceph.com 3 6 377 2 -298us[ -290us] +/- 56ms 2024-03-24T10:59:14.361 INFO:teuthology.orchestra.run.smithi096.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 4 -128us[ -119us] +/- 64ms 2024-03-24T10:59:14.361 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T10:59:14.362 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-24T10:59:14.375 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-24T10:59:14.376 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-24T10:59:14.386 DEBUG:teuthology.orchestra.run.smithi094:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-24T10:59:14.478 DEBUG:teuthology.orchestra.run.smithi096:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-24T10:59:14.591 DEBUG:teuthology.orchestra.run.smithi094:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-24T10:59:14.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-24T10:59:14.621 DEBUG:teuthology.orchestra.run.smithi096:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-24T10:59:14.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-24T10:59:14.662 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-24T10:59:14.674 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-24T10:59:14.707 INFO:teuthology.task.internal:Duration was 1840.068837 seconds 2024-03-24T10:59:14.708 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-24T10:59:14.720 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-24T10:59:14.721 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-24T10:59:14.723 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-24T10:59:14.764 INFO:teuthology.orchestra.run.smithi094.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-24T10:59:14.785 INFO:teuthology.orchestra.run.smithi096.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-24T10:59:15.128 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-24T10:59:15.129 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi094.front.sepia.ceph.com 2024-03-24T10:59:15.129 DEBUG:teuthology.orchestra.run.smithi094:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-24T10:59:15.146 INFO:teuthology.orchestra.run.smithi094.stderr:grep: /home/ubuntu/cephtest/archive/syslog/kern.log: No such file or directory 2024-03-24T10:59:15.148 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi096.front.sepia.ceph.com 2024-03-24T10:59:15.149 DEBUG:teuthology.orchestra.run.smithi096:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-24T10:59:15.176 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-24T10:59:15.176 DEBUG:teuthology.orchestra.run.smithi094:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-24T10:59:15.190 DEBUG:teuthology.orchestra.run.smithi096:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-24T10:59:15.254 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-24T10:59:15.254 DEBUG:teuthology.orchestra.run.smithi094:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-24T10:59:15.260 DEBUG:teuthology.orchestra.run.smithi096:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-24T10:59:15.931 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-24T10:59:15.944 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-24T10:59:15.944 DEBUG:teuthology.orchestra.run.smithi094:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-24T10:59:15.971 DEBUG:teuthology.orchestra.run.smithi096:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-24T10:59:16.008 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-24T10:59:16.020 DEBUG:teuthology.orchestra.run.smithi094:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-24T10:59:16.023 DEBUG:teuthology.orchestra.run.smithi096:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-24T10:59:16.046 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern = core 2024-03-24T10:59:16.083 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern = core 2024-03-24T10:59:16.106 DEBUG:teuthology.orchestra.run.smithi094:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-24T10:59:16.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-24T10:59:16.120 DEBUG:teuthology.orchestra.run.smithi096:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-24T10:59:16.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-24T10:59:16.165 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-24T10:59:16.177 INFO:teuthology.task.internal:Transferring archived files... 2024-03-24T10:59:16.178 DEBUG:teuthology.misc:Transferring archived files from smithi094:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-24_04:56:01-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7619070/remote/smithi094 2024-03-24T10:59:16.178 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-24T10:59:16.233 DEBUG:teuthology.misc:Transferring archived files from smithi096:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-24_04:56:01-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7619070/remote/smithi096 2024-03-24T10:59:16.233 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-24T10:59:16.283 INFO:teuthology.task.internal:Removing archive directory... 2024-03-24T10:59:16.284 DEBUG:teuthology.orchestra.run.smithi094:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-24T10:59:16.286 DEBUG:teuthology.orchestra.run.smithi096:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-24T10:59:16.336 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-24T10:59:16.349 INFO:teuthology.task.internal:Not uploading archives. 2024-03-24T10:59:16.350 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-24T10:59:16.361 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-24T10:59:16.361 DEBUG:teuthology.orchestra.run.smithi094:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-24T10:59:16.363 DEBUG:teuthology.orchestra.run.smithi096:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-24T10:59:16.375 INFO:teuthology.orchestra.run.smithi094.stdout: 265844 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 24 10:59 /home/ubuntu/cephtest 2024-03-24T10:59:16.394 INFO:teuthology.orchestra.run.smithi096.stdout: 265317 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 24 10:59 /home/ubuntu/cephtest 2024-03-24T10:59:16.396 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-24T10:59:16.408 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-24T10:59:16.435 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on smithi094 with status 123: "time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose --" 2024-03-24T10:59:16.436 INFO:teuthology.nuke:Checking targets against current locks 2024-03-24T10:59:16.463 DEBUG:teuthology.nuke:shortname: smithi094 2024-03-24T10:59:16.464 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-24T10:59:16.485 DEBUG:teuthology.nuke:shortname: smithi096 2024-03-24T10:59:16.486 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-24T10:59:16.489 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi094.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-24_04:56:01-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7619070', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-24 10:19:26.312724', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLUkEj7js5B2LOwuV3cIofIodSHX3qWulagmUJDM+flT/yFZrCi2nuYPov/3ek9RIJmcmdXob/juWqdK+ysRkPE='} 2024-03-24T10:59:16.513 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi096.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-24_04:56:01-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7619070', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-24 10:19:26.314032', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJWsIwubLvHhvCvUaQT2eDd4GX0NcfHStLR85hOuWiDl3z52omxHrLa5qsLj3fs88euB9Nv5EQ7jV88mScZ0laA='} 2024-03-24T10:59:16.542 INFO:teuthology.orchestra.console.smithi094:Power off 2024-03-24T10:59:16.543 DEBUG:teuthology.orchestra.console.smithi094:pexpect command: ipmitool -H smithi094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-24T10:59:16.567 DEBUG:teuthology.orchestra.console.smithi094:power off output: Chassis Power Control: Down/Off 2024-03-24T10:59:16.568 DEBUG:teuthology.orchestra.console.smithi094:pexpect command: ipmitool -H smithi094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-24T10:59:16.582 INFO:teuthology.orchestra.console.smithi096:Power off 2024-03-24T10:59:16.582 DEBUG:teuthology.orchestra.console.smithi096:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-24T10:59:16.591 DEBUG:teuthology.orchestra.console.smithi094:check power output: Chassis Power is on 2024-03-24T10:59:16.604 DEBUG:teuthology.orchestra.console.smithi096:power off output: Chassis Power Control: Down/Off 2024-03-24T10:59:16.604 DEBUG:teuthology.orchestra.console.smithi096:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-24T10:59:16.628 DEBUG:teuthology.orchestra.console.smithi096:check power output: Chassis Power is on 2024-03-24T10:59:18.609 INFO:teuthology.orchestra.run.smithi096.stderr:gzip: /var/log/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/ceph-osd.5.log: No such file or directory 2024-03-24T10:59:18.609 INFO:teuthology.orchestra.run.smithi096.stderr: 93.5% -- replaced with /var/log/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/ceph-osd.5.log.gz 2024-03-24T10:59:19.127 INFO:teuthology.orchestra.run.smithi096.stderr:gzip: /var/log/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/ceph-osd.4.log: No such file or directory 2024-03-24T10:59:19.127 INFO:teuthology.orchestra.run.smithi096.stderr: 93.3% -- replaced with /var/log/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/ceph-osd.4.log.gz 2024-03-24T10:59:19.844 INFO:teuthology.orchestra.run.smithi096.stderr:gzip: /var/log/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/ceph-osd.3.log: No such file or directory 2024-03-24T10:59:19.845 INFO:teuthology.orchestra.run.smithi096.stderr: 93.4% -- replaced with /var/log/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/ceph-osd.3.log.gz 2024-03-24T10:59:19.963 INFO:teuthology.orchestra.run.smithi096.stderr:gzip: /var/log/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi096.jtebhh.log: No such file or directory 2024-03-24T10:59:19.963 INFO:teuthology.orchestra.run.smithi096.stderr: 94.9% -- replaced with /var/log/ceph/386110d4-e9ca-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi096.jtebhh.log.gz 2024-03-24T10:59:20.592 DEBUG:teuthology.orchestra.console.smithi094:pexpect command: ipmitool -H smithi094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-24T10:59:20.629 DEBUG:teuthology.orchestra.console.smithi096:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-24T10:59:20.703 DEBUG:teuthology.orchestra.console.smithi094:check power output: Chassis Power is on 2024-03-24T10:59:20.738 DEBUG:teuthology.orchestra.console.smithi096:check power output: Chassis Power is on 2024-03-24T10:59:24.706 DEBUG:teuthology.orchestra.console.smithi094:pexpect command: ipmitool -H smithi094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-24T10:59:24.739 DEBUG:teuthology.orchestra.console.smithi096:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-24T10:59:24.816 DEBUG:teuthology.orchestra.console.smithi094:check power output: Chassis Power is off 2024-03-24T10:59:24.849 DEBUG:teuthology.orchestra.console.smithi096:check power output: Chassis Power is on 2024-03-24T10:59:24.918 INFO:teuthology.orchestra.console.smithi094:Power off completed 2024-03-24T10:59:28.850 DEBUG:teuthology.orchestra.console.smithi096:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-24T10:59:28.959 DEBUG:teuthology.orchestra.console.smithi096:check power output: Chassis Power is off 2024-03-24T10:59:29.060 INFO:teuthology.orchestra.console.smithi096:Power off completed 2024-03-24T10:59:29.165 INFO:teuthology.run:Summary data: description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 1840.068836927414 failure_reason: 'Command failed on smithi094 with status 123: "time sudo find /var/log/ceph /var/log/rbd-target-api -name ''*.log'' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose --"' flavor: default owner: scheduled_pdonnell@teuthology status: fail success: false 2024-03-24T10:59:29.165 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-24T10:59:29.258 INFO:teuthology.run:FAIL